Discussant Remarks, "Understanding the Complex Context of Implementing and Sustaining EBPs in the Real World"

Hi my name is Thaddeus Ferber, I’m the Vice President for Policy at the Forum for Youth Investment.

So for about as long as I can remember, I knew I wanted to make a difference. So my earliest thoughts were along these lines. But in particular, a formative thing for me personally, was the LA Riots, which happened when I was a young kid. And it was all well and good to sit in social studies class and learned about the Civil Right battles of the past that allowed us to have a society of equal opportunity but what I was seeing on the screen didn’t look like a very equal opportunity to me.

Right around then, there was a full page ad in the New York Times of a bunch of corporate CEOs coming together saying, “We’re going to work together on this issue, and we need your help.”  Well, that was great.  Here I was, a young teen, and Fortune 500 CEOs want my help, so I wrote them, at the address, a deeply passionate letter that I assure you, when I read it – I reread it a couple of years ago, was extraordinarily embarrassing in retrospect.

[Laughter]

And they sent back – I don’t know exactly what I thought they were gonna do next, besides have me join the CEO for a year of working with him or something like that, but what I got back was a special edition of Business Week magazine that was focusing on education.  And it put the spotlight on different programs that were working in different parts of the country.  This program, this mentoring program, increased test scores 10 percent; this tutoring program, increased test scores 15 percent, things along that line.

So, this seemed great – right?  I could just then look around and say, “Well, if we can get ten of these programs, each which improve things ten percent together, we’ll solve this problem.”

[Laughter]

This was, I should say, before I had had any stats or probability classes.

[Laughter]

But besides that, you know, besides that fallacy, I think what I’m hearing throughout this day is that we’re not gonna solve these problems by finding an individual program, evaluating it, and making it happen everywhere in the country.

And so, that’s really what I saw, as the wonderful speakers this morning, and over the course of the event, and all the wonderful work that you’re all doing is trying to figure out, “All right, if we’re not gonna achieve utopia one program at a time, what do we need to do?  And why is this approach not working?  And what do we need to do differently?”

So, you heard fantastic remarks by Gene Hall, talking about things related to what I consider replication or adoption in places across the country.  And merely taking something that worked in one place and plopping it down in the other and expecting it will be implemented and adopted the exact same way as in the first place is a fallacy, and we know that often doesn’t work.

But deeper beneath that, I think there’s a bigger issue, which is that you can’t – there’s really never one magic bullet.  There’s no one thing you can do to change kids’ lives.  And in pouring through everything I could get my hands on, I was kind of deflated by how minimal the effects of different programs in isolation were.

And the only thing that I ever saw predictive was the kind of curve that  you would expect that you’d want, if you are a young type looking to change the world, were things that didn’t get into one, any one particular thing, but looked broader than that.

So, for example, looking at resiliency research, and finding, “All right, let’s come up with some horrible, morbid thing that’s gonna make sure that all kids fail when that happens,” and zeroed in on things like death of a parent, and then we’re surprised to find, well, death of a parent by itself doesn’t lead to bad outcomes.  People are generally resilient.

Where things went bad at the population level was when you could start seeing lots of different problems all happening to the same kid, and lots – or lots of great things happened.  And so, the research that we were looking at, showing that if you just count the number of positive things and count the number of negative things, you’re going to be able to have a much better job of predicting success or failure than by delving into the details of any one particular program or intervention.

Well, that themselves then brings with whole new challenges of, “All right, well, if there’s lots of different programs, what’s the right configuration of programs that work, and how do they work together?”  And I think that’s what you were hearing also as well today.  And that when you’re doing lots of different - that’s a whole set of things - and trying to change lots of things all at the same time, it’s very hard to know what thing you’re doing or aren’t doing that works, as you heard John Roman speak to, it’s not easy to find out when the answer isn’t one single intervention, not one single program.  You know, you need lots of stuff that’s hard to parse out.

So, a couple of high level thoughts on that of when you – once you know you need lots of things happening at once, one is it seems that you just can’t let a thousand flowers bloom.  Right?  You could take that assumption and say, “All right, everyone just go do stuff, because we don’t which matters, and we know having a lot of them helps.”  And that is starting to be neither as efficient or as effective as people would want.  So, you actually do need a discipline to figuring out which things to do, in which order, and how they compliment each other.

Second is quality matters a great deal.  The quality with which a program is implemented often matters a whole lot more than the name on the front door.  Everything from how well the practices were adopted, and even the whole sense of it may not be a particular program so much as it might be individual staff practices.  And if you can get into the black box and start getting people to adopt those practices, that’s going to matter a whole lot.

And then the last is that those things that communities need that you put out, they have to be connected to each other.  I see Dan Cardinali in the room, who’s from Communities in Schools.  I believe he’s done some research that shows that there’s an added value if you show not just that you just have lots of services, but if you show that they’re delivered and connected in concert, that increases the Return on Investment.

All right, so that brings us then up to David’s presentation, which is, “All right, if you need a lot of things happening at once, and they have to be aligned, and they have to be connected, and they have to meet the community’s need, you need some group or entity facilitating that in some sort of partnerships.”  And I see really the work on evidence as it pertains to partnerships as the real cutting ground difficult stuff that’s going on right now.

What you’re hearing David talk about is that not every partnership works.  And we don’t know yet enough why some partnerships work and why some don’t.  There’s a lot of work going on, trying to do this practice-based evidence of trying to talk and qualitative work, to talk to lots of partnerships and get a sense of, what are the common factors that lead some to succeed and some to fail.  And I feel like we’re just kind of getting a standards of practice conversation coming together.  The next phase is going to have to be, can we find any evidence that supports our perception of why – what are the standards that we think lead to success and ones we don’t?

And then a last – I’ll leave you at the even higher meta level than David’s conversation, which was the conversations of when you go into communities, you often find multiple partnerships.  And so, just as we were talking about the challenges of studying multiple programs, multiple interventions, not only do we need to study what’s effective at a partnership, we have to understand what happens when partnerships come up in places where there’s lots of other partnerships and understanding how those partnerships play across each other.

So, I’ll leave you with – I don’t see Kris Moore from Child Trends in the office, but I saw her a couple days ago, and she was reflecting what a great, nerdy time this is in America.

[Laughter]

And it’s wonderful to have so many fellow nerds delving into this and really connecting the work to the visions that we all share of what this country is and where we should be going.  So, thank you.

[Applause]

[End of Audio]