Willing, Able, Ready: Supporting Evidence-Based/Informed Interventions in Organizations with Different Levels of Readiness

Okay, great.  Well, that’s a big “y’all” from South Carolina.  My name’s Abe Wandersman, as Celene mentioned, and I have had the honor of being part of the planning group for this as co-PI with David Osher and Project Director Allison Dymnicki, and with our federal colleagues and funders, but really colleagues – Sarah Oberlander and Laura Radel and everybody else.

This is – we’ve been waiting for this for a long time.  And I love Arthur’s comments about carrying bad outcomes.  That’s why we do this.  And what really matters to us is not so much what’s presented here, but what you take away and what you do with it.

So, the topic that I’m going to be covering is readiness.  And I was – I got a compliment yesterday about my new suit, which is actually a suit I bought in a place called Theory.  And I like that, because I’m theory kind of guy.

[Laughter]

I will tell you, though, that I did buy it at an outlet mall, ‘cause I – the prices are just outrageous if you go to a regular Theory shop. 

But anyway, a long time ago, Kurt Lewin talked about, “There is nothing so practical as a good theory.”  And I hadn’t really thought about that until yesterday, that really what I’m talking about is a kind of theory of readiness.  We really thought about it as a formula or a heuristic.  But really, it is a kind of theory.  And I think it connects well with what Karen and Arthur were talking about this morning.

So, this is the title, “Willing, Able, Ready – Supporting Evidence-based, Evidence-informed Interventions in Organizations with Different Levels of Readiness.”  The concept of readiness has come up quite a bit yesterday and today.  Paul brought it up a lot today.

So, I’m going to try to connect to the idea of, what is readiness?  How can we access differences in readiness, and how can we support organizational readiness to implement evidence-based, evidence-informed interventions?

So, an example that a number of people in the room are familiar with, but you can use this for lots of things you’re familiar with is a program called PREP, funded under the Affordable Care Act, Personal Responsibility and Education Programs, for teens, primarily teen pregnancy prevention oriented.  It’s in 44 states and the District of Columbia.  And the grants are made to state agencies to educate young people on both abstinence and contraception. They target young people who are homeless; in foster care; living in rural areas, or areas with high teen birth rates; from minority groups, including sexual minorities.  And there’s a big interest in replicating effective evidenced-based programs.  And there was a federal list that had 28 effective programs.  It’s now grown to 31.

And so, if you are a state that has PREP money, you are aware that there are a variety of organizations in your state, some of which seem more ready than others to implement evidence-based interventions.  But they may not be in the places or have the connections to the target group.

So, we really have to think about readiness, not only organizations that already know how to do evidence-based interventions, but are they the right ones to work with the youth that we’re interested in?  And also, the idea that Caryn mentioned this morning, that it’s not just the same old groups getting the money, but how do we build capacity more broadly?

So, in terms of readiness, there’s an assessment and a support.  We want to first figure out what readiness is.  There is a need for a straightforward method to assess readiness that provides information to customize support.

I’m going to briefly introduce parts of a framework that I co-developed with colleagues at CDC and colleagues from the University of South Carolina called “the Interactive Systems Framework for dissemination and implementation.”

Anyway, it was developed to bridge research and practice by both – by integrating research-to-practice models with models that are community centered and practice centered and start with the community and just don’t see the community or the practice as a robotic extension of the researcher.

So, one of the systems is called the delivery system.  These are the systems in the community, the CBOs, the community-based organizations; the health centers, et cetera.  And there are two kinds of capacity. The model heretofore has primarily focused on capacity building.

There’s a general capacity, and these are some of the kinds of constructs or ideas in general capacity.  What is that organization like?  Does it have a strong organizational identity?  Is it an organization on terms of organizational innovativeness that’s pretty accepting of doing new things, or are they very much tied to tradition and doing things the way they’ve always done?

Resource utilization.  Do they have the ability to really garner resources and use them efficiently and affectively?

Leadership.  Leadership is what we often think about.  How strong is the leader?  What kind of guidance do they have?  And also, a lot of these organizations have boards.  And we don’t often think about how the boards relate to some of these evidence-based interventions.

The structure.  How big is the organization?  What’s the decision making like?  Are there many kinds of people who could do the job, or are two people doing almost everything, and they’re pretty scattered and thin?

Climate.  What’s the feel of the organization?  Is it chaotic?  Is there a lot of turnover?  Does it feel secure?  Is it a warm and accepting environment, both for providers and for recipients?

And then staff capacity.  What kinds of staff are there?  What’s their professional experience?  What’s their professional credentials?  How many of them are there?  Et cetera.

These are the kinds of ways that we can think about how good an organization is at being a potential host for an evidence-based intervention.  A lot of these components can be thought of in a normal curve, that leadership can be – at the high end, there are some organizations that have really topnotch leaders.  There’s a whole bunch kind of in the middle, and then, of course, there are some at the low end.

And you can think of all those components of having some version of a normal curve.  We also need to think about innovation-specific capacity.  So, out of the 31 evidence-based interventions for teen pregnancy prevention, some are pretty easy to do.  For example, there’s one called What Could You Do that involves a clinic showing a 45-minute video.  That’s an evidence-based intervention.

There’s also Making Proud Choices, which is an eight-session curriculum, two hours – one hours each, and you have to – the providers need to learn all kinds of skills about how they work with those kids, including condom demonstrations.  There’s some real skills needed for a lot of that work.  Safer Choices is a two-year curriculum.

So, innovation-specific capacity is really important.  What are the demands required by a particular innovation?  And when we talk about innovation, our definition of innovation is something that’s new.  A program, or a policy, or a practice that’s new to an organization.  Okay?  It’s new to that environment or that community.

So, these are some of the elements of innovation-specific capacities.  And as I mentioned, they vary a lot in the knowledge and skills and abilities that they require.  Is there a program champion for that particular innovation?  Is there somebody influential in the organization who really wants to make that particular intervention or innovation work?

And what’s the climate like?  Some of these interventions and innovations really require a galvanizing of the organization to really wrap around it and do it, and it’s not just a little bit of a silo.

One of the things we’ve added, if you are familiar with the ISF – this is brand new for us, and I’m ashamed to say that even though I’ve been batting my head against brick walls for 20 years – that we’re finally getting around to thinking more systematically about motivation, talking with Gene Hall about how many of us sort of try – and he mentioned yesterday – to try to go over and not really think about the motivations as clearly as we should.  And we’ve started to do that recently.

So, in terms of motivation, here are some of the kinds of elements that might be thought of when an organization is considering doing a new innovation, something that’s new for them.  What’s the relative advantage?  How will this new thing compare to what we’re already doing?  Why will it be easier or better, or will it be easier or better?  And that will connect to some of the others.  Compatibility.  How compatible is this intervention?

So, some of those evidence-based interventions for teen pregnancy prevention, they may have been developed with some – needing some real advanced knowledge about sex ed., and you’re trying to work with middle school kids, and some of them might be more suited to high school kids.  So, you really have to think about the compatibility of that intervention with who’s being dealt with.

Complexity I’ve already indicated, with one being a 45-minute video, for example, and another involving a 2-year curriculum.  Different interventions have different amounts of complexity, and the motivation to do something like show a video can be quite different than requiring all the training and expense of doing a two-hour – a two-year curriculum.

Trial ability.  One of my grad students works for the state organization that does training in TA on teen pregnancy prevention.  And she was saying her experience is, an organization often wants to get some sense, before they go full scale, that this works in one or two places.  That’s it’s going to work well with their population.  They’re not going to get some really big pushback or rebellion about it.  They want to get some sense of satisfaction, both of the youth and the people providing it, and do some user satisfaction kinds of surveys.

Observability has to do with, can you actually feel and see a difference, or is there some sort of magic down the line?  And she talked about how they really want to see – how the organizations really want to get a sense of seeing that this really works.

Priority.  We all deal with this issue.  People can have capacity, but if something’s not a priority, if they are always surrounded by some other kind of emergency, and this is coming last or near last, all the other things sort of drop out.  So, really being clear about whether something’s going to be a priority is a really important thing.  And what makes something a priority?  Several of us have talked about the role of incentives and disincentives.

So, that’s the delivery system, whether they’re CBOs or other kinds of organizations.  Now, what we need to do, in the Interactive Systems Framework, is to have a way of supporting these organizations.  So, for example, with PREP, how do we get organizations ready to actually implement an evidence-based intervention?

So, we talk a lot about support systems which have tools, training, technical assistance.  And support systems also have a lot of these same elements.  What’s their general capacity?  If you are a national purveyor, how do you provide that training around the country?  Do you have people located regionally, or are you all in one place?  That really will affect the kinds of costs, et cetera, involved.

And training in TA organizations also have innovation-specific capacity.  There are no organizations, that I’m aware of, that can do evidence-based training on all 31 teen pregnancy prevention programs.  So, this is a way of funders being able to diagnose what kinds of capacities the support system is likely to be able to do in a feasible way.  And then we have motivation.

And then there’s the need to connect these systems.  How do we connect the support system with the delivery system?  How do we actually help them?

So, here’s – we have a logic model.  When we do our work, we have a logic model about how we’re trying to help build that capacity between the middle – between the box called “support system” and the delivery system, and this is our logic model.

And we’re very interested in something that we call an evidence-based system for innovation support.  And what that means is, if we think it’s important to be evidence based about our programs, then I raise the question, “Isn’t it important to be evidence based about how we do support, how we do tools, how we do training, how we do technical assistance?”

So, I’ll briefly describe this logic model metaphorically.  We want to achieve a certain desired outcome.  Let’s say it’s the ability to implement an evidence-based intervention with quality.  That’s the leftmost box.  What we know, and I mentioned this in the PREP example, is we often are starting out with organizations at different initial levels of capacity and motivation.  And that suggests that we can’t work with them as if one size fits all.

So, we have tools, and tools tend to be one size fits all.  It’s a book, and everybody reads the same book.  We have training which tries to give people a hands-on feel.  I like to use a metaphor of how we do this when we really care about something, and that’s teaching somebody to drive.  How many of you have had the joy of teaching somebody to drive?  Okay.

If you give them the book from the Motor Vehicle Bureau, do they learn to drive by reading that book or manual?  No.  You put them in a big class with 20 other kids, some of whom have been on their dad’s farm and driven a tractor, or they’ve had a go-kart or whatever, and some people have never been behind the wheel before.  But they’re all getting the same thing in that training.

So, I think of it as cost efficient, but not necessarily cost effective at getting them to that level of skill.  So, when you’re teaching somebody to drive, you have that individualized coaching or technical assistance, and you do it differently, depending on the press of that situation.

So, if they’re driving in the city of Philadelphia, they’re doing one thing.  If they’re driving on a rural highway, you teach them something else.  And if they’re driving on an interstate, you teach them something else.  And that’s one of the benefits of individualized coaching and TA -is making it relevant to them.

But we don’t give somebody a license because their mom or dad said, “He’s a good kid, and I’ve spent a lot of time with him, and you can trust him.  He’s ready to go.”  We actually have what we think of as quality assurance.  We have driver’s license tests.  And on those driver’s license tests, they don’t say, “Oh, you don’t like to parallel park?  That’s all right.  You don’t have to do what you’re not comfortable with.”

[Laughter]

That can undermine the quality of those driving skills, and the same kinds of stuff happens in a lot of our interventions.  So, we take that kind of thinking into doing evidence – to doing support: tools, training, technical assistance, and then quality assurance with a quality improvement component.  We want people to succeed.  We want people to get outcomes.  So, the system is iterative; it keeps building.  And what you see around the circumference is relationships.  We know that to do good training and TA, and particularly TA in coaching, you have to have a good relationship.  You really can’t connect with people and just tell them to do it.  They have to trust you.  They have to have some belief that you have some credibility.

So, if you do all those things, you come out on the other side presumably ready.  Okay?  And in this case we’re talking about ready to implement an evidence-based intervention.  What we’re in the process of doing now is trying to make this kind of – these kinds – these four components of support evidence based.

How do you – not all tools are equally good.  Not all training is equally good.  Not all technical assistance in coaching is equally good.  We want to build the evidence base for how to do them with quality.  I don’t have time to go into that, but I think the idea is pretty recognizable.

So, we’ve come up with a heuristic readiness equals motivation times general capacity.  And as Caryn has already said, we are talking about times, and times innovation-specific capacity.  If you have zero motivation, it doesn’t matter how much capacity you have.  And if you have only one of these kinds of capacity, you’re not going to be able to do it.

So, it leads us to this formula.  Okay?  So, in summary, what is readiness?  This is what I started with.  What is readiness?  How can we assess differences in readiness?  How can we support organizational readiness to implement evidence-based, evidence-informed interventions?

So, in terms of readiness, we’re talking about three major elements: the general capacity of the organization, the innovation-specific capacity, and the motivation.  And then we’re talking about a system of support that works with those organizations at their level to help them reach that level of readiness to implement an evidence-based intervention.

[Horns blowing]

[Laughter]

So, one of the things – I should end there, but I will say this –

[Laughter]

I think you get it.  The term is usually “ready, willing, able.”  And we really think it’s “willing, able” is what you need to be ready.