Studies that explicitly identified program implementation problems or suggested possible issues had smaller improvements on social competence than those that did not mention implementation at all or indicated no problems..

The most frequently cited implementation problems in the research were lack of adherence to a program manual or protocol, incomplete or inconsistent service delivery, and low participant attendance.

Put in place a systematic process for monitoring implementation so that you can determine if your program has an implementation problem, identify the problem and its sources, and plan how to address it. Without a system in place to monitor program operations and implementation, program managers may not have a full picture of any problems or, more importantly, what might be causing the problems. Monitoring implementation as part of an overall Continuous Quality Improvement (CQI) approach can help you diagnose problems and ensure that what you learn from monitoring is incorporated into staff training and professional development, program operations, and program design.

Assess Feasibility

Current systems. If you already have a system to monitor implementation, examine whether it provides you with the information you need to identify problems. For example, are you able to:
- Monitor program attendance through collection and analysis of administrative records or sign-in sheets?
- Monitor adherence to lesson plans or program guidance with fidelity checklists and identify points at which your program differs?
- Identify patterns and areas for improvement? Examine data to help identify possible causes or places for intervention and refinement.
- Assess service delivery quality with observations or participant surveys?
Fidelity standards. Does your program have specified thresholds of acceptable levels of implementation? For example, what parts of a program are essential to complete? What is the frequency and duration of a service to be considered full implementation?
Organizational capacity. If you do not have a way to systematically monitor implementation – or your current system does not capture critical elements of program performance – do you have capacity to introduce a new system or refine an existing one?
Organizational leadership. Is your organization’s leadership supportive and committed to integrating monitoring processes into the workflow?

Take Action

Consider any of the following courses of action depending on the results of your feasibility assessment:

Consider adding elements to your implementation monitoring system that may be helpful, such as benchmarks for acceptable levels of implementation based upon past program performance or minimum program requirements. Otherwise, wait until your program has accumulated enough data to set realistic benchmarks.
Use Continuous Quality Improvement or feedback loops to incorporate what you learn into staff training and program operations decisions.
Ensure staff have adequate training and ongoing support so they understand the importance of adhering to program guidelines, and have tools to enhance the quality of services they deliver.
Monitor data over time to see if there is improvement in the areas you are targeting – and to identify potential problems during the program period.
Consider working with an external evaluator, or conduct your own internal process evaluation to learn if your program is being implemented as planned and leading to desired results.

Breadcrumb

Monitor Implementation and Address Challenges

Outcome

Social Competence

Intervention Family

Effective Implementation