Defining Evidence Based Programs
Programs recognized as evidence-based have demonstrated the highest level of evidence of effectiveness based on the criteria below. These programs, if implemented with adherence to the program developer’s model, are likely to produce positive youth outcomes.
- Effectiveness demonstrated in rigorous scientific evaluations
It is important that program effectiveness is demonstrated using a study design with sufficient scientific rigor, preferably multiple independently replicated randomized or time series control trials. This increases confidence that the outcomes observed in the study are the result of the program, as opposed to some other unknown or external factor that might be contributing to the change. It also increases confidence that the program does not produce any iatrogenic or unintended harmful or negative effects.
- Effectiveness demonstrated in large studies with diverse populations or through multiple replications
When positive outcomes are found in large studies or across multiple studies, it increases our confidence that the outcomes are generalizable (i.e., apply across diverse populations and settings). Evidence of impact, seen in diverse populations (e.g., different socioeconomic, racial, and cultural groups) and diverse settings (e.g., urban, suburban, and rural areas), creates greater confidence that the same results can be generated across different types of populations, schools, and communities.
- Significant and sustained effects
Large longitudinal studies (ones that follow participants for several months or years) verify that positive effects are sustained over time. Unfortunately, many programs that demonstrate initial success fail to show long-term impacts after the intervention or may even show a decline in effectiveness. Also, programs sometimes have delayed impact and the full effects are not seen by the end of the intervention. Thus, it is important to assess impact, not just immediately following the program, but after sufficient time has elapsed.
Caution: Not All Program Lists Have Stringent Inclusion Criteria
- A variety of terms are used to refer to programs or approaches demonstrating varying levels of effectiveness, such as science-based, research-based, empirically supported, best practices, exemplary, model, and promising programs. It is important to realize that there is a continuum of effectiveness and that some programs promoted as effective may not meet all of the above criteria.
- The lowest levels of evidence are reflected in program assessments conducted through non-experimental designs or by endorsements of authorities with clinical experience only. In contrast, comprehensive, enduring, and effective prevention programs have strong study designs and scientific evidence that they reduce negative outcomes and lead to sustained positive impacts.
- Many online lists of prevention programs include programs which have undergone some form of study, whether scientifically rigorous or not, and that have produced some outcome findings, positive or negative. These sites include programs with varying degrees of research and it is left to the reader to make their own judgment about whether the evaluation results represent an effective program.
How to Identify Evidence-based Programs
- Internationally, The Cochrane Collaboration conducts the most rigorous reviews of evidence-based health-focused programs.
- Nationally, the Center for Healthy Youth Development (University of Colorado Boulder) publishes a list of evidence-based prevention programs based on stringent inclusion criteria. Blueprints Model and Promising Programs are identified for their rigorous research design and as prevention programs with demonstrated outcomes. Compared to any other national list of effective prevention programs, the Blueprints list is recognized as including only those prevention programs with the greatest evidence of effectiveness.
Further Information on the Criteria for Evidence-based Programs
- The National Center for Injury Prevention and Control - Centers for Disease Control and Prevention has published a guide explaining the continuum of evidence of effectiveness.
- The Standards of Evidence adopted by the Society for Prevention Research.
Steps to Selecting an Evidence Based Program
- Using local data identify risk and protective factors and problem behaviors in your community and prioritize short and long-term targets for population-based behavioral change.
- Identify an evidence-based program that addresses the locally prioritized targets and that is backed by strong evidence of a large effect in a population similar to the one you serve.
- Identify the key features of the identified program that must be implemented faithfully to achieve the desired outcomes. To identify these key features, review the website of the identified program and talk directly to the developer or program staff.
- Caution: There is strong evidence that seemingly small differences in the way a program is implemented can alter a program’s effects. It is importance to adhere to the model and implement the program as intended by the developer to ensure the same outcomes are achieved locally as were demonstrated in the research supporting the program.
- Conduct a community readiness assessment to ensure that capacity exists to implement the program with quality.
- If your site is deemed to have the appropriate resources to implement the program, work closely with the program developer and technical assistance providers to ensure appropriate training, implementation, and model adherence.
- Establish systems for outcome measurement and to monitor the extent of model adherence over time. Use the collected data to monitor implementation quality and model fidelity, to assess participant impacts, and to report to and generate support from community stakeholders and sustainability sources.
Evidence-based programs aim to not only impact outcomes for individual youth, but to change the population prevalence rates of a problem. The ultimate goal is to reduce current and future costs for expensive societal interventions, such as out-of-home placement, justice system involvement, drug and alcohol treatment, and social service and welfare usage. Evidence-based programs also have the potential to increase the health and well-being of youth, which promotes productivity as future citizens and increases tax revenue.