Frequently Asked Questions

• What is the Teen Pregnancy Prevention Evidence Review?
• Who sponsors the Teen Pregnancy Prevention Evidence Review?
• How many programs and components meet the Teen Pregnancy Prevention Evidence Review criteria for showing evidence of effectiveness?
• When were the reviews conducted?
• How does the U.S. Department of Health and Human Services use the results of the review?

What is the Teen Pregnancy Prevention Evidence Review?

Since 2009, the U.S. Department of Health and Human Services has sponsored a systematic review of the teen pregnancy prevention research literature. This review helps identify programs and components with evidence of effectiveness in reducing unintended teen pregnancy, sexually transmitted infections and HIV, and associated sexual risk behaviors. The main purpose of the Teen Pregnancy Prevention Evidence Review (TPPER) is to review research to examine study quality and assess whether programs or program components have demonstrated positive impacts on sexual risk behavior and sexual health outcomes. Eligible programs reflect a variety of approaches in the field (for example, positive youth development, sexual health education, sexual risk avoidance, clinic-based programs, and healthy relationships). Eligible program components must be (1) a clearly defined practice, procedure, policy, support, or organizational structure, potentially with documented steps for implementation with fidelity to facilitate replication, and (2) (theoretically) capable of being implemented independently, in conjunction with, or integrated into a TPP intervention. In addition to being a resource to organizations that work to prevent unintended teen pregnancy, the TPPER is used by the Office of Population Affairs’ Teen Pregnancy Prevention (TPP) grant program and the Administration for Children and Families’ Personal Responsibility Education Program (PREP) to inform which programs grantees can select for replication. If your program is federally funded, please check with your funding agency to determine how evidence-based program components may be implemented into your funded project.

The TPPER is managed by the Office of the Assistant Secretary for Planning and Evaluation in collaboration with the Office of Population Affairs, and the Administration for Children and Families’ Family and Youth Services Bureau within the U.S. Department of Health and Human Services. The TPPER is conducted through a contract with Mathematica.

Who sponsors the Teen Pregnancy Prevention Evidence Review?

The TPPER is a joint effort sponsored by three agencies in the U.S. Department of Health and Human Services: the Office of the Assistant Secretary for Planning and Evaluation, the Office of Population Affairs within the Office of the Assistant Secretary for Health, and the Family and Youth Services Bureau within the Administration for Children and Families.

How many programs and components meet the Teen Pregnancy Prevention Evidence Review criteria for showing evidence of effectiveness?

Forty-eight active programs have evaluation studies that meet the TPPER criteria for evidence of program effectiveness. The programs represent a variety of program approaches, including sexual risk avoidance, sexual health education, clinic-based, healthy relationship, and positive youth development approaches. Two program components have evaluation studies that meet the TPPER criteria for evidence of effectiveness.

When were the reviews conducted?

The findings from the initial review were released in spring 2010 as part of the former Office of Adolescent Health (now Office of Population Affairs) Teen Pregnancy Prevention grant announcement. The review findings are updated periodically as new research emerges. Findings from the most recent update were released in September 2024.

How does the U.S. Department of Health and Human Services use the results of the review?

Within the U.S. Department of Health and Human Services, the Office of Population Affairs’ Teen Pregnancy Prevention (TPP) program and the Administration for Children and Families’ Personal Responsibility Education Program rely on the TPP Evidence Review findings to inform which programs can be selected for replication by grantees. Please contact the appropriate program office for questions about whether specific programs are eligible for federal funding.

• What criteria did the U.S. Department of Health and Human Services use to conduct the review?
• How did the U.S. Department of Health and Human Services define high- and moderate-quality study ratings?
• Who conducted the reviews?
• Did the review require studies to have a randomized controlled trial evaluation?
• Do studies of program components need to meet the same standards as studies of programs?
• Did the review look only at U.S. studies?
• Did the age criterion of 19 or younger refer to the age of participants at the time of initial intervention or the maximum age of program participants?
• How did you handle outcome measures of poor or questionable quality?
• Did the review consider findings for subgroups of participants?
• Why wasn’t race or ethnicity included as one of the priority subgroups?
• Do the studies have to appear in peer-reviewed journals in order to be included in the review?
• What outcomes did the review consider when examining evidence of effectiveness?
• Does the Teen Pregnancy Prevention Evidence Review discuss the implementation readiness of programs?
• Will there be changes to the Teen Pregnancy Prevention Evidence Review criteria in the future?

What criteria did the U.S. Department of Health and Human Services use to conduct the review?

In developing the review criteria, the U.S. Department of Health and Human Services (HHS) drew on evidence standards used by several well-established evidence assessment projects and research and policy groups, such as the What Works Clearinghouse, Blueprints for Healthy Youth Development, and the National Registry of Evidence-Based Programs and Practices. Based on standards used in these other processes, this review defined the criteria for the quality of an evaluation study and the strength of evidence for a particular intervention. Using these criteria, HHS then defined a set of rigorous standards an evaluation must meet in order for a program to demonstrate evidence of effectiveness. HHS reviews and updates the standards periodically to stay current with best practices in evidence reviews. Each iteration of the protocol is available here on the Teen Pregnancy Prevention Evidence Review website. The latest protocol is version 7.0 (PDF, 22 pages). This document summarizes how the standards changed between versions 6.0 and 7.0.

How did the U.S. Department of Health and Human Services define high and moderate quality study ratings?

The high study quality rating was reserved for randomized controlled trials with low rates of sample attrition, no reassignment of sample members, no systematic differences in data collection between the research groups, and more than one subject or group (school, classrooms, and so on) in both the intervention and comparison conditions. The moderate study quality rating was considered for studies using quasi-experimental designs and for randomized controlled trials that did not meet all the review criteria for a high-quality rating. To meet the criteria for a moderate study quality rating, a study had to demonstrate equivalence of the intervention and comparison groups on race, age, and gender; report no systematic differences in data collection between the research groups; and have more than one subject or group (school, classroom, and so on) in both the intervention and comparison conditions. Studies based on samples of youth ages 14 or older also had to demonstrate baseline equivalence of the intervention and comparison groups on at least one behavioral outcome measure.

Who conducted the reviews?

Trained researchers from Mathematica conducted the reviews. Two team members assessed each impact study; the first member conducted a detailed review of the study following a protocol developed by Mathematica and approved by a U.S. Department of Health and Human Services interagency work group; the second member assessed and verified the review for accuracy and completeness.

Did the review require studies to have a randomized controlled trial evaluation?

No. In addition to studies that use a randomized design, the review considered quasi-experimental (also known as matched comparison group design) studies that do not employ random assignment.

Do studies of program components need to meet the same standards as studies of programs?

Yes, impact studies of program components need to meet the same evidence standards as studies of programs. Studies of components must test the effects of the component rigorously—for example, by randomizing some youth to an intervention and others to the same intervention with an additional component (such as booster sessions). This allows the study to separate the effects of the component from the effects of the rest of the intervention. (Note: some studies rigorously test combinations of a small number of components.) For a component to be deemed evidence based, studies of that component must also show evidence of effectiveness comparable to the evidence needed for a program (that is, it has favorable effects on reducing rates of unintended teen pregnancy, STIs, or associated sexual risk behaviors).

Did the review look only at U.S. studies?

Yes. The review was limited to studies of programs that were conducted in the United States.

Did the age criterion of 19 or younger refer to the age of participants at the time of initial intervention or the maximum age of program participants?

The age criterion of 19 or younger refers to age at the start of the intervention. Participants might have been older than 19 during the study period or when outcome measures were assessed.

How did you handle outcome measures of poor or questionable quality?

Measures with serious limitations in terms of their validity or interpretation were excluded. For example, the review did not consider reports from males of their female partners’ use of birth control pills, or scales of behavioral risk that combine multiple measures into a single outcome.

Did the review consider findings for subgroups of participants?

Yes. In addition to findings for the full study sample, the review considered findings for two subgroups based on (1) gender and (2) sexual experience at baseline. The review considered the same outcome measures for these subgroups as for the full study sample—namely, measures of sexual risk behavior and its health consequences.

Why wasn’t race or ethnicity included as one of the priority subgroups?

The subgroup assessment was limited to address concerns about “multiple comparisons” or “multiple hypothesis testing.” As the number of subgroups examined in a particular study increases, the probability of finding a statistically significant impact also increases, just by chance. To address this issue, we chose to limit the number of subgroups considered for providing evidence of effectiveness. When selecting these subgroups, there were many relevant options to consider, such as race/ethnicity, gender, sexual experience, socioeconomic status, family structure, and many others. The U.S. Department of Health and Human Services ultimately chose to focus on gender and baseline sexual experience as the two subgroups for the review to consider. Moreover, in many studies, the sample sizes are too small to assess impacts separately by race or ethnicity. Future rounds of review will consider ways to include more subgroups.

Do the studies have to appear in peer-reviewed journals in order to be included in the review?

No. The review is not limited to peer-reviewed journal articles. The review includes studies reported as part of book chapters, government reports, unpublished manuscripts, or other documents.

By not restricting the pool of eligible studies to peer-reviewed publications, we are able to identify more recently evaluated studies. In addition, not all peer-reviewed publication venues are the same in terms of quality of the review. Instead, we focus on the quality of the evaluation study and assess the impacts based on the established review criteria. We require authors of unpublished reports to provide complete information needed to assess the quality of the evaluation study and its outcomes, and we ask them to make their publication available to the public upon request, if it’s not published.

What outcomes did the review consider when examining evidence of effectiveness?

The U.S. Department of Health and Human Services determined that programs and components with evidence of effectiveness must demonstrate evidence of a favorable, statistically significant impact on at least one of the following outcomes: sexual activity (initiation; frequency; or rates of vaginal, oral, and/or anal sex); number of sexual partners; contraceptive use (consistency of use or one-time use, for condoms or another contraceptive method); sexually transmitted infections or HIV (including testing, as long as it isn’t part of the intervention); or pregnancy or birth. It is possible that programs effective in influencing these behaviors also affect other types of adolescent health-risk behaviors. However, to be included in the review, programs must examine impacts on at least one measure of sexual behavior or its health consequences.

Does the Teen Pregnancy Prevention Evidence Review discuss the implementation readiness of programs?

The Teen Pregnancy Prevention Evidence Review describes aspects of implementation readiness, including implementation requirements and guidance (including training materials and resources), and potential adaptations. If your evidence-based program implementation is federally funded, any potential adaptations to the program need to be approved by your funding agency.

Will there be changes to the Teen Pregnancy Prevention Evidence Review criteria in the future?

The Teen Pregnancy Prevention Evidence Review contract is ongoing and before each round of review we consider changes to the eligibility, quality and effectiveness criteria to stay current with the field. Therefore, there may be changes to the criteria in the future.

• Is there a list of studies that were reviewed and did not meet the U.S. Department of Health and Human Services’ criteria for evidence of effectiveness?
• How do I determine if a program has evidence of effectiveness?
• If a program is listed as evidence-based on the website, does that mean every version of it is considered evidence-based?
• Where can I find information about the effect sizes for each of the relevant behavioral outcomes?
• Why do the findings of the U.S. Department of Health and Human Services Teen Pregnancy Prevention Evidence Review differ from similar reviews I’ve seen from other groups?
• Why aren’t there more programs with evidence of effectiveness for high-risk populations such as youth in the child welfare system and Americans Indian or Alaska Native youth?
• I don’t understand or I disagree with the rating a study received.
• I’m looking for a particular program with evidence of effectiveness, but it does not show up. Why was this particular program excluded?
• How can I identify which components are present in the “evaluated version,” “current version,” or “both versions” of a program in the program component tables in each program’s profile?
• What is the difference between program components that are shown as “present” and “core” in the program component tables in each program’s profile?
• What is an evidence-based program component?
• How are the program component tables in each program’s profile related to the evidence-based program components?
• As a developer or practitioner, what can I learn from the new evidence-based components?

Is there a list of studies that were reviewed and did not meet the U.S. Department of Health and Human Services’ criteria for evidence of effectiveness?

A full list of the studies reviewed is available under the Reviewed Studies section of the website. Study Quality “N/A” means a study was ineligible for review (i.e., it did not meet the inclusion criteria in the review protocol.) For instance, a study would be ineligible if it did not examine impacts on sexual behaviors or their consequences and only looked at precursors such as knowledge or intentions. A study would also be ineligible if it was conducted outside the U.S. or with a sample whose average age was greater than 20 years old. If the search comes back with “0 Studies found”, it means that TPPER has not yet reviewed the evidence for the submitted program (or author).

How do I determine if a program has evidence of effectiveness?

Only programs with current evidence of effectiveness appear under the “Find a Program” tab. If the program is marked “Inactive,” that means the program had evidence of effectiveness, but the program is not publicly available or implementation is no longer supported, or the only evidence of effectiveness is older than 20 years, so the program no longer meets the eligibility criteria.

To determine if a program has current evidence of effectiveness, go to the “Find and Compare” feature to select a program(s) of interest. You can also use the built-in filters—for instance, to identify programs designed to serve a particular population or designed to be offered in a particular setting.

The evidence for each program is presented by outcome domain. The quantity, shape, and color of the symbols provide a summary of the evidence in the domain (for instance, favorable evidence or conflicting evidence). The size of the shape indicates the number of studies with a moderate or high rating that examined outcomes in that domain (not just studies that showed favorable effects in the domain). There is a key on the Find and Compare page that explains the symbols and ratings.

If a program is listed as evidence-based on the website, does that mean every version of it is considered evidence-based?

No. Only the version of the program that was evaluated and presented in a study that meets the review criteria for evidence of effectiveness is considered evidence based. Some programs have multiple distinct versions that have met these criteria; for example, All4You2! is an adaptation of All4You! Each of these two programs met the review criteria for evidence of effectiveness (in separate evaluation studies), and their evidence bases are presented separately on the website. Other programs have multiple versions or adaptations, but only one version has met the review criteria for evidence of effectiveness; for example, Peer Group Connection — High School (PGC-HS) has met the review criteria for evidence of effectiveness, but the TPPER has not yet reviewed evidence for Peer Group Connection — Middle School. Finally, an evidence-based program’s implementation profile may describe other versions available (such as a version that meets the requirements of California’s Healthy Youth Act), but a program may only have evidence of effectiveness for one of the versions. Thus, only the version of the program as described in the study that met the review criteria for evidence of effectiveness should be considered an evidence-based program. Grantees who are receiving federal funding to implement an evidence-based TPP program with fidelity may be required to implement programs as evaluated in the studies that presented the programs’ evidence of effectiveness. Reach out to funders for specific requirements.

Where can I find information about the effect sizes for each of the relevant behavioral outcomes?

For all studies with high or moderate quality, the effect sizes for each outcome are available by clicking on the study citation from either the Study Findings page of the associated program profile or by searching the Reviewed Studies database.

Why do the findings of the U.S. Department of Health and Human Services Teen Pregnancy Prevention Evidence Review differ from similar reviews I’ve seen from other groups?

Each evidence review uses a slightly different set of procedures and criteria. Although there is usually overlap across reviews in which programs meet evidence standards, differences in the criteria used to screen and assess studies might lead to some difference in the programs identified as evidence based. In addition, each evidence review might have assessed a study’s evidence of effectiveness on a different outcome. For example, a positive youth development program might have been reviewed as part of the Teen Pregnancy Prevention Evidence Review for evidence of effectiveness on contraceptive use behavior, and the same program might be included in another evidence review to assess impacts on educational outcomes, violent behavior, or other outcomes.

Why aren’t there more programs with evidence of effectiveness for high-risk populations such as youth in the child welfare system and Americans Indian or Alaska Native youth?

The review does not aim to identify programs that serve any specific population. Rather, the goal of the review is to identify programs with the strongest evidence of effectiveness. If certain high-risk populations were underrepresented across the reviewed studies, it points to a gap in the research literature and need for additional research to identify effective programs for these groups. If you are aware of studies for high-risk populations that have not been reviewed, please send them in during the next call for studies period. The next call for studies is expected in fall 2024.

I don’t understand or I disagree with the rating a study received.

The About the Review section of the website provides a detailed description of the process used to determine the study ratings and an explanation of the criteria. This information should help explain the rating given to any particular study.

I’m looking for a particular program with evidence of effectiveness, but it does not show up. Why was this particular program excluded?

There are five reasons why a program might not have met the Teen Pregnancy Prevention Evidence Review criteria for evidence of effectiveness.

1. The program was not evaluated.
2. The review team did not identify any studies of the program during the literature search or through the call for papers.
3. The program might have been evaluated, but the study did not meet screening criteria (for example, looking at an eligible outcome).
4. The program’s evaluation was not sufficiently rigorous.
5. The program did not provide evidence of positive impacts on one of the key outcome measures for the full study sample or a priority subgroup.

Studies falling into the last three categories would, however, be included in the list of reviewed studies.

How can I identify which components are present in the “evaluated version,” “current version,” or “both versions” of a program in the program component tables in each program’s profile?

Each evidence-based program’s developer or distributor was asked to report on the individual components within their TPP program and indicate whether the components are present in the evaluated version of the program, the current version of the program, or both versions of the program. To view the components that are present in the version of the program that was evaluated, navigate to the program component table within the program’s profile, click the dropdown menu under “Has component,” and select “Yes (evaluated version).” The table will display all the components that are present only in the evaluated (but not the current) version of the program. Next, select “Yes (both versions).” The table will now display all the components that are present in the evaluated version and the current version of the program. To view the components that are present in the current version of the program, but not the evaluated version, select “Yes (current version),” and the table will display the components that are present in the current version of the program but not the evaluated version of the program. To view the components that are present in any version (current, evaluated, or both), select “Yes,” and the table will display all the components selected as present in any version of the program. Note that for dosage components, the dosage itself is described in the Notes when available.

What is the difference between program components that are shown as “present” and “core” in the program component tables in each program’s profile?

Each evidence-based program’s developer or distributor was asked to report on the individual components within their TPP program and the subset of present components that they consider to be core. For each of the seven types of program components, developers or distributors could indicate whether the component was present or optional in the program (and in which version of their program, if applicable) and whether they consider each of these components to be core. Developers or distributers could designate a component as core if the component is central to the identity of the program based on hypotheses around the theory of change of the program, based on existing evidence that demonstrates that the components favorably affect outcomes, or for some other reason. Components designated as core components of the evidence-based program in the program component table are not necessarily components that have been rigorously tested and show evidence of effectiveness. Most often developers denoted components as core based on theory or experience in the field. For a list of evidence-based components please go to the Evidence-based Program Component page.

What is an evidence-based program component?

Evidence-based program components are elements or activities of a program that have been tested and shown to have evidence of effectiveness in improving sexual behavior outcomes, with that effectiveness distinct from the effects of any associated program. Evidence-based program components may have been implemented independently, in conjunction with, or integrated into a TPP program. For example, the component sexual health risk reduction counseling was implemented as part of HORIZONS, and findings indicated that inclusion of the specific sexual health risk reduction counseling component to the HORIZONS program yields additional impacts on sexual behavior outcomes. The results do not mean that adding sexual health risk reduction counseling to any program would be considered evidence based. If your program is federally funded, please check with your funding agency to determine how evidence-based program components may be implemented into your funded project.

How are the program component tables in each program’s profile related to the evidence-based program components?

The program component tables in each program’s profile show the elements and activities that make up each evidence-based program. They were coded by the program’s developers or distributors using the core components checklist developed by the Office of Population Affairs. On the other hand, the evidence-based program components are program components that were tested and found to be evidence based. Evidence-based components could be components of evidence-based programs or they could be tested separately from an evidence-based program. Website users should read the evidence-based component profiles to learn about what was tested in the study reviewed for the evidence-based component. Although the impact evaluations of program components attempt to isolate the effects of each component, it is important to interpret these effects in the context of the programs they are delivered with. We do not have enough evidence to indicate whether the tested components would work the same way if they were implemented with a different program.

As a developer or practitioner, what can I learn from the new evidence-based components?

Over time, as the body of evidence about components grows, we will be able to better understand whether a component or combination of components are effective on their own or whether they are only effective in specific contexts or in conjunction with specific programs. Because effectiveness research on components is only beginning to emerge in the TPPER, we cannot make general conclusions about whether the components with evidence of effectiveness identified through the TPPER would be effective if implemented in conjunction with different programs than they were evaluated with. In the future, after the TPPER compiles a strong evidence base for components, TPPER users will be able to select the most effective components for a specific program aim or population so that they can tailor their program design or implementation to the unique needs of their community.

• How often will the review be updated?
• When new programs are deemed evidence-based, will they be considered eligible for funding under future replication grant funding opportunities if new money becomes available?
• Will future rounds of the Teen Pregnancy Prevention Evidence Review assess research from new studies of programs that have already been reviewed and recognized as evidence-based?
• Can I submit a study for review?

How often will the review be updated?

The Teen Pregnancy Prevention Evidence Review program is currently active, with studies identified through May 2023 included in the summary of findings released in September 2024. We expect to update the website with findings from the next round of review in 2025.

When new programs are deemed evidence-based, will they be considered eligible for funding under future replication grant funding opportunities if new money becomes available?

It is up to the individual program offices to determine whether grantees will be able to replicate programs that show evidence of effectiveness in the future. Though the findings from the Teen Pregnancy Prevention Evidence Review might be used to inform any future funding announcements, program offices might consider additional factors when deciding which programs are eligible for replication.

Will future rounds of the Teen Pregnancy Prevention Evidence Review assess research from new studies of programs that have already been reviewed and recognized as evidence-based?

The Teen Pregnancy Prevention Evidence Review program is currently active, with studies identified through May 2023 included in the summary of findings released in September 2024. As the U.S. Department of Health and Human Services identifies new studies of existing evidence-based programs, it incorporates these studies into the review process so the studies can contribute to the evidence base for that program. The website will be updated with findings from the next round of review in 2025.

Can I submit a study for review?

Yes. We will post a call for studies on the home page of this website for the next round of review when it opens. In addition, a notice will go out to those who sign up for the youth.gov newsletter. We expect to open the next call for studies in 2024.