Association for Public Policy Analysis & Management

The current federal emphasis on evidence-based approaches to teen pregnancy prevention (TPP) began in 2010 with congressional authorization of the TPP program and creation of OAH. The TPP program was one of six early evidence-based initiatives proposed by the Obama administration and authorized by Congress to increase the use of data and evidence in social policy (Haskins and Margolis 2015). The program provides roughly $100 million annually to state and local organizations to implement evidence-based and promising new TPP programs, with most of the funding going towards replication and evaluation of programs that have some existing evidence of effectiveness (Tier 1 grantees). Tier 1 grantees selected evidence-based programs from a list of 28 existing programs and curricula that the U.S. Department of Health and Human Services (HHS) had identified as having demonstrated evidence of effectiveness in reducing teen pregnancy, STIs, or associated sexual risk behaviors.  Grantees with evaluation requirements had to design and implement rigorous experimental or quasi-experimental impact evaluations.

To support the implementation of evidence-based programs with fidelity, OAH created an infrastructure to support their first cohort of TPP Tier 1 grantees (2010-2015).  Systems and supports included: a general technical assistance (TA) contractor to provide resources and trainings on important implementation topics; a web-based performance management system for grantees to utilize for continuous quality improvement; a process and tools for monitoring fidelity; and regular monitoring and support from  Federal project officers. To support high quality impact evaluations, OAH contracted with an evaluation TA provider that provided continuous monitoring of and TA to each grantee-led evaluation.

Nonetheless, this first cohort of TPP Tier 1 grantees experienced challenges scaling-up and replicating evidence-based programs in real world settings (Margolis and Roper 2014) – challenges that the program developers understandably may not have experienced when initially developing programs on a smaller scale. For example, grantees found that the length and intensity of some popular evidence-based programs were difficult to accommodate in settings such as schools and clinics. Some of the grantee-led evaluations also struggled to meet quality standards, such as enrolling a sufficiently sized evaluation sample and retaining them.

This presentation will describe how OAH drew from and leveraged lessons learned from the first cohort of grantees to prepare their second cohort of TPP Tier 1 (2015-2020) grantees in conducting higher quality implementations and evaluations. For example, OAH funded intermediary organizations to support capacity building for assessing fit of the evidence based programs with the planned setting, and then implementing the programs with fidelity in those settings.  OAH has also developed a web-based comprehensive resource center on TPP programs that provides grantees with detailed training materials on such topics as effective implementation strategies, successful approaches for recruiting and retaining youth, cultural competence, and engaging high-risk populations. Additionally, upon release of the funding opportunity announcement, OAH provided potential applicants with access to resources and webinars to support the development of high quality evaluations, drawing directly upon the experiences of the cohort one grantees.