Association for Public Policy Analysis & Management

Since 2010 the US Department of Education has awarded 143 grants under the “Investing in Innovation Fund” (i3). The purpose of this program is focused both on supporting projects that have evidence of past success and positioning those projects to establish further evidence of success. This is accomplished through providing grants to “applicants with a record of improving student achievement and attainment” in hopes of expanding programs “that are demonstrated to have an impact on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increased college enrollment or completion rates.”

Considering the theme of this year’s APPAM conference, “The Golden Age of Evidence-Based Policy”, we propose to present a paper that considers the success of a funding mechanism designed to both produce high-quality programs and contribute to research. While we feel that much work has been done to understand the research provided by each individual project – the impact analyses and implementation analyses required by the Department of Education – less work has focused on the ongoing success of the educational programs themselves. How many programs under study are still in use in the schools and districts where they were studied? What reasons can be attributed to the sustainability or decay of programs? If lasting change was meant to ensure that successful programs are implemented and continue to be implemented in schools, has i3 contributed to lasting change?

For each of the 143 awarded grants since 2010, the full grant proposals and reviewer’s comments are publicly available on the i3 solicitation website. Our paper focuses on the awards from the first two years, 2010 and 2011, of grantees for two reasons; first, nearly half of all awards under this program were made in the first two years (49 in 2010 and 20 in 2011) allowing for a good representation of the overall program, and second this allows for the most amount of time for programs to have spent changing the educational landscape.

Our analysis focuses on the impacts that persist beyond the time of the investment, the knowledge generated and the lasting changes in schools. We gather this evidence from three sources: 1) For understanding the projects and learning who to contact regarding the projects, we focus on the proposals of the awarded grants, 2) to describe the evidence generated about programmatic impact, we search from published papers about the projects and interview a subset of the project evaluators, 3) to describe the lasting impact the programs have in the schools we rely on interviews with a subset of the program directors and contacts in the schools in which the programs were implemented.

Using these data we describe the lasting impact of a funding program that is focused on supporting evidence-based educational policy. This work has implications for the ways evidence is valued within solicitations and how evaluators can work to both support the innovations by providing formative feedback and by providing evidence of the project’s impact on their intended outcomes.