Association for Public Policy Analysis & Management

Efforts to dramatically improve chronically low-performing schools have begun to take root nationwide. Spurred in part by the School Improvement Grant (SIG) program as part of the American Recovery and Reinvestment Act of 2009 (ARRA), many states have undertaken initiatives to “turn around” such schools (Dee, 2012). In this study, I examine the effects of such policies in Massachusetts, a state with one of the highest-performing school systems in the country.

In January of 2010, Massachusetts passed legislation giving the state wider latitude to intervene in low-performing districts and schools. These policy changes positioned the state well in the federal competition for SIG dollars, and the state received $250 million in funding under the program in 2010. As part of the legislation, the state ranked all schools in terms of their history of performance and their improvements over the past few years, identifying the bottom 35 schools as Level IV schools. This designation made them eligible for SIG funds and required them to adopt one of a menu of school improvement models.

Two features of this policy lead to quasi-experimental designs that support credible causal inferences about the effects of these policies. First, schools were labeled as Level IV after testing in 2010, meaning that students in these schools experienced an arguably exogenous transition into a Level IV school. Using difference-in-differences and comparative interrupted time-series techniques, I estimate the effect of being in a Level IV school on student achievement. Second, the Level IV designation occurred according to a ranking formula, enabling a regression-discontinuity approach to estimate the effect of being identified as a Level IV school for schools near the cutoff. These complementary designs leverage different sources of exogenous variation and thus, together, support more credible causal inferences. Across the board, I find large, positive effects of being identified as a Level IV school. For example, in mathematics, being labeled as Level IV increases the achievement of the average student by 0.2 standard deviations.

In the next months, I will continue to extend this analysis by exploring the mechanisms through which turnarounds have had success. There are (at least) two possible explanations. First, they could have brought in more effective teachers, so aggregate student test scores increased. Second, they could have boosted the effectiveness of teachers already at the school. Distinguishing between these two hypotheses has important implications for policymakers. Using detailed data from Boston Public Schools, I will track the performance of individual teachers over time to disentangle these hypotheses.