Association for Public Policy Analysis & Management

Improving the outcomes of students in schools labeled as failing or underperforming has been a long-held policy priority, and yet there is little evidence to date on how to do this effectively at scale. Recent work has addressed the evaluation of the recent school improvement grant initiatives from the federal government and found suggestively positive evidence for schools required to undergo the most substantial changes in response to the policy (Dee, 2012; Heissel & Ladd, 2014; Strunk, Marsh, Hashim, Bush, & Weinstein, forthcoming). This work has added to earlier iterations of school turnaround and improvement models under No Child Left Behind (NCLB) and found mixed evidence (e.g., Taylor et. al 2010). And yet, even without a strong foundation of what works, policymakers in congress and at the state and local level have proposed that NCLB reauthorization include greater state flexibility around appropriate interventions for underperforming schools. Taken together this suggests a dire need for high quality research aimed at evaluating the efficacy and efficiency of current efforts.

We investigate the impact of the latest, and as yet unexamined, federal educational accountability policy aimed at school improvement, the waivers to the requirements of the Elementary and Secondary Education Act (ESEA). ESEA waivers in many ways gave states more power to dictate both the nature and substance of school-level interventions relative to NYLB. In Rhode Island, the focus of this study, the state not only identified the particular interventions a school or district would implement but also how many. This new policy approach combined with the fact that Rhode Island was one of few waiver states to maintain its state testing regime after waiver approval, provides a novel opportunity to evaluate the impact of accountability policy when schools, identified as underperforming, must participate in a set of mandated interventions.

Using student- and school-level data we compare the subsequent performance of students in schools just below and above the thresholds used to mandate several levels of prescriptive interventions to estimate the impact of the policy. The exogenous choice of cut-points to identify the performance categories based on underlying continuous measures of performance, as well as the decision to have schools in lower-ranked categories to undergo intense reform, allow for a high-quality evaluation of the policy’s impact. Using a fuzzy regression-discontinuity design, we model the effects of being just subject to different performance tiers subject to intervention on the subsequent educational outcomes of students in schools who were just induced into needing interventions compared to students in schools that just missed being identified.

We find that schools in warning status and implement relatively few interventions perform no differently relative to typical schools on subsequent measures of student performance. Conversely, we find evidence that the lower-performing focus schools do worse than comparable schools that are mandated to implement fewer interventions. As one of the first papers to explore the newest form of federal accountability it serves to inform policymakers in waiver states and federal officials poised to reauthorize the ESEA legislation.