We capitalize on a natural experiment in Rhode Island, borne out of the features of the school classification system, to identify the causal effect of being induced to implement intervention programs on student outcomes. Prior work on school accountability policies that categorize school performance has found mixed results regarding the impact on student outcomes (Dee & Jacob, 2011; Chiang, 2009; Rockoff & Turner, 2010; Rouse, Hannaway, Goldhaber, & Figlio, 2013; Winters & Cowen, 2012). Recent federal policy innovations, most notably the Race to the Top (RTTT) competition and the waivers from the requirements of the Elementary and Secondary Education Act (ESEA) have changed the policy and accountability landscape. In particular, many of the 39 states that have received a waiver from ESEA have implemented new accountability systems that use more complex metrics for gauging school performance that are then used to classify schools into performance categories.

Rhode Island is one such state that, as part of the conditions of its ESEA waiver, has introduced a six-tiered classification system for schools (i.e. priority, focus, warning, typical, commended, and leading) with the lowest three tiers of schools being required to implement interventions. School categorizations in Rhode Island are a function of an underlying continuous measure of school quality comprised of several measures of performance. The exogenous choice of cut-points to identify the performance categories, as well as the decision to have schools in lower-ranked categories to have to undergo intense reform, restructuring, or professional development, allow for a high-quality evaluation of the impact of this policy on student educational outcomes. Using a sharp regression-discontinuity design, we examine the subsequent educational outcomes of students in schools who were just identified as needing substantial school-improvement interventions compared to students in schools that just missed being identified as in need of these interventions. The underlying theory of action driving the classification system and interventions in Rhode Island seems to be that lower performing schools require more and more comprehensive interventions than slightly less poorly performing ones.  This theory of action seems shared by other states who have implemented similar programs (e.g. – Connecticut, Massachusetts)

Through a research partnership with the Rhode Island Department of Education, we link publically available classification and accountability data with student-level data to provide one of the first estimates of the causal impact of such policies for schools on the margin of being identified. Using data from the first two years of the implementation of the new classification system, we examine the policy impact on several relevant student outcomes. We measure the short-term cognitive impact of experiencing the interventions with state standardized test scores and participation rates. In addition, we also gauge the impact of these interventions on measures of student engagement using attendance, out-of-school suspensions, and high-school graduation as proxy measures. Preliminary analysis suggests that the impact of being just classified as needing intervention programs provides no short-term benefit to student outcomes and may have a negative impact on standardized test performance.