Association for Public Policy Analysis & Management

Turning around the 5% of U.S. public schools identified as “chronically low performing” has remained an elusive goal despite prioritization at the highest levels of government (Gewertz, 2009). Previous research has examined school turnaround (Dee, 2012; Borman et al., 2003). However, there have been few attempts to rigorously evaluate district-wide efforts, although some theorize that district-level reforms may be better suited to long-term improvement (Zavadsky, 2013).

State takeovers of low-performing districts in the pre-NCLB era had some success with financial management, but less with improving academic outcomes (Wong & Shen, 2002). In the context of test-based accountability, portfolio management represents one relatively new approach to district improvement, placing the district in the position of overseeing a supply of schools with diverse managers (Bulkley, Henig & Levin, 2010; Hill, Campbell & Gross, 2012).

Lawrence provides a unique opportunity to examine a post-NCLB state takeover and attempted turnaround of a chronically low-performing district through a form of portfolio management implementation. Ninety percent of Lawrence students are low-income and 70% are learning English as a second language. Due to persistent underperformance, the State took control of the District and appointed a receiver, who was given broad authorities and began implementing turnaround efforts in 2012-13. The strategy included increased school-level autonomy, higher expectations, human capital improvements (including changes to the teacher compensation system in year 2), data use, and increased learning time.

We rely on data for all Massachusetts students from 2008 to 2014 to conduct a difference-in-differences analysis comparing the achievement trends of Lawrence students to the trends of a comparable group who did not experience turnaround. Our results suggest that Lawrence students made substantial improvements in math achievement during the first year of the turnaround. This finding is robust to a number of different specifications. We do not find overall year 1 effects in reading. We are in the process of examining results for year 2 and will be ready to present these results for APPAM 2015.

We also examine the effects of one particular component of the receiver’s expanded learning time efforts: the “Acceleration Academies.” This program provided struggling students with targeted, data-driven instruction delivered by a select group of educators over vacation breaks, and has potential for scalability. We find that participation in a 2013 ELA Acceleration Academy positively predicts gains on ELA achievement, and math participation predicts improved math performance. We also use propensity score matching techniques to model the process of selection into Academies, and use the 2013-14 data to examine whether the large effects of the year 1 Acceleration Academies appear to persist into year 2 of the turnaround.

We illustrate that the size of the combined effect of participation in an Acceleration Academy plus the rest of the turnaround bundle is smaller than, but still comparable to the effects of injecting high-performing charter school practices into traditional public schools in Houston, Texas (Fryer, 2014) and to winning the lottery to attend an oversubscribed charter school in Massachusetts (Abdulkadiroglu et al., 2011; Angrist et al., 2010).