Association for Public Policy Analysis & Management

Ten years ago, many education reformers championed rigorous and consequential teacher evaluation as an intervention that would improve the effectiveness of the teacher workforce and, in turn, increase student outcomes. The federal government and prominent philanthropies encouraged such reforms through a variety of high-profile initiatives (e.g., Race to the Top, the Measures of Effective Teaching Project, and NCLB waivers). In response, most states and school districts designed and implemented new teacher evaluation systems (Steinberg & Donaldson, 2016). Recently, however, enthusiasm has dimmed in the face of evidence suggesting a mixed record of implementation and efficacy. Even in districts with evidence of efficacy, the early promise of teacher evaluation may not sustain as these systems mature and change.

It is against this backdrop that we provide new evidence on IMPACT, the controversial teacher evaluation system in the District of Columbia Public Schools (DCPS). Prior research documented that aspects of IMPACT initially improved teacher performance (Dee & Wyckoff, 2015) and student achievement (Adnot et al., 2017). In this paper, we examine the evolving design features of IMPACT and the corresponding effects of its incentives on teacher attrition and performance under this mature and redesigned system. Notably, the changes to IMPACT include a de-emphasis on evaluating teachers with conventional value-added scores and an increase in performance standards for teaching. These higher expectations include a new rating category (Developing) for lower-performing teachers who would have previously been considered Effective. Even in the absence of these design changes, however, the longer-term effects of IMPACT are an open empirical question. Such reforms might be sustained if they remained well-implemented and catalyzed positive changes in school culture and performance. Alternatively, their effects might be attenuated in the context of a changed teacher workforce, as well as in response to leadership turnover, shifts in organizational focus, and internal pressure to limit their most binding consequences. However, IMPACT has evolved in ways that may well influence its effectiveness.

In this paper, we examine IMPACT’s evolution and the effects of revised incentives on teacher attrition and development, examining effects descriptively as well as causally, by employing a regression discontinuity (RD) design for Minimally Effective (ME) teachers, who face an immediate risk of dismissal, and Developing (D) teachers, who face a less-immediate threat. Descriptively, we find that lower-performing teachers are substantially more likely to either leave DCPS or to improve their performance than higher-performing teachers. RD evidence demonstrates that facing an immediate, performance-based dismissal threat increased the voluntary attrition of ME teachers, as did the less-immediate threat for D teachers. We also find qualified evidence that such threats increased the performance of ME teachers who returned.

The continuingly large effects we identify suggest that rigorous teacher evaluation can be sustained over the long term. Together, differential attrition and changes in IMPACT scores are pivotal for altering the composition and quality of the DCPS teaching force. Given that DCPS serves a primarily low-income, high-need student body, such changes are essential for giving these students an opportunity to learn and to succeed.