Panel Paper: Does Teacher Effectiveness Translate Across School Contexts? Evidence from a Randomized Experiment

Saturday, November 4, 2017
Dusable (Hyatt Regency Chicago)

*Names in bold indicate Presenter

Matthew Kraft, John Papay and Manuel Monti-Nussbaum, Brown University


Researchers and policymakers have focused considerable attention in recent years on efforts to measure and improve teacher effectiveness. A well-established body of literature has documented that teachers are an important determinant of student achievement, but much less is known about the degree to which teacher effectiveness is influenced by school contexts where they work. Much of the existing literature on teacher effectiveness explicitly or implicitly assumes teacher effectiveness is fixed and completely portable across different school and classroom settings. A growing number of districts have enacted policies based on this assumption such as incentive bonuses for high-performing teachers to transfer to lower performing schools that often serve a larger share of minority and low-income students.

Previous research on whether teachers may be affected by a change in environment, students, and peers, is decidedly mixed. Several studies testing the validity of value-added measures, which capture teachers’ contributions to student performance on standardized tests, leverage teacher transfers to conduct event studies and find that teacher effectiveness is largely transferable (Xu et al., 2012; Chetty et al., 2014; Bacher-Hicks et al., 2015). Other studies, find evidence that some aspects of teacher effectiveness are specific to grade levels (Ost, 2015), school contexts (Kraft & Papay, 2014), and the quality of the match between teachers and schools (Jackson, 2013). These studies, however, are limited by the observational nature of the data they use. Teachers mostly choose to transfer to similar or higher performing schools, providing only a weak test of the portability of teacher effectiveness.

In this study, we examine the portability of teacher effectiveness using experimental data from the Talent Transfer Initiative (TTI) study conducted in 10 districts across seven states. In the TTI study, 81 high-performing teachers from high-achieving schools were incentivized with a $20,000 stipend to move and remain at low-achieving schools within their districts. Participating low-achieving elementary and middle schools with open positions were randomly assigned to be eligible to offer this incentive bonus. The transfer incentive further introduced a degree of exogeneity into who transferred and created large contrasts between transferring teachers prior and new schools.

We test the portability of teacher effectiveness in a difference-in-differences framework with teacher fixed effects. We find transferring from high-achieving to low-achieving schools decreased high-performing teachers’ contributions to student achievement by between 0.10 to 0.15 standard deviations in the first year. Our second difference, teachers who transferred into control schools, suggests that this is not simply due to the transfer itself or any loss of school-specific human capital, but the effect of a large change in school context. Estimates using a sub-sample of teachers for which a second-year of post-transfer data is available suggest that a substantial portion of this decline in effectiveness persists. These results are robust to the use of several alternative constructions of the comparison group. We conduct a range of exploratory analyses to examine whether this decline is due to differences in school contexts, poor match quality, or the loss of grade and/or school-specific human capital.