Association for Public Policy Analysis & Management

Improving the quality of instruction is one of the most significant and enduring problems of educational practice. Despite the importance of teacher quality to student achievement (see for example Chetty, Friedman, & Rockoff, 2014), efforts to evaluate teacher performance have, historically, done little to improve student outcomes. Recently, states and districts nationwide have revised their systems of teacher evaluation (Steinberg & Donaldson, 2016), often incorporating multiple methods of assessing and evaluating teachers, including student growth, rubric-based classroom observations, and student survey reports of teacher quality. In addition to establishing accountability provisions for persistently low-performing teachers, these systems provide new information about classroom practice to teachers and principals that is specific and potentially actionable. The district’s theory of action is that teachers reflect on the feedback, are connected with resources, and revise their instructional practices in ways that ultimately result in improved student learning. In fact, recent evidence suggests that teacher evaluation reforms have the potential to fulfill the promise of improving student outcomes (Taylor & Tyler, 2012; Steinberg & Sartain, 2015) and teacher performance (Dee & Wyckoff, 2015) via newly available information provided to teachers about their instructional practice.

In this study, we add to this literature by examining the following questions in the context of Chicago Public School’s (CPS) teacher evaluation system, Recognizing Educators Advancing Chicago Students (REACH):

1. Do teachers respond to evaluation ratings by improving their practice (e.g., via improved classroom observation ratings)?
2. Do students perceive improvements in their teachers’ classroom management and instructional practices (as measured by student survey reports)?
3. Do students show improved outcomes (i.e., test scores) as a result of the evaluation system?

For each question, we will also explore teacher-level and school-level heterogeneities to examine the extent to which improvements are driven by particular types of teachers (such as tenured versus non-tenured) or schools (such as those that serve students from different economic backgrounds). We rely on detailed administrative data for all students and teachers in non-charter Chicago Public Schools for each of the 2012-13 through 2017-18 school years. For teachers, we observe multiple teacher practice measures, including classroom observation ratings and value-added measures that are part of the evaluation system, as well as annual student survey reports of teachers’ classroom practices in grades 6-12. For students, the administrative data include demographic information and academic performance indicators including course grades and test scores from standardized exams.

The Chicago setting is an important test case of evaluation reform. CPS was an early adopter of more rigorous evaluations and has been implementing the same evaluation system since 2012. The analysis will include a mix of descriptive and quasi-experimental methods. We will rely on regression discontinuity methods to estimate the effect of evaluation reform (via teacher performance ratings) on teacher and student outcomes. Specifically, because each teacher receives a summative evaluation rating based on a continuous underlying evaluation score, we are able to estimate the effect of teacher ratings on teacher performance and student outcomes by comparing teachers at multiple evaluation rating thresholds.