Association for Public Policy Analysis & Management

Classroom observations are the most widely used K-12 teacher evaluation measure in the United States. However, far less is known about the properties of observation scores relative to student test score-based measures such as teacher value-added. Using statewide administrative data from Tennessee, I assess the predictive validity of classroom observation scores collected as part of the state’s teacher evaluation system on a variety of student outcomes. Specifically, I adopt the “teacher switching” design used in Chetty, Friedman, and Rockoff (2014) to examine whether teacher observation scores are valid predictors of (1) student achievement, (2) attendance, (3) discipline, (4) course-taking, and (5) high school graduation. I conduct analyses using observation scores as the sole measure of teacher quality in addition to those that regress student outcomes on both teachers’ observation and value-added scores. Multiple approaches for the covariate adjustment and shrinkage of observation scores will be considered.

The primary contributions of this analysis are twofold. First, this analysis will be among the first to examine the predictive validity of observation scores obtained from an at-scale, consequential teacher evaluation system, supplementing the existing body of evidence using data from experimental, low-stakes settings. Second, access to administrative data allows me to validate observation scores (and value-added estimates) against a range of non-test score measures. Evidence on the multi-dimensionality of teacher quality from Gershenson (2016), Jackson (2017), Blazar and Kraft (2017) and others suggests that a considerable amount of teacher-driven variation in non test-score outcomes is missed by teacher value-added scores. If this variation is captured in classroom observations, as will be assessed in this study, these results would provide strong evidence to support and inform the use of multiple measure teacher evaluation.