Classroom Context and Measured Teacher Performance: What Do Teacher Observation Scores Really Measure?
*Names in bold indicate Presenter
In this study, we focus on the relationship between the characteristics of a teacher’s class and a teacher’s performance. Using data from the Measures of Effective Teaching (MET) study, we leverage within-teacher variation in classroom composition to estimate the influence that student composition and classroom context have on measured teacher effectiveness, based on the Danielson Framework for Teaching (FFT) observation protocol. This approach allows us to control for fixed teacher quality to better isolate the idiosyncratic influence of classroom context. We further assess if measured teacher performance is more or less sensitive to classroom context when considering specific aspects of classroom management and instruction, and the extent to which the influence of classroom context is different for classroom generalists compared with subject specialists. Moreover, we address the nonrandom sorting of teachers to classes that has been shown elsewhere to bias measures of teacher effectiveness (Rothstein, 2009, 2010). To do so, we leverage the randomization of teachers to classes in the second year of the MET study, relying on the fact that some classrooms to which teachers were randomly assigned fully complied with their random teacher assignment, while other classes experienced partial compliance, and a third group of teachers were nonrandomly assigned to classes (full noncompliers).
We find that teacher performance, based on classroom observation, is significantly influenced by the context in which teachers work. In particular, students’ prior year (i.e., incoming) achievement is positively related to a teacher’s measured performance captured by the FFT. Incoming achievement is more strongly associated with observed ELA instruction than math instruction, and is also concentrated among subject specialists, who work with multiple classes of students within a single school day, compared with classroom generalist teachers who work with the same classroom of students all day, every day. Further, incoming student achievement significantly influences aspects of teacher performance that capture teachers’ interactions with their students, while core instructional practices are not prone to this influence. Finally, we offer evidence that the intentional matching of teachers to classrooms exacerbates the influence of incoming student achievement on measured teacher performance among ELA teachers. Taken together, these results suggest that, in the context of newly implemented teacher evaluation systems, greater caution must be taken when making high-stakes personnel decisions based largely on teachers’ classroom observation scores.