Association for Public Policy Analysis & Management

Teachers are among the most critical inputs in educational processes. A good teacher strongly affects not only student test scores but also their non-cognitive outcomes, educational attainment and long-term labor market performance including earnings in adulthood (Chetty, Friedman, & Rockoff, 2014; Jackson, 2016; Kane & Staiger, 2008; Rivkin, Hanushek, & Kain, 2005; Rockoff, 2004). Despite consistent research demonstrating the importance of teachers, researchers have made surprisingly little progress toward identifying the substance of teacher effects, and it is not clear which teacher practices most strongly affect teacher performance. New observational protocols for measuring teaching quality have been identified in recent years, but the larger understanding of effective practice is still in its infancy, leading to a knowledge gap that severely limits the formulation of effective education policies, including professional development and teacher evaluation.

The development of computational methods provides an unprecedented opportunity to measure teaching practices at a larger scale, lower cost, and greater consistency. The current paper uses novel “text as data” methods to analyze transcripts of classroom videos by leveraging computational power instead of human time. I use data from the Measures of Effective Teaching (MET) project (Kane & Staiger, 2012) to create measures of teacher practices. MET is to date the most ambitious research project in the United States that looks into teacher and teaching effectiveness in America’s K-12 education system. In total, more than 2,500 fourth- through ninth-grade teachers in 317 schools located in six districts participated in this study in a two-year span (AY 2009-2010 and AY 2010-2011). One unique feature of MET is that it collected four video records each year for every teacher participant. Using word-to-word transcriptions of those classroom videos, I apply several “text as data” methods, including Structural Topic Modelling (STM), “bag of words,” and sentiment analysis to create metrics of teacher-student interaction patterns and their language features.

After evaluating the validity and reliability of these computer-generated measures, I link those measures to multiple measures of teacher effectiveness – including those based on student academic performance, observational measures and student survey response measures, and ask whether more effective teachers conduct more of those practices. By further leveraging the second-year randomization of students to teachers, I investigate the causal effects of such teacher practices.

This research contributes to the knowledge on why some teachers are more effective than others and how to improve teacher effectiveness particularly for students most in need of high quality instruction. This research also provides a demonstration of how to use computational social science methods to study teaching and learning in K-12 classrooms.