Panel Paper: Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables

Friday, November 9, 2012 : 8:20 AM
Hanover B (Radisson Plaza Lord Baltimore Hotel)

*Names in bold indicate Presenter

Matthew Johnson, Stephen Lipscomb and Brian Gill, Mathematica Policy Research

Researchers involved with estimating value-added models (VAMs) of teacher effectiveness agree that the validity of the measures depends critically on their ability to isolate teachers’ contributions to their students’ achievement growth. However, existing VAMs differ in key aspects of their empirical specifications, leaving policymakers with little clear guidance in the aggregate on what factors are important for constructing a fair model. We examine the sensitivity of teacher value-added estimates to including or excluding student and/or peer background characteristics and a double-lagged achievement score. Using data from a medium-sized, urban district between 2008-09 and 2010-11, we find that teacher estimates are most sensitive to including or excluding class average variables such that 20-30 percent of top or bottom quintile teachers place in a different quintile under the alternate specification. Teacher effects are less sensitive to student-level background variables and double-lagged achievement. We also explore the sensitivity of teacher VAM estimates to reductions in the sample brought about by requiring baseline scores from two prior years instead of one. The findings indicate a very high degree of correlation (i.e., 0.99), suggesting that sample exclusions may not introduce appreciable bias in an effort to control better for students’ incoming achievement, although requiring two years of lagged scores reduces the number of teachers with VAM estimates.