Association for Public Policy Analysis & Management

Increased accountability over the past decade has led to the growth and prominence of standardized testing in U.S. schools and, subsequently, to increased test-preparation activities in classrooms (Au, 2007; Popham, 2001; Valli, Croninger, & Buese, 2012). Critics of high-stakes testing have long argued that test preparation detracts from students’ classroom experiences by crowding out more high-quality forms of instruction – often referred to as “inquiry-oriented,” “ambitious,” or “rich” instruction – in favor of routine practices aimed at boosting students’ test scores (Diamond, 2007; Popham, 1991; Popham, 2001; Valli & Chamblis, 2007). Some further hypothesize that low-quality test-preparation instruction results from the low cognitive demand of many high-stakes tests, which do not create incentives for teachers to engage students around ambitious instruction (Darling-Hammond & Adamson, 2014; Ennis, 1993; Fredericksen, 1984; Wiggins, 1991; Yeh, 2001).

However, to date, there exists little empirical evidence linking test preparation to instructional quality. Further, quantitative results come to mixed conclusions about the nature and quality of test-preparation oriented instruction (Firestone, Monfils, & Schorr, 2004; Hill, Blazar, & Lynch, 2015). To our knowledge, no rigorous analyses examine whether certain types of test-preparation practices – particularly “substantive” versus “non-substantive” coaching (Koretz, 2005, p.6) – are related to instructional quality or whether the quality of test-preparation practices varies across classrooms where students take different high-stakes tests.

This paper describes a study of 328 upper elementary math teachers in five urban school districts. Drawing on video-recorded lessons of their mathematics instruction, we examine whether teachers’ instructional quality differs between test-preparation and non test-preparation lessons. We further explore whether this relationship varies by the type of test-preparation activities in which teachers engaged their students or the specific tests that teachers were preparing their students to take. We attempt to gain some insight into the underlying causal relationship between test preparation and instructional quality by comparing instructional quality scores between lessons from the same teacher. This approach helps avoid bias due to individual teacher’s underlying quality and his or her propensity to engage in different levels of test preparation. As a robustness test, we instead condition our estimates on a rich set of controls to account for many of the factors that likely incent teachers to engage in different amounts of test preparation, including prior scores of teachers’ students on high-stakes math tests and teachers’ mathematical content knowledge.

Our results suggest that test-preparation on its own is unlikely to result in lower-quality instruction. We find only small and generally non-statistically significant differences in the quality of test-preparation and non test-preparation mathematics instruction. Further, we find inconsistent evidence on the role of test rigor. For example, we did not find any difference in the gap in instructional quality between teachers who engaged in high versus low levels of test preparation across districts, even though the test items differed substantially in their level of cognitive demand. Together, these findings call into question conventional wisdom describing a negative relationship between test preparation and instructional quality.