Association for Public Policy Analysis & Management

Each year, schools and districts spend roughly $12 billion on textbooks and other instructional materials. Too often, districts and states must make decisions about textbook selection in the absence of empirical evidence of the success of these materials in promoting student achievement.

A handful of studies have identified positive effects of individual textbooks on student achievement (Agodini et al., 2010; Bhatt & Koedel, 2012; Bhatt, Koedel & Lehmann, 2013; Koedel et al., 2016). However, the existing research has focused on a relatively small sample of textbooks – just one or two textbooks in a given study. Further, no research has examined the relationship between textbooks aligned to the new Common Core State Standards, as assessed on tests aligned to these standards.

To fill these gaps, we designed a study in three phases. First, we collected information about textbooks used in upper elementary mathematics across six U.S. States. In four states, Louisiana, Maryland, New Jersey, and Washington, we administered a survey to a representative sample of schools (N = 1,100). In two additional states, California and New Mexico, textbook information was publicly available through school accountability report cards or purchasing records, allowing us to include a rough census of schools. Second, we conducted a teacher survey in a random subset of schools (N = 350), focusing on those using one of the top seven market share textbooks identified from phase one. Survey questions provide information about the intensity of teacher use of the text versus other instructional materials, teachers’ background in mathematics, and the types of instructional supports teachers receive. Third, in order to examine the effectiveness of textbooks used in fourth and fifth grade classrooms we matched textbook information to administrative data and student test scores. Given that textbooks and instructional materials are not randomly assigned, we control for characteristics at the student, school, and district levels. We also examine whether changes in textbook adoption within schools are related to changes in student achievement outcomes.

Survey data indicate that the vast majority of schools have been responsive to the Common Core, purchasing textbooks aligned to these standards. Teachers generally believe that these new textbooks support their teaching of these standards and student learning. However, they receive few supports to implement the new curriculum in their classrooms. Our value-added estimates differ from prior work evaluating textbook effectiveness. We find that very little variation in student achievement is attributable to textbooks. While a handful of textbooks seem to produce moderate to large positive effects on student learning, these estimates are sensitive to the states (and sometimes years) in which they are estimated. Preliminary analyses suggest that the Common Core may be partly responsible for minimal textbook effects. We find more variation in textbook effects for materials not aligned to the Common Core relative to materials that are aligned to these standards. If these results hold up to additional tests, that would be indication that Common Core partly met its stated go by aligning materials and instruction to a common set of standards.