Association for Public Policy Analysis & Management

In 2002, President George W. Bush signed the No Child Left Behind (NCLB) Act into law. NCLB held schools accountable by requiring annual student assessments, making their results public, and applying sanctions to schools that failed to get sufficient numbers of students to reach proficiency in reading and math. Over the last ten years, a considerable amount of scholarship has been devoted to the impact of No Child Left Behind on school performance and student achievement. However, these evaluations have been challenged methodologically because NCLB was a national policy in which all states were subjected to the same federal requirements. As a consequence, many evaluations of NCLB have relied on repeated measures designs that examined abrupt changes in NAEP scores prior to and post implementation of NCLB. Hanushek and Raymond (2005), Dee and Jacob (2009), and M. Wong, Steiner, and Cook (2015) all found positive effects of NCLB, though the latter two found only significant effects for math but not for reading.

Under NCLB’s pre-waiver period, states set annual targets (in percentage of students proficient) that were ratcheted up over time. States had discretion over at least three important implementation decisions. First, they were allowed to select the measures used to assess students’ proficiency. Second, they could determine the steepness of the improvement trajectories that schools were required to follow. Third, they were allowed to introduce so-called “exemption rules” that include confidence interval and safe harbor rules. These policies effectively lowered the performance requirements for many schools. As such, although state proficiency requirements became more stringent over time, “exemption rules” also provided schools and districts with outlets to reduce accountability pressures. Taken together, these implementation choices introduced tremendous heterogeneity in the pressure placed on schools to improve, even during the pre-waiver period of No Child Left Behind.

The purpose of this study is three-fold. First, it is to describe the stringency with which states implemented NCLB policies between 2003 and 2011 (pre-waiver period of NCLB), and the ways in which stringency varied between states and across time. Second, the paper assesses the causal impacts of stringency in states’ accountability policies on school responses to meeting “adequate yearly progress” (AYP) requirements. Third, the paper introduces a new method for examining the effects of state stringency in AYP policies on student outcomes. To achieve these three aims, we introduce a new measure for describing the stringency of accountability rules under NCLB across states and years. Our proposed measure uses simulated AYP failure rates for a fixed sample of schools, thereby ensuring that the measure depends only on state policies, not on characteristics of schools within the state. Using this implementation measure, we are able to describe changes in stringency in AYP policies across time and states, as well as assess the impact of stringency in AYP policy on school and student outcomes. Overall, we find that as state policies became more stringent, schools did respond to AYP pressures under NCLB, although there was variation across states based on the types of policies that were implemented.