Association for Public Policy Analysis & Management

Although education reform has remained at the forefront of the domestic policy agenda in the United States, there is little consensus on the policy levers that are likely to be most effective in promoting school performance and student learning. Over the last 30 years, reform initiatives at federal and state levels have focused on holding schools “accountable” for student outcomes (most evidently under NCLB). However, observers of education reform argue that ESSA marks a return of state and local governments in American education policy. But was NCLB really such a centralized effort? Several researchers have already noted that state governments had substantial discretion in implementing NCLB standards (Davidson, Reback, Rockoff, & Schwartz, 2015).

Under NCLB’s pre­waiver period, states set annual proficiency targets (percentage of students proficient) to help schools and districts meet the 2014 federal mandate. However, states also had discretion over at least three important implementation decisions. First, they were allowed to select the measures used to assess students’ proficiency. Second, they could determine the slope of the improvement trajectories that schools were required to follow. Third, they were allowed to introduce so­called “exemption rules” that include confidence interval and safe harbor rules. These policies effectively lowered the performance requirements for many schools. As such, although state proficiency requirements became more stringent over time, “exemption rules” also provided schools and districts with outlets to reduce accountability pressures.

Until now, the implementation literature describing states’ responses to federal regulations under NCLB has been limited. Most studies focus on one to three years of states’ accountability policies or have looked at small, purposive samples of states that may not be representative of the United States as a whole. Additionally, most measures of accountability stringency have been based on one­dimensional indices of academic proficiency standards, such as test difficulty. Such one­dimensional measures may not capture the complex set of ways that state governments may weaken or strengthen the NCLB accountability standards. Finally, implementation studies of state accountability systems often are challenged by the possible correlation between implementation stringency and the population characteristics of schools and students within states

In this paper, we introduce a new measure for describing and measuring the "stringency" of states’ accountability policies. Our implementation measure is simulated AYP failure rates for a fixed sample of schools. A strength of our measure is that it reflects the multiple implementation decisions states made under NCLB for each year in the pre­waiver period. By focusing on how each state would score a fixed basket of schools, our measure is independent of population characteristics of the state. This is useful for establishing causal linkages between accountability policies and school and student outcomes. Using our stringency measure, our paper describes variations in state policy stringency under NCLB between 2003 and 2011. We then causally link state stringency rates to school and student achievement outcomes. Overall, we find that although schools responded to state accountability stringency, the same gains were not observed in students’ math and reading achievement scores, as measured by the NAEP.