Association for Public Policy Analysis & Management

Panel: New Evidence on School Accountability: Outcomes and Policy Differences

(Education)

States’ school accountability systems continue to use test measures under the 2015 Every Students Succeeds Act (ESSA). Studies of the 2001 No Child Left Behind Act and earlier accountability policies generally found moderately improved test performance, particularly in math (Figlio & Ladd, 2015; Figlio & Loeb, 2011). More can be learned, however, about test-based school accountability from new outcome measures and from differences in policy. While test scores are informative, other outcomes are relevant. Yet, few studies consider outcomes other than test scores. Further, few existing studies consider policy changes and differences between systems and across time. Not accounting for this variation in policy, evaluations of the effect of policy may be inaccurate. The papers in this panel address these gaps in evidence.

Three of the papers examine non-test score outcomes: college enrollment, cortisol levels, and non-achievement outcomes. The first paper, “College Enrollment and School Accountability,” uses postsecondary enrollment data from all eligible Title IV institutions from 1992-2010 to detect the effects of states’ school accountability policies. Preliminarily, the author does not find consistent evidence of an effect on college enrollment. The second paper, “Stress and Testing: How Students Physiologically Respond to High-Stakes Testing,” compares samples of cortisol, a stress hormone, from students in three charter schools during a high-stakes test week and a low-stakes test week. The authors find higher cortisol levels before high-stakes exams with large variation among students. The third paper, “Is School Accountability Healthy for Students? Evidence from NCLB Implementation Details,” examines how states’ implementation of accountability policies from 2003 to 2011 affected students’ non-achievement outcomes, including their engagement with exercise, smoking, risky sex, drug use, suspensions, and expulsions. The authors find that increased state accountability stringency does not seem to be related to changes in student non-achievement outcomes. The papers add to our understanding of meaningful non-test score outcomes.

The third paper capitalizes on Wong, Wing, Martin, and Krishnamachari’s (2018) measure of accountability stringency between states and over time. As policies differ between jurisdictions and are not static, effects can vary. Failing to measure these differences can mask the relationships of policies to outcomes. The fourth paper examines an increasingly relevant policy change within an accountability system.  In “Is the Pen Mightier Than the Keyboard? The Effect of Online Testing on Measured Student Achievement,” the authors assess the impact of switching to online tests from traditional paper tests in Massachusetts. The authors compare student test scores from online and paper versions of the same tests. The authors find test mode results in differences in math and English language arts (ELA), with little evidence for variation by student characteristic except for ELA tests where the difference is larger for lower achieving students. Accounting for policy differences is crucial for understanding effects.

Together, the four papers contribute new findings on the implications of accountability policy. The panel addresses understudied outcomes and the consequences of policy change. The papers are more relevant under ESSA, with states having more freedom to design their accountability systems.