Association for Public Policy Analysis & Management

The No Child Left Behind Act resulted in a abundance of assessment-based school interventions that aim to improve student performance. Interim assessments are school interventions designed to help teachers use evidence about student performance to modify and differentiate instruction accordingly in order to promote learning. Such assessments are viewed by many educational leaders in states and districts as promising levers for increasing student achievement (Davidson & Frohbieter, 2011).

The literature has produced inconclusive evidence about the effects of school assessment systems on student achievement. Carlson, Borman, and Robinson (2011) report small but significant impacts of the 4Sight assessment program on mathematics achievement in grades 3 through 8, but not on reading. Konstantopoulos, Miller and van der Ploeg (2013) found significant treatment effects are detected in grades 3 to 8, especially in third and fourth grade reading and in fifth and sixth grade mathematics. However, Cordray, Pion, Brandt, and Molefe’s (2012) study in fourth and fifth grades in schools in Illinois districts found no impact on reading performance.

This study examines the impact of a new diagnostic assessment system in Indiana. We estimated the effects of diagnostic assessments on mathematics and reading achievement using data from a randomized control trial (RCT) that was conducted in public schools in Indiana in 2010-2011. Our initial sample consisted of 70 schools that were randomly assigned to treatment (36 schools) or control (34 schools) conditions. Due to attrition the participating sample of schools was reduced to 55 (28 treatment and 27 control schools).  

Indiana state test (ISTEP+) scores were used as outcomes in grades 3-8. Terra NOVA tests were used as outcomes in grades K-2. The main independent variable denoted whether a school was assigned to the treatment (i.e., mCLASS, Acuity) or not (coded as 1 if school received treatment and 0 otherwise). Student and school covariates such as SES or school urbanicity were included in our models.

First, we conducted intention to treat analyses using student and school data from the initial sample of schools. Second, we used random assignment as an instrument to facilitate causal inferences of the implementation of interim assessments (see Wooldridge, 2010). Third, we conducted analyses using data from the participating sample of schools (treatment on the treated analyses).

Results indicated that the treatment effect was not statistically significant in grades K-8 or 3-8 both in mathematics and reading. The estimates were typically close to zero, except in fifth grade where they ranged between one and two tenths of a standard deviation, but were nonetheless insignificant. The treatment estimates in grades K-2 were negative and typically statistically significant both in mathematics and reading. This finding is unexpected and the magnitude of the negative effects is somewhat concerning because it suggests a disadvantage in early grades for students in schools that received the treatment (especially in kindergarten and second grade). Taken together the results consistently show statistically significant but negative effects in grades K-2, and statistically insignificant effects in later grades 3-8.