Panel Paper: Is the Pen Mightier Than the Keyboard? the Effect of Online Testing on Measured Student Achievement

Friday, November 9, 2018
8222 - Lobby Level (Marriott Wardman Park)

*Names in bold indicate Presenter

Benjamin Backes and James Cowan, American Institutes for Research

Nearly two dozen states now administer online exams. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students’ learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the fact that the state administers the same exam (PARCC) in online and offline formats each year during the transitional period. We find mode effects of about 0.10 standard deviations in math and 0.20 standard deviations in English language arts (ELA). There is little evidence of systematic variation in mode effects by student demographic groups, although on ELA tests they are larger for students scoring at the bottom of the achievement distribution.