Association for Public Policy Analysis & Management

Public schools in the United States spend an estimated $3 billion per year on digital content. However, there is often uneven usage of educational software, a large component of this budget item, across students, classrooms, and schools within a district, with many teachers falling short of vendor recommendations. We conduct a large field experiment with 1,327 teachers across four local education agencies, including district and charter management organizations, to examine the impact of a low-cost, teacher-focused intervention on student-level usage in grades K-8. Teachers in the treatment group received emails containing information on their students’ weekly usage compared to teachers in the same school, while the controls received no information. These “nudges,” on average, did not increase the teachers’ average student usage. However, teachers below the school average increased the average student usage by approximately 6 minutes (a 9% increase), while teachers above the school average decreased their usage by a similar amount. We plan to evaluate whether there are corresponding changes in the amount of software content that is completed by the students. We also plan to evaluate whether there is a change in Spring 2017 student test scores, although it is unlikely that a change in minutes of this magnitude will have a detectable effect. Our experiment provides preliminary evidence that targeted data-driven information can change classroom practice. Furthermore, we demonstrates how local education agencies can utilize weekly data from education technology to improve implementation at scale for relatively low cost.