Association for Public Policy Analysis & Management

We test whether personalized information intended to substitute for the loss of motivational interaction and monitoring inherent to in-person instruction can improve the performance of students in hybrid-format undergraduate classes. We conducted a field experiment with over 800 subjects taking an introductory economics course at a large, urban, public university system in the Spring of 2017. We stratify students within in each class by whether their GPA was above or below the median and then randomly assign students within each stratum to one of two groups (A or B). Those in group A receive four personalized emails prior to the midterm regarding either the amount of time they have spent online with the course material (the monitoring intervention) or suggestions that the student practice (ungraded) problems from each of four chapters available in the online platform (the motivation intervention). The problems are algorithmic, so students can do slightly different versions of the problem multiple times. The midterm includes four problems similar to these practice problems. Students in the face-to-face classes go over these same four problems in class. One question is whether students who receive nudges do better on those four questions than students in the same class who do not receive email nudges. We also compare the performance of students in the hybrid classes to those in the traditional face-to-face classes on the same four questions. The latter comparison is based on observational data as we did not randomize students between traditional and hybrid classes. However, we survey all students prior to the class to understand how students choose class formats in an effort to mitigate selection bias in the observational comparisons. In the second half of the course, only group B receives the four nudges/monitoring emails. Preliminary results find that light-touch interventions of this nature can modestly improve exam performance by encouraging either more consistent study habits or increased study time. While the effects do not compensate for performance losses from moving from traditional to hybrid formats, low-cost information provision may be one way to ameliorate learning disparities induced by modern delivery formats.