Association for Public Policy Analysis & Management

Over the last five years we have been working with college instructors to research how online exercises, text messages, emails, and electronic calendars can be used to improve academic achievement. The setup works remarkably well in getting students to engage and contemplate advice within their own contexts. Instructors impose a small participation grade at the start of a course for completing an online ‘warm-up exercise’. This leads to exposing virtually all students to randomized content designed to improve mindset, study habits and motivation. While some of these efforts reveal hints of improved study time, mental health and very enthusiastic feedback about user experience, the studies have yet to estimate markedly improved course grades or retention from my tested programs, including ones based on previous promising studies. This paper tries to make sense of what is happening using a statistical model and follow-up survey analysis.