Association for Public Policy Analysis & Management

K-12 education systems often use indicator-based “early warning systems” to identify students at risk of not graduating. An indicator-based approach typically produces a dichotomous measure of risk based on a snapshot of readily available measures of student behavior and performance. However, education systems are increasingly creating rich, longitudinal datasets with frequent, and even real-time, data updates of numerous, dynamic student measures including exam scores, course marks and daily attendance and tardiness. These rich datasets provide an opportunity for educators to compute frequent, more accurate and more nuanced predictions of students’ risk, which is extremely valuable given that high school students often can move from being on-track to graduation to off-track in a matter of weeks.

We will present a case study of an implementation of a rapid and iterative predictive modeling framework focused not only on high school graduation but also on other key education milestones. The framework was developed through a partnership between a nonprofit organization that supports district-run public high schools in New York City and an independent research organization. We describe the framework - a repeatable process that allows us to update predictive models as new information becomes available and to easily predict multiple milestones. We explore the value of machine-learning and other data-adaptive methods; we discuss the uncertainty of the predictions; and we present how predictions have been communicated to practitioners and used to guide intervention for students in 77 New York City high schools. Moreover, we discuss the value of a research-practitioner partnership in maximizing the quality, correct interpretation and use of predictive modeling results.