Association for Public Policy Analysis & Management

This study reports on a project conducted with a large urban school district to develop and use predictive analytics tools to identify students who would be predicted to succeed in high school courses that are more advanced than they might otherwise take. The project used longitudinal transcript data for grades 8-12 for 4 recent cohorts of students. Courses in all academic subjects were classified at 4 different levels, with classifications determined separately in each grade. Research on this project builds on previous work on early warning models that we have conducted with the district, with an important difference: the analysis focuses on identifying predictors of success rather than risk.

The first phase of the analysis developed discrete choice models of the determinants of selected course levels in each year in high school. The models captured the dynamics of course choices that reflected transitions from one course level to another. In all of the models prior grade (by level) and absenteeism rate were very strong predictors of choices, much more so than mathematics and reading scores. Transitions – up and down – were perhaps surprisingly common. Nonetheless, prior level was a strong predictor of next year level. This analysis confirmed our view that course level enrollments are the product of both initial conditions – course level choices made in prior grades – and transition choices made in each year.

In the second phase of the analysis separate predictive analytics models were developed for all subjects by current and prior grade levels so that it was possible to predict how a student would perform if they, for example, enrolled in an advanced (level 4) course in grade 10, having been enrolled in 9^th grade in a level 1, 2, 3, or 4 course. Course grades were used as the measure of success. The predictive power of these models was high, ranging from 0.3 in the 11^th grade science models to 0.45 in the 11^th grade mathematics models. The models were used to predict the expected grade in each of the 4 course levels and the probability that a student would receive at least a C grade. This analysis indicated that there were large numbers of students who would be predicted to do well if they enrolled in a higher level course, but did not do so. Indeed, the models suggested that there was vast under-enrollment in AP courses overall and particularly in some high schools.

The models were used to: (1) predict the potential demand for advanced courses and (2) provide to a selected group of high school in the district the names of students who could be encouraged to take a more advanced course than they might otherwise take. We are currently working with the district to evaluate the effects of providing this information to the district, to address scaling up of this enrollment initiative, and to refine implementation of the initiative so that it increases opportunity rather than serves as a mechanism for denying access to advanced courses.