Making the Cut: An Optimization Approach for Setting Cutpoints in Targeted Policy Interventions in Education
*Names in bold indicate Presenter
This paper borrows approaches used in the field of medicine for clinical diagnosis and proposes a flexible approach to cutpoint identification in EWI/EWS that minimizes misclassification error. Receiver operating characteristic (ROC) curve analysis lays out two related concepts that operationalize misclassification: (1) selectivity (true-positive rate) and (2) specificity (true-negative rate). The study proposes several objective functions of selectivity and specificity that can identify cutpoints that optimally separate target and non-target individuals. Further, flexibility is added to the objective functions by adding parameters that account for relative costs of misclassification and prevalence of the outcome of interest. The result is an approach that can identify optimal cutpoints under different contexts.
I illustrate the optimization methodology in two examples using school district data. First, I identify optimal cutscores on kindergarten and first grade assessments to identify students in need of early literacy intervention. The approach leverages longitudinal data to identify the score profiles of students who exhibit later struggles with literacy. Next, using detailed student daily attendance data, I produce thresholds on daily attendance rates at different time points throughout the school year that can be used to identify students at risk of becoming chronically absent (missing at least 10% of the school year). The thresholds can be used to provide these at-risk students with targeted supports during the school year to prevent them from missing too much class time. The optimization approach here has the potential to be useful moving forward as districts continue to adopt and implement EWI/EWS.