Panel Paper: The Promise of Early Warning Data in Education: Impact of the Early Warning Intervention and Monitoring System After One Year

Saturday, November 4, 2017
Picasso (Hyatt Regency Chicago)

*Names in bold indicate Presenter

Ann-Marie Faria, Nicholas Sorensen, Jessica Heppen, Jill Bowdon and Ryan Eisner, American Institutes for Research


Graduation rates are rising nationally, but dropout remains a persistent problem. Additionally, students from historically disadvantaged groups continue to graduate at rates below national averages. Early warning systems, which use research-based warning signs to identify students at risk of dropping out of high school, have emerged as one strategy for improving graduation rates. These warning signs can include indicators of engagement, behavior, and course performance during middle and high school (Allensworth & Easton, 2005, 2007; Balfanz, Herzog, & Mac Iver, 2007). More robust, comprehensive early warning systems also emphasize matching and assigning identified students with interventions to help them get on track for graduation and continued monitoring of students’ progress in these interventions.

Although state, district, and school leaders are adopting early warning systems to prevent dropout, little is known about the impact of early warning systems on students and schools. This study examined the impact of one early warning system – the Early Warning Intervention and Monitoring System (EWIMS). EWIMS provides schools guidance and site-based support to implement a seven-step data review process. This process is supported by use of an early warning data tool that incorporates validated indicators of attendance, course performance, and behavior to identify students who are at risk of not graduating on time.

The study randomly assigned 73 schools in three Midwestern states to use EWIMS during the 2014/15 school year or to continue with their usual practices for supporting at-risk students. The study included 37,671 grade 9 and 10 students (18,634 students in EWIMS schools and 19,037 students in control schools).

Findings show that EWIMS reduced the percentage of students at risk of not graduating on time due to chronic absence and course failure, but not low GPAs or suspensions. EWIMS also did not have a significant impact on student progress in school (having insufficient credits) or on measures of schoolwide data culture. Specific findings included:

  • The percentage of students who missed 10 percent or more of instructional time was statistically significantly lower in EWIMS schools (10 percent) than in control schools (14 percent)

  • The percentage of students who failed one or more courses was statistically significantly lower in EWIMS schools (21 percent) than in control schools (26 percent)

  • The percentage of students who had a low GPA (2.0 or lower) or who were suspended once or more was not statistically different between EWIMS and control schools.

  • The percentage of students who earned insufficient credits to be on track to graduate within four years was 14 in both EWIMS and control schools

Overall implementation of the EWIMS seven-step process was challenging. Nearly all participating schools received low scores of implementation. Nevertheless, this study provides rigorous evidence that even with limited implementation during the first year of adoption, using a comprehensive early warning system can reduce the percentage of students who are chronically absent or fail one or more courses. These short-term results are promising because chronic absence and course failures in grades 9 and 10 are two key indicators that students are off-track for graduation.

Full Paper:

See more of: Big Data in Education
See more of: Panel
<< Previous Paper | Next Paper >>

Navigation
Additional Resources