Association for Public Policy Analysis & Management

There is no agreement among educators about how to best support students who have fallen behind academically. High dosage tutoring represents one approach to individualizing instruction for struggling students that has demonstrated impressive results. However, two-to-one tutoring programs tend to come with a large price tag that, despite impressive benefit-cost ratios, could create challenges for scalability.

A less costly alternative to two-to-one tutoring is to provide small groups of struggling students with intensive instruction in a single subject over weeklong vacation breaks, delivered by regular classroom teachers selected based on evidence of teaching merit. For this approach, districts recruit teachers they consider to be high quality and have them work to help students who have fallen behind catch up through a relatively short burst of concentrated, small group, instructional time.

Several low-performing districts in the state of Massachusetts have deployed this strategy. Quasi-experimental evidence from the Lawrence Public Schools suggests that participation in these “Vacation Academy” programs produces sizable improvements on student test performance in both math and English Language Arts. However, previous research has been unable to completely rule out the possibility that selection bias explains part or all of the results.

To address this limitation, I conducted a field experiment, in the context of a set of low-performing middle schools undergoing turnaround reforms in Springfield, Massachusetts to study the causal effect of math-focused Vacation Academies during the 2015-16 school year. A total of 1,187 6^th and 7^thgrade students were randomized into treatment and control groups. A large majority of these students qualified for free or reduced price lunch (85 percent) and most were classified as either Hispanic (63 percent) or African American (22 percent).

Attendance at these “Vacation Academies” increased the probability students moved out of the “warning” and “failing” performance levels on Common Core-aligned math exams by between 16 and 22 percentage points and the program also had spillover effects on English Language Arts achievement. Importantly, Academy attendance also affected non-test score outcomes by improving course grades and reducing exposure to exclusionary discipline. Furthermore, eighty-six percent of participating students who responded to surveys about the program said they “had fun learning math.”

The program had particularly large impacts for traditionally disadvantaged subgroups. I also find suggestive evidence that progress was more dramatic for participants assigned to a single primary teacher for the entire week than for students who rotated through teachers specializing in particular lessons. Importantly, teacher selection was not highly competitive and the program cost 600 dollars per student, making it a potentially scalable approach to individualizing instruction.