Association for Public Policy Analysis & Management

There is an important body of research focusing on the effectiveness of distinct types of educational programs when it comes to improve the academic outcomes of low achieving students: one-to-one tutoring, small-group interventions, classroom instructional process approaches, after-school / out-of-school / summer programs, etc. Aspects like the dosage of the intervention and the characteristics of tutors (professionals, paraprofessionals, volunteers…) have also been under scrutiny. We contribute to this literature by providing evidence that a relatively low dosage small-group tutoring program conducted by not professional tutors in after-school settings can be an effective way of improving academic outcomes.

Since their launch in 2001 the Exit (Success) program has offered low achieving students in their last two years of primary education (5^th and 6^th grades) and first two years of lower secondary education (1^st and 2^nd grades) after-school tutoring sessions and school support in small groups. Such sessions, of 1.5 hours each, are conducted by university students twice a week throughout the academic year.

In the 2015-2016 academic year, the Exit program was operating in 39 secondary schools and 93 primary schools in the city of Barcelona (Spain), involving a total of 2,600 students. And for the first time, it was also being made available to students in their last two years of lower secondary education (3^rd and 4^th grades).

We take advantage of this circumstance to evaluate the impact of the program on the academic outcomes of 4^th grade students, a cohort which in 2015-2016 was for the first time eligible for the program. Using a difference-in-difference-in-difference (DDD) model with student fixed effects – we estimate the intention to treat effect of the program by comparing the changes in outcomes for 4^th graders (eligible for the first time for the program; these are 2,800 students) and 2^nd graders (always eligible for the program; 3,000 students) in schools which offer the Exit program, with changes in outcomes for students in these same grades in schools not taking part of the exit.

We focus our attention on three outcomes gathered from students’ final assessment at the end of 2015-2016: number of subjects failed, both core subjects (mathematics, Spanish, Catalan and English language, natural and social sciences) and non-core subjects; probability of passing all subjects and of passing each individual core subject; probability of being promoted to the next grade.

Results show positive and significant effects of being offered the program on relevant outcomes: 4^th grade students, who become eligible for the first time, experience a decrease in the number of core subjects failed of 0.5 subjects (meaning a reduction of 20% in core subjects failed), increase the probability of passing all core subjects by 11 percentage points (24% increase), and increase the probability of being promoted by 14 percentage points (33% increase). Although a cost-effective analysis of the program is still pending, the findings resulting from the impact evaluation would encourage local authorities to involve not participating schools in the program.