As concern over the academic performance of students from disadvantaged backgrounds has grown, so has the interest in academically focused after- and summer-school programs.  While intuitively, these types of out-of-school time (OST) programs should improve academic performance because they provide students with more instruction and time to practice new skills, evaluations of many of these programs do not find academic impacts.  The most common reasons advanced to explain the lack of results are that the programming was not of high enough academic quality, not long enough or not time-intensive enough.  But some worry the issue is more fundamental—perhaps students are unable to absorb additional academic instruction in a more social environment after a full school day, or given the voluntary nature of supplemental programs, students would not attend these academic programs frequently enough or long enough to impact academic achievement even if intensive, long-term programs were available. 

To help resolve these issues, we conducted a RCT of Higher Achievement—a high-quality, time-intensive, multi-year academic OST program targeted at middle school students.  If this “Cadillac” OST program cannot achieve academic impacts, then the skeptics are likely to be right—significant academic advancement cannot be achieved outside of the professionalized, mandatory school environment.  If it can, then next the field can consider whether less intensive programs could also have academic impacts. 

We find that the program significantly increases students’ problem solving and reading comprehension scores two years after baseline. This high-standards program also causes students to reassess their perceptions of their own academic abilities. However, we find no evidence that the program improves test scores after one-year or that the program changes students’ perceptions of the support they receive from adults or peers.  Interestingly, we do not find that the program improves these students’ test scores or other outcomes over the summer. However, this finding may be unique to the types of youth who applied to this intensive academic program, as neither treatment nor control group members experienced the typical reading and math “summer learning loss.”  Finally, we find evidence that the program increases students’ desire to attend competitive area high schools rather than their local public schools.

Interestingly, improvements in test scores were not preceded by improvements in self-reported attitudes or behaviors. In fact, at both the one- and two-year follow-ups, Higher Achievement youth were more likely to report engaging in certain negative behaviors—a pattern thatresearchers are continuing to investigate.