Panel: A Broader, Bolder Approach to Education Policy: Experimental Evidence On After-School and Summer Programming

Friday, November 9, 2012: 1:00 PM-2:30 PM
Salon D (Radisson Plaza Lord Baltimore Hotel)

*Names in bold indicate Presenter

Organizers:  Jacob Leos-Urbel, Claremont Graduate University
Moderators:  Jennifer Steele, RAND Corporation and Marc Stein, John Hopkins University
Chairs:  Karl Alexander, Johns Hopkins University

In her 2011 APPAM Presidential Address, Helen Ladd argued that “addressing the educational challenges faced by children from disadvantaged families will require a broader and bolder approach to education policy than the recent efforts to reform schools.” We have long known that children’s experiences outside of the traditional school day and year can have implications for their academic success. For instance, the summer learning loss literature suggests that differential summer experiences may contribute to the academic achievement gap that persists along racial and socio-economic lines. Yet, there are considerable gaps in our knowledge of how program and policy efforts after school and during the summer can influence educational outcomes. This panel contributes to this important policy question, with three papers that use experimental research designs to estimate the impact of summer and after-school programming on educational outcomes. “Achieving Academic Success Outside of School: A Randomized Controlled Trial of the Higher Achievement Program” examines a high-quality, time-intensive, multi-year academic out-of-school time program for middle school students. The program significantly increases students’ problem solving and reading comprehension scores two years after baseline, and also causes students to reassess their perceptions of their academic abilities. However, there is no evidence that the program improves test scores after one-year or that the program changes students’ perceptions of the support they receive from adults or peers. Interestingly, the program does not improve students’ outcomes over the summer. Further, improvements in test scores were not preceded by improvements in self-reported attitudes or behaviors. Summer school has been shown to improve academic achievement, but a major impediment to its success is poor attendance. "The Effect of Incentives for Increasing Summer School Attendance: Evidence from a Field Experiment" presents preliminary estimates of the impact of randomly-assigned incentives aimed at improving attendance at the Pittsburgh Public School’s Summer Dreamer’s Academy. The intervention, which includes treatments aimed at both students and parents, led to a statistically significant increase in attendance. Further analyses suggest that the parent incentives drove the effects of the combined treatment. Ongoing research examines impacts on student achievement, regular-year school attendance, and spillovers across siblings. Summer jobs have the potential to provide youth with new skills, income, and a positive developmental experience that may in turn improve academic success. However, little research has examined the impact of summer work on students’ educational outcomes. “What is a Summer Job Worth? The Causal Impact of Summer Youth Employment on Academic Outcomes: Evidence from a Large-Scale Lottery” studies New York City’s Summer Youth Employment Program (SYEP). Analyses validate the random nature of the SYEP lottery. SYEP leads to small significant increases in school attendance in the following year. SYEP also increases attempting some optional exams, but does not increase test scores. We believe these papers, which rigorously study a variety of summer and after-school interventions, belong together on one panel. This area is understudied and not well understood, and we believe that this panel will provide the foundation for a productive conversation about policy and management in education broadly defined.

Achieving Academic Success Outside of School: A Randomized Controlled Trial of the Higher Achievement Program
Leigh Linden, University of Texas, Austin, Carla Herrera, Public/Private Ventures and Jean Baldwin Grossman, Public/Private Ventures, Princeton University

The Effect of Incentives for Increasing Summer School Attendance: Evidence From a Field Experiment
Paco Martorell1, Trey Miller2, Lucrecia Santibanez3, Catherine Augustine3 and Jennifer Sloan McCombs3, (1)RAND Corporation, (2)Pardee RAND Graduate School, (3)RAND

The Equity and Effectiveness of a Market-Based Intervention for Increasing Student Achievement Under NCLB
Carolyn Heinrich1, Patricia E. Burch2, Annalee G. Good3, Hiren Nisar4, Mary S. Stewart3, Marcus Dillender1, Huiping Cheng3 and Rudy Acosta2, (1)University of Texas at Austin, (2)University of Southern California, (3)University of Wisconsin-Madison, (4)Abt Associates, Inc.

See more of: Education
See more of: Panel