Panel Paper: Final Implementation Findings from the Multi-Year Evaluation of the Massachusetts Expanded Learning (ELT) Initiative

Saturday, November 10, 2012 : 2:25 PM
International E (Sheraton Baltimore City Center Hotel)

*Names in bold indicate Presenter

Beth Gamse, Abt Associates and Amy Checkoway, Abt Associates, Inc.

The five-year Abt Associates evaluation of the Massachusetts Expanded Learning Time (ELT) initiative focuses on three questions: How has the ELT initiative been implemented? What are the outcomes of ELT for schools, students, and teachers? What is the relationship between implementation and outcomes?

The study collected annual implementation and impact data from ELT and matched comparison schools.  Teachers and students were surveyed about their perceptions and attitudes, and key ELT stakeholders were interviewed about detailed aspects of program implementation.  Longitudinal student-level achievement data and student-level demographic and behavior variables were provided by ESE. The impact analysis used a difference-of-differences approach that leverages pre-program data and data from matched comparison schools to produce estimated effects representing differences between ELT and comparison schools beyond what one might expect given pre-program measures and other secular initiatives affecting all schools.

The study’s key contributions include refinement of how to collect and analyze information about school time use and schedules as well as the development and application of an implementation index based on principles of effective ELT operation. The implementation index provides a measure of fidelity that can be used both to understand school-level implementation and to explore relationships between implementation and outcomes.