Panel Paper: Refining a Measure of Implementation: The Evaluation of Citizen Schools Expanded Learning Time

Saturday, November 8, 2014 : 10:35 AM
Galisteo (Convention Center)

*Names in bold indicate Presenter

Beth Gamse, Abt Associates and Alyssa Rulf Fountain, Abt Associates, Inc.
Closing academic achievement gaps between advantaged and disadvantaged students is important to federal, state, and local policymakers. Despite improvements in academic achievement over the past few decades, proficiency gaps remain across income, racial, and ethnic groups.[1] Expanded Learning Time (ELT) represents a promising approach for improving academic achievement, particularly in our chronically low-performing schools. According to the National Center for Time and Learning (NCTL), there are 1,033 ELT schools in 47 states and the District of Columbia. [2] Five states have the greatest concentrations of ELT schools: North Carolina (106); California (99); New Jersey (94); Massachusetts (91); and Illinois (87).  The majority of current ELT schools (60%) are charter schools, yet, increasingly, ELT programs are being adopted by traditional public schools. [3]

Since 1995, Citizen Schools has developed and supported an ELT model using an additional shift of educators and community volunteers to engage middle-schoolers in hands-on apprenticeships, while simultaneously providing individualized supports to ensure academic and future success. Theory suggests that as a result of ELT, students may become more engaged in school because of additional enrichment and learning opportunities, develop better communication skills due to more time with teachers and peers, and be less likely to engage in disruptive behavior because of less idle time. Ultimately, student achievement may improve as a result of additional learning time.

 The authors are conducting a five-year national evaluation of the Citizen Schools ELT model that includes both implementation and outcomes components.  The study pays particular attention to understanding the role of implementation in explaining observed impacts, and the authors have developed an implementation index (based on interviews and surveys) corresponding to seven core elements of the ELT model, including:

 1) planning;

2) leadership;

3) data collection;

4) training and professional development;

5) family/community engagement;

6) alignment/coordination between partner school and Citizen Schools; and

7) core components of the Citizen Schools ELT model.

 Each program element is measured by indicators drawn from surveys and interviews conducted over four academic years (2010-11, 2011-12, 2012-13, 2013-14).  Fidelity levels for each program element reflect increasing and corroborated evidence that it has been implemented as intended. The index also measures congruence among respondents about each element. Scores do not represent a measure of the perceived or observed quality, but rather presence/absence of specific indicators. The paper will present findings based on variation on the index scores for individual schools, the initiative as a whole, and over time.


[1] National Center for Education Statistics. (2013). The nation's report card: Trends in academic progress 2012 (NCES 2013–456). Washington, D.C.: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.


[2] National Center on Time and Learning (NCTL): Expanded-Time Schools Database. Retrieved from; see National Center on Time and Learning (2012). Mapping the Field: A Report on Expanded-Time Schools in America. National Center on Time and Learning, Boston, MA.


[3] National Center on Time and Learning (2012). Mapping the Field: A Report on Expanded-Time Schools in America. National Center on Time and Learning, Boston, MA.