Panel Paper: Using Mixed Methods to Identify and Scale up Effective High School Practices

Friday, November 9, 2012 : 2:00 PM
International C (Sheraton Baltimore City Center Hotel)

*Names in bold indicate Presenter

Thomas Smith, Vanderbilt University

This paper will detail a mixed methods project focused on identifying the combination of programs, practices, processes and policies that make some high schools in large urban districts particularly effective with low income students, minority students, and English language learners and then collaborating with partner school districts to develop processes to adapt and implement these practices in less effective schools.

We first analyzed student achievement data to estimate value added models in order to identify schools that were effective at improving student achievement in core subject areas for students in traditionally low performing subgroups while also reducing dropout rates. We then employed a combination of interviews, surveys, and observations to uncover what the effective schools in our partner districts were doing that contributed to their success and distinguished them from less effective schools in the same district. Over the course of a school year, we visited each of eight schools (4 high value added and 4 low value added) for a week in the fall, winter, and spring. Across these three visits we interviewed school administrators, counselors, and other support personnel, interviewed and observed the instructional practices of 16 core subject teachers for at least four class periods, conducted focus groups with non-core subject teachers, student activity leaders, and students; observed faculty and leadership meetings; and shadowed individual students as they moved from class to class throughout the schools day. Interview data were coded to understand how the schools enacted what the research literature suggests are essential components of effective schools, including: quality instruction, a rigorous curriculum, a culture of learning and professional behavior, connections to external communities, systemic use of data, systemic performance accountability, and learner-centered leadership. Observations of classroom instruction were coded using the CLASS-S rubric, which focuses on the level of emotional support, organizational support, and instructional support in classrooms. Analyses conducted based on data collected in the first of our partner districts in 2010-2011 suggest that a distinguishing characteristic between high and low value added schools was the degree to which the school was organized around personalizing students’ academic and social learning experiences. Data collection in our second partner district will be completed this spring and analyses will be completed in time for presentation at the conference.

These distinguishing characteristics will then form the basis of a “design challenge”, where district leaders, school leaders and teachers from both the studied high value schools as well as school targeted for intervention will collaborate on the design and implementation of a process to support the sharing and implementation of the distinguishing in three intervention schools in each district. Both quantitative and qualitative methods will be used  to both provide formative feedback to the district design team as it fine tunes the intervention design, as well as data to assess the impact of the intervention designs and scale up processes. Our goal is to develop, implement, and test new processes that other districts can use to scale up effective practices within the context of their own goals and context.