School districts and their leaders are at the center of public attention, and some of the most popular and aggressively promoted school reform efforts such as mayoral control and teacher evaluation are focused at the district level.  But when we turn from policy prescriptions that assume the importance of school districts and their leaders to empirical evidence on their impact, we move from a rich to a sparse landscape.  Little is known about the impact of school districts on student achievement.  And what seems to be known rests on a questionable set of methods and assumptions.

Our project will use multiple years of student-level data from all public schools in two states to determine the degree to which variation in student achievement is associated with districts vs. schools within districts vs. teachers within schools vs. students within classrooms.  It will examine whether (and if so, how) effective and ineffective districts can be reliably identified and explore alternative methods for doing so.  And it will explore whether there are distinctive characteristics of leaders of exceptional districts, such as previous experience and route into the superintendency (e.g., a non-traditional training program such as the Broad Superintendents Academy).

This project will be conducted using statewide student-level data from Florida (2001-02 through 2009-10) and North Carolina (1996-97 through 2009-10).  The research methods used will include variance decomposition, hierarchical linear models, and ordinary least squares (OLS) regression.  The variance decomposition analysis will use a four-level hierarchical model (where the levels are students, classrooms, schools, and districts) to partition the variance in student achievement into the proportion associated with each of the four levels (both with and without covariates).

The measures of district exceptionality will be calculated using two distinct approaches.  First, district value-added estimates will be computed as the coefficients on district fixed effects in regression analyses that control for students’ prior-year test scores as well as student, classroom, and school characteristics.  The second measure of district exceptionality will examine the extent to which districts, over time, have reduced the variance in student achievement associated with classrooms and schools (using the variance decomposition technique described above).  These two measures will be examined in terms of their robustness to plausible alternative specifications, their correlation with each other, and their reliability over time.  The association between district exceptionality and superintendent characteristics will be estimated by regressing the measures of district exceptionality on superintendent characteristics, controlling for district-level covariates.

Many of these methods have already been applied to examine effects of teachers and schools, but they have not been used to estimate district impacts using statewide longitudinal data.  Consequently, this project will provide new evidence on the question of the relative importance of school districts for student achievement as well as insights that would be useful to policymakers and reformers who intend to improve student achievement through district-level efforts.