Association for Public Policy Analysis & Management

Despite the rapid growth of charter schools in the past decade, their effects on achievement remain unclear. Inconsistent findings stem from differences in methods, samples, and research designs across studies. Small scale lottery studies on high performing charters produce impressive results, but large scale observational studies on the full range of charter schools are much less encouraging. To make matters worse, many large scale observational studies that aim for enhanced representativeness are based on only switchers (those who switch in and out of charters), a small and unrepresentative population which arguably defeats the purpose of this type of design. Thus, using statewide administrative data from elementary and middle school students in North Carolina between 2004 and 2014, we ask four questions:

1. Has the North Carolina charter school student population changed over time?
2. Are those who switch into and out of charter schools representative of the overall charter school population?
3. Is charter school student achievement growth larger than traditional public schools (TPSs) student achievement growth?
4. Does the charter school effect on test score growth vary by period or by race/ethnicity or by family poverty?

The charter school population in North Carolina has become whiter and less poor. In addition, the subset of the charter school population that has switched into charters has become a less representative and shrinking group. This empirical fact makes using a student fixed effects model an increasingly inappropriate method in which to investigate charter school effects. We support this claim with a discussion about the importance of drawing policy conclusions from studies with both strong internal and external validity. In short, we argue that both confounding bias and sample selection bias can introduce error into treatment effects about charter schools and that, in the absence of the size of each type of bias, there is no reason to privilege one over the other.

We estimate quadratic multilevel models with interval scaled test scores to study achievement growth differences between three groups: 1) those always observed in a TPS, 2) those always observed in a charter, and 3) those who switch into or out of a charter. We find, consistent with prior research, charter school performance has been improving over time. In recent years, while achievement growth has been similar or slightly higher in charter schools for white and non-economically disadvantaged students, it has been significantly higher in charter schools than in TPSs for black and economically advantaged students.

This study makes several key contributions to the literature on charter school effectiveness. First, our results confirm previous findings in North Carolina that charter schools are contributing to the growing racial segmentation of students by school sector. Second, this is the first study in North Carolina to examine the specific characteristics and representativeness of switchers and how this group has contributed to changes in the charter school student population over time. Finally, we analyze subgroup differences for socially and academically disadvantaged groups, an understudied area in prior research on North Carolina charter schools.