Association for Public Policy Analysis & Management

As a condition of the Investing in Innovation (i3) grant awarded to the KIPP network of charter schools, this paper addresses a key research question: does KIPP maintain its demonstrated effectiveness as it scales?Specifically, as KIPP scales up the number of schools and grades served, what is its impact on student achievement? Are the previously documented positive impacts on students maintained in existing schools? And how do impacts of existing schools compare with those of new schools established under the scale-up?

KIPP is intentionally expanding vertically—into elementary and high schools in cities and locations where KIPP middle schools are already established. Yet evidence on KIPP is focused on middle schools. Do the positive impacts we’ve observed persist to the same extent in elementary and high school? There are reasons to believe that they might not. At the elementary level, KIPP is not enrolling kids who have fallen behind in school for five or six years already. At the high school level, the issue is somewhat different: KIPP is educating students who most often have already been in KIPP (having attended a KIPP middle school) for up to four years. In this way, there may not be sufficient “room” to generate additional impacts for these students.

Second, KIPP is also expanding “horizontally”—into new cities, but also by adding new middle schools in existing markets. In this way, the “treatment” at a new school may not be the same as at the early (or existing) KIPP schools, since it may be harder to recruit teachers, attract students, and so forth.  On the other hand, schools in existing markets may be able to capitalize on economies of scale from regional structures and supports already in place.

In this paper, we estimate impacts of KIPP schools on student achievement using a combination of approaches, capitalizing on the advantages of both experimental and quasi-experimental designs (QEDs). Specifically, at each school level (elementary, middle, and high), we employed the most rigorous possible study design to understand the effect of KIPP schools on achievement. We present impact estimates on reading and math scores as follows:

1. Randomized Controlled Trial (RCT). A lottery-based RCT compares lottery winners (treatment group) to lottery losers (control group) at KIPP elementary and middle schools.
2. Matched-student QED. A student-level propensity-score-matched comparison group QED compares KIPP students (treatment group) to similar non–KIPP students (comparison group) at KIPP middle and high schools.

3. Matched-school QED. A student-level QED derived from school-level matching compares outcomes for KIPP middle school students with and without the opportunity to later attend a KIPP high school (comprising the treatment and comparison groups, respectively). One version of this model compares students within a region, over time (before and after the KIPP high school opens); another compares a single cohort of students across different regions with and without KIPP high schools.

Using these models, we attempt to address the main sources of variation across schools, and discuss the implications for scaling a school network of this type.