Association for Public Policy Analysis & Management

The Indiana Choice Scholarship Program (ICSP), one of the nation’s largest statewide school voucher programs, has over 35,000 participating students enrolled in over 300 religious and non-religious private schools as of the 2017-18 school year. In this study, we build upon our previous work by examining the heterogeneous impacts of the voucher program on student achievement over the program’s first six years (2011-12 through 2016-17). Specifically, we disentangle the variation in impacts by many school-level characteristics, including by the schools’ racial/ethnic and socioeconomic compositions; proportion of English-language learners and special education students enrolled; average academic achievement; total school enrollment; and principal and teacher measures of school climate and instructional and organizational supports.

We analyze these impacts using longitudinal, student-level administrative data from the Indiana Department of Education. We also augment the aggregate school-level measures in the administrative data with survey data from a state-representative sample of 200 private schools statewide, including responses from 191 principals and 1,609 teachers (part of a broader study of school effectiveness in Indiana during the 2016-17 school year).

We estimate the annual achievement impacts on achievement for students who use a voucher to transfer from a public to a private school. Specifically, we exactly match voucher and non-voucher students of the same sex and race/ethnicity and are in the same public school, grade, and year in the year prior to a voucher student transferring to a private school. We account for selection by incorporating these “matching cell” fixed effects into an OLS regression model that also includes controls for baseline and pre-baseline achievement. We disaggregate the main impacts by the aggregate composition and academic characteristics of private schools by including interaction terms in our models.

Overall, we find that voucher students who transfer to private schools experience statistically significant losses in mathematics achievement, particularly in the first two years, with null gains in English/language arts (ELA) in comparison to peers remaining in their previous public schools. The effects on math become null and turn positive in ELA the longer students participate in the program (i.e., by the third and further enrolled in a private school). The greatest degree of heterogeneity in impacts based on the demographic rather than academic composition of the school. These findings suggest that the composition of the private school may be more important for students choosing to use a voucher to transfer from a public school than the school’s aggregate academic characteristics. We are in the early phases of incorporating the survey data on school climate, instruction, and organizational measures into our analyses and will examine this variation within the subset of schools providing survey responses. All analyses with survey data will include appropriate survey weights.

We discuss these results in the context of the growth and modification of Indiana’s voucher program and additional qualitative and survey research conducted on the voucher program. We conclude the paper by suggesting future mixed-methods research that may help explain our findings and uncover school-level conditions under which voucher programs may be most effective.