Association for Public Policy Analysis & Management

Information plays a key role in markets with consumer choice. In education, information on school quality is gathered through standardized tests. The use of tests to assess school performance has became increasingly common. However, accountability systems resorting to tests are controversial among scholars and practitioners. Critics argue that high-stakes testing might generate undesirable behavioral responses that introduce distortions in the performance metric itself and the accomplishment of its goals (Neal, 2013). Despite increasing evidence of undesirable behavioral responses, quantification of these potential distortions and their consequences is lacking.

How large are distortions in school quality signals? What are the market consequences of these distortions? We study the Chilean educational system and show that distortions in school quality signals are large, and have significant consequences on school choice and the allocation of public programs. Chile has a market-oriented system, which uses standardized testing to generate school-specific quality metrics. These are used for quality assessment, performance evaluation and as disclosure system for school choice.

First, we show that low-performing students are underrepresented on test days. Using data on the universe of Chilean students, we compare daily attendance of test takers and non-takers across test and non-test days. High performing test-takers increase attendance on test days by 0.18s.d. more than low performing test-takers. Thus, a behavioral response to standardized testing is at work. This response varies considerably across schools. Second, we use multiple imputation to predict test scores of absent students and associated distortions in school test scores. Average distortions in the system are sizable: 0.1s.d. of school test scores. Distortions vary widely across schools, but are persistent within schools over time. Public, low quality, and for-profit schools display larger distortions. Third, we estimate a school choice model to quantify the implications of distortions. Providing undistorted school quality information would induce three percent of students to switch schools. The model implies that school switchers are willing to pay $117 yearly for undistorted quality information, with non-poor households willing to pay more than poor households. Fourth, we show that public programs are misallocated because of distortions. First, the government assigns bonuses to teachers in schools with high test scores. Thirteen percent of resources are misallocated each year, equivalent to $20MM since inception. Second, the government used test scores to classify schools and delivered this information to parents. Four percent of schools were incorrectly classified and this persuaded two percent of the incoming cohort of students to choose a different school.