Association for Public Policy Analysis & Management

Researchers and policymakers examining school choice often assume that parents have well-ordered and stable preferences when it comes to their most important decisions, including where to send their children for school. A common assumption is that parents will be able to make the “correct” choice as long as they have access to complete information through school finders or “school shopping websites” (e.g., Levin 2002). But preferences may be more malleable than many assume, and parents might be influenced by the intentional or unintentional decisions that designers make about how to present information—how much detail to include, what attributes to describe, how to format the data shown for each attribute, and how to sort the options. At the same time, currently there is little consensus about how this information should be presented (Glazerman 2017).

We conducted an experiment to test how several different design decisions might influence the degree to which parents might favor, say, academic performance over distance from home, when choosing a school. The experiment investigated how different formulations of a set of school profiles affect parents’ school choices, using a hierarchical Bayesian analytical model to take advantage of the study’s highly structured set of “treatments” when drawing inferences about impacts. We also measured parents’ understanding, satisfaction, and ease of use for each tested information display.

To examine these questions, we asked a sample of 3,500 low-income parents to evaluate 16 hypothetical schools. We systematically varied how schools were presented using a factorial design, testing the effects of altering the sort order of schools, the amount of information shown, the display format, the sources information, and the inclusion of district averages as a reference point for each school. Each participant was randomly assigned to view one of 72 potential displays, after which participants selected their preferred school and then answered follow-up survey items regarding their understanding of the information in the display and their user experience.

We show that the design manipulations in the study did affect parents’ understanding (based on factual questions about schools), satisfaction, perceptions of the ease of use of the information, and school rankings (i.e., which school characteristics guided their choices). Specifically, we found that several design choices can meaningfully increase the weight parents place on academic quality when selecting a school compared to other attributes (such as a school’s distance from home). These results suggest that relatively simple and inexpensive design choices can produce meaningful changes in how parents interpret, process and use the information in a school guide. The rich factorial structure of the experiment also allows us to explore more nuanced questions regarding which nudges work, how weak or strong they are, and the embedded tradeoffs between designs that may produce unintended consequences. Insights from the experiment have important implications in the field of school choice policy, and also reveal practical implications for designers of school information guides or other complex information displays.