Association for Public Policy Analysis & Management

Over decades of policy research, the labels quasi experiment and natural experiment have been applied to a wide range of causal study designs. Both labels imply causal evidence that comes close to a randomized experiment but, as we demonstrate, they are ambiguous terms and used inconsistently. Consequently, studies labelled quasi and natural experiments can confuse and even mislead policy makers and the public. Moreover, the terms lump together and fail to specify key dimensions of causal research designs, undermining the training of researchers and the commissioning of good causal research by policymakers. We briefly discuss historical and current usage of the terms quasi and natural experiment and their limitations. We then provide more precise definitions and propose a new taxonomy, grounded in traditional terms for key dimensions.

The resulting taxonomy differentiates causal study designs on the key dimensions of assignment exogeneity, researcher control, and intervention (see Table). Regarding the dimension of assignment exogeneity, we use the term “quasi” to distinguish non-random assignment from “randomized” assignment. The dimension of control refers to the researcher’s role in setting up the study and determining the assignment rules. A study can involve random assignment that is “uncontrolled,” for example when Oregon used a lottery to expand its Medicaid program, thus allowing researchers to study the causal effect of health insurance on healthcare utilization and health outcomes (Finkelstein et al 2012). In our taxonomy, this study is an “uncontrolled randomized experiment”. Finally, the dimension of intervention refers to a treatment or program that aims to influence the study outcome. In contrast, causes can be “natural” in the sense of being changes or actions, sometimes unplanned, that have an unintended effect on an outcome. Card’s (1990) study of the effect of the Mariel Boat Lift on Miami’s labor market, compared to other similar labor markets (not impacted by this sudden wave of immigration) is a “natural quasi experiment” in our taxonomy. The table below shows the taxonomy of study types that results from crossing these three dimensions of assignment, control and intervention.

We illustrate each design type in the taxonomy with a real-world policy research example, including several well-known natural and quasi experiments that have been influential in policy debates. We demonstrate the payoff from our proposed taxonomy: guiding students in how to conduct causal research; and improving the commissioning of causal research by practitioners and policymakers.

Assignment to treatments or conditions (X)	Random		Quasi- (Strong if as-if random, Weak if non-random)
Control by the investigator	Controlled	Uncontrolled	Controlled	Uncontrolled
Intervention Treatment or program aimed at influencing Y	Randomized controlled trial (RCT) Moving to Opportunity (MTO) study	Uncontrolled randomized experiment Oregon Medicaid expansion study	Controlled quasi-experiment Jobs Plus Evaluation	Uncontrolled quasi-experiment Mexican floors study
Natural change or action that unintentionally influences Y	Unintended effects of a randomized experiment Unintended effects of MTO	Natural randomized experiment Vietnam draft lottery study	Unintended effects of a controlled quasi-experiment Unintended effects of Jobs Plus	Natural quasi-experiment Mariel boatlift study