Poster Paper: Beyond Quasi and Natural Experiments: A New Taxonomy to Improve the Conduct and Interpretation of Causal Research

Thursday, November 8, 2018
Exhibit Hall C - Exhibit Level (Marriott Wardman Park)

*Names in bold indicate Presenter

Dahlia K. Remler, Baruch College, City University of New York and Gregg Van Ryzin, Rutgers University

Over decades of policy research, the labels quasi experiment and natural experiment have been applied to a wide range of causal study designs. Both labels imply causal evidence that comes close to a randomized experiment but, as we demonstrate, they are ambiguous terms and used inconsistently. Consequently, studies labelled quasi and natural experiments can confuse and even mislead policy makers and the public. Moreover, the terms lump together and fail to specify key dimensions of causal research designs, undermining the training of researchers and the commissioning of good causal research by policymakers. We briefly discuss historical and current usage of the terms quasi and natural experiment and their limitations. We then provide more precise definitions and propose a new taxonomy, grounded in traditional terms for key dimensions.

The resulting taxonomy differentiates causal study designs on the key dimensions of assignment exogeneity, researcher control, and intervention (see Table). Regarding the dimension of assignment exogeneity, we use the term “quasi” to distinguish non-random assignment from “randomized” assignment. The dimension of control refers to the researcher’s role in setting up the study and determining the assignment rules. A study can involve random assignment that is “uncontrolled,” for example when Oregon used a lottery to expand its Medicaid program, thus allowing researchers to study the causal effect of health insurance on healthcare utilization and health outcomes (Finkelstein et al 2012). In our taxonomy, this study is an “uncontrolled randomized experiment”. Finally, the dimension of intervention refers to a treatment or program that aims to influence the study outcome. In contrast, causes can be “natural” in the sense of being changes or actions, sometimes unplanned, that have an unintended effect on an outcome. Card’s (1990) study of the effect of the Mariel Boat Lift on Miami’s labor market, compared to other similar labor markets (not impacted by this sudden wave of immigration) is a “natural quasi experiment” in our taxonomy. The table below shows the taxonomy of study types that results from crossing these three dimensions of assignment, control and intervention.

We illustrate each design type in the taxonomy with a real-world policy research example, including several well-known natural and quasi experiments that have been influential in policy debates. We demonstrate the payoff from our proposed taxonomy: guiding students in how to conduct causal research; and improving the commissioning of causal research by practitioners and policymakers.


to treatments or conditions (X)



(Strong if as-if random,

Weak if non-random)


by the investigator






Treatment or program aimed at influencing Y

Randomized controlled trial (RCT)

Moving to Opportunity (MTO) study

Uncontrolled randomized experiment

Oregon Medicaid expansion study

Controlled quasi-experiment

Jobs Plus Evaluation

Uncontrolled quasi-experiment

Mexican floors study

Natural change or action that unintentionally influences Y

Unintended effects of a randomized experiment

Unintended effects of MTO

Natural randomized experiment

Vietnam draft lottery study

Unintended effects of a controlled quasi-experiment

Unintended effects of Jobs Plus

Natural quasi-experiment

Mariel boatlift study