Association for Public Policy Analysis & Management

Policy makers rely on a variety of data sources and research methods to understand mechanisms and estimate parameters that inform the implementation and funding of social programs. In the field of child development, for example, understanding the effects of human capital investments made by parents is important for the study of intergenerational mobility. In practice, however, these effects can only be determined using non-experimental techniques—often on existing secondary datasets—that aim at establishing causal links between investments and long-term outcomes. Furthermore, current researcher degrees of freedom in conducting these studies too easily allow detrimental research practices, such as specification searching and data mining, that pose a threat to the credibility of the research. The goal of this paper is to argue for applying open, transparent, and reproducible science practices in regression-based, causal-inference methods. These practices are becoming commonplace in research that uses experimental designs and collects primary data. This paper begins with a discussion of the importance of using credible evidence to inform policy-making and threats to its credibility. We then discuss the open, transparent, and reproducible science practices proposed as a solution to these credibility threats. To demonstrate, we apply these principals to an analysis of intergenerational mobility that uses administrative data from Norway to empirically examine the relationship between (the timing of) parental income and child educational outcomes. Making commonplace the use of open, transparent, and reproducible science practices will increase the credibility of causal evidence of the regression-based econometric techniques frequently used to inform policy analysis.