Association for Public Policy Analysis & Management

Panel: Moving Beyond Big Data: The Role of Qualitative Research in Understanding the Financial Lives of Americans

(Tools of Analysis: Methods, Data, Informatics and Research Design)

The mandate for quality research to inform policy has strongly emphasized the gold standard of large scale random control trial impact evaluation studies. While this focus has moved forward a rigorous approach to ensuring that policy programs have demonstrated impact at scale to determine whether public dollars should continue to be invested, some of the nuance and complexity of the social world has been lost from such studies. To provide a more balanced understanding of the impact of a policy, evidence-based policy must also be rooted in the social processes, norms, and contexts that can make or break policy interventions. Such research is best conducted from a research paradigm based in qualitative approaches that portray the richness and complexity of the social context in which a program operates. This panel will explore a range of different pieces of policy research that have emerged from the paradigm of constructing a deep and “thick” understanding of the social complexity surrounding a range of different programs. The panel will focus on drawing out answers to questions about how we can generalize from such work and where in the process of program evaluation such research has a place. The first paper in this panel by Li-Ning Huang from Fannie Mae will provide insight into the decision to conduct an ethnographically inspired study of first time homebuyer experiences. The paper explores the advantages that an in-depth qualitative approach provides in understanding the low-income first time homebuyer market. The second paper in the panel by Diana Eliot from the Pew Charitable Trusts explores the role of qualitative research in providing in-depth insights into the design of a larger scale study at Pew. The paper explores the ways that qualitative data informs our understanding of macro-economic processes and how policy responds. The third paper in this panel by Anna Jefferson, examines a study conducted by Abt Associates to better understand the financial decisions that people make. The study drew on an innovative approach of conducting focus groups followed by in-depth case studies of observed individual financial practices for a smaller sample. The paper highlights the biases that may be present in self-reporting of financial behaviors. The final and fourth paper of the panel by Hannah Thomas from Abt Associates, reports on how qualitative data can be used to inform policy in real time. The paper draws on the case of the City of Boston providing context on the advantages of qualitative research in providing real time data allowing policy makers to respond to an emerging policy crisis quickly, and priming other institutions to better understand lived family experiences.