Association for Public Policy Analysis & Management

Roundtable: Challenges and Strategies for Integrated Analysis in Mixed Methods Policy Research

(Tools of Analysis: Methods, Data, Informatics and Research Design)

Mixed methods are becoming more prevalent in policy research but can present many challenges to successfully carry out. While mixed methods are espoused in policy research and evaluation, it is often the case that qualitative and quantitative components of a study are designed and data are collected, housed, and analyzed separately. The qualitative and quantitative processes occur in parallel rather than in an integrated way. Having observed this in their own and many other organizations, the presenters offer an interactive panel, including a case example, of how software is one strategy to support more integrated mixed methods analyses. We begin with a brief grounding in what mixed methods research is, and strategies to assess when it can, and should, be used in projects. Dr. Lisa LeRoy, a Principal Associate in Abt Associates U.S. Health Division and director of Abt’s Qualitative Methods Center, will provide a brief overview of the different types of mixed methods research and evaluation and the strengths and weaknesses of each. Attendees will participate in open discussion and contribute from their individual experiences. Dr. LeRoy will utilize an interactive exercise to illuminate the challenges with combining methods. We will consider: What kinds of data can be used to answer evaluation questions? How can those data be integrated? What does integrated analysis look like? Dr. Anna Jefferson, Associate at Abt Associates in Social and Economic Policy, will provide a grounded example of integrated analysis from a recently completed study on consumers’ financial decision-making that used focus groups and surveys. This integrated approach produced stronger and qualitatively different results than would have been possible through parallel analyses of focus group and survey data. These materials are shown in a combined NVivo database to show software as one strategy to support integrated analyses. Dr. Cynthia Jacobs will orient participants to the types of mixed methods analyses and reports that can be generated using NVivo or similar qualitative analysis software. Dr. Jacobs is Regional Manager, Americas, for Training and Research Consultancy, at QSR International, the developer of NVivo and has worked with and trained hundreds of researchers and research teams in public health, the social sciences, and other contexts. The course is NOT hands-on software training. The workshop is an experiential session to discuss mixed methods questions, including the roles software can – and cannot – play in mixed methods analyses. The session will be moderated by Dr. Amy Castro Baker, Assistant Professor of Social Policy and Practice at University of Pennsylvania. Dr. Baker is currently developing a cost-effective mixed methods approach to supervised machine learning and Big Data policy analysis that involves direct partnership with public policy interest groups. The session will leave ample time for discussion of software and additional strategies for mixed methods research design, data collection, and analysis. The session will be useful for individuals both with, and without, prior mixed methods research experience.