Panel: [DATA] Survey Data Versus Administrative Data in Evaluation
(Tools of Analysis: Methods, Data, Informatics and Empirical Design)

Thursday, November 6, 2014: 8:30 AM-10:00 AM
Laguna (Convention Center)

*Names in bold indicate Presenter

Panel Organizers:  Burt S. Barnow, George Washington University
Panel Chairs:  Demetra Smith Nightingale, U.S. Department of Labor
Discussants:  Jacob Alex Klerman, Abt Associates

Do Estimated Impacts on Earnings Depend on the Source of Data Used to Measure Them? Evidence from Previous Social Experiments
Burt S. Barnow, George Washington University and David H. Greenberg, University of Maryland, Baltimore County

The Sensitivity of Impact Estimates to the Range of Data Sources Used: Analysis from a Canadian Experiment
Reuben Ford, Douwere Grekou, Isaac Kwakye and Claudia Nicholson, Social Research and Demonstration Corporation

Administrative and Survey Data in the UK Employment Retention and Advancement Demonstration
Richard Dorsett, National Institute of Economic and Social Research, Richard Hendra, MDRC and Philip K. Robins, University of Miami

There are two possible sources of data that can be used to estimate the impacts of social programs: data obtained from government agencies that maintain the data for use in administering their programs, and data collected from surveys of a research sample for the specific purpose of measuring program effects. Each type of data has its own advantages and disadvantages. Most importantly, they can produce different impact estimates. The proposed panel will include four papers that investigate this issue by comparing impacts estimated with survey-based data with those estimated with administrative-based data. All the papers focus on previously conducted social experiments that used data from both sources to estimate impacts. One of the papers provides an overview by describing the strengths and weaknesses of each data source and developing a simple model to investigate why differences between the impacts might occur. It then examines findings from eight previously conducted social experiments that estimated earning impacts with data from both sources. The remaining three papers provide more in-depth analyses of social experiments that were conducted in three different countries: the United States, Canada, and the United Kingdom. These experiments were all large-scale, and each used both administrative and survey data to estimate employment and earnings impacts. The papers examine whether that estimates produced by the two data sources differ, and if they do, the reasons why and adjustments they made to take account of known or suspected data problems. The proposed panel includes a suggested chair and discussant.