Association for Public Policy Analysis & Management

A growing literature documents the problems with relying on survey measures of program participation, which suffer from considerable reporting error, when conducting impact evaluations. Administrative data are considered the “gold standard” to overcoming these econometric challenges, but relatively little evidence exists on the potential problems with administrative records or econometric strategies to address them. We investigate these issues using data from the FoodAPS, which combines a panel of household purchases with a survey and linked administrative data on Supplemental Nutrition Assistance Program (SNAP) participation from both state enrollment records and Electronic Benefit Transfer card expenditures. We first document substantial missing data in the two administrative participation measures and show that they are only roughly as strongly correlated with each other as with self-reported participation. Next, we show that estimated misreporting rates vary considerably depending on assumptions used to consolidate the two administrative variables into a single “true” participation measure. Finally, we utilize information from all three participation variables to estimate the effects of SNAP on food insecurity, obesity, and the Healthy Eating Index. A Bayesian imputation method computes, for each individual, the point estimates of the probability of participation and the uncertainty associated with this imputation, which reflects both missing data and discrepancies across measures. Using this predicted probability as the measure of SNAP participation, we estimate OLS and IV models that account for uncertainty in the prediction using multiple imputation methods. We then compare the results to those obtained using each of the three SNAP measures separately, ignoring uncertainty.