Association for Public Policy Analysis & Management

The American Community Survey (ACS) is an essential tool for monitoring changes in health insurance coverage brought about by the Affordable Care Act (ACA). The ACS has several advantages over other surveys including a large sample size that supports state level estimates, an uninterrupted health insurance time series dating back to 2008, and a rich set of covariates. However, like all surveys the ACS imperfectly measures health insurance coverage type. This paper examines state by year differences in the quality of Medicaid estimates in the ACS.

Medicaid enrollees are known to under report their coverage. A recent paper found that in the 2009 ACS the false-negative rate among known enrollees (identified through individual level linkage to administrative data) was 21.6%, in line with the National Health Interview Survey and the Medical Expenditure Panel Survey. However, unlike other surveys, the 2009 ACS over-counted administrative data on an aggregated basis. The authors suggest that this over-count may have been caused by the broad scope of the ACS question which combines Medicaid/CHIP and all other government funded coverage for low-income populations. The goal of this paper is to determine if the 2009 over-count has persisted over time and if the ACA has affected the correspondence between aggregate estimates from the ACS and administrative sources.

We compiled a panel of state level administrative counts of Medicaid and CHIP enrollment from 2008 through 2014, obtained from the Centers for Medicare and Medicaid Services (CMS). We merged to these data state level counts of Medicaid/CHIP enrollment estimated from the ACS and a set of state-by-year characteristics including Medicaid expansion status and other program and demographic characteristics.

Preliminary results suggest that in 2014 the ACS undercounted Medicaid and CHIP enrolment reported by CMS by approximately 4.4 million individuals (61.7 million in ACS versus 66.1 million reported by CMS). This is remarkably different than 4.3 million person overcount in the 2009 file. The shift in concordance appears to have been gradual overtime through 2013 when the ACS essentially matched administrative counts. However, in 2014 there was a large jump in discordance. CMS suggests a growth in Medicaid enrollment between 2013 and 2014 of 8.3 million whereas the ACS estimates a change of 4.5 million. The undercount in the ACS was correlated with expansion status: in states that expanded Medicaid the ACS undercounted by 16% versus 1% in non-expansion states. States that had the largest growth in enrollment per CMS tended to have the largest discordance. Additional regression analyses will examine the independent contribution of various other program and demographic characteristics to differences between the ACS and administrative counts.

This paper demonstrates that measurement error to Medicaid/CHIP reporting in the ACS varies by year and by state Medicaid expansion status. This is an important finding because it suggests that studies of take-up may be biased by measurement error that is correlated with Medicaid eligibility policy. Our results underscore the need for administrative and survey data linkages as called for in the recent Murray-Ryan Evidence Based Policy Bill.