Association for Public Policy Analysis & Management

Research Objective: First, we motivate the need for understanding the accuracy of self-reports of health insurance coverage in surveys. Then, we detail the reverse record-check design for the CHIME study, including the stratified sampling strategy, the treatment arms, survey design, data collection procedures, and data matching procedures. We describe the flow of health insurance questions, decisions about how to classify health insurance type for individuals based on their self-report. Finally, we list the research questions and corresponding analysis to orient the audience to the presentation of the study results.

 

Study Design: Using administrative records from a private health plan, individuals known to be enrolled in a range of different coverage types (public and private) were selected and randomly assigned to one of two survey treatments: the Current Population Survey (CPS) recently redesigned Annual Social and Economic Supplement and the American Community Survey (ACS). Insurance coverage at the time of the survey was confirmed.

Population Studied:Non-elderly Minnesota adults enrolled in a health plan were surveyed by telephone in May and June of 2015. Samples were randomly selected from five different insurance type strata: (1) employer-sponsored insurance, (2) non-group coverage including (3) qualified health plans (QHP) from the Marketplace, (4) Medicaid, and (5) MinnesotaCare (Minnesota’s basic health plan).

 

Preliminary Findings:  The survey achieved an overall response rate of 22% collecting survey data on 6,644 individuals from 2,660 households. In 87% of the surveyed households, at least one non-elderly individual was matched to administrative enrollment data, giving us an analysis sample of 3,823 individuals from 2,306 households. We found only small differences between respondent households versus non-respondent households and between households assigned to the two treatment arms. We found that questions included in the survey and the responses provided were not always sufficient to accurately discern specific coverage types. Specifically, we examined survey reporting of plan type, getting coverage on the marketplace, premium and subsidy among those with known insurance type. We contrast two approaches to categorizing insurance type: (1) a conceptual approach based on survey responses that match expectations of coverage type characteristics (e.g., enrolled through the Marketplace, charged a monthly premium, etc) and (2) a data driven approach that leverages enrollment data to uncover maximally accurate reporting patterns in the survey data. Estimates of the impact of misreporting on population-based prevalence estimates of coverage are derived using Bayes methods.

Conclusions:  Matched survey and enrollment data provides an important opportunity to study how individuals understand their coverage and the capacity of population surveys to categorize health insurance type. The CHIME validation study provides information on the accuracy of self-reports regarding health insurance coverage that can be used to inform the design of future health insurance coverage survey questions as well as how best to categorize an individual’s coverage type based on their responses to various survey questions. Findings can be applied to other federal surveys to improve classification of insurance type.