Panel Paper: Using RCTs to Test Procedural Changes: Evidence From Medicaid Fraud Investigations

Friday, November 8, 2013 : 8:20 AM
DuPont Ballroom G (Washington Marriott)

*Names in bold indicate Presenter

Stephen Weinberg, State University of New York, Albany
This paper presents the results of a randomized control trial by a government agency in how it conducts Medicaid enrollment fraud investigations.

Medicaid enrollment fraud investigations are lengthy, expensive undertakings.  Consider a case that proceeds all the way to a “guilty” resolution.  When an individual comes to the agency’s attention as a potentially fraudulent enrollee, a supervisor makes an initial screening.  Then, an investigator reviews the target’s documents, conducts credit checks, accesses external databases, requests documents from third parties, and may even visit collateral contacts or gather evidence in the field.  The agency can then either refer to the case for criminal prosecution or can send the client a letter requesting an interview.  The interview “request” includes instructions to bring extensive documentation, including financial records.  The letter notes that the recipient may wish to bring an attorney and offers suggestions about free legal services.   The interview itself requires two agency investigators, and may require a translator.  This process can end in voluntary recoveries and Medicaid disenrollment or in civil litigation.

The agency decided to field-test a “stream-lined” version of its investigation protocol, designed to reduce some particularly burdensome steps.  The agency partnered with the author to  design and implement a randomized control trial.  From August, 2012 to January, 2013, new investigations were randomly assigned to either a “status quo” investigation or the stream-lined investigation.  The investigation targets were randomly chosen from a “high risk” list, ensuring that there were no unobserved variables guiding sample selection.  The agency also trained its investigators to record their “action steps” as they were undertaken, allowing the author to observe process measures such as how long each step took and which steps were needed.

The agency completed 520 investigations as part of the trial.  From a resource perspective, the stream-lined procedure was successful.  It reduced investigation time by 12% (after controlling for case characteristics, starting month, and investigation outcome).  Looking at successful investigations, the stream-lined procedure reduced the use of a particularly expensive step from 84% of cases to 53% of cases.  However, these savings in time and money came at a cost: it reduced the investigation success rate from 61% to 44% (controlling for case characteristics and starting month).

The agency plans to provide the author with additional case characteristics, to predict the types of cases for which the stream-lined process can be used without degrading the probability of success.  The paper will also discuss this follow-up analysis.  It will discuss general principles for successfully embedding a randomized control trial within an agency’s processes, such as the importance of close communication between the study designer and the agency’s “front line” staff.