Indiana University SPEA Edward J. Bloustein School of Planning and Public Policy University of Pennsylvania AIR American University

Panel Paper: Evaluation of the Effectiveness of a Low-Cost Statistical Methodology to Target Services to Participants of a Local Welfare-to-Work Program

Thursday, November 12, 2015 : 10:15 AM
Orchid B (Hyatt Regency Miami)

*Names in bold indicate Presenter

Randall Eberts, W.E. Upjohn Institute for Employment Research
This paper reports on an RCT experiment that evaluated the effectiveness of a low-cost statistical methodology to target services to participants of a local welfare-to-work program.   The purpose of the pilot was to improve the employment outcomes of participants by referring them to services that best meet their needs.  Funded by the USDOL and developed by the Upjohn Institute, the pilot referred welfare-to-work participants to one of three service providers based on a statistical algorithm that used administrative data to determine which provider offered services that were shown to be most effective for customers possessing specific characteristics and employment backgrounds.  Each provider offered different services and different approaches to delivering those services.  Before the pilot was established, the local Workforce Investment Board (LWIB) where the pilot took place randomly referred participants to the three different providers.  Therefore, the relationships between different types of services and employment outcomes for groups of participants with different characteristics were based on a randomized sample.  Using this sample, the observed employment outcomes were regressed against personal characteristics of the participants, and these relationships were then used to refer new enrollees to providers based on the enrollees’ personal characteristics and the recent experience of similar participants. The initiative demonstrated that customizing services based on participant characteristics could increase the effectiveness and efficiency of the intervention.  A random assignment evaluation of the pilot showed that targeting services in this way significantly increased the 90-day employment retention rate of participants by 20 percentage points, yielding a benefit-cost ratio of greater than three.  The paper will draw lessons from this experiment on how low-cost interventions have the potential to significantly improve the effectiveness of workforce programs.