Benefits, Costs, and Statistical Tests: Evidence from the Family Self-Sufficiency Program Interventions in the Work Rewards Demonstrations

Long, David; Long, David

This paper describes one of the first comprehensive benefit-cost assessments of the effectiveness of a federal Family Self-Sufficiency (FSS) program, which was conducted as part of Opportunity NYC–Work Rewards demonstration project.. It considers two interventions implemented as part of the demonstration to increase employment and earnings of families receiving Section 8 housing vouchers from New York City’s Department of Housing Preservation and Development. One is the FSS Program, which offers case management and escrow accounts to help voucher holders boost employment and build financial assets. The second is an enhanced version of FSS in which voucher holders are offered special workforce incentive payments to prepare for, obtain, and maintain employment. The experiences of families randomly assigned to FSS alone, enhanced FSS, and the control group were compared as part of the random assignment evaluation conducted by MDRC and sponsored by New York City’s Center for Economic Opportunity.

The benefit-cost analysis provides a fuller accounting of the economic value of the interventions, estimating the dollar value of the various outcome differences between these groups from the standpoints of families, taxpayers, and the general public (comprising families and taxpayers). For each outcome (such as earnings and housing assistance payments), as well as for the bottom-line estimates of net present value (based on all outcomes), the dollar value is estimated as a regression adjusted program-control outcome difference. Outcomes were measures for six years after random assignment and are extrapolated to cover a 10-year time horizon based on year-six results. Statistical significance tests are used to assess the likelihood that the bottom-line estimates of the value of the interventions are different from zero, and sensitivity tests are employed to assess how these estimates change when key assumptions are modified. To provide a fuller assessment of the statistical uncertainty surrounding these estimates, the analysis provides 90-percent and 75-percent confidence intervals around the point estimates.

The analysis finds that families assigned to the regular FSS program, without the special work incentives, especially those who were not working at the time of random assignment, are likely to come out ahead financially within a 10-year period, but the result is not statistically significant. For taxpayers, tax payments by participating families and reductions in outlays for transfer benefits eventually offset government expenditures for the program, leaving a modest, but insignificant net value. This generates a substantial, but statistically insignificant social net value. For the not-working subgroup, the net gain to families is improved by the use of rewards payments, and the measured gain over six years is significant. The 10-year estimates entail more uncertainty, but still suggest that the net return for those families is positive. It also appears that taxpayers break even after 10 years, despite the fact that the enhanced program is more expensive than FSS alone. The social net value of FSS plus incentives for this subgroup is large, but again not statistically different from zero, demonstrating the uncertainty surrounding estimates combining multiple outcomes covering 10 years.

Association for Public Policy Analysis & Management

Panel Paper: Benefits, Costs, and Statistical Tests: Evidence from the Family Self-Sufficiency Program Interventions in the Work Rewards Demonstrations