Association for Public Policy Analysis & Management

The first round of Health Profession Opportunity Grants (HPOG 1.0)—awarded by the U.S. Department of Health and Human Services (HHS) Administration for Children and Families (ACF) in 2010—funded health sector career pathways training for TANF recipients and other low-income individuals nationwide. Each grantee developed and implemented its own HPOG model, based on ACF guidelines and their own decisions about which program features and supports would be most effective in ensuring that participants enrolled in and completed their training. The impact evaluation of HPOG 1.0 involved randomizing 13,717 individuals across 42 HPOG programs to treatment or control groups.

This paper uses variation in program characteristics – including program design, local context, and population traits – to explore which characteristics associate with the magnitude of impact on four key outcomes: educational progress, employment, employment in healthcare, and earnings. The paper’s empirical specification approach uses forward stepwise regression to generate a set of candidate models and the Akaike Information Criterion (AIC) to select among these candidates. This procedure selects a parsimonious model and identifies the program characteristics most correlated with impact magnitude.

The particular characteristics associated with impacts vary by outcome; those that are successful in promoting one outcome are not necessarily successful in promoting others. For example, programs offering tuition assistance and other financial assistance tend to have larger impacts on educational progress but the program component is not identified by the selection procedure as important to explaining variation in impacts on employment, employment in healthcare, or on earnings. Similarly, the proportion of staff members who report that employment is the primary focus of the program is positively correlated with employment in healthcare, but this measure is not identified for inclusion in the model explaining impacts on educational progress, overall employment, or on earnings. The fact that an employment focus in a health sector training program is related to employment in healthcare but not necessarily overall employment is consistent with the kind of employment supports—e.g. connections with healthcare employers—offered by the programs.

As other papers in this panel, this analysis is non-experimental but intends to advance our knowledge of how to use experimental data to analyze the relative contribution of selected program characteristics to overall impacts. These findings are best used to generate hypotheses which could then be tested in an experimental evaluation.