Panel Paper: Learning “What Works” in Career Pathways Programming: The ISIS Evaluation

Saturday, November 9, 2013 : 10:05 AM
Mayfair Court (Westin Georgetown)

*Names in bold indicate Presenter

David Fein, Howard Rolston, David Judkins and Karen N Gardiner, Abt Associates, Inc.
The Innovative Strategies for Increasing Self-Sufficiency (ISIS) evaluation, sponsored by ACF, is using an experimental design to assess nine promising post-secondary career pathways programs that promote improvements in education, employment and well-being for low-skilled, economically disadvantaged adults and youth.  Although there is a great deal interest of in these programs among policy makers, practitioners, and researchers, they have not been rigorously evaluated.  Though the nine programs in ISIS utilize the basic building blocks in career pathways (assessment, basic skills and occupational skills training, supports, and employment connections), they vary on a substantial number of fundamental design features.  These include target populations, array of services provided, goals for participants, and timeframes for expecting goals to be achieved.  Although all programs share a primary goal of increasing earnings through access to and facilitating the completion of education, they take very different approaches, dedicating very different levels and kinds of resources, to get there.  The first section of this presentation will provide a broad overview of the ISIS evaluation design along with more in-depth profiles of program models.  This part of the presentation will focus on key questions career pathways programs raise and how ISIS will explore them. Examples of questions are how to engage non-traditional student populations in careers pathways programs, what are innovative strategies for improving post-secondary skills (e.g., acceleration, contextualization, active learning), and what are approaches to link career pathways steps and credentials and encourage participation in these pathways (as opposed to single steps/credentials).  The second part of the presentation will identify several implementation and impact study design challenges and evaluation team responses to them.  ISIS specifically, but evaluations of career pathways programs more generally, need to address challenges such as:  (1) describing quantitatively participants’ progression in and across training steps and between training and employment, (2) defining and measuring educational progress in comparable terms between treatment group and control group members across study programs, and (3) applying a theory of change in the analysis in order to identify important mediators of impacts on primary outcomes (education, employment, well-being).