Association for Public Policy Analysis & Management

Work history information reflected in résumés and job application forms is commonly used to screen job applicants; however, there is little consensus on how to systematically translate information about one’s past into predictors of future work outcomes. In this paper, we apply machine learning techniques to job application form data (including previous job descriptions and stated reasons for changing jobs) to develop measures that summarize candidate’s previous work experience and turnover history quantitatively and to predict future performance and turnover. Interpreting candidate’s work history in terms of relevance to the position they are applying for is challenging. Many studies use length of tenure in the same occupation or in the same organization. We develop a measure of work experience relevance that leverages all past positions and decades of accumulated knowledge about occupations embodied in the U.S. Department of Labor’s Occupational Information Network (O*NET), a comprehensive database designed to describe occupations in terms of knowledge, skills, abilities and other characteristics (KSAO) needed in each occupation. We proceeded in 4 steps: (1) train an algorithm (a Naïve Bayes Classifier) to map each candidate’s past position job-titles and job-description text to O*NET standard occupation codes, (2) map occupation codes to O*NET KSAO space. (3) measure distance in KSAO space between the past and desired position, and, (4) to get a single applicant-specific measure, average this distance across all the applicant’s past positions using a weighting function that favors more recent and longer-held positions.

We measure tenure history as the average deviation of applicant's tenure in prior jobs from the median tenure in each occupation. To summarize the reasons for turnover, we first took a small sample of unique reasons for leaving and manually categorized them into four variables: involuntary turnover, avoiding bad jobs, approaching better jobs and other reasons. We then apply supervised machine learning to applicants’ self-reported reasons for leaving each of their prior positions to categorize them. We empirically examine our model on a longitudinal sample of 16,071 applicants for public school teaching positions and predict subsequent work outcomes including student evaluations, expert observations of performance, value-added to student test scores, voluntary turnover, and involuntary turnover. We use regression models with Heckman corrections to predict post-hire performance (to account for the fact that we only observe outcomes on candidates who got hired) and corrected proportional hazard models to predict turnover. We find that work experience relevance and a history of approaching better jobs are linked to positive work outcomes, whereas a history of avoiding bad jobs was associated with negative outcomes. We also quantify the extent to which our model can improve the quality of selection the process relative to conventional methods of assessing work history, while lowering the risk of adverse impact. Specifically, we find that the candidates that our model would have recommended for hire outperformed the actual candidates that the district chose via conventional methods. Further, we find that following the model’s recommendations for would lead to hiring decisions that are uncorrelated with gender and race.