Rasch Analysis of the Kentucky Department of Juvenile Justice's Intensive Aftercare Selection Instrument
*Names in bold indicate Presenter
Using Rasch analysis, this study examines the YLS/CMI at the item level among a sample of high risk juveniles receiving Intensive Aftercare (IA) services from the Kentucky Department of Juvenile Justice. The analysis was conducted using the 42 standard YLS/CMI items and conducted in Winsteps. Data from 52 IA program participants were used. Juveniles had to receive a score of 23 or above to be initially included in IA. The Rasch results suggest the YLS/CMI generally functioned well as a diagnostic instrument, but had a reliability issue. The item-fit scores were appropriate, with item-infit statistics ranging from -1.3 to 1.0 and item-outfit statistics ranging from -1.2 to 1.7. Item difficulty levels fell between -3.30 and 2.51. These item difficulties suggest there was a range of easy-to-difficult items. There was some repetitive clustering of items around the 0.0 logit point, suggesting the model items could be reduced. The reliability results were mixed. The item-reliability was at .91, suggesting the difficulty order would not alter greatly with another sample (Bond & Fox, 2007). However, the person-reliability was at .37, suggesting that other sample subjects with similar risk levels would receive varying scores and that responses of the subjects may not remain stable longitudinally. This could indicate a problem with the YLS/CMI instrument or may be linked to rater scoring practices.
Understanding the performance of the YLS/CMI as a diagnostic instrument is vital, given its usages as a service screening tool and risk of recidivism identifier. Basing policy decisions or enacting policies using a diagnostic tool with validity issues is problematic. First, it can lead to policy decisions which misuse resources and place individuals in the broader community at increased risk. Secondly, it can lead to inadequately meeting the needs of the participants being screened by the instrument. For these reasons, understanding the item level performance of the YLS/CMI and similar diagnostic instruments is vital for effective policy decisions and resource allocation as well as increased public safety.