Despite increasing interest in the cognitive behavioral therapy (CBT) treatment-based probation program, relatively little is known about whether this program's effect on recidivism may vary according to probationer demographic characteristics. Using the rich data collected from a multi-year randomized field trial focusing on high risk probationers in Philadelphia and a rigorous Bayesian hierarchical Gamma-Poisson model, this paper tries to address this question, by comparing probationers who were assigned to the CBT program (treatment group) and those who were not (control group), conditional on probationer characteristics. The results show that the CBT program effect in reducing recidivism is more evident for the high-risk probationers who were at around 10-19 and 30-39 years old, who have experienced probation prior to the current probation, and who had a high ratio of "high (risk prediction)" votes from the random forest risk prediction model. Noticeably, the Bayesian approach detects more intention-to-treat (ITT) and the treatment-on-treated (ETT) effects than the frequentist null hypothesis significance testing (NHST) approach does. This is due to the Bayesian approach's strength of the relative robustness to small sample sizes and outliers impacts that are common problems in the criminal justice field.