Additive linear risk/need tools such as the LSI-R, YLS-CMI, COMPAS, etc. periodically require specific types of evaluation – instrument validation – to determine whether the respective instrument has relative predictive accuracy as currently implemented.
J-SAT has conducted validation studies of both juvenile and adult risk/need assessment tools to help agencies determine whether the tool they are using has at least moderate predictive accuracy (Area Under the Curve [AUC] values around .70). In some studies we have controlled for officer race/ethnic biases and in others we provided simple and straightforward analysis to establish the predictive co-efficients of the tool in the most expedient and cost-effective manner.
Our Holistic Methodology
One primary benefit of using J-SAT to conduct your agency’s risk/need validation centers around our extensive experience implementing a wide variety of assessment tools. We completely understand the premise that problems with tool prediction are more likely to center around how the tool is applied than particular psychometric properties of the tool. We don’t minimize either but we are especially sensitive to the adage: “It’s the hand that turns the screw” and we undertake tool validation with keen attention to surfacing the latter potential initial training and implementation problems.
A Cost-Effective Analysis
Due to our strong service-oriented values, we offer cost-effective solutions to agencies with needs for assessment tool validation. We are not interested in bloating the budget for conducting analytic services using basic statistical techniques (e.g., logistic regression, survival analysis, multiple regression and ROC’s). Providing the host agency can provide us a relatively clean ‘flat file’ of outcome data along with the respective metrics of the tool, we can produce a simple validation analysis for a fraction of the cost of our competition.