© 2008 The Royal College of Psychiatrists
Correspondence |
Shall evidence-based risk assessment be abandoned?
Mental Health Center Penetanguishene, 500 Church Street, Penetanguishene, ON L9M 1G3, Canada. Email: gharris{at}mhcp.on.ca
Mental Health Center Penetanguishene, Penetanguishene, Canada
Department of Psychology, Queen's University, Kingston, Ontario, Canada
Edited by Kiriakos Xenitidis and Colin Campbell
Hart et al1 argued that actuarial risk assessments (Violence Risk Appraisal Guide, VRAG, and Static-99) are `virtually meaningless'. They committed statistical error by misapplying confidence intervals (see http://www.mhcp-research.com/hmcrespond.htm or http://bjp.rcpsych.org/cgi/eletters/190/49/s60). Confidence intervals do not capture the `precision' of individual scores. The appropriate statistic is standard error of measurement, which for the VRAG indicates that any individual score carries less than 0.05 probability of misclassification by more than one category.2
Hart et al also erred in using `precision' and `accuracy' as synonyms. Accuracy is best assessed by sensitivity, specificity and their trade off. More than 40 evaluations of the VRAG (and the allied Sex Offender Risk Appraisal Guide) in approximately 8000 released prisoners, sex offenders, forensic patients, compulsorily admitted (`civil') psychiatric patients and other samples from seven countries have employed follow-ups from several weeks to 10 years. Predictive effects (in sensitivity–specificity terms) are large. Contrary to Hart et al, scores predict the speed and severity of recidivism. Most commonly, there are no statistically significant differences between observed rates and expectations based on norms,3 especially given variation predicted by Bayes' Rule. Hart et al's statistical argument does not and cannot refute empirical results supporting the accuracy of actuarial risk assessments in predicting the violent recidivism of offenders.
The notion that it is scientifically wrong to base individual decisions on such post-analytic groupings (inappropriately referred to as `group data' by Hart et al) has been thoroughly refuted.2,4 What should a clinician do when considering the release of one previously violent forensic patient? Hart et al implied that no decision should be made, recommending concentrating on subjectivist issues instead. We disagree. An actuarial tool (such as the VRAG or Static-99) is simply an efficient distillation of relevant empirical evidence. Actuarials do not afford certainty, but, as Hart et al acknowledged, are more valid than any other method. The undeniable superiority of actuarials means that their use can optimise the balance between public safety and offenders' civil liberties. Hart et al's advice to eschew risk-related decisions means less accurate decisions that cumulate in avoidable harm to victims, unnecessary restriction of offenders, or both.
REFERENCES
-
1
- Hart SD, Michie C, Cooke DJ. Precision of actuarial risk assessment
instruments. Br J Psychiatry 2007;
190 (suppl 49): s60
-5.
[Abstract/Free Full Text] 2 - Quinsey VL, Harris GT, Rice ME, Cormier CA. Violent Offenders: Appraising and Managing Risk (2nd edn). American Psychological Association, 2006. 3
- Harris GT, Rice ME. Characterising the value of actuarial violence risk assessment. Crim Justice Behav 2007; in press. 4
- Grove WM, Meehl PE. Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: the clinical–statistical controversy. Psychol Public Policy Law 1996; 2: 293 -323.[CrossRef]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||