| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |
|
|
|
||||||||||
|
||||||||||||
Correspondence |
South London & Maudsley NHS Trust, Maudsley Hospital, Denmark Hill, London SE5 8AZ
Dolan & Doyle (2000) provide a helpful review of clinical and actuarial measures in violence risk prediction. The evidence shows that prediction can be significantly better than chance. However, they present only one half of the story. How well do the best instruments perform in the real clinical world where prediction leads to action, including restrictions on the liberty of patients regarded as dangerous? False positives are very serious from an ethical (including resource allocation) point of view. Here we encounter the ‘base rate’ problem that the authors inexplicably fail to mention.
The rate at which violent acts occur in the population of interest is critical to the predictive abilities of any instrument. The authors reproduce a receiver operator characteristics (ROC) curve of a well-performing instrument which, as they say, shows the trade-off between the true positive rate and the false positive rate (or conversely the true negative rate). Where that trade-off should lie depends on the relative costs of false positives v. false negatives. One usually looks at the point of maximum perpendicular distance from the diagonal line. For this ROC, a true positive rate of 0.7 and a false positive rate of 0.3 (equivalent to a true negative rate of 0.7) is probably the optimum. A test has to predict accurately who will be violent as well as who will not be violent. Although this ROC is statistically significant against chance at the P < 0.001 level in predicting violence, how does it fare in practice?
It is difficult to describe how prediction instruments perform in a way that is easily comprehensible to non-mathematicians. Perhaps probability trees can help. Figure 1 shows a probability tree in which the essential data are presented in relation to a population in which 20% of patients will actually be violent during the follow-up period. Using the test represented by the ROC described, it can be seen that the positive predictive value, that is, the proportion of patients predicted by the test to be violent who indeed turn out to be violent, is 0.37. But this means also that the prediction will be wrong about six times out of ten. Perhaps a base rate of 20% is appropriate to some forensic populations. In a community mental health service, even an inner-city one, the rate of violent acts, of any severity, over a 6-month period is more likely to be around 6% (Shergill & Szmukler, 1998). Substituting the figures 6 and 94 in the probability tree the reader will discover that the positive predictive value drops to 0.14; that is, the prediction will be wrong almost nine times out of ten. For very serious violence, perhaps at a rate of 1%, the test will be wrong about 97 times out of a 100. For homicides, at around 1 in 10 000 per annum committed by patients with a psychosis, prediction is meaningless.
|
Rare events are inherently difficult to predict. Even a test with an impossible 0.9 accuracy for both true positives and true negatives will be wrong more than nine times out of ten at a base rate of 1%. Thus highly statistically significant ROC curves look very limited indeed in their practical application in a community context. How unfair is it then that mental health services in the UK seem to be expected to prevent what is, in practice, unpredictable?
REFERENCES
Dolan, M. & Doyle, M. (2000) Violence risk
prediction. Clinical and actuarial measures and the role of the Psychopathy
Checklist. British Journal of Psychiatry,
177,
303-311.
Shergill, S. S. & Szmukler, G. (1998) How predictable is violence and suicide in psychiatric practice? Journal of Mental Health, 7, 393-401.[CrossRef]
This article has been cited by other articles:
|
|
 |
|
  M. A. Norko and M. V. Baranoski The Prediction of Violence; Detection of Dangerousness Brief. Treat. Crisis Interven., February 1, 2008; 8(1): 73 - 91. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Turner and M. Salter Forensic psychiatry and general psychiatry: re-examining the relationship Psychiatr. Bull., January 1, 2008; 32(1): 2 - 6. [Full Text] [PDF]
|
|
|
|
 |
|
 P. E. Mullen Dangerous and severe personality disorder and in need of treatment The British Journal of Psychiatry, May 1, 2007; 190(49): s3 - s7. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. D. Hart, C. Michie, and D. J. Cooke Precision of actuarial risk assessment instruments: Evaluating the `margins of error' of group v. individual predictions of violence The British Journal of Psychiatry, May 1, 2007; 190(49): s60 - s65. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Higgins, D. Watts, J. Bindman, M. Slade, and G. Thornicroft Assessing violence risk in general adult psychiatry Psychiatr. Bull., April 1, 2005; 29(4): 131 - 133. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. SZMUKLER Risk assessment: 'numbers' and 'values' Psychiatr. Bull., June 1, 2003; 27(6): 205 - 207. [Full Text] [PDF]
|
|
|
|
 |
|
 L. A. Craig, K. D. Browne, and I. Stringer Risk Scales And Factors Predictive Of Sexual Offence Recidivism Trauma Violence Abuse, January 1, 2003; 4(1): 45 - 69. [Abstract] [PDF]
|
|
|
|
|
|
J. Stone and G. Szmukler An audit of risk assessment in an emergency setting Psychiatr. Bull., March 1, 2002; 26(3): 88 - 90. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Szmukler The mathematics of risk assessment for serious violence Psychiatr. Bull., September 1, 2001; 25(9): 359 - 359. [Full Text] [PDF]
|
|
|
|
|
|
A. Maden Practical application of structured risk assessment The British Journal of Psychiatry, May 1, 2001; 178(5): 479 - 479. [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |
