Hostname: page-component-8448b6f56d-cfpbc Total loading time: 0 Render date: 2024-04-16T11:58:02.701Z Has data issue: false hasContentIssue false

The futility of risk prediction in psychiatry

Published online by Cambridge University Press:  02 January 2018

Roger Mulder*
Affiliation:
Department of Psychological Medicine, University of Otago, Christchurch, New Zealand
Giles Newton-Howes
Affiliation:
Department of Psychological Medicine, University of Otago, Wellington, New Zealand
Jeremy W. Coid
Affiliation:
Violence Prevention Research Unit, Wolfson Institute of Preventive Medicine, Queen Mary University, London, UK
*
Roger Mulder, University of Otago, Christchurch, PO Box 4345, Christchurch 8140, New Zealand. Email: roger.mulder@otago.ac.nz
Rights & Permissions [Opens in a new window]

Summary

Significant efforts have been made to identify risk factors associated with suicide. However, the evidence suggests that risk categorisation may be of limited value, or worse, potentially harmful, confusing clinical thinking. We argue instead for a shift in focus towards real engagement with the individual patient, their specific problems and circumstances.

Type
Editorial
Copyright
Copyright © Royal College of Psychiatrists, 2016 

Suicide prevention is generally seen as a major priority for the mental health sector. This has led to significant research interest in the risk factors associated with suicide. Efforts have been made to group the identified risk factors in such a way that clinicians can categorise patients into low- and high-risk groups, with significant treatment implications, such as the choice to admit to hospital, voluntarily or not. A significant problem with such an approach is the low incidence of completed suicide, albeit the total global impact affects thousands of lives.

One way to address the problem of low base rates of suicide is to study suicide risk factors in groups where the base rate is higher than in the general population. This ‘enriched’ risk group not only provides for data of greater power, allowing better identification of the factors of interest, but is clinically relevant if the element ‘enriching’ the group is one that increases presentations to health services. An obvious group, and the subject of a systematic review in this issue of the BJPsych, Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 is people who have already self-harmed. Self-harm is more common than suicide, is of major clinical interest being a common reason for admission to hospital and, importantly, the risk of suicide is elevated between 50- and 100-fold in the year after self-harm. This implies that fewer participants will be necessary to identify meaningful risk factors.

The idea that risk factors can be identified and then used to predict the likelihood of suicide is appealing not only to clinicians, but also to hospital managers, politicians, families and patients. Even more appealing is the idea that these risk factors can be grouped into composite risk tools or scales that can quantify the risk of later suicide. These tools can be used by a very wide array of mental health clinicians, carry the kudos of science and provide reassurance to all groups that suicidality can be made understandable and easily manageable. Such tools are currently commonly used in clinical practice despite little evidence that they are effective. The Chan et al review in this issue of the BJPsych has two aims; the first is to identify risk factors in this enriched sample of individuals who have attempted suicide. Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 The second, and to our knowledge unique, aim is a review and meta-analysis of risk assessment scales in people who had self-harmed or were under specialist mental healthcare.

Risk factors

The results on risk factors associated with suicide are predictable; the four factors that emerged from the meta-analysis were previous episodes of self-harm, suicide intent, physical health problems and male gender. All these risk factors are common in clinical populations so that the positive predictive value is low and they are of little use clinically. This finding is reported in all studies that have looked at risk factors. Reference Mulder2 Large et al, Reference Large, Sharma, Cannon, Ryan and Nielssen3 for example, carried out a meta-analysis of another group with a high base rate of suicide: patients discharged from a psychiatric hospital. They reported that previous episodes of self-harm, depressive symptoms, male gender, suicidal ideas, unplanned discharge and recent social difficulty were moderately or weakly associated with post-discharge suicide. Again, all risk factors were common and they concluded that no factor, or combination of factors, was strongly associated with suicide. Perhaps it is finally time to acknowledge that rare events such as suicide – no matter that they are tragic for all involved or how much we wish to prevent them – are impossible to predict with a degree of accuracy that is clinically meaningful. Reference Szmukler4 This has significant clinical implications for our assessments of patients following a self-harm attempt and, importantly, implications for how we communicate the issues associated with future completed suicide to medical colleagues, families and the wider public.

Risk scales

Risk scales are widely used internationally, occasionally mandated for use in some health settings and routinely considered the essential component of risk assessment. This belies the evidence: Chan and colleagues could only find seven studies evaluating their efficacy. Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 Of these, only two scales, the Beck Hopelessness Scale (BHS) and the Suicide Intent Scale (SIS), had sufficient data points to conduct a meta-analysis. Both the BHS and SIS lacked sufficient specificity and sensitivity to be clinically useful. Without being sufficiently sensitive the tool will miss those who go on to die by suicide, providing false reassurance to staff and families. By lacking specificity many patients who would not have gone on to take their own life will be managed more aggressively, including admission to hospital, carrying clinical risk and implications as to the capacity of services to manage. Chan et al opine that risk scales are of dubious clinical utility at best and should not be used alone in clinical practice to assess the future risk of suicide. Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 To our knowledge only one study has prospectively examined the effect of risk assessment on patient outcome. Reference Troquete, van den Brink, Beintema, Mulder, van Os and Schoevers5 The participants were forensic out-patients and the outcome was violent and criminal behaviour. They reported no difference in outcome between a structured risk assessment protocol and treatment as usual.

The risks of risk assessment

Chan et al's review reinforces the limited usefulness of risk factors and risk scales in clinical practice. Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 This should be well known. What is less discussed is the potential harms that risk categorisation and risk scales can do to patients. The most obvious harm is that patients labelled ‘high risk’ may receive needlessly more restrictive treatment. This is counter to the principles of a recovery-focused service embedded into many countries' mental health strategies, and is in conflict with international treaties such as the Convention on the Rights of People with Disabilities that take a strongly libertarian stance. It may increase the stigma felt by those in mental distress and reduce the ability of services to engage in providing evidence-based care that could be of benefit to these very patients. Less obvious is that the majority of suicides occur in ‘low-risk’ groups primarily because they contain many more members than high-risk groups and the sensitivity of the instruments used to assess risk are, we can now say, too poor to overcome this size issue. This is a common problem in medicine with very low incidence outcomes. According to the National Confidential Inquiry into Suicide and Homicide in People with Mental Illness, 86% of suicides are in low-risk groups. Reference Appleby, Shaw, Kapur, Windfuhr, Ashton and Swinson6 Focusing healthcare resources on high-risk groups not only has the potential to directly harm them but may result in the misallocation of resources from some of the patients wrongly labelled low risk who require more intensive input.

In clinical practice, as Chan et al point out, risk assessment may provide false reassurance for clinicians and managers. Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 Some go further, positing that risk assessment is often an organisational attempt to tame anxiety rather than to improve patient care. Reference Undrill7 Patients may be detained not for treatment needs but because not detaining them produces intolerable anxiety in the staff involved in the assessment. The myth that certain risk factors can predict suicide leads to the belief that suicide may be the result of inadequate risk assessments in health services. The conflation of risk prediction and risk assessment confuses clinical thinking and is unhelpful. Our current preoccupation with risk prediction has the potential to harm patients, clinicians and the organisations in which they work. It has created a sense of unease among clinicians and a culture of blame when things go wrong. What can be done to change this?

First, we need to acknowledge the impossibility of predicting individual risk accurately and educate the public that this fact, although unfortunate, is true. Second, as Chan et al point out, we need to move away from assessment models that prioritise quantification of risks at the expense of understanding. Rather than using risk scales we need to focus on the individual; determining the specific factors that precipitate suicidal ideation and attempts in that person and identifying the qualitative personal factors that could increase the likelihood of later suicide. Although there is no empirical evidence this approach is effective, we need to accept this as the ‘art’ of psychiatry and accept it is difficult to quantify and examine empirically. Risk assessments should be a consensual process with the patient and clinicians striving towards a realistic conceptualisation of the risk then mutually deciding how best to manage it. This is a broad approach that forms part of a needs-based model of care, overcoming the problems associated with attempts at risk prediction. Third, detailed risk assessments should be curtailed. Accumulating more and more risk information of marginal utility is of little value and increasingly likely to harm the patient and the therapeutic alliance. Risk assessment should not form the basis of clinical decision-making. The aim of assessment is to provide optimal care and planning according to patient needs regardless of the perceived risk. Fourth, a major problem with risk research is the emphasis on identifying statistical predictors and not the causal factors for outcomes such as suicide. Research into factors leading to violence among discharged psychiatric patients has shown that delusions are a poor future predictor but strongly associated when mediated by anger, suggesting causality. Similarly, some risk factors may be poor predictors of future outcome when combined into scales, but if different statistical models are used that attempt to establish causal pathways, targets for preventive intervention can be identified. This corresponds to clinical experience and the approach recommended by Chen et al. Reference Chan, Bhatti, Meader, Stockton, Evans and O'Connor1 Clinical intervention will only be effective with factors that have some causal association with suicide. The search for ever better statistical predictors will not achieve this.

In conclusion, although it is possible to validly and reliably identify risk factors for completed suicide, the mathematics of risk prediction, whether clinical or using risk assessment tools, mean that risk categorisation can never assist in reducing suicide. Reference Paton, Large and Ryan8 Suicide is a sufficiently rare event and our current tools sufficiently blunt that ‘high-risk’ groups will contain mainly false positives and the majority of those who die by suicide will have been categorised as low risk. We need to accept that risk conceptualisation and prediction associated with completed suicide is a major public health concern, not a major psychiatric concern. The logical response to this for psychiatry is to abandon misguided attempts at risk prediction and encourage real engagement with the individual patient, their specific problems and circumstances.

Footnotes

See pp. 277–283, this issue.

Declaration of interest

None.

References

1 Chan, MKY, Bhatti, H, Meader, N, Stockton, S, Evans, J, O'Connor, RC, et al. Predicting suicide following self-harm: systematic review of risk factors and risk scales Br J Psychiatry 2016; 209: 277–83.CrossRefGoogle ScholarPubMed
2 Mulder, R. Problems with suicide risk assessment. Aust NZ J Psychiatry 2011; 45: 605–7.Google Scholar
3 Large, M, Sharma, S, Cannon, E, Ryan, C, Nielssen, O. Risk factors for suicide within a year of discharge from psychiatric hospital: a systematic meta-analysis. Aust NZ J Psychiatry 2011; 45: 619–28.Google Scholar
4 Szmukler, G. Risk assessment for suicide and violence is of extremely limited value in general psychiatric practice. Aust NZ J Psychiatry 2012; 46: 173–4.Google Scholar
5 Troquete, NAC, van den Brink, RHS, Beintema, H, Mulder, T, van Os, TWDP, Schoevers, RA, et al. Risk assessment and shared care planning in out-patient forensic psychiatry: cluster randomised controlled trial. Br J Psychiatry 2013; 202: 365–71.CrossRefGoogle ScholarPubMed
6 Appleby, L, Shaw, I, Kapur, N, Windfuhr, K, Ashton, A, Swinson, N, et al. Avoidable Deaths: Five Year Report by the National Confidential Inquiry into Suicide and Homicide by People with Mental Illness. University of Manchester, 2006.Google Scholar
7 Undrill, G. The risks of risk assessment. Adv Psychiatr Treat 2007; 13: 291–7.Google Scholar
8 Paton, MB, Large, MM, Ryan, CJ. Debate: clinical risk categorisation is valuable in the prevention of suicide and severe violence – no. Australas Psychiatry 2014; 22: 10–2.Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.