Background The MacArthur Violence Risk Assessment Study (MacVRAS) in the USA provided strong evidence to support an actuarial approach in community violence risk assessment.
Aims To examine the predictive accuracy of the MacVRAS measures, in addition to structured professional judgement, in a UK sample of patients discharged from in-patient care in the north-west of England.
Method A prospective study of 112 participants assessed pre-discharge and followed up at 24 weeks post-discharge. Pre-discharge measures were compared with prevalence of violent behaviour to determine predictive validity of risk factors.
Results Historical measures of risk and measures of psychopathy, impulsiveness and anger were highly predictive of community violence. The more dynamic clinical and risk management factors derived from structured professional judgement (rated atdischarge) added significant incremental validity to the historical factors in predicting community violence.
Conclusions Although static measures of risk relating to past history and personality make an important contribution to assessment of violence risk, consideration of current dynamic factors relating to illness and risk management significantly improves predictive accuracy.
In the past two decades there have been significant developments in the standardised assessment of violence risk and management (Monahan & Steadman, 1994; Douglas et al, 1999a; Dolan & Doyle, 2000), including the introduction of ‘structured professional judgement’ approaches to risk assessment (Webster et al, 2002; Douglas et al, 2003). The latter method is effectively an attempt to bridge the gap between clinical and actuarial approaches to risk assessment, by combining both elements into structured professional guidelines for clinical practice (Webster et al, 2002). Although both clinical and actuarial approaches to risk assessment have advantages and disadvantages (Hart, 1998), there are still relatively few studies that have tested the validity of this combined approach in a range of mental health settings. Some data are available from UK populations (Doyle et al, 2002; Gray et al, 2004) but most European data in this field come from outside the UK (e.g. Grann et al, 1999; Belfrage et al, 2001; Tengström, 2001). In general, the findings support the validity of measures of psychopathy, e.g. the Psychopathy Checklist – Screening Version (PCL–SV; Hart et al, 1995), or violence risk measures such as the Violence Risk Assessment Guide (VRAG; Webster et al, 1994) and the Historical Clinical Risk – 20 items scale (HCR–20; Webster et al, 1997) for institutional violence or recidivism in forensic cohorts. In view of the lack of UK data on the predictive accuracy of a range of established risk measures, and the general lack of prospective outcome data on structured professional judgement approaches to violence risk in the community, this study was developed to examine the predictive accuracy of a range of putative measures of violence risk in a representative sample of patients discharged from both civil and forensic psychiatric facilities in the north-west of England. As there is limited published statistical evidence to support the value of combined structured professional judgement approaches (Litwack, 2002; Webster et al, 2002; Douglas et al, 2003), we were particularly interested in assessing the incremental validity of the more clinical dynamic measures of risk included in the HCR–20 when added to the static, actuarial risk prediction equation.
The design was modelled on the MacArthur Violence Risk Assessment Study (MacVRAS; Monahan et al, 2001). However, we included patients discharged from both forensic and non-forensic psychiatric services to ensure that we had a representative sample of discharges. We also included additional measures such as the HCR–20 and the VRAG. The work was completed in five sites (three forensic medium secure units and two non-forensic units) in the northwest of England, as this region has a close geographical boundary with good links and tracking networks between district and forensic services. Representativeness of the sample was evaluated by comparing it with typical populations within the research sites against three indices: schizophrenia-spectrum disorder, gender and age. The sample had a lower proportion of people with schizophrenia-spectrum disorder: 69.6%, compared with the 76.8% in the research sites; fewer males: 67%, compared with 73.4% in the research sites; and the mean age of the sample was higher at 40 years compared with 37 years in the research sites. Patients were excluded if they were under 18 or over 65 years of age, unable to provide informed consent, unavailable because of leave or absence from ward, diagnosed primarily with learning disability or unable to read or understand English.
The North West Multi-site Research Ethics Committee approved the study, and written informed consent was obtained from all participants.
The plan was to recruit a minimum of 100 participants into the study, as this has been found to be more than sufficient to obtain significant results in previous prediction studies of this type (e.g. Doyle et al, 2002; Gray et al, 2004). A total of 129 participants were discharged during the 18-month study period. Of these, 112 (86.8%) completed the follow-up interviews. All participants were interviewed pre-discharge while in-patients, using a semi-structured interview schedule designed to elicit the information needed to score the standardised research instruments and minimise duplication of questions relating to similar domains. Nursing staff with good knowledge of participants were interviewed to gather collateral information needed to score key risk measures. A notification and tracking system was set up to ensure notification about all imminent or potential discharges across the sites, so that pre-discharge assessments could be prioritised and conducted accordingly. This system was checked regularly to ensure that no cases were missed.
Community violence was measured by completing the MacArthur Violence Risk Assessment Instrument (Monahan et al, 2001) with the participant and a collateral informant. The prevalence of community violence used in the analysis was based on official records, in addition to self-reports and collateral reports that were masked to baseline assessment measures. Data were also extracted from the Offenders Index at the Home Office. The primary outcome measure for the purpose of analyses was any violence in the 24-week period post-discharge.
The measures were chosen because they had demonstrated significant predictive validity in previous violence risk prediction studies, because they allowed comparison of historical, dispositional, clinical and contextual factors as described in the MacVRAS (Monahan et al, 2001) or because they were scales specifically designed to assess the risk of violence (i.e. HCR–20, VRAG). Measures were completed from data derived from the range of data sources cited above. The Novaco Anger Scale (NAS; Novaco, 2003) and the Barratt Impulsiveness Scale (BIS; Barratt, 1994) were self-report questionnaires.
The PCL–SV was chosen as the measure of psychopathy because it was designed for use in non-forensic samples (Hart et al, 1995). It has 12 items reflecting two parts. Part 1 reflects interpersonal and affective symptoms, and Part 2 reflects social deviance symptoms. Total scores range from 0 to 24, and scores of 18 or more are considered psychopathic in US studies.
The VRAG contains 12 items, attributed integer weights, ranging from – 5 to +12. The VRAG was designed for use with forensic populations, and three of the items rely upon rating of index offence. Participants with no index offence were given the lowest score possible for the three index-offence-related items.
The HCR–20 is a composite of 20 risk factors for violence. The ten historical factors relate to past relatively stable violence risk factors; the five clinical items reflect current, dynamic (changeable) correlates of violence; and the five risk management items focus on situational factors that may aggravate or mitigate risk. The HCR–20 is therefore sensitive to change, as the clinical and risk management items are dependent on current functioning and context and can act as a barometer of risk. In this study, the clinical and risk management items were rated at time of discharge and were used to examine the incremental validity of dynamic factors in addition to static factors. The total HCR–20 score reported here was a composite of the historical factors rated at baseline and the clinical and risk management scales rated at discharge.
The NAS is a 60-item self-report instrument that includes 48 items that measure three cognitive, arousal and behavioural domains of anger, each containing 16 items. Each domain has four sub-scales containing four items. The scale includes a 12-item anger regulation domain that provides information on cognitive, arousal and behavioural regulation of anger.
The BIS is a 30-item Likert-type self-report impulsiveness measure that has three sub-factors of impulsiveness; motor – acting without thinking, comprising 10 items; cognitive – making quick decisions, 8 items; non-planning – lack of concern for the future, 12 items.
Of the 129 participants who were discharged, complete data were available for 112, as 6 (3%) of the sample were transferred to another institution, 2 died before discharge (1%) and 9 were lost to followup (7%). The mean number of days to community follow-up was 168.47 (s.d.=16.88). The mean age of the community sample was 40 years (s.d.=11.5). The majority (75, 67%) were men. Almost all (104, 93%) were White. Over two-thirds of the sample (78, 70%) were discharged from district services and 34 (30%) were discharged from the three forensic sites. Nearly half the sample (52, 46%) had a primary diagnosis of schizophrenia, 8 (7%) of schizo-affective disorder, 18 (16%) of bipolar disorder, 15 (13%) of depression, 4 (4%) of personality disorder, 6 (5%) of substance misuse and 9 (8%) of other disorders or unknown. Thus, 78 (70%) of the sample had a serious mental illness diagnosis of either schizophrenia-spectrum disorder or bipolar disorder. Over half (59, 53%) were legally detained under the Mental Health Act 1983 at baseline assessment. Although less than a third were discharged from a forensic facility, 61 (54.5%) had a recorded criminal index offence for which they were receiving treatment or had been receiving ongoing treatment before the baseline assessment; 16 (14%) of the sample met the recommended cut-off score of >18 for psychopathy on the PCL–SV.
Community follow-up defining and measuring violence
Violence at follow-up was defined in accordance with the MacVRACS as:
‘...any acts that include battery that resulted in physical injury;
sexual assaults; assaultative acts that involved the use of a weapon; or
threats made with a weapon in hand’
(Monahan et al,
Data were analysed using SPSS for Windows version 10.1. Descriptive statistics described the sample. Interrater reliability checks were conducted for 20 cases on the historical items of the HCR–20 and the PCL–SV, as different raters had rated the same patients. Intraclass correlation coefficients were satisfactory between two researchers for the clinically rated historical items of the HCR–20 (0.97), PCL–SV total (0.97), PCL–SV factor 1 (0.85) and PCL–SV factor 2 (0.8). The interrater reliability between three raters based on seven cases was 0.99 for the VRAG, 0.85 and 0.83 for the clinical and risk management items of the HCR–20. Group differences between violent and non-violent samples were assessed using χ 2- and t-tests as appropriate. Receiver operating characteristic analysis was conducted to examine the predictive validity of the risk factors (Mossman, 1994). Logistic regression procedures were used to calculate odds ratios and examine the best predictive model for the dichotomous violence outcome measure based on the variables that were significant in univariate analysis. These procedures also controlled for possible confounding variables (age, length of stay, gender, forensic status).
Prevalence of violence
At follow-up 24 weeks post-discharge, using official records alone, only 10 participants (9%) would have been detected as having committed a violent act; 12 participants self-reported 16 acts of violence and 15 (13%) of the collaterals reported 46 acts of violence. When both self-reported violence and collateral information was merged, as in the MacVRAS, the prevalence of violence committed significantly increased to 19% (n=21; χ 2=42.49, d.f.=1, P<0.001 when compared with 9% when using records alone).
Comparison of violent and non-violent groups
There were no significant differences based on psychiatric diagnoses between violent and non-violent groups, but a higher proportion (38%) of those meeting the criteria for psychopathy (based on a cut-off of 18) were violent compared with those who scored below the cut-off (16%) (Table 1). There was no significant difference in the prevalence of violence between the forensic and non-forensic samples. There were no significant differences between violent and non-violent groups in terms of age, gender, ethnicity or presence of a clinical personality disorder diagnosis (Table 1). Those who were subject to the enhanced care programme approach (Department of Health, 2000) on discharge were significantly less likely to be violent in the 24 weeks after discharge (Table 1).
Predictive validity of risk scales
There were significant differences between violent and non-violent groups on all the baseline risk assessment scales, with the violent group having higher scores on all measures (Table 2). The PCL–SV and self-reported anger and impulsiveness demonstrated most significant differences between violent and non-violent groups. In the receiver operating characteristic analysis, which examined the predictive validity of the scales, the majority of measures were significantly predictive at the P<0.05 level but the accuracy level varied between scales (Table 3). For these analyses, the HCR–20 total was calculated according to the total historical items score at baseline and the total score of the ten clinical and risk management scores measured at discharge. The historical items scale of the HCR–20 measured at baseline had a moderate area under curve (AUC), whereas the HCR–20 total had the largest AUC at 0.797 (Table 3). The NAS total and sub-scales AUCs ranged from 0.696 to 0.723 for the cognitive sub-scale. The BIS cognitive sub-scale had the largest AUC (0.735). The VRAG had a relatively low AUC (0.657) and the PCL–SV and its sub-scales had moderate AUCs ranging from 0.666 to 0.687 (Table 3).
Incremental validity of the HCR–20 clinical and risk management items
To examine the relative contribution of the dynamic clinical and risk management factors of the HCR–20 measured at discharge, we used a series of logistic regression analyses based on hierarchical methods. To do this, a number of significant baseline factors (see below) were entered on the first step, and then the HCR–20 dynamic clinical and risk management scales were added to see whether the predictive model improved. Variables selected for entry were based on the scales or sub-scales of all measures that showed the most significant differences in the univariate and predictive receiver operating characteristic analysis. As the psychopathy score was entered as an individual item, we removed the psychopathy item from the historical items of the HCR–20 and VRAG to avoid conflation, as recommended in previous studies of this type (Douglas et al, 1999b). The factors entered in the first regression procedure (model 1, Table 4) were the total scores on the PCL–SV, historical items sub-scale (minus PCL–SV item), VRAG total (minus PCL–SV item), BIS cognitive sub-scale and NAS cognitive sub-scale. The regression procedure was repeated, adding the HCR–20 clinical and risk management scores rated at discharge (model 2; Table 4). Model 1, without the clinical and risk management scales total, demonstrated a highly significant chi-square value (23.53, P<0.001) and correctly classified 86% of the sample. However, only the BIS and NAS cognitive sub-scales independently predicted violence with significant odds ratios, where P<0.005 (Table 4). When the clinical and risk management scales total was added to the model (model 2), the chi-square statistic for the model improved (36.17, P<0.001) and the percentage of the sample correctly classified increased to 88%. In model 2, only the clinical and risk management total score independently predicted community violence post-discharge. Therefore, the HCR–20 clinical and risk management dynamic scales added significant incremental validity to the baseline measures.
In order to further test the predictive validity of the HCR–20 total score, further logistic regression procedures were conducted to control for possible confounding variables that have been identified in previous studies (e.g. Swanson et al, 1990). Therefore on step 1 the HCR–20 total was entered alone, whereas on step 2 age, gender, length of stay as in-patient and forensic status were added to examine the possible confounding effect of these variables. The HCR–20 total score significantly predicted post-discharge violence, and this remained the case on step 2 when age, gender, length of stay as in-patient and forensic status were added (Table 5). The adjusted odds ratio actually increased when confounding variables were entered, supporting the independent predictive accuracy of the HCR–20 for post-discharge violence
As far as we are aware, this is the first UK prospective study of community violence that has been modelled on the recent MacVRAS in the USA, which is now regarded as one of the definitive studies in the violence risk assessment field in view of its rigour and sample size (n=939). Although our sample is notably smaller (n=112), we have included individuals with a greater range of baseline characteristics by recruiting a forensic cohort. Unlike the MacVRAS, we did not exclude individuals over 40 years of age, because this would not be representative of discharges and forensic patients have longer lengths of stay than civil psychiatric patients. By doing this, we had a fairly representative cohort of patients discharged into the community. The age, gender and diagnostic profiles of our sample were fairly typical of the profiles of patients admitted to all the research sites using routine data sources and previous research studies. More homogeneous samples, in terms of specific conditions or diagnoses, make it easier to control for variability and confounding factors, although more heterogeneous samples (such as this) have the advantage of being more representative of actual clinical populations.
Comparison with findings from the MacArthur Violence Risk Assessment Study
Despite the differences between this study and the MacVRAS, we found that the results were generally very similar, suggesting cross-cultural validity in a number of measures. The mean follow-up period of approximately 24 weeks in this study was comparable with the 20-week follow-up in the MacVRAS, where the rate of violence at 20 weeks follow-up was 18.7%. This is comparable with our data (19%) for a 24-week follow-up period. We found similarly that the inclusion of collateral information significantly enhanced the detection of violent behaviour in the community in this UK sample. Previous US studies have also highlighted the value of collateral informants in this type of research (Steadman et al, 1998; Monahan et al, 2001). If we had relied on official records alone, we would have detected only half of the incidents that occurred, and this might have limited our ability to accurately assess the validity of the key measures. The limitation of treating violence as a binary outcome should also be noted, as those committing frequent, severe acts of violence can be classified with those committing only one. Multiple statistical comparisons were made in this study, thereby increasing the risk of spurious results. However, we are confident in the validity of our results in view of the consistency and significance of findings across different measures and the similarities between our findings and previous research.
We found a higher rate of psychopathy in our sample (14%) than the MacVRAS sample where only 8% met the criteria. This is not surprising, as we had included a forensic sample, and previous studies have suggested that at least 25% of forensic patients would meet the criteria for psychopathy (Hart et al, 1995; Doyle et al, 2002).
In terms of the predictive accuracy of key measures, we found that the PCL–SV, VRAG and HCR–20 significantly predicted violence in the community. This fits with data from previous US studies (e.g. Rice, 1997; Douglas et al, 1999b; Skeem & Mulvey, 2001; Harris et al, 2002). The lower predictive accuracy of the VRAG compared with previous studies (e.g. Rice, 1997) is likely to be due to the facts that in this cohort nearly half of the participants did not have an offending history and the tool was rated in a non-standard way. The VRAG was developed with a forensic sample and, as three items are offence-related, the VRAG is likely to be a better predictor in populations with a history of offending behaviour.
In this sample we found that BIS impulsiveness and NAS anger problems (particularly the cognitive components) were significantly predictive of subsequent violence. The MacVRAS found similar but less powerful relationships with impulsiveness and anger as measured by the BIS and NAS, whereas anger and impulsiveness have been found to be associated with subsequent violence in several other studies (Segal et al, 1988; Novaco & Renwick, 1998). These findings suggest that self-report measures of anger and impulsiveness, that are easily administered and scored, may have some clinical utility in identifying those at risk of subsequent violence. The findings also suggest that previous criticisms and scepticism about the value of self-report questionnaires in risk assessment in forensic samples (e.g. Hart et al, 1995) may be overestimated. However, it should be noted that in research settings, where the findings from self-report data have no direct clinical impact, it is possible that the respondents are more honest than when these measures are administered for clinical purposes and their answers may affect release decisions.
We found no striking relationship between specific diagnosis and future community violence. The lack of a relationship might be explained by the relatively low base rate of violence, small sample size and general lack of statistical power. Nevertheless, contradictory findings might reflect real differences in the levels of supervision in the samples studied. Further, our findings supported the important effect of aftercare arrangements as a protective factor; an enhanced level of the care programme approach was found to be protective against violence after discharge. Treatment, engagement, compliance and restrictions in the community are possible confounders in this study, and this is clearly an area that requires research in the future. In this study, we did not find that substance misuse or a clinical diagnosis of personality disorder per se were specifically associated with subsequent violence, although both these factors have been reported as robust risk predictors in previous studies (Swanson et al, 1990; Widiger & Trull, 1994; Steadman et al, 1998; Monahan et al, 2001). There are a number of reasons why there are conflicting findings in the literature, and these may be the result of variation in the characteristics of the samples (civil or forensic), differences in assessment of personality disorder (clinical or research-based) and differences in information sources (self-report or collateral or official records or combined). Future studies need to take these factors into consideration in study designs.
Psychopathy and the HCR–20
It is noteworthy that, as with numerous previous studies, psychopathy was predictive of future violence. What is surprising is that this predictive accuracy was not as high as might have been expected based on previous findings, and that the accuracy was surpassed by measures of anger and impulsiveness. This seems to fit with the recent findings of Skeem et al (2005), where measures of personality traits and antagonism were more important than psychopathy in explaining violent outcome in the MacVRAS sample.
Our main finding was that the HCR–20 (which was not used in the MacVRAS) was the most robust predictor of subsequent community violence, and that the clinical and risk management items (which are dynamic in nature) do add significant incremental validity to the assessment of risk, over and above that of more static factors such as those listed under the historical scale of the HCR–20. Although the proportion correctly classified increased modestly from 86% to 88%, more importantly, when the clinical and risk management scales total was added to the original model, it was found to be the only significant predictor.
Structural professional judgement
The heterogeneity of violence risk factors found in this study suggest that reliance on findings based on historical aggregate data, essential for epidemiological studies and potentially useful for clinical decision making, may be limited in their applicability to individual patients. Overall, our findings highlight the importance of considering current social functioning, mental state and contextual factors in decision making. Furthermore, our data suggest that the HCR–20 has reasonable cross-cultural validity, as our findings fit with other international studies highlighting the predictive accuracy of this measure in a range of settings, including Canada (Douglas et al, 1999b), Scotland (D. J. Cooke, personal communication, 2006) and Sweden (Grann et al, 1999). However, as with other structured risk assessments, it should be noted that the level of supervision provided on release can attenuate the predictive accuracy of this measure for post-discharge violence. This was demonstrated by Dolan & Khawaja (2004), who noted that the HCR–20 predicted self-report violence and readmission, but not officially recorded violence, as supervising staff were using readmission as an effective management strategy. Previous writers in this field have noted this phenomenon (Hart, 1998; Douglas et al, 2003). Our evidence suggests that, contrary to arguments by those supporting the superiority of actuarial assessments, clinical and risk management factors are very important and enhanced levels of care do make an important contribution, at least in the short term.
Implications for clinical practice
According to our findings, it is possible that risk management strategies will be more successful if they are feasible, treat active symptoms of mental illness, address attitudinal, impulsiveness and emotional-regulation problems, reduce the likelihood of non-compliance and improve insight. There is clearly a need to use a combination of strategies to characterise individual violence risk in the long, medium and short term, and this can only be done if clinical teams have a good knowledge and understanding of idiosyncratic historical, clinical and risk management factors that apply to individuals. Measures such as the HCR–20 provide a very clear outline of the factors that clinicians should consider in the formulation of risk and, like all structured professional judgement approaches to risk assessment, measures such as the HCR–20 are designed to help clinicians provide a more transparent and structured method of recording their risk assessments. Records of assessments are becoming increasingly important in inquiries into clinical practice following untoward events, and measures such as the HCR–20 have value in enhancing the rationale for clinical risk judgements. By reviewing change in clinical and risk management items, it may also be possible to assess the impact of current interventions and monitor progress, while systematically tracking change in all key domains that have been identified as treatment targets. The latter approach should make intuitive sense to clinicians and reflect good clinical practice in risk assessment.
We thank Barbara Tomenson, University of Manchester, for her statistical advice. The study was supported by a grant from the National Health Service North West Regional Research & Development Directorate in England. Ongoing support provided by the National Forensic Mental Health Research and Development Programme.
- Received December 30, 2005.
- Revision received June 6, 2006.
- Accepted August 1, 2006.
- © 2006 Royal College of Psychiatrists