Yale University Department of Psychiatry, New Haven, Connecticut, USA
Health Services Research Department, David Goldberg Centre, Institute of Psychiatry, London, UK
Correspondence: Dr Alec Buchanan, Yale University Department of Psychiatry, 34 Park Street, New Haven,CT 06519, USA. E-mail: alec.buchanan{at}yale.edu
|
|
|---|
Aims To quantify the contributions made by three types of information demographic data alone, demographic and criminal record and demographic, criminal record and legal class of disorder to the prediction of criminal conviction in patients.
Method All 425 patients discharged from the three special (high secure) hospitals in England and Wales over 2 years were followed for 10.5 years. The contribution of eachtype of information was described in terms of the area under the receiver operating characteristic curve (AUC) and the number needed to detain (NND).
Results The AUCvalues using the three types of information were 0.66, 0.72 and 0.73 respectively. Prediction based on the full model using an optimal probability cut-off implies an NND of 2. The AUCs for serious offences were 0.67, 0.69 and 0.75 respectively.
Conclusions For long-term prediction of conviction on any charge, information on legal class adds little to the accuracy of predictions made using only a patients age, gender and criminal record. In the prediction of serious offences alone the contribution of legal class is significant.
|
|
|---|
We aimed to quantify the contributions made by three types of information to the prediction of criminal conviction in patients leaving high secure psychiatric care. We examined three hypotheses: first, that data relating to past offending would contribute the most to predictive accuracy; second, that this effect would be independent of the period covered by the prediction; and third, that this effect would be independent of the nature of the offence being predicted.
|
|
|---|
The Mental Health Act 1983 requires, for detention exceeding 28 days, the allocation of a patients mental disorder to one or more of four classes: mental illness, psychopathic disorder, mental impairment and severe mental impairment. The legal and administrative nature of these classes distinguishes them from clinical diagnoses. Many patients have more than one diagnosis, not all of which are reflected in the legal class they are assigned. Significantly also, a patients legal classification does not usually change as his or her condition fluctuates. Legal classes nevertheless contain diagnostically distinct groups of patients. The most recent survey of all three special hospitals found an 89% overlap (and a kappa coefficient of 0.8) between legal class and clinical diagnosis (Taylor et al, 1998). Thus, 85% of special hospital patients classified as mentally ill meet ICD10 (World Health Organization, 1992) criteria for psychosis and only 3% of patients who meet those criteria do not carry a mental illness classification (Taylor et al, 1998). Ninety-three per cent of men in Broadmoor special hospital and 97% of women in all three special hospitals who are classified as having a psychopathic disorder have one or more Axis II diagnoses under DSMIII (American Psychiatric Association, 1980). The mean numbers of DSMIII Axis II categories for those classified as having psychopathic disorder are 2.7 for men and 3.7 for women (Coid, 1992) and the most prevalent categories are antisocial, borderline and narcissistic (Reiss et al, 1999). All special hospital patients with ICD10 severe learning difficulties have been found to be detained under severe mental impairment and 88% of those with mild to moderate learning disabilities are detained under mental impairment (Taylor et al, 1998).
The first hypothesis was examined using any criminal conviction recorded within 10.5 years of discharge as the dependent variable. To investigate the second and third hypotheses the dependent variable was changed, first to reflect convictions received at different points over the 10.5 years and second to include only conviction on a serious charge. Serious was operationally defined to comprise murder, attempted murder, threat or conspiracy to murder, manslaughter, wounding, grievous bodily harm, actual bodily harm (multiple), child stealing, buggery, attempted buggery, rape, indecent assault, incest, gross indecency with children, robbery, kidnapping, aggravated burglary and arson. Buggery, attempted buggery, incest and gross indecency with children do not require non-consensual contact. They were included because at the time the members of the sample were convicted they were usually charged in England and Wales in cases of child abuse where the necessity of the victim giving evidence was sought to be avoided.
Characteristics of the sample
Of the 425 patients in the sample, 349 (82%) were men, and the mean age on
discharge was 38 years (range 1875, s.d.=10.9). Further data were
missing for one patient. The Mental Health Act 1983 allows a patient to be
placed in more than one class. Only 24 (6%) of the sample had received a dual
classification, however, suggesting that legal class had been allocated to
reflect the primary diagnosis rather than the full range of the
patients psychopathology. One hundred and ninety (45%) had been
classified as mentally ill, 141 (33%) as having psychopathic disorder, 72
(17%) as mentally impaired and 45 (11%) as severely mentally impaired.
Patients had been convicted of a mean of 4.6 and median of 2 offences (range 038, interquartile range 16) at the time of their discharge. On discharge, 281 (66%) went to be patients in other hospitals, 122 (29%) went home or to supported accommodation and 22 (5%) went back to prison. For patients categorised as mentally ill, the proportion becoming patients elsewhere was 68% and the proportion going home or to supported accommodation was 24%. For those categorised as having a psychopathic disorder these proportions were 60% and 36% respectively and for patients categorised as mentally impaired they were 76% and 23%. Eighty-seven (20%) had been convicted of an offence within 2.5 years of discharge, 118 (28%) within 6.5 years and 134 (32%) within 10.5 years. For patients with mental illness these proportions were 17%, 23% and 29% respectively; for patients with personality disorder they were 29%, 40% and 44% and for those with mental impairment they were 15%, 20% and 25%. When other variables were controlled for, the correlates of conviction for this sample were age, number of convictions at discharge and legal class of psychopathic disorder. Prior to controlling for other variables, correlates included additionally gender and destination on discharge (Buchanan, 1998).
Statistical procedure
The variables and the order in which they were entered were chosen to place
recognised risk factors for conviction
(Bonta et al, 1998;
Monahan et al, 2001)
in a sequence that reflected the resources required to obtain the information
that they comprise. Logistic regression equations were fitted to the data with
each of the outcomes as dependent variables. The equations used as independent
variables: first, age and gender; then, in addition, the number of convictions
the patient had received prior to leaving special hospital care; and, third,
in addition, each patients legal classification. The change in
likelihood at each of the three stages was used to assess the statistical
significance of the new information added (twice the change in log likelihood
having an approximate chi-squared distribution with degrees of freedom equal
to the change in the number of parameters).
The predicted probabilities of each patient being convicted were then used to generate the coordinates of a receiver operating characteristic curve. Receiver operating characteristic analyses describe the accuracy of a procedure for classification. The curves are generated by plotting sensitivity against 1 specificity at different thresholds. When used in conjunction with logistic regression, as here, the area under the curve (AUC) is the probability that a randomly selected case will score higher than a randomly selected non-case on the linear predictor (that is, the risk factors combined using the regression weights).
Receiver operating characteristic analyses were developed in the 1950s to describe the performance of radar (Altman & Bland, 1994). They have been used to describe the independent contributions of Gleason score, prostate-specific antigen and clinical judgement in staging prostate cancer (Partin et al, 1997; Swets et al, 2000), in the evaluation of screening programmes for hypercholesterolaemia (Umans-Eckenhausen et al, 2001) and in the identification of predictors of treatment response in multiple sclerosis (Wandinger et al, 2003). They have also been used to describe the success of attempts to predict general (Rice & Harris, 1995) and sexual (Hanson & Thornton, 2000) recidivism. They have not been employed previously to describe the contributions of different types of information to the prediction of conviction in a psychiatric population.
The description of the accuracy of a procedure for classification that a receiver operating characteristic analysis provides through the AUC does not vary with the base rate and takes into account that the sensitivity and specificity of any procedure based on a quantitative measure (such as probability of conviction) will change with the threshold that the procedure uses in order to classify. We computed the areas under the receiver operating characteristic curve at each stage and bootstrapped each change in these with 1000 replications to obtain the 95% bias-corrected confidence interval (Efron & Tibshirani, 1993). Where this excluded zero the change is noted as being significant at an approximate critical level of 0.05. Sensitivity and specificity were calculated using two probability cutoffs for predicting a case as positive: the first was 0.5 and the second was the value that maximised the sum of the sensitivity and specificity. This latter approach causes the choice of cut-off to change depending on the outcome, and is optimal (in terms of the costs of misclassification) if the cost of a false negative result is the same as that of a false positive one (Altman, 1995).
Using the sample prevalence, the number needed to detain (NND; Buchanan & Leese, 2001) was then calculated for the optimal cut-off. The NND is the number of people who would have to be detained in order to prevent one person from being convicted. It is the inverse of positive predictive value and, unlike the AUC, is a base-rate sensitive measure that can only be calculated when the prevalence of the behaviour to be prevented is known. Also unlike the AUC, however, it describes the practical consequences of using a prediction technique to detain those thought likely to offend. Because the NND is sensitive to the rate of the behaviour to be prevented it varies with the length of time in this case 2.5 years, 6.5 years or 10.5 years over which a prediction is made.
Destination on discharge is associated with conviction in this sample (Buchanan, 1998) and patients categorised as having a psychopathic disorder were more likely to be discharged to unsupported settings. The final models for 10.5 years were refitted including destination as an independent variable in case this could have been a confounding factor. Two sensitivity analyses were also conducted. In the first, the AUC and other measures of predictive ability were recalculated using jack-knifing. Jack-knifing is a method of cross-validation in which cases are left out one by one, the model re-estimated with each omission and the new model used to predict the omitted case. It was used to estimate the loss of predictive accuracy that would result from using the same prediction technique with a different sample. In the second sensitivity analysis the effect of dual classification was examined by treating legal class as a single, mutually exclusive, categorical variable. Finally, we examined whether the relative contributions of the different types of information derived from the sequence in which information was added to the model.
|
|
|---|
|
View this table: [in a new window] | Table 1 Results of entering three types of data in three logistic regressions: age and gender; age, gender and prior convictions; and age, gender, prior convictions and legal class of mental disorder. The dependent variable is conviction of any offence within 10.5 years of discharge from special (high secure) hospital |
|
View this table: [in a new window] | Table 2 Predictive accuracy of three regression equations age and gender (AG), age, gender and prior convictions (AG+C) and age, gender, prior convictions and legal class of disorder (AG+C+D) measured in terms of the log likelihood ratio, the area under the receiver operating characteristic curve and number needed to detain |
![]() View larger version (14K): [in a new window] [as a PowerPoint slide] |
Fig. 1 Receiver operating characteristic curves: predictions of any offence within
10.5 years. AG, age and gender; AG+C, age, gender and prior conviction;
AG+C+D, age, gender, prior conviction and legal class of disorder.
|
![]() View larger version (13K): [in a new window] [as a PowerPoint slide] |
Fig. 2 Receiver operating characteristic curves: prediction of a serious offence
within 10.5 years. AG, age and gender; AG+C, age, gender and prior
convictions; AG+C+D, age, gender, prior convictions and legal class of
disorder.
|
The NND shows that, applied to this population where the prevalence of any conviction in 10.5 years is 32% and using the optimal cut-off probability (in this instance of 0.30), the model including all three types of information would result in the detention of two people in order to prevent one conviction.
Table 2 also shows the results of testing the second and third hypotheses. The time over which predictions are made does not appear to affect the relative importance of different types of information to the prediction of conviction. The pattern of incremental improvement from age and gender through age, gender and prior convictions to age, gender, prior convictions and legal class is maintained when the dependent variable is changed to reflect conviction by 2.5 years and 6.5 years.
For conviction on a serious charge, the addition of criminal record data to age and gender, although significant in terms of the likelihood analysis, does not add significantly to the AUC. However, in contrast to the situation for all offences, the subsequent addition of information relating to legal class does add to the ability to predict. This difference in findings for serious offences compared with all offences is evident from comparing Figs 1 and 2. Further examination of the data showed that the ability of legal class to predict serious offences could be explained by a correlation between psychopathic disorder and subsequent sexual offending (child stealing, buggery, attempted buggery, rape, indecent assault, incest, gross indecency with children; odds ratio compared with those not classed under psychopathic disorder 4.09, P=0.047).
The NND indicates that, applied to this population where the prevalence of conviction on a serious charge in 10.5 years is 14% and using the optimal probability cut-off (in this instance 0.20), the model would result in the detention of between three and four people to prevent one serious conviction.
Destination after discharge was also examined as a potential explanatory variable, along with demographic information, prior convictions and clinical information, in the 10.5-year models for predicting any offence and serious offences. Those going home or to supported accommodation were more likely to commit an offence than those going as patients to other hospitals (OR=2.97, P < 0.001, for any offence; OR=2.03, P=0.021, for serious offences). However, as had been the case when destination was not controlled for, adding clinically derived information improved the accuracy of predictions of serious offences (AUC 0.720 increasing to 0.768) but not of offences in general (AUC 0.762 increasing to 0.766).
Two sensitivity analyses were performed. In the first the predictions from the 10.5-year models were jack-knifed. There was negligible change (1% change in each of sensitivity and specificity for cut-off probability 0.5). In the second, legal class was converted into a single, mutually exclusive, categorical variable. The areas under the curve for the two 10.5-year models did not change significantly.
The relative contribution of each type of information did not depend on the sequence in which information was added to the model. For all offences, prior conviction and demographic data continued to be significantly associated with conviction when they were added after legal class (not shown). For serious offences alone, legal class when added to demographic information is significant at P=0.001. The AUC for these two types of information is 0.74. Number of prior convictions when added to this model is significant at P=0.02 but increases the AUC by only 0.01 to 0.75 (NS at P=0.05).
|
|
|---|
This study and previous research
We have demonstrated the applicability of a statistical approach used
elsewhere in medicine to the quantification of the contributions made by
different kinds of information in the prediction of criminal conviction. The
ability of the approach to quantify the relative contributions of those
variables does not appear to be dependent on the sequence in which information
is added to the regression equations. The results of jack-knifing suggest that
the approach will be robust when used on other similar samples. The results
nevertheless require replication using different samples and different
variables, including variables that relate to a patients clinical
condition.
Our finding that demographic information and data from an individuals criminal record make significant contributions to the prediction of conviction on any charge, whereas legal class does not add to the accuracy that can be achieved using these two sources of information alone, is consistent with reviews suggesting that the long-term predictors of recidivism are the same for offenders with or without a psychiatric disorder (Bonta et al, 1998). That this remains the case regardless of the period over which predictions are made (when that period is measured in years) is consistent with earlier work showing that the overall accuracy of predictions is independent of time at risk (Mossman, 1994; Buchanan & Leese, 2001). The finding that legal class is more important in respect of serious convictions suggests, contrary to what had been thought to be the case, that predictors operate differently according to offence type.
For serious convictions the NND is higher than that for all convictions because the prevalence of serious convictions is lower and NND is a base-rate sensitive measure. The probability cut-off was chosen in a neutral way (to maximise the sum of sensitivity and specificity) and other valid choices could lead to lower values of NND, albeit at the expense of a higher false negative rate. It is possible that more sophisticated statistical methods, for example classification and regression tree analysis, would permit more accurate predictions to be made. Such methods demand substantially larger data-sets than logistic regression, however, in order to obtain robust predictors (Thomas et al, 2005).
Our AUC values should be compared with those reported elsewhere only with caution. The logistic model used for prediction has not been validated by applying it to a different sample from that used to estimate the parameters. The results of jack-knifing suggest that, at least for similar samples, the consequent loss of predictive accuracy would not be great, probably because in this case the sample size is reasonably large in comparison with the number of predictors. Such validation nevertheless invariably produces some shrinkage in the AUC.
Mossman (1994) described a mean AUC for all prediction techniques of 0.78 and areas of 0.71 and 0.67 for validated actuarial and clinical approaches respectively. We report AUC values for any offence and serious offences of 0.73 and 0.75 respectively. The regression equations in this study used fewer actuarial and clinical data than previous studies. With the above reservation, therefore, the results suggest that when long-term predictions of conviction on any charge are sought, the incremental value of information beyond that contained in basic demographic data and a persons criminal record may be limited. The results suggest also that the clinical information generating a legal classification of psychopathic disorder is more important, and criminal record less important, when only serious offences are sought to be predicted. We are not aware of a previous quantitative description of such an effect but the finding is consistent with suggestions that criminal record is a more potent predictor of general recidivism than of more serious crimes (Bonta et al, 1998). It may be that a diagnosis of personality disorder, and hence a classification of psychopathic disorder, is operating as a proxy for histories of violence and other criminal behaviour of which the clinical team is aware but of which the person has not been convicted, and that this effect is more pronounced when the offence is serious. A second possibility is that the legal class psychopathic disorder is operating as a proxy for psychopathy (Hare, 1970) and that psychopathy is independently associated with conviction after discharge (Harris et al, 1991). It is unclear from previous research, however, that such an effect would be limited to serious offending (Cooke et al, 2001; Stadtland et al, 2005).
A third possibility is that adding the legal class psychopathic disorder permits the prediction model to benefit from the reduced heterogeneity of offending by people convicted of sexual offences (Hanson & Bussière, 1998). Our method classified the majority of sexual offences as serious (Buchanan, 1998) and any breeding true of sexual offending therefore increased the rate of serious offending disproportionately. Further inspection of our data suggested that this was the most likely explanation. Patients classed as having a psychopathic disorder were more likely than other patients to have been convicted of a serious sexual offence prior to discharge (psychopathic disorder 30%, mental illness 7.9%, mental impairment 21%, severe mental impairment 2%).
Assessment and resources
The quality of any risk assessment is limited by the availability of
accurate information (Holloway,
1997). The amount and quality of the information that can be
obtained is limited, in turn, by resources. Some information required by
actuarial techniques, such as age and gender, is readily available. Other
data, such as those contained in a criminal record, are easy to obtain in some
cases but not in others. Information such as diagnosis requires more detailed
and time-consuming assessment. These results suggest that
screening patients for a history of violent behaviour may be as effective as
more complicated and resource-intensive assessments of the risk of future
violence, at least where all types of conviction are sought to be prevented.
This seems especially likely if, as others have suggested, patients themselves
will report most of the violence that can be ascertained by other means
(Steadman et al,
1998).
Implications for future research
Special hospital patients in England and Wales are atypical of psychiatric
patients. Further research using this approach should examine whether the
relative contributions of different types of information are the same for
different samples. It should also use the full range of actuarial and clinical
variables that have been shown to be correlated with conviction. Of the
clinical data not examined here, a history of substance misuse seems the most
obvious candidate to assist in the assessment of risk
(Harris & Rice, 1997;
Steadman et al,
1998). Not all aspects of a persons criminal record will be
of equal importance, and age at first conviction (for instance) may be a more
powerful predictor than total number of convictions received.
|
|
|---|
LIMITATIONS
|
|
|---|
|
|
|---|
Related articles in BJP:
This article has been cited by other articles:
![]() |
A. Buchanan Risk of Violence by Psychiatric Patients: Beyond the "Actuarial Versus Clinical" Assessment Debate Psychiatr Serv, February 1, 2008; 59(2): 184 - 190. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. T Tyrer An agitation of contrary opinions The British Journal of Psychiatry, May 1, 2007; 190(49): s1 - s2. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||