Background Few studies have investigated factors which predict inappropriate terminations (drop-out) of clinical contact with mental health services.
Aims To identify patient and treatment characteristics associated with dropping out of contact with community-based psychiatric services (CPS).
Method A 3-month cohort of patients attending the CPS was followed up for 2 years, to identify drop-outs.
Results We identified 495 patients who had had at least one psychiatric contact of whom 261 had complete ratings for the Global Assessment of Functioning and the Verona Service Satisfaction Scale. In the year after the index contact, 70 terminated contact with the CPS; of these, 44 were rated as having inappropriate terminations (the ‘drop-out’ group) and 26 had appropriate terminations of contact. Drop-outs were younger, less likely to be married and their previous length of contact with services was shorter. No drop-outs had a diagnosis of schizophrenia. Multivariate analysis revealed predictors of dropping out.
Conclusions In a CPS targeted to patients with severe mental illnesses, those who drop out of care are younger patients without psychoses who are generally satisfied with their treatment.
It is reasonable to assume that an episode of care should usually end when an episode of illness finishes. When these two end-points do not coincide, we need to understand why there has been an end to a period of treatment, although the illness has not yet been resolved. To clarify this, it is necessary first to define both types of episode and then to operationalise such definitions.
An ‘episode of care’ can be simply defined as the time interval between a first service contact for a mental health problem and a ‘ last’ contact with the services. The most useful definition of last contact in the field of mental health care, which has been tested using case register data, is ‘a contact, after which there is a gap of 90 days or more without any further contact’ (Tansella et al, 1995). This has been applied to the end of a single episode of care, but may not be a sufficiently long period of time without contact to establish that treatment has truly been terminated. By contrast, in this paper we define an ‘illness episode’ as ‘the time interval between the onset or recurrence of a mental health problem and its resolution or remission’.
This perspective can be developed by considering the nature of terminations to service contact, where these can be seen as either appropriate or inappropriate. By ‘appropriate terminations’ of contact, we mean those which occur when a clinical resolution or remission has taken place, or those cases when, for some other reason, staff and patient agree that treatment should be stopped. Inappropriate terminations are those which occur when there has not been a clinical resolution or an agreed termination, and they are referred to, in this paper, as ‘drop-out cases’. They are identified after excluding those patients who died or moved away from the local catchment area.
Previous research has shown that socio-demographic factors, such as age, marital status and living situation, may be important to predict such drop-outs (Trepka, 1986; Tehrani et al, 1996; Young et al, 2000). Other predictors of dropping out identified previously are: clinical setting, patient satisfaction (Pekarik, 1983; Tehrani et al, 1996; Young et al, 2000) and severity of clinical status (Robin, 1976).
Although it has been estimated, for example, that between 26% and 40% of patients may inappropriately leave out-patient follow-up care in a 1-year period (Pekarik, 1983; Tehrani et al, 1996; Young et al, 2000) and that this event is considered as an indicator of low quality of care (Grassi, 2000), until now, no studies have investigated those dropping out of care from an integrated community mental health service which aims to optimise continuity of care (Thornicroft & Tansella, 1999), nor have any used a comprehensive catchment area case register to ascertain cases and to evaluate their patterns of care.
The aim of this study is to identify patient and treatment characteristics associated with the likelihood of dropping out of contact with local community-based psychiatric services, so that services can identify the measures necessary to reduce inappropriate terminations of clinical contact.
The study was conducted in South Verona (about 75 000 inhabitants), an area that includes part of the city of Verona and two small neighbouring towns. The main agency providing psychiatric care for the adult population is the South Verona Community Psychiatric Service (CPS), which is run by the Section of Psychiatry, Department of Medicine and Public Health, University of Verona.
The CPS supplies a wide range of well-integrated hospital and community services. With the exception of hospital nurses, all staff work both within and outside the hospital. This ensures continuity of care through the different phases of treatment and across the various components of service provision (Sytema et al, 1997). Two private in-patient units (with a total of 220 beds), an out-patient service for children and adolescents, an out-patient service for those with addictions and a small number of general hospital neurological wards also provide psychiatric care to the residents in the Province of Verona, a wider area that includes South Verona (Tansella et al, 1998).
The South Verona Psychiatric Case Register
The South Verona Psychiatric Case Register (PCR), which began in 1979, records socio-demographic characteristics, past psychiatric and medical history, clinical data, and contacts with psychiatrists, psychologists, social workers and psychiatric nurses. The PCR collects information not only from the South Verona CPS but also from all public and private psychiatric services of the Province of Verona. Contacts with general practitioners (GPs), psychiatrists and psychologists in private practice are not reported to the PCR. Each attendance at an out-patient clinic and each home visit is counted as a contact. The PCR also routinely records details of patients who leave the catchment area and those who die. Estimates of time spent for each out-patient contact and each domiciliary visit are routinely recorded by the professionals providing care (Tansella et al, 1998), as are the types and numbers of professionals involved. This information forms the basis for calculating costs of specialist mental health care (Amaddeo et al, 1997).
This study is part of the South Verona Outcome Project, in which cross-sectional standardised assessments of patients in contact with the South Verona CPS have been made each year since 1994. Both first-ever patients and patients already in contact with the service are assessed, using several outcome measures, but in this study we report only the use of the Global Assessment of Functioning Scale (GAF) and the Verona Service Satisfaction Scale (VSSS) for all those seen by a psychiatrist or a psychologist. The Outcome Project study excludes contacts which take place in the casualty department or in the liaison psychiatry department because of logistical difficulties in assessing patients in these settings. Full details of the design of the Outcome Project are given in Ruggeri et al (1998).
The official Italian versions of GAF and VSSS were used. The GAF is a measure of individual well-being in the previous month on a continuous scale, where 0 denotes extremely severe dysfunction and 90 extremely good function (Endicott et al, 1976). The VSSS consists of 54 items covering 7 dimensions of the patient's experience of mental health services in the previous year: overall satisfaction, the skills and behaviour of professionals, information, access, efficacy, type of intervention and involvement of relatives items are rated on a 5-point Likert scale (1=terrible; 5=excellent) (Ruggeri & Dall'Agnola, 1993; Ruggeri et al, 1994).
All key professionals were trained in the correct use of these standardised instruments. Interrater reliability for GAF scores was assessed during the project and was always higher than 0.70 (intraclass correlation coefficient). If necessary, the research team helped the patients to complete the VSSS and assessed their understanding of items and coherence of assessments; confidentiality was fully preserved. The test-retest reliability and the validity of the VSSS have been assessed previously and proved to be good (Ruggeri & Dall'Agnola, 1993; Ruggeri et al, 2000).
This study includes all first-ever and all previously treated patients who were seen in the cross-sectional assessment period between October and December 1994, and for whom both GAF and VSSS were completed. Using the PCR, each patient was followed-up for 2 years after his/her first contact during the 3-month assessment period. Patients who died or moved away from the catchment area during the first year after the index contact were excluded from the study.
Drop-out patients were defined as those who (a) had a period without psychiatric contacts lasting at least 365 consecutive days, either immediately after the index contact or after further occasional contacts occurring in the following year and (b) those whose termination of treatment was not rated as appropriate.
To rate appropriateness of termination of contact, we considered the reason. This was independently assessed from the case notes of the last recorded contact and rated by a psychiatrist. From these case records, a rating was made for each patient in the following categories who terminated contact: (a) clinical resolution of the episode, (b) termination agreed between patient and clinician for other reasons, (c) termination not agreed, or (d) referral to the GP. When the reason for termination of contact could not be assessed from the case notes, the psychiatrist used the case notes to make a GAF rating of the overall functional level of each patient during the month preceding the date of the last recorded contact. This rating was blind to all previous GAF ratings and to the status of the patient in terms of contact termination. This retrospective method of rating the GAF from case records has been shown to be highly reliable (Mirandola et al, 2000).
Using the information gathered in the steps outlined above, an appropriate termination of contact with services was defined as applying to: (i) those patients for whom the recorded reason of termination referred to categories (a), (b) or (d) above or (ii) those patients in which the clinical condition at termination showed only a minor degree of disability/symptom severity, as shown by a GAF score of 70, for the month preceding the date of termination of contact, indicating only a mild degree of disability.
For each patient, the following data were collected in relation to the index contact.
Socio-demographic and diagnostic information (ICD-10 clinical descriptions, case register diagnosis; World Health Organization, 1992), which is reported in the following groups: schizophrenia and related disorders, affective disorders, anxiety-related and somatoform disorders, personality disorders and other disorders.
Presence of severe mental illness, defined using the criteria in Ruggeri et al (2000): (a) GAF score ≤ 50 and (b) over 2 years of contact with mental health services.
Length of contact with services, before entry into the study.
GAF score at entry into the study.
VSSS score at entry into the study.
Psychiatric care received in the 365 days before entry into the study (date on which the GAF and VSSS were rated): admitted or not admitted to hospital, number of day care contacts, number of out-patient contacts, number of domiciliary visits (data from the PCR).
Direct costs in the year preceding entry into the study. Costs were attached to each service contact recorded on the PCR so as to give the best local estimates of long-run marginal opportunity costs. Direct costs included are those concerning contacts with public and private specialist mental health services; costs of care provided by GPs, private psychiatrists and psychologists, and medication payments by patients were excluded. As we decided to use the most recent and more comprehensive list of unit costs, costs are expressed in Italian lire at 1999 price levels. All indirect costs were excluded from this study. The cost of out-patient contacts was calculated by taking the cost of a working minute for the different professionals (psychiatrists, psychologists, social workers, nurses, etc.) and multiplying this value by the estimated time spent in each contact. For the in-patient services, the cost per day was calculated, and for the rehabilitation groups and day care, an estimate was made of the cost per contact (taking into account the contact duration). The costs of private in-patient clinics were based on the prices paid by the Italian National Health Service. The support and treatment services listed here are mainly delivered, coordinated and funded by the public sector health service. Full details on the preparation of the unit cost list and the cost calculation have been reported elsewhere (Amaddeo et al, 1997, 1998).
The probability of being a drop-out was assessed by logistical regression. Since the study only included subjects (‘respondents’) for whom complete GAF and VSSS data were available, weights were applied to make the sample representative of all the eligible patients (patients who had been seen in the cross-sectional assessment in October—December 1994). The dependent variable was patient status (drop-out or not drop-out). The independent variables were: socio-demographic characteristics (gender, age, marital status, living situation, education and employment status); clinical characteristics (diagnosis and whether the patient had a severe mental illness or not); length of contact with services before entry into the study; service utilisation characteristics of the patients in the year preceding entry into the study (days of admission to hospital, days of contact with day care, number of contacts with out-patients or domiciliary care and total service costs in the previous year); patient overall functioning at entry into the study as rated by GAF score; and patient satisfaction characteristics at entry into the study, as rated by the VSSS total score and the scores in the seven VSSS dimensions.
Weights for non-response were proportional to the inverse probability of responding, estimated from a logistical regression on the whole group of eligible patients. The weights are greater for respondents with a lower response probability, who are therefore underrepresented in the analysed sample (Iannacchione et al, 1991). The independent variables were socio-demographic and clinical information available both for respondents and non-respondents, and the dependent variable was the response status. A ‘missing at random’ mechanism for non-response is assumed, given the characteristics included in the logistical regression model (Brick & Kalton, 1996). All statistical analyses were performed using STATA Release 7.0 (STATA Corporation, 2000).
A total of 495 patients were identified as having had at least one contact with a psychiatrist or a psychologist between October and December 1994. This 3-month cohort can be considered as representative of all patients who receive care locally from our community-based mental health service. Of those, 354 completed GAF and 261 completed both GAF and VSSS. Table 1 shows the number of patients who, in the 365 days after the index contact: moved away from the South Verona area (n=8), died (n=4), terminated contact with the South Verona CPS (n=70) and were still in contact (n=179). Of the 70 patients who terminated contact with services, 44 (17% of those who completed both the GAF and the VSSS) were rated as having in-appropriate terminations (and are therefore defined as the drop-out group) and 26 were rated as having appropriate terminations.
Table 2 presents the comparison of socio-demographic, clinical and psychiatric history data between drop-out and other patients (univariate analysis). Significant demographic and clinical differences were found between patients who remained in treatment and the drop-out group. Compared with the other patients, drop-outs were younger (F=4.88, P=0.002), and were less likely to be married, (F=3.32, P=0.037). The length of contact with services before entry into the study was greater for patients who remained in contact (F=7.72, P=0.0005). No significant differences between groups were found for gender, living condition, educational level or employment status. There were significant differences between groups for diagnosis (F=5.58, P=0.0002). Compared with drop-outs, patients who stayed in care were more likely to have severe mental illnesses and less likely to suffer from anxiety and somatoform disorders. Of the 44 patients who left care inappropriately, none had schizophrenia and only 4 were classified as having severe mental illness.
Table 3 shows the comparison of GAF and VSSS scores between drop-out patients and patients remaining in contact. Since there were no patients with a diagnosis of schizophrenia among the drop-outs, we divided those who remained in care into those suffering from schizophrenia and those who were not. Using a one-way analysis of variance (ANOVA), significant differences between these three groups were found for mean GAF scores (F=6.39, P=0.0002). Patients with schizophrenia remaining in contact had a lower mean score (52.5), whereas those without schizophrenia and drop-out patients had similar mean scores (62.1 v. 62.5). These differences are also clinically relevant because in the GAF, the range score from 50 to 60 is used to describe a moderate-to-severe level of impairment of symptoms and functioning, and the range from 60 to 70 is used to describe a mild-to-moderate level. For the VSSS total score and sub-scale scores, a trend was found only for lower satisfaction scores in drop-out patients.
Table 4 compares service utilisation during the previous year by drop-out patients, patients without schizophrenia who remained in care and those with schizophrenia who remained in care (excluding first-ever patients who, by definition, had received no contact in the previous year).
Table 5 shows the direct costs (Italian lire at 1999 prices) of care provided in the year preceding the entry into the study (weighted data). The drop-out group had received a much lower level of in-patient, sheltered residential, day and community care than those patients remaining in contact with services. The total costs for the drop-out group for the year preceding the index contact were much less than for patients with and without schizophrenia who continued contact over the follow-up period. Differences were statistically significant for day care, community care and total costs. Table 5 also strongly suggests that the clinical service is successfully targeted to patients with schizophrenia in terms of the balance of expenditure and clinical interventions.
All patients with schizophrenia remained in contact with services during the study period, so the diagnosis of schizophrenia was a perfect predictor for not dropping out. Table 6 shows the final logistic regression model calculated for the remaining 177 patients (excluding those with a diagnosis of schizophrenia, n=46). A backward selection was performed, and in the initial model, socio-demographic characteristics (living condition, working status, educational level, marital status), clinical variables (diagnosis, GAF, length of contact, number of contacts in the previous year), total costs and satisfaction with services were introduced. The prediction formula used in Table 6 and an example of a prediction for a typical patient are shown in the Appendix.
It was found that lower age, less use of day care, and less use of out-patient care in the previous year all increased the risk of dropping out of treatment. Premature termination of treatment was not associated with the other socio-demographic characteristics, psychiatric history or diagnosis (except schizophrenia), in-patient days or community care contact in the previous year.
No significant effect on dropping out was found for GAF score, VSSS total score and for total costs. Instead, associations were found between some aspects of patient satisfaction with services and dropping out: patients with greater satisfaction with the skill and behaviour of professionals had a greater probability of staying in contact, whereas those who were more satisfied with the type of intervention received were 5.9 times more likely to drop out.
This study, combining data from a PCR with data from other sources, aimed to identify characteristics associated with inappropriate termination of care which could be used in practice to reduce the numbers of such drop-outs within a community mental health service. As the study was conducted in a case register area, it was possible to operationalise termination of care over a relatively long period of time, and to relate these PCR data with standardised patient outcome measures of disability and service satisfaction, collected in routine clinical care. To ensure that the patients included in the drop-out group were only those who had discontinued ongoing contact with services, we adopted a very stringent criterion for eligibility, i.e. that no contact had taken place with any public or private service reporting to the case register for at least 365 consecutive days. In addition, we were able to identify (among those individuals who had lost service contact) those who had died, or who had moved out of the catchment area (in this study, 2.8% of patients, a proportion which is no different from the overall rate of emigration for the whole resident South Verona population). Those patients who terminated episodes of treatment for appropriate reasons were also identified. After this detailed, multi-stage procedure, the remainder who lost contact with services for more than the year following entry to the study were therefore the ‘ drop-out’ group.
The decision to include information on disability/symptomatology and satisfaction with services as possible predictors of drop-out reduced the size of our sample from 495 (all 3-month period prevalent treated cases) to 261 (those for whom both GAF and VSSS were complete). However, a well-established weighting procedure (see ‘Statistical analysis’ above) was used to ensure that the patient data included in the study were adjusted to be representative of all treated prevalent cases. This study extends previous work by drawing on case register data, using the types, amounts and costs of care received in the year prior to the index contact as potential predictor variables. This approach has the advantage of realistically categorising patients who end contact with care and who are relatively well (GAF > 70) as appropriate discontinuations, even if no formal agreement to discharge has been reached between clinician and patient, as in the study by Percudani et al (2002).
A limitation of our study was that it did not include follow-up details of patients who came under the treatment of private psychiatrists or private psychologists, or those who sought care from GPs without a transfer from the CPS. These limits are common to all studies using case registers, which do not usually include data from these sources.
Rate of inappropriate termination of contact (drop-out)
Among the 261 patients included in the cross-sectional South Verona Outcome Project we found that 17% (44) had an inappropriate termination of contact during the year after the index contact. Our findings are not consistent with the results of other studies (Pekarik, 1983; Tehrani et al, 1996; Young et al, 2000), which estimated that between 26% and 40% of patients may inappropriately leave out-patient care in a 1-year period. The lower drop-out found in our study may be explained by several factors. First, the South Verona service is designed to promote continuity of care, especially for people with severe mental illness (none of the patients with schizophrenia dropped out), which explains why only 27% (70 out of 261) of the total group discontinued contact during the year after the index contact. Second, previous studies have defined the concept of inappropriate termination of treatment less stringently (Baekeland & Lundwall, 1975; Louks et al, 1989; Koch & Gillis, 1991; Mohl et al, 1991), relating dropping out to the number of out-patient visits made or to the length of time in treatment (Atwood & Beck, 1985; Dworkin et al, 1986; Axelrod & Wetzler, 1989; Mohl et al, 1991). In these investigations, a patient was considered to be a drop-out if he or she terminated treatment before an arbitrary cut-off point, whether the clinician agreed with the termination or not. By contrast, our definition distinguished between appropriate and inappropriate termination. In addition, the possibility that some of our drop-out patients did not, in fact, drop out of contact with services, but rather transferred care to other providers not reporting to the case register, would further reduce the proportion of cases dropping out of care, and would increase the difference between our findings and those of previous studies. In these respects, the South Verona CPS acts more as an assertive outreach team than as a general adult mental health service within the UK context (Department of Health, 2001). This is because it has relatively fewer referrals of patients with lesser disability than catchment area teams in Britain (which often combine both consultation assessments at the request of primary care practitioners and the treatment of a longer-term case-load of patients with greater disability), as shown by previous UK—Italian comparative studies (Amaddeo et al, 1995; Gater et al, 1995).
Variables associated with dropping out of care
Our findings relating drop-out to younger age are consistent with the results of Kline & King (1973), Molnar & Pinchoff (1993), Tehrani et al (1996) and Young et al (2000), although the finding from the univariate analysis that drop-outs are more likely both to be younger and unmarried must be interpreted with caution, as these two variables are often associated in psychiatric datasets. In terms of clinical status, our results show that there was a significant difference in the level of disability between drop-out patients and patients who remained in contact. These results are consistent with those of Robin (1976). However, a recent controlled prospective study at a psychiatric out-patient service in London (without an outreach service) showed that those who missed appointments were more unwell and had higher levels of disability than those who did attend (Killaspy et al, 2000). This study, however, referred only to loss of contact with the out-patient component of the service, rather than with any part of the service, and therefore addressed a more restricted issue. In addition, our results show that direct costs of patients who dropped out of treatment in the previous year were significantly lower than those of patients both with and without schizophrenia, who remained in contact.
Predictors of dropping out of care
For the multivariate analysis, we excluded the 46 patients with schizophrenia because none of them dropped out of care in the year following the index contact; the diagnosis of schizophrenia thus might be considered a perfect predictor of non-drop-out. This is because the South Verona CPS is deliberately targeted at those with severe mental illness and if such a patient fails to attend for an appointment, the staff actively arrange to visit them at home to ensure continuity of clinical contact.
Termination of treatment might be assumed to represent a behavioural sign of dissatisfaction, so a strong inverse relationship between inappropriate termination of treatment and satisfaction with care might be expected. Our use of a detailed service satisfaction scale allowed us to differentiate between different aspects of satisfaction. In terms of the bivariate analysis, we found a trend, but no significant differences, for lower satisfaction in drop-out patients (Table 3). However, when multivariate analyses were performed to identify predictors of dropping out, after excluding patients with a diagnosis of schizophrenia, satisfaction with type of intervention received became the most significant predictor. This sub-scale summarises ratings made on 17 items of the VSSS referring to patients' perceptions of a wide range of treatment and care received, from medication to sheltered work and advice on welfare benefits. This suggests that, for a group of patients predominantly without psychoses, dropping out of contact with services is strongly associated with being more satisfied with the interventions received in the period prior to the index contact and this implies that, from the perspective of these patients, the termination of contact was appropriate.
These results indicate that different criteria may be used by staff and by patients not suffering from schizophrenia when judging at which point to discontinue clinical contact, and that these different priorities may well warrant a more detailed investigation. This leads us to the tentative conclusion that a basis for planning a mutually agreed termination of treatment by clinicians should be open discussion with patients as to whether they are satisfied with the type and amount of treatment received, and when they feel that they have had sufficient care.
At the same time, as expected, patients who are less satisfied with the professional skills and behaviour (rating professionalism, competence and thoroughness of staff) are also more likely to drop out of care. Therefore, these aspects of satisfaction may also have important consequences for whether patients without a diagnosis of schizophrenia allow continuing clinical contact, and therefore potentially effective treatment, to take place at all. This also indicates a further avenue for research, namely the interrelationships between the processes and the outcomes of care, in this case where the processes are rated from a patient perspective.
Clinical Implications and Limitations
Only 17% of patients seeking care from a community-based psychiatric service (none with a diagnosis of schizophrenia) had inappropriate terminations of contact (drop-out).
Self-rated satisfaction with treatment using a standardised scale, for patients not diagnosed with schizophrenia, is highly predictive of patients dropping out of care in the subsequent year.
Patients who are less satisfied with the professional skills and behaviour of staff are also more likely to drop out of care. This also indicates important interrelationships between the processes and the outcomes of care, in this case where the processes are rated from a patient perspective.
This study did not include follow-up details of patients who came under the treatment of private psychiatrists, private psychologists or general practitioners. This would further reduce the already low proportion of cases dropping out of care.
Generalisability may be limited since this study was conducted on a single site.
Self-assessments by patients of their reasons for dropping out, and their own ratings of their disability or symptom severity, are not included.
An example of a prediction for a typical patient could be made using the following formula: where: a is the constant value of the regression
So, for a patient with:
Satisfaction with professionals' skills and behaviour=4
Satisfaction with type of intervention=3
Day care contacts in the previous year=40
Out-patient contacts in the previous year=10
and considering that the constant value of the logistic regression is equal to -0.704:
Thus, P<0.5. As P=0 is not ‘drop-out’ and P=1 is ‘drop-out’, then the patient with these characteristics has a high probability of not dropping out of care.
We acknowledge the contributions to this study by Dr Rossella Beneduce, who made the GAF assessments from case records, and Dr Morven Leese, who gave very helpful statistical advice.
- Received June 15, 2001.
- Revision received April 15, 2002.
- Accepted June 11, 2002.
- © 2002 Royal College of Psychiatrists