Hostname: page-component-7c8c6479df-r7xzm Total loading time: 0 Render date: 2024-03-28T06:12:29.701Z Has data issue: false hasContentIssue false

Collaborative care for depression in primary care

Making sense of a complex intervention: systematic review and meta-regression

Published online by Cambridge University Press:  02 January 2018

Peter Bower*
Affiliation:
National Primary Care Research and Development Centre, University of Manchester
Simon Gilbody
Affiliation:
Department of Health Sciences, University of York
David Richards
Affiliation:
Department of Health Sciences, University of York
Janine Fletcher
Affiliation:
Department of Nursing, Midwifery and Health Visiting, University of Manchester
Alex Sutton
Affiliation:
Department of Health Sciences, University of Leicester, UK
*
Dr Peter Bower, National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL, UK. Email: peter.bower@manchester.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Background

The management of depression in primary care is a significant issue for health services worldwide. ‘Collaborative care’ interventions are effective, but little is known about which aspects of these complex interventions are essential.

Aims

To use meta-regression to identify ‘active ingredients' in collaborative care models for depression in primary care.

Method

Studies were identified using systematic searches of electronic databases. The content of collaborative care interventions was coded, together with outcome data on antidepressant use and depressive symptoms. Meta-regression was used to examine relationships between intervention content and outcomes.

Results

There was no significant predictor of the effect of collaborative care on antidepressant use. Key predictors of depressive symptom outcomes included systematic identification of patients, professional background of staff and specialist supervision.

Conclusions

Meta-regression may be useful in examining ‘active ingredients' in complex interventions in mental health.

Type
Review Article
Copyright
Copyright © Royal College of Psychiatrists, 2006 

Depression is prevalent in primary care, but current management is suboptimal (Reference Simon and Von KorffSimon & Von Korff, 1995). There is increasing evidence of the effectiveness of ‘collaborative care’ (Reference Gilbody, Whitty and GrimshawGilbody et al, 2003), a multifaceted organisational intervention involving new staff and ways of working (Reference Von Korff and GoldbergVon Korff & Goldberg, 2001). However, collaborative care interventions vary in content and intensity, and it is unclear which aspects are crucial determinants of effectiveness (the ‘active ingredients’). Most of the current collaborative care literature derives from the USA, and designing collaborative care interventions for use in other settings requires an understanding of these ‘active ingredients’.

Collaborative care is an example of a complex intervention, involving a number of separate mechanisms, where the ‘active ingredient’ is difficult to specify (Reference Campbell, Fitzpatrick and HainesCampbell et al, 2000). If different collaborative care interventions vary in their inclusion of ‘active ingredients’, then this should lead to significant variation in outcomes. Such variation in outcomes in a meta-analysis is described as statistical heterogeneity. Meta-regression is a method used to explore statistical heterogeneity in meta-analysis (Reference Sutton, Abrams and JonesSutton et al, 1998; Reference Thompson and HigginsThompson & Higgins, 2002).

A phased approach to the development of complex interventions has been proposed (Reference Campbell, Fitzpatrick and HainesCampbell et al, 2000). Modelling of complex interventions, where the ‘active ingredients’ are explored, is a critical step in the phased model prior to further trials. However, there are relatively few examples of the phased model in the literature (Reference Bradley, Kinmonth and MantBradley et al, 1999; Reference Campbell, Fitzpatrick and HainesCampbell et al, 2000; Medical Research Council, 2000; Reference LoebLoeb, 2002) and a lack of consensus as to the optimal modelling methods.

The authors are developing and testing a collaborative care intervention in the UK using the phased approach, and used meta-regression to examine the relationship between the content of collaborative care interventions and outcomes, to identify ‘active ingredients’ and thus assist in the design of a UK collaborative care model for the care of depression.

METHOD

Data sources

We based our meta-regression on a systematic review. A published systematic review of organisational interventions in primary care mental health completed by S.G. was used as the initial source of studies (Reference Gilbody, Whitty and GrimshawGilbody et al, 2003); this review included collaborative care as well as other types of organisational interventions used to improve the management of depression. Searches included Medline, EMBASE, CINAHL, PsycINFO, the Cochrane Library and the Database of Abstracts of Reviews of Effectiveness (DARE), and were run from the inception date of each database to 2003. We updated the comprehensive search strategies from this review (without language restriction) to June 2004 to find recent studies, and then conducted a second update to November 2005 (Fig. 1). Details of the exact search methods and a table of excluded studies are available from the authors upon request. From this comprehensive database, we then identified the subset of collaborative care studies.

Fig. 1 QUOROM (Quality of Reporting Meta-analyses) flow diagram.

Inclusion criteria

The population of interest was patients with depressive symptoms or diagnosed depressive disorders in primary care settings. Primary care was defined as the provision of medical care by professionals who provide first contact and ongoing care to patients, regardless of the patient's age, gender or presenting problem.

Although we have published a broad typology of models of quality improvement which includes collaborative care (Reference Bower and GilbodyBower & Gilbody, 2005), developing precise inclusion criteria for such complex interventions is more problematic, because by definition it is not clear a priori which mechanisms have to be in place in order to define an intervention as ‘collaborative care’. Therefore, any definition is potentially arbitrary, reflected by published reviews of collaborative care that disagree on which studies and interventions are included and excluded (Reference Von Korff and GoldbergVon Korff & Goldberg, 2001; Reference Gilbody, Whitty and GrimshawGilbody et al, 2003; Reference Bijl, van Marwijk and de HaanBijl et al, 2004).

The purpose of the study was to examine the relationship between variation in the interventions within collaborative care studies, and outcomes. Therefore, we used a broad definition, and defined ‘collaborative care’ as a multifaceted organisational intervention, which could include a number of components:

  1. (a) the introduction of a new role (case manager) into primary care, to assist in the management of patients with depression through structured and systematic delivery of interventions;

  2. (b) the introduction of mechanisms to foster closer liaison between primary care clinicians and mental health specialists (including case managers) around individual patient care;

  3. (c) the introduction of mechanisms to collect and share information on the progress of individual patients.

We excluded educational and training interventions and the provision of brief psychological therapy where these were the sole intervention and were not supported by other enhancements of care outlined above. The full list of studies is given in Table 1.

Table 1 Studies included in the review

Study Reference Setting Unit of randomisation Sample size n Patient population Antidepressant use data? Depressive symptoms data?
Adler 2004 Adler et al (Reference Adler, Bungay and Wilson2004) USA Patient 533 Adults with major depression or dysthymia Yes Yes
Akerblad 2003 Bungay et al (Reference Bungay, Adler and Rogers2004) Sweden General practitioner 1031 Adults with major depression and an indication for antidepressants Yes Yes
Araya 2003 Araya et al (Reference Araya, Rojas and Fritsch2003) Chile Patient 240 Adult women with major depression Yes Yes
Blanchard 1995 Blanchard et al (Reference Blanchard, Waterreus and Mann1995) UK Patient 96 Elderly people with depression warranting clinical intervention Yes Yes
Brook 2003 Brook et al (Reference Brook, van Hout and Nieuwenhuyse2003a ,Reference Brook, van Hout and Nieuwenhuyse b ) The Netherlands Patient 147 Adults with depressive complaints, prescribed new antidepressant No Yes
Bruce 2004 Coyne et al (Reference Coyne, Brown and Datto2001); Bruce et al (Reference Bruce, Ten Have and Reynolds2004) USA Practice 598 Elderly people with major depression, dysthymia and minor depression Yes Yes
Callahan 1994 Callahan et al (Reference Callahan, Hendrie and Dittus1994) USA Practice 175 Elderly people with newly diagnosed depression Yes Yes
Capoccia 2004 Boudreau et al (Reference Boudreau, Capoccia and Sullivan2002); Capoccia et al (Reference Capoccia, Boudreau and Blough2004) USA Patient 74 Adults with depression, prescribed a new antidepressant Yes Yes
Coleman 1999 Coleman et al (Reference Coleman, Grothaus and Sandhu1999) USA Practice 169 Frail elderly people No Yes
Datto 2003 Datto et al (Reference Datto, Thompson and Horowitz2003) USA Practice 61 Adults with depressive symptoms No Yes
Dietrich 2004 Dietrich et al (Reference Dietrich, Oxman and Williams2004a ,Reference Dietrich, Oxman and Williams b ) USA Practice 405 Adults with major depression and dysthymia, starting/changing treatment Yes Yes
Finley 1999 Finley et al (Reference Finley, Rens and Gess1999, Reference Finley, Rens and Pont2003) USA Patient 125 Adults with current major depression, prescribed a new antidepressant Yes Yes
Hunkeler 2000 Hunkeler et al (Reference Hunkeler, Meresman and Hargreaves2000) USA Patient 302 Adults with major depression or dysthymia, prescribed a new antidepressant Yes Yes
Katon 1995 Katon et al (Reference Katon, Von Korff and Lin1995); Von Korff et al (Reference Von Korff, Katon and Bush1998) USA Patient 217 Adults with depression, prescribed a new antidepressant Yes Yes
Katon 1996 Katon et al (Reference Katon, Robinson and Von Korff1996); Von Korff et al (Reference Von Korff, Katon and Bush1998) USA Patient 153 Adults with depression, prescribed a new antidepressant Yes Yes
Katon 1999 Katon et al (Reference Katon, Von Korff and Lin1999); Simon et al (Reference Simon, Katon and Von Korff2001a ) USA Patient 228 Adults on antidepressants, at high risk of persistent depression, recurrent depression or dysthymia Yes Yes
Katon 2001 Katon et al (Reference Katon, Rutter and Ludman2001a ), Simon et al (Reference Simon, Von Korff and Ludman2002) USA Patient 386 Adults, prescribed a new antidepressant, at high risk of relapse Yes Yes
Katon 2004 Katon et al (Reference Katon, Von Korff and Lin2003, Reference Katon, Von Korff and Lin2004) USA Patient 329 Adults with diabetes with depressive symptoms No Yes
Katzelnick 2000 Katzelnick et al (Reference Katzelnick, Simon and Pearson2000); Simon et al (Reference Simon, Manning and Katzelnick2001b ) USA Practice 407 Adults, high users of services, with depressive symptoms Yes Yes
Mann 1998 Mann et al (Reference Mann, Blizard and Murray1998) UK Patient 419 Adults with depression Yes Yes
Oslin 2003 Oslin et al (Reference Oslin, Sayers and Ross2003) USA Physician 97 Adults with depression or dysthymia, at risk drinking No Yes
Peveler 1999 Peveler et al (Reference Peveler, George and Kinmonth1999) UK Patient 160 Diagnosis of depression, prescribed a new antidepressant Yes Yes
Rickles 2003 Rickles (Reference Rickles2003) USA Patient 63 Prescribed a new antidepressant No Yes
Rost 2001a Rost et al (Reference Rost, Nutting and Smith2000, Reference Rost, Nutting and Smith2001); Pyne et al (Reference Pyne, Smith and Fortney2003) USA Practice 243 Adults with major depression, prescribed a new antidepressant, recently treated Yes Yes
Rost 2001b As above USA Practice 189 Adults with major depression, prescribed a new antidepressant, beginning new episode Yes Yes
Simon 2000 Simon et al (Reference Simon, Von Korff and Rutter2000) USA Patient 392 Adults with depression, prescribed a new antidepressant Yes Yes
Simon 2004a Simon et al (Reference Simon, Ludman and Tutty2004) USA Patient 402 Adults with depression, prescribed a new antidepressant Yes Yes
Simon 2004b As above USA Patient 393 Adults with depression, prescribed a new antidepressant Yes Yes
Swindle 2003 Swindle et al (Reference Swindle, Rao and Helmy2003) USA Firm 268 Adults with major depression, dysthymia or partially remitted major depression Yes Yes
Unutzer 2002 Unutzer et al (Reference Unutzer, Katon and Williams2001a , Reference Unutzer, Katon and Callahan2003) USA Patient 1801 Elderly people with major depression, dysthymia, or both Yes Yes
Wells 2000a Wells et al (Reference Wells, Sherbourne and Schoenbaum2000); Sherbourne et al (Reference Sherbourne, Wells and Duan2001); Schoenbaum et al (Reference Schoenbaum, Unutzer and Sherbourne2001); Unutzer et al (Reference Unutzer, Rubenstein and Katon2001b ); Wells et al (Reference Wells, Sherbourne and Schoenbaum2004) USA Practice 867 Adults with major depression or dysthymia Yes Yes
Wells 2000b As above USA Practice 932 Adults with major depression or dysthymia Yes Yes
Whooley 2000 Whooley et al (Reference Whooley, Stone and Soghikian2000) USA Practice 331 Elderly people with depressive symptoms Yes Yes
Wilkinson 1993 Wilkinson et al (Reference Wilkinson, Allen and Marshall1993) UK Patient 61 Adults with depression, prescribed a new antidepressant Yes Yes

Data extraction

Content of collaborative care

We initially tested published coding schemes relating to quality improvement (Reference Weingarten, Henning and BadamgaravWeingarten et al, 2002; Reference Bero, Eccles and GrilliBero et al, 2006), but these lacked the detail to capture the specific issues of relevance to collaborative care. Therefore, a basic coding frame was developed on the basis of the ‘prototypical’ collaborative care model, described in terms of the three roles potentially involved in patient care: primary care provider, mental health specialist and case manager (Reference Katon, Von Korff and LinKaton et al, 2001b ). Variables were created relating to the professional background of each worker and additional intervention-specific training. These codes were then supplemented by variables describing the potential interprofessional relationships (e.g. specialist supervision of the case manager, and case manager feedback of information to the primary care provider). Because professional–patient contact within collaborative care is focused on the case manager–patient relationship, we added variables relating to the intensity and nature of that contact. Finally, we added three variables related to the characteristics of the patients and study setting (see Appendix).

After piloting the data extraction among the team, data from each study were extracted by two different members of the research team working independently. There was no formal measurement of reliability, but disagreements were few and were resolved by discussion. Owing to inconsistent reporting of data in the published papers we were only able to extract comprehensive data on 8 of the original 27 variables (see Table 2).

Table 2 Intervention content variables (n=34)

Characteristic n
Setting
    USA 27
    Non-USA 7
Recruitment method
    Systematic identification 22
    Referral by clinicians 12
Patient population
    Patients with depression 16
    Patients with depression specifically 18
    willing to take antidepressants
Primary care physician training
    Training provided 15
    No training provided 19
Case manager background
    Mental health 17
    Non-mental health 17
Case management sessions
    4 or fewer 13
    5-7 11
    8+ 10
Case manager supervision
    Regular/planned 24
    Other arrangements 10
Case management content
    Medication management alone 21
    Medication management plus 13
    psychological therapy

Concealment of allocation is the quality attribute with the best evidence for an association with outcomes (Schultz & Grimes, 2002), and we extracted data on concealment to test whether outcomes were related to study quality.

Intervention outcomes

Collaborative care interventions often seek to improve adherence to antidepressant medication, and the first outcome measure was changes in measures of antidepressant use. Most studies reported data in dichotomous form, e.g. the proportion of patients taking antidepressants or meeting standardised guidelines for antidepressant use.

The second outcome measure was reduction in depressive symptoms. A wide variety of outcomes were reported at a number of different time points. Because the meta-regression required as large a sample of studies as possible for reliable analysis (Reference Thompson and HigginsThompson & Higgins, 2002), we restricted the analysis to short-term outcomes (approximately 6 months after randomisation), as these outcomes were by far the most frequently reported. Where alternative measures of depressive outcomes were reported within the same study, the data extracted were chosen on the basis of an a priori decision rule which extracted any identified primary outcome first, and then prioritised observer-rated scales over self-report measures where available.

We extracted all measures of anti-depressant use as dichotomous outcomes, analysed using odds ratios. Measures of depressive symptoms included a mix of dichotomous and continuous outcomes. We translated continuous measures to a standardised effect size, i.e. the mean of the intervention group minus the mean of the control group, divided by the pooled standard deviation. We translated outcomes reported as dichotomous variables to standardised effect sizes using the logit transformation (Reference Lipsey and WilsonLipsey & Wilson, 2001). In 5 of 62 (8%) comparisons, missing data (e.g. standard deviations) were imputed from other relevant studies, in line with accepted practice (Reference Furukawa, Barbui and CiprianiFurukawa et al, 2006).

Previous reviews have identified that unit of analysis errors are common in the evaluation of collaborative care (Reference Gilbody, Whitty and GrimshawGilbody et al, 2003), making studies more susceptible to type 1 errors. We identified all studies using cluster randomisation and where necessary adjusted the precision of these studies in the meta-analysis using methods recommended by the Effective Practice and Organisation of Care (EPOC) group of the Cochrane Collaboration (Reference Bero, Eccles and GrilliBero et al, 2006) and assuming an intraclass correlation of 0.02. The effects of adjustment for clustering were examined in a sensitivity analysis using intraclass correlations of 0.00 and 0.05 (Reference Donner and KlarDonner & Klar, 2002).

Analysis

Analyses were conducted in Stata version 8 for Windows, using the metan and metareg macros. The initial meta-analyses used random effects modelling (Reference Sutton, Abrams and JonesSutton et al, 1998) to provide an overall pooled measure of effect of collaborative care on the two outcomes. However, the main focus of the analysis was on heterogeneity. Heterogeneity was measured using the I 2 statistic, which estimates the percentage of total variation across studies that can be attributed to heterogeneity rather than chance. As a guide, I 2 values of 25% may be considered low, 50% moderate and 75% high (Reference Higgins, Thompson and DeeksHiggins et al, 2003).

The main analysis used random effects meta-regression, which provided estimates of the relationships between eight intervention content variables and the two outcomes. The permutation test was used to calculate P values (using 1000 Monte Carlo simulations) and to reduce the chance of spurious false-positive findings (Reference Higgins and ThompsonHiggins & Thompson, 2004). The amount of heterogeneity explained by the intervention content variables was examined by reductions in the I 2 statistic. Initial univariate analyses (using a criterion of significance of P<0.10) were followed by estimation of a multivariate model. The multivariate model was not based on any automated selection procedure, but involved examination of a number of candidate models involving different combinations of variables. The final model was chosen on the basis of the greatest reduction in heterogeneity. A secondary meta-regression provided an estimate of the relationships between the two outcomes (i.e. whether antidepressant use predicted depressive symptoms).

RESULTS

We identified 28 published studies of collaborative care interventions with outcome data on antidepressant use and 34 studies with outcome data on depressive symptoms (Table 1). Intervention content variables are summarised in Table 2.

Meta-analysis

We found a positive effect of collaborative care on antidepressant use (odds ratio 1.92, 95% CI 1.54–2.39; Fig. 2) and depressive outcomes (standardised mean difference 0.24, 95% CI 0.17–0.32; Fig. 3). The I 2 estimates of inconsistency were 80% and 54% respectively.

Fig. 2 Meta-analysis of antidepressant use. Note: the Wells (2000) and Simon (2004) studies involved two intervention groups compared against a single control; to avoid double-counting the controls, the sample size and event rate in the control were divided by 2. The Rost 2001 study data are only available analysed in two subgroups, rather than as an overall analysis; in our analysis these subgroups were treated as separate comparisons.

Fig. 3 Meta-analysis of depressive symptoms (see note for Fig. 2).

Meta-regression

Analyses of the effects of intervention content variables are shown in Tables 3 and 4. There was insufficient variability in quality of allocation concealment, as most studies were rated as ‘not clear’, and this variable was not used as a covariate in the final analysis.

Table 3 Univariate analysis of associations between intervention content variables and antidepressant use

Variable Category 1 Category 2 Log odds ratio regression coefficient (95% CI) P I 2 (%)
Study setting Outside USA USA 0.076 (-0.558 to 0.710) 0.804 80.2
Patient sample Patients with depression Patients with depression willing to take antidepressants -0.123 (-0.631 to 0.385) 0.647 80.1
Recruitment method Referral Systematic identification 0.345 (-0.167 to 0.858) 0.183 78.5
Primary care physician training No training provided Training provided 0.328 (-0.163 to 0.818) 0.194 79.5
Case manager background Non-mental health professional Mental health professional 0.220 (-0.280 to 0.721) 0.393 78.9
Content of case management Medication management Medication management plus psychotherapeutic processes -0.104 (-0.617 to 0.409) 0.683 80.5
Supervision of case manager None or variable Regular and planned 0.039 (-0.549 to 0.627) 0.906 80.4
Case management sessions1 -0.053 (-0.126 to 0.020) 0.151 79.6

Table 4 Univariate analysis of associations between intervention content variables and depressive symptoms

Variable Category 1 Category 2 Effect size regression coefficient (95% CI) P I 2 (%)
Study setting Outside USA USA 0.007 (-0.193 to 0.206) 0.930 54.4
Patient sample Patients with depression Patients with depression willing to take antidepressants -0.087 (-0.243 to 0.070) 0.285 52.1
Recruitment method Referral Systematic identification 0.146 (-0.014 to 0.306) 0.061 47.8
Primary care physician training No training provided Training provided 0.093 (-0.065 to 0.252) 0.237 54.9
Case manager background Non-mental health professional Mental health professional 0.187 (0.046 to 0.327) 0.004 42.7
Content of case management Medication management Medication management plus psychotherapeutic processes 0.093 (-0.064 to 0.250) 0.206 50.7
Supervision of case manager None or variable Regular and planned 0.169 (0.002 to 0.337) 0.033 49.3
Case management sessions1 0.015 (-0.008 to 0.039) 0.174 50.9

None of the intervention content variables was significantly associated with anti-depressant use, and no multivariate model was estimated. Three intervention content variables predicted improvement in depressive symptoms: recruitment by systematic identification (P=0.061), case managers having a specific mental health background (P=0.004) and provision of regular supervision for case managers (P=0.033), which reduced the overall heterogeneity I 2 from 54% to 48% and 43 to 49% respectively.

In multivariate analysis, four intervention content variables produced the most robust meta-regression in relation to depressive symptom outcomes. The analysis indicated that non-US studies (P=0.038), recruiting through systematic identification of patients (P=0.081) and using case managers having a specific mental health background (P=0.027) who received regular supervision (P=0.055) were more effective. The combination of these four covariates reduced the overall heterogeneity to 36% (low to moderate between study heterogeneity). The inclusion of ‘setting’ (which was not statistically significant in the univariate analyses) reflects the fact that the multivariate analysis accounts for both the relationships between each intervention content variable and the outcome, and the relationships between intervention content variables (Reference Tabachnick and FidellTabachnick & Fidell, 2001).

The meta-regression of the relationships between antidepressant use and depressive symptoms showed a positive association (β coefficient 0.20, 95% CI 0.02–0.38, P=0.028; Fig. 4).

Fig. 4 Relationship between antidepressant use outcomes and depressive symptoms outcomes.

The results of these analyses were not substantively influenced by the sensitivity analysis using estimates of intracluster correlations of 0.00 and 0.05.

DISCUSSION

Overall, the analysis showed an interesting pattern of results. No variable predicted variation in relation to our first outcome, antidepressant use. However, the study did identify several predictors of the second outcome, depressive symptoms. Furthermore, antidepressant use did predict depressive symptom outcomes, which suggests that effects of collaborative care on the latter may be mediated through changes in the former. However, it is not clear whether this association would remain significant if the antidepressant use variables were analysed alongside the other intervention content variables. Clearly the causal pathways between intervention content variables, intermediate outcomes (such as antidepressant use) and final outcomes (such as depressive symptoms) are potentially complex, and analytic techniques such as path analysis might be useful in examining these relationships further.

If the associations between intervention content variables and depressive symptom outcomes are robust, they have interesting implications for the design of collaborative care interventions. For example, the use of case managers with a mental health background and regular specialist supervision both predict outcomes, which suggests that expertise is important. This may have implications for the involvement of the new paraprofessional graduate workers in collaborative care models (Reference Whitty and GilbodyWhitty & Gilbody, 2005).

Clearly the meta-regression cannot determine the process by which expertise has its influence. This may relate to specific technical skills, such as knowledge of anti-depressants or the effective use of psychotherapeutic techniques, or may reflect non-specific skills, such as the ability to engage with patients or to work effectively in collaboration with other professionals. Exploration of this issue might benefit from qualitative research on the nature of patient–professional and interprofessional contact in collaborative care, and the influence of context and organisational variables (Reference Weaver, Tyrer and RitchieWeaver et al, 2003).

However, models of care which require personnel with significant expertise are likely to be more difficult to implement in some contexts, which may limit their usefulness, reflecting the potential tension between ‘efficacy’ as demonstrated in trials and ‘effectiveness’ in routine contexts. Also, models using expert personnel may be more costly, which raises issues about trade-offs between effectiveness and cost that need to be considered when designing collaborative care interventions.

Limitations of the systematic review

As a complex intervention, collaborative care defies simple definition. Our decisions about inclusion and exclusion were informed by our previous conceptual work (Reference Bower and GilbodyBower & Gilbody, 2005), but we took a liberal approach to inclusion precisely because the study focused on the degree to which variability in collaborative care models influenced outcomes. Clearly the inclusion or exclusion of particular studies may have important implications, and thus our findings should be considered exploratory rather than definitive. It should also be noted that most studies were conducted in the USA and the results may not generalise to other contexts. Setting was a significant predictor in the multivariate analysis.

The validity of the coding scheme used to extract data on the interventions has not been confirmed. As noted previously, there were problems of inconsistent reporting and missing data in the published studies. A significant proportion of intervention content variables could not be included as they were not reported consistently, and it is unlikely that it would have been possible to extract data on many additional issues. However, it remains possible that other variables might be more effective predictors than those included in our analyses.

The difficulties encountered in deriving a full description of the interventions echoes traditional problems with poor reporting in randomised trials. There may be a case for adopting a more standardised approach to the reporting of the content of complex interventions (equivalent to CONSORT (Consolidated Standards of Reporting Trials; Reference Moher, Schutz and AltmanMoher et al, 2001) and QUOROM (Quality of Reporting Metaanalyses; Reference Moher, Cook and EastwoodMoher et al, 1999) in order to overcome these problems. The proliferation of web-based journal archives for the presentation of data outside the word limits of the paper-based journals provides an appropriate platform. However, determining the appropriate content and structure of such standardised reports would be challenging, given the potential range of processes involved in complex interventions.

Limitations of the meta-regression technique

The technique of meta-regression has several limitations (Reference Thompson and HigginsThompson & Higgins, 2002). The analysis represents an observational association only, because meta-regression across trials does not have the benefits of randomisation. Equally, statistical power to detect useful associations using meta-regression is limited by (among other things) the number of available studies (Reference Lambert, Sutton and AbramsLambert et al, 2002). Outliers may have a large influence, particularly in the context of a limited sample size. The multivariate model described earlier was found to be sensitive to the particular variables included in the analysis. It should also be noted that the analysis will not be able to detect ‘active ingredients’ that are necessary but do not vary between interventions. Furthermore, it is possible that with certain variables, such as the number of case management sessions, the relationship with average numbers of sessions across trials may not be the same as the relationship within trials. Only individual patient data analysis could overcome this ‘ecological fallacy’ (Reference Thompson and HigginsThompson & Higgins, 2002).

Finally, the analyses were not controlled for quality criteria. The a priori quality criterion (concealment of allocation) showed little variation, as the majority of studies failed to report this adequately. However, it is not clear whether inadequate reporting of concealment always reflects inadequate methods (Reference Soares, Daniels and KumarSoares et al, 2004; Reference Pildal, Chan and HrobjartssonPildal et al, 2005).

Alternatives to meta-regression in the analysis of complex interventions

The controversy over fidelity to assertive community treatment and outcomes (Reference Fiander, Burns and McHugoFiander et al, 2003) indicates that the identification and measurement of ‘active ingredients’ in mental health interventions has important implications for both research and service provision (Reference Marshall and CreedMarshall & Creed, 2000). It is therefore critical to consider the optimal methods of identifying ‘active ingredients’. Our study has shown that the use of meta-regression is feasible but has limitations. The key issue is how well meta-regression compares with the available alternatives, which include clinical expertise, qualitative work, theoretical models and ‘dismantling’ or ‘factorial’ trials.

Clinical expertise is a potentially useful source of hypotheses, and rigorous qualitative work is ideally suited to capture the complexity of care processes, and is especially useful at exploring the perspectives of stakeholders and illuminating context (Reference Weaver, Tyrer and RitchieWeaver et al, 2003; Reference Marshall, Lockwood and LewisMarshall et al, 2004). However, it is unclear whether patients and professionals can reliably identify ‘active ingredients’. Acknowledgement of the limitations of clinical expertise in identifying causal mechanisms is fundamental to evidence-based medicine, and patients will presumably face many of the same challenges as professionals. Insights from theoretical models are another useful source, but few theoretical models within mental health services research are so well validated that they provide a comprehensive description of ‘active ingredients’, and complex mental health issues such as depression will have many competing theories. Although theory is a necessary aspect of the development of a complex intervention, it will rarely be sufficient.

Dismantling and factorial studies test different combinations of ingredients within a randomised comparison. Relevant examples exist in the collaborative care literature. For example, a recent study compared outcomes in patients randomised to a depression care programme (including systematic follow-up) and systematic follow-up alone. There was no difference in outcomes, suggesting that systematic follow-up is critical (Reference Vergouwen, Bakker and BurgerVergouwen et al, 2005). The advantage of such designs is that randomisation is preserved, allowing causal inference. However, the use of such costly designs to identify ‘active ingredients’ may not always be the optimal use of limited research resources.

Clearly comparisons of the different methods are required, and the intervention development currently being conducted by the authors also includes qualitative work which can be compared with the findings of the meta-regression. It is likely that complex interventions will increasingly be required to improve patient care within mental health, and the evaluation of such interventions raises particular challenges. Although there are potential problems with the application of meta-regression, we conclude that the technique has potential in developing useful insights into the active ingredients in complex interventions in mental health, and thus assist in the design and evaluation of future interventions.

Appendix Initial intervention content variables

Variable Description
Setting What was the geographical location of the study?
Patient population Did the population include all patients with depression, or was it restricted to patients who had depression and were currently taking, or willing to take, antidepressants?
Screening Were patients referred by their primary care providers, or systematically identified (e.g. through screening)?
Primary care providers
    PCP professional group What was the professional background of the primary care providers?
    PCP training and education What training and education did the primary care providers receive?
    PCP EBM guidelines? Did the primary care providers receive an evidence-based guideline?
    PCP T+E time How much time was involved in the training of the primary care providers?
    PCP T+E materials What other materials were used in the training of the primary care providers?
Case managers
    CM professional group What was the professional background of the case managers?
    CM training and education What training and education did the case managers receive?
    CM T+E time How much time was involved in the training of the case managers?
    CM T+E materials What other materials were used in the training of the case managers?
    CM session number planned How many case management sessions were planned?
    CM session frequency planned How often were case management sessions designed to be delivered?
    CM session duration planned What was the planned duration of case management sessions?
    CM total time planned What was the total planned time for the case management?
    CM session number delivered How many case management sessions were delivered?
    CM session frequency delivered How often were case management sessions delivered?
    CM session duration delivered What was the actual duration of case management sessions?
    CM total time delivered What was the actual total time for the case management?
    CM intervention content What was the content of the case management sessions?
    CM intervention patient materials What patient materials were used in the case management session?
    CM liaison with PCP How did the case manager liaise with the primary care provider?
Specialist care
    Specialist professional group What was the professional background of the specialist?
    Specialist training and education What training and education did the specialist receive?
    Specialist liaison with PCP How did the specialist liaise with the primary care provider?
    Specialist liaison with CM How did the specialist liaise with the case manager?

Acknowledgements

The study was implemented in the context of research led by D.R. (funded by the Medical Research Council) in which P.B. and S.G. are grant holders. P.B. and J.F. are funded through the Department of Health, but the views expressed in this paper are those of the authors alone. P.B. thanks John Cape for useful input into the paper.

Footnotes

Declaration of interest

None.

References

Adler, D., Bungay, K., Wilson, I., et al (2004) The impact of a pharmacist intervention on 6 month outcomes in depressed primary care patients. General Hospital Psychiatry, 26, 199209.Google Scholar
Araya, R., Rojas, G., Fritsch, R., et al (2003) Treating depression in primary care in low income women in Santiago, Chile: a randomised controlled trial. Lancet, 361, 9951000.Google Scholar
Bero, L., Eccles, M., Grilli, R., et al (2006) About the Cochrane Collaboration (Art. No. EPOC). Cochrane Review Groups Issue 2. Chichester: Wiley Interscience.Google Scholar
Bijl, D., van Marwijk, W., de Haan, M., et al (2004) Effectiveness of disease management programmes for recognition, diagnosis and treatment of depression in primary care: a review. European Journal of General Practice, 10, 612.CrossRefGoogle Scholar
Blanchard, M., Waterreus, A. & Mann, A. (1995) The effect of primary care nurse intervention upon older people screened as depressed. International Journal of Geriatric Psychiatry, 10, 289298.CrossRefGoogle Scholar
Boudreau, D., Capoccia, K., Sullivan, S., et al (2002) Collaborative care to improve outcomes in major depression. Annals of Pharmacotherapy, 36, 585591.CrossRefGoogle ScholarPubMed
Bower, P. & Gilbody, S. (2005) Managing common mental health disorders in primary care: conceptual models and evidence base. BMJ, 330, 839842.CrossRefGoogle ScholarPubMed
Bradley, F., Kinmonth, A., Mant, D., et al (1999) Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project (SHIP). BMJ, 318, 711715.CrossRefGoogle ScholarPubMed
Brook, O., van Hout, H., Nieuwenhuyse, H., et al (2003a) Effects of coaching by community pharmacists on psychological symptoms of antidepressant users: a randomised controlled trial. European Neuropsychopharmacology, 13, 347354.CrossRefGoogle ScholarPubMed
Brook, O., van Hout, H., Nieuwenhuyse, H., et al (2003b) Impact of coaching by community pharmacists on drug attitude of depressive primary care patients and acceptability to patients: a randomized controlled trial. European Neuropsychopharmacology, 13, 19.CrossRefGoogle ScholarPubMed
Bruce, M., Ten Have, T., Reynolds, C., et al (2004) Reducing suicidal ideation and depressive symptoms in depressed older primary care patients. JAMA, 291, 10811091.CrossRefGoogle ScholarPubMed
Bungay, K., Adler, D., Rogers, W., et al (2004) Description of a clinical pharmacist intervention administered to primary care patients with depression. General Hospital Psychiatry, 26, 210218.CrossRefGoogle ScholarPubMed
Callahan, C., Hendrie, H., Dittus, R., et al (1994) Improving treatment of late life depression in primary care: a randomized clinical trial. Journal of the American Geriatrics Society, 42, 839846.CrossRefGoogle ScholarPubMed
Campbell, M., Fitzpatrick, R., Haines, A., et al (2000) Framework for design and evaluation of complex interventions to improve health. BMJ, 321, 694696.CrossRefGoogle ScholarPubMed
Capoccia, K., Boudreau, D., Blough, D., et al (2004) Randomized trial of pharmacist interventions to improve depression care and outcomes in primary care. American Journal of Health System Pharmacy, 61, 364372.CrossRefGoogle ScholarPubMed
Coleman, E., Grothaus, L., Sandhu, N., et al (1999) Chronic care clinics: a randomized controlled trial of a new model of primary care for frail older adults. Journal of the American Geriatrics Society, 47, 775783.CrossRefGoogle ScholarPubMed
Coyne, J., Brown, G., Datto, C., et al (2001) The benefits of a broader perspective in case-finding for disease management of depression: early lessons from the PROSPECT study. International Journal of Geriatric Psychiatry, 16, 570576.Google Scholar
Datto, C., Thompson, R., Horowitz, D., et al (2003) The pilot study of a telephone disease management program for depression. General Hospital Psychiatry, 25, 169177.CrossRefGoogle ScholarPubMed
Dietrich, A., Oxman, T., Williams, J., et al (2004a) Going to scale: re-engineering systems for primary care treatment of depression. Annals of Family Medicine, 2, 301304.CrossRefGoogle ScholarPubMed
Dietrich, A., Oxman, T., Williams, J., et al (2004b) Re-engineering systems for the treatment of depression in primary care: cluster randomised controlled trial. BMJ, 329, 605.Google Scholar
Donner, A. & Klar, N. (2002) Issues in the metaanalysis of cluster randomized trials. Statistics in Medicine, 21, 29712980.CrossRefGoogle ScholarPubMed
Fiander, M., Burns, T., McHugo, G., et al (2003) Assertive community treatment across the Atlantic: comparison of model fidelity in the UK and USA. British Journal of Psychiatry, 182, 248254.CrossRefGoogle ScholarPubMed
Finley, P., Rens, H., Gess, S., et al (1999) Case management of depression by clinical pharmacists in a primary care setting. Formulary, 34, 864870.Google Scholar
Finley, P., Rens, H., Pont, J., et al (2003) Impact of a collaborative care model on depression in a primary care setting: a randomized controlled trial. Pharmacotherapy, 23, 11751185.CrossRefGoogle Scholar
Furukawa, T., Barbui, C., Cipriani, A., et al (2006) Imputing missing standard deviations in meta-analyses can provide accurate results. Journal of Clinical Epidemiology, 59, 710.CrossRefGoogle ScholarPubMed
Gilbody, S., Whitty, P., Grimshaw, J., et al (2003) Educational and organisational interventions to improve the management of depression in primary care: a systematic review. JAMA, 289, 31453151.CrossRefGoogle ScholarPubMed
Higgins, J. & Thompson, S. (2004) Controlling the risk of spurious findings from meta-regression. Statistics in Medicine, 23, 16631682.CrossRefGoogle ScholarPubMed
Higgins, J., Thompson, S., Deeks, J., et al (2003) Measuring inconsistency in meta-analyses. BMJ, 327, 557560.CrossRefGoogle ScholarPubMed
Hunkeler, E., Meresman, J., Hargreaves, W., et al (2000) Efficacy of nurse telehealth care and peer support in augmenting treatment of depression in primary care. Archives of Family Medicine, 9, 700708.Google Scholar
Katon, W., Von Korff, M., Lin, E., et al (1995) Collaborative management to achieve treatment guidelines: impact on depression in primary care. JAMA, 273, 10261031.CrossRefGoogle ScholarPubMed
Katon, W., Robinson, P., Von Korff, M., et al (1996) A multifaceted intervention to improve treatment of depression in primary care. Archives of General Psychiatry, 53, 924932.Google Scholar
Katon, W., Von Korff, M., Lin, E., et al (1999) Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial. Archives of General Psychiatry, 56, 11091115.CrossRefGoogle ScholarPubMed
Katon, W., Rutter, C., Ludman, E., et al (2001a) A randomized trial of relapse prevention of depression in primary care. Archives of General Psychiatry, 58, 241247.CrossRefGoogle ScholarPubMed
Katon, W., Von Korff, M., Lin, E., et al (2001b) Rethinking practitioner roles in chronic illness: the specialist, primary care physician and the practice nurse. General Hospital Psychiatry, 23, 138144.CrossRefGoogle ScholarPubMed
Katon, W., Von Korff, M., Lin, E., et al (2003) Improving primary care treatment of depression among patients with diabetes mellitus: the design of the pathways study. General Hospital Psychiatry, 25, 158168.CrossRefGoogle ScholarPubMed
Katon, W., Von Korff, M., Lin, E., et al (2004) The Pathways Study: a randomized trial of collaborative care in patients with diabetes and depression. Archives of General Psychiatry, 61, 10421049.Google Scholar
Katzelnick, D., Simon, G., Pearson, S., et al (2000) Randomized trial of a depression management program in high utilizers of medical care. Archives of Family Medicine, 9, 345351.CrossRefGoogle ScholarPubMed
Lambert, P., Sutton, A., Abrams, K., et al (2002) A comparison of summary patient-level covariates in metaregression with individual patient data meta-analysis. Journal of Clinical Epidemiology, 55, 8694.CrossRefGoogle ScholarPubMed
Lipsey, M. & Wilson, D. (2001) Practical Meta-Analysis. Thousand Oaks, CA: Sage.Google Scholar
Loeb, M. (2002) Application of the development stages of a cluster randomized trial to a framework for evaluating complex health interventions. BMC Health Services Research, 2, 13.CrossRefGoogle ScholarPubMed
Mann, A., Blizard, R., Murray, J., et al (1998) An evaluation of practice nurses working with general practitioners to treat people with depression. British Journal of General Practice, 48, 875879.Google Scholar
Marshall, M. & Creed, F. (2000) Assertive community treatment – is it the future of community care in the UK? International Review of Psychiatry, 12, 191196.Google Scholar
Marshall, M., Lockwood, A., Lewis, S., et al (2004) Essential elements of an early intervention service for psychosis: the opinions of expert clinicians. BMC Psychiatry, 4, 17.Google Scholar
Medical Research Council (2000) A Framework for Development and Evaluation of RCTs for Complex Interventions to Improve Health. London: MRC.Google Scholar
Moher, D., Cook, D., Eastwood, S., et al (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Lancet, 354, 18961900.Google Scholar
Moher, D., Schutz, K. & Altman, D. (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet, 354, 18961900.CrossRefGoogle Scholar
Oslin, D., Sayers, S., Ross, J., et al (2003) Disease management for depression and at risk drinking via telephone in an older population of veterans. Psychosomatic Medicine, 65, 931937.CrossRefGoogle Scholar
Peveler, R., George, C., Kinmonth, A., et al (1999) Effect of antidepressant drug counselling and information leaflets on adherence to drug treatment in primary care: randomised controlled trial. BMJ, 319, 612615.CrossRefGoogle ScholarPubMed
Pildal, J., Chan, A., Hrobjartsson, A., et al (2005) Comparisons of descriptions of allocation concealment in trial protocols and published reports: cohort study. BMJ, 330, 1049.CrossRefGoogle ScholarPubMed
Pyne, J., Smith, J., Fortney, J., et al (2003) Cost-effectiveness of aprimary care intervention for depressed females. Journal of Affective Disorders, 74, 2332.Google Scholar
Rickles, N. (2003) A Randomized Controlled Study Evaluating the Impact of an Antidepressant Monitoring Program on Consumer Outcomes. PhD Thesis, University of Wisconsin, Madison.Google Scholar
Rost, K., Nutting, P., Smith, J., et al (2000) Designing and implementing a primary care intervention trial to improve the quality and outcome of care for major depression. General Hospital Psychiatry, 22, 6677.CrossRefGoogle ScholarPubMed
Rost, K., Nutting, P., Smith, J., et al (2001) Improving depression out comes in community primary care practice: a randomized trial of the QuEST intervention. Journal of General Internal Medicine, 16, 143149.Google Scholar
Schoenbaum, M., Unutzer, J., Sherbourne, C., et al (2001) Cost-effectiveness of practice-initiated quality improvement for depression: results of a randomized controlled trial. JAMA, 286, 13251330.Google Scholar
Schulz, K. & Grimes, D. (2002) Allocation concealment in randomised trials: defending against deciphering. Lancet, 359, 614618.CrossRefGoogle ScholarPubMed
Sherbourne, C., Wells, K., Duan, N., et al (2001) Long-term effectiveness of disseminating quality improvement for depression in primary care. Archives of General Psychiatry, 58, 696703.Google Scholar
Simon, G. & Von Korff, M. (1995) Recognition, management and outcomes of depression in primary care. Archives of Family Medicine, 4, 99105.Google Scholar
Simon, G., Von Korff, M., Rutter, C., et al (2000) Randomised trial of monitoring, feedback and management of care by telephone to improve treatment of depression in primary care. BMJ, 320, 550554.CrossRefGoogle ScholarPubMed
Simon, G., Katon, W., Von Korff, M., et al (2001a) Cost-effectiveness of a collaborative care program for primary care patients with persistent depression. American Journal of Psychiatry, 158, 16381644.Google Scholar
Simon, G., Manning, W., Katzelnick, D., et al (2001b) Cost-effectiveness of systematic depression treatment for high utilisers of general medical care. Archives of General Psychiatry, 58, 181187.Google Scholar
Simon, G., Von Korff, M., Ludman, E., et al (2002) Cost-effectiveness of a program to prevent depression relapse in primary care. Medical Care, 40, 941950.Google Scholar
Simon, G., Ludman, E., Tutty, S., et al (2004) Telephone psychotherapy and telephone care management for primary care patients starting antidepressant treatment: a randomized controlled trial. JAMA, 292, 935942.Google Scholar
Soares, H., Daniels, S., Kumar, A., et al (2004) Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ, 328, 2225.Google Scholar
Sutton, A., Abrams, K., Jones, D., et al (1998) Systematic reviews of trials and other studies. Health Technology Assessment, 2, 1276.Google Scholar
Swindle, R., Rao, J., Helmy, A., et al (2003) Integrating clinical nurse specialists into the treatment of primary care patients with depression. International Journal of Psychiatry in Medicine, 33, 1737.Google Scholar
Tabachnick, B. & Fidell, L. (2001) Using Multivariate Statistics. Boston: Allyn & Bacon.Google Scholar
Thompson, S. & Higgins, J. (2002) How should metaregression analyses be undertaken and interpreted? Statistics in Medicine, 21, 15591573.Google Scholar
Unutzer, J., Katon, W., Williams, J., et al (2001a) Improving primary care for depression in late life: the design of a multicenter randomized trial. Medical Care, 39, 785799.Google Scholar
Unutzer, J., Rubenstein, L., Katon, W., et al (2001b) Two-year effects of quality improvement programs on medication management for depression. Archives of General Psychiatry, 58, 935942.Google Scholar
Unutzer, J., Katon, W., Callahan, C., et al (2003) Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA, 288, 28362845.Google Scholar
Vergouwen, A., Bakker, A., Burger, H., et al (2005) A cluster randomized trial comparing two interventions to improve treatment of major depression in primary care. Psychological Medicine, 35, 2535.Google Scholar
Von Korff, M. & Goldberg, D. (2001) Improving outcomes in depression. BMJ, 323, 948949.Google Scholar
Von Korff, M., Katon, W., Bush, T., et al (1998) Treatment costs, cost offset, and cost effectiveness of collaborative management of depression. Psychosomatic Medicine, 60, 143149.Google Scholar
Weaver, T., Tyrer, P., Ritchie, J., et al (2003) Assessing the value of assertive outreach: qualitative study of process and outcome generation in the UK700 trial. British Journal of Psychiatry, 183, 437445.Google Scholar
Weingarten, S., Henning, J., Badamgarav, E., et al (2002) Interventions used in disease management programmes for patients with chronic illnesses – which ones work? Meta-analysis of published reports. BMJ, 325, 925.Google Scholar
Wells, K., Sherbourne, C., Schoenbaum, M., et al (2000) Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial. JAMA, 283, 212220.CrossRefGoogle ScholarPubMed
Wells, K., Sherbourne, C., Schoenbaum, M., et al (2004) Five year impact of quality improvement for depression: results of a group-level randomized controlled trial. Archives of General Psychiatry, 61, 378386.CrossRefGoogle ScholarPubMed
Whitty, P. & Gilbody, S. (2005) NICE, but will they help people with depression? The new National Institute for Clinical Excellence depression guidelines. British Journal of Psychiatry, 186, 177178.Google Scholar
Whooley, M., Stone, B. & Soghikian, K. (2000) Randomized trial of case-finding for depression in elderly primary care patients. Journal of General Internal Medicine, 15, 293300.Google Scholar
Wilkinson, G., Allen, P., Marshall, E., et al (1993) The role of the practice nurse in the management of depression in general practice: treatment adherence to antidepressant medication. Psychological Medicine, 23, 229237.CrossRefGoogle ScholarPubMed
Figure 0

Fig. 1 QUOROM (Quality of Reporting Meta-analyses) flow diagram.

Figure 1

Table 1 Studies included in the review

Figure 2

Table 2 Intervention content variables (n=34)

Figure 3

Fig. 2 Meta-analysis of antidepressant use. Note: the Wells (2000) and Simon (2004) studies involved two intervention groups compared against a single control; to avoid double-counting the controls, the sample size and event rate in the control were divided by 2. The Rost 2001 study data are only available analysed in two subgroups, rather than as an overall analysis; in our analysis these subgroups were treated as separate comparisons.

Figure 4

Fig. 3 Meta-analysis of depressive symptoms (see note for Fig. 2).

Figure 5

Table 3 Univariate analysis of associations between intervention content variables and antidepressant use

Figure 6

Table 4 Univariate analysis of associations between intervention content variables and depressive symptoms

Figure 7

Fig. 4 Relationship between antidepressant use outcomes and depressive symptoms outcomes.

Figure 8

Appendix Initial intervention content variables

Submit a response

eLetters

No eLetters have been published for this article.