Comparative randomised trial of online cognitive–behavioural therapy and an information website for depression: 12-month outcomes

Andrew Mackinnon , Kathleen M. Griffiths , Helen Christensen

Abstract

Background

An earlier paper reported positive outcomes immediately following access to a cognitive–behavioural therapy (CBT) internet intervention and a depression information website.

Aims

To determine 6-month and 12-month outcomes of the interventions relative to a placebo control condition.

Method

Participants allocated to the depression information website condition, the CBT internet intervention and the placebo control condition were followed up at 6 and 12 months by survey. The primary outcome measure was the Center for Epidemiologic Studies – Depression (CES–D) scale. Of 525 participants recruited, 79% completed their assigned intervention and 60% were retained at 12-month follow-up. Attrition was handled using mixed-model repeated-measures ANOVA.

Results

Both the CBT site and depression information sites were associated with statistically significant benefits in CES–D score reduction compared with controls at post-test. At 6-months follow-up, the reduction seen for the people allocated to the CBT arm remained significant, whereas that for the depression information website arm was not. At 12 months, both active interventions were statistically significantly superior to the control condition.

Conclusions

There is some evidence that benefits of these brief internet interventions are retained beyond their endpoint. The impact of human support provided by interviewers in the trial must be considered when comparing these outcomes to routine use of both internet interventions.

Randomised controlled trials demonstrate that internet-delivered brief cognitive–behavioural therapy (CBT) may be effective in reducing depression.13 Whether gains observed at the end of internet intervention programmes are maintained over longer periods has not yet been definitively established. Only one internet depression programme has reported any follow-up data; however, the longest follow-up period of this study was only 16 weeks.2 Evidence regarding whether internet CBT programmes produce similar long-term improvements to those associated with face-to-face CBT interventions is equivocal with studies reporting conflicting results.4 Attrition in both trial and routine use of internet interventions is high.5,6 Thus, an important issue in follow-up trials is that of the appropriate management of data missing because of participant withdrawal. The widely used approaches of completer analysis and last observation carried forward (LOCF) have well-established deficiencies in reaching reliable conclusions regarding the effectiveness of treatments.710

The aims of the present study were to examine whether the benefits of internet-delivered brief psychoeducation (BluePages) and CBT (MoodGYM) observed at the trial end-point1 persisted at 6 and 12 months relative to a credible placebo-controlled condition and to determine whether post-test differences found using LOCF were robust to more defensible approaches when dealing with participant withdrawal.

Method

Participants

The original trial recruited 525 participants with elevated depression scores from the community. Participants were randomised to three conditions: MoodGYM, a modularised internet CBT programme (http://moodgym.anu.edu.au); BluePages, a web-based education site which provides information about a range of psychological, medical and alternative treatments, and recommends those supported by scientific evidence (http://bluepages.anu.edu.au); and an attention control condition, consisting of targeted questions about the participant's lifestyle. Recruitment and retention details are shown in Fig. 1. All participants were telephoned by lay interviewers weekly during the 6-week intervention period to report on progress and encourage adherence. Participants were excluded if they were already undertaking CBT, did not have access to the internet or had language difficulties that were likely to interfere with their understanding of the interventions.

Fig. 1

Flow of participants through trial. a. Participants who were not assessed on a particular measurement occasion but who were assessed subsequently.

Procedures

The original trial was registered as ISRCTN77824516. Participants completed questionnaires delivered by mail at screening, pre-test, post-test and 6 and 12 months after the trial. Those in the CBT internet intervention and depression information website conditions were provided with a log in identification number and a manual containing information about MoodGYM or BluePages respectively. MoodGYM comprises five interactive modules, including cognitive restructuring, pleasant activities and assertiveness training, problem-solving and downloadable relaxation sessions. The modules were made available sequentially weekly. All interactive exercises were stored in a personalised workbook, and at the conclusion of each module an individualised feedback form was available for print out. BluePages provides information about the symptoms of depression and its diagnosis, includes evidence-based information about medical, psychological, alternative and lifestyle interventions, provides information about depression resources and sources of help, and includes a focused depression search engine that permits the individual to search across sites on depression worldwide. Each week participants were directed to one of five sections of the website. Interviewers maintained weekly telephone contact with participants in all conditions over the period of the intervention (a total of six 10-min contacts). Participants were telephoned up to three times to remind them to complete post-intervention questionnaires. There was no other contact with participants over the 12-month period following the intervention period, although the websites were still accessible to participants.

Measures

The primary outcome measure was the 20-item Center for Epidemiologic Studies – Depression (CES–D) scale (range 0 to 60). Higher scores represent greater psychological distress. Scores 16 or higher are recognised as indicating the presence of clinically significant depression.11 Demographic information was also collected, including age, gender, marital status, years of education and previous history of depression.

Statistical analysis

All analyses were undertaken using mixed-model repeated-measures (MMRM) ANOVA with measurement occasion as a within-groups factor and intervention as a between-groups factor. Relationships between observations at different occasions were modelled as an unstructured covariance matrix. Planned contrasts compared changes from baseline under each intervention at post-test, 6- and 12-month follow-up.

These analyses differ from the previously published analysis of post-test data in which observations missing at post-test were imputed by assuming that participants who withdrew were unchanged.1 Mixed-model repeated measures uses all available data and does not involve any substitution of missing values with supposed or estimated values. The assumptions underlying MMRM allow `missingness' to be related to observed variables in the analysis but not to unobserved (missing) values. This is referred to as `missing at random.' In addition, MMRM does not impose the constraint inherent in conventional repeated-measures ANOVA that observations have the same dispersion on all occasions of measurement and are equally correlated. Rather, these attributes may be unconstrained, as in the current paper, or may be modelled. Mixed-model repeated measures is recognised as an emerging, principled method of analysing trial data.12 These analyses were undertaken using the MIXED procedure in SAS release 9.1 for Windows.13

Results

Participant withdrawal

Table 1 and Fig. 1 show participant retention in the three conditions. Nearly all missing data were due to participant withdrawal (i.e. leaving the study and not re-entering). A very small number of participants had missing observations at post-test or 6-month follow-up, but provided responses at a later stage.

View this table:
Table 1

CES-D observed means, standard deviations, minima and maxima, and sample size for each intervention at pre-test, post-test, 6- and 12-months follow-up

At post-test (i.e. withdrawal before or during the intervention period), withdrawal was not significantly related to age or gender. Those in the active interventions were more likely to withdraw than controls (OR=4.70, 95% CI 2.11–10.49 and OR=2.45, 95% CI 1.04–5.77 for the CBT invention and depression information website respectively). However, in the control group, participants with higher pre-test CES–D values were more likely to withdraw (OR per CES–D unit 1.07, 95% CI 1.02–1.11). This relationship did not exist with the internet interventions. Subsequent loss to follow-up was unrelated to age or to treatment condition. Males were more likely to be lost to follow-up, but this difference was significant only at 12-months follow-up (OR=2.52, 95% CI 1.59–3.98). Total withdrawal during treatment and subsequent loss to follow-up at 12 months from baseline were 48%, 35% and 30% for the CBT internet intervention, depression information website and control condition respectively (Fig. 1).

Outcome

Table 1 and Fig. 2 display mean CES–D scores on each measurement occasion as a function of the three conditions. The overall interaction of condition and occasion was significant (F[6,379.2]=2.90, P<0.01) indicating different patterns of change of the interventions over time. In order to investigate differences between conditions at each occasion of measurement, a series of planned contrasts were computed. For post-test, 6- and 12-month follow-up, differences in change from baseline between each of the three conditions were tested (Table 2). Differences between each active intervention and the control condition were of primary interest but differences between the active interventions were also explored.

View this table:
Table 2

Mixed-model repeated-measures ANOVAs of CES–D scores at post-test, 6- and 12-month follow-up with contrasts between interventions on each occasion of measurement compared with baseline

Fig. 2

Estimated marginal means for CES–D scores estimated under occasion × intervention model. CBT, cognitive–behavioural therapy; CES–D, Center for Epidemiologic Studies – Depression scale.

Post-test

Analyses of post-test outcome demonstrated that both the CBT internet intervention and depression information website conditions reduced depression symptoms to a greater extent than did the control condition (F[1,447.0]=14.79, P<0.001 and F[1,439.3]=8.13, P=0.005 respectively). Participants in the CBT internet intervention would be expected to achieve a reduction of 4.3 points on the CES–D more than those in the control group (effect size 0.38). Similarly, participants assigned to the depression information website would be expected to achieve a reduction of 3.2 points on the CES–D more than those in the control group (effect size 0.29). No statistically significant difference was found between the CBT internet intervention and the depression information website conditions on this or any subsequent occasion.

These results buttress previously reported results obtained using LOCF at post-test.1 Mean decline from baseline for the CBT internet intervention and the depression information website estimated by carrying forward observations was approximately 1 point less than found in current analyses, a lower estimate of benefit compared with placebo. While this might appear to support the use of LOCF in yielding conservative estimates of effectiveness, use of this technique would have inflated the statistical significance of some of the contrasts reported in this study.

Six-month follow-up

The relative advantage of the active interventions compared with the control group declined by approximately 1 point on the CES–D by 6 months after post-test assessments. Statistical tests showed that the benefits of the CBT internet intervention remained significant (F[1,405.5]=4.49, P=0.035), while that of the depression information website did not (F[1,396.6]=2.80, P=0.095). Effect sizes were reduced to 0.27 and 0.21 respectively.

Twelve-month follow-up

Twelve months after post-test, the pattern of decline of CES–D scores in all conditions was maintained. There was a slight increase in the benefits of the active interventions compared with controls so that both attained statistical significance (F[1,388.7]=4.09, P=0.044 and F[1,376.7]=5.11, P=0.024 respectively). Effect sizes were 0.27 and 0.29 respectively.

Outcomes in participants with clinically significant depression at pre-test

In keeping with its pragmatic community-sample focus, all participants were retained in the study and in the primary analysis regardless of whether they had substantial decline in symptoms from recruitment screening to programme commencement. Participants with low pre-test scores on the CES–D could show little, if any, benefit from any of the interventions (although they had potential to relapse or to improve on secondary outcomes such as stigma or mental health literacy). Pharmaceutical trials frequently exclude such participants. In order to evaluate the effects of the two interventions in individuals with significant depression at pre-test, the analyses described were repeated including only the 369 participants with pre-test CES–D scores of 16 or greater.

Results for this subgroup showed the same overall pattern as the full sample. At post-test, both interventions were significantly better than the control condition (F[1,307.3]=13.75, P<0.001 and F[1,302.7]=9.86, P=0.002 respectively) and, as might be expected, conferred a benefit approximately 1 point greater on the CES–D than in the whole sample. At 6-month follow-up the internet CBT programme remained significant, delivering a benefit of 4.2 on the CES–D compared with controls, while the depression website was not (F[1,275.6]=6.23, P=0.013 and F[1,270.6]=2.25, P=0.134 respectively). At 12-month follow-up, the internet CBT programme was not significant while the depression website maintained statistical significance as found in the full sample (F[1,266.1]=2.03, P=0.156 and F[1,256.5]=4.22, P=0.042 respectively). The benefits of each intervention compared with controls were comparable to those found in the full sample. No comparisons between the two active interventions were statistically significant.

Discussion

Participants in all conditions in this study continued to show reductions in depression symptomatology after post-test. Analysis of follow-up data provides some evidence that the reduction in CES–D scores in both the depression information website and the CBT internet intervention conditions was greater than for controls and persisted over the 12-month period post-test. The study confirms the conclusions regarding the effectiveness of these programmes at the end of the interventions, doing so using a defensible approach to participant withdrawal rather than relying on the unsupportable LOCF method. Similar conclusions can be drawn for the subgroup of participants meeting screening criteria for clinically significant depression when commencing the interventions.

It is important that differences between the interventions as tested under trial conditions and their normal implementation are appreciated. In order to encourage adherence, collect response data and equate with human intervention in the control conditions, participants were contacted on six occasions at weekly intervals. Interviewers directed participants to the next phase of each intervention. Human contact is not a feature of routine use of either the CBT internet or psychoeducational website intervention. An automated trial of MoodGYM with no human intervention reported higher rates of withdrawal than the current study but comparable levels of effectiveness at the end of the intervention.14 Thus the effectiveness of this intervention cannot to be attributed to a synergy of the internet CBT programme and human support in its use. However, human support through telephone contact is highly likely to be responsible for participant retention.

The effects of two active interventions on CES–D scores were similar throughout the trial, although it can be presumed that their mechanisms of action differed. Cognitive–behavioural therapy is an established treatment for depression, whose efficacy has been demonstrated in a variety of modes of delivery. Mechanisms by which psychoeducation for depression, as delivered by BluePages, is effective have not been extensively explored. It might be supposed that participants exposed to this site may be more likely to take up evidenced-based treatment. However, at post-test in this trial, we found that these sites were associated with a self-reported reduction in ineffective and possibly detrimental interventions rather than in increased uptake of evidence-based treatment.15

The findings support growing evidence that brief interventions, including those that are primarily information-based, may have lasting effects on symptomatology. This outcome is consistent with comparable `low intervention' programmes. Brief interventions are increasingly recognised as providing positive treatment outcomes. There is some evidence for the effectiveness of brief `passive' educational interventions in producing positive mental health outcomes16 but other studies have been less encouraging.17,18

There are a number of challenges for research into the effectiveness of internet-based interventions for depression. These include the comparison of internet CBT and information provision against the gold standard of antidepressant use – this would provide a convincing test of the potential benefits of this novel approach to therapy delivery. A clinical trial in a medical setting could provide a test examination of this issue. However, community-focused interventions differ from clinical treatment research in that they aim to provide self-help or deliverable educational programmes that can be accessed by individuals who cannot, or who choose not to, seek help from professionals. Therefore, research should seek to delineate the circumstances and presentations under which these self-managed, early intervention programmes may be used with equivalent or better outcomes than conventional psychotherapies or pharmacotherapies and, equally important, to determine the circumstances in which they should be avoided. There is also a need to determine which aspects of the interventions are associated with better outcomes. This is particularly critical for internet approaches as, in normal settings, many users persevere with websites for only brief periods. Optimising information presentation and therapy delivery is thus a high priority, as is making sites responsive to the characteristics and needs of individuals.

The limitations of this research need to be articulated. Care should be exercised in immediately accepting these analyses as providing evidence of reliable longer-term benefits. While the estimated benefits of internet CBT and the depression website relative to control were comparatively stable, statistical tests of any advantage beyond post-test hovered around the conventional cutoff point for significance. This reflects increasing uncertainty surrounding these estimates owing to withdrawal after the end of the programmes. The results presented might be best interpreted as providing `some positive evidence' of ongoing benefit rather than as overwhelming support. As the statistical significance of the outcomes of both interventions becomes equivocal over time, the utility of brief booster programmes warrants consideration and research.

This study provides some evidence that the effects on depressive symptomatology of an internet-based CBT and an information-based intervention persist beyond those recorded at treatment cessation. At 6- and 12-month follow-up the size of the benefit of these interventions compared with the control condition were relatively stable, albeit that an equivocal pattern of statistical significance was present. There were no significant differences between the two active treatments over the course of the study. Further clinical research should compare these intervention modalities with current conventional treatments, including face-to-face psychotherapy and antidepressant medication. Further community research should investigate the circumstances under which these interventions prevent or reduce symptoms, reduce stigma, increase help-seeking, maintain treatment participation, and offer knowledge and self-help that is sought and of value to individuals.

Acknowledgments

This study was supported by the Australian National Health and Medical Research Council grant No. 366781 to H.C. and grant No. 332950 to A.M.

  • Received October 7, 2006.
  • Revision received August 29, 2007.
  • Accepted October 24, 2007.

References

View Abstract