Computerised cognitive–behavioural therapy (CCBT) might offer a solution to the current undertreatment of depression.
To determine the clinical effectiveness of online, unsupported CCBT for depression in primary care.
Three hundred and three people with depression were randomly allocated to one of three groups: Colour Your Life; treatment as usual (TAU) by a general practitioner; or Colour Your Life and TAU combined. Colour Your Life is an online, multimedia, interactive CCBT programme. No assistance was offered. We had a 6-month follow-up period.
No significant differences in outcome between the three interventions were found in the intention-to-treat and per protocol analyses.
Online, unsupported CCBT did not outperform usual care, and the combination of both did not have additional effects. Decrease in depressive symptoms in people with moderate to severe depression was moderate in all three interventions. Online CCBT without support is not beneficial for all individuals with depression.
Although cognitive–behavioural therapy (CBT) is an effective treatment for depression1,2 many people with depression in primary care remain untreated.3 An effective, acceptable and feasible solution might be computerised CBT (CCBT).4 Computerised cognitive–behavioural therapy can vary greatly in terms of technologies used and amount of additional support. To our knowledge, only one study so far investigated the efficacy of CCBT for depression in primary care.5 It was shown that CCBT (delivered on a computer in the general practice) is more effective than usual general practitioner (GP) care. Nevertheless, the effectiveness of CCBT via the internet in primary care remains to be evaluated as well as the effects of CCBT combined with usual GP care. In a randomised trial, we addressed these issues by examining the effectiveness of online, unsupported CCBT (the Colour Your Life programme) for depression in primary care. In another study, this intervention was equally effective as group CBT in people over 50 years old with subthreshold depression.6 We hypothesised that CCBT would be superior to usual GP care, and that the combination of CCBT and usual GP care would be more effective than CCBT alone. The Medical and Ethical Committee of Maastricht University approved the study protocol. The study is registered at The Netherlands Trial Register, part of the Dutch Cochrane Centre (ISRCTN47481236).
Study population and recruitment
Participants were recruited from the general population by means of a large-scale internet-based screening in the south of The Netherlands. A random selection of individuals was sent an invitation letter to complete an online screening questionnaire. Potentially eligible participants were invited to visit the research centre to assess final eligibility. Participants were eligible if they met the following criteria: age 18 to 65; access to the internet at home; at least mild to moderate depressive complaints (Beck Depression Inventory II (BDI–II)7 score ≥16); duration of depressive complaints 3 months or more; no current psychological treatment for depression; no continuous antidepressant treatment for at least 3 months prior to entry; fluent in Dutch language; no alcohol and/or drug dependence; and no severe psychiatric comorbidity. To determine DSM–III–R8 Axis I diagnoses the computerised Composite International Diagnostic Interview (CIDI–auto)9 was used. Full details of the study method have been described elsewhere.10
After informed consent was obtained, participants were randomly allocated to one of three groups: online CCBT without support (the Colour Your Life programme); treatment as usual (TAU) by a GP; or online, unsupported CCBT and TAU combined. Baseline assessment took place at the research centre before randomisation on a computer. All follow-up assessments took place at home via the internet. Preceding an upcoming assessment point participants received an email alert. Individuals received financial compensation for internet use (e25).
The CCBT programme, named Colour Your Life,11 is an online, multimedia, interactive computer program for depression. Colour Your Life is based on the principles of CBT and on the Dutch version of ‘The Coping with Depression Course’ of Lewinsohn.12,13 It consists of eight 30-min sessions and a ninth booster session, although the duration of sessions can vary among users. At the end of each session homework assignments are given. Participants were advised to complete one session per week. Participants were given log-in codes by the researchers and they accessed CCBT at home. No assistance was offered. Colour Your Life was originally developed for people over 50 years old6 and was adapted for an adult population (18 to 65 years) for the current study.
Treatment as usual was delivered by the participant’s own GP who was advised to follow guidelines from the Dutch College of General Practitioners.14 Treatment as usual can include 4 to 5 consultations, held every second week, and antidepressant treatment if indicated.
For each person, it was assessed whether the interventions received were according to ‘protocol’. Adherence to CCBT was defined as being exposed to all essential steps of the intervention, which was operationalised as having completed five or more sessions. Treatment as usual adherence was defined as receiving at least four consultations or antidepressant medication. Computerised cognitive–behavioural therapy plus TAU adherence was defined as a combination of both.
Secondary outcomes included the following measures. General psychological distress was measured with the Symptom Checklist 90 (SCL–90). Scores range from 90 (no distress) to 450 (very severe distress).17,18 The Work and Social Adjustment Scale was used to assess impairment in social functioning attributable to depression.19 A high score is indicative of severe impairment (score range 0–40). The 36-item short-form Health Survey (SF–36) was used to assess specific features of quality of life.20–22 We used the two most relevant subscales: role limitations caused by emotional problems and general mental health. High scores indicate high levels of quality of life (score range 0–100). The intensity of dysfunctional beliefs was assessed with the Dysfunctional Attitude Scale form A.23 We used a 17-item version,24 with a score range of 17 to 119. The higher the score, the more dysfunctional attitudes an individual reports.
Additional measures included a healthcare use questionnaire that we developed for the study, which measured self-reported use of GP care, antidepressant medication and specialist care.
All outcomes were assessed at baseline and at 3-month follow-up. In addition, the BDI–II and the Dysfunctional Attitude Scale form A–17 were also assessed at 2 months and 6 months. The healthcare use questionnaire was assessed monthly.
Power calculations were based on elementary head-to-head comparisons of CCBT v. usual care and CCBT v. combination treatment (t-test). We calculated that a sample size of 84 participants per group was needed to detect a change score of 5 (s.d. = 5.25) on the BDI–II (power 90%, α = 0.05). Adjusting for potential withdrawal from the study (20%), we estimated that 100 participants per group were needed.
Preliminary tests for distribution and outliers were performed. Skewness and kurtosis did not indicate substantial deviations from normality for all outcomes. The analyses were based on the intention-to-treat principle (i.e. those who provided follow-up data irrespective of treatment adherence). Only intermittent missing data were imputed (n = 5) by calculating the mean of the values of a previous and a subsequent time point. Missing data as a result of loss to follow-up were not replaced by imputed values. We tested all effects at the P<0.05 level (two-tailed). All analyses were carried out using SPSS (version 15.0.1 for Windows).
First, to test the main hypotheses, repeated-measures analyses of variance (ANOVAs) were performed. In case of significant time×group interactions, contrasts were conducted comparing changes from baseline to each subsequent time point for each pair of groups separately. Then, we computed improvement effect sizes for BDI–II scores for each time point according to Cohen’s d statistic,25 defined as (Mt0–Mtk)/sd(Mt0–Mtk). Between-group effect sizes were determined by calculating the difference in improvement effect sizes between two groups. Next, we determined the proportion of participants who made clinically meaningful changes on the BDI–II using the methodology of Jacobson and Truax.26 This approach is based on two components: reliable change, i.e. a decrease of 9 points; and clinically significant change, i.e. a score below 12. Chi-squared tests were used to test the frequency differences in reliable change, in clinically significant change, and in reliable change plus clinically significant change between the three groups. Based on the reliable change plus clinically significant change proportions, the number needed to treat was calculated.27 Finally, we conducted per protocol analyses for treatment adherers only, using repeated-measures ANOVAs for the BDI–II scores.
Figure 1 presents the flow of the participants. Recruitment took place from December 2005 to June 2007. Follow-up ended in December 2007. Three hundred and three people with depression were enrolled in the study. At 6-month follow-up, data were available for 275 participants (attrition rate 9.2%). Reasons for loss to follow-up were: too time-consuming (n = 8), personal circumstances or medical illness other than a mental disorder (n = 5) and no reason was given (n = 15). There seemed to be no baseline differences between participants who completed all assessments and those who were lost to follow-up (lowest P = 0.10). Baseline characteristics of all participants are shown in Table 1, stratified according to intervention group. Randomisation was successful; the characteristics are fairly similar in all three groups, although gender is not equally distributed across the groups.
Outcome of the interventions
Table 2 depicts the means and standard deviations of the clinical outcomes at follow-up. There were no significant group×time interactions on the primary outcome measure as well as most secondary outcomes (all P>0.29). A significant interaction effect was only found for the Work and Social Adjustment Scale (P = 0.03). Contrasts revealed that CCBT plus TAU resulted in a significantly greater reduction on the Work and Social Adjustment Scale compared with CCBT alone (F1,188 = 5.63, P = 0.02) and TAU alone (F1,188 = 4.35, P = 0.04). Since gender could have confounded the outcomes, we corrected for this in ancillary analyses by adding gender to the model as a between-group factor. This did not result in a significant outcome for the BDI–II (group×time: F6,536 = 0.94, P = 0.47) nor was there a significant interaction between group and gender (F2,270 = 0.47, P = 0.63). We therefore omitted this correction from all further analyses.
Regarding the magnitude of the effects, all three groups had medium to large improvement effect sizes, whereas between-group effect sizes were trivial (Table 3). Small negative between-group effect sizes were also found, indicating an effect in the opposite direction to that hypothesised.
Reliable and clinical change
Table 4 shows the proportion of participants in each group who showed a reliable change, a clinically significant change and both for each time point. There were no significant differences between the three groups (all P>0.12). In Fig. 2 the percentage of participants with both a reliable change and a clinically significant change are graphically shown. We calculated the number needed to treat (NNT) with TAU as the reference group. At 6 months the NNT were 72 and 25 respectively for CCBT and CCBT plus TAU.
In Table 5, treatment adherence and the use of healthcare services are shown for each group during the 6 months after baseline. As was expected, there were some significant differences between the three groups (P<0.05). More participants in TAU and CCBT plus TAU visited their GP for depressive complaints compared with those in the CCBT alone group. Concerning the use of CCBT, more participants in the CCBT plus TAU group completed the last session compared with the CCBT alone group. Furthermore, more individuals in the TAU group received specialist mental healthcare than in the other groups, and they received it earlier. Finally, only a small proportion of participants in each group received an adequate dosage of treatment (TAU: at least four consultations or prescribed antidepressants; CCBT: at least five sessions).
Per protocol analyses
First, we compared the outcomes on the BDI–II only for those who adhered to the treatment. Group×time interaction was not statistically significant (F6,148 = 0.85, P = 0.53). We repeated these analyses using a less strict definition of adherence in the CCBT plus TAU group, i.e. adherence was defined as either an adequate dosage of CCBT or an adequate dosage of TAU (n = 59). Again no significant interaction effect was obtained (F6,242 = 1.67, P = 0.13).
Next, we compared the BDI–II scores between those who adhered to the treatment protocol and those who did not within each intervention group. None of these within-group tests revealed statistically significant adherence×time interaction effects (CCBT: F3,87 = 0.59, P = 0.62; TAU: F3,86 = 2.03, P = 0.12; CCBT+TAU: F3,89 = 0.76, P = 0.52). When we used the less strict definition of adherence in the CCBT plus TAU group, a small effect, albeit clinically negligible, for adherence×time interaction was obtained (F3,88 = 2.70, P = 0.05). Contrasts revealed no differences between those who adhered to the treatment and those who did not for each change score (P>0.30).
Since initial depressive severity was high (as can be concluded from the baseline BDI–II scores in Table 2), we conducted ancillary subgroup analyses. First, subgroups were formed according to the initial median score on the BDI–II for the total group; low scores = BDI<27, and high scores = BDI≥27. No statistically significant group×time interaction effects were found in each subgroup (BDI<27: F6,262 = 1.27, P = 0.27; BDI≥27: F6,270 = 1.15, P = 0.34). Second, subgroups were formed based on the presence or absence of a major depressive episode. In the no major depressive episode subgroup, the group×time interaction was not statistically significant (F6,84 = 1.02, P = 0.42), whereas a significant interaction was obtained in the major depressive episode group (F6,448 = 2.25, P = 0.04). Contrasts revealed a significant effect only for change from baseline to 3-month follow-up in favour of the CCBT plus TAU group compared with the TAU alone group (F1,161 = 6.03, P = 0.02).
In contrast to our hypotheses, the findings suggest that there are no meaningful differences between CCBT, TAU, and CCBT plus TAU combined during 6 months of follow-up in terms of depressive severity, quality of life, dysfunctional beliefs and general psychological distress. Although we found medium improvement effect sizes in depressive severity for all interventions, the between-group effect sizes were rather small or even negative. Moreover, per protocol analysis between and within groups revealed no differences between the interventions either. Finally, we found that treatment adherence was low in all interventions. It should be noted that at 3 months, a significant effect was found for social functioning in favour of the combined treatment. We are reluctant to interpret this effect given the high number of statistical tests we performed.
To be able to compare the effects of CCBT in our study with previous ones, we calculated the usual Cohen’s d (i.e. dividing the pre–post difference by the pooled standard deviation) for the 2-month follow-up period. Our improvement effect size for Colour Your Life (d = 0.84) was smaller than in the previous study on Colour Your Life (d = 1.00)6 and than in the previous primary care study5 (d = 1.27). The between-group effect size for Colour Your Life relative to TAU was smaller (d = 0.20) than found for Colour Your Life in a previous study6 (d = 0.55), CCBT in primary care5 (d = 0.65), online CCBT with support (d = 1.05),28 and CCBT with shortened face-to-face therapy (d = 1.14).29 Comparable effect sizes were found for bibliotherapy with minimal contact in primary care (d = 0.18),30 and online CCBT without support (d = 0.22).31 It should be noted here that most of these studies used other comparison groups, making it difficult to directly compare the effects.
There are several reasons that may explain the small effects in our study. First of all, one might argue that Colour Your Life itself was less effective than other CCBT programmes. Results from the previous study using Colour Your Life seemed very promising for subthreshold depression in people over 50 years old.6,32 However, in that study, baseline assessment of the primary outcome was conducted after randomisation, which could have violated the results.
Second, the way CCBT was offered might explain the different outcomes. Various technologies can be used, ranging from the telephone to CD–ROMs, hi-tech computers, palmtops and the internet.33–36 The latest interventions use highly sophisticated computer systems, which might stimulate and improve engagement and motivation.37 Although Colour Your Life makes full use of the current technologies, this might not have been enough to stimulate treatment adherence. We think that the lack of clinician support might account for the poor adherence and response to online, unsupported CCBT that we observed. Similar studies also showed a lack of response,31,38,39 whereas studies that offered some form of support reported more treatment adherence and larger effects with online CCBT for several psychological disorders.28,33,40–42 Poor treatment adherence in our study might thus have masked potential effects of the interventions, but our ancillary per protocol analyses did not reveal differences between the treatment groups either. We did observe a small trend in favour of the group receiving CCBT plus TAU as opposed to both single therapies for reliable change and clinically significant change, and in favour of the subgroup with a major depressive episode. However, our combined group cannot be seen as supported self-help, since the GP was not directly involved in the CCBT intervention.
Third, the low effect sizes in all three interventions might be attributable to our study sample, which was more severely depressed than in previous studies.5,28 The mean starting levels on the BDI–II were even higher than generally seen in primary care patients with depression.43 Chronicity might also have negatively influenced the outcome.44 Unfortunately, we do not have details of the exact duration of each current depressive episode. Moreover, because of our recruitment strategy our sample did not consist of active help-seekers, despite the high severity of their depression. This might have resulted in less-motivated participants.
Finally, we should note that none of our interventions did particularly well. Clinical improvement was approximately 30% in all groups. Given the fact that the response rate with pill-placebo is generally high in depression (i.e. 30–40%),45,46 we might have observed the natural course of depression in the current study. Nevertheless, improvement in our study was somewhat greater than seen in patients with depression on waiting lists.47
Our findings might have several implications for the primary care treatment of depression. First, treatment might only be indicated for those who ‘ get stuck’ in their depression, since depressive symptoms seem to improve over time without adhering to treatment, as was shown in the current study. Second, for more severe depression online CCBT offered with some support might be more helpful. Third, this group of people with more severe depression might also fare better in secondary mental healthcare, where they can receive psychotherapy or antidepressant medication, for which larger effect sizes have been found relative to our effect size.48,49 Fourth, careful implementation of unsupported online self-help is warranted. Stepped-care and collaborative care models might be viable options.50,51 Fifth, if large differences in costs between the interventions exist, this might be a reason to choose one primary care treatment over another. Only one study so far has conducted an economic evaluation of CCBT.52 It was shown that supported CCBT was both more effective and more costly compared with usual GP care. When willing to pay for an additional unit of effect, CCBT could be very cost-effective.52 In a further paper, we will report the economic evaluation of Colour Your Life without support in primary care. Finally, qualitative process evaluation and information on individuals’ experiences (e.g. treatment satisfaction) might also help to decide which treatment should be given to an individual. The acceptability of CCBT both before and after treatment (e.g. expectancy, credibility and satisfaction) has rarely been assessed in research.53 Taken together, more work needs to be done to optimise treatment adherence in CCBT, to determine the best way of providing online and unsupported CCBT in actual practice and to determine for whom CCBT is best suited.
We feel that our results cannot be explained by clear methodological flaws. Our large sample size (n = 303) provided us with sufficient power to detect significant differences between the interventions. Furthermore, we had a relatively low attrition rate, so we feel confident that no biases occurred as a result of missing data. Finally, we were able to recruit participants from the general population. Unlike in samples selected in general practices or clinics, no biases occurred as a result of help-seeking behaviour of individuals and illness recognition by physicians, which is often a problem in depression.54
Some limitations of the present study should also be noted. All our outcomes were measured online and one might question the equality of computerised questionnaires and paper and pen versions. However, there are sufficient indications that computerised and paper and pen questionnaires show similar construct validity.55–57 Furthermore, we relied on self-report measures at follow-up and, as a result, we have no information on actual diagnoses of depressive episodes at follow-up. Finally, it should be noted that the number of participants included is merely a fraction of the original recruitment population (i.e. 0.14%), despite the high prevalence of depression in the community.58,59 Participants had to come to the research centre for an intake, which could have reduced the number of applicants, but could also have increased the adherence. Overall, the low response rate in the current study might be discouraging to the belief of many researchers that online CCBT can be disseminated to large parts of the general population.60
In the current study we were unable to confirm the previously reported high effectiveness of CCBT using online, unsupported Colour Your Life. Moreover, adding Colour Your Life to treatment as usual had no extra beneficial effects. These findings cannot merely be explained by the lack of treatment adherence, since per protocol analyses showed no differences between the interventions either. It is entirely possible that we have observed natural, although not complete, recovery in a group of people with moderate and severe depression who showed a decrease in symptoms irrespective of the nature of the intervention they received. Computerised cognitive–behavioural therapy, offered online without any support, is not beneficial for all people with depression (nor is any treatment of any kind). Adding therapist support to CCBT or treatment in secondary mental healthcare might have yielded better results in this group of people with depression.
We thank Annie Hendriks and Greet Kellens for their assistance during the study and Rosanne Janssen for the development of the infrastructure for online data collection.
- Received May 8, 2008.
- Revision received October 7, 2008.
- Accepted November 11, 2008.
- © 2009 Royal College of Psychiatrists