Hostname: page-component-7c8c6479df-nwzlb Total loading time: 0 Render date: 2024-03-28T02:14:27.944Z Has data issue: false hasContentIssue false

Invited commentary on … Confusing procedures with process in cognitive bias modification research

Published online by Cambridge University Press:  02 January 2018

Ioana A. Cristea*
Affiliation:
Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, Cluj-Napoca, Romania Meta-Research Innovation Center at Stanford, Stanford University, Stanford, California, USA
Robin N. Kok
Affiliation:
Department of Psychology, University of Southern Denmark, Odense, Denmark Centre for Innovative Medical Technology, Odense University Hospital, Odense, Denmark Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, The Netherlands
Pim Cuijpers
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, The Netherlands
*
Ioana A. Cristea, Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, Republicii Street 37, 400015, Cluj-Napoca, Romania. Email: ioana.cristea@ubbcluj.ro
Rights & Permissions [Opens in a new window]

Summary

The notion that cognitive bias modification should be appraised exclusively on the basis of trials where its postulated mechanisms were successfully changed starkly contradicts the standards of evidence-based psychotherapy. In the laboratory or as a treatment, cognitive bias modification cannot continue to eschew the rigorous scrutiny applied to other interventions.

Type
Invited commentary
Copyright
Copyright © The Royal College of Psychiatrists 2017 

Selective exclusion of studies or outcomes from meta-analyses based on post-hoc criteria or applied without transparency or a solid theoretical justification is a pernicious practice that distorts results, Reference Page, McKenzie, Kirkham, Dwan, Kramer and Green1 thus usually considered a grave error. Yet Grafton and colleagues seem to do exactly that with our meta-analysis. Reference Cristea, Kok and Cuijpers2 They misconstrue it as investigating whether cognitive bias modification (CBM) alters ‘emotional vulnerability’, a vague concept of uncertain clinical relevance. Instead, as evident throughout, we examined post-intervention anxiety and depression outcomes. Using our data for anxiety outcomes at post-test, they employ three arbitrary filters to selectively exclude studies and subsequently perform a strictly qualitative and unclear classification of the remaining ones.

The most conspicuous filter is the exclusion of ten studies for measuring ‘resting mood state’ instead of ‘emotional vulnerability’. Eight of these used the State–Trait Anxiety Inventory (STAI)-State, one a specific phobia inventory, Reference Harris and Menzies3 another social anxiety measures. Reference Heeren, Reese, McNally and Philippot4 Hence, we are at a loss as to what the authors mean, as all of these measure symptoms of ‘anxiety’. Moreover, all our effect size calculations were, as described, at post-test. If for instance authors of a trial would use the STAI-State at both post-test and after a so-called stressor task, we only considered the former. So, in this sense everything was ‘resting mood’. Furthermore, if Grafton et al deemed state anxiety measures as improper, they should have also excluded them from effect size calculations for the other included trials. Their analysis nonetheless retained five other studies that solely used the STAI-State. Reference Hirsch, Mathews and Clark5,Reference Murphy, Hirsch, Mathews, Smith and Clark6 Finally, Grafton et al do not substantiate their reanalysis with any actual data analysis, except for tallying findings as yes or no. We undertook this task for them. Presuming they intended to exclude state anxiety, we recalculated effect size for the 12 remaining studies measuring anxiety. The eight where bias change occurred resulted into a small Hedges g of 0.38, virtually identical to our original findings Reference Cristea, Kok and Cuijpers2 (Duval-Tweedie publication bias adjusted g = 0.28). We conducted meta-regression analyses combining bias change with other significant moderators of outcome. Reference Cristea, Kok and Cuijpers2 Bias change no longer predicted outcomes (Table 1).

Table 1 Meta-regression analysis for bias change, alone and in combination with other significant predictors of outcome Reference Cristea, Kok and Cuijpers2

Bias change
Alone And participant
compensation
And delivery And impact
factor
b = 0.42, P = 0.032 b = 0.37, P = 0.18 b = 0.25, P = 0.50 b = 0.24, P = 0.38

More generally, the claim that CBM should be assessed for effectiveness only in the presence of change in its postulated mechanisms conflicts with the current standards for evaluating psychotherapies. For instance, the effectiveness of cognitive–behavioural therapy Reference Cuijpers, Berking, Andersson, Quigley, Kleiboer and Dobson7 is not restricted to trials where dysfunctional thoughts were successfully changed. Process variables are commonly conjectural, unclear, multiple, confounded with outcome measures, assessed in miscellaneous ways, and produce contradictory results. Even when a hypothesised process changes in a trial, it does not follow this is indeed a mechanism of change. Reference Kazdin8 For CBM, the nature and direction of bias change needed to engender symptom change have been targets of speculation and debate, Reference Koster and Bernstein9,Reference MacLeod and Mathews10 although posited as self-evident facts by Grafton et al.

Ultimately, CBM researchers should decide at which table they want to sit. If CBM is cast as a laboratory development, encouraging but as yet inconsequential for clinical practice, exploring procedures to modify assumed processes is an adequate goal. Conversely, if – as repeatedly claimed Reference MacLeod and Mathews10 – CBM is a promising psychotherapy for use on patients and in clinical trials, it should comply with the same standards as all psychotherapies. These standards involve evaluating effectiveness on clinically relevant outcomes, using all available evidence, as we did, Reference Cristea, Kok and Cuijpers2 and cannot hinge on whether or not purported processes have changed. Grafton et al summarily gloss over other serious problems we evidenced, such as lack of effects for clinical samples, pervasive publication bias and low study quality. Vague and debatable distinctions qualitatively applied post-hoc to a subset of the available data cannot substitute for modest if extant symptom change.

Footnotes

See analysis, pp. 266–271, this issue.

Declaration of Interest

None

References

1 Page, MJ, McKenzie, JE, Kirkham, J, Dwan, K, Kramer, S, Green, S, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev 2014; 10: MR000035.CrossRefGoogle Scholar
2 Cristea, IA, Kok, RN, Cuijpers, P. Efficacy of cognitive bias modification interventions in anxiety and depression: meta-analysis. Br J Psychiatry 2015; 206: 716.Google Scholar
3 Harris, LM, Menzies, RG. Changing attentional bias: can it effect self-reported anxiety? Anxiety Stress Coping 1998; 11: 167–79.CrossRefGoogle Scholar
4 Heeren, A, Reese, HE, McNally, RJ, Philippot, P. Attention training toward and away from threat in social phobia: effects on subjective, behavioral, and physiological measures of anxiety. Behav Res Ther 2012; 50: 30–9.Google Scholar
5 Hirsch, CR, Mathews, A, Clark, DM. Inducing an interpretation bias changes self-imagery: a preliminary investigation. Behav Res Ther 2007; 45: 2173–81.CrossRefGoogle Scholar
6 Murphy, R, Hirsch, CR, Mathews, A, Smith, K, Clark, DM. Facilitating a benign interpretation bias in a high socially anxious population. Behav Res Ther 2007; 45: 1517–29.CrossRefGoogle Scholar
7 Cuijpers, P, Berking, M, Andersson, G, Quigley, L, Kleiboer, A, Dobson, KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry 2013; 58: 376–85.Google Scholar
8 Kazdin, AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol 2007; 3: 127.Google Scholar
9 Koster, EHW, Bernstein, A. Introduction to the special issue on cognitive bias modification: taking a step back to move forward? J Behav Ther Exp Psychiatry 2015; 49: 14.CrossRefGoogle Scholar
10 MacLeod, C, Mathews, A. Cognitive bias modification approaches to anxiety. Annu Rev Clin Psychol 2012; 8: 189217.CrossRefGoogle Scholar
Figure 0

Table 1 Meta-regression analysis for bias change, alone and in combination with other significant predictors of outcome2

Submit a response

eLetters

No eLetters have been published for this article.