Hostname: page-component-7c8c6479df-r7xzm Total loading time: 0 Render date: 2024-03-17T07:50:41.232Z Has data issue: false hasContentIssue false

Abortion and mental health: guidelines for proper scientific conduct ignored

Published online by Cambridge University Press:  02 January 2018

Chelsea B. Polis
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, USA. Email: cpolis@jhsph.edu
Vignetta E. Charles
Affiliation:
AIDS United, USA
Robert Wm. Blum
Affiliation:
Department of Population, Family and Reproductive Health, Johns Hopkins Bloomberg School of Public Health, USA
William H. Gates Sr
Affiliation:
Department of Population, Family and Reproductive Health, Johns Hopkins Bloomberg School of Public Health, USA
Rights & Permissions [Opens in a new window]

Abstract

Type
Columns
Copyright
Copyright © Royal College of Psychiatrists, 2012 

Priscilla Coleman’s recent meta-analysis Reference Coleman1 ignores guidelines for proper scientific conduct of meta-analyses of observational data. Her results violate at least three major principles of meta-analysis: she fails to assess the underlying validity of included studies; she fails to examine statistical heterogeneity; and she illogically combines estimates for distinct outcomes. Furthermore, she accuses previous reviews of lacking ‘reasonable justification’ for declining to quantitatively summarise effects, when declining to do so actually reflected sound epidemiological judgement.

Coleman contends that ‘Through a process of systematically combining the quantitative results from numerous studies addressing the same basic question… far more reliable results are produced than from particular studies that are limited in size and scope’. However, expert consensus suggests that ‘the likelihood that the treatment effect reported in a systematic review approximates the truth depends on the validity of the included studies…’. Reference Liberati, Altman, Tetzlaff, Mulrow, Gotzsche and Ioannidis2 Coleman fails to assess the validity of included studies and erroneously asserts that ‘as a methodology wherein studies are weighted based on objective scientific criteria, meta-analysis offers a logical, more objective alternative to qualitative reviews…’. In fact, studies in meta-analyses are typically weighted by sample size, which is not always related to study quality, Reference Egger, Smith, Schneider, Egger, Smith and Altman3 and decisions on which studies to include and how to include them remain subjective. If poor-quality studies are included, as occurred in Coleman’s review, a poor-quality quantitative estimate will be generated. Coleman combines statistically heterogeneous results, and illogically combines effect estimates for outcomes that vary substantially (i.e. marijuana use and suicide), thus generating a summary estimate void of meaning or utility.

Meta-analysis of observational data can be useful when carefully conducted. However, it is essential that a summary estimate be accompanied by a qualitative description of risk of bias in included studies (which Coleman’s review lacked) since ‘potential biases in the original studies, relative to biases in RCTs, make the calculation of a single summary estimate of effect of exposure potentially misleading’. Reference Stroup, Berlin, Morton, Olkin, Williamson and Rennie4

Coleman ignores other essential requirements of a high-quality statistical meta-analysis. Reference Liberati, Altman, Tetzlaff, Mulrow, Gotzsche and Ioannidis2 She makes no attempt to present a replicable search strategy or article selection diagram. She attempts to justify excluding articles prior to 1995 by noting that study methodology has improved, but fails to adequately justify selected cut-off dates. Ultimately, she includes multiple methodologically weak studies, and excludes at least two older but methodologically stronger studies. She authored her review alone, despite Cochrane and PRISMA recommendations to involve multiple reviewers to reduce the possibility of investigator bias or error. Reference Liberati, Altman, Tetzlaff, Mulrow, Gotzsche and Ioannidis2,Reference Higgins and Green5

Coleman makes disingenuous accusations about previous reviews. For example, she claims that our 2008 systematic review Reference Charles, Polis, Sridhara and Blum6 ‘overlooked’ ten articles which met inclusion criteria, and ‘lacked sufficient methodologically based selection criteria’. This unfounded attack is puzzling, particularly since in 2008, we directly emailed to Coleman the reasons (consistent with our methodologically based selection criteria detailed on p. 437) for excluding seven of these ten articles. The remaining three (not previously enquired about) also fail to meet inclusion criteria: two had a follow-up period of less than 90 days and the other compared medical v. surgical termination.

Coleman continues to ignore the scientific importance of accounting for pregnancy intention in this body of literature. If women who abort (many of which are unintended pregnancies) are compared against women who deliver (many of which are intended pregnancies), effects of unintended pregnancy are difficult to disentangle from effects of abortion. Circumstances surrounding an intentional v. an unintentional conception or pregnancy may be related to mental health outcomes. Most aborted pregnancies in the USA were unintended. Reference Finer and Henshaw7 Coleman wrongly assumes that since nearly half of pregnancies in the USA are unintended, most births are too, failing to acknowledge that almost half of unintended pregnancies end in abortion. Reference Finer and Zolna8 Thus, her assertion that ‘the majority of women in the control groups in studies comparing abortion with term pregnancy actually delivered unintended pregnancies even if the variable was not directly assessed’ has no empirical grounding. Similarly, her assertion that a ‘no pregnancy’ group may be a ‘cleaner’ comparison group ignores the fact that the ‘no pregnancy’ group would not have experienced unintended pregnancy.

The scientific validity and rigour of Priscilla Coleman’s work has been questioned before. Reference Steinberg and Finer9 However, we are surprised and disappointed that the multiple egregious scientific errors in her review went undetected by the editorial or peer-review process of the British Journal of Psychiatry.

References

1 Coleman, PK. Abortion and mental health: quantitative synthesis and analysis of research published 1995–2009. Br J Psychiatry 2011; 199: 180–6.Google Scholar
2 Liberati, A, Altman, DG, Tetzlaff, J, Mulrow, C, Gotzsche, PC, Ioannidis, JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009; 6: e1000100.Google Scholar
3 Egger, M, Smith, G, Schneider, M. Systematic reviews of observational studies. In Systematic Reviews in Health Care: Meta-Analysis in Context (eds Egger, M, Smith, G, Altman, D): 211–27. BMJ Publishing Group, 2001.CrossRefGoogle Scholar
4 Stroup, DF, Berlin, JA, Morton, SC, Olkin, I, Williamson, GD, Rennie, D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA 2000; 283: 2008–12.CrossRefGoogle ScholarPubMed
5 Higgins, JPT, Green, S (eds). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. The Cochrane Collaboration, 2011.Google Scholar
6 Charles, VE, Polis, CB, Sridhara, SK, Blum, RW. Abortion and long-term mental health outcomes: a systematic review of the evidence. Contraception 2008; 78: 436–50.Google Scholar
7 Finer, L, Henshaw, S. Disparities in rates of unintended pregnancy in the United States, 1994 and 2001. Perspect Sex Reprod Health 2006; 38: 90–5.Google Scholar
8 Finer, L, Zolna, M. Unintended pregnancy in the United States: incidence and disparities, 2006. Contraception 2011; 84: 478–85.Google Scholar
9 Steinberg, JR, Finer, LB. Examining the association of abortion history and current mental health: a reanalysis of the National Comorbidity Survey using a common-risk-factors model. Soc Sci Med 2011; 72: 7282.Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.