Department of Psychiatry, Yale School of Medicine, and Connecticut Mental Health Center, New Haven, Connecticut
Center for Pharmacoeconomic Studies, University of Texas, Austin, Texas
Center for Pharmacoeconomic Studies, University of Texas, and Office of the Medical Director, Texas Department of Mental Health and Mental Retardation, Austin, Texas
Department of Psychiatry, Yale School of Medicine, and VA Northeast Program Evaluation Center, West Haven, Connecticut, and Yale School of Medicine Departments of Epidemiology and Public Health, New Haven, Connecticut
Department of Psychiatry, Yale School of Medicine; Connecticut Mental Health Center, New Haven, Connecticut, USA
Correspondence: Dr C. Bruce Baker, Department of Psychiatry, Yale School of Medicine, CMHC Rm 38B, 34 Park St, New Haven, CT 06519, USA. Tel: (203) 974 7051; fax: (203) 974 7057; e-mail: Bruce.Baker{at}Yale.edu
Declaration of interest Range of industry and non-industry funding received, detailed in Acknowledgements.
|
|
|---|
Aims To determine whether there is an association between sponsorship and quantitative outcomes in pharmacoeconomic studies of antidepressants.
Method Using all identifiable articles with original comparative quantitative cost or cost-effectiveness outcomes for antidepressants, we performed contingency table analyses of study sponsorship and design v. study outcome.
Results Studies sponsored by selective serotonin reuptake inhibitor (SSRI) manufacturers favoured SSRIs over tricyclic antidepressants more than non-industry-sponsored studies. Studies sponsored by manufacturers of newer antidepressants favoured these drugs more than did non-industry-sponsored studies. Among industry-sponsored studies, modelling studies favoured the sponsor's drug more than did administrative studies. Industry-sponsored modelling studies were more favourable to industry than were non-industry-sponsored ones.
Conclusions Pharmacoeconomic studies of antidepressants reveal clear associations of study sponsorship with quantitative outcome.
|
|
|---|
We studied associations between sponsorship and study design with quantitative outcome in pharmacoeconomic studies by examining the test case of antidepressants. We asked the following primary questions. First, is there an association between industry v. non-industry sponsorship of studies and quantitative conclusions? Second, among industry-sponsored studies and between industry-sponsored v. non-industry-sponsored studies, is there an association between study design and quantitative conclusions?
|
|
|---|
Study sample
To locate reports of pharmacoeconomic studies of antidepressant drugs we
used the Cochrane Library, Medline and Health-STAR databases supplemented by
manual searches based on the references cited in the studies located through
the databases. We searched for all articles between 1987 - the year the first
'newer' antidepressant, fluoxetine, received US Food and Drug Administration
(FDA) approval - and April 2001. The search terms we used in Medline and
HealthSTAR were COST-BENEFIT ANALYSIS or COST SAVINGS or
DRUG COSTS or COST-EFFECTIVENESS (text word) and
ANTIDEPRESSIVE AGENTS or ANTIDEPRESSANT (text word). The search term
we used in the Cochrane Library was ANTIDEPRESSIVE AGENTS. We identified 46
articles (Jonsson & Bebbington,
1993,
1994;
Hatziandreu et al,
1994; Le Pen et al,
1994; McFarland,
1994; Sclar et al,
1994,
1995,
1998,
1999;
Stewart, 1994;
Anton & Revicki, 1995; Einarson et al, 1995,
1997;
Lapierre et al, 1995;
Nuijten et al, 1995;
Revicki et al, 1995,
1997;
Skaer et al, 1995; Bentkover & Feighner, 1996;
Forder et al, 1996;
Hylan et al, 1996,
1998;
Montgomery et al,
1996; Smith & Sherrill,
1996; Croghan et al,
1997,
2000;
Melton et al, 1997; Obenchain et al,
1997; Woods & Rizzo,
1997; Boyer et al,
1998; Canadian Coordinating
Office for Health Technology Assessment, 1998; Crown et al, 1998;
Simon & Fishman, 1998; Thompson et al, 1998;
Brown et al,
1999a,b;
Griffiths et al,
1999; Nurnberg et al,
1999; Russell et al,
1999; Simon et al,
1999; Borghi & Guest,
2000; Sullivan et al,
2000; Casciano et al,
2001; Doyle et al,
2001; Poret et al,
2001; Wan et al,
2002). We excluded two studies
(Boyer et al, 1998; Simon et al, 1999)
because they were randomised trials, unlike all the other studies, which were
modelling studies or analyses of administrative databases. The remaining
articles represent 45 separate studies. Two articles report the results of one
study (Jonsson & Bebbington,
1993,
1994). Two articles report two
studies each, one in-patient, one out-patient (Einarson et al,
1995,
1997). Two articles reported
slight variations on two studies, one in-patient and one out-patient
(Casciano et al,
2001; Doyle et al,
2001).
Classification of studies
For the primary analysis we categorised each study according to whether it
was industry-sponsored. The study was categorised as industry-sponsored if at
least one author was listed as a pharmaceutical company employee, or an
acknowledgement listed pharmaceutical company support; otherwise, it was
categorised as non-industry-sponsored. For secondary analyses we categorised
studies authored by industry employees separately from studies only listing
financial support.
Study sponsors were categorised by product into those manufacturing selective serotonin reuptake inhibitors (SSRIs: fluoxetine, sertraline, paroxetine and citalopram) or atypical antidepressant drugs (venlafaxine, bupropion and mirtazapine).
Operationalisation of outcomes
For either of the questions posed in our study no single means of
operationalising the issue of which antidepressant was favoured could be
applied to all studies. Therefore, we performed separate analyses using
alternative operationalisations. Specifically, for question one (the industry
v. non-industry comparison), no single standard was applicable that
allowed analysis of all 46 studies. Seemingly simple standards such as
sponsors antidepressant favoured' could not apply: in
non-industry-sponsored studies, there is no sponsors
antidepressant'. In our primary depressant'. analysis of industry-sponsored
v. non-industry-sponsored studies we examined whether the outcome
favoured SSRIs or tricyclic antidepressants (TCAs), excluding studies
sponsored by atypical antidepressant manufacturers. To allow
analysis of the latter studies, we performed an alternative analysis based on
whether the outcome favoured the newest antidepressant
(newness was based upon date of FDA approval). In this analysis,
studies in which the sponsor's drug was not the newest were excluded.
In addressing our second question, regarding the association of study design with bias on outcome, we examined the issue both within industry-sponsored trials and between industry-sponsored and non-industry-sponsored trials. Within the first group we looked at the association of modelling v. administrative study designs with outcome. We operationalised the outcomes and groups in two alternative ways: favouring the newest drug among all industry-sponsored studies, or favouring the sponsored drug among all industry-sponsored studies.
In examining the association of study design with outcome between industry v. non-industry sponsors, we compared the outcome patterns within modelling studies. We could not compare outcome patterns in administrative data studies given there was only one such non-industry-sponsored study. We operationalised outcomes in two alternative ways: favouring the newest drug, or favouring SSRIs v. TCAs.
Rating study outcomes
Initially two of the authors (C.B.B. and M.N.J.) independently categorised
sponsorship and outcomes of each study. If their ratings were inconsistent, a
third author (S.W.W.) rated the study. Initial ratings agreed in all cases but
one.
Most studies contained several outcomes. However, we wished to rate a single outcome from each study and employed the following decision rules to select that outcome. First, we selected only quantitative outcomes. Second, among base case and variants, we selected the base case. Third, among outcomes adjusted for bias and unadjusted outcomes, we selected the adjusted outcome. Fourth, among outcomes for various time periods, we selected the longest period. Fifth, among multiple pharmacoeconomic indicators, we selected a single outcome on the basis of the following rules: if only cost outcomes were reported, we chose total costs over more limited costs; if cost and cost-effectiveness outcomes were reported, we chose cost-effectiveness outcomes; and if more than one type of cost-effectiveness ratio was reported, we chose incremental over average ratios. Sixth, if results were reported separately for individual countries, we selected the results for the UK and the USA.
After selecting a single outcome for each study, the researchers rated each study as favourable, neutral or unfavourable for the drug of interest, depending on the particular analysis (e.g. SSRI in the SSRI v. TCA analysis, or newest antidepressant in the newest v. older antidepressant analysis): favourable meant that a drug's quantitative cost-effectiveness results were unequalled by any of the other drugs in the study; neutral meant that although other drugs' results might be equal to it, none surpassed the drug of interest; and unfavourable meant that other drugs' results did surpass the drug of interest. Raters used all available information to judge differences in outcomes among drugs. If the study reported statistical significance, raters based their judgements on statistically significant differences. If the study did not report statistical significance, raters based their judgements on the reported numerical differences. With quality-adjusted life-years (QALYs), raters judged a treatment superior if marginal cost-effectiveness was less than US$20 000 per QALY, a common applied limit (Laupacis et al, 1992). Subsequently, we performed a sensitivity analysis by varying the marginal threshold between $20 000 and $100 000 per QALY.
The following is an example of how raters applied the rules noted above to designate a specific study as favourable, neutral or unfavourable. In the SSRI v. tricyclic or heterocyclic antidepressant analysis of the Hatziandreu study (Hatziandreu et al, 1994) the preceding rules led raters to judge that the study favoured the SSRI. The study reported the base case incremental cost-effectiveness ratio to be £2172 ($3692) for each QALY gained by using the SSRI rather than the TCA. This cost per QALY gained is less than the $20 000 per QALY cut-off noted in the raters' decision rules; therefore, the study was rated as favourable for the SSRI.
In addition to the planned analyses described above, we performed two exploratory analyses: one was based on the number of industry authors and the second was based on the ordinal position of any industry authors. Neither of these analyses yielded a significant association.
We analysed the association between sponsorship and outcome using Fisher's
exact test as generalised for 2x3 tables. We chose contingency table
analysis rather than a meta-analytic technique because of the qualitative
heterogeneity of the pharmacoeconomic outcome types across studies, which
ranged from direct costs per patient, to direct costs per treatment success,
to direct costs per symptom-free day, to lifetime direct costs per discounted
QALY. We judged it inappropriate to transform these qualitatively disparate
types of outcomes into a common effect size. We selected the 0.05
level, two-tailed.
|
|
|---|
|
View this table: [in a new window] | Table 1 Economic outcome studies of antidepressant therapy sponsored by manufacturers of selective serotonin reuptake inhibitors |
|
View this table: [in a new window] | Table 2 Economic outcome studies of antidepressant therapy sponsored by manufacturers of atypical antidepressants |
|
View this table: [in a new window] | Table 3 Non-industry-sponsored economic outcome studies of antidepressant therapy |
Association of sponsorship
SSRI v. TCA analysis
For the primary analysis of industry v. non-industry sponsorship
of SSRI v. TCA studies, six of seven non-industry-sponsored studies
were eligible for analysis (see Table
3). Seventeen industry studies were eligible (see Tables
1 and
2).
Distribution and results for Fisher's exact test are noted in Table 4. The association between industry sponsorship and outcome favouring SSRIs v. TCAs was statistically significant. Each of the two secondary analyses contrasting studies with industry-employed authors v. non-industry-sponsored studies and contrasting studies with industry funding alone v. non-industry-sponsored studies demonstrated a statistically significant association between industry sponsorship and outcome favouring SSRIs v. TCAs, with probability values of 0.0420 and 0.0163 respectively.
|
View this table: [in a new window] | Table 4 Fisher's exact test: distributions and probabilities |
New v. old antidepressant analysis
All non-industry-sponsored studies were eligible (see
Table 3). Thirty-three
industry-sponsored studies (see Tables
1 and
2) were eligible. Distribution
and results of Fisher's exact test are noted in
Table 4. The association
between industry sponsorship and outcome favouring the newest antidepressant
was statistically significant. Each of the two secondary analyses contrasting
studies with industry-employed authors v. non-industry-sponsored
studies and contrasting studies with industry funding alone v.
non-industry-sponsored studies demonstrated a statistically significant
association between industry sponsorship and outcome favouring the newest
antidepressant, with probability values of 0.0047 and 0.0018 respectively.
Association between study design and sponsorship bias
Question 1
Within industry-sponsored studies, is there a difference in tendency to
favour the sponsor's drug over a competitor's drug or drug class, based on
type of study design? For the principal analysis, favouring the
sponsors drug or drug class was defined based on favouring the
newest drug among all manufacturer-sponsored studies. Thirty-three
industry-sponsored studies were eligible (see Tables
1 and
2). Distribution and results of
the Fisher's exact test are noted in Table
4. The association between modelling v. administrative
study design and outcome favouring the newest drug was statistically
significant.
For our alternative analysis based on whether the sponsor's drug or drug class won, regardless of whether it was newest, all 38 industry-sponsored studies were eligible (see Tables 1 and 2). This analysis yielded a probability value of 0.0011, consistent with the results in the primary analysis.
Question 2
Between industry-sponsored and non-industry-sponsored modelling design
studies, is there a difference in outcome patterns? For the principal analysis
of this question we examined the patterns of favouring the newest drug.
Nineteen industry studies (see Tables
1 and
2) and five non-industry
studies (see Table 3) were
eligible. The distribution and results of the Fisher's exact test are noted in
Table 4. The association
between industry v. non-industry sponsorship of modelling studies and
outcome favouring the newest drug was statistically significant. Each of the
two secondary analyses contrasting studies with industry-employed authors
v. non-industry-sponsored studies and contrasting studies with
industry funding alone v. non-industry-sponsored studies demonstrated
a statistically significant association between industry sponsorship and
outcome favouring the newest antidepressant in modelling studies, with
probability values of 0.0010 and 0.0100 respectively.
In an alternative analysis we examined the patterns of favouring SSRIs v. favouring TCAs in modelling studies. We performed this analysis with the five eligible non-industry-sponsored studies (see Table 3) contrasted first with all twelve eligible industry-sponsored modelling studies (see Tables 1 and 2) that included SSRI v. TCA comparisons, and then with the six eligible modelling studies sponsored by SSRI manufacturers (see Table 1) that included SSRI v. TCA comparisons. The results of the Fisher's exact test in the two cases were 0.0139 and 0.0151 respectively, indicating that the tendency for industry-sponsored simulations to favour SSRIs more often than non-industry-sponsored studies is unlikely to be due to chance.
The sensitivity analysis varying the marginal cost-effectiveness threshold from $20 000 to $100 000 per QALY did not change any of the results reported above.
|
|
|---|
Consistency with prior studies
Our overall finding of sponsorship bias is consistent with prior studies in
that all three prior fully reported studies found some association between
study sponsorship and outcomes (Azimi &
Welch, 1998; Friedberg et
al, 1999; Neumann et
al, 2000a). However, any detailed comparison between
our study and previous studies is necessarily limited given that the previous
studies mixed drugs, devices and other health interventions, and mixed various
classes of medicines (Azimi & Welch,
1998; Neumann et al,
2000a); focused on qualitative conclusions
(Friedberg et al,
1999); and used various definitions to select the specific study
outcomes to be analysed (Azimi & Welch,
1998; Friedberg et
al, 1999; Neumann et
al, 2000a). The study by Friedberg et al
(1999) of oncology drugs is
perhaps most comparable with our current study, given their focus on a single
pharmaceutical class and their categorisation of study conclusions as
favourable, neutral or unfavourable, although they focused on qualitative
rather than quantitative conclusions. Like our study, that of Friedberg et
al did find an association between study conclusion and funding
source.
Support for concern about modelling studies
In addition to supporting the general concern about sponsorship bias in
pharmacoeconomic studies, our findings support the more specific concerns that
have been raised about the potential for bias in modelling studies
(Luce, 1995;
O'Brien, 1996;
Sheldon, 1996;
Maynard & Cookson, 1998;
McCabe & Dixon, 2000). Such support stems from the combination of our two findings regarding study
design: among industry studies, modelling studies are more favourable to the
sponsor than administrative studies, and in a comparison of industry-sponsored
and non-industry-sponsored modelling studies, studies sponsored by industry
are significantly more favourable to industry.
Limitations of our study
Our study has clear limitations. Randomised pharmacoeconomic trials could
not be compared on the basis of sponsorship because there were only two such
trials in this area. Relatively few non-industry-sponsored studies were
available. We examined only one class of medications; analyses of other
classes of medications should be conducted.
Bias v. accuracy
Although we have demonstrated several associations between study
sponsorship and outcome, these associations do not suggest which (if either)
side presents a more accurate estimate of relative pharmacoeconomic outcome.
Both industry-supported and non-industry-supported researchers may be subject
to forces that could potentially bias their work
(Yee & Hillman, 1997; Drummond, 1998;
Rennie & Luft, 2000).
Additionally, journal editorial processes can result in a biased sample of
studies being published. It has been observed that journals tend to publish
studies with positive rather than negative results
(Freemantle & Mason,
1997).
Causes of bias
Many ideas have been offered to explain how sponsorship could result in
biased reported outcomes (Udrarhelyi
et al, 1992;
Freemantle & Mason, 1997;
Drummond, 1998;
Cook, 1999;
Neumann et al,
2000a; Rennie &
Luft, 2000). Industry, motivated to enhance sales of its products,
might only pursue studies on products and select comparators that would yield
favourable results. They might select biased populations within administrative
data-sets, overtly or subtly influence analytical methods or models, or veto
submission for publication of studies yielding unfavourable results.
Non-industry-sponsored researchers might bias the studies submitted for
publication in similar ways, although perhaps from different motivations such
as controlling formulary costs, personal or academic rivalries, or career
promotion.
We are unable to pinpoint the causes of bias among the reports analysed here. Examination of the individual studies does not reveal a common element that differ differentiates industry-sponsored from non-industry-sponsored studies; rather, the methodological limitations in the studies vary widely. These limitations have been discussed extensively elsewhere (Hotopf et al, 1996; Woods & Baker, 1997, 2002). However, at least two suggested causes seem unlikely. First, some commentators have noted the potential role of selection bias - i.e. the tendency of researchers not to submit and of journals not to publish small studies or studies with negative statistical outcomes (Freemantle & Mason, 1997; Neumann, 1998). This would help to explain how an overall preponderance of statistically positive studies could exist even if there were true uncertainty about alternative medications (Djulbegovic et al, 2000). The difference we have shown between industry-sponsored and non-industry-sponsored studies suggests that submission or editorial selection bias based on statistical significance alone does not adequately explain the bias in the present case. Second, it has been suggested that a particular sponsor weeds out weak alternatives among its drugs in early preliminary processes; therefore, drugs that reach the stage of being marketed are strong competitors and likely to yield analyses that favour the sponsor's drug (Gagnon, 2000). However, these same strong competitors performed less well in non-industry-sponsored studies, as shown clearly in our analysis of outcomes favouring either SSRIs or TCAs. Moreover, it should be noted that in the 18 studies with head-to-head comparisons among such strong competitors, the sponsor's drug lost only once (Einarson et al, 1995).
Bias in efficacy v. pharmacoeconomic studies
It is not possible to comment about whether the bias revealed in the
current study of pharmacoeconomic reports of antidepressants is any greater or
less than the sponsorship bias that may exist in efficacy studies of
antidepressants. There is no published report on sponsorship bias in efficacy
studies in any medication category within psychiatry. The only published
report devoted to such quantitative analysis of psychiatric medications is a
letter reviewing efficacy studies of any psychiatric medication in one journal
over a 1-year period (Mandelkern,
1999). This author reported a tally for industry-supported studies
of 16 favourable to the manufacturer's drug and none unfavourable, and for
unsupported studies 10 favourable and 6 unfavourable, concluding that there
was a correlation between source of support and efficacy outcome. In other
areas of medicine, bias has been demonstrated repeatedly in efficacy studies
(Davidson, 1986; Rochon et al, 1994;
Stelfox et al, 1998;
Djulbegovic et al,
2000). A study of sponsorship bias in efficacy trials of
antidepressants would provide a useful comparison for our study.
It is important for pharmacoeconomic studies to attempt to give estimates that are as accurate and uninfluenced by bias as possible, given the large and growing number of health care dollars spent on medications. Pharmaceutical sales for North America were reported to be US$153 billion in 2000, representing a 14% growth over the previous year (IMS Health, 2001). Owing to the importance of cost constraint in medicine the volume of pharmacoeconomic research has been growing (Detsky, 1994) and is linked to governmental purchasing decisions in some jurisdictions (Canadian Coordinating Office for Health Technology Assessment, 1994; Ontario Ministry of Health, 1994; Australian Government, 1995). However as we noted previously, financial and other incentives create strong motives for bias. Our results for antidepressants suggest that actual bias related to sponsorship appears to exist, although whether or how the bias and specific motives are related cannot be determined. Until the mechanisms producing the bias are better understood, interpretation of results from pharmacoeconomic studies should take sponsorship into account.
|
|
|---|
LIMITATIONS
|
|
|---|
|
|
|---|
Related articles in BJP:
This article has been cited by other articles:
![]() |
C. Barbui, A. Cipriani, T. A Furukawa, G. Salanti, J. P T Higgins, R. Churchill, N. Watanabe, A. Nakagawa, I. M Omori, and J. R Geddes Making the best use of available evidence: the case of new generation antidepressants: A response to: Are all antidepressants equal? Evid. Based Ment. Health, November 1, 2009; 12(4): 101 - 104. [Full Text] [PDF] |
||||
![]() |
K. Okike, M. S. Kocher, E. X. Wei, C. T. Mehlman, and M. Bhandari Accuracy of Conflict-of-Interest Disclosures Reported by Physicians N. Engl. J. Med., October 8, 2009; 361(15): 1466 - 1474. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Jutel and D. B Menkes "But Doctors do it...": Nurses' Views of Gifts and Information from the Pharmaceutical Industry Ann. Pharmacother., June 1, 2009; 43(6): 1057 - 1063. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Timimi Child psychiatry and its relationship with the pharmaceutical industry: theoretical and practical issues Adv. Psychiatr. Treat., January 1, 2008; 14(1): 3 - 9. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. TYRER The British Journal of Psychiatry, December 1, 2006; 189(6): 576 - 576. [Full Text] [PDF] |
||||
![]() |
C. M Bell, D. R Urbach, J. G Ray, A. Bayoumi, A. B Rosen, D. Greenberg, and P. J Neumann Bias in published cost effectiveness studies: systematic review BMJ, March 25, 2006; 332(7543): 699 - 703. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. G. Edwards Newer v. older antidepressants in long-term pharmacotherapy: REVISITING... PREVENTION OF RELAPSE AND RECURRENCE OF DEPRESSION Adv. Psychiatr. Treat., May 1, 2005; 11(3): 184 - 194. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Shooter Dancing with the Devil? A personal view of psychiatry's relationships with the pharmaceutical industry Psychiatr. Bull., March 1, 2005; 29(3): 81 - 83. [Full Text] [PDF] |
||||
![]() |
J. R Lexchin Implications of Pharmaceutical Industry Funding on Clinical Research Ann. Pharmacother., January 1, 2005; 39(1): 194 - 197. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. S. BAUER Dr. Bauer Replies Am J Psychiatry, October 1, 2004; 161(10): 1929 - 1929. [Full Text] [PDF] |
||||
![]() |
Other articles noted Evid. Based Ment. Health, August 1, 2004; 7(3): 67 - 68. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||