Declaration of interest
G.T. has received grants for stigma-related research in the past 5 years from Lundbeck UK and from the National Institute for Health Research, and has acted as a consultant to the UK Office of the Chief Scientist.
Evidence on the economic impact of social marketing anti-stigma campaigns in relation to people with mental illness is limited.
To describe the economic impact of the Time to Change (TTC) anti-stigma social marketing campaign, including the potential effects on the wider economy.
Data collected for the evaluation of TTC were combined with the social marketing campaign expenditure data to investigate differences in knowledge, attitudes and behaviour in relation to campaign awareness. To evaluate the return on investment, we applied a decision model that estimated the impact on employment for people with depression.
Based on average national social marketing campaign costs, the economic benefits outweighed costs even if the campaign resulted in only 1% more people with depression accessing services and gaining employment if they experienced a health improvement. The cost per person with improved intended behaviour was at most £4 if we assume the campaign was responsible for 50% of the change. Costs associated with improved knowledge and attitudes, however, were more variable.
The findings suggest that the TTC anti-stigma social marketing campaign is a potentially cost-effective and low-cost intervention for reducing the impact of stigma on people with mental health problems.
Evidence of the economic impact of social marketing anti-stigma campaigns in relation to people with mental illness is limited.1,2 Some evidence does exist for the cost-effectiveness of social marketing programmes for other behavioural health issues such as alcohol misuse and smoking,3,4 but economic evaluations of social marketing campaigns are generally lacking. Social marketing campaigns have the potential to reach large audiences; however, their effectiveness might be reduced by lack of audience insight, an overcrowded media market or ineffective message delivery.5,6 Moreover, the sustainability of social marketing campaign effects is questionable. Given the high costs of social marketing and advertising, evaluation of the cost in relation to outcomes is fundamental.6,7 It is possible that reductions in stigma and discrimination might lead to better and more timely access to services in addition to increased rates of employment,8-11 which could translate to monetary benefits. Additionally, better attitudes among the general public are associated with lower self-stigma and perceived discrimination and higher empowerment among people with mental health problems.12 Funding for mental health services has lagged behind that for other health conditions, and it is especially important that this limited investment has clear economic benefits.13,14 Operationalisation of the impact of anti-stigma social marketing campaigns in relation to mental health may hinder evaluation, as definition and measurement of outcome targets are challenging in comparison with campaigns that address more defined behaviours such as smoking. Research into the costs associated with stigma and discrimination against people with mental health problems, however, is emerging,15 and McCrone et al have developed an innovative model to assess the cost-effectiveness of anti-stigma campaigns for people with mental health problems.1 We applied this newly developed model in conjunction with the Time to Change (TTC) social marketing campaign evaluation data to investigate the economic impact of the campaign, including the potential effects on the wider economy.
Data collected for the evaluation of the TTC social marketing campaign (measures and data collection methods for the evaluation of the anti-stigma social marketing campaign are described in detail elsewhere; see Evans-Lacko et al,16 this supplement) were combined with the social marketing campaign expenditure data to investigate differences in knowledge, attitudes and behaviour in relation to campaign awareness. Data on knowledge, attitude and intended behaviour outcomes in addition to campaign awareness were collected before and after each burst of marketing activity (including six bursts in total) by an online market research panel.17 Approximately 1000 participants, restricted to the campaign target population (i.e. residents of England, aged 25-45 years and of middle-income socioeconomic groups), were recruited for each burst. Quota sampling ensured equal distributions of age, gender and socioeconomic group across time points and that the sample was geographically representative of the population in England.
Prompted campaign awareness was assessed for each type of media and/or activity used by TTC. Individuals who reported seeing any of the advertisements were categorised as ‘campaign aware’ whereas those who responded ‘no’ or ‘don’t know’ were categorised as ‘not campaign aware’. Campaign awareness associated with the post-burst stage pertained to awareness of the specific media activity immediately preceding the survey, whereas awareness during the pre-burst stage referred to recall of the media used in the previous campaign burst.
Mental health-related knowledge was measured by the Mental Health Knowledge Schedule (MAKS).18 This comprises six items covering stigma-related mental health knowledge areas (help-seeking, recognition, support, employment, treatment and recovery) and six items that enquire about classification of various conditions as mental illnesses.19 Overall test-retest reliability of the MAKS is 0.71 and the overall internal consistency among items is 0.65.18
Mental health-related attitudes were assessed based on three items from the Community Attitudes toward the Mentally Ill (CAMI) scale.20 These items were chosen a priori in collaboration with the campaign developers and approved by an expert panel including service users and international stigma experts, and allowed the evaluation results to be directly compared with the concurrent Department of Health national surveys on mental illness, which have used the CAMI since 1993 (for additional information on selection of items see Evans-Lacko et al,16 this supplement). These items assessed attitudes regarding commonality (‘Virtually anyone can become mentally ill’); responsibility (‘People with mental health problems should not be given any responsibility’) and dangerousness (‘People with mental health problems are far less of a danger than most people suppose’). All items addressed messages that were explicitly included in the campaign: for example the ‘1 in 4’ message was a part of the myths/facts campaign and addressed commonality; the ‘Schizo movie’ advert aimed to challenge stereotypes about mental illness and dangerousness (www.youtube.com/watch?v=6IBgkks_jLw); and the personal testimonials of people with mental illness included in the campaign demonstrated real stories of empowerment.
Mental health-related intended behaviour (the level of intended future contact with people with mental health problems) was measured by the Reported and Intended Behaviour Scale (RIBS).21 We assessed changes in four intended behaviour outcomes (domains comprised living with, working with, living nearby and continuing a relationship with someone with a mental health problem). Assessment of intended behaviour allowed for brief and feasible evaluation at the population level. Overall test-retest reliability of the RIBS was 0.75 and the overall internal consistency of the scale was 0.85.
To estimate costs associated with changes in knowledge, attitudes and behaviour associated with the TTC campaign, we first determined the percentage agreement with each knowledge, attitude and intended behaviour item among those aware v. not aware of the TTC campaign. To improve generalisability of the estimate to the English population, survey sampling weights were developed to match characteristics of the recruited sample to the target population in England according to prevalence rates of ethnicity within geographic region reported by the Office for National Statistics. Additionally, to minimise bias associated with campaign awareness (i.e. potential endogenous confounders associated with campaign awareness and outcomes), we applied stabilised inverse probability weights.22 These allow for modelling of the independent relationship between exposure to the campaign and stigma outcomes and minimise potential confounding associated with campaign exposure.23-25 To calculate these weights the conditional probability of campaign awareness or propensity for campaign exposure was estimated by fitting a logistic regression model of awareness using all potential confounders (gender, socioeconomic group, ethnic group and familiarity/knowing someone with a mental health problem). Weighting observations by the inverse probability of exposure can introduce excessive variability to the estimates, thus we used stabilised weights that normalised the conditional probability of awareness by the marginal predicted probability of awareness.26,27 The marginal predicted probability of awareness was estimated by fitting a simple logistic regression model containing only the intercept term. Finally, a separate stabilised inverse probability weight (conditional probability of awareness/marginal probability of awareness) was applied to each observation. The stabilised inverse probability weights allow for better estimation of standard errors because the distribution of weights is shrunk around 1. All stigma outcomes were subsequently calculated using a combination of inverse probability weights and sampling weights.
The difference in agreement with each statement among those who were campaign aware v. not aware was calculated and the percentage difference was then extrapolated to the number of individuals in the target population who were campaign aware. Based on the campaign awareness level of 44.1% (average awareness across bursts) this translates to 6 348 711 being campaign aware, assuming a target population of 14 396 171 (number of individuals in England aged 25-45 years). It might be that the campaign effects would be limited to the target group and would only include those in socioeconomic groups B, C1 and C2, in which case the population would be smaller and the subsequent costs per person with changed knowledge, attitudes and intended behaviour would be higher. As it is unclear how much of the change in stigma outcomes could be attributed to the TTC campaign, we performed a sensitivity analysis for each outcome with success rates ranging from 10% to 100%. These figures were then combined with data on the costs of the campaign in the form of a cost-effectiveness ratio.
Return on investment analysis
To evaluate the return on investment for the TTC campaign we applied a decision model previously described by McCrone et al which estimates the impact that reduced stigma/discrimination has on employment for people with depression.1 The model assumes that some people with depression will not seek help because of the stigma that they experience or anticipate, and that employment opportunities for people with depression are diminished because of discrimination by potential employers. Based on findings from Scotland’s ‘see me’ campaign, McCrone et al assumed a 9% point reduction in those not seeking treatment owing to stigma/discrimination in the presence of a campaign and a similar impact on employment rates. We need to be cautious when generalising from one campaign to another, thus these analyses explored a number of scenarios in relation to TTC. These are that first, the TTC campaign resulted in increased service use of between 1 and 10 percentage points but had no separate impact on employers; second, the TTC campaign did not result in increased service use but did increase employment rates for those with depression by 1-10%; and third, the campaign resulted in both increased service use and employment rates (again by 1-10%). The economic value of increased work is estimated to be £9446, based on the average increase in wages following increased work time.28 This is assuming a low level of annual earnings and as such the results here may be underestimates. All statistical analyses were carried out using SAS version 9.1 for Windows 7.
Campaign expenditure varied significantly, with burst 6 costing the least at £520 000 and burst 5 costing the most at £1.31 million. This resulted in an average campaign burst expenditure of £748 000.
Cost of change in public knowledge
For the MAKS knowledge items, differences in agreement between campaign aware v. not aware individuals ranged from –0.3% (‘Most people with mental health problems go to a healthcare professional to get help’) to 8.0% (‘Most people with mental health problems want to have paid employment’) (Table 1). Although we were able to minimise confounding associated with campaign awareness through the use of inverse probability score weighting, it is still unclear how much of the difference between groups could be attributed to the campaign. Thus, Fig. 1 presents a sensitivity analysis of the cost per person with changed knowledge based on a 10-100% success rate (see also online Figs DS1 and DS2). The higher the rate of success the greater the proportion of people with a change in knowledge due to the campaign, and therefore the cost per person declines as success rate increases. If the campaign is responsible for 100% of the change then the cost per person with changed knowledge ranges from £1.47 for the MAKS employment item to £4.28 for the medication item. If we use a more conservative estimate, that the campaign is responsible for 50% of the change, then the costs range from £2.95 (employment) to £8.56 (medication). As the item about seeking help from a professional was not associated with an increase, this was not analysed.
Cost of change in public attitudes
For the CAMI attitude items, differences in agreement between campaign aware v. not aware individuals ranged from 1.8% (‘People with mental health problems should not be given any responsibility’) to 7.9% (‘People with mental health problems are far less of a danger than most people suppose’) (see Table 1). Figure 2 demonstrates that if the campaign is responsible for 100% of the change then the cost per person with improved attitudes ranges from £1.25 for the dangerousness item to £5.47 for the responsibility item (see also online Figs DS3 and DS4). If we use the more conservative estimate that the campaign is responsible for 50% of the change the costs range from £2.50 (dangerousness) to £10.94 (responsibility).
Cost of change in public intended behaviour
For the RIBS intended behaviour items, differences in agreement between campaign aware v. not aware individuals ranged from 5.1% (‘In the future I would be willing to continue a relationship with a friend who developed a mental health problem’) to 8.8% (‘In the future I would be willing to work with someone with a mental health problem’) (see Table 1). Figure 3 demonstrates that if the campaign is responsible for 100% of the change then the cost per person with improved intended behaviour ranges from £1.93 (continue a relationship) to £1.12 (work with someone) (see also online Figs DS5 and DS6). If we use the more conservative estimate that the campaign is responsible for 50% of the change the costs range from £2.24 (work with someone) to £3.86 (continue a relationship).
Return on investment
Tables 2, 3, 4 describe the economic gain as a result of the TTC anti-stigma social marketing campaign for a hypothetical individual with depression. The gain consists of increased work (with wage rates used as a proxy for extra production) minus the extra service costs incurred through increased uptake. Table 2 suggests that compared with a situation in which 50% of people with depression do not access services owing to stigma and discrimination, the TTC anti-stigma social marketing campaign resulting in 49% not accessing services owing to stigma or discrimination (1% success) would result in more people experiencing a health gain and therefore more also having increased work time. Averaged across all English adults with depression, this results in £3 per person of economic gain. When we extrapolate this rate to adults in the target age range of 25-44 years the return on investment is estimated to be between 0.7 and 1.9, depending on campaign cost. If we assume a 10% success rate the return on investment increases to between 6.7 and 18.5.
Taking together all of the 40 campaign cost-success rate combinations the return is above 1 for all but two cases (where campaign costs are very high and the success rate is low). If the campaign had no effect on service use or employment, the return on investment would be zero and there would be a loss equal to the value of the campaign. If the campaign affects employment rates but does not separately affect uptake of services then the return on investment ranges from £8.1 million (high cost and low success rate) to £223.5 million (low cost and high success rate) (Table 3). This is because there is an employment effect for those with health improvements regardless of whether they are accessing services because of reduced stigma/discrimination. In all campaign cost-success rate combinations the return on investment is well above 1. The worst scenario is that with a campaign cost of £2 million and only a 1% success rate the return is 8 times the investment.
Table 4 demonstrates that the largest returns on investment follow a campaign that influences both service use and employment rates. In this scenario the figures range from £8.8 million to £257.5 million. Notably, it is also possible that there is no effect on either service use or employment, and that the return on investment would therefore be zero.
Overall TTC programme cost
The total TTC programme cost £20.5 million. The ratio between this and the total cost of mental health problems is lower than the ratio between campaign costs for stroke and obesity and the respective costs of these clinical areas (Table 5).
Our findings suggest that the Time to Change anti-stigma social marketing campaign is a potentially cost-effective and low-cost intervention for improving intended behaviour among the general public in addition to some (but not all) types of mental health-related knowledge and attitudes. Moreover, when taking into account potential effects on the wider economy such as service utilisation and employment, the economic benefits outweigh the financial costs of the campaign for almost every scenario. Finally, when comparing the costs of the entire TTC programme in relation to other types of public health campaigns, the TTC costs are relatively low when considering the global burden of disease consequent upon neuropsychiatric disorders.29-31
We estimated relatively low costs for changes in intended behaviour and varied costs associated with knowledge and attitude outcomes. If the campaign is responsible for 50% of the intended behaviour change, the cost per person with improved intended behaviour is at most £4. The higher relative impact on behavioural outcome in relation to knowledge and attitudes is consistent with other findings in relation to the TTC programme (see Evans-Lacko et al16 and Thornicroft et al,32 this supplement). The cost associated with the three attitude items varied, with the responsibility item being associated with the highest cost. It is notable that the dangerousness item was associated with greatest improvement - and thus lowest cost - among the three attitude items (i.e. £2.50 if we assume a 50% success rate). Previous work suggests that this belief is deeply rooted and persists over time.33,34 Moreover, it has been suggested that a weakness of anti-stigma campaigns is that they would not be successful at addressing entrenched attitudes about difficult topics such as dangerousness.35 In response, the TTC anti-stigma social marketing campaign developed the ‘Schizo film’ advert which specifically addressed this stereotype. These findings suggest that targeted media addressing the dangerousness and mental illness stereotype may be an effective and low-cost approach, although its effectiveness will be influenced by the extent of news media reporting that supports this stereotype (see Thornicroft et al, this supplement32).
The return on investment figures demonstrate positive economic benefits in relation to campaign costs of up to £2 million. These figures are likely to be underestimates because the analysis focused only on people with depression (because that is where there is most evidence about employment effects following treatment). The TTC campaign, however, aims to reduce stigma and discrimination across all mental health conditions. In addition, we have assumed employment gains only for people with health improvements, whereas a successful campaign might result in employment gains for those with no health improvement also.
Strengths and limitations
Our assessment of campaign awareness included a detailed questioning process to assess prompted and unprompted campaign awareness for each type of media. However, we could only assess self-reported campaign awareness and not actual awareness. Additionally, although we were able to minimise the endogeneity bias associated with campaign awareness through the use of inverse propensity weights, our model may not have accounted for this completely. Additionally, although the effects of the campaign may increase over time and may vary according to the campaign message, our data did not allow us to measure serial awareness and thus our analysis was based on average estimates across all bursts. A related limitation is that it is impossible to know the impact on knowledge, attitudes and intended behaviour that can be attributed to the TTC campaign rather than to other contemporaneous influences present in England. Additionally, as the campaign has been running from 2009 to 2012, it is unclear how sustainable these effects would be over the long term without the presence of the campaign. Finally, our evaluation includes assessment of knowledge, attitude and intended behaviour, and thus the economic effects depend on these changes being reflected in actual behaviour. Thus, we must also consider the possibility that the campaign has no effect on behaviour and hence no economic benefit. Evaluation of discrimination experienced by people using mental health services over the same period (see Corker et al, this supplement36) suggests that discriminatory behaviour against people with mental illness is decreasing; however, we cannot be certain how much of these effects are directly related to the TTC campaign or to changes in public stigma.
For this evaluation we focused only on certain effects, and given that the campaign will have had other impacts, the findings here are conservative. Moreover, knowledge, attitude and intended behaviour change are not mutually exclusive outcomes. That is to say, if spending on the campaign results in views about the psychotherapy statement changing, then views about the other statements are likely to change also.
This study suggests that mass media campaigns can produce positive changes associated with economic benefits across a large target group. As the media, delivery and specific campaign message used by different anti-stigma campaigns are likely to vary according to population needs, budget and organisations involved in the development,37,38 the findings associated with these specific items may not generalise to all anti-stigma social marketing campaigns.39,40 Additionally, it is possible that targeting anti-stigma efforts towards specific ‘high-impact’ subgroups (e.g. campaigns focused on employers, teachers or healthcare staff) might be more cost-effective. Thus, the opportunity costs of working with broad rather than specific target groups or employing other types of anti-stigma interventions should be considered and investigated in future research. As funding in mental health generally has lagged behind that for other health conditions, it is especially important to demonstrate economic impact for mental health-related programmes.13,14 To demonstrate value an intervention must be effective, and therefore it is important to understand the effect that spending money on TTC has on the lives of people with mental illness, as well as potential savings in some areas. Additional research has begun to make this case (see Corker et al, this supplement36); however, a better understanding of the potential range of effects is needed. These initial findings, however, should give policy makers more confidence around investment in anti-stigma social marketing interventions such as the TTC campaign.
We thank Sue Baker (Time to Change), Maggie Gibbons (Mental Health Media), Paul Farmer (MIND), Paul Corry (Rethink Mental Illness) for their collaboration on the evaluation. CH, PM and GT are supported by a National Institute for Health Research (NIHR) Applied Programme grant awarded to the South London and Maudsley NHS Foundation Trust. GT is also supported in relation to the NIHR Specialist Mental Health Biomedical Research Centre at the Institute of Psychiatry, King’s College London and the South London and Maudsley NHS Foundation Trust (GT). CH is also supported by grants from Guy’s and St Thomas’s Charity and the Maudsley Charity. All opinions expressed here are solely those of the authors.
The evaluation of Time to Change 2007-11 was funded by the Big Lottery Fund, Comic Relief, and SHiFT.
- Royal College of Psychiatrists
Royal College of Psychiatrists, This paper accords with the NIH Public Access policy and is governed by the licence available athttp://www.rcpsych.ac.uk/pdf/NIH%20licence%20agreement.pdf