Changes in met perceived need for mental healthcare in Australia from 1997 to 2007

Graham N. Meadows , Irene Bobevski
  • Declaration of interest

    None.

Abstract

Background

The Australian National Survey of Mental Health and Wellbeing conducted in 1997 and 2007 allows for exploration of changes in perceptions of mental healthcare. Major demographic, economic and health-behaviour changes have unfolded in Australia during this time. Governments also have increased spending on mental health services and been active in mental healthcare policy-making and implementation.

Aims

To compare rates of meeting of perceived need for mental healthcare between these two surveys dealing with adjustable sources of error.

Method

Combined data-sets from 1997 and 2007 were analysed to provide weighted population estimates. Selection, indirect standardisation to a common reference population and multinomial logistic regression were employed, addressing bias and confounding.

Results

Between 1997 and 2007 perceived need for information, counselling and skills training among people who consulted a general practitioner, psychiatrist, or psychologist for mental health reasons has increased. Within these service users, increases in perceived responses from services are evident among people who have perceived needs for information provision and counselling.

Conclusions

Policy and service changes are among possible causal explanations. Generally, trends are in the direction that policy changes were intended to achieve, giving some encouragement that these initiatives have had some effect. Reduced unmet perceived need suggests improved access to some interventions. However, the proportion of service responses to perceived need seen as sufficient is generally unchanged. This suggests the adequacy of treatments offered, as perceived by the Australian public, may not have improved and that a continued focus on quality of care is important for the future.

National epidemiological surveys of mental disorders conducted in 1997 and 2007 established the Australian National Survey of Mental Health and Wellbeing (NSMHWB) programme. Assessment of perceived needs for care and perceived sufficiency of service response to such perceived needs featured in both surveys. This mirrored a policy shift towards greater valuing of the patients’ perspective in assessment and design of mental health services.1 Although expert assessment of need would yield important insights, perceived need assessment is more practicable in a large survey. The first-person perspective has also been argued to be an important complementary perspective in understanding patterns of service use.2 In this, survey methods can reflect realities of clinical care within which asking the patient their view of what might be helpful and monitoring their view of treatment offered is good practice. Self-ratings of symptoms have been seen to predict measures of quality of life above comparable clinician ratings, and changes in patient-rated unmet need have been shown to predict improvement in quality of life.3,4

The Perceived Need for Care Questionnaire (PNCQ) was rigorously developed to measure user appraisals of mental healthcare needs and service responses.5,6 Analyses from Australia have been reported by multiple research groups.710 The PNCQ has been adopted in other programmes internationally1112 with translation into Cantonese, Dutch, French and Spanish. One important objective of better service provision would be decreased prevalence of symptomatic disorders but changes in instrumentation between the Composite International Diagnostic Interview (CIDI) v.2.1 used in 199713 and v.3.0 used in 200714 are such that these comparisons cannot be made with any confidence. Time anchors for questions critical to diagnostic classifications differ between the instruments so that even if an attempt were to be made to construct common severity criteria such as has been done in other survey contexts,15 it is very doubtful that any valid comparisons could be made. The PNCQ stands in contrast to the much-changed CIDI in that core PNCQ content from 1997 was purposively retained in 2007 for comparability. If careful attention is paid to case inclusion and coding, then the PNCQ as used in 1997 and 2007 can yield closely comparable results. An earlier PNCQ-based paper compared crude rates of met perceived need for 1997 and 2007. Overall estimates of population met perceived need were 5.7% in 1997 and 6.2% in 2007. Met perceived needs for information, medication, counselling and skills training all seemingly increasing by between 1 and 2% of the adult population.8 This might indicate substantial improvement in meeting of perceived needs through 1997 to 20078 but limited access to the data at that time meant that key sources of bias could not be excluded. With the confidentialised unit record file from the 2007 survey more recently available to researchers,16 this paper extends examination of perceived need for care. These analyses go much further in controlling for sources of bias in assessing whether the profile of perceived needs has changed from 1997 to 2007.

Specifically then, in this study we aim to describe changes in patterns of perceived need for mental healthcare between 1997 and 2007 in Australia. We cannot expect to establish causation through such a descriptive study; nevertheless, specific hypotheses are that public awareness campaigns conducted between 1997 and 2007 will have led to more people in contact with services for mental healthcare having perceived needs for care in 2007 than in 1997, while the reforms designed to improve access and provision of care should mean these perceived needs will be more often responded to and more frequently met. In view of the evidence of improved mental health literacy and significant increase in the provision of psychological services during the 1997–2007 decade, it would be reasonable to conjecture that changes in perceived need and meeting of need would be most prominent in the areas of mental health information provision and psychological treatment.

Method

The 1997 and 2007 NSMHWB were conducted by the Australian Bureau of Statistics on a stratified multistage probability sample of the Australian population, including participants from all eight states and territories. The 1997 survey17 included people aged 18 to 75+ with no upper limit on age. It had a sample size of 10 641 and 78% response. The 2007 survey included people aged 16 to 85 years. Sample size was 8841 with a lower 60% response rate. The Australian Bureau of Statistics examined response bias in 2007 through comparisons with other national and international data sources and a non-response follow-up survey. This indicated that non-respondents may have had distress scores up to 10% higher,16 so possibly a marginally higher prevalence of mental disorders. Sensitivity analyses conducted by the Australian Bureau of Statistics suggested that the differential response bias seemed unlikely of itself to lead to statistically different findings between the two surveys.16 Details of survey conduct, administration, quality control, issues of response rates and weighting all have been described elsewhere.18

Both surveys included the following measures that we shall consider in this paper. For 12 months prior to interview self-report of mental health services use was captured, then perceived needs and barriers to care as measured by the PNCQ.6 The PNCQ uses self-ratings to assess five areas of perceived need for mental healthcare:

  1. information (about mental illness, its treatment and available services);

  2. medication (medicine or tablets for mental health problems);

  3. counselling (psychotherapy, cognitive–behavioural therapy, counselling);

  4. social intervention (practical help to sort housing or money problems); and

  5. skills training (help to improve ability to work, care for oneself or home, or to use time in other ways).

Participants are categorised based on their responses as either being in perceived need or having no perceived need for mental healthcare in each of the above five areas. For those with perceived need, perceived needs are further categorised as either: fully met (if they received as much help as they perceived was needed); partially met (if they received help, but not as much as they perceived was needed); and unmet (if a perceived need existed, but no help was received).

The PNCQ as used in 20078 was extended from the version used in 1997 but changes were made with attention to the importance of comparison of results from the two surveys. Mostly, changes were added as branch questions and so did not change the text of the stem questions as asked in 1997. Some items in perceived-need areas (d) and (e) above that were grouped in 1997 were asked separately in 2007 but content was common. Changes to CIDI affected the entry rules for the PNCQ between the surveys for people who did not use services, but in both surveys all users of any service for a mental health problem were asked the PNCQ questions. Hence, restricting analyses to a common set of service users removes CIDI changes as a source of selection bias in comparisons. For this paper we restrict consideration of PNCQ findings to people with service use in the previous year and to PNCQ content common between 1997 and 2007.

Service use was determined by asking people whether they consulted listed providers in the previous 12 months. There were changes in lists offered in 1997 and 2007, but questions for consulting general practitioners (GPs), psychiatrists and psychologists were identical. Therefore, analysis was restricted to people who consulted at least one of these three providers for mental health reasons in the previous 12 months. Most people who used mental health services in 1997 (84%) and in 2007 (86%) consulted at least one of a GP, a psychiatrist or a psychologist.

Differences between the two survey times in levels of psychological distress might confound findings in regard to perceived need and we will consider this. Psychological distress in the previous 4 weeks was assessed by the Kessler Psychological Distress Scale (K-10).1921 The K-10 score is based on experience of ten emotional distress items in the previous 30 days rated on a Likert scale from ‘not at all’ to ‘all the time’. The ten items were included in both the 1997 and 2007 surveys, although with some minor wording and item-ordering differences. Final scores range from 10 to 50, with higher scores indicating higher levels of distress. Score bands of different levels of distress have been established by Australian research19 (10–15 low; 16–21 medium, 22–29 high; and 30–50 very high).

Analysis

The Australian Bureau of Statistics confidentialised unit record files for 1997 and 2007 use similar weighting methodology. Each included sample weights derived both from design information and post-stratification weighting for calibration with population benchmarks, as well as a set of replicate weights for variance estimation using the jacknife replication technique. The jacknife replication method derives outputs from repeated analyses using the replicate weights so as to take account of design error in the calculation of statistics.16

The 1997 survey contained 30 replicate weights and the 2007 survey contained 60 replicate weights corresponding to each sample weight. The approach to working with combined data-sets was based on well-established and documented methodology used to compare cross-sectional health surveys with similar weighting techniques, such as the Health Information National Trends Survey (HINTS)22 and the California Health Interview Survey.23 Ninety new replicate weights were created such that only the respective replicate weights for each survey contributed to the variance estimation for that survey.

To make the age range as comparable as possible between the two surveys and within constraints on available age-range coding within the released data-set, we restricted the analysis for this paper to people aged 20–69. Data were analysed using Stata 9.0 Intercooled for Windows XP.

The larger Australian population in 2007 also had a changed age–gender distribution that could make direct comparison of unstandardised rates misleading. In these analyses the 1997 survey population structure was indirectly standardised by age and gender to the 2007 population by using post-stratification weights based on the 2007 age–gender bands. Comparisons of rates are thereby corrected for these demographic shifts.

Within each perceived need category and associated level of perceived need, percentage frequencies and percentage differences 1997–2007 with confidence intervals were calculated, with statistical testing for differences in proportions. As a further investigation of the extent of changes in perceived need between 1997 and 2007, frequencies of perceived needs per person were also compared, using a chi-squared test.

Multinomial logistic regressions were carried out to investigate in more depth the relative changes of proportions of fully met v. unmet perceived need, partially met v. unmet perceived need, and fully met v. partially met perceived need for each perceived need category over time, controlling for age and gender. A separate multinomial regression was carried out for each perceived need category.

To establish whether levels of psychological distress have changed, people who had consulted either a GP, a psychiatrist or a psychologist for mental health reasons in the previous 12 months were compared on their K-10 score bands with a chi-squared test.

Results

The 1997–2007 percentage frequencies for each perceived need category are presented in Table 1, also differences and significance of differences in proportions. For 1997 and 2007, the most frequently endorsed perceived need is for counselling, followed by medication and information. Least-endorsed perceived needs are for skills training and social intervention.

View this table:
Table 1

Percentage frequencies, percentage differences and confidence intervals for the five perceived-need categories and for overall perceived need for the 1997 and the 2007 surveys, for people who consulted a general practitioner, a psychiatrist or a psychologist for mental health reasons

Table 1 shows a consistent increase in perceived need across all categories from 1997 to 2007. The largest increases, which are statistically significant, occurred for information (18.3%) followed by skills training (9.6%) and counselling (5.9%). There was a significant increase in the total number of perceived needs per person in 2007 compared with 1997 (χ2(1,n = 1909) = 15.9, P<0.01). In 2007 there were more people with a total of three to five perceived needs (47.4%, 95% CI 42.2–52.7) compared with 1997 (34.1%, 95% CI 30.1–38.3). In 2007 there also were fewer people with zero to two perceived needs (52.6%, 95% CI 47.3–57.8) compared with 1997 (65.9%, 95% CI 61.8–69.9).

Table 2 examines percentage frequencies and differences with associated confidence intervals and significance between 1997 and 2007 for fully met, partially met and unmet perceived need for each of the five categories. Table 2 shows that consistently across both survey years the larger proportion of perceived need for medication, counselling and information is mostly met in that rank-order. There are large proportions of unmet perceived need for skills training and particularly for social interventions. There is a significant decrease in overall unmet perceived need (–4.1%), in unmet perceived need for information (–12.2%) and counselling (–7.4%) from 1997 to 2007. There are no significant changes in part met and fully met perceived need.

View this table:
Table 2

Percentages, percentage differences and confidence intervals for fully met, partly met and unmet perceived need for the five perceived need categories and for overall perceived-need for people who consulted either a general practitioner, a psychiatrist or a psychologist for mental health reasons and reported a perceived need in the respective category

A separate multinomial regression was carried out for each perceived-need category with perceived-need category as the dependent variable, year of survey as the independent variable (using 1997 as the reference category) and gender and age as covariates (Table 3). These results indicate that for information and counselling, and for the overall perceived-needs category, in 2007 compared with 1997, people reported significantly higher proportions of fully met perceived need and partially met perceived need than unmet perceived need. There were no significant changes in proportions of fully met perceived need compared with partly met perceived need between 1997 and 2007.

View this table:
Table 3

Multinomial logistic regressions with perceived-need category as the dependent variable, year of survey as the independent variable (using 1997 as the reference category) and controlling for gender and age for people who consulted either a general practitioner, a psychiatrist or a psychologist for mental health reasons and reported a perceived need in the respective category

It appears that people who expressed a perceived need for information or counselling were more likely to get at least some help for these perceived needs in 2007 than in 1997 (therefore less unmet perceived need). However, for the people who did obtain help for information or counselling, there is no significant change between 1997 and 2007 in the proportion of people receiving help of this kind who judged it sufficient for their perceived needs (that is ‘met’ v. ‘part met’ perceived need).

There were no significant changes in the meeting of perceived need for medication, social intervention and skills training between 1997 and 2007. However, there is a trend for a higher proportion of fully met perceived need compared with only partially met perceived need for social intervention, which almost reaches statistical significance.

In summary, the above results indicate that differences in increased perceived need and in improved service response from 1997 to 2007 are most evident in the areas of information provision and counselling. A possible confounder for these findings would be present if there were appreciable differences in levels of symptoms typically found among service users in the different surveys. To allow for consideration of this, among people who consulted any of a GP, a psychiatrist or a psychologist for mental health reasons, the percentage of respondents in each score band of the K-10 was calculated for 1997 and 2007. In 1997 the percentages of respondents within each K-10 score band were as follows: low 34.1% (95% CI 30.1–38.4); medium 31.9% (95% CI 28.6–35.3); high 21.5% (95% CI 18.0–25.5); and very high 12.5% (95% CI 10.7–14.6). In 2007 these results were: low 29.9% (95% CI 24.4–36.1); medium 32.3% (95% CI 27.9–36.9); high 24.9% (95% CI 21.1–29.2); and very high 12.9% (95% CI 9.8–16.9). No significant differences were found in the K-10 score bands between 1997 and 2007 (χ2(3,n= 1909) = 0.81, P = 0.48). This suggests that changes in distress levels are unlikely to be strong sources of confounding in these comparisons.

Although Australian Bureau of Statistics sensitivity analyses suggested generally that non-response would be an unlikely cause of such significant differences,16 we examined sensitivity of significant differences in unmet need among service users to a hypothetical scenario of changed response rates. To consider the possible effect that an increase in the 2007 response rate might have had, we simulated a 78% response rate in 2007, as in 1997. To do this we added statistics from a notional extra 18% of respondents to the 2007 survey. We assumed the same rate of service use among simulated new responders but with unmet need among service users scaled up by first, a factor of 1.2, then more extreme values of 1.5 and 2. Unweighted relative risk ratios and confidence intervals24 were calculated for proportions of unmet need in 1997 v. 2007 among service users for any need, information and counselling. Relative risk ratios remained significant for any unmet need under all three scenarios and for counselling for the first scenario. The finding for perceived need for information was more sensitive and significance at P<0.05 was not maintained in these alternative scenarios.

Discussion

Changes in Australia between 1997 and 2007

Our findings need to be considered within the context of changes in health services as well as social, demographic and economic changes between 1997 and 2007. By the time of the 1997 NSMHWB, the transition from institutional to community-based care was well under way in Australia. Changes continued through the further decade: annual expenditure on mental healthcare nationally increased from $164 million dollars to $251 million;25 new psychotropics appeared, with novel antidepressants becoming widely used. There were major structural changes in primary and specialist mental healthcare, with promotion of training for primary care staff and efforts to change public attitudes.26 National survey work confirmed that recognition by members of the public of a vignette depicting someone with a depressive disorder doubled from 1996 to 200327 to over 60%. This change varied in degree across the eight states and territories but positive change was present and substantial across the whole country.27 New government funding for psychologists provided for 6 to 12 sessions of psychological treatment following a GP referral in a scheme that has had very positive systematic evaluation.2830 Other changes in Australia during 1997–2007 should be noted: the Australian population increased from 18.3 to 20.8 million; the average age of the population increased from 35.9 to 37.8 years.31 Unemployment declined from a population rate of 8.3 to 4.3%.32 Gross domestic product increased from AUD $145 billion in 1997 to AUD $286 billion in 2007.33 The proportion of working age adults with a bachelors degree or above increased from 15.6% of the population to 24.1%. Population smoking rates declined from 26 to 19%,34 whereas recent work suggests that as Australian wines have become typically stronger, alcohol consumption in this period has appreciably increased.35 The surveys in 1997 and 2007 therefore provide an opportunity to examine changed experience of care in the population going along with these changes.

Main findings

Between 1997 and 2007, among Australians who consulted a GP, a psychiatrist, or a psychologist for their mental healthcare, we have found a significant increase in perceived need for information, counselling and skills training. There was also a significant apparent increase over this time period in the total number of perceived needs per person. There has been a decrease in unmet perceived need relative to fully met and partially met perceived need for information and counselling, as well as overall, among the above service users. Hence, a higher proportion of service users who perceive a need for information and counselling got at least some perceived response to these particular perceived needs in 2007 than in 1997. However, there were no significant changes in the proportions of fully met perceived need relative to partially met perceived need, suggesting that once help was received for a particular need there was no clear evidence here of changes in consumers’ perceptions of whether the amount of help received was sufficient. Central estimates suggest this latter finding is not likely to be attributable to type II error.

Considering limitations of this study, despite all efforts by the Australian Bureau of Statistics to achieve a representative sample, differences in response rates were present and so varying selection bias could account for some of these apparent effects. It is possible, although not established, that prevalence of depression in some groups in Australia may have increased, and that could affect the overall need profile.36,37 The sections of the questionnaires considered here were set in interviews of different length and content, and thus the possibility of order effects from the surrounding interview cannot be excluded. There was some disaggregation of stem items in the 2007 PNCQ regarding social intervention and skills training8 that could have increased levels of perceived need endorsement here, although this finding was also present in the other items that were identical between the two surveys.

Although acknowledging these limitations, we have in these analyses paid careful attention to issues of comparability of instrumentation bias and possible population confounders. Sensitivity analyses suggest that key findings are robust to a reasonable range of likely response bias. We suggest that a balanced view of the findings is that among users of services from the providers examined, there has been a change in the perceptions of need and of how well it is being met, at least in the two domains broadly of talking therapies and of provision of information. These changes are in the direction of the a priori hypotheses we set out to examine and can be seen as aligned with the hoped-for outcomes from changes in Australia that have targeted service provision and public education. The changes in mental health literacy that are known to have occurred may have contributed to more needs being identified27 and the changes in service delivery nationwide could have contributed to greater service response once need was perceived.26 Nonetheless, the design of this work is such that causation cannot be established. Other influences on well-being and optimism about life or about the effectiveness of services might have had causal roles; when we compare the Australian public of 2007 with that of 1997 we find a public that is more affluent, often better educated and informed, with substantially changed patterns of substance use. Each of these changes and more could have played a part in generating these findings.

Large proportions of unmet perceived need remained over time for social intervention and skills training. In regard to these two perceived needs, it is important to point out that the analyses were constrained for the purposes of ensuring comparability to people who had consulted a GP, a psychiatrist or a psychologist. None of these three types of provider is specifically trained in the areas of social intervention and skills training, which are usually the domains of other professionals, such as social workers and occupational therapists.

In population survey work from the UK, between 1993 and 2000 increased use of medication was evident but not increased psychological treatments.38 In the present study medication provision also trended towards increase, although not significantly. The PNCQ typically rates medication needs as mostly met,7,8 and this likely reflects the difficulty that patients as consumers may have in identifying whether prescribed therapy is adequate to their needs. A patient as consumer may be more able to reach a judgement as to whether a psychological treatment has been provided adequately or not, even though this judgement will not necessarily correspond with adherence to evidence-based practice. Had the time period observed in this British work been applicable to the Australian National Survey programme, then we might have seen very different results. In Australia there was a major surge of antidepressant prescribing in the early and mid-1990s, which then flattened in the early 2000s.39 In contrast, the developments in psychological therapies provided in general practice in Australia have been much more a phenomenon of the first decade of the 21st century.

Implications

These findings should be considered for their implications for policy in Australia. If the findings of this work had been contrary to the directional hypotheses we began with, then this would certainly have been discouraging for policy-makers in Australia. More hopefully, we have found some changes in the anticipated directions. The range of other changes in the country that cannot be controlled for means that the findings cannot firmly be ascribed to government policy. They, however, provide some encouragement that mental health policy-making along with other social changes, is contributing to a positive shift in the engagement of consumers and providers in a number of forms of mental healthcare. Of some concern, the findings do not provide confirmation that once treatment is accessed it is any more likely to be perceived as sufficient. Although the capacity of the system to make some response to perceived needs from consumers may indeed have increased, this evidence supports the continuing need to strive towards improvements in the quality of this response.

Funding

This work was conducted within the resources of the Southern Health Adult Psychiatry Research, Training and Evaluation Centre (Southern Synergy), which receives significant financial support from Southern Health.

Acknowledgments

We thank all the participants in the reported surveys. We acknowledge the Australian Government Department of Health and Aging and the Australian Bureau of Statistics for their essential roles in support and conduct of both the National Survey of Mental Health and Wellbeing 2007 and 1997.

Footnotes

  • See editorial, pp. 443–444, this issue.

  • Received October 7, 2010.
  • Revision received June 16, 2011.
  • Accepted July 28, 2011.

References

View Abstract