Declaration of interest
Clinicians have considerable difficulty identifying and helping those people with alcohol problems but no previous study has looked at this systematically.
To determine clinicians’ ability to routinely identify broadly defined alcohol problems.
Data were extracted and rated by two authors, according to PRISMA standard and QUADAS criteria. Studies that examined the diagnostic accuracy of clinicians’ opinion regarding the presence of alcohol problems as well as their written notation were evaluated.
A comprehensive search identified 48 studies that looked at the routine ability of clinicians to identify alcohol problems (12 in primary care, 31 in general hospitals and 5 in psychiatric settings). A total of 39 examined alcohol use disorder, 5 alcohol dependence and 4 intoxication. We separated studies into those using self-report and those using interview. The diagnostic sensitivity of primary care physicians (general practitioners) in the identification of alcohol use disorder was 41.7% (95% CI 23.0–61.7) but alcohol problems were recorded correctly in only 27.3% (95% CI 16.9–39.1) of primary care records. Hospital staff identified 52.4% (95% CI 35.9–68.7) of cases and made correct notations in 37.2% (95% CI 28.4–46.4) of case notes. Mental health professionals were able to correctly identify alcohol use disorder in 54.7% (95% CI 16.8–89.6) of cases. There were limited data regarding alcohol dependency and intoxication. Hospital staff were able to detect 41.7% (95% CI 16.5–69.5) of people with alcohol dependency and 89.8% (95% CI 70.4–99.4) of those acutely intoxicated. Specificity data were sparse.
Clinicians may consider simple screening methods such as self-report tools rather than relying on unassisted clinical judgement but the added value of screening over and above clinical diagnosis remains unclear.
Alcohol problems are a significant public health problem.1,2 Alcohol consumption has been estimated to cause 3.8% of all deaths and 4.6% of disability.3 Alcohol problems in general include alcohol use disorder, alcohol dependence and acute intoxication. Alcohol use disorders include a spectrum of excessive drinking often also described as alcohol abuse (DSM-IV),4 hazardous drinking (WHO)5 or harmful drinking (ICD-10)6 (see Appendix 1). In the general population hazardous drinking is seen in 30–40%,7 with lifetime alcohol misuse or dependence found in about 10% compared with 16–36% of out-patients.8,9 In primary care approximately 7–30% of attendees have at-risk drinking or an alcohol use problem.2,10–12 In hospital settings the point prevalence of alcohol use disorder varies between 7 and 25%13–18 and approximately 4% have alcohol dependence.19 In psychiatric out-patients with serious mental illness, a recent meta-analysis showed rates of 10% (current use) and 20% (lifetime use) for alcohol use disorder.20 The highest rates are seen in psychiatric in-patients where prevalence rates may be as high as 50%.21–23 In spite of these high prevalence rates it seems that only a minority of alcohol problems are detected and treated. Studies conducted in the USA, Australia, UK and Finland indicate that clinicians frequently do not screen for alcohol use disorder and fail to address the problem in at least a third to a half of cases even when the diagnosis is known.24–28 In most cases, diagnosis is made by clinical judgement without the use of scales, blood tests or reference to diagnostic criteria.29,30 Patient surveys suggest that only 30–40% are asked about their alcohol habits31–33 and a small percentage of those with alcohol problems report receiving advice to cut down.34 Several effective treatment packages including brief alcohol interventions have been developed and are potentially effective.35–37 However, such interventions can only be effective when alcohol problems are recognised. Numerous studies of screening tools and biomarkers have been conducted but it is important to clarify how much improvement in the identification of alcohol problems occurs with their use above and beyond that achieved from routine clinical judgement alone.38 Early research suggests that about a third of individuals with alcohol problems are detected by their general practitioner.28 The comparable detection rate from general hospital and psychiatric settings is unknown, although some previous work has suggested that hospital specialists detect most people with drinking problems at admission.39
The aim of this study was to clarify accuracy of clinical judgement as well as clinical recording of clinicians working in (a) primary care, (b) general hospital and (c) psychiatric settings in identifying broadly defined alcohol problems. We hypothesised that mental health specialists would have the greatest success and primary care doctors the least success when working without assistance of scales or tools.
Inclusion and exclusion criteria
The principal inclusion criteria were studies that examined the clinical judgement and clinical recording of clinicians in the identification of those with alcohol problems. We defined alcohol problems broadly to give a representative picture of routine practice. We grouped these problems into alcohol use disorder with and without dependence. We allowed studies to use a self-report or interview-based criterion definition of alcohol use disorder. Although we did not specifically exclude ICD-10-based studies, all interview-based studies used DSM criteria (Appendix 1). In an attempt to minimise heterogeneity we looked at studies examining acute intoxication and alcohol dependence separately from alcohol use disorder without intoxication or dependence. We excluded studies that did not present data on alcohol use, were based on vignettes, had insufficient primary data or failed to employ an adequate criterion standard (e.g. studies based on self-reported alcohol use).40–46
Search and information sources
A systematic search, critical appraisal and meta-analysis were conducted. The following abstract databases were searched from inception to September 2011. In MEDLINE/Embase (266 hits) and PsycINFO (20 hits), relevant keywords, MESH terms, title terms and limits were applied (available from the author on request). In four full-text collections (Science Direct, Ingenta Select, Ovid Full text, Blackwell-Wiley Interscience) the search terms were used as a full text search and citation search (261 hits). The abstract databases SCOPUS (179 hits) and Web of Knowledge (113 hits) were searched, using relevant search terms as a text word search, and using key papers in a reverse citation search. Non-English language articles were searched and one relevant study was found. We also contacted several experts in the field for unpublished and very recently published work.
Data extraction and appraisal
We adhered to standards in the PRISMA guidelines for meta-analyses.47 Data extraction was conducted independently by two authors using a data extraction form in Microsoft excel. The form was developed from previous systematic reviews of diagnostic accuracy according to principles of PRISMA and the Cochrane Collaboration (available from the author on request).48 Variables extracted were country of study, setting (e.g. primary care, general hospital), patient characteristics (e.g. age, gender), reference standard (including cut-off if relevant), method used to determine clinician judgement, sample size, positive cases and negative cases (as identified by reference standard), sensitivity, specificity, true positives (i.e. clinician judgement and reference standard both suggest alcoholism), false positives (i.e. clinician judgement suggests alcoholism but reference standard does not), false negatives (clinician judgement indicates no alcoholism but reference standard suggests alcoholism) and true negatives (both clinician judgement and reference standard both judge no alcoholism). To establish validity of the data extraction for the primary outcomes, true positive, false positives, false negatives and true negatives extracted from papers were recalculated from prevalence, sensitivity and specificity data in order to identify any inconsistencies or errors in extraction. Any inconsistencies were resolved by double-checking data from the paper and discussion with one of the authors (A.J.M.). Appraisal of each article was conducted by all authors independently using QUADAS.49 This is a standardised quality appraisal form and is the recommended tool for a number of organisations such as the Cochrane Collaboration and the National Institute for Health and Clinical Excellence.
Given high heterogeneity, we used random effects bivariate meta-analysis to synthesise the data and provide pooled estimates of sensitivity and specificity using the metandi commands in Stata 10 on Windows. This method fits a two-level model, with independent binomial distributions for the true positives and true negatives conditional on the sensitivity and specificity in each study, and a bivariate normal model for the logit transforms of sensitivity and specificity between studies.50 A summary receiver operator characteristic (ROC) curve, where each data point represents a separate study, was then constructed using the bivariate model to produce a 95% confidence ellipse within ROC space. Heterogeneity was assessed using the I2 statistic.51 Partial verification bias, differential verification bias and incorporation bias was assessed for each study (online Table DS1). Finally, publication bias was assessed formally using Begg-Mazumdar’s test.52
Study description and methods
We identified 48 studies of clinical accuracy including 39 on alcohol use disorder, 4 concerning intoxication and 5 examining alcohol dependence (Fig. 1). The sample size of individual studies ranged from 35 to 3014 individuals (mean 490.6, s.d. = 644.7). Twenty-one studies identified alcohol use disorder on the basis of a structured interview, four intoxication studies used blood alcohol concentration and the remainder used self-report measures of alcohol use (online Table DS2). High heterogeneity was found for most analyses (I ranged from 92.0 to 94.5% depending on analysis). Publication bias was assessed using funnel plots but no evidence of bias was detected (Fig. 2).
We examined the prevalence of each type of alcohol problem according to setting. In primary care the pooled prevalence of alcohol use disorder was 16.7% (95% CI 10.0–24.6). It was 12.1% (95% CI 7.2–18.0) when identified by interview and 22.7% (95% CI 10.5–37.9) when identified by self-report. In hospital settings the prevalence of alcohol use disorder was 33.5% (95% CI 16.2–53.5%) (Martin et al53 was excluded from the prevalence calculation as the study pre-selected a high-risk sample). However, the prevalence was 43.4% (95% CI 5.1–87.5) when identified by interview and 28.1% (95% CI 20.5–36.5) when identified by self-report. In mental health settings the prevalence of alcohol use disorder was 21.7% (95% CI 10.4–35.7), with insufficient studies to stratify by self-report/interview. Finally, the prevalence of alcohol dependence was 12.1% (n = 4, 95% CI 9.3–15.1) in hospital settings, whereas the prevalence of intoxication reached 52% (n = 2, 95% CI 7.7–94.0) identified by analysing blood alcohol levels in emergency departments.
Identification by primary care physicians
Alcohol use disorder
Across 12 studies, involving 10 997 people with problem drinking, primary care physicians recorded alcohol problems in medical records in 27.3% (95% CI 16.9–39.1), but actually recognised alcohol use disorder in 41.7% (95% CI 23.0–61.7) using clinical judgement (Table 1). In studies relying on interview-based gold standard detection the sensitivity was 44.0% (95% CI 21.4–68.0). There were only two studies with specificity data, both based on clinical judgement, and primary care physicians correctly reassured 93.1% (95% CI 86.7–97.6) of people without a drinking problem (detection specificity). At a prevalence of 20%, the positive predictive value would be 60.2% and the negative predictive value 86.5%. Thus, a primary care practitioner would typically identify 8 cases, missing 12. They would correctly identify 75 non-drinkers, falsely diagnosing 5. Thus the fraction correctly identified would be 83%.
Alcohol dependence and alcohol intoxication
No studies were found.
Identification by hospital staff in medical settings
Alcohol use disorder
Across 23 studies involving 10 837 people with alcohol use disorder, health professionals correctly recorded alcohol problems in 37.2% (95% CI 28.4–46.4) of case notes. Based on clinical judgement, at interview they correctly identified 52.4% (95% CI 35.9–68.7). There was no difference in detection sensitivity when stratified by gold standard (robust clinical interview) 41.2% (95% CI 28.8–54.2) v. self-report 42.7% (95% CI 31.1–54.6). In nine studies reporting specificity, hospital staff identified 93.1% (95% CI 89.1–96.3) of those without problem drinking with no difference by case ascertainment or outcome method.
At a prevalence of 20%, the positive predictive value would be 52.5% and the negative predictive value 88.2%. Thus, hospital doctors would typically identify 10 cases, missing 10. Hospital doctors would typically correctly identify 71 non-drinkers, falsely diagnosing 9. Thus, the fraction correctly identified would be 81%.
Alcohol dependence and alcohol intoxication
There were four studies assessing the identification of alcohol dependence in hospital settings. Hospital staff accurately recorded 41.7% (95% CI 16.5–69.5) of such cases in medical notes. However, no data were reported on specificity in these settings.
Four studies examined the ability of doctors and nurses working in trauma centres to identify acute alcohol intoxication (defined by a high blood alcohol concentration). Health professionals were able to identify intoxication in 89.8% (95% CI 70.4–99.4) of cases based on clinical judgement and recorded this in the notes in 75.6% (95% CI 68.1–82.3). However, their specificity (based on clinical judgement) was low at 61.4% (n = 2, 95% CI 11.4–98.7).
Identification by mental health professionals
Alcohol use disorder
We located four studies involving a small sample of 784 patients, but as there was only one study using a chart review method (showing a sensitivity of 28.2%), we were only able to pool studies of clinical judgement, reducing the sample size to 384. Mental health professionals identified 54.7% (95% CI 16.8–89.6) of those with alcohol use disorder. Based on clinical judgement, their detection specificity was 83.6% (95% CI 56.3–98.8). At a prevalence of 20%, the positive predictive value would be 45.4% and negative predictive value 88.1%. Thus, a mental health professional would typically identify 11 cases, missing 9. They would correctly identify 67 non-drinkers, falsely diagnosing 13. Thus, the fraction correctly identified would be 78%.
Alcohol dependence and alcohol intoxication
Only one small study was found. Rienzi54 reported that mental health practitioners had a sensitivity of 82.9% (95% CI 67.3–91.9) when looking for self-reported alcohol dependency (defined using the Michigan Alcoholism Screening Test (MAST)).
Figure 3 shows the results for the hierarchical summary ROC curve analysis for clinical identification of alcohol use disorder.
We found 39 studies examining the clinical identification of alcohol use disorder, 5 studies involving alcohol dependence and 4 involving acute alcohol intoxication, with a total sample of 23 472 participants. Although the overall sample size was large there was sparse data on dependence and intoxication, especially in mental health settings (Table 1). Our findings indicate that all healthcare professionals have considerable difficulty with the identification of problem drinking in clinical practice, identifying about half of those with alcohol use disorder based on clinical judgement and correctly recording alcohol use disorder in the notes in only one in three cases. It should be noted that this data were based on single assessments to inform clinical opinion. Only one study examined the effect of cumulative assessment, finding that detection sensitivity improved from 16 to 34% after three consultations.56 Previous studies have found that clinicians have most difficulty identifying individuals with milder alcohol problems and better success with dependence but we found no significant difference.46 The most successful group were emergency department specialists who were able to identify acute alcohol intoxication in nine out of ten people. This is in itself important, as at least 33% of people seen in the emergency department for trauma have evidence of legal intoxication.55,57 However, even here emergency department specialists made a significant number of false positive errors.
Using clinical judgement primary care physicians identified about four in ten of attendees with alcohol use disorder, although their medical records were accurate in less than three out of ten. This is in line with previous work suggesting that most of those presenting in primary care are not detected routinely.28 Recognition by hospital specialists and mental health professionals has been much less discussed.44 Hospital staff also had difficulty with recognition with about half of people with alcohol use disorder identified. This is important because alcohol use disorder can exacerbate severity of illness and prognosis in people with several mental disorders such as schizophrenia and depression.58–60 Our finding of lower accuracy in indentifying problems among mental health professionals may be surprising but it has been previously found that alcohol problems are often not discussed even in mental health settings.61,62 Further, most mental health practitioners are not expert in alcohol problems, rarely use standardised instruments for alcohol problems and have variable access to dedicated specialist alcohol services.63–65
Barriers to the recognition of alcohol problems
Many factors have been cited as barriers to appropriate and prompt recognition. These include clinician confidence as to what constitutes alcohol misuse,66 inadequate training,67 lack of contractual incentives,68 lack of time,69 fear of labelling due to the stigma associated with substance misuse70 and a belief that patients will not honestly disclose their drinking practices.71–73 In most cases patients accept being questioned about their drinking habits.74,75 Our data on the similar prevalence of alcohol problems by self-report compared with interview and lack of substantial diagnostic differences by criterion reference do not support the hypothesis that people will not disclose their drinking history if asked in a sensitive manner. Therefore we suggest that the most significant modifiable predictor remains the willingness of the clinician to ask about alcohol habits appropriately.76 In self-report surveys, health professionals report that they often enquire about drinking behaviours.77–79 Yet they also express the belief that clinical questions will not be answered honestly by patients70 and are concerned that asking about drinking might harm the patient–provider relationship.72,80
Observational studies of clinician enquiry show that, in general, screening for alcohol problems is not routine in primary care29,30,81,82 or in specialist settings.83 Several studies found that clinicians discuss alcohol use in about 10–15% of consultations but few discussions are specifically prompted by concerns over drinking habits.84–86,83 On videotaped or observed interviews, alcohol-related discussions were often superficial and yielded little information regarding patients’ drinking practices.82,87,88 Interviews where at-risk drinking discussions took place typically lasted only 1–2 min.89,90 Of those clinicians that look for alcohol problems, nearly all prefer asking quantity–frequency questions, about a third say they use the CAGE questions, and 15% cite use of biochemical markers.83,91,92 D’Amico et al examined the practices of physicians towards over 7000 individuals visiting them.93 The practitioners asked 29% about their drinking (and 44% of problem drinkers about their drinking) over 1 year. Of those asked about alcohol problems, only 21% received advice (49% in the case of problem drinkers).93 Clinicians are least likely to raise the subject of problem drinking with White people, women and widows,86 prioritising discussion with healthier, younger males who misuse tobacco and alcohol.94 Less studied is the issue of whether clinicians may also find distinguishing problematic alcohol use from non-problematic use difficult.
In response to these concerns the Institute of Medicine, the National Institute on Alcohol Abuse and Alcoholism (NIAAA), the American Medical Association and the American Society of Addiction Medicine have all recommended that clinicians routinely ask patients about alcohol use.79,95–97 However, the Scottish Intercollegiate Guidelines Network advocates clinical assessment with judicious use of questionnaires only where there is suspicion of alcohol problems.98 The NIAAA and the US Preventive Services Task Force (USPSTF) recommend population screening to identify problem drinking; that is, clinicians should ask all attendees whether they drink, and assess the specific quantity, frequency and pattern of consumption, but they did not recommend a specific tool.99 In 2004 the USPSTF recommended that screening should be accompanied by behavioural counselling interventions to reduce alcohol misuse by adults in primary care settings.100 The NIAAA also recommended targeted screening (case finding) in that all patients who drink alcohol should be screened with the CAGE questions.101 To date, variations of the AUDIT (Alcohol Use Disorder Identification Test), CAGE and MAST have been the most common questionnaires for alcohol problems but these tools are difficult to use in a primary care practice.72,79,82,102,103 No single laboratory test or combination of tests has been shown to be appropriate for screening.104,105
The main limitation in this data synthesis is lack of data from some settings and a lack of consistency in terminology for alcohol use disorder.106,107 It is disappointing that few studies were conducted in Europe, and none in the UK. Some problems in terminology are to be anticipated given we have examined studies spanning more than 25 years of clinical practice. A second limitation is the reliance on self-report criterion methods such as the CAGE, AUDIT and MAST in some studies. However, we adjusted for this by examining both interview-based and self-report standards separately. In primary care and medical settings there was no difference in sensitivity or specificity but in mental health settings, in studies relying on an interview-based gold standard, detection sensitivity was significantly lower 36.0% (95% CI 16.5–58.2) compared with self-report 79.8% (95% CI 70.0–88.1). In addition, we found no statistically significant difference in the prevalence of alcohol use disorder whether defined by interview or self-report.
Given the clear findings that most, if not all health professionals struggle to accurately identify those with alcohol problems (including dependency) is there any evidence that interventions improve recognition rates? There is some evidence that education programmes can improve diagnostic habits.108 A meta-analysis of 15 methods in 12 trials aimed at increasing primary care practitioner rates of screening for, and giving advice about, hazardous and harmful alcohol consumption concluded it is possible to increase the engagement of physicians although effects of identification of alcohol problems were unclear.109 There is an urgent need to trial combined screening, education and brief alcohol interventions in adequate samples in both mental health and general hospital settings. We suggest that such trials specifically measure detection sensitivity and detection specificity as well as patient outcomes. We also suggest that such trials compare the performance of screening against unassisted clinical accuracy in order to clarify which methods most help clinicians identify people with problem drinking.
Thanks to the staff of the postgraduate library, Leicester General Hospital. Thanks also to Alex Sutton, Professor of Medical Statistics, University of Leicester for statistical advice.
- Received December 22, 2010.
- Revision received September 19, 2011.
- Accepted December 19, 2011.
- Royal College of Psychiatrists