Institute of Psychiatry, Kings College London, London, UK
Psychology Dept., Institute of Psychiatry, Kings College London, UK
Department of Psychological Medicine, Institute of Psychiatry, Kings College London, UK
Department of Psychological Medicine, Institute of Psychiatry, Kings College London, UK, and Department of Psychiatry and Neuropsychology, Maastricht University, the Netherlands
Department of Psychological Medicine, Institute of Psychiatry, Kings College London, UK
Correspondence: Dr Caroline Brett, CASPD, Salomons, David Salomons Estate, Broomhill Road, Southborough, Tunbridge Wells, Kent, TN3 0TG. Email: cb239{at}canterbury.ac.uk
|
|
|---|
Aims To develop and validate a semi-structured interview (the Appraisals of Anomalous Experiences Interview; AANEX) to assess (a) anomalous experiences and (b) appraisal, contextual and response variables.
Method Following initial piloting, construct validity was tested via cross-sectional comparison of data from clinical and non-clinical samples with anomalous experiences. Interrater reliability was also assessed.
Results Scores from AANEX measuring appraisals, responses and social support differentiated the clinical and nonclinical groups. Interrater reliability was satisfactory for 65 ofthe 71 items. Six items were subsequently amended.
Conclusions The AANEX is a valid multidimensional instrument that provides a detailed assessment of psychotic-like experiences and subjective variables relevantto the development of a need for clinical care.
|
|
|---|
|
|
|---|
|
View this table: [in a new window] | Table 1 Demographic details of groups |
The diagnosed group included 14 people recruited from an in-patient unit and linked community team specialising in the treatment of people with a first or second episode of psychosis (Lambeth Early Onset Team, LEO), and 21 people recruited via a specialist tertiary service providing psychological interventions for out-patients with psychosis (Psychological Intervention Clinic for Outpatients with Psychosis; PICuP), both in the South London and Maudsley Trust, UK.
The group with at-risk mental state was recruited through Outreach and Support in South London (OASIS), also based in the South London and Maudsley Trust, a clinical service for people meeting the Personal Assessment and Crisis Evaluation Clinic (PACE) criteria for an at-risk mental state (Phillips et al, 2000).
The undiagnosed group was recruited from multiple sources through advertisement to obtain a sample with as much variety in socio-economic and cultural background as possible. All volunteers were screened for suitability with a self-report questionnaire which enquired about lifetime incidence of the range of anomalous experiences that make up the inventory section of the AANEX. Only individuals with at least occasional experiences of any Schneiderian first-rank symptom which was not directly related to drug use and occurred under conditions of clear consciousness were invited to participate. Only those whose anomalous experiences had commenced more than 5 years previously but who had never been treated or sought clinical help were included. This was to distinguish this group from those with at-risk mental state by minimising the likelihood that they would develop a need for care relative to these experiences in the future.
Exclusion criteria for all three groups were inability to speak fluent English, a history of neurological problems, head injury or epilepsy, evidence of current substance dependence and estimated current IQ<70.
The undiagnosed and diagnosed groups did not differ significantly significantly in age; however, those with at-risk mental state were significantly younger than the undiagnosed group (mean difference 0.362, 95% 0.201–0.524, P<0.001), as a result of the selection criteria (i.e. anomalous experiences having been present for at least 5 years). There were no group differences in gender ratio or ratio of British Whites to other ethnic categories. There were significant group differences in estimated IQ (Bonferroni corrected), with the undiagnosed group having a higher mean IQ than either of the clinical groups (undiagnosed>diagnosed, mean difference 26.4, 95% CI 10.2–42.6; P<0.001; undiagnosed>at-risk mental state, mean difference 16.9, 95% CI 33.3–0.48, P=0.04).
Development of the AANEX
The AANEX was developed in sequential stages, beginning with exploratory
in-depth interviews. Initially, a preliminary interview schedule was created
to explore the variables suggested to be pertinent by the cognitive models
(e.g. Fowler, 2000;
Garety et al, 2001)
and to gather information that might suggest additional dimensions of
interest. Interviews were then carried out with six participants with a
history of anomalous experiences, three of whom had been treated for a
psychotic disorder and three of whom had not.
On the basis of the information gained from the preliminary interviews, a more structured interview schedule was developed. The in-depth interviews demonstrated that peoples appraisals and responses do not remain static but change over time. For this reason, a format was developed that enabled assessment of more than one time period in an individuals life, including the first onset and current experiences, and also other intermediate time periods. Sets of items can be repeated to yield scores for each relevant time period and/or each particular form of anomalous experience that is reported, giving different response sets.
The next stage involved the piloting of this interview schedule on 10 participants, including individuals diagnosed with a psychotic disorder, those considered to be at risk of psychosis and those who had not received a diagnosis.
All elements of the interview schedule were tested repeatedly in a process of iterative modification and trial, until a final version was reached, which only incorporated useful and reliable questions that yielded easily and usefully codable responses. Particular attention was paid to establishing a flexible structure that would allow responses of different groups to specific experiences to be compared. The development of the scoring scheme, using ordinal (rather than dichotomous) ratings and open-ended questions, aimed to reflect and capture the continua of belief, feelings and responses expressed by the participants.
A cross-sectional pilot comparison was then carried out between a sample of undiagnosed participants (n=8) and diagnosed participants (n=8), using the final version of the interview. Results included two trends towards significant differences in types of appraisal between the two groups, despite the small samples. The interview was acceptable to all participants. Copies of the AANEX Interview and the users manual are available from C.M.C.B.
Components of the AANEX
AANEX-Inventory
The first section of the AANEX is the inventory, which includes items
reflecting Schneiderian first-rank symptoms and anomalies of perception,
cognition, affect and individuation (sense of distinction
between self and other), as well as some paranormal experiences.
This was generated by drawing on experiences enquired about in the Wisconsin
Manual for Assessing Psychotic-like Experience WMAPE;
Kwapil et al, 1999),
the Comprehensive Assessment of At-Risk Mental States (CAARMS; PACE clinic,
2000 version; Yung, 2000) and
items taken from the Bonn Scale for the Assessment of Basic Symptoms (BSABS;
Gross et al, 1987),
which were reported to be predictive of subsequent psychotic illness
(Klosterkotter et al,
1996), as well as additional experiences drawn from the accounts
of participants in the early stages of piloting.
Particular emphasis was placed on distinguishing experiences from their appraisals but at the same time developing probe questions that allowed for endorsement of the experience by a range of individuals with differing interpretations. Several experiences were included which might usually be considered examples of delusional content but which emerged during the pilot phases as common, relatively autochthonous experiences (e.g. the sense of revelation of insights or personal missions). The inventory generates two sets of scores: lifetime and state. For the lifetime scores each item is rated between 1 and 5 based on the frequency or pervasiveness of the experience across an individuals lifetime. The data can be summarised into five lifetime component scores, derived from a principal components analysis (further details on request).
The state scores capture an individuals experiences during a specified period, such as the time of assessment, or the period when particular experiences first began. A subset of 19 key items (Appendix) are rated between 0 and 2 (absent, marginal and present) at each selected time point and can be summarised into four state component scores. For each component the scores of each of the 19 items were multiplied by the correlation between the item and the component, and the results summed to yield a continuous component score. The anomalous experiences reported at each time point are then used to anchor the second section of the interview.
AANEX-CAR
The second section assesses appraisals, context and response
(AANEX–CAR) pertaining to particular anomalous experiences endorsed from
the inventory. It can also be used independently from the inventory to explore
anomalies elicited with other clinical instruments. Responses are rated on
Likert scales ranging from 0 to 2, 1 to 5, or 0 to 5. Verbatim responses can
be recorded for all items for qualitative analysis.
Domains relevant for assessment were initially identified on the basis of psychological models of the development of psychosis published by Bentall (1990), Garety et al (2001) and Morrison (2001). Further domains were subsequently derived from the in-depth and pilot interviews. The variables assessed by the final version of the AANEX can be seen in Table 2. Probe questions used to elicit the data are shown in the data supplement to the online version of this paper.
|
View this table: [in a new window] | Table 2 Summary of domains and items comprising the AANEX, and the scoring scheme for each item.1 |
The format is flexible: subsections can be omitted to constitute a brief form which assesses a persons current style of appraising and responding to anomalous experiences (administration time 10–15 min). The full format, which is reported in this paper, provides a comprehensive assessment of an individuals history of experiencing different types of anomalies, and the changes in their interpretation and response style from first onset to the present (administration time varies widely depending on the length of the individual history and range of experiences assessed).
Procedure
Participants were administered the AANEX as part of a wider battery of
tests. Interviews were carried out in person, and were tape-recorded with the
consent of the participants. Initial ratings made at the time of the interview
were checked on the basis of the taped interviews.
A subset of participants (n=9) were administered the AANEX over two sessions (no more than a week apart) when the interview was too long or tiring to be completed in a single sitting. The administration time varied from 45 to 300 min (mean 130 minutes), depending upon the individuals style of response, and the number of different anomalies and time points to which the interview questions were anchored. The number of response sets ranged from 1 to 18, with a mean of 3.7 per person. All participants were paid an honorarium for their time.
Statistical analyses
Since all of the variables being tested were ordinal (either rated between
0 and 2 or 1 and 5), the group comparisons were made using multinomial
logistic regression analyses (Tabachnick
& Fidell, 2001). The responses of the three groups were pooled
and group was entered as two dummy variables, with the undiagnosed group as
the reference category. P values of predictor variables were based on
the Wald statistic. Since analysis of the AANEX data needed to account for the
fact that repeated observations within participants (i.e. appraisals of
different kinds of anomalies or at different time points) were not
independent, robust standard errors were used to account for the correlation
between repeated observations from the same participant. The group scores for
the interview variables in Table
3 have been summarised using means and standard deviations,
treating the scales as interval data for this purpose only. Statistical
analyses were conducted using STATA version 8.1.
|
View this table: [in a new window] | Table 3 Summary of item scores for the three groups of participants |
|
|
|---|
The raters scored the interviews from tapes on the basis of the written scoring manuals following a brief training session. This involved both raters initially being given the same transcribed interview to score. Their scores were checked and any areas of concern were discussed and clarified. The raters were masked to the diagnostic status of the participants.
The degree of agreement between C.M.C.B. and those of the independent raters (treated as one data-set) was analysed pairwise using weighted kappa. The weight matrices used for the comparison of ratings were determined for each variable, taking into account the rating scale, whether it was ordinal and the relative intervals between each anchor on the scale.
AANEX–inventory
The average kappa for all 40 items was 0.67, which can be interpreted as
substantial agreement; 92.5% of the items had at least fair agreement
(>0.4), 42.5% had substantial agreement (>0.6) and 17.5% had almost
perfect agreement (>0.8).
Of the 40 items, receptivity, reference experiences and thought pressure achieved weighted kappa <0.4. The raw percentage agreements for these three items were 62%, 79% and 54% respectively. The reference experiences items ranged between 3 and 5 in the data-set compared; combined with the substantial raw agreement, this suggests that the low kappa value might have reflected the incomplete range of scores being represented. However, the receptivity and thought pressure scores in the data-set ranged between 1 and 5 and 1 and 4 respectively, suggesting that the kappa value was not overly conservative.
Subsequently, the scoring guidelines were altered to facilitate the rating of these items. Further assessment is necessary to establish the interrater reliability of the revised version. The data reported here were collected by C.M.C.B. only.
AANEX–CAR
The level of agreement for the variables assessing the categories of
appraisal, dimensions of appraisal (valence, dangerousness, externality and
agency), emotional response, context and implications did not fall below 0.4;
65% had at least substantial agreement and 35% almost perfect agreement.
The subsection assessing cognitive and behavioural response to anomalies contained three items that did not achieve satisfactory interrater reliability: reappraisal, rumination and neutral response. Of these, the reappraisal item showed raw agreement of over 80%, whereas the latter two items showed raw agreement of 66% and 58% respectively, suggesting that amendments are required to either the scoring guidelines or the categorisation of responses to this section. The scoring guidelines have subsequently been amended and further work is needed to establish the current reliability of these rating categories. The other three categories in this section (avoidance, cognitive control and immersion) demonstrated fair to substantial agreement. Data from the rumination and neutral response categories have not been included in the analyses currently reported. Future data generated by the probe item may be analysed to identify alternative schemes of categorisation.
Validity
Content validity was addressed by developing items on the basis of both the
psychological literature and a range of existing clinical measures, as well as
in-depth interviews with a range of individuals having anomalous experiences.
It was not feasible to assess concurrent validity since no single existing
instrument measures the same variables. The cross-sectional study reported
here assesses the construct validity of the AANEX by exploring whether the
interview can distinguish those with and without a need for care in the
context of psychotic-like experiences. Specifically, based on the model of
Garety et al (2001),
good construct validity would be demonstrated by:
Therefore, cross-sectional analyses relevant to these predictions are reported, based on a subset of the full data generated by the AANEX.
Group comparisons
Table 3 summarises the item
scores for the three groups and the results of the regression analyses. The
state component scores were entered as covariates in all the
analyses. The following sections report the analyses carried out to test
construct validity via the predictions of the model of Garety et al
(2001).
Emotional response
There was a significant main effect of group on negative emotional
response, which was accounted for by significantly higher odds of distress in
the diagnosed and at-risk mental state groups than the undiagnosed group
(diagnosed OR=4.01, 95% CI 2.18–7.37; at-risk mental state, OR=2.94, 95%
CI 1.62–5.33). There was also a significant main effect of group on
positive emotional response, reflecting significantly higher scores in the
undiagnosed group than both the diagnosed group (OR=0.21, 95% CI
0.12–0.40) and the group with at-risk mental state (OR= 0.28, 95% CI
0.12–0.64). However, there was no significant effect of group on neutral
arousal (P=0.341).
These findings were corroborated by the participants self-ratings for anxiety and excitement in response to their anomalies. There was a significant main effect of group on anxiety, reflecting increased odds of higher anxiety in both clinical groups relative to the undiagnosed group (diagnosed: OR=3.11, 95% CI 1.65–5.86, P<0.001; at-risk mental state OR=2.22, 95% CI=1.20–4.10, P=0.011). Conversely, there was a significant main effect of group on excitement, reflecting lower odds of higher excitement in the clinical groups relative to the undiagnosed group (diagnosed, OR=0.28, 95% CI 0.15–0.52, P<0.001; at-risk mental state OR=0.40, 95% CI 0.19–0.84, P=0.016).
In summary, the first prediction was partially borne out in that the diagnosed and at-risk mental state groups reported greater distress and less positive affect in relation to anomalies, but not greater arousal than the undiagnosed group.
Dimensions of appraisal
There was a significant effect of group on the appraised valence of
anomalies, with the undiagnosed group reporting more positive appraisals than
the diagnosed (OR=0.19, 95% CI 0.11–0.32) or at-risk mental state groups
(OR=0.37, 95% CI 0.20–0.66). There was also a significant group effect
on appraised dangerousness of anomalies, with the diagnosed group appraising
their experiences as more dangerous to them than the undiagnosed group
(OR=2.85, 95% CI 1.60–5.06, P=0.01). The diagnosed group was
also more likely than the undiagnosed group to appraise their experiences as
being caused by some agency rather than an impersonal cause (OR=2.36, 95% CI
1.22–4.55, P=0.01).
The diagnosed group was associated with increased odds of making an external appraisal relative to the undiagnosed group (OR=2.08, 95% CI 1.16–3.74, P=0.01). However, when the state component scores were incorporated into the regression model there was no longer any significant predictive value for the diagnosed group and the group with at-risk mental state had reduced odds of an external appraisal, although this was only marginally significant (OR=0.56, 95% CI 0.31–1.00, P=0.05).
In summary, the prediction was predominantly fulfilled since the clinical groups appraised anomalies as more negative, dangerous and personally caused than the undiagnosed group. However, the clinical groups did not appraise anomalies as being more externally caused when the types of anomalies were controlled for.
Categories of appraisal
Group contributed significantly to the prediction of four of the categories
of appraisal. The diagnosed group was significantly more likely to make a
biological appraisal (OR=2.39, 95% CI 1.05–5.45,
P=0.039), and an other people appraisal (OR= 9.01, 95%
CI 4.02–20.22, P<0.001) than the undiagnosed group.
The diagnosed group was significantly less likely than the undiagnosed group to make a psychological appraisal (OR= 0.34, 95% CI 0.15–0.76, P=0.008). Both the clinical groups were less likely to make a normalising appraisal than the undiagnosed group (diagnosed, OR=0.16, 95% CI 0.07–0.40, P<0.001; at-risk mental state, OR=0.27, 95% CI 0.11–0.64, P=0.003).
There was no group effect on the likelihood of making a drug-related (P=0.35), no interpretation (P=0.23), supernatural (P=0.19) or spiritual appraisal (P=0.28). For the latter category, there was a significant effect of group (reflecting increased odds of spiritual appraisal in the undiagnosed group compared with both clinical groups) before the state component scores were entered into the model, indicating that group differences were secondary to the association between particular types of anomaly and this type of appraisal.
In summary, the prediction was partially fulfilled, since the diagnosed group made more other people, and less psychological appraisals than the undiagnosed group. However, the diagnosed group was more likely to make biological appraisals and there was no difference in the likelihood of making supernatural or drug-related appraisals between the two groups.
Perceived controllability and social support/understanding
The undiagnosed group was more likely to report higher perceived control
over their anomalies than the diagnosed (OR=0.42, 95% CI 0.21–0.84,
P=0.014) and at-risk mental state (OR=0.43, 95% CI 0.20–0.90,
P=0.025) groups, in accordance with the prediction.
The undiagnosed group was more likely to report higher perceived social understanding relative to their experiences than either clinical group (diagnosed, OR=0.15, 95% CI 0.08–0.29, P<0.001; at-risk mental state, OR=0.28, 95% CI 0.13–0.62, P=0.002), in accordance with the prediction.
Cognitive and behavioural responses
The diagnosed group was more likely to report responses categorised as
avoidance (OR=2.31, 95% CI 1.08–4.98, P=0.03), cognitive
control (OR=3.04, 95% CI 1.33–6.92, P=0.008) and immersion
(OR=1.94, 95% CI 1.12–3.36, P=0.019), relative to the
undiagnosed group. The at-risk mental state group was also more
likely to report cognitive control (OR=7.18, 95% CI 3.08–16.73,
P<0.001) and reappraisal responses (OR=3.10, 95% CI
1.32–7.30, P=0.010) relative to the undiagnosed group.
Therefore, in accordance with the prediction, different patterns of response
were reported by the undiagnosed group compared with the clinical groups.
|
|
|---|
The results predominantly met the predictions suggested to indicate good construct validity. Out of the 20 variables to which the five predictions pertained, the clinical and non-clinical groups were differentiated in the predicted direction on 15 variables.
The undiagnosed group of participants was characterised by significantly different styles of appraisal, response and context to those in the clinical groups, as predicted by cognitive models of psychosis (Garety et al, 2001). Overall the undiagnosed participants reported appraising their experiences as relatively more positive and benign, with a more positive and less negative emotional response. Interestingly, on average this group did not report any less arousal in relation to their experiences than the clinical groups, suggesting that the reduced distress did not reflect a lack of emotional engagement with the anomalies. Another finding that may be related to the reduced distress in this group was the higher level of perceived controllability of the anomalies. However, even in this group, the mean rating for controllability was only just above minimal.
The undiagnosed participants were also less likely to report avoidant responses or to employ cognitive control strategies than both the clinical groups. Although they were less likely than the diagnosed group to act on the basis of their experiences (the immersion category), they were also less likely to reappraise their experiences than at-risk mental state participants, suggesting that they had found a way of appraising their experiences that was (subjectively) coherent and adaptive.
The undiagnosed participants were also much less likely than the diagnosed group to form a paranoid appraisal, such as thinking that other people or agencies were causing the experiences, and were more likely to think that the experiences were caused by something psychological. However, the results did not suggest that an externalising appraisal per se marked the defining decision leading to clinically relevant psychotic symptoms: the undiagnosed participants also reported externalising appraisals such as those falling in the supernatural category.
Moreover, the results suggest that the preponderance of externalising appraisals in the diagnosed group was secondary to differences in the sorts of anomalies predominating at the time points under discussion, compared with the undiagnosed group, suggesting that certain types of anomaly tend to elicit external appraisals. Similarly, the finding that the excess of spiritual appraisals in the undiagnosed group was accounted for by the inclusion of the state component scores suggests that certain experiences, more common in the response sets of the undiagnosed group, tended to elicit spiritual appraisals. As suggested by Garety et al (2001), and demonstrated by differences in appraisal when holding the effects of type of anomaly constant, appraisals are relevant to transition from anomalous experience to clinically relevant symptom; however, the relationship appears more complex than simply turning on an externalising decision.
Overall, and taking into account variance in the kinds of experiences being described, the undiagnosed group was much more likely to consider that their experiences were part of the spectrum of normal human experience. This may reflect another characteristic of the group: the higher perceived levels of understanding of their experiences among their social group. In relation to early intervention strategies, these findings suggest that normalising approaches towards anomalies reported by those seeking help may be invaluable. Moreover, facilitating access to experts by experience, people who have had the same experiences and coped with them, could be a useful strategy for supporting the normalising approach.
Limitations
There are some limitations to the generalisability of these preliminary
findings, because of the nature of the undiagnosed sample. Relative to the
clinical groups, the undiagnosed participants were more likely to come from
White ethnic backgrounds (although there were more non-British people), which
may be relevant to their styles of response and appraisal, as well as their
social context. They also differed significantly from the diagnosed group in
terms of estimated IQ. Although this may not affect appraisal processes
per se, it might represent an additional factor influencing the
development of need for care: the undiagnosed group had higher than average
IQ, which might possibly act as a protective factor through allowing more
sophisticated appraisals and responses. However, it is not known whether this
is characteristic of all those in the general population with at least
occasional experiences of any first-rank symptom.
Furthermore, although the number and types of anomalies occurring at each time point were controlled for, the frequency or severity of any particular type was not. It is likely that variance in frequency has an impact on appraisals and response. Nevertheless, the undiagnosed group was selected on the basis of reporting comparable anomalous experiences to the clinical groups, and individuals reporting only infrequent experiences were not included.
Implications
Overall, the initial results suggest that the AANEX has the potential to
elicit information that may clarify the nature of the continuum of psychotic
and psychotic-like experiences, and the complex and multifactorial development
of distress and need for care relative to these experiences. The interview may
allow this clarification by assessing a range of components of experience
considered to vary across the continuum, such as types and frequency of
anomalous experience, interpretations, and emotional and contextual factors.
In relation to this, it has demonstrated utility as a clinical tool for
differentiating anomalous experiences from their appraisals, thereby improving
understanding of the persons subjective experience and how they might
benefit from psychological interventions.
The majority of the interview schedule was found to have good levels of reliability, indicating that the dimensions assessed and scoring schemes are robust, despite the complexity of the material being elicited. The small number of items that required amendment have been refined on the basis of the data gathered in this study, aiming for clearer, more unidimensional definition.
The form and process of the interview was acceptable to participants from both clinical and non-clinical populations, and sensitive to the differing ways in which psychotic-like experiences may be interpreted. These characteristics differentiate the AANEX from existing symptom rating scales that assess only pre-specified clinical forms of anomalous experiences (e.g. the Scale for Assessment of Positive Symptoms (Andreasen, 1984) or the Positive and Negative Syndrome Scale; Kay et al, 1987), and also from other scales suitable for populations without psychosis that do not assess relevant context, appraisal and response variables (e.g. the CAARMS (Yung, 2000) or WMAPE (Kwapil et al, 1999)). The flexible structure of the AANEX facilitates its use for either detailed investigations of individuals experiences or briefer assessments that may be repeated at several time points (e.g. pre- and post-intervention or during a follow-up period). The brief form of the AANEX is currently being piloted as a repeated assessment during a follow-up of clients of OASIS with at-risk mental state.
|
View this table: [in a new window] | APPENDIX |
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||