The British Journal of Psychiatry
Diagnostic stability in a Dutch psychosis incidence cohort
Natalie D. Veen, Jean-Paul Selten, Diede Schols, Winfried Laan, Hans W. Hoek, Ingeborg van der Tweel, Rene S. Kahn


Background No study outside the UK has examined the diagnostic stability of psychotic disorders in a population-based sample.

Aims To determine diagnostic stability in a Dutch population-based psychosis incidence cohort, to examine the frequencies of diagnostic shifts to and from schizophrenic disorders and to report the revised relative risks of schizophrenic disorders for immigrants.

Method A 30-month follow-up study assessed the cohort (n=181) by means of face-to-face diagnostic interviews.

Results Diagnostic stability of schizophrenic disorders was high (91%), but lower for other psychotic disorders. At follow-up, the initial diagnosis was adjusted to schizophrenic disorder more often than that the reverse occurred. Almost half (49%) of the patients who were not initially diagnosed as having a schizophrenic disorder received this diagnosis at follow-up. The relative risks for most immigrant groups were stable.

Conclusions Schizophrenic disorders are underdiagnosed, rather than overdiagnosed, at first presentation.

Although stability of diagnosis over time is an important issue in psychotic disorders, there have been only two studies of population-based samples, both conducted in the UK (Amin et al, 1999; Goater et al, 1999). We therefore conducted a follow-up study in The Netherlands of a population-based incidence cohort recruited in The Hague (Selten et al, 2001) and re-diagnosed all cohort members 30 months after their first contact. The primary aim of our study was to report diagnostic stability, defined as the proportion of patients who received a follow-up diagnosis in the same main category as in the incidence study. Second, we examined the frequencies of two particular diagnostic shifts, namely the shift from schizophrenic disorder (DSM–IV categories schizophrenia, schizophreniform or schizoaffective disorder; American Psychiatric Association, 1994) to any other category, and the shift the other way round. Third, we report the revised incidence rates of schizophrenic disorders and the revised relative risks for immigrant groups.


Incidence study

Full details of the recruitment of the incidence cohort have been described by Selten et al (2001). Briefly, all people aged 15–54 years living in The Hague who consulted a physician for the first time about a (suspected) psychotic disorder during the period April 1997 to April 1999 were referred to the study. Physicians and psychiatrists in the psychiatric hospitals and out-patient clinics were informed repeatedly about the study, as were those working in the prison, the addiction treatment centres and the general hospitals and more than 200 general practitioners. Patients with a substance-induced psychotic disorder were excluded. A resident in psychiatry conducted a diagnostic interview, the Comprehensive Assessment of Symptoms and History (CASH; Andreasen et al, 1992), and a research psychiatric nurse interviewed a key informant for each patient, using the Instrument for the Retrospective Assessment of the Onset of Schizophrenia (IRAOS; Häfner et al, 1992). An official interpreter was asked to help in the administration of the CASH or IRAOS, if necessary. Additional information was obtained from the treating physician or retrieved from the patient's medical file. The researchers wrote a history of the patient's illness, omitting any clue to the patient's ethnicity. This history was discussed during a diagnostic meeting, which included the researchers and two psychiatrists. The latter made a DSM–IV diagnosis. The incidence cohort consisted of 181 patients.

Follow-up study

Two and a half years (mean 30.2 months, s.d.=3.7) after the first contact the patients were approached for a repetition of the diagnostic assessments. The resident in psychiatry (N.D.V.) interviewed the patients using a follow-up version of the CASH (CASH–UP; Ho et al, 1998) and obtained information from the treating physician and the patient's medical file. The research nurse collected key data from informants using the IRAOS–UP, a modified version of the IRAOS. If necessary, an interpreter assisted in the administration of interviews to participants who were not native Dutch speakers. As in the earlier study, the researchers used all available information to compile a history, omitting the initial diagnosis and the patient's ethnicity. The procedure of the diagnostic meeting was identical to that of the incidence study. J.-P.S. participated in all of the meetings of both the incidence and follow-up studies.

Three members of the original cohort could not be traced, two had died, seven refused to participate in the follow-up study, and one had insufficient information in her medical file. Thus, for 168 participants there was sufficient information available on which to base a diagnosis at the second assessment. For 99 patients information was available from three sources (CASH–UP, IRAOS–UP and the medical file), for 40 patients information was available from CASH–UP and the medical file, for 5 patients information was available from IRAOS–UP and the medical file, and for 24 patients information was available from the medical file and the treating physician. It was not possible to obtain key-informant data for 64 patients; in 25 of these patients this was due to lack of family or friends. There was no association between the number of data sources used and diagnostic stability (χ2-test, P=0.39).

Definition of immigrant groups

Four groups of immigrants were delineated: those from Morocco, Surinam, Turkey and other countries. First-generation (i.e. those not born in The Netherlands) and second-generation (Dutch-born) immigrants were combined into one group. People born in The Netherlands and whose parents were born in The Netherlands are referred to as native Dutch.

Data analysis

Diagnostic stability

Diagnostic stability was defined as the proportion of patients whose diagnosis at follow-up was in the same main category as in the incidence study. Four main categories were delineated:

  1. schizophrenic disorders (including DSM–IV categories schizophrenia, schizophreniform disorder and schizoaffective disorder);

  2. major depressive disorder and bipolar disorder with psychotic features;

  3. other non-organic psychotic disorders (delusional disorder, brief psychotic disorder and psychotic disorder not otherwise specified);

  4. organic psychotic disorders (psychotic disorder due to a general medical condition and substance-induced psychotic disorders).

The diagnostic stabilities of brief psychotic disorders and schizophreniform disorders were evaluated separately.

Diagnostic shift towards and away from schizophrenic disorders

The diagnostic shift away from schizophrenic disorders to any of the other diagnostic main categories was evaluated and compared with the shift in the reverse direction, using McNemar's test for paired proportions. These diagnostic shifts were also evaluated for different sections of the population.

Incidence and relative risks of schizophrenic disorders

To calculate the incidence of schizophrenic disorders and the relative risks for immigrant groups, data were combined for the patients who had received this diagnosis at follow-up (n=125) and for the 8 patients who had received this diagnosis at the initial assessment but could not be assessed in the follow-up. The incidence after exclusion of the 36 patients who were not admitted to hospital early in the course of their disorder was also calculated. For the crude incidence rate, the number of cases was divided by the number of person-years at risk (same denominator as in the incidence study). This rate was standardised by direct standardisation for age and gender to the Dutch population on 1 January 1998. In order to compute 95% confidence intervals a Poisson distribution was assumed (MacMahon & Trichopoulous, 1996). Age-adjusted relative risks for schizophrenic disorders in immigrant groups, by gender and generation, were computed with Poisson regression analysis using EGRET (Cytel Software, Cambridge, MA, USA).


Diagnostic stability

Table 1 shows the diagnostic stability of the main diagnostic categories. In 120 of 168 patients (71%), the follow-up diagnosis was in the same main category as the diagnosis made during the incidence study. The diagnostic stability of schizophrenic disorders was 91%, compared with 67% for psychotic mood disorders and 30% for other non-organic psychotic disorder. As for specific diagnostic categories, the diagnosis of brief psychotic disorder (DSM–IV code 298.8) was stable in 5 of 13 patients (38%). At follow-up, 6 of the 13 patients were given a diagnosis of schizophrenic disorder and 2 a psychotic disorder not otherwise specified. The diagnosis schizophreniform disorder (DSM–IV 295.40) was stable in 5 of 29 patients (17%). As expected, most of the 29 patients (n=19; 65.5%) received the diagnosis schizophrenia or schizoaffective disorder at follow-up and thus remained within the main category of schizophrenic disorders. Four of the 29 were diagnosed with psychotic disorder not otherwise specified and one was diagnosed with amphetamine-induced psychotic disorder.

View this table:
Table 1

Stability of diagnosis across 30-month interval

Diagnostic shifts towards and away from schizophrenic disorders

In Table 2 the diagnostic shifts to and from schizophrenic disorders is shown for the native Dutch and immigrant groups. Of the 67 patients who were initially not diagnosed with a schizophrenic disorder about half (n=33) received this diagnosis at the follow-up assessment. The diagnostic shift from any other diagnosis to the main category of schizophrenic disorders occurred significantly more often than the shift from schizophrenic disorders to any other diagnosis (33 v. 9 patients; McNemar's test Z=12.6, P<0.001). In all sections of the population there was an increase in schizophrenic disorders, except in the Surinamese group, where the diagnostic shift to and from schizophrenic disorder was the same (n=3 each way). For Turkish immigrants the increase was especially marked, with the diagnosis of 6 of 10 patients being changed to a schizophrenic disorder at follow-up and none of the previous diagnoses of schizophrenic disorder being changed to another diagnosis. Owing to the small size of the groups, it was not appropriate to test for differences between the groups.

View this table:
Table 2

Shifts from and to the diagnosis of schizophrenic disorder after 30 months' follow-up, categorised by immigrant group

Incidence rates of schizophrenic disorders

The revised crude annual incidence rate of schizophrenic disorders in The Hague was 2.6 (95% CI 1.8–3.7) per 10 000. The difference between the crude and the standardised incidence rates was minimal. The annual crude incidence rate after exclusion of the 36 patients who were not hospitallised early in the course of their disorder was 1.9 (95% CI 1.5–2.3) per 10 000.

Relative risks for immigrant groups

Table 3 shows the (5-year) age-adjusted relative risks for schizophrenic disorders in immigrant groups, by gender and generation. For almost all groups there was little difference from the risks reported in our earlier study. An exception is the revised relative risk for Turkish-born men, which was found to be significantly increased.

View this table:
Table 3

Age-adjusted relative risks of schizophrenic disorder for immigrant group, diagnosed after 30 months' follow-up, by gender


This study showed a high diagnostic stability for the main category of schizophrenic disorders. Furthermore, almost half of the patients in this cohort who were initially not diagnosed as having a schizophrenic disorder were found to have this disorder at follow-up. There were only minor changes in the relative risks for immigrant groups.

Interpretation of diagnostic shifts

There are different sources of diagnostic instability, which include subject variance (true changes in the patient), information variance (e.g. more information available at the follow-up assessment), observation variance (different interpretations of same stimuli) and criterion variance (e.g. two observers use different criteria for diagnosing a delusion) (Spitzer et al, 1975). In order to reduce observation and criterion variance, we used similar diagnostic instruments at both assessments, the same procedures at the diagnostic meetings and the same criteria for classification. However, a limitation of the study was that the diagnosticians at the follow-up assessment were not masked to the purpose of the study. Most ‘new’ cases of schizophrenic disorder at follow-up had received the diagnosis ‘ psychotic disorder not otherwise specified’ at baseline, a diagnosis that was often made because the information was insufficient for a specific diagnosis. Consequently, a likely explanation for many diagnostic shifts is that the patient (or a relative) disclosed more information pertinent to the schizophrenia syndrome after the initial assessment. This might also explain the relatively high rates of diagnostic shift to schizophrenic disorder for Turkish and Moroccan immigrants. At the initial assessment the researchers sometimes had difficulties in gathering sufficient information from those who did not speak Dutch.

A second explanation for diagnostic changes is that they were necessitated by true changes in the clinical picture. One patient, for example, was initially diagnosed with a bipolar disorder on account of a depressive and a manic episode with mood-congruent psychotic symptoms. During the follow-up period, however, his mood was normal but he suffered from acoustic hallucinations and negative symptoms.


One clinical implication of this study is that the use of an extensive diagnostic protocol makes it possible to diagnose schizophrenic disorders reliably at their first presentation. This is important because early treatment and psycho-education of patients and their families may improve the course of the disorder (Lieberman & Fenton, 2000; Malla et al, 2002). Moreover, physicians should be aware that even if a patient with a first episode of psychosis is diagnosed as having a disorder other than a schizophrenic disorder, there is a distinct possibility that this diagnosis will be adjusted to a schizophrenic disorder at a later date. It is therefore important that these patients are not lost from sight.

There are also implications for research. First, studies on risk factors and course of schizophrenic disorders should include all patients with a first psychotic episode and not only those initially given a diagnosis of schizophrenic disorder. Second, first-contact rates constitute an underestimation of the true incidence rates. The revised annual incidence rate of schizophrenic disorders was 2.6 per 10 000, compared with 2.1 (95% CI 1.7–2.5) per 10 000 obtained in the incidence study. The changes in relative risks for immigrant groups were small.

Comparison with earlier reports

We replicated the main results of the Nottingham study. Amin et al (1999) and Harrison et al (1999) used similar methods and found a diagnostic stability of 83% for DSM–III–R schizophrenia after 3 years, with no significant differences between natives and immigrants from the Caribbean. Goater et al (1999) carried out similar research in London and also reported no significant association between ethnicity and diagnostic stability. Other studies in this field included only hospitalised patients, and none compared natives with immigrants (e.g. Tsuang et al, 1981; Fennig et al, 1994; Rabinowitz et al, 1994; Chen et al, 1996; Schwartz et al, 2000; Forrester et al, 2001).

Strengths of the study

The strengths of this study lie in its population-based design and the extensive diagnostic procedures, including direct patient interviews and direct key-informant interviews at initial and follow-up evaluations. The cohort was large, and enough information was available to enable a reliable follow-up diagnosis to be made for 93% of the original cohort. Finally, the diagnosis was made by psychiatrists who were masked to ethnicity and the previous diagnosis.

In conclusion, the study's findings indicate that at first presentation, underdiagnosis of schizophrenic disorders is more frequent than overdiagnosis.

Clinical Implications and Limitations


  1. The stability of the diagnosis of schizophrenic disorder is high when a detailed diagnostic protocol is used.

  2. Underdiagnosis of schizophrenic disorders at first contact is more frequent than overdiagnosis.

  3. The incidence of schizophrenic disorders among some immigrant groups to The Netherlands is greater than in the native Dutch population.


  1. At follow-up a face-to-face diagnostic interview was administered to only 139 of the original cohort of 181 patients (77%).

  2. In 13 cases (7%) there was insufficient information for a diagnosis at follow-up.

  3. The diagnostic stability of substance-induced psychotic disorders could not be examined, because this diagnosis was an exclusion criterion in the incidence study.


We thank Wilma Feller, psychiatric nurse, and Jan-Dirk Blom, psychiatrist, for their cooperation with this study.

  • Received November 24, 2003.
  • Revision received June 10, 2004.
  • Accepted July 9, 2004.


View Abstract