Background

Formal DSM [1] or ICD [2] diagnoses are now required for admission and treatment in most mental health facilities and settings. In addition the diagnoses generated for this purpose also figure prominently in treatment planning, as clinicians shape interventions to address the diagnoses assigned [3].

But research does not support unstructured interviews as reliable means to standard diagnoses. When colleagues of the same discipline working in the same clinical setting were often unable to agree about an individual's diagnosis even when they were presented with exactly the same information, researchers concluded that this situation needed to be remediated and began to develop structured interviews [3, 4].

Even experienced clinical interviewers are not reliable diagnosticians when compared which each other or when compared with structured interviews.

Tables 1 and 2 present a review of studies on agreement between clinicians' diagnoses and diagnoses generated with standardized interview procedures in child and adolescent psychiatry. These studies use generally J Cohen's kappa [5] and concern both inpatient and outpatient as well as community populations.

Table 1 Studies comparing clinical diagnoses with diagnoses generated with diagnostic interviews
Full size table
Table 2 Studies comparing clinical diagnoses with diagnoses generated with diagnostic interviews (cont.)
Full size table

Our review consistently revealed low or moderate levels of agreement, except for K-SADS. Additionally, findings showing poor agreement were generally robust across multiple methodological variations, as for instance assessing agreement at the level of broader diagnostic clusters. Agreement is usually higher for externalizing diagnostic categories than for internalizing ones and many authors suggest that diagnoses of anxiety and depression can be missed using an unstructured interview.

Research is still needed to establish the utility of diagnostic interviews in clinical settings. By encouraging clinicians to follow standard diagnostic and interviewing methods, structured interviews promote more consistent diagnostic practices and help justify therapeutic interventions and outcomes. Miller [6, 7] suggested that because the structured interview yields precise diagnostic data, appropriate treatments may be delivered earlier, leading to more rapid recovery and shorter hospital stays.

However, the instrument is not a substitute for clinical judgement. As McClellan and Werry [8] pointed out, psychiatric decision making depends on the integration of informations from diverse sources and perspectives, including the patient and family interviews, the mental status examination, collateral informants (teachers) and other treatment providers. The pre-eminent role of the clinician must be recognized and preserved.

In in-patient clinical settings, it is of particular importance that diagnoses can be reliably made. Adolescents who need admission in a psychiatric unit are often seriously disturbed and show considerable impairment. Many of them need acute admission because the assessing clinician has detected a significant suicidal risk. In this context, the project of implementing a semi-structured diagnostic interview (here the K-SADS-PL) into clinical practice has been welcomed by the staff members as well as by parents. The majority expected the instrument to provide more precise nosological data, so that diagnoses could be more reliable and appropriate treatment would be delivered earlier.

Aim

To evaluate the impact that introducing the interview had on diagnoses, we reviewed charts of the previous three years of consecutively admitted patients evaluated using routine psychiatric evaluation, before we started using the Kiddie-SADS-PL. Then we reviewed the charts of the next twelve months, patients being evaluated by adding the instrument to routine practice.

Methods

Clinical context

The adolescent unit of the Department of Child and Adolescent Psychiatry of the Landspítali University Hospital in Reykjavík, is the only psychiatric ward for adolescents in Iceland, admitting each year between 70 and 80 patients from 12 to 17 years of age, from all parts of the country. The main reasons for admission are severe behavioural and/or emotional disturbances with severe functional impairment and often suicidality (61% of cases in the period 2001–2004), 53% being acute admissions. Mean length of stay is 43 days for the period 2001–2004 (SD = 46.74). Adolescents presenting with alcohol and drug abuse as a predominant problem are referred to other service providers, such as social and child welfare services. The population admitted is culturally homogeneous and its geographic distribution throughout the eight regions of the country is representative of the general population for the same age category (12 to 17 years old). Since the unit is the only facility in the country providing psychiatric in-patient treatment for adolescents, we assume that our population is representative of the most severe range of psychiatric morbidity of the adolescent clinical population in Iceland.

Participants

Participants were all adolescents admitted to the in-patient unit of the Department of Child and Adolescent Psychiatry, Landspítali University Hospital in Reykjavík, through the years 2001, 2002, 2003 and 2004 (N = 333 admissions). Boys: 43% (n = 144); Girls: 57% (n = 189). Age: 12 to 17 years-old (Mean: 14.8; SD = 1.33), 71% are between 14 and 16.

The study was approved by the Data Protection Authority and the National Bioethics Comittee in Iceland.

Measures

Clinical diagnoses were made on the basis of admission history, mental status, nursing obervations, psychometric and psychoeducational testing and treatment course. Consensus diagnoses were used and in case of disagreement, discussion with ward staff members helped to clarify information in the chart. Diagnoses were assigned according to the symptomatology present at the time of admission.

The Kiddie-SADS interview has changed since its original publication [9] and is currently available in different DSM-IV format versions: the Kiddie-SADS-P IVR (Present State), the Kiddie-SADS-L (Lifetime), the Kiddie-SADS-PL (Present and Lifetime Version) and the Kiddie-SADS-E (Epidemiological) [10]. The present study used the K-SADS-PL, which has several strengths [11]. It has strong content validity because it was designed to tap pre-specified diagnostic criteria and includes detailed probes useful in eliciting clinically meaningful information. It is also the only instrument that provides global and diagnosis-specific impairment ratings to facilitate the determination of "caseness". In addition, the Kiddie-SADS-PL provides a clinician-friendly front and screening examination which may result in a more efficient shorter interview.

The Icelandic version of the K-SADS-PL was developed by classic translation-back translation technique with a bilingual expert committee assessing equivalence in several dimensions [12].

The inter-rater reliability of the Icelandic version was assessed by re-rating 15 randomly selected interviews [12]. Experienced and trained clinicians rated the interviews independently either by videotape or by attending the interview session (three clinical psychologists and one child and adolescent psychiatrist took part in the project). Reliability was satisfactory for most diagnostic categories (kappa = .44 – 1.00), except for Mania (.31). We also examined inter-rater reliability at the symptom level separately for each diagnostic area, with kappa values calculated for each item. Average values at the symptom level within each diagnostic category ranged from .48 (Social Phobia) to .98 (Oppositional Defiant Disorder).

Additionally, we obtained correlations of numbers of diagnostic criteria met between raters and found them satisfactory in all diagnostic categories (Pearson's r = .76 – 1.00).

Finally, we examined inter-rater reliability across main diagnostic areas surveyed in the screen interview, in order to estimate agreement in utilization of skip-out criteria. The average agreement evaluated by calculation of Kappa statistics across the diagnostic areas studied was .90 (range = .57 through 1.00).

The convergent and divergent validity of the skip-out screens and most frequent diagnoses generated with the Icelandic version of the K-SADS-PL was determined in another study using an adolescent clinical inpatient sample (N = 86) against eleven standard self-report or parent-report rating scales which had already been translated, adapted and in most cases validated in Iceland: rating scales of depression [13, 14], anxiety [15, 16], ADHD [17, 18], behavioral and other psychiatric problems [17, 19–21]. The results indicated that the Icelandic version of K-SADS-PL generates valid DSM-IV depression, anxiety and behavioral diagnoses in severely affected adolescent in-patients. Divergent validity was only partially supported in our very comorbid clinical sample.

Procedure

1. We reviewed clinical charts of the previous three years of consecutively admitted patients evaluated using routine non-structured psychiatric evaluation (N = 248). Assessments had been made with unstructured clinical interviews and consensus observation within the unit, the ICD-10 diagnoses being assigned by six experienced child and adolescent psychiatrists and appearing in the records, as well as assessment of suicidality. Diagnoses had been assigned according to the symptomatology present at the time of admission.

2. Then we reviewed charts of all consecutive admissions during the next twelve months (N = 85), patients being evaluated with the structured interview Kiddie-SADS-PL in addition to routine diagnostic procedures. As the main official diagnostic classification system in European countries is ICD-10 for both clinical and research purposes, results of Kiddie-SADS interviews algorithms have been translated into ICD-10. A few additional questions were included in the interviews for ICD-10 criteria not covered by the K-SADS-PL [22].

Two coders checked to verify accurate utilization of DSM-IV and ICD-10 algorithms for assignment of final diagnosis. Suicidality was assessed with the diagnostic interview.

In both study periods, combined diagnoses according to ICD-10 (F41.2, F90.1 and F92) were categorized as two co-morbid disorders.

The comparability of the two populations was evaluated by reviewing variables other than diagnoses: age, sex, geographic distribution, mean length of admission and several risk factors:

parents' separation or divorce

having moved or changed school more than twice during the six months prior to admission

a history of being bullied during the six months prior to admission

a confirmed history of child neglect, physical, sexual or emotional abuse

at least one parent having a confirmed history of psychiatric disease

at least one parent having a confirmed history of mental and behavioural disorder due to psychoactive substance use

These risk factors were identified during admission with clinical interviews and detailed and comprehensive evaluation of the patient's history. Only documented and reported cases of child abuse or neglect were considered. Definitions of the various maltreatment categories are those commonly accepted [23].

Six patients could not be evaluated with the Kiddie-SADS (Four with Autism or related Pervasive Developmental Disorders, one with Mental Retardation and one with severe Language Disorder), and in four cases parents didn't participate or couldn't be reached for interviews. In twelve other cases, only parents could be interviewed and diagnoses were generated by combining information collected with routine patient evaluation in the unit. In all other cases, diagnoses were generated by combining information from parents' interview with information from adolescent interview and routine patient evaluation.

The first author of the study was the only Kiddie-SADS interviewer.

The adolescents' interviews (n = 63) were conducted on the day of admission in 5 cases (7.9%), but 73 days after admission in one case (interval of time: 0 to 73 days, mean = 13.4, SD = 13.3; median = 9).

The parents' interviews (n = 75) were conducted on the day of admission in 4 cases (5.3%), but 71 days after admission in one case (interval of time: 0 to 71 days, mean = 11.0, SD = 12.9; median = 6).

Diagnoses were assigned according to the symptomatology present at the time of admission.

Statistical analysis

The Statistical Package for Social Sciences (SPSS) was used for data analysis. For comparisons between groups on categorical variables (sex, geographic area, presence of risk factors, assigned diagnoses), Chi-Square tests were applied and odds ratios were calculated. Independent-samples t-tests were conducted to compare groups on continuous variables (age, mean length of admission).

Results

Comparability of the two populations

There were no statistically significant differences (p > .05) between the two groups according to the following variables:

Age and mean length of admission (Table 3)

Table 3 Comparability of the two populations: age and mean length of admission
Full size table

Sex and 9 risk factors listed before (Table 4)

Table 4 Comparability of the two populations: sex and risk factors
Full size table

Geographic distribution: the odds ratios for the 8 different regions ranged from .29 to 2.38; the proportion of patients from Reykjavík vs. other (rural) regions slightly increased between the two study period (from 63.7% to 70.6%).

Impact on diagnoses

We observed considerable changes in the frequencies of several diagnoses and in co-morbidity rates (Table 5).

Table 5 Comparison of prevalences in clinical samples without and with K-SADS
Full size table

The mean number of diagnoses assigned to each patient admitted rose from 2.4 (SD = 1.2) to 3.4 (SD = 1.5). Only 13% of all admissions received only one diagnosis during the second period of the study, against 29% during the first period.

We observed a very significant increase in the number of patients diagnosed with depressive disorders (ICD-10 Severe depressive episode F32.2, Moderate depressive episode F32.1, Dysthymia F34.1) and with anxiety disorders (ICD-10 Phobic anxiety disorder of childhood F93.1, Social anxiety disorder of childhood/Social phobia F93.2/F40.1, Separation anxiety disorder of childhood F93.0, Overanxious/Generalized anxiety disorder of childhood F93.8/F41.1).

The number of patients diagnosed with Eating disorders F50 increased significantly.

We observed also a significant increase in the rate of patients diagnosed with psychotic and bipolar disorders (ICD-10 Bipolar affective disorder F31, Manic episode F30, Acute and transient psychotic disorders F23) and with any behavioural disorder but this was only because of Oppositional defiant disorder and other conduct disorders F91; the rate of patients diagnosed with Hyperkinetic disorders F90 didn't increase significantly between the two study periods.

There was no significant change in the number of patients diagnosed with Mental and behavioural disorders due to psychoactive substance use F10–F19, and Stress and adjustment disorders F43.

The rate of suicide attempts or self-harm didn't change significantly between the two periods but the rate of patients detected with suicidal thoughts increased dramatically with the use of the semi-structured diagnostic interview.

Discussion

This study was conducted to assess the impact of implementing a structured interview on diagnoses, in a clinical population of severely affected adolescents presenting a range of symptoms that suggest multiple diagnostic possibilities. The results show considerable increase in the rates of patients diagnosed with several main diagnostic categories (e.g. depressive, anxiety, bipolar and disruptive disorders) suggesting that those disorders were likely under-reported when using unstructured conventional routine assessment procedures. The rate of co-morbidity increased markedly. Using the structured diagnostic instrument, the non-specific diagnoses "Other" or "Unspecified" were much less frequently assigned and diagnoses were made that had probably been missed before. Importantly, the interview helped to detect suicidal ideation.

Under-evaluation of depressive disorders in a clinical population has been reported before [22], as well as under-detection of anxiety disorders [24] or bipolar disorders [25, 26].

But we must be aware that there may have been changes of the patients' characteristics between the both time periods of our study. This bias is however limited by the fact that several variables other than diagnoses have been reviewed to evaluate the comparability of the two samples.

Additionally the first author of the study was the only Kiddie-SADS interviewer, which could constitute a bias. He was also one of the clinicians assigning diagnoses during the first period, which limits the validity of our comparisons. These biases are however limited by the fact that an evaluation of inter-rater reliability indicated a high rate of agreement between the interviewer and other experienced clinicians.

It has been our experience that adding the standardized diagnostic instrument allowed more precise diagnostic evaluations; some authors [6, 7] have suggested that this may lead to more appropriate treatment delivery. Despite the time needed for interviews, patients and their families usually reacted positively, feeling satisfied that they were evaluated thouroughly, which was in line with other studies [27]. Traditional psychiatric evaluations and psychodynamic formulations were not abandonned, but the highly detailed symptomatic assessment helped the staff and therapists to set up cognitive and behavioural interventions during and after admissions.

In the present study, the introduction of the structured diagnostic interview has led the clinician assigning diagnoses to use more extensively the concept of co-morbidity, according to DSM-IV classification diagnostic system, and abandon the ICD-10 philosophy of emphasizing one main diagnosis in each patient.

Conclusion

This study provides arguments for the utility of diagnostic interviews in inpatient clinical settings. The major differences in diagnostic assignment rates between the two periods are in line with other research findings that have suggested a mismatch between diagnoses in practice and diagnoses in research. A useful objective for future studies will be to understand this mismatch and how to address it. Major changes in diagnostic rates underline the need for better understanding of factors that affect clinical diagnostic assignment in "real world" settings.