Journal of Autism and Developmental Disorders

, Volume 44, Issue 8, pp 1918–1932

How Will DSM-5 Affect Autism Diagnosis? A Systematic Literature Review and Meta-analysis

Authors

    • Department of Health Policy and ManagementColumbia University Mailman School of Public Health
    • Office of Scholarship and Research DevelopmentColumbia University School of Nursing
  • Arlene M. Smaldone
    • Columbia University School of Nursing
  • Elizabeth G. Cohn
    • Columbia University School of Nursing
Original Paper

DOI: 10.1007/s10803-014-2065-2

Cite this article as:
Kulage, K.M., Smaldone, A.M. & Cohn, E.G. J Autism Dev Disord (2014) 44: 1918. doi:10.1007/s10803-014-2065-2

Abstract

We conducted a systematic review and meta-analysis to determine the effect of changes to the Diagnostic and Statistical Manual (DSM)-5 on autism spectrum disorder (ASD) and explore policy implications. We identified 418 studies; 14 met inclusion criteria. Studies consistently reported decreases in ASD diagnosis (range 7.3–68.4 %) using DSM-5 criteria. There were statistically significant pooled decreases in ASD [31 % (20–44), p = 0.006] and DSM-IV-TR subgroups of Autistic disorder [22 % (16–29), p < 0.001] and pervasive developmental disorder-not otherwise specified (PDD-NOS) [70 % (55–82), p = 0.01]; however, Asperger’s disorder pooled decrease was not significant [70 % (26–94), p = 0.38]. DSM-5 will likely decrease the number of individuals diagnosed with ASD, particularly the PDD-NOS subgroup. Research is needed on policies regarding services for individuals lacking diagnosis but requiring assistance.

Keywords

DSM-5Autism spectrum disorderPDD-NOSDiagnosisPublic health policy

Introduction

The prevalence of autism spectrum disorders has steadily increased over the past decade. In 2012, the Centers for Disease Control and Prevention reported the prevalence of autism spectrum disorders under the Diagnostic and Statistical Manual (DSM), Fourth Edition, Text Revision (DSM-IV-TR) as one in 88 children aged eight years old in surveillance year 2008 across the sites of the Autism and Developmental Disabilities Monitoring (ADDM) Network (Centers for Disease Control and Prevention 2012). When compared with findings for earlier ADDM surveillance years, an estimated increase in autism prevalence of 23 % was reported when compared with 2006 data (9 per 1,000 children) and an estimated increase in ASD prevalence of 78 % when compared with 2002 data (6.4 per 1,000 children) (Centers for Disease Control and Prevention 2012). Increasing rates have generated concern (King and Bearman 2009) and led to the emergence of autism as a major public health concern in the United States (King and Bearman 2009; Newschaffer and Curran 2003; Rossi et al. 2013). In addition, it is unknown whether these rates are indicative of a true increase in incidence of the disorders or are due to broader diagnostic criteria or increased awareness (Johnson and Myers 2007; Peterson and Barbel 2013).

DSM-IV-TR Versus DSM-5 Autism Diagnosis

Under the category of pervasive developmental disorders (PDD) in the DSM-IV-TR, three unique autism spectrum disorders [Autistic disorder (AD), Asperger’s disorder, and pervasive developmental disorder-not otherwise specified (PDD-NOS)] represent a wide range in symptomatology and severity. Diagnosis of these disorders relies on multiple behavioral criteria and sub-criteria which can be combined in numerous ways; in fact, there are a total of 2,027 possible combinations of criteria in the DSM-IV-TR to arrive at a diagnostic threshold for any one of these three autism spectrum disorders (McPartland et al. 2012). Notably, a diagnosis of PDD-NOS indicates a severe, pervasive impairment in the development of reciprocal social interaction coupled with either impairment in verbal or nonverbal communication skills or the presence of stereotyped behavior, interests, and activities—but criteria for another PDD or related disorder such as Schizophrenia is not met. Thus, PDD-NOS has been described as the “catch-all” autism diagnosis (APA 2012a). Although the diagnosis criteria for AD, Asperger’s disorder, and PDD-NOS are defined, researchers have found that these criteria are not consistently applied across different clinics and treatment centers (APA 2012a; Glasson et al. 2008; Matson et al. 2012; McClure et al. 2010; Robertson et al. 2013).

In May 2013, the APA published the Fifth Edition of the DSM (DSM-5) after a 14-year revision process, and one of the most controversial changes was that the DSM-IV-TR autism subgroups of AD, Asperger’s Disorder, and PDD-NOS were combined into one broad diagnosis—autism spectrum disorder (ASD) (APA 2013a). In addition, ASD in DSM-5 now includes only two main behavior categories because social interaction and communication have been collapsed into one criterion. For an ASD diagnosis, an individual must meet four broad criteria which include meeting all three distinctions of the social communication and interaction (SCI) criteria and two out of four distinctions of the restrictive, repetitive behavior (RRB) criteria. There are significantly fewer ways to arrive at a diagnostic threshold for ASD in the DSM-5, which includes only 11 possible combinations of criteria (McPartland et al. 2012). As the diagnosis of autism cannot be confirmed with a laboratory or other diagnostic test, the clinician must rely on DSM descriptive criteria as the “gold standard” for diagnosis. Therefore, changes in behavioral criteria from DSM-IV-TR to DSM-5 may have far reaching ramifications.

Objectives

Because the new DSM-5 criteria for ASD have the potential to affect the number of children and adults who currently have or may become eligible for access to care and insurance coverage, the objectives of this systematic literature review and meta-analysis were to: (1) estimate the changes in frequency of ASD diagnosis based on the proposed DSM-5 criteria; (2) determine the ASD subgroups most likely to be affected by the changes in DSM-5 criteria; and (3) present public health policy implications of implementation of DSM-5 ASD criteria.

Methods

Search Strategy and Inclusion Criteria

We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting on the studies included in our review and meta-analysis (Moher et al. 2009). The Cumulative Index of Nursing and Allied Health Literature (CINAHL), the Educational Resource Information Center (ERIC) databases (EBSCO and Dialog Classic), Web of Science, PSYCInfo, and PubMed were searched. We linked 16 sets of key words with Boolean “AND” logic, including: DSM-5/DSM 5/DSM-V/DSM V “AND” autism/autistic/Asperger’s/pervasive developmental disorder. At this phase of the search, studies were included if they were: (1) an original article; (2) written in English; and (3) published in a peer-reviewed journal (including online only and Epub ahead of print) between January 1, 2011 and March 31, 2013. This date range was chosen based upon the 2010 publication date of the first DSM-5 draft criteria for ASD.

During the screening process, two authors (KK, EC) examined article titles and abstracts for three additional content inclusion criteria (4, 5, and 6). Studies needed to (4) employ prospective or retrospective study designs; (5) compare application of DSM-IV-TR and either the 2010 or 2011 APA draft criteria for the proposed DSM-5 autism spectrum disorder (ASD) diagnosis to populations at risk for or previously diagnosed with ASD and/or one of three ASD subgroups (AD, Asperger’s disorder, or PDD-NOS); and (6) report results as raw data or percentages of subjects meeting diagnostic criteria using both sets of criteria. If it was unclear whether a study met criterion 5 or 6 based on a review of the abstract, the study was conservatively included for full-text review. The reference lists of identified studies were also hand-searched to identify additional studies that may have been missed in the electronic search.

Data Extraction

Two authors (KK, EC) independently extracted data from each study and compared results to arrive at a consensus. Information was collected on the country where the study was conducted; study design; data sources; age range; size of the sample; number diagnosed with ASD or its subgroups (if available) under DSM-IV-TR criteria; and measurement tools used. Next, information was ascertained on the change in frequency of ASD diagnosis when the DSM-5 draft criteria were applied to the same sample and/or subsamples, including number and percent reduction in diagnosis. The draft version of DSM-5 ASD criteria used in the assessment was also identified: 2010 (Mattila et al. 2011) versus 2011(You et al. 2011). For studies which also applied modified versions of the DSM-5 draft criteria to the same population, this information was collected and corresponding changes in rates of ASD diagnosis, presented in subgroups when available, was documented.

Other notable findings extracted included statistical significance, researcher remarks on specificity and sensitivity of DSM-IV-TR versus DSM-5 diagnosis criteria, and indications of which subgroups of ASD the studies suggest would be more affected. Additionally, studies’ reports of potential rates of Social Communication Disorder (SCD), a new diagnosis that appears under the Neurodevelopment Disorders subcategory of Communication Disorders in the DSM-5 (APA 2012a, b) and intended to capture individuals with symptoms of PDD-NOS (APA 2013b) were noted. Although a non-autistic disorder, SCD is characterized by verbal and nonverbal communication deficiencies that are not attributed to low cognitive ability, and symptoms include difficulty in spoken and written language as well as inappropriate responses in conversation (APA 2013b).

Quality Appraisal

For rating the scientific rigor of individual studies, we used the Quality Appraisal of Reliability Studies (QAREL) (Lucas et al. 2010). The QAREL was developed for use in systematic reviews and meta-analyses to gauge the quality of studies of diagnostic reliability. It is an 11-item checklist which explores seven principles representing the appropriateness of subjects, qualification of examiners, examiner blinding, order effects of examination, suitability of the time interval between repeated measurements, appropriate test application and interpretation, and statistical analysis of inter- or intra-rater agreement. Each item on QAREL can be answered “yes,” “no,” or “unclear.” In addition, five items include the option “not applicable.” Each author independently rated the studies and then collectively reviewed results to come to a consensus score for each study.

To inform reviewer responses to several QAREL items, it was essential to determine standards to evaluate the diagnosis of ASD under the DSM-IV-TR. Because there isn’t a universal “gold standard,” we defined our standards based on the findings of a 2013 systematic literature review (Falkmer et al. 2013). This requires (1) a multi-disciplinary team assessment of behavioral, historical, and parent-report information; (2) clinical judgment using the DSM-IV-TR or International Classification of Diseases (ICD) criteria; and (3) use of two evidence-based assessment tools: both the Autism Diagnostic Observation Schedule (ADOS) (Lord et al. 1999) and the Autism Diagnostic Interview-Revised (ADI-R) (Mazefsky et al. 2013; Rutter et al. 2002).

Data Analysis

To address the study aims, we conducted two meta-analyses. In the first pooled analysis, all studies that met inclusion criteria were included to examine the changes in frequency of ASD diagnosis based on the proposed DSM-5 criteria. Data were extracted as sample size of subjects meeting DSM-IV-TR criteria and number no longer meeting the diagnostic criteria when DSM-5 was applied and computed as the proportion of those who would no longer retain their ASD dignaosis. A pooled effect was estimated for the proportion of subjects who no longer met criteria for ASD diagnosis using a random effects meta-analysis model. Results are presented as a forest plot and heterogeneity was assessed using Cochran Q and I2 statistics. To examine differences within studies that might explain heterogeneity, we conducted sensitivity analyses by sample age, country where the study was conducted, study design, and study quality. To examine the risk of publication bias, we constructed a funnel plot.

For the second pooled analysis, we included studies that examined the differences in ASD diagnosis by DSM-IV-TR subgroup (AD, Asperger’s disorder, PDD-NOS). Data were extracted as sample size of subjects meeting DSM-IV-TR criteria and number no longer meeting the diagnostic criteria when DSM-5 was applied and computed as the proportion of those who would no longer retain their ASD dignaosis. A pooled effect was estimated for the proportion of subjects who no longer met criteria for each subgroup using random effects meta-analysis models. The heterogeneity of each model was assessed using Cochran Q and I2 statistics. Data were analyzed using comprehensive meta-analysis (CMA) statistical software (Biostat, Inc.) with results presented as forest plots.

Results

Figure 1 presents details of the literature review. A total of 418 records were initially identified in the database search phase; following removal of duplicates, 235 articles were deemed eligible for screening. After screening the titles and abstracts, 206 articles were excluded, leaving 29 eligible for full-text assessment. No additional publications were identified by hand searching the reference lists of these articles. Thirteen studies were subsequently excluded after the full-text review. Finally, three studies used the same sample, so two of these were excluded. A total of 14 studies were included in the systematic review and the first pooled analysis; seven studies that examined ASD subgroups were eligible for the additional pooled analyses.
https://static-content.springer.com/image/art%3A10.1007%2Fs10803-014-2065-2/MediaObjects/10803_2014_2065_Fig1_HTML.gif
Fig. 1

PRISMA flow diagram for the systematic literature review. *If criterion 5 or 6 was unclear based on a review of the abstract, we conservatively included the article for full-text review

Study Quality

Figure 2 summarizes the results of the quality appraisal of the 14 studies. All studies used an appropriate sample of subjects and employed an appropriate time-interval between DSM-IV-TR and DSM-5 measurement. In the majority of studies, appropriately credentialed raters conducted the behavioral observations, administered instruments, provided diagnoses (n = 10) (Dickerson Mayes et al. 2013; Gibbs et al. 2012; Huerta et al. 2012; Matson et al. 2012b, c; Mattila et al. 2011; Mazefsky et al. 2013; McPartland et al. 2012; Taheri and Perry 2012; Wilson et al. 2013), and correctly applied and interpreted the instruments or criteria for diagnoses (n = 12) (Beighley et al. 2013; Dickerson Mayes et al. 2013; Gibbs et al. 2012; Huerta et al. 2012; Matson et al. 2012b, c; Mattila et al. 2011; Mazefsky et al. 2013; McPartland et al. 2012; Taheri and Perry 2012; Wilson et al. 2013; Worley and Matson 2012). The most commonly used screening instruments were the ADOS and the ADI-R (n = 5) (Gibbs et al. 2012; Huerta et al. 2012; Mattila et al. 2011; Mazefsky et al. 2013; Wilson et al. 2013), which were consistently used in tandem across studies, followed by the DSM-IV-TR/ICD-10 Checklist (n = 4) (Beighley et al. 2013; Matson et al. 2012b; Neal et al. 2012; Worley and Matson 2012). A total of 12 measurement tools were used across studies. One study examined the effects of utilizing the ADOS and ADI-R separately versus pooling across patients and found a high variability between the two instruments (Mazefsky et al. 2013). Notably, only one study (Wilson et al. 2013) utilized all three screening instruments as specified as our standard for ASD diagnosis (Falkmer et al. 2013). Only two studies reported inter and/or intra-rater reliability (Matson et al. 2012c; Taheri and Perry 2012). The most consistent area of weakness in study quality was lack of blinding with only one study reporting that raters were blinded to the results of DSM-IV-TR (Matson et al. 2012). When evaluated by the 11 QAREL components for overall quality and rigor, only five of the 14 studies (Huerta et al. 2012; Matson et al. 2012c; Mattila et al. 2011; McPartland et al. 2012; Taheri and Perry 2012) met at least half of the applicable quality criteria.
https://static-content.springer.com/image/art%3A10.1007%2Fs10803-014-2065-2/MediaObjects/10803_2014_2065_Fig2_HTML.gif
Fig. 2

Study quality appraisal results using the QAREL checklist

Qualitative Synthesis

Study Type, Demographics, and Data Sources

Table 1 provides a descriptive summary of each study. All were observational studies. Seven studies were retrospective (Beighley et al. 2013; Huerta et al. 2012; Matson et al. 2012; Mazefsky et al. 2013; McPartland et al. 2012; Taheri and Perry 2012; You et al. 2011), six were prospective (Gibbs et al. 2012; Matson et al. 2012; Mattila et al. 2011; Neal et al. 2012; Wilson et al. 2013; Worley and Matson 2012), and one study examined one retrospective sample and a second prospective sample (Dickerson Mayes et al. 2013); for meta-analysis purposes, this is included as a retrospective study. The majority of the studies (n = 8) were conducted in the US (Beighley et al. 2013; Dickerson Mayes et al. 2013; Matson et al. 2012b, c; Mazefsky et al. 2013; Neal et al. 2012; Worley and Matson 2012; You et al. 2011). Sample sizes ranged from 25 (Dickerson Mayes et al. 2013) to 5,484 (Mattila et al. 2011) subjects; the number meeting diagnostic criteria for any ASD under DSM-IV-TR ranged from 17 (Dickerson Mayes et al. 2013) to 2,130 (Huerta et al. 2012). Samples were heterogeneous in terms of age, risk for ASD, and data sources. One study restricted its sample to toddlers (17–36 months) (Matson et al. 2012c); eight included toddlers and children between 1 and 18 years (Beighley et al. 2013; Dickerson Mayes et al. 2013; Gibbs et al. 2012; Huerta et al. 2012; Mattila et al. 2011; Neal et al. 2012; Taheri and Perry 2012; Worley and Matson 2012); two restricted inclusion to children and adults ≥4 years (Matson et al. 2012b; Wilson et al. 2013); and three included all age ranges (Mazefsky et al. 2013; McPartland et al. 2012; You et al. 2011). Most studies screened a broad population for ASD, some of whom were at risk for ASD (Beighley et al. 2013; Dickerson Mayes et al. 2013; Gibbs et al. 2012; Matson et al. 2012b, c; Mattila et al. 2011; Neal et al. 2012; Wilson et al. 2013; You et al. 2011), whereas other studies employed the DSM-5 criteria using subjects currently diagnosed with ASD under DSM-IV-TR (Huerta et al. 2012; Mazefsky et al. 2013; McPartland et al. 2012; Taheri and Perry 2012; Worley and Matson 2012). The studies also utilized a wide range of data sources. Retrospective studies included study populations from database and chart reviews (Beighley et al. 2013; Dickerson Mayes et al. 2013; Huerta et al. 2012; You et al. 2011); developmental programs and centers (Matson et al. 2012c); field trials (McPartland et al. 2012); and data from previous research studies (Mazefsky et al. 2013; Taheri and Perry 2012). Prospective studies included tertiary referral services (Gibbs et al. 2012); diagnostic, outpatient, and specialty clinics (Neal et al. 2012; Wilson et al. 2013); residential centers (Matson et al. 2012b); schools (Mattila et al. 2011); and advocacy and support groups (Worley and Matson 2012).
Table 1

Characteristics of the included studies

Author, location, study type, data sources

Sample characteristics

Measurement tools

DSM-IV-TR diagnoses (including subgroups)

DSM-5 diagnoses

Reduction in diagnoses using full DSM-5 criteriaa

Reduction in diagnoses using study-relaxed DSM-5 criteria

Quality Score

Beighley et al. (2013)

US

Retrospective database review

Outpatient clinics, schools, and community organizations

N = 459

Ages 2–18 years

DSM-IV-TR/ICD-10 Checklist

ASD-PB-C

328 ASD

219 ASD

33.2 % ASD

Not applicable

3/7

Dickerson Mayes et al. (2013)

US

(Study Population 1) Retrospective chart review

Penn State Department of Psychiatry practice site

N = 100

Ages 1–16 years

CASD

67 ASD

50 AD

17 PDD-NOS

57 ASD

50 AD

7 PDD-NOS

15 % ASD

0 % AD

58.8 % PDD-NOS

Not applicable

4/9

(Study Population 2) Prospectiveb

Penn State Children’s Hospital Behavior and Developmental specialty clinic

N = 25

Ages 1–3 years

CASD

Parent diagnostic interview

17 ASD

14 AD

3 PDD-NOS

14 ASD

13 AD

1 PDD-NOS

17.6 % ASD

7.1 % AD

66.7 % PDD-NOS

Not applicable

 

Gibbs et al. (2012)

Australia

Prospective

Tertiary referral service for diagnostic assessment for autism

N = 132

Ages 2–16 years

ADOS

ADI-R

Informal observations

Background reports from teachers and other professionals

111 ASD

59 AD

18 Asperger’s

34 PDD-NOS

85 ASD

53 AD

15 Asperger’s

17 PDD-NOS

23.4 % ASD

10.2 % AD

16.6 % Asperger’s

50 % PDD-NOS

12.6 % ASD with Relaxed SCIc

10.8 % ASD with Relaxed RRBd

4/10

Huerta et al. (2012)

International

Retrospective database review

(Study Population 1) Simon Simplex Collection

N = 2,130

Ages 2–17 years

ADOS

ADI-R

Cognitive or developmental testing

2,130 ASD

1,942 ASD

8.8 % ASD

Not applicable

6/10

(Study Population 2) Collaborative Programs of Excellence in Autism

N = 1,021

Ages 2–17 years

 

858 ASD

795 ASD

7.3 % ASD

Not applicable

 

(Study Population 3) University of Michigan Autism and Communication Disorders Center

N = 1,992

Ages 2–17 years

 

1,465 ASD

1,305 ASD

10.9 % ASD

Not applicable

 

Matson et al. (2012b)

US

Prospective

Intellectually disabled adult residents of two developmental centers

N = 330

Ages 18–88 years

DSM-IV-TR/ICD-10 Checklist

156 ASD

99 ASD

36.5 % ASD

Not applicable

4/10

Matson et al. (2012c)

US

Retrospective chart review

Louisiana’s EarlySteps Program

N = 2,721

Ages 17–36 months

BISCUIT-Part I

M-CHAT

BDI-2

795 ASD

453 AD

342 PDD-NOS

415 ASD

343 AD

72 PDD-NOS

47.8 % ASD

24.3 % AD

78.9 % PDD-NOS

Not applicable

9/10

Mattila et al. (2011)

Finland

Prospective

304 schools in Northern Ostrobothnia Hospital District

N = 5,484

Age 8 years

ADOS

ADI-R

26 ASD

15 AD

11 Asperger’s

12 ASD

12 AD

0 Asperger’s

53.8 % ASD

20 % AD

100 % Asperger’s

3.9 % ASD

0 % AD

9.1 % Asperger’s with Relaxed SCI

6/10

Mazefsky et al. (2013)

US

Retrospective file review

Prior research subjects

N = 498

Ages 5–61 years

ADOS

ADI-R

(Separately and then pooled together)

430 ASD by ADOS

488 ASD by ADI-R

345 ASD by both ADOS and ADI-R

136 ASD by ADOS

403 ASD by ADI-R

302 ASD by both ADOS and ADI-R

68.4 % ASD by ADOS

17.4 % ASD by ADI-R

12.5 % ASD by pooling across patients assessed with both instruments

Not applicable

5/11

McPartland et al. (2012)

International

Retrospective secondary data analysis

Sample from multisite field trial of DSM-IV

N = 933

Ages 1–43 years

DSM-IV field trial clinical diagnosis

657 ASD

450 AD

48 Asperger’s

159 PDD-NOS

398 ASD

341 AD

12 Asperger’s

45 PDD-NOS

39.4 % ASD

24.2 % AD

75 % Asperger’s

71.7 % PDD-NOS

Not applicable

5/9

Neal et al. (2012)

US

Prospective

University outpatient clinic in Louisiana

N = 63

Ages 3–18 years

ASD-OC

DSM-IV-TR/ICD-10 Symptom Checklist

38 ASD

21 ASD

44.7 % ASD

Not applicable

4/10

Taheri and Perry (2012)

Canada

Retrospective file review

All available files from several studies of behavioral intervention done in authors’ lab in past five years

N = 131

Ages 2–12 years

CARS

Vineland Adaptive Behavior Scales-II

129 ASD

93 AD

36 PDD-NOS

82 ASD

75 AD

6 PDD-NOS

36.4 % ASD

19.4 % AD

83.3 % PDD-NOS

25.6 % ASD with Relaxed RRB

15.5 % ASD with Relaxed SCI and RRBe

7/11

Wilson et al. (2013)

United Kingdom

Prospective

National tertiary ASD diagnostic clinic of adults without an intellectual disability

N = 158

Ages 18–65 years

ICD-10R

ADOS

ADI-R

80 ASD

61 ASD

23.7 % ASD

15 % ASD with Relaxed SCI

10 % ASD with Relaxed RRB

1.2 % ASD with Relaxed SCI and RRB

5/11

Worley and Matson (2012)

US

Prospective

Advocacy groups, support groups, schools, and an outpatient clinic

N = 208

Ages 3–16 years

DSM-IV-TR/ICD-10 Checklist

ASD-DC

180 ASD

121 ASD

33 % ASD

Not applicable

3/7

You et al. (2011)

US

Retrospective chart review

Department of Developmental Medicine in Children’s Hospital Boston

N = 163

Ages 18 months–56 years

DSM-IV clinical evaluation

135 ASD

93 AD

3 Asperger’s

39 PDD-NOS

57 ASD

56 AD

0 Asperger’s

1 PDD-NOS

57.8 % ASD

40 % AD

100 % Asperger’s

97.5 % PDD-NOS

Not applicable

3/10

The abbreviation of “ASD” under DSM-IV-TR refers to group of three diagnoses under the autism spectrum: autistic disorder, Asperger’s disorder, and Pervasive Developmental Disorder—Not Otherwise Specified, and “ASD” under DSM-5 refers to the new diagnosis of autism spectrum disorder

Quality Score QAREL indicators met/total applicable, ASD-PB-C autism spectrum disorders-problem behaviors for children, CASD checklist for autism spectrum disorder, ADOS autism spectrum screening questionnaire autism diagnosis observation schedule, ADI-R autism diagnostic interview-revised, BISCUIT-Part I baby and infant screen for children with autism traits-part I, M-CHAT modified checklist for autism in toddlers, BDI-2 battelle developmental inventory, 2nd edition, ASD-OC autism spectrum disorder observation for children, CARS DSM-IV checklist childhood autism rating scales, ASD-DC autism spectrum disorder-diagnostic for children

aIn prospective studies, all individuals were evaluated for both a DSM-IV-TR and a DSM-5 ASD diagnosis on the same day

bFull DSM-5 criteria = Must meet 3 out of 3 Criteria A—social communication and interaction (SCI) categories and must meet 2 out of 4 Criteria B—Restrictive, Repetitive Behaviors (RRB) categories

cRelaxed SCI = Must meet 2 out of 3 instead of 3 out of 3 SCI and 2 out of 4 RRB

dRelaxed RRB = Must meet 3 out of 3 SCI and must meet 1 out of 4 instead of 2 out of 4 RRB

eRelaxed SCI and RRB = Must meet 2 out of 3 instead of 3 out of 3 SCI and must meet 1 out of 4 instead of 2 out of 4 RRB

Full DSM-5 Criteria

The majority of studies (n = 11) (Beighley et al. 2013; Dickerson Mayes et al. 2013; Huerta et al. 2012; Matson et al. 2012b; Mazefsky et al. 2013; McPartland et al. 2012; Neal et al. 2012; Taheri and Perry 2012; Wilson et al. 2013; Worley and Matson 2012; You et al. 2011) utilized the 2011 DSM-5 draft criteria. There were no substantial differences in findings between studies that used the 2010 criteria (Gibbs et al. 2012; Matson et al. 2012c; Mattila et al. 2011) versus the 2011 DSM-5 criteria. When applying the full ASD DSM-5 draft criteria to study populations previously or prospectively diagnosed with DSM-IV-TR autism, which encompasses the three subtypes of AD, Asperger’s disorder, or PDD-NOS, in all studies the prevalence of ASD was reduced but varied widely. The percent reduction in ASD diagnoses using DSM-5 draft criteria ranged from 7.3 % (Huerta et al. 2012) to 68.4 % (Mazefsky et al. 2013). When individually examining samples used in the studies, four studies demonstrated reduction rates of 7.3–25 % in ASD diagnoses using DSM-5 criteria (Dickerson Mayes et al. 2013; Gibbs et al. 2012; Huerta et al. 2012; Wilson et al. 2013); seven demonstrated reduction rates of 25–50 % (Beighley et al. 2013; Matson et al. 2012b, c; McPartland et al. 2012; Neal et al. 2012; Taheri and Perry 2012; Worley and Matson 2012); and three demonstrated reduction rates of 50–68.4 % (Mattila et al. 2011; Mazefsky et al. 2013; You et al. 2011).

Consistent across studies, all individuals who met DSM-5 criteria for ASD also met DSM-IV-TR criteria for autism (AD, Asperger’s, or PDD-NOS). In half of the studies (n = 7) (Gibbs et al. 2012; Matson et al. 2012b; Mattila et al. 2011; McPartland et al. 2012; Taheri and Perry 2012; Wilson et al. 2013; Worley and Matson 2012) researchers interpreted this finding as an indication that DSM-5 criteria are more specific but less sensitive than DSM-IV-TR criteria. Furthermore, some researchers remarked that those who failed to retain their DSM-IV-TR diagnosis under DSM-5 continued to exhibit significant autism symptoms as compared to non-autistic controls (Dickerson Mayes et al. 2013; Worley and Matson 2012). Others found a similar level of symptom severity in individuals who did not fully meet DSM-5 ASD diagnostic criteria (e.g., met 3 of 3 SCI criteria but only 1 of 4 instead of 2 of 4 RRB criteria) as compared to those who did (Beighley et al. 2013; Matson et al. 2012c; Neal et al. 2012).

Study-Relaxed DSM-5 Criteria

In four studies, researchers also applied “relaxed” DSM-5 criteria to their study samples to determine the effect on ASD diagnosis (Gibbs et al. 2012; Mattila et al. 2011; Taheri and Perry 2012; Wilson et al. 2013). These included relaxing one of the SCI criteria (must meet 2 out of 3 instead of 3 out of 3; Relaxed SCI) (Gibbs et al. 2012; Mattila et al. 2011; Wilson et al. 2013); one of the RRB criteria (must meet 1 out of 4 instead of 2 out of 4; Relaxed RRB) (Gibbs et al. 2012; Taheri and Perry 2012; Wilson et al. 2013); or both (Relaxed SCI and RRB) (Taheri and Perry 2012; Wilson et al. 2013). Findings were consistent across studies with a decrease in the percent reduction rate of ASD diagnoses when modified criteria were applied. When Gibbs et al. applied Relaxed SCI criteria, the percent reduction in ASD diagnoses decreased by 46.2 %, and when Relaxed RRB criteria were applied, the percent reduction decreased by 53.8 % (2012). Wilson et al. reported remarkably similar findings with a decrease of 36.7 % with Relaxed SCI criteria, and 57.8 % with Relaxed RRB criteria (2013). When both Relaxed SCI and RRB criteria were applied, Taheri and Perry reported a 15.5 % reduction in ASD diagnosis rates (vs. 36.4 %) (2012), and Wilson et al. reported a 1.2 % reduction (vs. 23.7 %) (2013). Finally, when Relaxed SCI criteria were applied to Asperger’s and AD subgroups by Mattila et al., the percent reduction decreased from 100 to 9.1 % and from 20 to 0 %, respectively (2011).

Social Communication Disorder (SCD)

Four studies examined the proportion of subjects who met the DSM-5 criteria for SCD in their respective samples (Dickerson Mayes et al. 2013; Huerta et al. 2012; Taheri and Perry 2012; Wilson et al. 2013). The proportion of subjects who no longer met criteria for ASD but did meet criteria for diagnosis of SCD varied widely, ranging from 4.2 % (2/48) (Taheri and Perry 2012) to 63.2 % (12/19) (Wilson et al. 2013). In addition, Huerta et al. found that 75 of the 5,134 individuals (1.5 %) assessed across their three study samples would qualify for an SCD diagnosis (2012).

Quantitative Synthesis

Data representing 7,517 subjects with ASD were included in the first pooled analysis. Using a random effects model, the pooled effect suggests a 31 % [95 % confidence interval (CI) 20–44, p < 0.001] reduction in ASD diagnosis (Q = 945, p < 0.001; I2 = 98.6) when DSM-5 criteria were applied (Fig. 3). Heterogeneity between and within studies was high. To examine the high level of heterogeneity between and within studies, we performed sensitivity analyses to identify if there were meaningful variables responsible; results are presented in Table 2. Variables examined included study samples, country where the study was conducted, study design, and study quality. Of these variables, only the age of the study samples demonstrated significant differences in percent reduction of ASD diagnosis ranging from 22.7 % (n = 3; samples included children and adults) to 53.8 % (n = 1; sample included children only). Figure 4 presents the funnel plot. The open circles indicate each of the 14 individual studies included in the meta-analysis, and the filled circles indicate potentially missing studies. Its overall symmetry suggests that publication bias is not present. Further, addition of potentially missing studies does not significantly change the pooled effect.
https://static-content.springer.com/image/art%3A10.1007%2Fs10803-014-2065-2/MediaObjects/10803_2014_2065_Fig3_HTML.gif
Fig. 3

Forest plots of the 14 included studies representing the proportion of individuals who met criteria for an Autism Spectrum Disorder (ASD) diagnosis under DSM-IV-TR but not for DSM-5 ASD. Squares represent effect sizes of individual studies with extended lines denoting 95 % confidence intervals. Sizes of squares indicate the weight of each study based on sample size using random effects analysis. The diamond represents the estimated pooled effect size

Table 2

Sensitivity analyses

Variable

Number of studies

Decrease in ASD diagnosis when DSM-5 criteria applied (%)

95 % confidence interval

Age of study samplesa

 Toddlers only (ages ≤3)

1

47.8

44.3–51.3

 Toddlers/children (ages ≤18)

7

25.6

14.1–41.8

 Children only (ages 4–18)

1

53.8

35.0–71.6

 Children/adults (ages ≥4)

3

22.7

10.5–42.4

 Toddlers/Children/Adults (all ages)

2

48.1

30.9–65.8

Country

 US

8

33.4

23.5–45.0

 International

6

28.3

13.3–50.5

Study design

 Prospective

6

33.7

26.8–41.4

 Retrospectiveb

8

28.5

15.2–47.1

Study quality

 Met < half of applicable quality criteria

9

29.6

21.0–39.9

 Met ≥ half of applicable quality criteria

5

34.2

14.5–61.4

aSignificant differences between subgroups p < 0.001

bIncludes study which examined one sample retrospectively and a second sample prospectively (Dickerson Mayes et al. 2013)

https://static-content.springer.com/image/art%3A10.1007%2Fs10803-014-2065-2/MediaObjects/10803_2014_2065_Fig4_HTML.gif
Fig. 4

Funnel plot represents differences in proportion of those diagnosed with ASD using DSM-5 versus DSM-IV-TR criteria. Plot shows the standard error of the difference in proportion (Y axis) versus the reported percent not captured by DSM-5 (X axis) using a random effects model. The vertical line indicates the pooled effect estimate. The open circles indicate each of the 14 individual studies included in the meta-analysis, and thefilled circles indicate potentially missing studies. The open diamond indicates the pooled effect size and 95 % confidence interval for meta-analysis, and the filled diamond indicates pooled effect size and 95 % confidence internal when missing studies suggested by publication bias analysis are included

To determine the ASD subgroups most likely to be affected by the changes in DSM-5 criteria, the seven studies which examined the impact of DSM-5 criteria on one or more specific DSM-IV-TR subgroups within ASD were analyzed separately (Dickerson Mayes et al. 2013; Gibbs et al. 2012; Matson et al. 2012c; Mattila et al. 2011; McPartland et al. 2012; Taheri and Perry 2012; You et al. 2011). Of these, one study (Dickerson Mayes et al. 2013) reported findings of two samples which were combined for purposes of meta-analysis. Individual studies varied widely regarding the potential impact of DSM-5 criteria, particularly for Asperger’s disorder (range 17–96 % decrease) and PDD-NOS (range 25–97 %). For the pooled analysis of these studies examining DSM-IV-TR subgroups, data representing 1,227 subjects with AD, 80 subjects with Asperger’s disorder, and 630 subjects with PDD-NOS were included. Using random effects models, the pooled effects suggest a 22 % [95 % confidence interval (CI) 16–29, p < 0.001] reduction in AD diagnosis (Q = 27.7, p < 0.001; I2 = 78.4) and a 70 % (95 % CI 25–97, p = 0.01) reduction in PDD-NOS (Q = 39.4, p < 0.001; I2 = 87.3) when DSM-5 criteria were applied; however, the reduction for Asperger’s disorder was not significant [70 % (95 % CI 17–96), p = 0.38] (Q = 18.3, p < 0.001; I2 = 83.6). Heterogeneity between and within studies was high in all models. Forest plots illustrating these findings are included in Figure 5.
https://static-content.springer.com/image/art%3A10.1007%2Fs10803-014-2065-2/MediaObjects/10803_2014_2065_Fig5_HTML.gif
Fig. 5

Forest plots of Autistic disorder (top), Asperger’s disorder (middle), and pervasive development disorder—not otherwise specified (bottom) representing the proportion of individuals who met criteria for diagnosis under DSM-IV-TR criteria but not for DSM-5 autism spectrum disorder. Squares represent effect sizes of individual studies with extended lines denoting 95 % confidence intervals. Sizes of squares indicate the weight of each study based on sample size using random effects analysis. The diamond represents the estimated pooled effect size

Discussion

Implications for Future Research and Practice

A clinician’s accuracy of diagnosis is the first step in defining a treatment plan for a patient (APA 2012a). Since “a formal diagnosis of an ASD is often used as a ‘gatekeeper’ for services and support” (Wilson et al. 2013), accuracy of diagnosis is critical in order to enable these individuals to obtain needed medical, educational, and community-based services. However, the validity of the DSM-5 has been challenged by groups such as the National Institute of Mental Health (Lane, May 4, 2013). Results of our systematic review and meta-analysis highlight the need for consistency in diagnosing ASD as well as areas for improvement and clarification in the new DSM-5 ASD criteria. Notably, only in one study (Wilson et al. 2013) did the researchers employ all criteria identified by Falkmer et al. (2013) as standard for an ASD diagnosis, emphasizing the need for a universally recognized and consistently applied “gold standard” for determining an ASD diagnosis under the DSM-5.

A systematic literature review by Woolfenden et al. found that AD in the DSM-IV-TR is a fairly stable diagnosis, while Asperger’s disorder and PDD-NOS are relatively unstable, supporting the more stringent DSM-5 criteria (2012). In fact, several reviewed studies found that DSM-5 ASD criteria have a lower sensitivity but a higher specificity as compared to the DSM-IV-TR (Gibbs et al. 2012; McPartland et al. 2012; Worley and Matson 2012). Our findings show that when researchers applied modified DSM-5 criteria by relaxing SCI, RRB, or both, the percent reduction rate of ASD diagnoses in DSM-5 when compared to DSM-IV-TR consistently decreased, further supporting a decreased sensitivity under DSM-5 (McPartland et al. 2012). Nevertheless, the DSM-5 Neurodevelopmental Work Group believes “a single umbrella disorder will improve the diagnosis of ASD without limiting the sensitivity of the criteria, or substantially changing the number of children being diagnosed” (APA 2012a). However, findings of our systematic review not only suggest that sensitivity of DSM-5 ASD criteria will be reduced in order to achieve the higher specificity, but that the number of children who will be diagnosed with ASD under DSM-5 criteria will significantly decrease, with the DSM-IV-TR diagnosis of PDD-NOS likely to be the most affected. Future efforts are needed to clarify a gold standard diagnosis for ASD and then explore the sensitivity and specificity of the DSM-5 criteria to determine the extent of its impact on individuals with developmental disorders.

Although our meta-analysis demonstrated reductions in DSM-5 diagnoses for individuals who would have formerly met criteria for Asperger’s disorder, these reductions were not significant. However, since only four studies specifically examined the effect of DSM-5 on individuals who would have received a DSM-IV-TR Asperger’s disorder diagnosis, and samples in those studies were small, they may have been underpowered to detect a significant decrease. Future research is needed, therefore, to specifically examine the impact of implementation of DSM-5 on individuals who would have received a diagnosis of Asperger’s disorder.

Finally, our sensitivity analyses exploring heterogeneity between and within studies revealed that percent reduction of ASD diagnosis significantly varied by age of the study sample with the highest reduction in the study which included only children. However, since only one study examined this age group exclusively, future studies are warranted to determine if the number of diagnoses among children, in particular, will be disproportionately affected by the new DSM-5 criteria.

DSM-5’s new diagnostic category, SCD, although outside the autism spectrum, is intended to provide diagnostic coverage for those individuals with symptoms in the social-communication domain but who have never displayed repetitive, restricted behaviors or interests (Wilson et al. 2013), in essence providing them with a safety-net diagnosis. Greaves-Lord et al. (2013) indicate that SCD might be a suitable alternative diagnosis for individuals not fully meeting the new DSM-5 criteria for ASD, while Robison refers to SCD as “autism lite” (Robison, January 17, 2013). The APA describes the new diagnosis of SCD as having the ability to bring individuals’ “social and communication deficits out of the shadows of a ‘not otherwise specified’ label to help them get the services and treatment they need,” and reports that many individuals with such symptoms who may have been previously lumped under the PDD-NOS diagnosis would meet the definition of SCD (APA 2013b). However, our findings reveal significant shortcomings for SCD relative to its intended purpose; only a minority of individuals who met DSM-IV-TR criteria for PDD-NOS and fail to meet ASD DSM-5 criteria will qualify for a diagnosis of SCD. This is in contrast to the findings reported from the DSM-5 Field Trial which indicated that the decreases in ASD diagnoses at several sites were offset by movement into SCD diagnoses (Regier et al. 2013). The possible failure of SCD to “capture” individuals who would have been previously diagnosed with PDD-NOS may have future practice implications that practitioners need to consider when evaluating individuals for ASD using DSM-5 criteria. It is unclear how different SCD is from ASD, and for individuals who do receive the alternate diagnosis of SCD, which clinical care or public health services will be available and what they will quality for is not yet known. Therefore, future research is needed to evaluate the overall impact and implications of this new diagnosis and the degree to which it differs from ASD.

Public Health Policy Implications

More than half of the studies included in this systematic review and meta-analysis demonstrated ASD reduction rates between 25-68 % when applying DSM-5 criteria. Therefore, it is likely that a large number of individuals will fall outside of DSM-5 severity thresholds for receiving state-funded, school-supported, and/or insurance-covered services for their developmental, social, and communication deficiencies. The wide range in rates of reduction of ASD diagnosis in the DSM-5 is likely due to the fact that methods for diagnosing ASD were not operationalized consistently across studies, including the variety of instruments, measures, and methods used to screen for and diagnose ASD, as well as the wide range of sample ages and overall lack of a “gold standard” for diagnosis. Therefore, policy makers may need to consider whether or not to rely solely on DSM-5 ASD diagnoses when establishing guidelines for receipt of services. This may be particularly important for certain age (e.g., toddlers) and concomitant disorder (e.g., mental disabilities) subgroups. In fact, a study by Barton et al. (2013) found that toddlers were particularly liable to lose their ASD diagnosis under DSM-5 criteria, the very age when intervention may have the greatest impact. Therefore, policy makers should also consider other diagnostic thresholds or indicators for continuation of services and other benefits to guide health care plan benefits. On June 4, 2013, the State of Connecticut responded to this issue by passing a bill guaranteeing that anyone previously diagnosed with an ASD prior to DSM-5 will not lose their benefits under the state’s 2009 autism insurance reform law (Autism Speaks, May 31, 2013; State of Connecticut, June 5, 2013). This may indicate a need for states to develop policies regarding access to services for individuals previously diagnosed with an ASD, particularly those with PDD-NOS, the subgroup which our systematic review found would be most affected. Research is needed to determine if and how other states respond to implementation of DSM-5.

APA criteria explicitly state that individuals with a well-established DSM-IV-TR diagnosis of AD, Asperger’s disorder, or PDD-NOS should retain the diagnosis in the DSM-5 (APA 2013a). However, the issue of infants and children displaying Asperger’s disorder or PDD-NOS symptomatology but lacking a formal DSM-IV-TR diagnosis is not addressed. Although infants and toddlers who fail to meet developmental milestones but do not qualify for a DSM-5 ASD diagnosis would possibly still have access to early intervention services (Dickerson Mayes et al. 2013), these benefits end at the age of three years under federal law (Montes et al. 2009). The new DSM-5 criteria may affect a wide range of individuals, from childhood through adulthood, who may no longer qualify for state-supported assistance such as Medicaid, a key resource for persons with developmental disabilities (Hemp et al. 2002; Ruble et al. 2005) and the single largest public payer of behavioral health services (Mark et al. 2003; Ruble et al. 2005). Access to school-based services and private insurance benefits may also be affected as 31 states require insurers to provide coverage for the treatment of autism (National Conference of State Legislatures, August 2012). This could potentially affect individuals’ access to services that they still need and could benefit from even though they no longer have or fail to receive a formal ASD diagnosis under DSM-5. This is of particular concern for children who lack an ASD diagnosis but who would still benefit from social and educational assistance, services which would give them the best likelihood of success as an independent adult. Since obtaining a formal diagnosis of ASD is often the sole means for an individual to qualify for services (Matson et al. 2008) and early intervention is key for improved outcomes (Hu 2012; Johnson and Myers 2007), it is critical that new public health policies aimed at addressing the needs of these children be designed to fill this gap.

Limitations

Our systematic review has several limitations. Included studies were restricted to those published in the English language and in peer-reviewed journals. In addition, studies presented as posters or oral presentations at research meetings were not captured by our review.

Conclusions

Research is needed to examine the impact of implementation of the new DSM-5 criteria to work toward establishing a “gold standard” for diagnosis to prevent wide variability of diagnosis patterns across clinics and other settings. Future studies should also examine the implications of the new SCD diagnosis and whether it appropriately captures individuals formerly diagnosed with PDD-NOS and enables eligibility for state-funded services. Policy makers may want to consider alternatives to the DSM-5 criteria thresholds for receipt of services to achieve better long-term outcomes, particularly for individuals who formerly would have been captured by the PDD-NOS DSM-IV-TR autism subgroup. State intervention may be required to ensure that these individuals who may lose or fail to receive an ASD diagnosis will have continued access to public health support services; therefore, future research is needed to determine how states respond regarding insurance coverage and services for individuals without an ASD diagnosis but who still may require assistance.

Acknowledgements

Elizabeth G. Cohn’s work on this project was supported by the Robert Wood Johnson Foundation Nurse Faculty Scholars Program.

Conflict of interest

The authors declare they have no conflict of interest.

Ethical standards

This systematic literature review and meta-analysis did not involve any human subjects research.

Supplementary material

10803_2014_2065_MOESM1_ESM.doc (54 kb)
Supplementary material 1 (DOC 54 kb)

Copyright information

© Springer Science+Business Media New York 2014