Abstract
Controversy exists regarding the DSM-5 criteria for ASD. This study tested the psychometric properties of the DSM-5 model and determined how well it performed across different gender, IQ, and DSM-IV-TR sub-type, using clinically collected data on 227 subjects (median age = 3.95 years, majority had IQ > 70). DSM-5 was psychometrically superior to the DSM-IV-TR model (Comparative Fit Index of 0.970 vs 0.879, respectively). Measurement invariance revealed good model fit across gender and IQ. Younger children tended to meet fewer diagnostic criteria. Those with autistic disorder were more likely to meet social communication and repetitive behaviors criteria (p < .001) than those with PDD-NOS. DSM-5 is a robust model but will identify a different, albeit overlapping population of individuals compared to DSM-IV-TR.
Similar content being viewed by others
Notes
In essence favoring more complex models (Breivik and Olsson 2001).
The sample size adjusted BIC-SABIC was preferred to the corrected AIC (Hurvich and Tsai 1989) for the following reason: the AICc has proven useful for the time series autoregressive models for which it was originally developed. There is to date little evidence on its utility in other types of analyses (Brockwell and Davis 1991; McQuarrie and Tsai 1998). With small, less complex models and medium sample sizes, as was the present case, both AIC and AICc will generate similar estimates. Based on the early work of Rafterty (1995), Gignac and Watkins (2013) have recommended that effect sizes need to be suggested for AIC and BIC. They recommended that difference AIC/BIC values of 2, 6, 10 or >10 units reflect “weak”, “positive”, “strong”, and “very strong” effects in favor of the simpler model.
References
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723.
Akaike, H. (1980). Likelihood and the Bayes procedure. In J. M. Bernardo (Ed.), Bayesian Statistics (Vol. 31, pp. 143–166). Valencia: University Press.
American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders-IV-Text Revision. Washington, DC: American Psychiatric Publishing.
American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition. Arlington, VA: American Psychiatric Publishing.
Amir, R. E., Van den Veyver, I. B., Wan, M., Tran, C. Q., Francke, U., & Zoghbi, H. Y. (1999). Rett syndrome is caused by mutations in X-linked MECP2, encoding methyl-CpG-binding protein 2. Nature Genetics, 23(2), 185–188. doi:10.1038/13810.
Barton, M. L., Robins, D. L., Jashar, D., Brennan, L., & Fein, D. (2013). Sensitivity and specificity of proposed DSM-5 criteria for autism spectrum disorder in toddlers. Journal of Autism and Developmental Disorders, 43(5), 1184–1195. doi:10.1007/s10803-013-1817-8.
Bayley, N. (2006). Manual for the Bayley Scales of Infant and Toddler Development (3rd ed.). San Antonio: The Psychological Corporation.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606.
Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
Breivik, E., & Olsson, U. (2001). Adding variables to improve fit: the effect of model size on fit assessment in Lisrel. In R. Cudeck, S. du Toit, & D. Sorbom (Eds.), Structural equation modeling: Present and future (pp. 169–194). Lincolnwood, IL: Scientific Software International.
Brockwell, P. J., & Davis, R. A. (1991). Time series: theory and methods (2nd ed.). New York: Springer.
Charan, S. H. (2012). Childhood disintegrative disorder. Journal of Pediatric Neurosciences, 7(1), 55–57. doi:10.4103/1817-1745.97627.
Elliott, C. (2007). Differential ability scales (2nd ed.). San Antonio: Pearson.
Enders, C. K., & Tofighi, D. (2008). The impact of misspecifying class-specific residual variances in growth mixture models. Structural Equation Modeling: A Multidisciplinary Journal, 15, 75–95.
Frazier, T. W., Youngstrom, E. A., Kubu, C. S., Sinclair, L., & Rezai, A. (2008). Exploratory and confirmatory factor analysis of the autism diagnostic interview-revised. Journal of Autism and Developmental Disorders, 38(3), 474–480. doi:10.1007/s10803-007-0415-z.
Frazier, T. W., Youngstrom, E. A., Speer, L., Law, P., Constantino, J., Findling, R. L., et al. (2012). Validation of proposed DSM-5 criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 28–40 e3. doi:10.1016/j.jaac.2011.09.021.
Gibbs, V., Aldridge, F., Chandler, F., Witzlsperger, E., & Smith, K. (2012). Brief report: an exploratory study comparing diagnostic outcomes for autism spectrum disorders under DSM-IV-TR with the proposed DSM-5 revision. Journal of Autism and Developmental Disorders, 42(8), 1750–1756. doi:10.1007/s10803-012-1560-6.
Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639–662.
Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007). The Autism Diagnostic Observation Schedule: revised algorithms for improved diagnostic validity. Journal of Autism and Developmental Disorders, 37(4), 613–627. doi:10.1007/s10803-006-0280-1.
Guthrie, W., Swineford, L. B., Wetherby, A. M., & Lord, C. (2013). Comparison of DSM-IV and DSM-5 factor structure models for toddlers with autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 52(8), 797–805 e2. doi:10.1016/j.jaac.2013.05.004.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer.
Happe, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9(10), 1218–1220. doi:10.1038/nn1770.
Hu, L. T., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling concepts, issues, and applications (pp. 76–99). London: Sage.
Hu, L. T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.
Huerta, M., Bishop, S. L., Duncan, A., Hus, V., & Lord, C. (2012). Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. The American Journal of Psychiatry, 169(10), 1056–1064. doi:10.1176/appi.ajp.2012.12020276.
Hurvich, C. M., & Tsai, C.-L. (1989). Regression and time series model selection in small samples. Biometrika, 76(2), 297–307.
Insel, T. R. (2014). The NIMH Research Domain Criteria (RDoC) Project: precision medicine for psychiatry. The American Journal of Psychiatry, 171(4), 395–397. doi:10.1176/appi.ajp.2014.14020138.
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, P. S., Quinn, K., et al. (2010). Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. doi:10.1176/appi.ajp.2010.09091379.
Joreskog, K. (1973). A general method for estimating a linear structural equation system. In A. S. Goldberger & O. D. Duncan (Eds.), Structural equation models in the social sciences (pp. 85–112). New York: Seminar Press.
Kenny, D. A. (2012). Measuring model fit. Retrieved October 22, 2012 from http://www.davidakenny.net/cm/fit.htm.
Kent, R. G., Carrington, S. J., LeCouteur, A., Gould, J., Wing, L., Maljaars, J., et al. (2013). Diagnosing Autism Spectrum Disorder: who will get a DSM-5 diagnosis? Journal of Child Psychology and Psychiatry, 54(11), 1242–1250. doi:10.1111/jcpp.12085.
Kim, Y. S., Fombonne, E., Koh, Y. J., Kim, S. J., Cheon, K. A., & Leventhal, B. L. (2014). A comparison of DSM-IV pervasive developmental disorder and DSM-5 autism spectrum disorder prevalence in an epidemiologic sample. Journal of the American Academy of Child and Adolescent Psychiatry, 53(5), 500–508. doi:10.1016/j.jaac.2013.12.021.
Kulage, K. M., Smaldone, A. M., & Cohn, E. G. (2014). How Will DSM-5 affect autism diagnosis? A systematic literature review and meta-analysis. Journal of Autism and Developmental Disorders, 44(8), 1918–1932. doi:10.1007/s10803-014-2065-2.
Lecavalier, L., Gadow, K. D., DeVincent, C. J., Houts, C., & Edwards, M. C. (2009). Deconstructing the PDD clinical phenotype: Internal validity of the DSM-IV. Journal of Child Psychology and Psychiatry, 50(10), 1246–1254. doi:10.1111/j.1469-7610.2009.02104.x.
Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis. Mahwah, NJ: Lawrence.
Lord, C., & Jones, R. M. (2012). Annual research review: Re-thinking the classification of autism spectrum disorders. Journal of Child Psychology and Psychiatry, 53(5), 490–509. doi:10.1111/j.1469-7610.2012.02547.x.
Lord, C., Petkova, E., Hus, V., Gan, W., Lu, F., Martin, D. M., et al. (2012). A multisite study of the clinical diagnosis of different autism spectrum disorders. Archives of General Psychiatry, 69(3), 306–313. doi:10.1001/archgenpsychiatry.2011.148.
Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavare, P. C., et al. (2000). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149.
Maglione, M. A., Gans, D., Das, L., Timbie, J., & Kasari, C. (2012). Nonmedical interventions for children with ASD: Recommended guidelines and further research needs. Pediatrics, 130(Suppl 2), S169–S178. doi:10.1542/peds.2012-0900O.
Mahoney, W. J., Szatmari, P., Maclean, J. E., Bryson, S. E., Bartolucci, G., Walter, S. D., et al. (1998). Reliability and accuracy of differentiating pervasive developmental disorder subtypes. Journal of the American Academy of Child and Adolescent Psychiatry, 37(3), 278–285. doi:10.1097/00004583-199803000-00012.
Mandy, W. P., Charman, T., & Skuse, D. H. (2012). Testing the construct validity of proposed criteria for DSM-5 autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 41–50. doi:10.1016/j.jaac.2011.10.013.
Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit in structural equation models. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics. A Festschrift for Roderick P. McDonald. Mahwah, NJ: Lawrence Erlbaum.
Matson, J. L., Hattier, M. A., & Williams, L. W. (2012a). How does relaxing the algorithm for autism affect DSM-V prevalence rates? Journal of Autism and Developmental Disorders, 42(8), 1549–1556. doi:10.1007/s10803-012-1582-0.
Matson, J. L., Kozlowski, A. M., Hattier, M. A., Horovitz, H., & Sipes, M. (2012b). DSM-IV vs DSM-5 diagnostic criteria for toddlers with autism. Developmental Neurorehabilitation, 15(3), 185–190. doi:10.3109/17518423.2012.672341.
McPartland, J. C., Reichow, B., & Volkmar, F. R. (2012). Sensitivity and specificity of proposed DSM-5 diagnostic criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(4), 368–383. doi:10.1016/j.jaac.2012.01.007.
McQuarrie, A. D. R., & Tsai, C.-L. (1998). Regression and time series model selection. Singapore: World Scientific.
Muthen, L. K., & Muthen, B. O. (2007). Mplus user’s guide 4. Los Angeles, CA: Muthen & Muthen.
Norris, M., Lecavalier, L., & Edwards, M. C. (2012). The structure of autism symptoms as measured by the autism diagnostic observation schedule. Journal of Autism and Developmental Disorders, 42(6), 1075–1086. doi:10.1007/s10803-011-1348-0.
Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.
Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: The University of Chicago Press.
Raykov, T. (2005). Studying group and time invariance in maximal reliability for multiple-component measuring instruments via covariance structure modelling. The British Journal of Mathematical and Statistical Psychology, 58(Pt 2), 301–317. doi:10.1348/000711005X38591.
Raykov, T., & Marcoulides, G. (2000). A first course in structural equation modeling. Mahwah, NJ: Lawrence.
Reise, S. (1990). A comparison of item and person fit methods of assessing model fit in IRT. Applied Psychological Measurement, 42, 127–137.
Rieske, R. D., Matson, J. L., Beighley, J. S., Cervantes, P. E., Goldin, R. L., & Jang, J. (2013). Comorbid psychopathology rates in children diagnosed with autism spectrum disorders according to the DSM-IV-TR and the proposed DSM-5. Developmental Neurorehabilitation: Advance online publication. doi:10.3109/17518423.2013.790519.
Rigdon, E. E. (1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling, 3(4), 369–379.
Ronald, A., Happe, F., Bolton, P., Butcher, M., Price, T. S., & Plomin, R. (2006). Genetic heterogeneity between the three components of the autism spectrum: A twin study. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 691–699. doi:10.1097/01.chi.0000215325.13058.9d.
Schwarz, G. E. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.
Sideridis, G. D., Simos, P., Papanicolaou, A., & Fletcher, J. (2014). Using structural equation modeling to assess functional connectivity in the brain: Power and sample size considerations. Educational and Psychological Measurement, 74, 733–758.
Smith, E. V, Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231.
Smith, R. M., Schumacker, R. E., & Bush, M. J. (1998). Using item mean squares to evaluate fit to the Rasch model. Journal of Outcome Measurement, 2, 66–78.
Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180.
Steiger, J. H. (2000). Point Estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to Hayduk and Glaser. Structural Equation Modeling, 7(2), 149–162.
Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42, 893–898.
Stuive, I., Kiers, H. A. L., Timmerman, M. E., & ten Berge, J. M. F. (2008). The empirical verification of an assignment of items to subtests: The oblique multiple group method versus the confirmatory common factor method. Educational and Psychological Measurement, 68(6), 923–939.
Taheri, A., & Perry, A. (2012). Exploring the proposed DSM-5 criteria in a clinical sample. Journal of Autism and Developmental Disorders, 42(9), 1810–1817. doi:10.1007/s10803-012-1599-4.
Tofghi, D., & Enders, C. K. (2007). Identifying the correct number of classes in mixture models. In G. R. Hancock & K. M. Samulelsen (Eds.), Advances in latent variable mixture models (pp. 317–341). Greenwich, CT: Information Age.
Tucker, L. R., & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrica, 38, 1–10.
Widaman, K. F., & Thompson, J. S. (2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8(1), 16–37.
Worley, J. A., & Matson, J. L. (2012). Comparing symptoms of autism spectrum disorders using the current DSM-IV-TR diagnostic criteria and the proposed DSM-V diagnostic criteria. Research in Autism Spectrum Disorders, 6, 965–970.
Young, R. L., & Rodi, M. L. (2013). Redefining Autism Spectrum Disorder Using DSM-5: The Implications of the Proposed DSM-5 Criteria for Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 44(4), 758–765. doi:10.1007/s10803-013-1927-3.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1
Multi-level model estimated to test the potential influence of inter-rater variability (bias).
Level-1 Model
Level-2 Model
The prediction of DSM-5 scores is a function of the intercept β 0j and the partial regression coefficients related to inter-rater team variability β 1j participant’s age β 2j gender β 3j and DSM-IV-TR variability β 4j plus the error of estimate r ij . The classification variable at level-2 involved the number of different diagnostic teams.
The presence of differential rater effects would be suggestive of bias in the criteria employed to clinically diagnose children with ASD. Thus, it represented an important prerequisite to validly testing the different classification systems.
Appendix 2
See Table 5.
Rights and permissions
About this article
Cite this article
Harstad, E.B., Fogler, J., Sideridis, G. et al. Comparing Diagnostic Outcomes of Autism Spectrum Disorder Using DSM-IV-TR and DSM-5 Criteria. J Autism Dev Disord 45, 1437–1450 (2015). https://doi.org/10.1007/s10803-014-2306-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10803-014-2306-4