Skip to main content
Log in

Comprehensive Comparison of Self-administered Questionnaires for Measuring Quantitative Autistic Traits in Adults

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

We comprehensively compared all available questionnaires for measuring quantitative autistic traits (QATs) in terms of reliability and construct validity in 3,147 non-clinical and 60 clinical subjects with normal intelligence. We examined four full-length forms, the Subthreshold Autism Trait Questionnaire (SATQ), the Broader Autism Phenotype Questionnaire, the Social Responsiveness Scale2-Adult Self report (SRS2-AS), and the Autism-Spectrum Quotient (AQ). The SRS2-AS and the AQ each had several short forms that we also examined, bringing the total to 11 forms. Though all QAT questionnaires showed acceptable levels of test–retest reliability, the AQ and SRS2-AS, including their short forms, exhibited poor internal consistency and discriminant validity, respectively. The SATQ excelled in terms of classical test theory and due to its short length.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  • Allison, C., Auyeung, B., & Baron-Cohen, S. (2012). Toward brief “Red Flags” for autism screening: The Short Autism Spectrum Quotient and the Short Quantitative Checklist for Autism in toddlers in 1,000 cases and 3,000 controls [corrected]. Journal of the American Academy of Child and Adolescent Psychiatry, 51(2), 202–212.

    Article  PubMed  Google Scholar 

  • Austin, E. J. (2005). Personality correlates of the broader autism phenotype as assessed by the Autism Spectrum Quotient (AQ). Personality and Individual Differences, 38(2), 451–460.

    Article  Google Scholar 

  • Bailey, A., Palferman, S., Heavey, L., & Le Couteur, A. (1998). Autism: The phenotype in relatives. Journal of Autism and Developmental Disorders, 28(5), 369–392.

    Article  PubMed  Google Scholar 

  • Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The autism-spectrum quotient (AQ): Evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31(1), 5–17.

    Article  PubMed  Google Scholar 

  • Bolton, P., Macdonald, H., Pickles, A., Rios, P., Goode, S., Crowson, M., et al. (1994). A case–control family history study of autism. Journal of Child Psychology and Psychiatry Child Psychology and Psychiatry, and Allied Disciplines, 35(5), 877–900.

    Article  Google Scholar 

  • Boulet, J., & Boss, M. W. (1991). Reliability and validity of the brief symptom inventory. Psychological Assessment, 3(3), 433–437.

    Article  Google Scholar 

  • Brugha, T. S., McManus, S., Smith, J., Scott, F. J., Meltzer, H., Purdon, S., et al. (2012). Validating two survey methods for identifying cases of autism spectrum disorder among adults in the community. Psychological Medicine, 42(3), 647–656.

    Article  PubMed  Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105.

    Article  PubMed  Google Scholar 

  • Canty, A., & Ripley, B. (2012). boot: Bootstrap R (S-Plus) functions.

  • Constantino, J. N., Abbacchi, A. M., Lavesser, P. D., Reed, H., Givens, L., Chiang, L., et al. (2009). Developmental course of autistic social impairment in males. Development and Psychopathology, 21(1), 127–138.

    Article  PubMed Central  PubMed  Google Scholar 

  • Constantino, J. N., & Cruber, C. P. (2012). Social Responsiveness Scale, second edition (SRS-2). Los Angeles, CA: Western Psychological Services.

    Google Scholar 

  • Constantino, J. N., Lavesser, P. D., Zhang, Y., Abbacchi, A. M., Gray, T., & Todd, R. D. (2007). Rapid quantitative assessment of autistic social impairment by classroom teachers. Journal of the American Academy of Child and Adolescent Psychiatry, 46(12), 1668–1676.

    Article  PubMed  Google Scholar 

  • Constantino, J. N., & Todd, R. D. (2003). Autistic traits in the general population: A twin study. Archives of General Psychiatry, 60(5), 524–530.

    Article  PubMed  Google Scholar 

  • Davidson, J., Goin-Kochel, R. P., Green-Snyder, L. A., Hundley, R. J., Warren, Z., & Peters, S. U. (2012). Expression of the broad autism phenotype in simplex autism families from the simons simplex collection. Journal of Autism and Developmental Disorders. doi:10.1007/s10803-012-1492-1.

  • DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3), 837–845.

    Article  PubMed  Google Scholar 

  • Derogatis, L. R. (1977). SCL-90R (revised version) manual I. Baltimore: Johns Hopkins University School of Medicine.

    Google Scholar 

  • Derogatis, L. R., & Melisaratos, N. (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13(03), 595–605.

    Article  PubMed  Google Scholar 

  • Duku, E., Vaillancourt, T., Szatmari, P., Georgiades, S., Zwaigenbaum, L., Smith, I. M., et al. (2013). Investigating the measurement properties of the social responsiveness scale in preschool children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(4), 860–868.

    Article  PubMed  Google Scholar 

  • Efron, B., & Tibshirani, R. (1994). An introduction to the bootstrap (Chapman & Hall/CRC monographs on statistics & applied probability) (1st ed.). New York: Chapman and Hall/CRC.

    Google Scholar 

  • Fleiss, J. L. (1999). The design and analysis of clinical experiments. Hoboken, NJ: Wiley.

    Book  Google Scholar 

  • Folstein, S., & Rutter, M. (1977). Infantile autism: A genetic study of 21 twin pairs. Journal of Child Psychology and Psychiatry Child Psychology and Psychiatry, and Allied Disciplines, 18(4), 297–321.

    Article  Google Scholar 

  • Froom, P., Melamed, S., Kristal-Boneh, E., Benbassat, J., & Ribak, J. (1999). Healthy volunteer effect in industrial workers. Journal of Clinical Epidemiology, 52(8), 731–735.

    Article  PubMed  Google Scholar 

  • Fujita, K., Maekawa, H., Dairoku, K., & Yamanaka, K. (2006). A Japanese version of the WAIS-III. Tokyo: Nihon Bunka Kagakusha. Tokyo: Nihon Bunka Kagakusha (in Japanese).

  • Fukuhara, S., Bito, S., Green, J., Hsiao, A., & Kurokawa, K. (1998). Translation, adaptation, and validation of the SF-36 Health Survey for use in Japan. Journal of Clinical Epidemiology, 51(11), 1037–1044.

    Article  PubMed  Google Scholar 

  • Fukuhara, S., & Suzukamo, Y. (2004). Manual of the SF-36v2 Japanese version: Institute for Health Outcomes and Process Evaluation research. Kyoto: Institute for Health Outcomes and Process Evaluation Research.

    Google Scholar 

  • Furukawa, T. A., Kawakami, N., Saitoh, M., Ono, Y., Nakane, Y., Nakamura, Y., et al. (2008). The performance of the Japanese version of the K6 and K10 in the World Mental Health Survey Japan. International Journal of Methods in Psychiatric Research, 17(3), 152–158.

    Article  PubMed  Google Scholar 

  • Gamer, M., Lemon, J., Fellows, I., & Singh, R. (2010). irr: Various coefficients of interrater reliability and agreement for quantative, ordinal, and nominal data. R package version 1.4-9.

  • Hoekstra, R. A., Bartels, M., Cath, D. C., & Boomsma, D. I. (2008). Factor structure, reliability and criterion validity of the Autism-Spectrum Quotient (AQ): A study in Dutch population and patient groups. Journal of Autism and Developmental Disorders, 38(8), 1555–1566.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hoekstra, R. A., Vinkhuyzen, A. A. E., Wheelwright, S., Bartels, M., Boomsma, D. I., Baron-Cohen, S., et al. (2011). The construction and validation of an abridged version of the autism-spectrum quotient (AQ-Short). Journal of Autism and Developmental Disorders, 41(5), 589–596.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hurley, R. S. E., Losh, M., Parlier, M., Reznick, J. S., & Piven, J. (2007). The broad autism phenotype questionnaire. Journal of Autism and Developmental Disorders, 37(9), 1679–1690.

    Article  PubMed  Google Scholar 

  • Hurst, R. M., Nelson-Gray, R. O., Mitchell, J. T., & Kwapil, T. R. (2007). The relationship of Asperger’s characteristics and schizotypal personality traits in a non-clinical adult sample. Journal of Autism and Developmental Disorders, 37(9), 1711–1720.

    Article  PubMed  Google Scholar 

  • Ingersoll, B., Hopwood, C. J., Wainer, A., & Brent Donnellan, M. (2011). A comparison of three self-report measures of the broader autism phenotype in a non-clinical sample. Journal of Autism and Developmental Disorders, 41(12), 1646–1657.

    Article  PubMed  Google Scholar 

  • Japanese version of WISC III and Publication Committee. (1998). A Japanese version of WISC III. Tokyo: Nihon Bunka Kagakusha. (in Japanese).

    Google Scholar 

  • Johns, M. W. (1991). A new method for measuring daytime sleepiness: The Epworth sleepiness scale. Sleep, 14(6), 540–545.

    PubMed  Google Scholar 

  • Kanne, S. M., Christ, S. E., & Reiersen, A. M. (2009). Psychiatric symptoms and psychosocial difficulties in young adults with autistic traits. Journal of Autism and Developmental Disorders, 39(6), 827–833.

    Article  PubMed  Google Scholar 

  • Kanne, S. M., Wang, J., & Christ, S. E. (2012). The Subthreshold Autism Trait Questionnaire (SATQ): Development of a brief self-report measure of subthreshold autism traits. Journal of Autism and Developmental Disorders, 42(5), 769–780.

    Article  PubMed  Google Scholar 

  • Kanner, L. (1943). Autistic disturbances of affective contact. Nervous Child, 2, 217–250.

    Google Scholar 

  • Kawakami, N., Arai, S., & Okoyama, K. (1993). Reliability and validity of the Japanese version of the CAGE alcoholism screening test. Japanese Journal of Hygiene, 48(1), 401.

    Google Scholar 

  • Kessler, R. C., Andrews, G., Colpe, L. J., Hiripi, E., Mroczek, D. K., Normand, S. L. T., et al. (2002). Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine, 32(6), 959–976.

    Article  PubMed  Google Scholar 

  • Kirshner, B., & Guyatt, G. (1985). A methodological framework for assessing health indices. Journal of Chronic Diseases, 38(1), 27–36.

    Article  PubMed  Google Scholar 

  • Kloosterman, P. H., Keefer, K. V., Kelley, E. A., Summerfeldt, L. J., & Parker, J. D. A. (2011). Evaluation of the factor structure of the Autism-Spectrum Quotient. Personality and Individual Differences, 50(2), 310–314.

    Article  Google Scholar 

  • Kurita, H., Koyama, T., & Osada, H. (2005). Autism-Spectrum Quotient-Japanese version and its short forms for screening normally intelligent persons with pervasive developmental disorders. Psychiatry and Clinical Neurosciences, 59(4), 490–496.

    Article  PubMed  Google Scholar 

  • Lau, W. Y.-P., Gau, S. S.-F., Chiu, Y.-N., Wu, Y.-Y., Chou, W.-J., Liu, S.-K., et al. (2013). Psychometric properties of the Chinese version of the Autism Spectrum Quotient (AQ). Research in Developmental Disabilities, 34(1), 294–305.

    Article  PubMed  Google Scholar 

  • Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., et al. (2000). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.

    Article  PubMed  Google Scholar 

  • Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24(5), 659–685.

    Article  PubMed  Google Scholar 

  • Mandy, W. P. L., Charman, T., & Skuse, D. H. (2012). Testing the construct validity of proposed criteria for DSM-5 autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 41–50.

    Article  PubMed  Google Scholar 

  • Mandy, W. P. L., & Skuse, D. H. (2008). Research review: What is the association between the social-communication element of autism and repetitive interests, behaviours and activities? Journal of Child Psychology and Psychiatry, 49(8), 795–808.

    Article  PubMed  Google Scholar 

  • Mayfield, D., McLeod, G., & Hall, P. (1974). The CAGE questionnaire: Validation of a new alcoholism screening instrument. The American Journal of Psychiatry, 131(10), 1121–1123.

    PubMed  Google Scholar 

  • McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.

    Article  Google Scholar 

  • McHorney, C. A., Ware, J. E., Lu, J. F., & Sherbourne, C. D. (1994). The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Medical Care, 32(1), 40–66.

    Article  PubMed  Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749.

    Article  Google Scholar 

  • Micali, N., Chakrabarti, S., & Fombonne, E. (2004). The broad autism phenotype: Findings from an epidemiological survey. Autism, 8(1), 21–37.

    Article  PubMed  Google Scholar 

  • Nunnally, J., & Bernstein, I. (1994). Psychometric theory. rds.epi-ucsf.org. New York: Nunnally, Jum C., and Ira H. Bernstein. Psychometric theory. No. 972. McGraw-Hill Humanities/Social Sciences/Languages, 1994.

  • Piersma, H. L., Boes, J. L., & Reaume, W. M. (1994). Unidimensionality of the Brief Symptom Inventory (BSI) in adult and adolescent inpatients. Journal of Personality Assessment, 63(2), 338–344.

    Article  PubMed  Google Scholar 

  • Piven, J., Gayle, J., Chase, G. A., Fink, B., Landa, R., Wzorek, M. M., et al. (1990). A family history study of neuropsychiatric disorders in the adult siblings of autistic individuals. Journal of the American Academy of Child and Adolescent Psychiatry, 29(2), 177–183.

    Article  PubMed  Google Scholar 

  • Piven, J., Palmer, P., Jacobi, D., Childress, D., & Arndt, S. (1997). Broader autism phenotype: Evidence from a family history study of multiple-incidence autism families. The American Journal of Psychiatry, 154(2), 185–190.

    PubMed  Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. The Journal of Applied Psychology, 88(5), 879–903.

    Article  PubMed  Google Scholar 

  • R Development CoreTeam. (2012). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

    Google Scholar 

  • Ransohoff, D. F., & Feinstein, A. R. (1978). Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. The New England Journal of Medicine, 299(17), 926–930.

    Article  PubMed  Google Scholar 

  • Reiersen, A. M., Constantino, J. N., Grimmer, M., Martin, N. G., & Todd, R. D. (2008). Evidence for shared genetic influences on self-reported ADHD and autistic symptoms in young adult Australian twins. Twin Research and Human Genetics, 11(6), 579–585.

    Article  PubMed Central  PubMed  Google Scholar 

  • Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response analysis. Journal of Statistical Software, 17(5), 1–25.

    Google Scholar 

  • Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., et al. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12, 77.

    Article  PubMed Central  PubMed  Google Scholar 

  • Ruta, L., Mazzone, D., Mazzone, L., Wheelwright, S., & Baron-Cohen, S. (2012). The Autism-Spectrum Quotient—Italian version: A cross-cultural confirmation of the broader autism phenotype. Journal of Autism and Developmental Disorders, 42(4), 625–633.

    Article  PubMed  Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.

    Article  PubMed  Google Scholar 

  • St Pourcain, B., Wang, K., Glessner, J. T., Golding, J., Steer, C., Ring, S. M., et al. (2010). Association between a high-risk autism locus on 5p14 and social communication spectrum phenotypes in the general population. The American journal of psychiatry, 167(11), 1364–1372.

    Article  PubMed Central  PubMed  Google Scholar 

  • Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 82(2), 245.

    Article  Google Scholar 

  • Streiner, D. L., & Norman, G. R. (2008). Health Measurement Scales: A practical guide to their development and use. New York, NY: Oxford University Press.

    Book  Google Scholar 

  • Takegami, M., Suzukamo, Y., Wakita, T., Noguchi, H., Chin, K., Kadotani, H., et al. (2009). Development of a Japanese version of the Epworth Sleepiness Scale (JESS) based on item response theory. Sleep Medicine, 10(5), 556–565.

    Article  PubMed  Google Scholar 

  • Tanaka Institute of Education. (1987). Tanaka-Binet Intelligence Scale. Tokyo: Taken. (in Japanese).

    Google Scholar 

  • Tomioka, M., Shimura, M., Hidaka, M., & Kubo, C. (2008). The reliability and validity of a Japanese version of symptom checklist 90 revised. BioPsychoSocial Medicine, 2, 19.

    Article  PubMed Central  PubMed  Google Scholar 

  • Wakabayashi, A., Baron-Cohen, S., & Wheelwright, S. (2006a). Are autistic traits an independent personality dimension? A study of the Autism-Spectrum Quotient (AQ) and the NEO-PI-R. Personality and Individual Differences, 41(5), 873–883.

    Article  Google Scholar 

  • Wakabayashi, A., Baron-Cohen, S., Wheelwright, S., & Tojo, Y. (2006b). The Autism-Spectrum Quotient (AQ) in Japan: A cross-cultural comparison. Journal of Autism and Developmental Disorders, 36(2), 263–270.

    Article  PubMed  Google Scholar 

  • Ware, J. E., & Gandek, B. (1998). Methods for testing data quality, scaling assumptions, and reliability. Journal of Clinical Epidemiology, 51(11), 945–952.

    Article  PubMed  Google Scholar 

  • Ware, J. E., Kosinski, M., & Dewey, J. E. (2000). How to score version 2 of the SF-36 Health Survey. Lincoln, RI: QualityMetric Incorporated.

    Google Scholar 

  • Watson, D., & Clark, L. A. (1984). Negative affectivity: The disposition to experience aversive emotional states. Psychological Bulletin, 96(3), 465–490.

    Article  PubMed  Google Scholar 

  • Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., et al. (2005). Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR task force for translation and cultural adaptation. Value in Health, 8(2), 94–104.

    Article  PubMed  Google Scholar 

  • Woodbury-Smith, M. R., Robinson, J., Wheelwright, S., & Baron-Cohen, S. (2005). Screening adults for Asperger Syndrome using the AQ: A preliminary study of its diagnostic validity in clinical practice. Journal of Autism and Developmental Disorders, 35(3), 331–335.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This study was supported by the Grants-in-Aid for Scientific Research (Basic Research B: No. 22300095, Basic Research C: No. 24590807 and No. 23591723) from the Ministry of Education, Culture, Sports, Science and Technology in Japan. The authors would like to thank the following people for their assistance with data collection: Dr. Taishi Miyachi, Dr. Masayuki Imaeda, Dr. Atsuro Yamada, Dr. Takao Maeda, Dr. Susumu Adachi, Dr. Hiroshi Hirai, and Dr. Chiho Goto. They also thank Prof. John N. Constantino and Dr. Christian P. Gruber for their useful and constructive comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Takeshi Nishiyama.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Text (DOC 32 kb)

Supplementary Table 1 (DOC 36 kb)

Supplementary Table 2 (DOC 38 kb)

Supplementary Table 3 (DOC 42 kb)

Supplementary Table 4 (DOC 73 kb)

Supplementary Table 5 (DOC 57 kb)

Supplementary Table 6 (DOC 52 kb)

Supplementary Table 7 (DOC 76 kb)

Supplementary Table 8 (DOC 36 kb)

Supplementary Table 9 (DOC 49 kb)

Supplementary Table 10 (DOC 50 kb)

Supplementary Table 11 (DOC 48 kb)

10803_2013_2020_MOESM13_ESM.tif

Distribution and Q-Q plots of scores from the SRS2-AS short forms according to disease status. (Supplementary Figure 1, TIFF 15005 kb)

10803_2013_2020_MOESM14_ESM.tif

Distribution and Q-Q plot of scores from the AQ short forms according to disease status. (Supplementary Figure 2, TIFF 39461 kb)

10803_2013_2020_MOESM15_ESM.tif

ROC curves for the four full-scale questionnaires of autistic traits to discriminate ASD patients from non-ASD controls. (Supplementary Figure 3, TIFF 39 kb)

10803_2013_2020_MOESM16_ESM.tif

ROC curves for the SRS2-AS and its abridged forms to discriminate ASD patients from non-ASD controls. (Supplementary Figure 4, TIFF 36 kb)

10803_2013_2020_MOESM17_ESM.tif

ROC curves for the AQ and its abridged forms to discriminate ASD patients from non-ASD controls. (Supplementary Figure 5, TIFF 43 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nishiyama, T., Suzuki, M., Adachi, K. et al. Comprehensive Comparison of Self-administered Questionnaires for Measuring Quantitative Autistic Traits in Adults. J Autism Dev Disord 44, 993–1007 (2014). https://doi.org/10.1007/s10803-013-2020-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-013-2020-7

Keywords

Navigation