Journal of Autism and Developmental Disorders

, Volume 47, Issue 9, pp 2770–2782 | Cite as

Examining the Role of Race, Ethnicity, and Gender on Social and Behavioral Ratings Within the Autism Diagnostic Observation Schedule

  • Ashley J. Harrison
  • Kristin A. Long
  • Douglas C. Tommet
  • Richard N. Jones
Original Paper


The Autism Diagnostic Observation Schedule (ADOS) is widely used to assess symptoms of autism spectrum disorder (ASD). Given well-documented differences in social behaviors across cultures, this study examined whether item-level biases exist in ADOS scores across sociodemographic groups (race, ethnicity, and gender). We examined a subset of ten ADOS items among participants (N = 2458). Holding level of overall ADOS behavioral symptoms constant, we found significant item level bias (measurement noninvariance) for race and ethnicity on three ADOS items. Item-level bias was not apparent across gender. Although the magnitude of bias was small, our findings highlight the need to reevaluate norms and operational definitions used in assessments to increase ASD diagnostic accuracy among culturally-diverse groups.


Autism spectrum disorder Cross-cultural Social norms ADOS Measurement bias Race Ethnicity 



Ashley J. Harrison’s work on this project was supported by a NIMH T32 fellowship to (5T32MH019927_20). The authors would like to express their appreciation to the following students for their help with this project and manuscript, Margaret Naughton, Rebecca Miller, and Alejandra Perez-Ramirez and the families that participated in the Simons Simplex Collection data set.

Author Contributions

AJH and RNJ conceived of the study and designed the study. AJH requested SSC data access. RNJ and DCT performed the statistical analysis and the provided the written interpretation of the data analyses. AJH and KAL helped interpret the data and aided in contextualizing the findings in the cross-cultural literature. All authors drafted and revised the manuscript.

Compliance with Ethical Standards

Conflict of interests

All the authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all participants included in the study.


  1. Al-Salehi, S. M., & Al-Hifthy, E. H. (2009). Autism in Saudi Arabia: Presentation, clinical correlates and comorbidity. Transcultural Psychiatry, 46(2), 340–347.CrossRefPubMedGoogle Scholar
  2. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washington, DC: American Psychiatric Association.Google Scholar
  3. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental health disorders (5th ed.). Washington, DC: American Psychiatric Association.CrossRefGoogle Scholar
  4. Begeer, S., El Bouk, S., Boussaid, W., Terwogt, M. M., & Koot, H. M. (2009). Underdiagnosis and referral bias of autism in ethnic minorities. Journal of Autism and Developmental Disorders, 39(1), 142–148. doi: 10.1007/s10803-008-0611-5.CrossRefPubMedGoogle Scholar
  5. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238.CrossRefPubMedGoogle Scholar
  6. Bracken, B. A., & Barona, A. (1991). State of the art procedures for translating, validating and using psychoeducational tests in cross-cultural assessment. School Psychology International, 12, 119–132. doi: 10.1177/0143034391121010.CrossRefGoogle Scholar
  7. Browne, M. W., Cudeck, R., Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit. Sage Focus Editions, 154, 136–136.Google Scholar
  8. Caron, K. G., Schaaf, R. C., Benevides, T. W., & Gal, E. (2012). Cross-cultural comparison of sensory behaviors in children with autism. American Journal of Occupational Therapy, 66(5), e77–e80.CrossRefPubMedGoogle Scholar
  9. Carter, J. A., Lees, J. A., Murira, G. M., Gona, J., Neville, B. G., & Newton, C. R. (2005). Issues in the development of cross-cultural assessments of speech and language for children. International Journal of Language & Communication Disorders, 40(4), 385–401. doi: 10.1080/13682820500057301.CrossRefGoogle Scholar
  10. Chaidez, V., Hansen, R. L., & Hertz-Picciotto, I. (2012). Autism spectrum disorders in Hispanics and non-Hispanics. Autism: The International Journal of Research and Practice, 16(4), 381–397. doi: 10.1177/1362361311434787.CrossRefGoogle Scholar
  11. Chua, H. F., Boland, J. E., & Nisbett, R. E. (2005). Cultural variation in eye movements during scene perception. Proceedings of the National Academy of Sciences of the United States of America, 102(35), 12629–12633. doi: 10.1073/pnas.0506162102.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Chung, K.-M., Jung, W., Yang, J.-W., Ben-Itzchak, E., Zachor, D. A., Furniss, F., … Barker, A. A. (2012). Cross cultural differences in challenging behaviors of children with autism spectrum disorders: An international examination between Israel, South Korea, the United Kingdom, and the United States of America. Research in Autism Spectrum Disorders, 6(2), 881–889.CrossRefGoogle Scholar
  13. Collett, P. (1971). Training Englishmen in the non-verbal behaviour of Arabs. International Journal of Psychology, 6(3), 209–215. doi: 10.1080/00207597108246684.CrossRefGoogle Scholar
  14. Constantino, J. N., & Gruber, C. P. (2012). Social responsiveness scale, second edition: SRS-2. Los Angeles: Western Psychological Services.Google Scholar
  15. Daley, T. C. (2004). From symptom recognition to diagnosis: children with autism in urban India. Social Science & Medicine, 58(7), 1323–1335.CrossRefGoogle Scholar
  16. El Bouk, S., Boussaid, W., Meerum Terwogt, M., & Koot, H. (2009). Underdiagnosis and referral bias of autism in ethnic minorities. Journal of Autism and Developmental Disorders, 39(1), 142–148.CrossRefGoogle Scholar
  17. Elfenbein, H. A. (2013). Nonverbal dialects and accents in facial expressions of emotion. Emotion Review, 5(1), 90–96. doi: 10.1177/175407391245133.CrossRefGoogle Scholar
  18. Elfenbein, H. A., Beaupré, M., Lévesque, M., & Hess, U. (2007). Toward a dialect theory: Cultural differences in the expression and recognition of posed facial expressions. Emotion, 7(1), 131. doi: 10.1037/1528-3542.7.1.131.CrossRefPubMedGoogle Scholar
  19. Ember, C. R., & Cunnar, C. M. (2015). Children’s play and work: The relevance of cross-cultural ethnographic research for archaeologists. Childhood in the Past, 8(2), 87–103. doi: 10.1179/1758571615Z.00000000031.CrossRefGoogle Scholar
  20. Farver, J. M., & Howes, C. (1993). Cultural differences in American and Mexican mother-child pretend play. Merrill-Palmer Quarterly, 39, 344–358.Google Scholar
  21. Fischbach, G. D., & Lord, C. (2010). The Simons Simplex Collection: a resource for identification of autism genetic risk factors. Neuron, 68(2), 192–195.CrossRefPubMedGoogle Scholar
  22. Freeth, M., Milne, E., Sheppard, E., & Ramachandran, R. (2014). Autism across cultures: Perspectives from non-western cultures and implications for research. In F. R. Volkmar, S. J. Rogers, R. Paul, & K. A. Pelphrey (Eds.), Handbook of autism and pervasive developmental disorders, (4th ed.,Vol. 2, pp. 997–1013). Hoboken, NJ: Wiley.Google Scholar
  23. Freeth, M., Sheppard, E., Ramachandran, R., & Milne, E. (2013). A cross-cultural comparison of autistic traits in the UK, India and Malaysia. Journal of Autism and Developmental Disorders, 43(11), 2569–2583.CrossRefPubMedGoogle Scholar
  24. Fugita, S. S., Wexley, K. N., & Hillery, J. M. (1974). Black-White differences in nonverbal behavior in an interview setting. Journal of Applied Social Psychology, 4(4), 343–350. doi: 10.1111/j.1559-1816.1974.tb02606.x.CrossRefGoogle Scholar
  25. Gotham, K., Risi, S., Dawson, G., Tager-Flusberg, H., Joseph, R., Carter, A., … Hyman, S. L. (2008). A replication of the Autism Diagnostic Observation Schedule (ADOS) revised algorithms. Journal of the American Academy of Child & Adolescent Psychiatry, 47(6), 642–651. doi: 10.1097/CHI.0b013e31816bffb7.CrossRefGoogle Scholar
  26. Hall, W. S., & Freedle, R. O. (1975). Culture and language: The black American experience. Washington D.C.: Hemisphere Publishing.Google Scholar
  27. Hall, W. S., Reder, S., & Cole, M. (1975). Story recall in young Black and White children: Effects of racial group membership, race of experimenter, and dialect. Developmental Psychology, 11(5), 628. doi: 10.1037/0012-1649.11.5.628.CrossRefGoogle Scholar
  28. Hess, U., Adams, R., & Kleck, R. (2005). Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition & Emotion, 19(4), 515–536. doi: 10.1080/02699930441000364.CrossRefGoogle Scholar
  29. Hiller, R. M., Young, R. L., & Weber, N. (2015). Sex differences in pre-diagnosis concerns for children later diagnosed with autism spectrum disorder. Autism: The International Journal of Research and Practice, 20(1), 75–84. doi: 10.1177/1362361314568899.CrossRefGoogle Scholar
  30. Hilton, C. L., Fitzgerald, R. T., Jackson, K. M., Maxim, R. A., Bosworth, C. C., Shattuck, P. T., … Constantino, J. N. (2010). Brief report: Under-representation of African Americans in autism genetic research: A rationale for inclusion of subjects representing diverse family structures. Journal of Autism and Developmental Disorders, 40(5), 633–639. doi: 10.1007/s10803-009-0905-2.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Hu, L.-T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424.CrossRefGoogle Scholar
  32. Jack, R. E., Garrod, O. G., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences, 109(19), 7241–7244. doi: 10.1097/01.mlr.0000245250.50114.0f.CrossRefGoogle Scholar
  33. Jones, R. N. (2006). Identification of measurement differences between English and Spanish language versions of the Mini-Mental State Examination: detecting differential item functioning using MIMIC modeling. Medical Care, 44(11), S124–S133.CrossRefPubMedGoogle Scholar
  34. Kirkovski, M., Enticott, P. G., & Fitzgerald, P. B. (2013). A review of the role of female gender in autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(11), 2584–2603.CrossRefPubMedGoogle Scholar
  35. Knapp, M. L., Hall, J. A., & Horgan, T. G. (2013). Nonverbal communication in human interaction. Boston: Wadsworth Cengage Learning.Google Scholar
  36. Knickmeyer, R. C., Wheelwright, S., & Baron-Cohen, S. B. (2008). Sex-typical play: Masculinization/defeminization in girls with an autism spectrum condition. Journal of Autism and Developmental Disorders, 38(6), 1028–1035. doi: 10.1007/s10803-007-0475-0.CrossRefPubMedGoogle Scholar
  37. Kochman, T. (1981). Black and white styles in conflict: Chicago: University of Chicago Press.Google Scholar
  38. LaFrance, M., & Hecht, M. A. (2000). Gender and smiling: A meta-analysis. In A. Fischer (Ed.), Gender and emotion: Social psychological perspectives (pp. 118). Cambridge, UK: Cambridge University Press.Google Scholar
  39. LaFrance, M., & Mayo, C. (1976). Racial differences in gaze behavior during conversations: Two systematic observational studies. Journal of Personality and Social Psychology, 33(5), 547.CrossRefGoogle Scholar
  40. Lai, M.-C., Lombardo, M., Auyeung, B., Chakrabarti, B., & Baron-Cohen, S. (2015). Sex/gender differences and autism. Journal of the American Academy of Child & Adolescent Psychiatry, 54(1), 11–24.CrossRefGoogle Scholar
  41. Lancy, D. F. (2007). Accounting for variability in mother–child play. American Anthropologist, 109(2), 273–284. doi: 10.1525/aa.2007.109.2.273.CrossRefGoogle Scholar
  42. Landa, R., & Garrett-Mayer, E. (2006). Development in infants with autism spectrum disorders: A prospective study. Journal of Child Psychology & Psychiatry, 47(6), 629–638. doi: 10.1111/j.1469-7610.2006.01531.x.CrossRefGoogle Scholar
  43. Liptak, G. S., Benzoni, L. B., Mruzek, D. W., Nolan, K. W., Thingvoll, M. A., Wade, C. M., & Fryer, G. E. (2008). Disparities in diagnosis and access to health services for children with autism: Data from the National Survey of Children’s Health. Journal of Developmental & Behavioral Pediatrics, 29(3), 152–160.CrossRefGoogle Scholar
  44. Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., ... Rutter, M. (2000). The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.CrossRefPubMedGoogle Scholar
  45. Lord, C., Rutter, M., DiLavore, P., & Risi, S. (2002). Autism Diagnostic Observation Schedule: ADOS. Los Angeles: Western Psychological Services.Google Scholar
  46. Lord, C., Rutter, M., DiLavore, P., Risi, S., Gotham, K., & Bishop, S. L. (2012). Autism Diagnsotic Observation Schedule, second edition (ADOS-2) manual (Part 1): Modules 1–4. Torrance, CA: Western Psychological Services.Google Scholar
  47. Magaña, S., Lopez, K., Aguinaga, A., & Morton, H. (2013). Access to diagnosis and treatment services among Latino children with autism spectrum disorders. Intellectual and Developmental Disabilities, 51(3), 141–153. doi: 10.1352/1934-9556-51.3.141.CrossRefPubMedGoogle Scholar
  48. Magiati, I., Goh, D. A., Lim, S. J., Gan, D. Z. Q., Leong, J., Allison, C., … Saw, S. M. (2015). The psychometric properties of the Quantitative-Checklist for Autism in Toddlers (Q-CHAT) as a measure of autistic traits in a community sample of Singaporean infants and toddlers. Molecular Autism, 6(1), 40.CrossRefPubMedPubMedCentralGoogle Scholar
  49. Mandell, D. S., Listerud, J., Levy, S. E., & Pinto-Martin, J. A. (2002). Race differences in the age at diagnosis among Medicaid-eligible children with autism. Journal of the American Academy of Child & Adolescent Psychiatry, 41(12), 1447–1453. doi: 10.1097/01.CHI.0000024863.60748.53.CrossRefGoogle Scholar
  50. Mandell, D. S., Wiggins, L. D., Carpenter, L. A., Daniels, J., DiGuiseppi, C., Durkin, M. S., Giarelli, E., … Pinto-Martin, J. A. (2009). Racial/ethnic disparities in the identification of children with autism spectrum disorders. American Journal of Public Health, 99(3), 493.CrossRefPubMedPubMedCentralGoogle Scholar
  51. Marsh, A. A., Elfenbein, H. A., & Ambady, N. (2003). Nonverbal “accents” cultural differences in facial expressions of emotion. Psychological Science, 14(4), 373–376.CrossRefPubMedGoogle Scholar
  52. Matson, J. L., Matheis, M., Burns, C., Esposito, G., Venuti, P., Pisula, E., … Kamio, Y. (2017). Examining cross-cultural differences in autism spectrum disorder: A multinational comparison from Greece, Italy, Japan, Poland, and the United States. European Psychiatry, 42, 70–76.CrossRefPubMedGoogle Scholar
  53. Matson, J. L., Nebel-Schwalm, M., & Matson, M. L. (2007). A review of methodological issues in the differential diagnosis of autism spectrum disorders in children. Research in Autism Spectrum Disorders, 1(1), 38–54. doi: 10.1016/j.rasd.2006.07.004.CrossRefGoogle Scholar
  54. Matson, J. L., Worley, J. A., Fodstad, J. C., Chung, K.-M., Suh, D., Jhin, H. K., … Furniss, F. (2011). A multinational study examining the cross cultural differences in reported symptoms of autism spectrum disorders: Israel, South Korea, the United Kingdom, and the United States of America. Research in Autism Spectrum Disorders, 5(4), 1598–1604.CrossRefGoogle Scholar
  55. Mazefsky, C. A., & Oswald, D. P. (2006). The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting. Autism: The International Journal of Research and Practice, 10(6), 533–549.CrossRefGoogle Scholar
  56. McCarthy, A., Lee, K., Itakura, S., & Muir, D. W. (2006). Cultural display rules drive eye gaze during thinking. Journal of Cross-Cultural Psychology, 37(6), 717–722. doi: 10.1177/0022022106292079.CrossRefPubMedPubMedCentralGoogle Scholar
  57. McClure, E. B. (2000). A meta-analytic review of sex differences in facial expression processing and their development in infants, children, and adolescents. Psychological Bulletin, 126(3), 424. doi: 10.1037/0033-2909.126.3.424.CrossRefPubMedGoogle Scholar
  58. Mundy, P. (1995). Joint attention and social-emotional approach behavior in children with autism. Development and Psychopathology, 7(01), 63–82. doi: 10.1017/S0954579400006349.CrossRefGoogle Scholar
  59. Muthén, L. K., & Muthén, B. O. (2005). Mplus: Statistical analysis with latent variables: User’s guide. Los Angeles: Muthén & Muthén.Google Scholar
  60. Norbury, C. F., & Sparks, A. (2013). Difference or disorder? Cultural issues in understanding neurodevelopmental disorders. Developmental Psychology, 49(1), 45. doi: 10.1037/a0027446.CrossRefPubMedGoogle Scholar
  61. Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2005). Evidence-based assessment of autism spectrum disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34(3), 523–540. doi: 10.1207/s15374424jccp3403_8.CrossRefPubMedGoogle Scholar
  62. Pae, T.-I., & Park, G.-P. (2006). Examining the relationship between differential item functioning and differential test functioning. Language Testing, 23(4), 475–496.CrossRefGoogle Scholar
  63. Richesson, R., Anderson, M., & Smerek, M. (2014). Race/ethnicity data standards. Retrieved from
  64. Runnels, J. (2013). Measuring differential item and test functioning across academic disciplines. Language Testing in Asia, 3(1), 9. doi: 10.1186/2229-0443-3-9.CrossRefGoogle Scholar
  65. Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In Innovations in Multivariate Statistical Analysis, pp. 233–247. New York: Springer.CrossRefGoogle Scholar
  66. Schofield, T. J., Parke, R. D., Kim, Y., & Coltrane, S. (2008). Bridging the acculturation gap: Parent-child relationship quality as a moderator in Mexican American families. Developmental Psychology, 44(4), 1190. doi: 10.1037/a0012529.CrossRefPubMedGoogle Scholar
  67. Sell, N. K., Giarelli, E., Blum, N., Hanlon, A. L., & Levy, S. E. (2012). A comparison of autism spectrum disorder DSM-IV criteria and associated features among African American and white children in Philadelphia County. Disability and Health Journal, 5(1), 9–17. doi: 10.1016/j.dhjo.2011.08.002.CrossRefPubMedGoogle Scholar
  68. Simons. (2010). SFARI base/SSC researcher welcome packet. Retrieved April 19, 2010, from
  69. Sipes, M., Furniss, F., Matson, J. L., & Hattier, M. (2012). A multinational study examining the cross cultural differences in social skills of children with autism spectrum disorders: A comparison between the United Kingdom and the United States of America. Journal of Developmental and Physical Disabilities, 24(2), 145–154.CrossRefGoogle Scholar
  70. Soto, S., Linas, K., Jacobstein, D., Biel, M., Migdal, T., & Anthony, B. J. (2014). A review of cultural adaptations of screening tools for autism spectrum disorders. Autism, 19(6), 646–661.CrossRefPubMedGoogle Scholar
  71. United States Census Bureau. (2017). Race. Retrieved from
  72. Vrana, S. R., & Rollock, D. (2002). The role of ethnicity, gender, emotional content, and contextual differences in physiological, expressive, and self-reported emotional responses to imagery. Cognition & Emotion, 16(1), 165–192. doi: 10.1080/0269993014300018.CrossRefGoogle Scholar
  73. Western Psychological Services. (2016). Published translations. Retrieved from
  74. Yuki, M., Maddux, W. W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43(2), 303–311. doi: 10.1016/j.jesp.2006.02.004.CrossRefGoogle Scholar
  75. Zebrowitz, L. A., Kikuchi, M., & Fellous, J.-M. (2010). Facial resemblance to emotions: Group differences, impression effects, and race stereotypes. Journal of Personality and Social Psychology, 98(2), 175. doi: 10.1037/a0017990.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Ashley J. Harrison
    • 1
  • Kristin A. Long
    • 2
  • Douglas C. Tommet
    • 3
  • Richard N. Jones
    • 3
  1. 1.Department of Educational PsychologyUniversity of GeorgiaAthensUSA
  2. 2.Boston UniversityBostonUSA
  3. 3.Alpert Medical School of Brown UniversityProvidenceUSA

Personalised recommendations