Advertisement

Journal of Autism and Developmental Disorders

, Volume 48, Issue 6, pp 1932–1944 | Cite as

An Initial Psychometric Evaluation of the Joint Attention Protocol

  • Sallie W. NowellEmail author
  • Linda R. Watson
  • Richard A. Faldowski
  • Grace T. Baranek
Original Paper

Abstract

The goal of this paper is to examine the psychometric properties of a live-coded behavioral measure of joint attention, the Attention-Following and Initiating Joint Attention Protocol (JA Protocol), in order to assist researchers and clinicians in identifying when this measure may meet their joint attention assessment needs. Data from 260 children with autism spectrum disorder, developmental delay, or typical development between the ages of 2 and 12 years were used to evaluate this measure using quality standards for measurement. Overall, the JA Protocol demonstrated good psychometric properties. Recommendations and limitations for use of this measure based on psychometric analysis results are reported.

Keywords

Joint attention Psychometric Measurement Autism ASD 

Notes

Acknowledgments

This research was supported in part by grants from The National Institute for Child Health and Human Development (R01-HD42168, P30-HD003110) to the University of North Carolina at Chapel Hill. The authors would like to thank the families who participated in the Sensory Experiences Project. The authors also gratefully acknowledge the contributions of Rachel Rhyne in refining the Attention Following and Initiating Joint Attention Protocol manual and for her leadership in collecting test–retest data.

Author Contributions

SWN drafted the manuscript and analysed the data. LRW was a co-investigator for the parent study, led data collection for both of the JA Protocol samples, contributed to writing the introduction, methods, and conclusion sections, and edited the manuscript. RAF co-developed the data analysis plan, assisted SWN with conducting the analyses, contributed to writing the results section for the paper, and edited the manuscript. GTB was the principal investigator for the parent study, contributed to writing the introduction and conclusion sections, and edited the manuscript.

Funding

This research was supported in part by grants from The National Institute for Child Health and Human Development. R01-HD42168 to Grace Baranek and P30-HD003110 to the Carolina Institute for Developmental Disabilities (current PI: Gabriel Dichter) at the University of North Carolina at Chapel Hill.

Compliance with Ethical Standards

Conflict of interest

Dr. Watson and Dr. Baranek have intellectual property rights related to the JA Protocol as authors. Ms. Nowell and Dr. Faldowski declare no conflicts of interest. The authors declare they have no financial conflicts of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Baranek, G. T., Watson, L. R., Boyd, B. A., Poe, M. D., David, F. J., & McGuire, L. (2013). Hyporesponsiveness to social and nonsocial sensory stimuli in children with autism, children with developmental delays, and typically developing children. Development and Psychopathology, 25(2), 307–320.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Baron-Cohen, S. (1991). Precursors to a theory of mind: Understanding attention in others. In A. Whiten (Ed.), Natural theories of mind. Oxford: Basil Blackwell.Google Scholar
  3. Bates, E. (1976). Language and context. New York: Academic Press.Google Scholar
  4. Bonett, D. G., & Price, R. M. (2005). Inferential methods for the tetrachoric correlation coefficient. Journal of Educational and Behavioral Statistics, 30, 213.CrossRefGoogle Scholar
  5. Bracken, B. A. (1987). Limitations of preschool instruments and standards for minimal levels of technical adequacy. Journal of Psychoeducational Assessment, 5(4), 313–326.CrossRefGoogle Scholar
  6. Brady, N. C., Steeples, T., & Fleming, K. (2005). Effects of prelinguistic communication levels on initiation and repair of communication in children with disabilities. Journal of Speech, Language, and Hearing Research, 48(5), 1098–1113.CrossRefPubMedGoogle Scholar
  7. Brown, M. B. (1977). Algorithm AS 116: The tetrachoric correlation and its standard error. Applied Statistics, 26, 343–351.CrossRefGoogle Scholar
  8. Bruinsma, Y., Koegel, R. L., & Koegel, L. K. (2004). Joint attention and children with autism: A review of the literature. Mental Retardation and Developmental Disabilities Research Reviews, 10(3), 169–175.CrossRefPubMedGoogle Scholar
  9. Bruner, J. (1981). The social context of language acquisition. Language & Communication, 1(2), 155–178.CrossRefGoogle Scholar
  10. Carpenter, M., Nagell, K., & Tomasello, M. (1998). Social cognition, joint attention, and communicative competence from 9 to 15 months of age. Monographs of the Society for Research in Child Development, 63, i-174.CrossRefGoogle Scholar
  11. Charman, T. (2003). Why is joint attention a pivotal skill in autism? Philosophical Transactions of the Royal Society of London B: Biological Sciences, 358(1430), 315–324.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284.CrossRefGoogle Scholar
  13. Cicchetti, D. V., & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86, 127–137PubMedGoogle Scholar
  14. Cicchetti, D. V., & Sparrow, S. S. (1990). Assessment of adaptive behavior in young children. In J. H. Johnson & J. Goldman (Eds.), Pergamon general psychology series, 163. Developmental assessment in clinical child psychology: A handbook (pp. 173–196). Elmsford, NY: Pergamon Press.Google Scholar
  15. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.CrossRefGoogle Scholar
  16. Crais, E., Douglas, D. D., & Campbell, C. C. (2004). The intersection of the development of gestures and intentionality. Journal of Speech, Language, and Hearing Research, 47(3), 678–694.CrossRefPubMedGoogle Scholar
  17. Crais, E. R., Watson, L. R., & Baranek, G. T. (2009). Use of gesture development in profiling children’s prelinguistic communication skills. American Journal of Speech-Language Pathology, 18(1), 95–108.CrossRefPubMedGoogle Scholar
  18. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart and Winston.Google Scholar
  19. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.CrossRefGoogle Scholar
  20. DeVon, H. A., Block, M. E., Moyle-Wright, P., Ernst, D. M., Hayden, S. J., Lazzara, D. J., … Kostas-Polston, E. (2007). A psychometric toolbox for testing validity and reliability. Journal of Nursing scholarship, 39(2), 155–164.CrossRefPubMedGoogle Scholar
  21. Drasgow, F. (1988). Polychoric and polyserial correlations. In L. Kotz & N. L. Johnson (Eds.), Encyclopedia of Statistical Sciences (Vol 7, pp. 69–74). New York: Wiley.Google Scholar
  22. Friberg, J. C. (2010). Considerations for test selection: How do validity and reliability impact diagnostic decisions? Child Language Teaching and Therapy, 26(1), 77–92.CrossRefGoogle Scholar
  23. Furr, M. (2011). Core principles, best practices, and an overview of scale construction. In M. Furr (Ed.), Scale construction and psychometrics for social and personality psychology (pp. 4–16) London: SAGE Publications Ltd.CrossRefGoogle Scholar
  24. Harris, B. (1988). Tetrachoric correlation coefficient. In L. Kotz & N. L. Johnson (Eds.), Encyclopedia of Statistical Sciences (Vol 9, pp. 223–225). New York: Wiley.Google Scholar
  25. Hurwitz, S., & Watson, L. R. (2015). Joint attention revisited: Finding strengths among children with autism. Autism.  https://doi.org/10.1177/1362361315593536.PubMedPubMedCentralGoogle Scholar
  26. Hus, V., Gotham, K., & Lord, C. (2014). Standardizing ADOS domain scores: Separating severity of social affect and restricted and repetitive behaviors. Journal of Autism and Developmental Disorders, 40, 1–13.Google Scholar
  27. Kasari, C., Brady, N., Lord, C., & Tager-Flusberg, H. (2013). Assessing the minimally verbal school-aged child with autism spectrum disorder. Autism Research, 6(6), 479–493.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Kasari, C., Gulsrud, A., Freeman, S., Paparella, T., & Hellemann, G. (2012). Longitudinal follow-up of children with autism receiving targeted interventions on joint attention and play. Journal of the American Academy of Child & Adolescent Psychiatry, 51(5), 487–495.CrossRefGoogle Scholar
  29. Klin, A., Saulnier, C., Tsatsanis, K., & Volkmar, F. R. (2005). Clinical evaluation in autism spectrum disorders: Psychological assessment within a transdisciplinary framework. Handbook of Autism and Pervasive Developmental Disorders, 2, 772–798.Google Scholar
  30. Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., … Streiner, D. L. (2011). Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. International Journal of Nursing Studies, 48(6), 661–671.CrossRefPubMedGoogle Scholar
  31. Leekam, S. R., Hunnisett, E., & Moore, C. (1998). Targets and cues: Gaze-following in children with autism. Journal of Child Psychology and Psychiatry, 39(7), 951–962.CrossRefPubMedGoogle Scholar
  32. Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., … Rutter, M. (2000). The Autism Diagnostic Observation Schedule–Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205–223.CrossRefPubMedGoogle Scholar
  33. Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2002). Autism diagnostic observation schedule manual. Los Angeles: Western Psychological Services.Google Scholar
  34. Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. (2012). Autism diagnostic observation schedule: ADOS-2. Los Angeles, CA: Western Psychological Services.Google Scholar
  35. Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24(5), 659–685.CrossRefPubMedGoogle Scholar
  36. Masur, E. F. (1983). Gestural development, dual-directional signaling, and the transition to words. Journal of Psycholinguistic Research, 12(2), 93–109.CrossRefGoogle Scholar
  37. McCauley, R. J., & Swisher, L. (1984). Psychometric review of language and articulation tests for preschool children. Journal of Speech and Hearing Disorders, 49(1), 34–42.CrossRefPubMedGoogle Scholar
  38. Mullen, E. M. (1995). Mullen scales of early learning. Circle Pines, MN: American Guidance Service.Google Scholar
  39. Mundy, P., Delgado, C., Block, J., Venezia, M., Hogan, A., & Seibert, J. (2003). Early Social Communication Scales (ESCS). Coral Gables, FL: University of Miami.Google Scholar
  40. Mundy, P., & Newell, L. (2007). Attention, joint attention, and social cognition. Current Directions in Psychological Science, 16(5), 269–274.CrossRefPubMedPubMedCentralGoogle Scholar
  41. Mundy, P., Novotny, S., Swain-Lerro, L., McIntyre, N., Zajic, M., & Oswald, T. (2017). Joint-attention and the social phenotype of school-aged children with ASD. Journal of Autism and Developmental Disorders, 47(5), 1423–1435.CrossRefPubMedGoogle Scholar
  42. Mundy, P., Sigman, M., & Kasari, C. (1990). A longitudinal study of joint attention and language development in autistic children. Journal of Autism and Developmental Disorders, 20(1), 115–128.CrossRefPubMedGoogle Scholar
  43. Muthen, L. K., & Muthen, B. O. (1998–2017). Mplus user’s guide (7th ed.). Los Angeles, CA: Authors.Google Scholar
  44. Naglieri, J. A., & Chambers, K. M. (2009). Psychometric issues and current scales for assessing autism spectrum disorders. In S. Goldstein, J. A. Naglieri, & S. Ozonoff (Eds.), Assessment of autism spectrum disorders (pp. 55–90). New York: Guilford.Google Scholar
  45. Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2005). Evidence-based assessment of autism spectrum disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34(3), 523–540.CrossRefPubMedGoogle Scholar
  46. Plante, E., & Vance, R. (1994). Selection of preschool language tests a data-based approach. Language, Speech, and Hearing Services in Schools, 25(1), 15–24.CrossRefGoogle Scholar
  47. Poon, K. K., Watson, L. R., Baranek, G. T., & Poe, M. D. (2012). To what extent do joint attention, imitation, and object play behaviors in infancy predict later communication and intellectual functioning in ASD? Journal of Autism and Developmental Disorders, 42(6), 1064–1074.CrossRefPubMedPubMedCentralGoogle Scholar
  48. Rutter, M., Bailey, A., & Lord, C. (2003). Social communication questionnaire. Los Angeles: Western Psychological Services.Google Scholar
  49. Sigman, M., Ruskin, E., Arbelle, S., Corona, R., Dissanayake, C., Espinosa, M., … Robinson, B. F. (1999). Continuity and change in the social competence of children with autism, Down syndrome, and developmental delays. Monographs of the Society for Research in Child Development, 64, i-139.Google Scholar
  50. Sparrow, S., Balla, D., & Cicchetti, D. (1984). Vineland Adaptive Behavior Scales. Circle Pines, MN: American Guidance Service.Google Scholar
  51. Stone, W. L., Coonrod, E. E., & Ousley, O. Y. (2000). Brief report: Screening tool for autism in two-year-olds (STAT): Development and preliminary data. Journal of Autism and Developmental Disorders, 30(6), 607–612.CrossRefPubMedGoogle Scholar
  52. Stone, W. L., Ousley, O. Y., Yoder, P. J., Hogan, K. L., & Hepburn, S. L. (1997). Nonverbal communication in two-and three-year-old children with autism. Journal of Autism and Developmental Disorders, 27(6), 677–696.CrossRefPubMedGoogle Scholar
  53. Stone, W. L., & Yoder, P. J. (2001). Predicting spoken language level in children with autism spectrum disorders. Autism, 5(4), 341–361.CrossRefPubMedGoogle Scholar
  54. Tomasello, M. (1995). Joint attention as social cognition. In C. Moore & P. J. Dunham (Eds.), Joint attention: Its origins and role in development (pp. 103–130). Hillsdale, NJ: Lawrence Erlbaum AssociatesGoogle Scholar
  55. Uebersax, J. S. (2017). Introduction to the tetrachoric and polychoric correlation coefficients. Retrieved May 7, 2017, from http://www.john-uebersax.com/stat/tetra.htm.
  56. Watson, L. R., Poston, V., Baranek, G. T. (2003). The attention-following and initiating joint attention protocol (JA protocol). Unpublished manuscript.Google Scholar
  57. Wetherby, A., & Prizant, B. (2002). Communication and symbolic behavior scales developmental profile—first normed edition. Baltimore: Paul H. Brookes.Google Scholar
  58. Zimmerman, I. L., Steiner, V. G., & Pond, R. E. (2002). Preschool Language Scale–Fourth edition examiner’s manual. San Antonio, TX: Psychological Corporation.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Sallie W. Nowell
    • 1
    • 3
    Email author
  • Linda R. Watson
    • 1
  • Richard A. Faldowski
    • 1
  • Grace T. Baranek
    • 2
  1. 1.Department of Allied Health SciencesThe University of North Carolina at Chapel HillChapel HillUSA
  2. 2.Chan Division of Occupational Science and Occupational TherapyThe University of Southern CaliforniaLos AngelesUSA
  3. 3.Division of Speech & Hearing SciencesThe University of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations