Measuring Anxiety as a Treatment Endpoint in Youth with Autism Spectrum Disorder

Abstract

Despite the high rate of anxiety in individuals with autism spectrum disorder (ASD), measuring anxiety in ASD is fraught with uncertainty. This is due, in part, to incomplete consensus on the manifestations of anxiety in this population. Autism Speaks assembled a panel of experts to conduct a systematic review of available measures for anxiety in youth with ASD. To complete the review, the panel held monthly conference calls and two face-to-face meetings over a fourteen-month period. Thirty eight published studies were reviewed and ten assessment measures were examined: four were deemed appropriate for use in clinical trials, although with conditions; three were judged to be potentially appropriate, while three were considered not useful for clinical trials assessing anxiety. Despite recent advances, additional relevant, reliable and valid outcome measures are needed to evaluate treatments for anxiety in ASD.

This is a preview of subscription content, log in to check access.

References

  1. Achenbach, T. M. (1991). Manual for the child behavior checklist/4-18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.

    Google Scholar 

  2. Achenbach, T. M. (2005). Advancing assessment of children and adolescents: Commentary on evidence-based assessment of child and adolescent disorders. Journal of Clinical Child and Adolescent Psychology, 34(3), 541–547.

    PubMed  Article  Google Scholar 

  3. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.

    Google Scholar 

  4. Aman, M. G., De Smedt, G., Derivan, A., Lyons, B., Findling, R. L., & Risperidone Disruptive Behavior Study Group. (2002). Double-blind, placebo-controlled study of risperidone for the treatment of disruptive behaviors in children with subaverage intelligence. American Journal of Psychiatry, 159(8), 1337–1346.

    Google Scholar 

  5. Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985). The aberrant behavior checklist—A behavior rating-scale for the assessment of treatment effects. American Journal of Mental Deficiency, 89(5), 485–491.

    PubMed  Google Scholar 

  6. Aman, M. G., Tasse, M. J., Rojahn, J., & Hammer, D. (1996). The Nisonger CBRF: A child behavior rating form for children with developmental disabilities. Research in Developmental Disabilities, 17, 41–57.

    PubMed  Article  Google Scholar 

  7. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revised). Washington, DC: American Psychiatric Association.

  8. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Association.

    Google Scholar 

  9. Barmish, A. J., & Kendall, P. C. (2005). Should parents be co-clients in cognitive-behavioral therapy for anxious youth? Journal of Clinical Child and Adolescent Psychology, 34(3), 569–581.

    PubMed  Article  Google Scholar 

  10. Birmaher, B., Brent, D. A., Chiapetta, L., Bridge, J., Monga, S., & Baugher, M. (1999). Psychometric properties of the screen for child anxiety related emotional disorders (SCARED): A replication study. Journal of the American Academy of Child and Adolescent Psychiatry, 38(10), 1230–1236.

    PubMed  Article  Google Scholar 

  11. Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., et al. (1997). The screen for child anxiety related emotional disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), 545–553.

    PubMed  Article  Google Scholar 

  12. Cartwright-Hatton, S., McNally, D., Field, A. P., Rust, S., Laskey, B., Dixon, C., et al. (2011). A new parenting-based group intervention for young anxious children: Results of a randomized controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 50(3), 242–251.e6.

    PubMed  Article  Google Scholar 

  13. Centers for Disease Control and Prevention. (2012). Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, 14 sites, United States, 2008. Morbidity and Mortality Weekly Report. Surveillance Summaries. Volume 61, Number 3.

  14. Centre for Childhood Disability Research. (2004). Outcome measures rating form guidelines. www.canchild.ca. Accessed March 19, 2013.

  15. Chalfant, A. M., Rapee, R., & Carroll, L. (2007). Treating anxiety disorders in children with high functioning autism spectrum disorders: A controlled trial. Journal of Autism and Developmental Disorders, 37(10), 1842–1857.

    PubMed  Article  Google Scholar 

  16. Chorpita, B. F., Moffitt, C. E., & Gray, J. A. (2005). Psychometric properties of the Revised Child Anxiety and Depression Scale in a clinical sample. Behaviour Research and Therapy, 43, 309–322.

    PubMed  Article  Google Scholar 

  17. Chorpita, B. F., & Nakamura, B. J. (2008). Dynamic structure in diagnostic structured interviewing: A comparative test of accuracy and efficiency. Journal of Psychopathology and Behavioral Assessment, 30, 52–60.

    Article  Google Scholar 

  18. Chorpita, B. F., Yim, L. M., Moffitt, C. E., Umemoto, L. A., & Francis, S. E. (2000). Assessment of symptoms of DSM-IV anxiety and depression in children: A Revised Child Anxiety and Depression Scale. Behaviour Research and Therapy, 38, 835–855.

    PubMed  Article  Google Scholar 

  19. Costello, E. J., Egger, H. L., & Angold, A. (2005). The developmental epidemiology of anxiety disorders: Phenomenology, prevalence, and comorbidity. Child and Adolescent Psychiatric Clinics of North America, 14(4), 631–648, vii.

    Google Scholar 

  20. Curry, J. F., & Craighead, W. E. (1990). Attributional style in clinically depressed and conduct disordered adolescents. Journal of Consulting and Clinical Psychology, 58, 109–115.

    PubMed  Article  Google Scholar 

  21. Dierker, L. C., Albano, A. M., Clarke, G. N., Heimberg, R. G., Kendall, P. C., Merikangas, K. R., et al. (2001). Screening for anxiety and depression in early adolescence. Journal of the American Academy of Child and Adolescent Psychiatry, 40(8), 929–936.

    PubMed  Article  Google Scholar 

  22. Diggle, P. J., Heagerty, P., Liang, K.-Y., & Zeger, S. L. (2002). Analysis of Longitudinal Data (2nd ed.). Oxford: Oxford University Press.

  23. Dutra, L., Campbell, L., & Westen, D. (2004). Quantifying clinical judgment in the assessment of adolescent psychopathology: Reliability, validity, and factor structure of the child behavior checklist for clinician report. Journal of Clinical Psychology, 60(1), 65–85.

    PubMed  Article  Google Scholar 

  24. Ebesutani, C., Bernstein, A., Nakamura, B. J., Chopita, B. F., Higa-McMillan, C. K., & Weisz, J. R. (2010). Concurrent validity of the child behavior checklist DSM-oriented scales: Correspondence with DSM diagnoses and comparison to syndrome scales. Journal of Psychopathology and Behavioural Assessment, 32(3), 373–384.

    Article  Google Scholar 

  25. Esbensen, A. J., Rojahn, J., Aman, M. G., & Ruedrich, S. (2003). Reliability and validity of an assessment instrument for anxiety, depression, and mood among individuals with mental retardation. Journal of Autism and Developmental Disorders, 33(6), 617–629.

    PubMed  Article  Google Scholar 

  26. Evans, D. W., Canavera, K., Kleinpeter, F. L., Maccubbin, E., & Taga, K. (2005). The fears, phobias and anxieties of children with autism spectrum disorders and down syndrome: Comparisons with developmentally and chronologically age matched children. Child Psychiatry and Human Development, 36(1), 3–26.

    PubMed  Article  Google Scholar 

  27. Fischbach, G. D., & Lord, C. (2010). The Simons simplex collection: A resource for identification of autism genetic risk factors. Neuron, 68(2), 192–195.

    PubMed  Article  Google Scholar 

  28. Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Satistical methods for rates and proportions. Wiley series in probability and statistics (3rd ed.). Hoboken, NJ: Wiley-Interscience.

    Google Scholar 

  29. Fombonne, E. (2009). The epidemiology of pervasive developmental disorders. Pediatric Research, 65(6), 591–598.

    PubMed  Article  Google Scholar 

  30. Gadow, K. D., Devincent, C. J., Pomeroy, J., & Azizian, A. (2005). Comparison of DSM-IV symptoms in elementary school-age children with PDD versus clinic and community samples. Autism: The International Journal of Research and Practice, 9(4), 392–415.

    Article  Google Scholar 

  31. Gadow, K. D., & Sprafkin, J. (2002). Child symptom inventory-4 screening and norms manual. Stony Brook, NY: Checkmate Plus.

    Google Scholar 

  32. Ginsburg, G. S., Keeton, C. P., Drazdowski, T. K., & Riddle, M. A. (2011b). The utility of clinicians ratings of anxiety using the Pediatric Anxiety Rating Scale (PARS). Child and Youth Care Forum 40(2), 93–105.

    Google Scholar 

  33. Ginsburg, G. S., Kendall, P. C., Sakolsky, D., Compton, S. N., Piacentini, J., Albano, A. M., et al. (2011a). Remission after acute treatment in children and adolescents with anxiety disorders: Findings from the CAMS. Journal of Consulting and Clinical Psychology, 79(6), 806.

    PubMed Central  PubMed  Article  Google Scholar 

  34. Gotham, K., Bishop, S. L., Hus, V., Huerta, M., Lund, S., Buja, A., et al. (2012). Exploring the relationship between anxiety and insistence on sameness in autism spectrum disorders. Autism Research, 6(1), 33–41.

    Google Scholar 

  35. Grills, A. E., & Ollendick, T. H. (2003). Multiple informant agreement and the anxiety disorders interview schedule for parents and children. Journal of the American Academy of Child and Adolescent Psychiatry, 42(1), 30–40.

    Google Scholar 

  36. Grondhuis, S. N., & Aman, M. G. (2012). Assessment of anxiety in children and adolescents with autism spectrum disorders. Research in Autism Spectrum Disorders, 6(4), 1345–1365.

    Article  Google Scholar 

  37. Hallett, V., Lecavalier, L., Sukhodolsky, D. G., Cipriano, N., Aman, M. G., McCracken, J. T., et al. (2013a). Exploring the manifestations of anxiety in children with autism spectrum disorders. Journal of Autism and Developmental Disorders. [Epub ahead of print].

  38. Hallett, V., Ronald, A., Colvert, E., Ames, C., Woodhouse, E., Lietz, S., et al. (2013b). Exploring anxiety symptoms in a large-scale twin study of children with autism spectrum disorders, their co-twins and controls. Journal of Child Psychology and Psychiatry, 54, 1176–1185.

    Google Scholar 

  39. Hallett, V., Ronald, A., Rijsdijk, F., & Eley, T. C. (2009). Phenotypic and genetic differentiation of anxiety-related behaviors in middle childhood. Depression and Anxiety, 26(4), 316–324.

    PubMed  Article  Google Scholar 

  40. Hallett, V., Ronald, A., Rijsdijk, F., & Happé, F. (2010). Association of autistic-like and internalizing traits during childhood: A longitudinal twin study. American Journal of Psychiatry, 167(7), 809–817.

    PubMed  Article  Google Scholar 

  41. Kerns, C. M., & Kendall, P. C. (2012). The presentation and classification of anxiety in autism spectrum disorder. Clinical Psychology: Science and Practice, 19(4), 323–347.

    Google Scholar 

  42. King, B. H., Hollander, E., Sikich, L., McCracken, J. T., Scahill, L., Bregman, J. D., et al. (2009). Lack of efficacy of citalopram in children with autism spectrum disorders and high levels of repetitive behavior: Citalopram ineffective in children with autism. Archives of General Psychiatry, 66(6), 583–590.

    PubMed  Article  Google Scholar 

  43. Laughren, T. P. (2003). Comorbid mood disorders and medical illness: A Food and Drug Administration perspective. Biological Psychiatry, 54(3), 195–199.

    PubMed  Article  Google Scholar 

  44. Laughren, T., & Levin, R. (2006). Food and Drug Administration perspective on negative symptoms in schizophrenia as a target for a drug treatment claim. Schizophrenia Bulletin, 32(2), 220–222.

    PubMed Central  PubMed  Article  Google Scholar 

  45. Lecavalier, L. (2006). Behavioral and emotional problems in young people with pervasive developmental disorders: Relative prevalence, effects of subject characteristics, and empirical classification. Journal of Autism and Developmental Disorders, 36(8), 1101–1114.

    PubMed  Article  Google Scholar 

  46. Lecavalier, L., Aman, M. G., Hammer, D., Stoica, W., & Matthews, G. (2004). Factor Analysis of the Nisonger Child Behavior Rating Form in Children with Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 34, 709–721.

    PubMed  Article  Google Scholar 

  47. LeDoux, J. (1998). Fear and the brain: Where have we been, and where are we going? Biological Psychiatry, 44(12), 1229–1238.

    PubMed  Article  Google Scholar 

  48. Leyfer, O. T., Folstein, S. E., Bacalman, S., Davis, N. O., Dinh, E., Morgan, J., et al. (2006). Comorbid psychiatric disorders in children with autism: Interview development and rates of disorders. Journal of Autism and Developmental Disorders, 36(7), 849–861.

    PubMed  Article  Google Scholar 

  49. March, J. S., Parker, J. D., Sullivan, K., Stallings, P., & Conners, C. K. (1997). The Multidimensional Anxiety Scale for Children (MASC): Factor structure, reliability, and validity. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), 554–565.

    PubMed  Article  Google Scholar 

  50. Marcus, R. N., Owen, R., Kamen, L., Manos, G., McQuade, R. D., Carson, W. H., et al. (2009). A placebo-controlled, fixed-dose study of aripiprazole in children and adolescents with irritability associated with autistic disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 48(11), 1110–1119.

    PubMed  Article  Google Scholar 

  51. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

    Google Scholar 

  52. Perrin, S., & Last, C. G. (1992). Do childhood anxiety measures measure anxiety? Journal of Abnormal Child Psychology, 20, 567–578.

    PubMed  Article  Google Scholar 

  53. Reaven, J., Blakeley-Smith, A., Culhane-Shelburne, K., & Hepburn, S. (2012). Group cognitive behavior therapy for children with high-functioning autism spectrum disorders and anxiety: A randomized trial. Journal of Child Psychology and Psychiatry, 53(4), 410–419.

    PubMed  Article  Google Scholar 

  54. Reaven, J., Blakeley-Smith, A., Nichols, S., Dasari, M., Flanigan, E., & Hepburn, S. (2009). Cognitive behavioral group treatment for anxiety symptoms in children with high functioning autism spectrum disorders: A pilot study. Focus on Autism and Other Developmental Disabilities, 24, 27–37.

    Google Scholar 

  55. Renno, P., & Wood, J. (in press). Discriminant and convergent validity of the anxiety construct in children with autism spectrum disorders. Journal of Autism and Developmental Disorders.

  56. Research Units on Pediatric Psychopharmacology (RUPP) Autism Network. (2002). Risperidone in children with autism and serious behavioral problems. New England Journal of Medicine, 347(5), 314–321.

    Article  Google Scholar 

  57. Research Units on Pediatric Psychopharmacology Anxiety Study Group. (2001). Fluvoxamine for the treatment of anxiety disorders in children and adolescents. New England Journal of Medicine, 344, 1279–1285.

    Article  Google Scholar 

  58. Research Units on Pediatric Psychopharmacology Anxiety Study Group. (2002). The pediatric anxiety rating scale (PARS): Development and psychometric properties. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 1061–1069.

    Article  Google Scholar 

  59. Research Units on Pediatric Psychopharmacology Autism Network. (2005). Randomized, controlled, crossover trial of methylphenidate in pervasive developmental disorders with hyperactivity. Archives of General Psychiatry, 62(11), 1266–1274.

    Article  Google Scholar 

  60. Reynolds, C. R., & Richmond, B. O. (1978). What I think and feel: A revised measure of child’s manifest anxiety. Journal of Abnormal Child Psychology, 6, 271–280.

    PubMed  Article  Google Scholar 

  61. Silverman, W. K., & Albano, A. M. (1996). Anxiety disorders interview schedule for DSM-IV. Oxford: Oxford University Press.

    Google Scholar 

  62. Silverman, W. K., Saavedra, L. M., & Pina, A. A. (2001). Test–retest reliability of anxiety symptoms and diagnoses with the anxiety disorders interview schedule for DSM-IV: Child and parent versions. Journal of the American Academy of Child and Adolescent Psychiatry, 40(8), 937–944.

    PubMed  Article  Google Scholar 

  63. Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., & Baird, G. (2008). Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry, 47(8), 921–929.

    PubMed  Article  Google Scholar 

  64. Snyder, R., Turgay, A., Aman, M. G., Binder, C., Fisman, S., Carroll, A., et al. (2002). Effects of risperidone on conduct and disruptive behavior disorders in children with subaverage IQs. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 1026–1036.

    PubMed  Article  Google Scholar 

  65. Storch, E. A., Arnold, E. B., Lewin, A. B., Nadeau, J. M., Jones, A. M., De Nadai, A. S., et al. (2013). The effect of cognitive-behavioral therapy versus treatment as usual for anxiety in children with autism spectrum disorders: a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 52(2), 132–142.

    PubMed  Article  Google Scholar 

  66. Storch, E. A., Ehrenreich, J., Wood, J. J., Jones, A. M., De Nadai, A. S., Lewin, A. B., et al. (2012b). Multiple informant agreement on the anxiety disorders interview schedule for youth with autism spectrum disorders. Journal of Child and Adolescent Psychopharmacology, 22(4), 292–299.

  67. Storch, E. A., Wood, J. J., Ehrenreich May, J., Lewin, A. B., Jones, A. M., Park, J. M., et al. (2012a). Convergent and divergent validity and reliability of the Pediatric Anxiety Rating Scale. Journal of Autism and Developmental Disorders, 42, 2374–2382.

    PubMed  Article  Google Scholar 

  68. Su, L., Wang, K., Fan, F., Su, Y., & Gao, X. (2008). Reliability and validity of the screen for child anxiety related emotional disorders (SCARED) in Chinese children. Journal of Anxiety Disorders, 22(4), 612–621.

    PubMed  Article  Google Scholar 

  69. Sukhodolsky, D. G., Scahill, L., Gadow, K. D., Arnold, L. E., Aman, M. G., McDougle, C. J., et al. (2008). Parent-rated anxiety symptoms in children with pervasive developmental disorders: Frequency and association with core autism symptoms and cognitive functioning. Journal of Abnormal Child Psychology, 36, 117–128.

    PubMed  Article  Google Scholar 

  70. Thayler, N. S., Kazemi, E., & Wood, J. J. (2010). Measuring anxiety in youth with learning disabilities: Reliability and validity of the Multidimensional Anxiety Scale for Children (MASC). Child Psychiatry and Human Development, 41(5), 501–514.

    Article  Google Scholar 

  71. van Steensel, F. J., Bogels, S. M., & Perrin, S. (2011). Anxiety disorders in children and adolescents with autistic spectrum disorders: A meta-analysis. Clinical Child and Family Psychology Review, 14(3), 302–317.

    Google Scholar 

  72. Walkup, J. T., Albano, A. M., Piacentini, J., Birmaher, B., Compton, S. N., Sherrill, J. T., et al. (2008). Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. New England Journal of Medicine, 359(26), 2753–2766.

    PubMed Central  PubMed  Article  Google Scholar 

  73. Weisbrot, D. M., Gadow, K. D., DeVincent, C. J., & Pomeroy, J. (2005). The presentation of anxiety in children with pervasive developmental disorders. Journal of Child and Adolescent Psychopharmacology, 15(3), 477–496.

    PubMed  Article  Google Scholar 

  74. White, S. W., Ollendick, T., Albano, A. M., Oswald, D., Johnson, C., Southam-Gerow, M. A., et al. (2013). Randomized controlled trial: Multimodal anxiety and social skill intervention for adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 43(2), 382–394.

    PubMed Central  PubMed  Article  Google Scholar 

  75. White, S. W., Oswald, D., Ollendick, T., & Scahill, L. (2009). Anxiety in children and adolescents with autism spectrum disorders. Clinical Psychology Review, 29(3), 216–229.

    PubMed Central  PubMed  Article  Google Scholar 

  76. Wood, J. J., Drahota, A., Sze, K. M., Har, K., Chiu, A., & Langer, D. (2009a). Cognitive behavioral therapy for anxiety in children with autism spectrum disorders: A randomized, controlled trial. Journal of Child Psychology and Psychiatry, 50, 224–234.

    PubMed  Article  Google Scholar 

  77. Wood, J. J., & Gadow, K. D. (2010). Exploring the nature and function of anxiety in youth with autism spectrum disorders. Clinical Psychology: Research and Practice, 17, 281–292.

    Google Scholar 

  78. Wood, J. J., McLeod, B., Piacentini, J. C., & Sigman, M. (2009b). One-year follow-up of family versus child CBT for anxiety disorders: Exploring the roles of child age and parental intrusiveness. Child Psychiatry and Human Development, 40, 301–316.

    PubMed Central  PubMed  Article  Google Scholar 

  79. Wood, J. J., Piacentini, J. C., Bergman, R. L., McCracken, J., & Barrios, V. (2002). Concurrent validity of the anxiety disorders section of the anxiety disorders interview schedule for DSM-IV: Child and parent versions. Journal of Clinical Child and Adolescent Psychology, 31(3), 335–342.

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

Autism Speaks provided resources for the collaborative activities of this workgroup.

Conflict of interest

Dr. Scahill: Roche, consultant; Pfizer, consultant; Bracket, consultant, BioMarin, consultant. Shire, research support; Roche, research support; Pfizer, research support. Dr. Aman: Roche, consultant; Bristol-Meyers Squibb, consultant, research grant; CogState Inc, Investigator training, Forest, consultant; Pfizer, consultant; Supernus, consultant; Johnson & Johnson, research grant. Dr. Handen has received research support from Eli Lilly, Curemark and Roche. Dr. King reports serving as a consultant to Biomarin and Neuropharm and as an unpaid consultant to Forest, Nastech, and Seaside Therapeutics. He has received or has pending research grant support from Neuropharm and Seaside Therapeutics. Dr. Pearson has received research support from Curemark LLC and Forest, has pending support from BioMarin and Novartis, and has served as a consultant to Curemark LLC and Bracket. Drs. Horrigan and Jones are currently employed at Neuren Pharmaceuticals. Dr. Cook has served as a consultant and has received research grant support from Seaside Therapeutics. Dr. Dawson is on the Professional Advisory Board for Integragen, Inc.; Consultant, Nastech, Inc., Seaside Therapeutics, Inc. Drs. Lecavalier, Wood, Hallett, Sullivan, and Ms. Grondhuis report no financial relationships with commercial interests.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Lawrence Scahill.

Appendix: Descriptive and Evaluative Criteria for Measures Reviewed (for details on rating of all instruments reviewed contact the authors)

Appendix: Descriptive and Evaluative Criteria for Measures Reviewed (for details on rating of all instruments reviewed contact the authors)

  1. 1.

    Respondent type indicates who reports on behavior, including self, caregiver, clinician, or other person (sibling, classmate).

  2. 2.

    Age and requirement of verbal skills reflects the degree to which the measure requires verbal skills by the person with ASD and the extent to which it can be used across a range of developmental level and chronological age. To be reliable and valid, some measures may rely less on language and more on other communication skills. For example, individuals may have language but may not have the pragmatic skills to meet the demands of some tests. Other measures may require the individual to respond within a specific time period or require a motoric response. These tests may require adequate receptive language skills. Abilities and knowledge in these areas may vary for children with ASD. The notation in this category indicates whether verbal skills are required by the instrument to assess the domain of interest.

  3. 3.

    Comment on the use in ASD, DD population or both follows from the unique challenges of assessing behavior in youth with ASD. A “yes” in this column indicates that the instrument has been studied in one or more samples of individuals with ASD (e.g., community sample, clinical sample, an intervention trial).

  4. 4.

    Clinical relevance reflects the degree to which the measure captures dimensions of anxiety that are applicable to youth with ASD.

  5. 5.

    Reliability reflects the extent to which the measure assesses anxiety consistently. Measures of reliability include internal consistency (items or dimensions contribute evenly to the total score), test–retest reliability (correlation of score over brief periods of time), inter-rater reliability (level of agreement across raters). Inter-rater reliability is particularly important in clinician-rated instruments (e.g., Pediatric Anxiety Rating Scale). The demand for inter-rater reliability may be lower for informant-based measures (e.g., parents and teachers may not be in agreement about a child’s behavior given the differences in setting). The following rating system was adapted from the Centre for Childhood Disability Research (2004) for this review.

    • Excellent (e.g., publication of more than 3 studies with reliability statistics in the excellent range plus use of the measure in at least two clinical trials with adequate statistical power.

    • Good (e.g. 2 reliability studies completed with adequate to excellent reliability values plus use of at least one clinical trial with adequate statistical power.

    • Adequate (e.g. 1 reliability study completed with adequate to excellent reliability values)

    • Poor (e.g., reliability studies poorly completed, or reliability studies showing inadequate levels of reliability)

    • No evidence available

  6. 6.

    Validity reflects the degree to which the instrument measures what it purports to measure. The review considered (a) content validity, i.e., The degree to which the measure adequately reflects anxiety; (b) construct validity, i.e., the degree to which the measure positively correlates with another measure of anxiety (convergent validity) and the degree to which it does not correlate with a measure presumed to measure a construct separate from anxiety (divergent validity). The following ratings were used:

    Content Validity

    • Excellent: (e.g., expert judgment or statistical method (e.g., factor analysis) was used to determine that the measure adequately covers the domain of interest without inclusion of unrelated material;

    • Adequate: Has reported content validity but no specific method was used

    • Poor: Instrument is not comprehensive

    • No evidence available

      Construct Validity

    • Excellent: (e.g., more than 2 independent, experimental studies have shown adequate agreement with a gold standard measure, stronger for overall evaluation if determination of both convergent and divergent validity reported)

    • Adequate: (e.g. 1–2 studies demonstrate confirmation of theoretical formulation; determination of either convergent or divergent validity reported)

    • Poor: (e.g., construct validation poorly documented or construct validity not supported by the study)

    • No evidence available

  7. 7.

    Sensitivity to change. Measures with sufficient assay sensitivity are typically used as the primary endpoint in pivotal clinical trials in support of regulatory approval for a new drug or device, or new indications for existing drug or device. Validity and reliability are pre-requisites for assay sensitivity, but high levels of validity and reliability do not guarantee sensitivity to change. Measures of assay sensitivity include effect sizes and receiver operating characteristic curves (Diggle et al. 2002). The assessment of assay sensitivity may include review of design elements (e.g., entry criteria, length of trial, and selection of control condition) and properties of the measure itself (scoring range, number of items). Although sensitivity to change is essential for a measure to be useful as a primary endpoint, failure of a given measure to demonstrate sensitivity to change may reflect misapplication of the measure, design problems in the trial or lack of efficacy for the intervention. Therefore, in this review, failure of a measure to demonstrate sensitivity to change would not result in discounting the potential for the measure as a primary endpoint. Ratings of demonstrated sensitivity to change were defined as follows:

    • Excellent: More than 2 experimental trials have demonstrated that measure captures changes in outcome (improvement or decline), in a clear and consistent manner

    • Adequate: 1–2 studies (experimental or quasi-experimental) have demonstrated that the measure captures changes in outcome, but evidence is incomplete or inconclusive. For example, evidence is limited to only one or two studies; evidence is available for more than two studies, but findings across studies are not consistent. An assessment of adequate could also be given if available evidence is inconclusive due to study design limitations making it difficult for the instrument to detect change due to variability. For example, a relatively small sample with unclear entry criteria resulting in large variability on the primary outcome.

    • Poor: Available studies have failed to show the capacity of the measure to differentiate active from a control condition.

    • No evidence available or limited evidence due to type of study (open label, small pilot trial, randomized trial with design limitations, single dose study only).

  8. 8.

    Burden reflects the time, demand structure or difficulty associated with collection of the measure. The review considered: (1) the individual with autism, (2) other persons who would complete the measure, such as parents, teachers, and clinicians, and (3) the investigator. For the individual with ASD, considerations included whether the test involves an unusually high level of demand, such as sitting motionless for an extended period of time, or an expectation that a person with ASD might find uncomfortable or particularly challenging, such as interacting with groups of people, being touched, or required to comprehend complex, verbal instructions. The length of a measure and frequency of measurement could also affect burden. For informants such as a parent, clinician, or teacher, the review also included length and frequency of the test, level of literacy required, or features of the test that might make the informant uncomfortable, such as inquiring about highly personal matters. For clinician-rated interviews, considerations include the level of training or specific qualifications required to conduct the measure, need for detailed supervision following collection of the measure, whether specialized and/or costly equipment is needed, high cost of administration (e.g., need for complex coding of behavior before analyzing data) and time required to complete the rating. Ratings of burden were classified as follows:

    • Low or none: The participant is observed in a natural environment for 15 min or less or takes 15–30 min to complete the measure by a trained observer. The measure is relatively brief (<20 min), includes current observable behaviors (e.g., past month or less) and requires simple scale (e.g., 0–3). There is little or no distressing content. Measure is easily downloadable, free of charge, and simple to score and interpret. Test is available and has been validated in more than one other language other than English.

    • Medium: The participant is observed or engaged with an unfamiliar interviewer/reporter, for 15–30 min or informant-based rating that takes >20 min to complete. An informant rating includes a long list of questions (e.g., >60) or assessment may have <60 questions, but requires complex directions or responses. There is low to moderate levels of verbal or pragmatic comprehension required. There may be some distressing content. It uses a Likert or Likert type scale, and the rater is required to interact and observe with participant while simultaneously recording responses. The rating may require a manual; however it does not require extensive training. Measure is not free of charge. Test is offered in at least one other language than English, but may not be validated for use in non-English languages. Need for conversion of raw to standardized scores to be calculated by an additional program at a cost.

    • High: The participant is required to spend 30 min or more performing tasks that require sustained attention or the assessment requires extensive preparation (e.g., fMRI) or participant training or high levels of verbal/pragmatic comprehension that may involve complex or distressing responses. The rater is required to interact and observe with participant while simultaneously recording responses using extended semi-structured or non-standardized measures. The measure requires extensive training, must be administered by clinician with at least master’s level and the measure requires >45 min to administer. Measure may not require extended time to administer, but it may involve distressing content or evoke distressing responses. Use of the tool is costly and administration would require investigator to provide additional training to personnel. Test is only available in English and requires use of translator.

  9. 9.

    Overall rating involved the inclusion of each category Most weight was placed on clinical relevance, psychometric soundness, and use in ASD population. However, sensitivity to change and burden were also considered when determining rating as follows:

    • Appropriate: Measure achieved high ratings on clinical relevance; good to excellent reliability and validity with information available on all pertinent indices (e.g., test–retest, internal consistency and construct validity) across a range of youth with ASD or developmentally disabled population. On clinician-rated measures, high priority was given to inter-rater reliability (consistency across informant ratings such as parents and teachers was less important). If sensitivity to change has been demonstrated in an ASD or developmentally disabled population, this enhanced the assessment of the rating. The instrument has low to medium burden to individual, parent, and clinician.

    • Appropriate with Conditions: Measure was considered appropriate with one or more qualifications. The psychometric properties have been evaluated in an ASD or developmentally disabled population, but available information is restricted to a specific age range or a specific subgroup such as high functioning ASD. Measure was considered relevant with good to excellent reliability and validity with information on several, but not all, pertinent indices (e.g., limited information on test–retest reliability or convergent validity). On clinician-rated measures, high priority was given to inter-rater reliability (consistency across informant ratings such as parents and teachers was less important). However, it may not be available in multiple languages and translation would be needed for a large-scale international study. Measure has medium or high level of burden and some data on sensitivity to change.

    • Potentially Appropriate: The instrument measures a clinically relevant outcome; however, there is emerging or inconsistent reliability and validity in ASD or developmentally disabled populations. The measure may have demonstrated validity for screening or diagnosis, but no evidence supporting its use as an outcome measure. Measure may be appropriate for a subgroup in the ASD population, but more study is needed to establish the reliability, validity and usefulness as an outcome measure.

    • Unproven: The instrument measures a clinically relevant outcome. However, reliability and validity have been inadequately evaluated or not evaluated. Measure may have evidence for use in other pediatric populations, but has little or no evidence supporting its use in an ASD or developmentally disabled populations.

    • Not Appropriate: Measure is not relevant to the outcome of interest or was developed for screening purposes rather than as a change measure. Available information indicates poor reliability, validity or both.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lecavalier, L., Wood, J.J., Halladay, A.K. et al. Measuring Anxiety as a Treatment Endpoint in Youth with Autism Spectrum Disorder. J Autism Dev Disord 44, 1128–1143 (2014). https://doi.org/10.1007/s10803-013-1974-9

Download citation

Keywords

  • Autism spectrum disorder
  • Anxiety
  • Instrument
  • Measure
  • Assessment
  • Treatment
  • Intervention