Using Expert Panels to Examine the Content Validity and Inter-Rater Reliability of the ABLLS-R
The assessment literature cites several instruments used to assess the skills of children with an autism spectrum disorder (ASD) diagnosis, but many lack adequate empirical support for their psychometric properties. The Assessment of Basic Language and Learning Skills-Revised (ABLLS-R) is a popular assessment used by clinicians to measure the skills of children with ASD. Despite its widespread use, the ABLLS-R contains limited research on its psychometric properties. The current study sought to extend the recent research on the psychometric properties of the ABLLS-R by using data obtained from two separate panels of expert raters to evaluate its content validity and the inter-rater reliability of its scores. Our results demonstrate evidence of content validity as at least five out of six expert panel members rated 441 out of the 544 ABLLS-R items (or 81% of the items in the assessment) as “essential.” We also found evidence of excellent inter-rater reliability (intraclass correlation coefficient = .95, p < .001) across the ABLLS-R scores obtained from a second panel of expert raters. These findings extend the existing literature and further document the ABLLS-R as a valid instrument that yields reliable scores.
KeywordsAutism Assessments ABLLS-R Reliability Validity Psychometric properties
Compliance with Ethical Standards
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
Conflict of Interest
The first and second authors declare that they have no conflict of interest. The third author owns stock in the company that publishes the ABLLS-R.
- Aman, M.G., Novotny, S., Samango-Sprouse, C., Lecavalier, L., Leonard, E., Gadow, K.D…Chez, M. (2004). Outcome measures for clinical drug trials in autism. CNS Spectrums, 9, 36–47. https://doi.org/10.1017/S1092852900008348.
- American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (2014). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
- American Medical Association. (2014). CPT assistant. Chicago: American Medical Association.Google Scholar
- Center for Disease Control and Prevention (2015). Data and statistics. Retrieved from http://www.cdc.gov/NCBDDD/autism/data.html
- Every Student Succeeds Act, Pub. L. No. 114–95 § 114 Stat. 1177 (2015).Google Scholar
- Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004).Google Scholar
- Landers, R. N. (2011). Computing intraclass correlations (ICC) as estimates of inter-rater reliability in SPSS. Retrieved from http://neoacademic.com/2011/11/16/computing-intraclass-correlations-icc-as-estimates-of-inter-rater-reliability-in-spss/#.U6z9uChhmGl. doi:10.15200/winn.143518.81744.
- Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x.CrossRefGoogle Scholar
- Malkin, A., Dixon, M. R., Speelman, R. C., & Luke, N. (2016). Evaluating the relationships between the PEAK Relations Training System-Direct Training Module, Assessment of Basic Language and Learning Skills-Revised, and the Vineland Adaptive Behavior Scales-II. Journal of Developmental and Physical Disabilities. https://doi.org/10.1007/s10882-016-9527-8.
- Partington, J. W. (2010a). The Assessment of Basic Language and Learning Skills-Revised. Pleasant Hill: Behavior Analysts, Inc..Google Scholar
- Partington, J. W. (2010b). The Assessment of Basic Language and Learning Skills-Revised: Scoring Instructions and IEP Development Guide. Pleasant Hill: Behavior Analysts, Inc..Google Scholar
- Partington, J. W., Bailey, A., & Partington, S. W. (2016). A pilot study examining the test-retest and internal consistency reliability of the ABLLS-R. Journal of Psychoeducational Assessment. https://doi.org/10.1177/0734282916678348.
- Salkind, N. (2013). Encyclopedia of research design: Content validity. Thousand Oaks: SAGE Publications, Inc..Google Scholar
- Sundberg, M. (2008). VB-MAPP Verbal Behavior Milestones Assessment and Placement Program: A language and social skills assessment program for children with autism or other developmental disabilities: Guide. Concord: AVB Press.Google Scholar
- Thompson, T. (2007). Making Sense of Autism. Baltimore: Paul H. Brookes Publishing Co..Google Scholar
- Thompson, T. (2011). Individualized Autism Intervention for Young Children. Baltimore: Paul H. Brookes Publishing Co..Google Scholar