Abstract
We determined inter-rater agreement for the VB-MAPP, an instrument sometimes used in planning educational goals and evaluating intervention effects for young people with autism. A pair of raters independently rated each of 32 children diagnosed with autism. Intraclass correlation coefficients for the total Milestones and Barrier scores were 0.876 and 0.629, respectively, indicating good and moderate reliability. There was variability in reliability in the different domains of the Milestones Assessment, with most indicating moderate reliability, and most of the individual Barriers Assessment domains indicating poor reliability. These are the first data relevant to the reliability of the VB-MAPP, they suggest that further evaluation of its reliability is merited and that a high reliability for individual domains should not be assumed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th edn.). Arlington: American Psychiatric Publishing.
Barnes, C. S., Mellor, J. R., & Rehfeldt, R. A. (2014). Implementing the verbal behavior milestones assessment and placement program (VB-MAPP). Analysis of Verbal Behavior, 30, 36–47. https://doi.org/10.1007/s40616-013-0004-5.
Carlson, J. F., Geisinger, K. F., & Jonson, J. L. (Eds.). (2017). The twentieth mental measurements yearbook. Lincoln: Buros Center for Testing.
Dixon, M. R. (2014). The PEAK relational training system: Direct training module. Carbondale: Shawnee Scientific.
Dixon, M. R., Belisle, J., Stanley, C., Rowsey, K., Daar, J. H., & Szekely, S. (2015). Toward a behavior analysis of complex language for children with autism: Evaluating the relationship between PEAK and the VB-MAPP. Journal of Developmental and Physical Disabilities, 27, 223–233. https://doi.org/10.1007/s10882-014-9410-4.
Dixon, M. R., Wiggins, S. H., & Belisle, J. (2018). The effectiveness of the peak relational training system and corresponding changes on the VB-MAPP for young adults with autism. Journal of Applied Behavior Analysis, 51, 321–334. https://doi.org/10.1002/jaba.448.
Dunne, S., Foody, M., Barnes-Holmes, Y., Barnes-Holmes, D., & Murphy, C. (2014). Facilitating repertoires of coordination, opposition distinction, and comparison in young children with autism. Behavioral Development Bulletin, 19, 37–47. https://doi.org/10.1037/h0100576
Gould, E., Dixon, D. R., Najdowski, A. C., Smith, M. N., & Tarbox, J. (2011). A review of assessments for determining the content of early intensive behavioral intervention programs for autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 990–1002. https://doi.org/10.1016/j.rasd.2011.01.012.
Grannan, L., & Rehfeldt, R. (2012). Emergent intraverbal responses via tact and match-to-sample instruction. Journal of Applied Behavior Analysis, 45, 601–605. https://doi.org/10.1901/jaba.2012.45-601.
Gunby, K. V., Carr, J. E., & LeBlanc, L. A. (2010). Teaching abduction-prevention skills to children with autism. Journal of Applied Behavior Analysis, 41, 107–112. doi:0.1901/jaba.2010.43-107.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163. https://doi.org/10.1016/j.jcm.2016.02.012.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
Lotfizadeh, A. D., Kazemi, E., Pompa-Craven, P., & Eldevik, S. (2018). Moderate effects of low-intensity behavioral intervention. Behavior Modification. https://doi.org/10.1177/0145445518796204.
Page, T. J., & Iwata, B. A. (1986). Interobserver agreement. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis (pp. 99–126). Boston: Springer.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. https://doi.org/10.1037/0033-2909.86.2.420.
Skinner, B. F. (1957). Verbal behavior. Englewood Cliffs: Prentice Hall.
Stolte, M., Hodgetts, S., & Smith, V. (2016). A critical review of outcome measures used to evaluate the effectiveness of comprehensive, community based treatment for young children with ASD. Research in Autism Spectrum Disorders, 23, 221–234. https://doi.org/10.1016/j.rasd.2015.12.009.
Sundberg, M. L. (2008). Verbal behavior milestones assessment and placement program: The VB-MAPP. Concord: AVB Press.
Sundberg, M. L. (2014). The verbal behavior milestones assessment and placement program: The VB-MAPP (2nd edn.). Concord: AVB Press.
Walter, S. D., Eliasziw, M., & Donner, A. (1998). Sample size and optimal designs for reliability studies. Statistics in Medicine, 17(1), 101–110. https://doi.org/10.1002/(SICI)1097-0258(19980115)17:1%3C101::AID-SIM727%3E3.0.CO;2-E.
Watkins, M. W., & Pacheco, M. (2000). Interobserver agreement in behavioral research: Importance and calculation. Journal of Behavioral Education, 10, 205–212. https://doi.org/10.1023/A:1012295615144.
Acknowledgments
Special thanks to Alyssa Kavner for her assistance with data entry and IOA calculations.
Author information
Authors and Affiliations
Contributions
All authors contributed equally to this study and to the preparation of this manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical Approval
All procedures performed were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was approved by a Human Subjects Institutional Review Board (HSIRB).
Informed Consent
Informed consent was obtained from all individual participants included in the study prior to the start of services.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Montallana, K.L., Gard, B.M., Lotfizadeh, A.D. et al. Inter-Rater Agreement for the Milestones and Barriers Assessments of the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP). J Autism Dev Disord 49, 2015–2023 (2019). https://doi.org/10.1007/s10803-019-03879-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10803-019-03879-4