Advertisement

Inter-Rater Agreement for the Milestones and Barriers Assessments of the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP)

  • Khrystle L. Montallana
  • Brendan M. Gard
  • Amin D. LotfizadehEmail author
  • Alan Poling
OriginalPaper
  • 50 Downloads

Abstract

We determined inter-rater agreement for the VB-MAPP, an instrument sometimes used in planning educational goals and evaluating intervention effects for young people with autism. A pair of raters independently rated each of 32 children diagnosed with autism. Intraclass correlation coefficients for the total Milestones and Barrier scores were 0.876 and 0.629, respectively, indicating good and moderate reliability. There was variability in reliability in the different domains of the Milestones Assessment, with most indicating moderate reliability, and most of the individual Barriers Assessment domains indicating poor reliability. These are the first data relevant to the reliability of the VB-MAPP, they suggest that further evaluation of its reliability is merited and that a high reliability for individual domains should not be assumed.

Keywords

Verbal Behavior Milestones Assessment and Placement Program Language assessment Verbal behavior Autism spectrum disorder Inter-rater agreement Reliability 

Notes

Acknowledgments

Special thanks to Alyssa Kavner for her assistance with data entry and IOA calculations.

Author Contributions

All authors contributed equally to this study and to the preparation of this manuscript.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was approved by a Human Subjects Institutional Review Board (HSIRB).

Informed Consent

Informed consent was obtained from all individual participants included in the study prior to the start of services.

References

  1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.Google Scholar
  2. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th edn.). Arlington: American Psychiatric Publishing.CrossRefGoogle Scholar
  3. Barnes, C. S., Mellor, J. R., & Rehfeldt, R. A. (2014). Implementing the verbal behavior milestones assessment and placement program (VB-MAPP). Analysis of Verbal Behavior, 30, 36–47.  https://doi.org/10.1007/s40616-013-0004-5.CrossRefPubMedGoogle Scholar
  4. Carlson, J. F., Geisinger, K. F., & Jonson, J. L. (Eds.). (2017). The twentieth mental measurements yearbook. Lincoln: Buros Center for Testing.Google Scholar
  5. Dixon, M. R. (2014). The PEAK relational training system: Direct training module. Carbondale: Shawnee Scientific.Google Scholar
  6. Dixon, M. R., Belisle, J., Stanley, C., Rowsey, K., Daar, J. H., & Szekely, S. (2015). Toward a behavior analysis of complex language for children with autism: Evaluating the relationship between PEAK and the VB-MAPP. Journal of Developmental and Physical Disabilities, 27, 223–233.  https://doi.org/10.1007/s10882-014-9410-4.CrossRefGoogle Scholar
  7. Dixon, M. R., Wiggins, S. H., & Belisle, J. (2018). The effectiveness of the peak relational training system and corresponding changes on the VB-MAPP for young adults with autism. Journal of Applied Behavior Analysis, 51, 321–334.  https://doi.org/10.1002/jaba.448.CrossRefPubMedGoogle Scholar
  8. Dunne, S., Foody, M., Barnes-Holmes, Y., Barnes-Holmes, D., & Murphy, C. (2014). Facilitating repertoires of coordination, opposition distinction, and comparison in young children with autism. Behavioral Development Bulletin, 19, 37–47.  https://doi.org/10.1037/h0100576 CrossRefGoogle Scholar
  9. Gould, E., Dixon, D. R., Najdowski, A. C., Smith, M. N., & Tarbox, J. (2011). A review of assessments for determining the content of early intensive behavioral intervention programs for autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 990–1002.  https://doi.org/10.1016/j.rasd.2011.01.012.CrossRefGoogle Scholar
  10. Grannan, L., & Rehfeldt, R. (2012). Emergent intraverbal responses via tact and match-to-sample instruction. Journal of Applied Behavior Analysis, 45, 601–605.  https://doi.org/10.1901/jaba.2012.45-601.CrossRefPubMedGoogle Scholar
  11. Gunby, K. V., Carr, J. E., & LeBlanc, L. A. (2010). Teaching abduction-prevention skills to children with autism. Journal of Applied Behavior Analysis, 41, 107–112. doi:0.1901/jaba.2010.43-107.CrossRefGoogle Scholar
  12. Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163.  https://doi.org/10.1016/j.jcm.2016.02.012.CrossRefPubMedGoogle Scholar
  13. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefPubMedGoogle Scholar
  14. Lotfizadeh, A. D., Kazemi, E., Pompa-Craven, P., & Eldevik, S. (2018). Moderate effects of low-intensity behavioral intervention. Behavior Modification.  https://doi.org/10.1177/0145445518796204.PubMedGoogle Scholar
  15. Page, T. J., & Iwata, B. A. (1986). Interobserver agreement. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis (pp. 99–126). Boston: Springer.CrossRefGoogle Scholar
  16. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.  https://doi.org/10.1037/0033-2909.86.2.420.CrossRefPubMedGoogle Scholar
  17. Skinner, B. F. (1957). Verbal behavior. Englewood Cliffs: Prentice Hall.CrossRefGoogle Scholar
  18. Stolte, M., Hodgetts, S., & Smith, V. (2016). A critical review of outcome measures used to evaluate the effectiveness of comprehensive, community based treatment for young children with ASD. Research in Autism Spectrum Disorders, 23, 221–234.  https://doi.org/10.1016/j.rasd.2015.12.009.CrossRefGoogle Scholar
  19. Sundberg, M. L. (2008). Verbal behavior milestones assessment and placement program: The VB-MAPP. Concord: AVB Press.Google Scholar
  20. Sundberg, M. L. (2014). The verbal behavior milestones assessment and placement program: The VB-MAPP (2nd edn.). Concord: AVB Press.Google Scholar
  21. Walter, S. D., Eliasziw, M., & Donner, A. (1998). Sample size and optimal designs for reliability studies. Statistics in Medicine, 17(1), 101–110.  https://doi.org/10.1002/(SICI)1097-0258(19980115)17:1%3C101::AID-SIM727%3E3.0.CO;2-E.CrossRefPubMedGoogle Scholar
  22. Watkins, M. W., & Pacheco, M. (2000). Interobserver agreement in behavioral research: Importance and calculation. Journal of Behavioral Education, 10, 205–212.  https://doi.org/10.1023/A:1012295615144.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Khrystle L. Montallana
    • 1
  • Brendan M. Gard
    • 2
  • Amin D. Lotfizadeh
    • 3
    Email author
  • Alan Poling
    • 4
  1. 1.Autism Services Training DivisionEasterseals Southern CaliforniaValenciaUSA
  2. 2.Autism Services Assessment DivisionEasterseals Southern CaliforniaOxnardUSA
  3. 3.Autism Services Research DivisionEasterseals Southern CaliforniaValenciaUSA
  4. 4.Psychology DepartmentWestern Michigan UniversityKalamazooUSA

Personalised recommendations