When the Right Measure Doesn’t Exist: A Novel Motivational Interviewing Tool for Community Programs

  • Johanna Creswell BáezEmail author
  • Rachel Galanis
  • Molly Magill


This report describes the reliability and validity of the Motivational Interviewing Evaluation Rubric (MIER). The MIER is a novel MI evaluation tool for use in supervision with community-based providers. In the MIER, there are 15 items targeting the following sub-dimensions: Spirit (four items), Process (six items), and Skills (five items). Two raters engaged in 10 h of training and practice on the use of the MIER. Data were session audio-files from youth justice program providers. Inter-rater reliability results showed good performance for the measure. Specifically, 2-way mixed Intraclass Coefficients were excellent, ranging from 0.81 to 0.92 for sub-dimension scores, and Kappa ratings per item were fair to substantial (k =  0.35 to 0.87). Internal consistency reliability for Spirit, Process, and Skills scores ranged from a = .72 to 0.78. In concurrent validity analyses, MIER sub-dimension scores were positively and moderately correlated with relevant Motivational Interviewing Treatment Integrity (4.2) criterion scores (rs = .45 to 0.67; p < .005). The MIER is a promising observational measure for guiding supervision in community youth programs.


Fidelity monitoring Community based programs Implementation science Motivational Interviewing Psychometric 



We would like to specifically acknowledge Elizabeth Zarate and Jada Nyankson for their heartfelt work on this project, along with the staff at Southwest Key Programs Milwaukee. We would also like to acknowledge those who supported the initial work, Hillary England and DeAna Swan, and to Cassandra Harper and Justin Walthers who helped with project management and assisted with data collection, respectively. Lastly, thanks to Matthea Marquart for introducing Dr. Magill to our community-based work.


Dr. Magill was paid a consultation fee by Southwest Key Programs Inc.

Compliance with Ethical Standards

Conflicts of interest

Dr. Baez was employed at Southwest Key Programs Inc. for part of this research project and Rachel Galanis was a Clinical Director at Southwest Key Programs Inc. during this research project.


  1. Cicchetti, V. (1994). Guidelines, criteria and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment,6, 284–290.CrossRefGoogle Scholar
  2. Cohen, J. (1968). Weighed kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin,70, 213–220.CrossRefGoogle Scholar
  3. Cronbach, L. J. (1970). Essentials of psychological testing (3rd ed.). New York: Harper & Row.Google Scholar
  4. Cushing, C. C., Jensen, C. D., Miller, M. B., & Leffingwell, T. R. (2014). Meta-analysis of motivational interviewing for adolescent health behavior: Efficacy beyond substance use. Journal of Consulting and Clinical Psychology,82, 1212–1218.CrossRefGoogle Scholar
  5. Fu, S. S., Roth, C., Battaglia, C. T., Nelson, D. B., Farmer, M. M., Do, T., … Zillich, A. J. (2015). Training primary care clinicians in motivational interviewing: a comparison of two models. Patient Education and Counseling,98, 61–68.CrossRefGoogle Scholar
  6. Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics,33, 363–374.CrossRefGoogle Scholar
  7. Leeman, J., Calancie, L., Hartman, M. A., Escoffery, C. T., Herrmann, A. K., Tague, L. E., … Samuel-Hodge, C. (2015). What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review. Implementation Science,10, 80.CrossRefGoogle Scholar
  8. Lundahl, B., & Burke, B. L. (2009). The effectiveness and applicability of motivational interviewing: A practice-friendly review of four meta-analyses. Journal of Clinical Psychology,65, 1232–1245.CrossRefGoogle Scholar
  9. Madson, M. B., & Campbell, T. C. (2006). Measures of fidelity in motivational enhancement: A systematic review. Journal of Substance Abuse Treatment,31, 67–73.CrossRefGoogle Scholar
  10. Madson, M. B., Campbell, T. C., Barrett, D. E., Brondino, M. J., & Melchert, T. P. (2005). Development of the motivational interviewing supervision and training scale. Psychology of Addictive Behaviors,19, 303–310.CrossRefGoogle Scholar
  11. Madson, M. B., Loignon, A. C., & Lane, C. (2009). Training in motivational interviewing: A systematic review. Journal of Substance Abuse Treatment,36, 101–109.CrossRefGoogle Scholar
  12. Madson, M. B., Shumacker, J. A., Baer, J. S., & Martino, S. (2016). Motivational interviewing for substance use: Mapping out the next generation of research. Journal of Substance Abuse Treatment,65, 1–5.CrossRefGoogle Scholar
  13. Manuel, J. K., Moyers, T. B., & Houck, J. M. (2011). Motivational interviewing: A review of coding systems. Poster session presented at the Motivational Interviewing Network of Trainers (MINT), Fort Wayne, IN.Google Scholar
  14. Martino, S., Ball, S. A., Nich, C., Frankforter, T. L., & Carroll, K. M. (2008). Community program therapist adherence and competence in motivational enhancement therapy. Drug and Alcohol Dependence, 96(1–2), 37–48.CrossRefGoogle Scholar
  15. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods,1, 30–46.CrossRefGoogle Scholar
  16. Miller, W. R., & Rollnick, S. (2013). Motivational interviewing: Helping people change (3rd ed.). New York: Guilford Press.Google Scholar
  17. Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology,72, 1050–1062.CrossRefGoogle Scholar
  18. Moyers, T. B., Manuel, J. K., & Ernst, D. (2014). Motivational interviewing treatment integrity coding manual 4.1. Unpublished Manual.Google Scholar
  19. Moyers, T., Martin, T., Catley, D., Harris, K. J., & Ahluwalia, J. S. (2003). Assessing the integrity of motivational interviewing interventions: Reliability of the motivational interviewing skills code. Behavioural and Cognitive Psychotherapy,31, 177–184.CrossRefGoogle Scholar
  20. Moyers, T. B., Martin, T., Manuel, J. K., Hendrickson, S. L., & Miller, W. R. (2005). Assessing competence in the use of motivational interviewing. Journal of Substance Abuse Treatment,28, 19–26.CrossRefGoogle Scholar
  21. Moyers, T. B., Martin, T., Manuel, J. K., Miller, W. R., & Ernst, D. (2010). Revised global scales: Motivational interviewing treatment integrity 3.1.1 (MITI 3.1.1) [Unpublished manuscript]. Center on Alcoholism, Substance Abuse and Addictions (CASAA), University of New Mexico.Google Scholar
  22. Moyers, T. B., Rowell, L. N., Manuel, J. K., Ernst, D., & Houck, J. M. (2016). The Motivational Interviewing Treatment Integrity Code (MITI 4): Rationale, preliminary reliability and validity. Journal of Substance Abuse Treatment,65, 36–42.CrossRefGoogle Scholar
  23. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.Google Scholar
  24. Nuro, K. F., Maccarelli, L., Martino, S., Ball, S. A., Baker, S. M., Rounsaville, B. J., & Carroll, K. M. (2005). Yale adherence and competence scale (YACSII) guidelines. West Haven, CT: Yale University.Google Scholar
  25. Pecukonis, E., Greeno, E., Hodorowicz, M., Park, H., Ting, L., Moyers, T., … Wirt, C. (2016). Teaching motivational interviewing to child welfare social work students using live supervision and standardized clients: A randomized controlled trial. Journal of the Society for Social Work and Research,7(3), 479–505.CrossRefGoogle Scholar
  26. Stanhope, V., Tennille, J., Bohrman, C., & Hamovitch, E. (2016). Motivational interviewing: Creating a leadership role for social work in the era of healthcare reform. Social Work in Public Health,31, 474–480.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.School of Social WorkColumbia UniversityNew YorkUSA
  2. 2.St. Charles Youth and Family ServicesMilwaukeeUSA
  3. 3.School of Public HealthBrown UniversityProvidenceUSA

Personalised recommendations