Abstract
This report describes the reliability and validity of the Motivational Interviewing Evaluation Rubric (MIER). The MIER is a novel MI evaluation tool for use in supervision with community-based providers. In the MIER, there are 15 items targeting the following sub-dimensions: Spirit (four items), Process (six items), and Skills (five items). Two raters engaged in 10 h of training and practice on the use of the MIER. Data were session audio-files from youth justice program providers. Inter-rater reliability results showed good performance for the measure. Specifically, 2-way mixed Intraclass Coefficients were excellent, ranging from 0.81 to 0.92 for sub-dimension scores, and Kappa ratings per item were fair to substantial (k = 0.35 to 0.87). Internal consistency reliability for Spirit, Process, and Skills scores ranged from a = .72 to 0.78. In concurrent validity analyses, MIER sub-dimension scores were positively and moderately correlated with relevant Motivational Interviewing Treatment Integrity (4.2) criterion scores (rs = .45 to 0.67; p < .005). The MIER is a promising observational measure for guiding supervision in community youth programs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cicchetti, V. (1994). Guidelines, criteria and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.
Cohen, J. (1968). Weighed kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.
Cronbach, L. J. (1970). Essentials of psychological testing (3rd ed.). New York: Harper & Row.
Cushing, C. C., Jensen, C. D., Miller, M. B., & Leffingwell, T. R. (2014). Meta-analysis of motivational interviewing for adolescent health behavior: Efficacy beyond substance use. Journal of Consulting and Clinical Psychology, 82, 1212–1218.
Fu, S. S., Roth, C., Battaglia, C. T., Nelson, D. B., Farmer, M. M., Do, T., … Zillich, A. J. (2015). Training primary care clinicians in motivational interviewing: a comparison of two models. Patient Education and Counseling, 98, 61–68.
Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 33, 363–374.
Leeman, J., Calancie, L., Hartman, M. A., Escoffery, C. T., Herrmann, A. K., Tague, L. E., … Samuel-Hodge, C. (2015). What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review. Implementation Science, 10, 80.
Lundahl, B., & Burke, B. L. (2009). The effectiveness and applicability of motivational interviewing: A practice-friendly review of four meta-analyses. Journal of Clinical Psychology, 65, 1232–1245.
Madson, M. B., & Campbell, T. C. (2006). Measures of fidelity in motivational enhancement: A systematic review. Journal of Substance Abuse Treatment, 31, 67–73.
Madson, M. B., Campbell, T. C., Barrett, D. E., Brondino, M. J., & Melchert, T. P. (2005). Development of the motivational interviewing supervision and training scale. Psychology of Addictive Behaviors, 19, 303–310.
Madson, M. B., Loignon, A. C., & Lane, C. (2009). Training in motivational interviewing: A systematic review. Journal of Substance Abuse Treatment, 36, 101–109.
Madson, M. B., Shumacker, J. A., Baer, J. S., & Martino, S. (2016). Motivational interviewing for substance use: Mapping out the next generation of research. Journal of Substance Abuse Treatment, 65, 1–5.
Manuel, J. K., Moyers, T. B., & Houck, J. M. (2011). Motivational interviewing: A review of coding systems. Poster session presented at the Motivational Interviewing Network of Trainers (MINT), Fort Wayne, IN.
Martino, S., Ball, S. A., Nich, C., Frankforter, T. L., & Carroll, K. M. (2008). Community program therapist adherence and competence in motivational enhancement therapy. Drug and Alcohol Dependence, 96(1–2), 37–48.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.
Miller, W. R., & Rollnick, S. (2013). Motivational interviewing: Helping people change (3rd ed.). New York: Guilford Press.
Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050–1062.
Moyers, T. B., Manuel, J. K., & Ernst, D. (2014). Motivational interviewing treatment integrity coding manual 4.1. Unpublished Manual.
Moyers, T., Martin, T., Catley, D., Harris, K. J., & Ahluwalia, J. S. (2003). Assessing the integrity of motivational interviewing interventions: Reliability of the motivational interviewing skills code. Behavioural and Cognitive Psychotherapy, 31, 177–184.
Moyers, T. B., Martin, T., Manuel, J. K., Hendrickson, S. L., & Miller, W. R. (2005). Assessing competence in the use of motivational interviewing. Journal of Substance Abuse Treatment, 28, 19–26.
Moyers, T. B., Martin, T., Manuel, J. K., Miller, W. R., & Ernst, D. (2010). Revised global scales: Motivational interviewing treatment integrity 3.1.1 (MITI 3.1.1) [Unpublished manuscript]. Center on Alcoholism, Substance Abuse and Addictions (CASAA), University of New Mexico.
Moyers, T. B., Rowell, L. N., Manuel, J. K., Ernst, D., & Houck, J. M. (2016). The Motivational Interviewing Treatment Integrity Code (MITI 4): Rationale, preliminary reliability and validity. Journal of Substance Abuse Treatment, 65, 36–42.
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
Nuro, K. F., Maccarelli, L., Martino, S., Ball, S. A., Baker, S. M., Rounsaville, B. J., & Carroll, K. M. (2005). Yale adherence and competence scale (YACSII) guidelines. West Haven, CT: Yale University.
Pecukonis, E., Greeno, E., Hodorowicz, M., Park, H., Ting, L., Moyers, T., … Wirt, C. (2016). Teaching motivational interviewing to child welfare social work students using live supervision and standardized clients: A randomized controlled trial. Journal of the Society for Social Work and Research, 7(3), 479–505.
Stanhope, V., Tennille, J., Bohrman, C., & Hamovitch, E. (2016). Motivational interviewing: Creating a leadership role for social work in the era of healthcare reform. Social Work in Public Health, 31, 474–480.
Acknowledgements
We would like to specifically acknowledge Elizabeth Zarate and Jada Nyankson for their heartfelt work on this project, along with the staff at Southwest Key Programs Milwaukee. We would also like to acknowledge those who supported the initial work, Hillary England and DeAna Swan, and to Cassandra Harper and Justin Walthers who helped with project management and assisted with data collection, respectively. Lastly, thanks to Matthea Marquart for introducing Dr. Magill to our community-based work.
Funding
Dr. Magill was paid a consultation fee by Southwest Key Programs Inc.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
Dr. Baez was employed at Southwest Key Programs Inc. for part of this research project and Rachel Galanis was a Clinical Director at Southwest Key Programs Inc. during this research project.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Báez, J.C., Galanis, R. & Magill, M. When the Right Measure Doesn’t Exist: A Novel Motivational Interviewing Tool for Community Programs. Child Adolesc Soc Work J 37, 195–205 (2020). https://doi.org/10.1007/s10560-020-00650-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10560-020-00650-y