Abstract
Objective
Qualitative studies of the Multiple Mini Interview (MMI) have investigated the attitudes and thoughts of prospective students and interviewers (i.e., raters) on the MMI interview, but none have examined rater’s written assessments. Concerns regarding what the MMI measures, especially across and within each interview, have sparked investigations to determine how and what raters are measuring. Raters communicate their student evaluation(s) through numerical ratings and written comments that provide score context. This study explores rater’s written comments to better understand the specific information gathered during the MMI process that contributes to interviewee evaluations.
Methods
Randomized data from two US medical schools were examined with no numerical scores or other information about the interviewee provided to reviewers. In reviewing the rater comments, common words and phrases were identified to help construct themes that characterized the content (domains). Authors reviewed each other’s notes and comments regarding themes and worked together to verify themes for accuracy.
Results
Using a directed content approach to content analysis and reviewing the rater’s comments, the results indicate that raters are focused on seven different domains: perspective taking, presentation, qualities, communication, coherence, comprehension, and non-verbal. Many of the rater comments contained multiple themes.
Conclusion
Raters’ MMI comments provide the context for numerical scores allowing admissions committees to more fully understand a candidate’s strengths or weaknesses. Identifying the themes in rater comments can ultimately assist the admissions committee to more comprehensively understand assessment elements that raters are using and consider important during the MMI evaluation.
Similar content being viewed by others
References
Khan KZ, Gaunt K, Ramachandran S, Pushkar P. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: Organisation & Administration. Med Teach. 2013;35(9):e1447–63.
Ponnamperuma G, et al. The long case and its modifications: a literature review, vol. 43; 2009. p. 936–41.
Regehr G, Freeman R, Hodges B, Russell L. Assessing the generalizability of OSCE measures across content domains. Acad Med. 1999;74(12):1320–2.
Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ. 2004;38(3):314–26.
Eva KW, Reiter HI, Rosenfeld J, Norman GR. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med. 2004;79(10 Suppl):S40–2.
Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J, Norman GR. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ. 2009;43(8):767–75.
Kumar K, Roberts C, Rothnie I, du Fresne C, Walton M. Experiences of the multiple mini-interview: a qualitative analysis. Med Educ. 2009;43(4):360–7.
Alweis RL, Fitzpatrick C, Donato AA. Rater perceptions of bias using the multiple mini-interview format: a qualitative study. J Educ Train Stud. 2015;3(5):52–8.
Zaidi NB, Swoboda C, Wang LL, Manuel RS. Variance in attributes assessed by the multiple mini-interview. Med Teach. 2014;36(9):794–8.
Eva KW, Macala C. Multiple mini-interview test characteristics: ‘tis better to ask candidates to recall than to imagine. Med Educ. 2014;48(6):604–13.
Tavares W, Ginsburg S, Eva KW. Selecting and simplifying: rater performance and behavior when considering multiple competencies. Teach Learn Med. 2016;28(1):41–51.
Axelson RD, Kreiter CD. Rater and occasion impacts on the reliability of pre-admission assessments. Med Educ. 2009;43(12):1198–202.
Cunnington JP, Neville AJ, Norman GR. The risks of thoroughness: reliability and validity of global ratings and checklists in an OSCE. Adv Health Sci Educ Theory Pract. 1996;1(3):227–33.
Manuel, R.S. MMI training sessions for rater. Use of Comments, sessions 2014;1-9 (slide 33). Retrieved from https://med.uc.edu/medicalstudentadmissions/interviewday/training. Accessed 9 July 2014.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
Glaser BG, Strauss A. The discovery of grounded theory: strategies for qualitative research. New York: Aldine De Gruyter; 1967. p. 271.
Wilkerson L, Rose M. Learning from thenarrative comments of standardized patients during an objective structured clinical examination of fourthyear medical students. ERIC Clearinghouse on Assessment and Evaluation. 2001.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
School A – IRB approval #07-08-23-02-E.
School B – The study does not meet the criteria for human subject’s research.
Informed Consent
NA
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Manuel, R.S., Dickens, L. & Young, K. Qualitative Analysis of Multiple Mini Interview Interviewer Comments. Med.Sci.Educ. 29, 941–945 (2019). https://doi.org/10.1007/s40670-019-00778-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-019-00778-2