Qualitative Analysis of Multiple Mini Interview Interviewer Comments

Abstract

Objective

Qualitative studies of the Multiple Mini Interview (MMI) have investigated the attitudes and thoughts of prospective students and interviewers (i.e., raters) on the MMI interview, but none have examined rater’s written assessments. Concerns regarding what the MMI measures, especially across and within each interview, have sparked investigations to determine how and what raters are measuring. Raters communicate their student evaluation(s) through numerical ratings and written comments that provide score context. This study explores rater’s written comments to better understand the specific information gathered during the MMI process that contributes to interviewee evaluations.

Methods

Randomized data from two US medical schools were examined with no numerical scores or other information about the interviewee provided to reviewers. In reviewing the rater comments, common words and phrases were identified to help construct themes that characterized the content (domains). Authors reviewed each other’s notes and comments regarding themes and worked together to verify themes for accuracy.

Results

Using a directed content approach to content analysis and reviewing the rater’s comments, the results indicate that raters are focused on seven different domains: perspective taking, presentation, qualities, communication, coherence, comprehension, and non-verbal. Many of the rater comments contained multiple themes.

Conclusion

Raters’ MMI comments provide the context for numerical scores allowing admissions committees to more fully understand a candidate’s strengths or weaknesses. Identifying the themes in rater comments can ultimately assist the admissions committee to more comprehensively understand assessment elements that raters are using and consider important during the MMI evaluation.