INTRODUCTION

The tradition of Morbidity and Mortality (M&M) conference originated from the work of the Anesthesia Study Commission of the Philadelphia County Medical Society in the 1930s and 1940s. The Commission held monthly meetings during which they reviewed selected cases involving perioperative fatality, held an open discussion, and voted on whether the death in each case could have been prevented by the anesthesiologist1. This deliberate effort to learn from negative patient outcomes gave rise to the current landscape of nearly ubiquitous M&M conferences. Some 90% of US internal medicine residency programs report holding M&M or an equivalent conference2, and all US general surgery residency programs are required to host a weekly M&M (or similar) conference3.

Despite their prevalence, consensus regarding the definition, format, and purpose of M&M conferences has remained elusive. Several analyses have documented variability in the conference content, including whether cases involve adverse events, whether errors are explicitly labeled as such4, whether the presenter performs a literature review5, and whether a formalized method is used for discussion of error2. These programmatic differences are a manifestation of a deeper lack of consensus regarding the educational purpose of M&M. In an ethnographic study of a general internal medicine M&M, attendees described their understanding of the key roles of M&M, including gaining new medical content knowledge, role modeling of attitudes for trainees, and receiving emotional support from colleagues6. Some authors have identified the conference as a venue for exposing trainees to quality improvement methods and tools7,8, or initiating systems changes to promote patient safety9. Others have argued that M&M represents a poor tool for the analysis and prevention of error, and that the essential purpose of the conference relates to professional identity formation through a shared space for self-reflection, humility, and consideration of humanistic aspects of practice10,11,12.

Gregor and Taylor13 argue that the apparent tension between M&M as an educational conference and as a venue for addressing quality issues is resolved through the application of experiential learning theory. In their model, reflection on the case and on one’s own clinical decision-making generates “experiential learning by proxy,” leading to improved clinical skills which are incorporated into practice. As described by Schön and others14, reflection on situations of “uncertainty, uniqueness, and value conflict,” common features of M&M cases, is an important strategy for the development of professional competence, particularly in postgraduate and continuing medical education15. A shared reflection that incorporates multiple perspectives may be more effective16, consistent with the finding that increased discussion and collaborative learning are associated with higher perceived educational value of surgical M&Ms17. As a form of case-based learning, M&M could link theoretical and applied knowledge, promote clinical reasoning, and encourage decision-making relevant to care of the whole patient18.

While previous studies document several distinct viewpoints on educational goals of M&M, less is known about the educational outcomes of the conferences: what types of learning are actually taking place? We investigated this question by analyzing learning outcomes statements generated by participants at one Department of Medicine’s M&M conference. We sought to characterize the conference content identified by participants as central to their learning, and to describe how participants proposed to apply that learning toward change at the individual or system level. In addition, we searched for evidence that the conference provides an opportunity for reflection regarding the shared experiences of physicians. By cataloguing the learning outcomes documented in these responses, our goal was to shed light on the range of educational roles this conference fulfills, and, by extension, provide insight regarding the role of M&M conferences in general. A better understanding of the nature of learning that takes place at M&M could provide useful guidance for educators and course planners who seek to develop or improve an M&M conference.

METHODS

Setting and Participants

Morbidity, Mortality, and Improvement (MM&I) is a weekly continuing education and core residency conference of the Department of Medicine at Dartmouth-Hitchcock Medical Center (DHMC). For each conference, an internal medicine resident selects a case to present—typically a case in which the resident directly participated in the patient’s care. Cases generally involve a medical error, near miss, or unanticipated negative outcome; one or more diagnostic or therapeutic dilemmas; and/or challenges related to care coordination, communication, or resource utilization. A primary objective of MM&I is to provide a safe forum in which to discuss cognitive and systems errors, patient safety concerns, and opportunities to improve team-based care.

The conference is usually attended by approximately 100 participants, including faculty from diverse clinical specialties, trainees, medical students, advanced practice providers, nurses, community physicians, and administrative staff. The discussion is facilitated by a faculty member (usually author K.A.K.), who poses questions regarding differential diagnosis, management, and approaches to challenging aspects of the case. This includes questions posed to the entire group and questions targeted to individuals with relevant expertise. Attendees respond to the facilitator’s questions and raise additional questions or concerns about the case. All conference attendees are invited to contribute to the discussion, and while American Board of Internal Medicine (ABIM) certified faculty represent a minority of attendees, they are generally the principle discussants. In each case, opportunities to improve future care are explicitly explored.

Beginning in 2017, the DHMC Department of Medicine registered its MM&I conference as a Maintenance of Certification (MOC) eligible activity with the ABIM. Following each conference, ABIM-certified participants can request MOC credit using an internally developed web-based platform. For this study, we reviewed all entries submitted during the 2017–2018 academic year that were awarded MOC credit. Because responses were collected as part of the MOC process, all study participants were physicians who were board-certified in internal medicine and its subspecialties.

Data Source

All participants requesting MOC for MM&I are required to submit responses to the following prompts:

  1. 1.

    What did you learn in today’s conference that influences you to change an element of your practice, or that reinforces the way you are currently practicing?

  2. 2.

    Please identify at least one element of practice that you are committed to changing or maintaining as a result of what you have learned.

Commitment to Change (CTC) statements such as these promote reflection regarding how a clinician will apply what they have learned19, and are associated with an increased likelihood of actual change in practice20. Importantly, CTC statements can identify learning outcomes that were unanticipated by course planners21.

The MM&I facilitator reviews participants’ CTC statements on a secure web-based platform and determines whether they meet the meaningful engagement threshold for MOC credit, which requires writing a reflective statement regarding the content of the conference and committing to change or maintain a related element of practice22. The facilitator summarizes the lessons learned each week, and the summary is e-mailed as feedback to all participants in the MOC process, consistent with ABIM standards22. For this study, responses to CTC prompts for conferences occurring between July 1, 2017, and June 30, 2018, were downloaded to a Microsoft Excel spreadsheet. Prior to performing the analysis, the data were de-identified with individual names replaced by a numeric participant identification code. No other identifying information was linked to the data.

Analysis

For each eligible MOC submission, we reviewed and analyzed the answers to both CTC prompts as a single comment using directed content analysis. In directed content analysis, qualitative data are analyzed using preliminary codes based on current frameworks or theory. Data that cannot be accurately coded are re-analyzed to determine whether they represent new codes or sub-categories of existing codes. This approach both incorporates and informs the prior framework or theory, and allows for descriptive comparison of the relative frequency of the codes in the data sample23.

We generated preliminary codes based on the literature describing the educational content and goals of M&M. In order to explore questions raised by the current literature regarding both the content that is important for learning and the manner in which learning could be applied, we developed a codebook to categorize comments in two major dimensions: content themes (the type of knowledge, skill, or attitude addressed in the learning); and learning implementation domains (the manner and context in which learning would be applied). We included a “shared experience” code for comments that included reflections on the experience of providing patient care or the meaning of being a physician, since this has been described as an important function of M&M.

Using these preliminary codes, we attempted to map every CTC comment to at least one learning implementation domain and at least one content theme, and to determine whether the comment included a description of the shared experience of practicing medicine. When a comment included multiple distinct statements about learning, each statement was mapped to a single learning implementation domain; thus, a multi-part comment could be mapped to multiple domains. For content theme mapping, distinct statements within a comment could be mapped to multiple content themes (e.g., End of Life and Patient/Family Communication, or Diagnostic Testing and Cost of Care).

Each author independently reviewed the first 10% of comments chronologically, mapping comments to codes in a Microsoft Excel spreadsheet. We used the logical test function in Excel to identify areas of inter-coder disagreement. To ensure that content themes were comprehensive and that the definitions of the codes were clear and mutually exclusive, we discussed all comments for which there was disagreement, or for which either a content theme or learning implementation domain code could not be assigned, then collaboratively revised and clarified the codes.

Following this preliminary review, each author independently reviewed and assigned codes for the remainder of the data set. We again identified and discussed areas of uncertainty and differences in code assignments, refined code definitions, and reached agreement on coding for the entire data set.

Final codes for content themes map to all six primary ACGME competencies, as demonstrated in Table 1. Definitions of learning implementation domains and shared experience are described in Table 2. The Appendix contains clarifying guidelines for content theme coding generated to resolve inter-coder discrepancies. Due to ambiguous comment language, there were three comments for which no content theme could be determined and one comment for which no learning implementation domain could be determined.

Table 1 Content themes and associated ACGME competencies
Full size table
Table 2 Definitions for learning implementation domains and shared experience category
Full size table

After coding all comments, we counted the occurrences of the content themes and learning implementation domains at each conference session, and counted the content themes and learning implementation domains described by each learner across the conference series. We compared the relative frequency of content themes and learning domains, described how often shared experience comments were included, and explored whether there were dominant themes or learning domains in each conference session and for individual learners.

The study plan was reviewed by the Dartmouth-Hitchcock Health Institutional Review Board and deemed Exempt on 1/8/2020 (reference number STUDY02000249).

RESULTS

Over the 1-year study period, there were 44 MM&I sessions including 347 qualifying comments, an average of 7.9 per session (range 2–18). Comments were submitted by 49 participants, with the number of comments by a single participant ranging from 1 to 33.

Content Themes

The 16 content themes appeared with varying frequency in participants’ CTC statements. The four most common themes (Principles of Patient Care, Disease Treatment, Disease Presentation, and Patient/Family Communication) each appeared in greater than 25% of comments. The four least common themes (Ethical Issues, Systems – National, Systems – Regional, and Provider Burnout/Moral Distress) each appeared in fewer than 3% of comments (Fig. 1). A mean of 6.5 (range 2–11) content themes were generated per conference session. Some sessions were dominated by a single theme: all nine comments from session #27, for example, included at least one statement that indicated learning about Disease Treatment. In fact, all nine comments from session #27 discussed anticoagulant selection, monitoring, or reversal. Other sessions included a greater diversity of themes. The ten comments from session #24, for example, were spread relatively evenly across eight themes, including Disease Presentation (“I will remember babesiosis as a potential rare cause of hemolysis in SCD patients who receive frequent transfusions”), Systems – Institution (“Lack of screening at many hospital blood banks for [Babesia] increases the likelihood of undetected transmission”), and Interprofessional Communication (“Reach out to colleagues in hematopathology for personal reads on peripheral smears”).

Fig. 1
figure 1

Distribution of comments by content theme. Because comments were frequently mapped to multiple themes, the sum of the comments in all themes is greater than the total number of comments

Full size image

Learning Implementation Domains

The predominant domain in which participants planned to implement what they learned from the conference was Clinical Knowledge and Skills (271; 78% of comments), followed by Systems Knowledge (119; 34%), Clinical Reasoning (98; 28%), and Systems Change (31; 9%). We considered whether the type of learning occurring at MM&I might vary from one session to the next, given the heterogeneity in the case content. Instead of between-session variation, we found substantial within-session variation, with all but one (2%) of the sessions generating learning in multiple domains. The single conference that generated comments mapping to one learning implementation domain included only two comments. Nine sessions (20%) generated comments mapping to two domains, 20 (45%) generated comments mapping to three domains, and 14 (31%) generated comments mapping to all four domains. For example, event #12 included learning in Clinical Knowledge and Skills (“Reinforced my understanding of the clinical presentation for pulmonary embolism”); Clinical Reasoning (“In my teaching, underscore to students the power of anchoring bias”); Systems Knowledge (“make sure the PCP know[s]…to continue the work up”); and Systems Change (“VTE ppx really needs to be something that we opt out of, instead of something that we consider”). Figure 2 demonstrates the number of comments from each session that mapped to each learning implementation domain.

Fig. 2
figure 2

Learning implementation domains at each conference session. Each column represents a single conference session, and each row represents a learning implementation domain. The shading indicates the number of comments in each domain at a given session. Sessions are arranged from left to right in order of increasing relative frequency of learning implementation domains other than Clinical Knowledge and Skills. CKS = Clinical Knowledge and Skills; SK = Systems Knowledge; SC = Systems Change; CR = Clinical Reasoning

Full size image

Although learning in the domain of Systems Change was relatively infrequent, comments in this area often involved an explicit action plan for improvement. Examples include “provide stronger support and education for fellows who perform telephone triage” and “use [the AHRQ Suspected UTI SBAR Toolkit] to help teach students/residents issues around Abx stewardship.”

We assessed whether individual participants tended to derive certain types of learning from MM&I based on their interests or learning needs. As with the conference session–level analysis, however, we found that participants tended to describe learning in multiple domains (Fig. 3). For the analysis of participant-level data, we excluded 13 participants who submitted responses for only one conference session. The remaining 36 participants commented on between 2 and 33 sessions, (mean of 7.1). Of these 36 learners, 29 (78%) reported learning in three or four domains. All had at least one instance of learning in the Clinical Knowledge and Skills domain, but participants varied in how strongly they leaned toward this domain. Participant #50, for example, submitted nine comments, all of which included learning in the area of Clinical Knowledge and Skills (e.g., “Massive bleeding and transfusion can be associated with refractory coagulopathy”), and only two comments related to any other domain. Participant #28, on the other hand, demonstrated learning in all four domains, with nearly equal numbers for Clinical Knowledge and Skills (e.g., “fungal endocarditis and septic emboli should not be anticoagulated”), Systems Knowledge (e.g., “I was glad to hear the new urine tox screen includes fentanyl”), and Clinical Reasoning (e.g., “be aware for opioid prejudice when treating these patients…be aware of bias and self-reflect”).

Fig. 3
figure 3

Learning implementation domains for each conference participant. Each column represents a single participant, and each row represents a learning implementation domain. The shading indicates the number of comments in each domain demonstrated by a given participant. Individual comments may include components that address multiple domains. This figure includes only those participants who provided comments for ≥ 2 events. Participants are arranged from left to right in order of increasing frequency of learning implementation domains other than Clinical Knowledge and Skills. CKS = Clinical Knowledge and Skills; SK = Systems Knowledge; SC = Systems Change; CR = Clinical Reasoning

Full size image

Shared Experience

Of the 347 comments submitted, 49 (14%) included statements coded in the Shared Experience category. Twenty-eight conference sessions (64%) included at least one Shared Experience statement. These statements addressed topics such as the limitations and fallibility of providers (e.g., “[This case] reminds me that our ability to predict whether someone will go home is tenuous. Also, that we can't save everyone.”); awe about the capabilities of modern medicine (e.g., “The Laboratory and blood blank are pretty amazing and at times we don't always understand all they do”); emotional impacts of the practice of medicine (e.g., “Goal orientated care can cause distress among providers and staff”); and challenges physicians face on a day-to-day basis (e.g., “How difficult it is to treat patient’s [sic] with complex psychosocial problems without additional resources”).

DISCUSSION

This analysis paints a detailed picture of faculty participants’ self-identified learning from one institution’s internal medicine M&M conference during the 2017–2018 academic year. Although learning outcomes related to the application of clinical knowledge and skills predominated, nearly every session generated learning involving diverse content themes applicable across multiple domains. Similarly, most participants derived multiple types of learning from the conference. This suggests that M&M can simultaneously serve a variety of educational functions. In this context, structuring the conference to achieve learning in only one area, such as quality improvement or error prevention, could result in a lost opportunity to engage and meet the needs of learners. Discussing complex cases reflecting real-world practice allows physicians to apply multiple competencies as each case evolves, building or consolidating content knowledge, clinical reasoning, systems improvement skills, and humility5,13.

It is notable that a majority of conference sessions produced at least one comment reflecting on the wonder, challenges, and emotional impacts of practicing medicine, despite the fact that these types of comments may not fit a strict definition of something “learned” from the conference. We anticipate that if participants were prompted to reflect on how the conference impacted their identity as a physician, their sense of wonder or humility, their feelings of shared purpose with other practitioners, or explorations of other affective domains, the number of comments in this area would likely be higher. Nonetheless, the presence of these comments in our sample illustrates that some attendees derive from MM&I an enhanced sense of community and professional identity, and incorporate these reflections in their “lessons learned.” The power of reflection in M&M may extend beyond individual learning and include a communal function in which “meaning is constructed within a community of professional discourse16,” as part of a Community of Practice. First described by Lave and Wenger24, Communities of Practice are defined as “groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly25.” Trainees in medicine, surgery, nursing, and many other professions gradually acquire competence through participation and discussion within their Communities of Practice26,27,28. Our results suggest that, beyond clinical competence, M&M provides a shared space for health professionals to reflect on the “countless cases where misfortune evokes humility, responsibility, remorse, even agony11.”

The learner-reported data analyzed in this study allow for a more granular examination of learning at M&M than has been previously reported. Rather than discussing their feelings toward the M&M conference in the abstract, participants provided timely and specific information about learning from individual conference sessions. The large number of sessions and attendees included in the data set reduces the risk that learning outcomes were skewed by the content of a limited number of sessions. Despite the relatively brief nature of the comments included in the data set, many comments provided a detailed and nuanced picture of the participants’ reported learning outcomes.

Our findings may not generalize to the learning occurring at M&M conferences in different specialties and at different institutions, but rather describes the learning that is achieved at one internal medicine conference. Our data set further does not give direct insight into the learning that participants hope to get from M&M conferences; one useful direction for further investigation would be to explore the reasons why participants attend MM&I and whether these motivations align with learning outcomes.

Because comments were provided as part of the MOC process, there is some risk for selection bias related to the inclusion of only ABIM-certified physicians in the sample, as well as the possibility that among ABIM-certified physicians, those requesting MOC credit may not be representative of the entire group. An interesting future direction of study would be to extend the prompts to all conference participants and compare learning outcomes reported across different learner types. Prior research suggests there are differences in how residents and staff physicians perceive the role of M&M6, so we would hypothesize differences between these groups and others.

Some individuals participated in the MOC process much more frequently than others, and these individuals’ perspectives are bound to be overrepresented in our data. Our analysis utilized participants’ self-assessments of their learning outcomes, and there may be discrepancies between actual or applied learning and reported learning. The prompts used to generate self-identified learning outcomes may also have influenced the type of learning reported. For example, because the prompts specifically elicited learning that could change the individual’s practice of medicine, aspects of healthcare perceived by participants as outside of their control may have been underrepresented. This possibility is supported by the relative infrequency of learning outcomes related to Systems Change in our sample. The pre-identified learning objectives of the conference might also prompt participants to focus their responses toward those content areas, or might be prioritized in the discussion as a result of directed questioning by the facilitator.

This study demonstrates that physicians attending an academic internal medicine M&M conference experience rich and varied learning outcomes from the conference, including enhanced clinical knowledge, systems knowledge, and clinical reasoning skills. Learning consistently occurs across these domains regardless of the focus of the individual conference session. Some participants also appear to engage in the conference as a space for reflecting on the nature and meaning of practicing medicine. These findings reinforce our commitment to incorporating a variety of cases in MM&I, using a flexible approach to explore the unique learning opportunities of each case, and more intentionally inviting reflection on the experience of providing care.