Critical care echocardiography (CCE) is a noninvasive diagnostic tool that has, in many countries, largely replaced invasive cardiovascular monitoring.1,2 International consensus panels agree that all intensivists should be able to perform basic CCE.3,4,5,6,7,8 Major professional societies in Europe and North America have implemented certification processes to assess intensivists’ competencies in CCE.9,10,11,12 Achieving proficiency in CCE has thus become a key component of safe critical care practice, yet the optimal path for intensivists to master the skill remains unknown.13,14,15

While many European countries have fully integrated echocardiography into critical care clinical practice and training, other countries, especially in North America, continue to depend on cardiology services to perform and interpret CCE. A major barrier to CCE uptake in North America is the lack of CCE-certified staff intensivists.16,17 Without certified intensivists providing longitudinal supervision, trainees may not achieve competence and may interpret CCE findings erroneously, thereby potentially compromising patient care.18 Numerous publications have reported on the successful implementation of short CCE workshops and basic curricula in critical care and emergency medicine, but the literature on advanced CCE training remains scarce.19,20,21,22,23,24,25,26 There are concerns that short, basic CCE courses are inadequate for safe clinical practice.13,14,15,27,28,29 Yet, a paucity of evidence informs the design of advanced CCE training, despite the recent development of international certifications in Europe and North America.10,11,18 Only a few authors have reported the implementation of advanced CCE programs.16,21,22

We developed and implemented a longitudinal echocardiography training program for advanced CCE, based on recommendations from professional societies and principles of curriculum design.30,31,32,33 The primary objective of the present work was to evaluate our advanced CCE program to better understand reasons for low completion rates. Our secondary objective was to assess if two curricular changes implemented after the first year of the program increased program completion. The curricular changes that we investigated were 1) distributed supervised CCE sessions conditional to learners’ independent practice (vs predefined supervised blocks) and 2) an optional echocardiography laboratory (echo lab) rotation.18

Methods

Initial structure of and subsequent changes to the echocardiography training program

The advanced CCE program took place within the Interdepartmental Division of Critical Care Medicine (IDCCM) of the University of Toronto (Toronto, ON, Canada), which is composed of approximately 70 attending physicians with different specialty backgrounds (e.g., anesthesiology, internal medicine) and 70 senior critical care trainees (fellows) per year. Only two IDCCM attending physicians were formally trained to perform CCE, and only one was certified for advanced CCE. Enrollment in the course was voluntary. The advanced CCE program consisted of a combination of theoretical and practical components (Table 1). Practical sessions included independent (unsupervised) and supervised hands-on practice in the intensive care unit (ICU). Learners submitted unsupervised CCE electronically. To complete the program, each learner had to collect a portfolio of 150 transthoracic echocardiograms with their corresponding reports (as per international recommendations)11,30,34 and attend at least half of the didactic sessions (see Electronic Supplementary Material [ESM] eAppendix 1 for additional details). The portfolio could comprise both unsupervised and supervised echocardiograms, including those performed in the echo lab. All echocardiograms submitted were reviewed and graded according to previously published scales.35 Reports generated by the learners were also reviewed. For learners aiming to obtain international certification, echocardiograms submitted as part of our training program were included in the learner’s portfolio required for international certification. The practical component of the training was tailored to the learners’ previous experience. Learners without previous experience were introduced to basic CCE and progressed to advanced CCE throughout the training. Low engagement and shortage of instructors during year 1 led us to modify the program for subsequent iterations in two ways. First, we optimized hands-on CCE practice by adopting a distributed conditional supervision model. To increase independent practice with delayed feedback (i.e., aiming toward “deliberate practice”), we required submission of a minimal number of unsupervised echocardiograms every month to “earn” access to direct supervision (Table 1).36,37,38,39,40,41,42,43 In year 2, 15 monthly echocardiograms’ submission allowed learners to access as many supervised sessions as available; in year 3, one two-hour supervised session could be obtained for each seven echocardiograms submitted. We hypothesized that this change would increase learners’ motivation to practice independently between supervised sessions, while optimizing our educational resource use. Additionally, based on concepts in psychomotor learning, we distributed the supervision over time (one- to two-hour sessions throughout the year upon request, Fig. 1) instead of using a massed practice schedule (i.e., one- to two-week blocks of supervised practice each year).44,45 Second, we provided an opportunity to complete an optional two- to four-week rotation in the echo lab, where learners could practice on cardiology outpatients under supervision of a sonographer. Learners could choose to complete a rotation in the echo lab (by using one of their critical care program elective blocks) if spots were available for noncardiology trainees.

Table 1 Advanced CCE training structure and modifications
Full size table
Fig. 1
figure 1

Distribution of supervised practice over the three years of implementation. The first year offered en-bloc supervision of one week at a time (two in total throughout the year). Supervision was organized based on the learner’s schedule and instructors’ availability. For the subsequent years, supervised practice was divided in multiple two-hour sessions. Only learners who had submitted echocardiograms for review by the instructors were offered supervised sessions.

Full size image

Study design, participants, and setting

This prospective cohort study followed a multiphase, convergent mixed methods design with the purpose of triangulation and complementarity. The study was conducted within the IDCCM of the University of Toronto. All IDCCM fellows and staff physicians who enrolled in the advanced CCE program between September 2015 and June 2018 were eligible to participate in the evaluation study if they attended more than three didactic sessions. Learners who did not have a portfolio requirement (i.e., attending physicians during year 1) were excluded from the analysis. Study participation was voluntary. Quantitative and qualitative data (with equal priority) were collected between September 2015 and June 2018. Approval from the Institutional Research and Ethics Board was obtained, and participants gave written informed consent.

Qualitative analysis: exposure, outcomes, and analyses

We conducted semistructured interviews of a purposeful sample of consenting physicians who either completed or did not fully complete the program between 2015 and 2018 (see ESM eAppendix 2). We performed a purposive sampling to achieve maximum variation.46,47 Two investigators (L. D., A. J.) who were not course instructors conducted all interviews, which were audio-recorded and transcribed verbatim. We began inductive content analysis of the deidentified transcripts immediately and classified the findings into themes, subthemes, and matched quotes. We categorized interviews according to the interviewees’ level of engagement using their number of echocardiograms submitted for review (low: < 50 CCE, moderate: 50 to 100, and high: > 100 CCE). Our initial set of 12 interviews included learners from years 1 to 3 but failed to capture learners who had submitted a large number of echocardiograms. We then purposefully recruited such learners to expand our understanding of learners’ engagement (theoretical sampling). We modified the interview guide iteratively to include questions pertaining to emerging themes such as the experience in the echo lab. One of the authors (G. D.) conducted an inductive thematic content analysis of the transcripts by performing line-by-line coding and identifying emerging themes. A second researcher (D. P.) also coded the initial transcripts. Preliminary interpretations were reviewed, discussed, and agreed upon between the two co-investigators. As the qualitative analysis progressed, certain interpretations of the transcripts called for additional quantitative analyses as aforementioned. Interpretations also raised questions about the role of motivation in learners’ engagement with the program. An exploration of motivational theories informed a subsequent deductive analysis of the data set. The interview process was discontinued when data saturation was reached.48,49,50 Both investigators reflected, throughout the research process, on their position as clinician, educator, and teacher, and its impact on the analyses.

Quantitative analysis

For the quantitative analyses, we collected attendance and number of echocardiograms performed independently and under supervision. We used descriptive statistics to report the primary outcome, which was program completion defined as attendance of more than 50% of the theoretical sessions and completion of 150 echocardiograms (supervised and unsupervised). In the secondary analyses, we estimated the effects of conditional vs unconditional supervision on the rate of submission of unsupervised echocardiograms. Furthermore, we calculated the effect of an echo lab rotation on the submission rate of unsupervised echocardiograms. Analyses were conducted with a Bayesian framework using R (Version 3.5.3) and JAGS (Version 4.3.0).51 Further details are described in ESM eAppendix 3.

Results

Learners’ characteristics, engagement level, and program completion rate

Sixty-five learners enrolled in the program between September 2015 and June 2018. We excluded 14 learners (22%) who attended less than three didactic sessions, and six learners (9%) without CCE portfolio requirements (ICU attendings from year 1). A total of 45 learners (69%) were included in the final analysis. In total (including unsupervised, supervised, and echo lab echocardiograms), learners performed a median [interquartile range] of 32 [14–83] echocardiograms over the course of their enrollment in the program. Eight learners (17%) performed a total of ≥ 100 echocardiograms in a year. Four of these eight learners had previously performed less than 50 CCEs, and two had no previous CCE experience. Fourteen learners (31%) completed a rotation in the echo lab. Four learners (9%) completed their portfolio (one within a year, and three within three years) and attended more than 50% of the didactic sessions, thereby fulfilling criteria for program completion. Eleven learners eventually wrote a certifying examination with a success rate of 100%. One wrote the European Diploma of Echocardiography and ten wrote the Examination of Special Competence in Critical Care Examination of the American National Board of Echocardiography (CCEeXAM); of note, the first iteration of the CCEeXAM took place in January 2019. The four learners who completed their portfolio are now diplomates of the CCEeXAM (i.e., passed the CCEeXAM and performed at least 150 complete transthoracic CCEs). An illustration of the study profile and details on participant characteristics are reported in Fig. 2 and Tables 2 and 3.

Fig. 2
figure 2

Study profile

Full size image
Table 2 Participant characteristics
Full size table
Table 3 Submission of echocardiograms and attendance stratified by year
Full size table

Qualitative analysis of learners’ perceptions of the program

We conducted 18 individual interviews. The following abbreviations are used to describe the quotes: interviewee (I), fellow (F), attending (A), and cohort (C), with cohorts defined as cohort 1 (academic year 2015–2016), cohort 2 (2016–2017), and cohort 3 (2017–2018). Low, moderate, and high refers to engagement with independent practice as aforementioned. Quantified qualitative data are summarized in ESM eTable 1. Additional quotes are reported in ESM eTable 2.

Enrolling in a critical care echocardiography course with great expectations

Learners viewed echocardiography as an essential tool to assess and manage critically ill patients. They stated three main reasons to enroll: 1) to perform echocardiography independently from cardiologists, 2) to monitor patient’s clinical changes and responses to treatments, and 3) to use a noninvasive technique.

“I really want to know what happens [with my patients] and I feel there's a lot of constraint when I rely on other people” (I-13: F/C2-3/moderate).

Certain learners chose an advanced CCE program to mitigate potential risks of superficial CCE learning, while others aimed to prepare for a formal certification. This was perceived as an advantage in a competitive employment market.

“There are risks with any diagnostic modality if their use is inappropriate […] you need some form of standards, an expertise to be able to do it and interpret it. Otherwise, it may easily give you a false sense of security” (I-15: F/C2/high).

“The reality is that most people who do this course are more interested in all aspects of critical care and in passing the exam and getting certified” (I-13, F/C3/moderate).

Facing the reality of learning a complex skill in a challenging environment

Learners unanimously mentioned the lack of protected time as a significant barrier to training.

“It was not just the course; it was everything else that was going on at the same time. You know … there was lots of things going on like … work and academic work and personally as well” (I-10, F/C2/moderate).

Learners also commented on the complexity of learning echocardiographic skills for critically ill patients, whose anatomical and physiologic dynamic changes complicated acquisition and interpretation of images. Learners reported a range of negative emotions, such as frustration and stress, while practicing independently in the ICU. In addition, they felt infantilized and punished by the conditional access to supervision based on independent practice.

“It was also hard because if you were doing the sessions on your own, you’d just get the view and after 10 minutes you just gave up, because you can’t stay there for hours […]” (I-17, F/C3/moderate).

“The treatment that we got was a bit like I [was] back at a five years old school and the fact that by not doing one thing [i.e., independent practice] we were punished without supervised sessions really put me off” (I-9: F/C2/low).

Benefiting from direct supervision

Learners felt that direct supervision in the ICU and in the echo laboratory was crucial to their learning and continuing motivation.

“Without [the supervised] sessions, […] I wouldn't have continued the course. I think the sessions are the core part of the training program” (I-15, F/C2/high).

“The instructors should not be expecting [trainees] to do unsupervised sessions without having about a couple of weeks of echo lab training or supervised training because without that, it’s just a spiral down. It’s a vicious cycle and we won’t learn and we won’t be motivated” (I-4, F/C1/moderate).

Protected time in the echo lab was perceived as invaluable. Yet, learners acknowledged that performing echocardiography in the ICU was more challenging than in the echo lab, and recalled experiencing difficulties in transferring CCE skills acquired in the echo lab to ICU patients:

“The second year was better [and] I have to attribute it a lot to the fact that I did the echo lab training. So, I got my confidence […] and then now the learning is more about how to adjust the views to an ICU patient […]” (I-4, F/C1/moderate).

Rising or not rising to the challenge: sustaining motivation

Learners described two types of response to difficulties encountered in the program. Certain trainees disengaged with the program, felt demoralized about different aspects of the course, and eventually gave up.

“Maybe six months or so [into the training], I just thought, I actually just can’t do this anymore” (I-10, F/C2/moderate).

Learners explained their loss of motivation in three ways: a perceived lack of performance improvement over time, personal circumstances that interfered with their ability to complete the program requirements, and realization of a misalignment between the program and their personal objectives.

“I felt that I wasn’t getting much out of the echoes because I wasn't having supervised sessions” (I-9, F/C2/low).

“Maybe I did not read enough around it afterwards […] because other folks did manage it and did complete it, so obviously, it’s doable. It might have just been my circumstances at the time […]. Some concepts may [have been] too advanced for intensivists, [who] wanted to learn basic or intermediate echo skills” (I-10, F/C2).

Conversely, other learners recalled investing more effort and engaging in additional independent practice when facing training challenges. Realizing their learning progress and seeing the potential impact of CCE on the management of their patients eventually contributed to maintaining high levels of motivation.

“When I was not confident, I just put more effort into training and then I started feeling more confident, which gave me more interest in the course, because it showed me that what I was doing was actually working, that I had actually learned things. […] It was the day I diagnosed an aortic dissection in a patient with an echo […] [that I realized] the many diagnoses I was able to make and the patients that I was able to help by knowing echocardiography” (I-14, F/C3/high).

When reflecting on their experience, highly engaged learners also reported stress when performing CCE initially, but eventually enjoyed it.

Quantitative effect of curricular changes on learners’ engagement

Teaching attendance and echocardiogram submissions are presented in ESM eTable 3. Learners submitted unsupervised echocardiograms with a 1.11-fold higher rate when supervision was conditional than when supervision was unconditional (rate ratio, 1.11; 95% credible interval [CrI], 1.08 to 1.14) (ESM eTable 3 and eTable 4). Rotation in the echo lab was associated with a subsequent decrease in the rate of submitting unsupervised echocardiograms (Fig. 3 and ESM eTable 5). Trainees who went to the echo lab either did not submit any unsupervised echocardiograms after the echo lab rotation (odds ratio of submitting echocardiograms, 0.2; 95 % CrI, 0.1 to 0.33) or submitted at a substantially lower rate (rate ratio of submitting echocardiograms, 0.10; 95% CrI, 0.09 to 0.14).

Fig. 3
figure 3

Critical care echocardiograms submission pre- and post-echo lab rotation. The dotted vertical lines represent the rotation in the echo lab. In blue are the number of echocardiograms submitted monthly before the rotation in the echo lab. The red lines represent the monthly number of echocardiograms submitted after the rotation in the echo lab.

Full size image

Discussion

We found that learners reported lack of protected time and limited access to supervision as the main barriers to successful program completion. Nevertheless, unconditional access to en-bloc supervision or rotation in the echo lab (sustained supervision by a sonographer) did not increase independent practice. We assumed that offering supervision as a reward for regular independent practice would motivate learners to practice more independently between supervised sessions and would increase program completion rates. Providing shorter (two-hour) but more frequent supervised sessions distributed throughout the year was expected to foster consolidation of newly acquired skills compared with longer (week-long), but infrequent (twice a year) supervised sessions. Our assumption was based on existing evidence showing that distributive practice can lead to superior learning outcomes compared with en-bloc practice.44,45 In implementing these changes, we hoped to motivate learners and to identify trainees who, by showing that they had practiced independently, were more likely to benefit from further supervision.52 We were also cognizant of the potential risk of “too much” supervision, which may limit learners’ ability to learn to perform independently.53 Surprisingly, the implementation of conditional and distributed supervision was perceived negatively by many learners and failed to increase the rate of training completion, despite having a positive impact on independent practice, as measured by the number of unsupervised echocardiograms submitted for review.

Our findings indicated that throughout the three years of the advanced CCE program implementation, many learners lost their motivation early in the academic year. There were different reasons for this: 1) discouragement in response to the skill complexity and lack of perceived progress, 2) negative emotions experienced during independent practice, 3) unintended negative perception of the conditional supervision implemented after year 1 and, 4) learners’ realization of a mismatch between their goals and the course objectives. Our assumption that the promise of supervision would be globally perceived as a reward and increase learners’ motivation to engage in unsupervised practice was incorrect. As described by Deci, contingent rewards may have a negative effect on intrinsic motivation.54,55,56 Applying contingent supervision may have blunted learner’s sense of autonomy and volition, resulting in lower intrinsic motivation. Interestingly, whereas some learners became disengaged, others embraced the challenges encountered by investing more efforts in training. As they started seeing the benefits of their efforts and of their newly acquired skills in clinical practice, they were increasingly motivated to pursue training, which they eventually reportedly enjoyed.

Additionally, we observed that, following the echo lab rotation, trainees submitted even fewer echocardiograms than before. We speculate that as learners gained confidence throughout their echo lab rotation, they may have deemed subsequent independent practice unnecessary.57 Possibly facing a steep de-learning curve without immediate skill consolidation after the echo lab rotation, they may have faced unexpected challenges when resuming independent practice on challenging ICU patients.28,44,58

Our low rate of program completion is similar to those described in previous reports.16,22 Likewise, technical challenges (especially without supervision), clinical duties conflicting with educational activities, variable willingness to persist despite challenges, and personal factors were identified as major barriers.16 Our study also suggests that underlying factors, such as motivation and persistence when facing failure, may be playing a role, which aligns with self-regulated learning theory.59 Indeed, learning advanced echocardiography in the critical care context appears to be more challenging than learning echocardiography on cardiology patients. Moreover, learning advanced CCE differs from echocardiography training for cardiologists, which heavily relies on extended time in the echo lab where supervision (by a cardiologist or sonographer) is not a limitation. Advanced CCE learners may not expect a slow learning curve, a realization that may lead them to abandon training unless they remain intrinsically motivated and have personal goals aligned with curricular objectives.

Our study has several limitations. First, the sample size was small, which is why we used Bayesian instead of frequentist statistics to analyze the data. Second, at the time of program inception, our units did not have the equipment to perform transesophageal echocardiography, so our program was restricted to transthoracic echocardiography. Furthermore, the paucity of highly trained echocardiographers among critical care attendings may have been a limitation. Indeed, learners had limited opportunities to observe the use and benefit of echocardiography, and to obtain real-time feedback from their primary clinical supervisor during their daily clinical activities. Third, relevant residual confounding (explaining low completion) cannot be excluded. Fourth, interviews were analyzed by two investigators, one of whom had an instrumental role in the design and delivery of the program. To mitigate narrow interpretations, we incorporated reflexivity throughout the research process. Fifth, although our study elicited possible explanations for low engagement, further research is needed to clarify differences between learners that maintained motivation throughout training and achieved CCE competence and those that failed to do so.

It may be argued that our program was too ambitious, especially for learners without previous basic CCE experience; however, the requirement of 150 echocardiograms was based on international recommendations and matched the requirements for North American certification.11 Furthermore, learners could complete their portfolio in more than one year and could remotely submit their echocardiograms for review. Lastly, we did not explore if and to what extent supervision impacted trainees’ learning curve. The fact that supervision failed to increase learners’ engagement and program completion does not invalidate its importance. Indeed, given the technical challenges of performing echocardiography on critically ill patients, novices may be unable to discern whether low-quality echocardiographic images relate to patients’ factors or to their inexperience.

In summary, lack of time and limited access to supervised practice were the major barriers reported by learners. Nevertheless, the provision of protected time in the echo lab and direct supervision in a critical care setting did not increase the likelihood of trainees completing their portfolios. Other factors such as sustained motivation and persistence may play significant roles in learners’ engagement and training completion. Further research is needed to better understand potential modifiers of motivation and engagement of learners enrolled in advanced CCE training.