Advertisement

Evaluation of a new goal-directed training curriculum for point-of-care ultrasound in the emergency department: impact on physician self-confidence and ultrasound skills

  • Di Shi
  • Jihai LiuEmail author
  • Jun Xu
  • Huadong Zhu
  • Xuezhong Yu
Open Access
Original Article
  • 134 Downloads

Abstract

Purpose

Developing countries need effective and efficient training curriculum for the point-of-care ultrasound (POCUS) in the emergency department (ED). We have developed a new goal-directed training curriculum focusing on critical POCUS procedures used in ED.

Methods

To evaluate the impact of the new POCUS curriculum on ED physicians’ performance/self-confidence, we carried out a quasi-experimental trial at ED training center of Peking Union Medical College Hospital, in which we trained two groups of physicians using either traditional curriculum or the new goal-directed curriculum. We measured the confidence in performing Focused Assessment with Sonography in Trauma, thoracic, vascular, ultrasound-guided puncture, echocardiography and undifferentiated shock diagnostic ultrasound at baseline, training completion and 1 month after training. We also measured the performance skills at the time of training completion. The relative value of the new curriculum was evaluated by differences in the confidence and performance measurements, with control for baseline measurements and confounding characteristics in univariate analyses and multivariate linear regression models.

Results

After training, both groups of trainees reported significantly increased self-confidence in performing POCUS as compared with baseline. Trainees with the new curriculum had statistically significant higher self-confidence increase and performance scores (p < 0.05), e.g., increase in confidence for diagnosis of undifferentiated shock at training completion = 3.14 vs. 1.85, 1 month after training = 2.22 vs. 1.56, for new and traditional curriculum, respectively. The new curriculum also resulted in a higher number of shock POCUS done within 1 month after training: 1.73 vs. 0, and higher overall performance scores: 165 vs. 113. The findings were robust when controlling for imbalanced baseline characteristics in multivariate regression models.

Conclusion

We conclude that a goal-directed, intensive but brief ED POCUS curriculum significantly increases trainee self-confidence, performance, and promote trainees to perform more procedures.

Keywords

Emergency medicine Ultrasound Education 

Abbreviations

US

Ultrasound

POCUS

Point-of-care ultrasound

ED

Emergency department

EM

Emergency medicine

ER

Emergency room

PUMCH

Peking Union Medical College Hospital

RUSH

Rapid ultrasound for shock and hypotension

FAST

Focused assessment with sonography in trauma

UGP

Ultrasound-guided puncture

THIRD

Tamponade, heart, inferior vena cava, respiratory system, deep venous thrombosis

REDCap

Research electronic data capture

Background

Many clinical settings have adopted the point-of-care ultrasound (POCUS) in practices [1, 2, 3, 4, 5]. Recognizing the value of the POCUS and the need for adequate training, various clinical settings have developed POCUS training curriculums [6, 7, 8, 9, 10, 11]. The unique characteristics of emergency clinical workflow require dedicated training for POCUS in the emergency department (ED). ED physicians need to perform the image acquisition on the spot and extract information in a timely fashion to aid the patient diagnosis and treatment. On the other hand, the application of POCUS in ED can be intensely focused on the specific use cases and often driven by common goals of emergency medicine (EM) patient management.

Although various EM POCUS training guidelines have been developed [12, 13, 14], POCUS training curriculums in ED remained quite varying between institutions, as recognized by the International Federation for Emergency Medicine in 2015 [6, 15]. Typical curriculums usually take several days including hours of didactic training followed by practices on proctored scans. The lengthy sessions of studying the course materials, interactive online learning, modular learning, and hands-on experiences are to ensure the trainee will acquire the knowledge and skills to efficiently utilizing POCUS. However, time-consuming sessions are expensive to institute, especially for developing countries with a considerable gap in physician POCUS expertise and limited resources. Curriculums that delivers both efficiency and quality are in great need to enable trainees in achieving a solid understanding of the anatomic structure, obtaining high-quality images in standard views, recognizing pathological findings and supporting clinical decisions.

With the aid of new technology of 3-D ultrasound image recording, we have developed a new, brief, goal-directed curriculum for POCUS training in ED. We expect the goal-directed curriculum, requiring the same amount of training time, will outperform the previous standard curriculum by improving trainee self-confidence and increasing performance carrying out POCUS. This study aimed first to introduce the new brief POCUS curriculum and then to evaluate the impact of the new curriculum on trainee performance/self-confidence in relative to the standard curriculum.

Methods

Study design and participants

This study was a quasi-experimental trial carried out to investigate the impact of two different POCUS curriculums on ED physicians’ performance/self-confidence. We conducted the trial at the ED training center of Peking Union Medical College Hospital (PUMCH). We recruited participants enrolled in PUMCH emergency POCUS training course from 28 December 2015 to 17 June 2017. Trainees enrolled before 31 December 2016 underwent training using the traditional curriculum, and those enrolled after received training using the new curriculum. As the enrollment to the PUMCH bedside ultrasound training course was open to all interested emergency physicians nationwide, there was no requirement for prior ultrasound education or hands-on experiences, and no specific control for trainees’ allocation to either the traditional or the new goal-directed curriculum.

We approached the two consecutive cohorts of trainees for consent and included only consent trainees in the study. We further excluded trainees who failed to finish the course or/and did not respond to follow-up. The institutional board of PUMCH reviewed and approved the study, considering this study presented no more than minimal risk to subjects.

Study interventions

For both standard and new curriculums, trainees participated in a two and half day training course in which they formed study groups of about four to seven members. One or two teaching physicians were responsible to coach individual study group in hands-on sessions.

Traditional curriculum

Subjects enrolled in the training program before 31 December 2016 received training using a traditional curriculum. The curriculum started with an introductory lecture on the fundamental ultrasound knowledge, which covered physics and instrumentation of emergency ultrasound, followed by lectures on how to obtain standard ultrasound views without emphasis on relevant ultrasound anatomy, or normal/pathological images interpretation. After the educational lectures, the program provided hands-on practices on simulated manikins for trainees to obtain standard views with the guidance of instructors. The curriculum completed with the introduction of Rapid Ultrasound for Shock and Hypotension (RUSH) protocol and a lecture for using POCUS to aid the diagnosis of undifferentiated shock.

New goal-directed curriculum

Subjects enrolled in 2017 underwent training using an improved, goal-directed curriculum. We identified the five core applications that ED physicians commonly perform to assess patients in their anatomical structures and underlying pathophysiological changes: Focused Assessment with Sonography in Trauma (FAST), thoracic, vascular, ultrasound-guided puncture (UGP) and echocardiography. Details of the five ultrasound applications are included in Appendix 1. The learning objectives of the training were to facilitate trainees to correctly carry out the above five ultrasound applications and obtain accurate assessment through a limited number of standard views. Also, we retained our emphasis on the POCUS procedure for undifferentiated shock diagnosis and introduced the “Tamponade/Tension, pneumothorax, Heart, Inferior vena cava, Respiratory system, Deep venous thrombosis/Aorta Dissection” (THIRD) procedure in the new curriculum [16].

Before formal courses, we provided trainees with pre-course learning materials to familiar with the necessary knowledge of ultrasound and the consensus of evaluation for undifferentiated shock patients. The pre-course materials contained reading literature and web-based video sessions. The ED of PUMCH has published details about the pre-course learning in May 2017 [16].

During the formal course, we improved the first introductory lecture for basic ultrasound knowledge by integrating hands-on practice with coaching, which included ultrasound probe holding, probe placement/movements, understanding of three ultrasound plains of tridimensional organ, explanation on four sides of the view, and summary on six steps of obtaining a high-quality view.

After the introductory lecture on basic ultrasound knowledge, we also changed the sequence of subsequent training sections with the improvement of contents and technologies. Subjects had the first lecture on the introduction of anatomy with 3D video records and photos of dissecting real organ. The second lecture provided the tips to gain a standard view in details and compared 3D images with normal ultrasound views. Trainees practiced drawing ultrasound views on simulated manikins after this session. Each trainee was assigned to a group for hands-on practice instructed by group instructor, the ratio of instructor to trainee was 1:7. Once the trainee performed required standard view on the manikins, simulated 3-D images were presented to the trainee to demonstrate various pathological findings under that particular standard view. We also provided standard patients with no pathological conditions to trainees alongside with the simulated manikins. Subjects had the last lecture on the interpretation of pathological ultrasound images with the real clinical case.

The final step of the new curriculum was to review and practice in groups all critical ultrasound diagnostic/treatment procedures (FAST, thoracic, vascular, UGP, and echocardiography) with emphasis on the THIRD protocol for undifferentiated shocks. Instructors presented simulated case scenarios and corresponded ultrasound findings while trainees were obtaining images on models.

Measurements and outcomes

We conducted a baseline survey of study participants before training for basic demographics and prior ultrasound experiences, which include seven single choice questions and one open-ended question. The participants were requested to rate for the self-confidence in performing of five core applications: FAST, thoracic, vascular, UGP, echocardiography and diagnosing undifferentiated shock (RUSH for old curriculum or THIRD for new curriculum). The confidence measurements ranged from 1 to 5 with 1 being the least confident and 5 being the most confident.

Multiple course instructors assessed trainees’ performance in carrying out the ED ultrasound procedures (FAST, thoracic, echocardiography, and UGP) after training. To give a fair assessment for trainees enrolled in the study, we assigned course instructors to evaluate the performance of trainees who they were not in direct coaching with during hands-on and group sessions. However, we could not completely blind the instructors of the trainees’ seniority and experiences. Instead, we specifically required an objective assessment of the trainees’ skill level using an anchored rating scale as the assessment tool: We assign score of one to participants who were unable to obtain a view and unable to explain normal image even with extensive instructor’s coaching; three when they were able to obtain a low-quality view and give some of the image interpretation with some coaching; five when they were able to gain a standard view with high image quality and make accurate image interpretation without assistance. The total performance score summing up the 4 subcomponents, 36 items ranges from 36 to 180. All course instructors received prior standardized training in the right using of the assessment tool.

One month after the completion of the training, we contacted trainees about the reassessment of self-confidence and post-training ultrasound practices. We chose to follow-up on trainees’ self-confidence 1 month after the training because we believe by that time trainees would have had enough time to digest the learning in the course and make enough attempts to practice POCUS independently.

This survey was administered online using the research electronic data capture (REDCap) medical data system.

Statistical analysis

An independent statistician analyzed the data. The primary outcome was trainees’ performance measured after the training. Secondary outcomes included self-confidence measured at the completion of training and 1 month after. To control for the differential distribution of baseline self-confidence between groups, we calculated the differences between the post-training and the baseline self-confidence measurements. To compare the traditional curriculum and new goal-directed curriculum, we tabulated trainees’ baseline characteristics and outcomes by training groups, with means and standard deviations presented for continuous variables, and counts and percentages for categorical variables. We carried out univariate statistical comparisons between training groups using the two-sample t test supplemented by the two-sample Wilcoxon rank sum test (given the small sample size and skewed distributions of variables) for continuous variables, and the Fisher's exact test for categorical variables.

We also performed multivariate linear regression analyses to examine the independent association between training programs and outcomes while controlling for the potential baseline confounders. Only imbalanced baseline characteristics between the two trainee cohorts were included in the multivariate regression model (either p value < 0.05 or relative differences greater than 10%). The statistical significance threshold was set at p < 0.05. Individuals with missing data were excluded from analysis. All data analyses were performed using STATA 12.0 (StataCorp. 2011. Stata Statistical Software: Release 12. College Station, TX: StataCorp LP.).

Patient and public involvement

Our study participants were not considered as patients, and there was no other public involvement other than the trainees enrolled in PUMCH emergency POCUS training course. The study’s results were disseminated to the participants who chose to receive updates after.

Results

Overall sample

During the two consecutive training programs, we enrolled and consented 66 trainees from over 20 hospitals throughout China. We excluded two trainees who did not finish the course for personal reasons. Twenty-seven trainees enrolled before 2017 received traditional POCUS training, while 37 trainees received training using the new curriculum.

On average, trainees had 5.58 years of practice experiences in ED, and 51.6% had some prior ED ultrasound training. Many indicated that they had performed ED ultrasound before, with the most frequently performed being ultrasound-guided puncture (45.3%), followed by vascular (37.5%), echocardiography (28.1%), and the least performed being diagnostic procedures for undifferentiated shock (7.8%). Similarly, the prior experiences with different ultrasound procedures were also reflected in the corresponding baseline self-confidence, with diagnosing undifferentiated shock being the one with the lowest confidence. However, a small proportion of them indicated that they were independently performing ED ultrasound before training (9.4%) (Table 1).

After training, all trainees reported significantly increased self-confidence in performing ED ultrasound as compared with baseline. One month after training completion, the confidence tended to decrease moderately from the end of training but was still higher than the baseline measurements. For example, confidences in performing shock protocol were 1.50, 3.44 and 2.59 for baseline, training completion and 1 month after training, respectively (Tables 1, 2).

Trainees’ baseline characteristics by training curriculums

Trainees received traditional/new training programs that differ in several baseline characteristics. Trainees with new curriculum tended to have more years practicing in ED (7.41 vs. 3.07, p < 0.001), more likely to have prior ED ultrasound training (64.9% vs. 33.3%, p < 0.001), performed slightly more ED ultrasound before (total amount: 17.35 vs. 13.70, p = 0.208; counts of types of ED ultrasound: 2.03 vs. 1.19, p = 0.060). Trainees who received the new curriculum also had higher baseline self-confidence measurements than those who received traditional curriculum (Table 1). However, trainees with new curriculum are less likely to indicate they were independently performing ED ultrasound (8.1% vs. 11.1%) with an insignificant p value of 0.691 at the threshold of 0.05.
Table 1

Baseline characteristics of trainees before ER ultrasound training programs

Baseline characteristics

All

Traditional program

Goal-directed program

p value

p value*

Years practicing in ER

5.58 ± 4.56

3.07 ± 2.22

7.41 ± 4.97

0.000

0.000

Prior ER US training (yes)

33 (51.6)

9 (33.3)

24 (64.9)

0.022

 

Independently Performing ER ultrasound (yes)

6 (9.4)

3 (11.1)

3 (8.1)

0.691

 

Number of ER US performed

15.81 ± 27.75

13.70 ± 23.06

17.35 ± 30.95

0.208

0.591

Number of ER US performed (categorical)

 0–10

45 (70.3)

19 (70.4)

26 (70.3)

0.999

 

 11–120

19 (29.7)

8 (29.6)

11 (29.7)

  

Number of ER US performed per month

2.83 ± 5.44

1.33 ± 2.09

3.92 ± 6.77

0.091

0.034

Counts of types of ER US performed

1.67 ± 2.02

1.19 ± 1.80

2.03 ± 2.13

0.060

0.092

Types of ER US performed

 FAST (yes)

14 (21.9)

7 (25.9)

7 (18.9)

0.551

 

 Thoracic (yes)

13 (20.3)

2 (7.4)

11 (29.7)

0.033

 

 Vascular (yes)

24 (37.5)

9 (33.3)

15 (40.5)

0.609

 

 USP (yes)

29 (45.3)

9 (33.3)

20 (54.1)

0.130

 

 Echocardiography (yes)

18 (28.1)

4 (14.8)

14 (37.8)

0.053

 

 Shock (yes)

5 (7.8)

0 (0.0)

5 (13.5)

0.068

 

Months since last ER US

 ≦6 months

48 (75.0)

21 (77.8)

27 (73.0)

0.682

 

  > 6 months

14 (21.9)

6 (22.2)

8 (21.6)

  

 Never performed

2 (3.1)

0 (0.0)

2 (5.4)

  

Pre-training self-confidence performing

 FAST

1.77 ± 0.96

1.59 ± 0.84

1.89 ± 1.02

0.210

0.205

 Thoracic

1.83 ± 1.05

1.48 ± 0.75

2.08 ± 1.16

0.024

0.015

 Vascular

2.03 ± 0.96

1.52 ± 0.75

2.41 ± 0.93

0.000

0.000

 USP

2.25 ± 1.18

1.70 ± 0.99

2.65 ± 1.16

0.001

0.001

 Echocardiography

1.78 ± 0.90

1.52 ± 0.80

1.97 ± 0.93

0.032

0.040

 Shock

1.50 ± 0.76

1.33 ± 0.55

1.62 ± 0.86

0.199

0.109

*Supplemental p values calculated from two-sample Wilcoxon rank sum test

Trainees’ post-training measurements by training curriculums

Trainees with new curriculum had statistically significantly higher measurements in self-confidence after training, reflected in numbers both at training completion and at 1 month after training. Even after adjusting for the differential distribution of baseline self-confidence by calculating the absolute delta of post-training confidence from baseline, the new curriculum was associated with higher trainee self-confidence (post-training vs. baseline and 1-month post-training vs. baseline). For example, in FAST performance, the confidence measurements were 3.92 (new) vs. 3.37 (traditional) for post-training, 3.27 vs. 2.26 for 1-month post-training, 2.03 vs. 1.78 for increased confidence post-training as relative to baseline, and 1.38 vs. 0.67 for increased confidence 1-month post-training as relative to baseline.

Trainees with new curriculum also ended up performing more ultrasound post-training than trainees received traditional curriculum (14.2 vs. 1.44, p < 0.001) and had higher performance scores at training completion (165.76 vs. 113.44, p < 0.001).

When adjusting for potential confounding factors such as imbalanced baseline characteristics and baseline confidence levels, the new curriculum was consistently associated with higher increases in self-confidence measured at either training completion or 1 month after (except for ultrasound-guided punctures), more substantial number of ultrasound performed in 1-month post-training, and higher performance scores at training completion (Table 3).
Table 2

Outcome assessments after ER ultrasound training

Outcomes

All

Traditional program

Goal-directed program

p value

p value*

Post-training self-confidence

 

 FAST

3.69 ± 0.50

3.37 ± 0.49

3.92 ± 0.36

0.000

0.000

 Thoracic

3.63 ± 0.65

3.19 ± 0.62

3.95 ± 0.47

0.000

0.000

 Vascular

3.69 ± 0.66

3.19 ± 0.56

4.05 ± 0.47

0.000

0.000

 UGP

3.72 ± 0.81

3.44 ± 0.58

3.92 ± 0.89

0.002

0.012

 Echocardiography

3.58 ± 0.64

3.19 ± 0.62

3.86 ± 0.48

0.000

0.000

 Shock

3.44 ± 0.75

2.89 ± 0.64

3.84 ± 0.55

0.000

0.000

One-month post-training self-confidence

 FAST

2.84 ± 0.88

2.26 ± 0.45

3.27 ± 0.87

0.000

0.000

 Thoracic

3.16 ± 0.95

2.52 ± 0.58

3.62 ± 0.89

0.000

0.000

 Vascular

3.02 ± 1.03

2.22 ± 0.58

3.59 ± 0.90

0.000

0.000

 UGP

2.94 ± 1.01

2.30 ± 0.67

3.41 ± 0.96

0.000

0.000

 Echocardiography

2.75 ± 1.01

1.89 ± 0.64

3.38 ± 0.72

0.000

0.000

 Shock

2.59 ± 0.92

1.85 ± 0.60

3.14 ± 0.71

0.000

0.000

Delta of self-confidence (post-training vs. baseline)

 

 FAST

1.92 ± 0.82

1.78 ± 0.64

2.03 ± 0.93

0.120

0.208

 Thoracic

1.80 ± 0.80

1.70 ± 0.54

1.86 ± 0.95

0.176

0.393

 Vascular

1.66 ± 0.74

1.67 ± 0.68

1.65 ± 0.79

0.730

0.922

 UGP

1.47 ± 0.84

1.74 ± 0.71

1.27 ± 0.87

0.022

0.021

 Echocardiography

1.80 ± 0.74

1.67 ± 0.55

1.89 ± 0.84

0.180

0.203

 Shock

1.94 ± 0.81

1.56 ± 0.58

2.22 ± 0.85

0.001

0.000

Delta of self-confidence (1-month post-training vs. baseline)

 FAST

1.08 ± 1.31

0.67 ± 1.00

1.38 ± 1.44

0.004

0.023

 Thoracic

1.33 ± 1.38

1.04 ± 0.98

1.54 ± 1.59

0.021

0.124

 Vascular

0.98 ± 1.09

0.70 ± 0.87

1.19 ± 1.20

0.027

0.065

 UGP

0.69 ± 1.14

0.59 ± 1.05

0.76 ± 1.21

0.526

0.564

 Echocardiography

0.97 ± 1.22

0.37 ± 1.08

1.41 ± 1.14

0.000

0.000

 Shock

1.09 ± 1.06

0.52 ± 0.89

1.51 ± 0.99

0.000

0.000

Number of ultrasound performed within 1-month post-training

 Total ultrasound

8.83 ± 11.49

1.44 ± 2.04

14.22 ± 12.54

0.000

0.000

 FAST

0.97 ± 1.53

0.07 ± 0.27

1.62 ± 1.74

0.000

0.000

 Thoracic

2.30 ± 3.93

0.44 ± 0.75

3.65 ± 4.71

0.000

0.000

 Vascular

1.91 ± 2.65

0.15 ± 0.60

3.19 ± 2.84

0.000

0.000

 UGP

1.00 ± 1.75

0.48 ± 0.75

1.38 ± 2.15

0.279

0.023

 Echocardiography

2.89 ± 4.22

0.33 ± 0.78

4.76 ± 4.71

0.000

0.000

 Shock

1.00 ± 1.94

0.00 ± 0.00

1.73 ± 2.31

0.000

0.000

Performance score post-training

 

 Total score

143.69 ± 28.07

113.44 ± 12.80

165.76 ± 8.58

0.000

0.000

 FAST

44.84 ± 9.41

35.30 ± 5.78

51.81 ± 3.53

0.000

0.000

 Thoracic

28.98 ± 5.24

23.89 ± 2.72

32.70 ± 3.00

0.000

0.000

 Vascular

31.20 ± 6.94

24.44 ± 4.20

36.14 ± 3.52

0.000

0.000

 Echocardiography

34.58 ± 7.40

26.52 ± 2.24

40.46 ± 2.83

0.000

0.000

*Supplemental p values calculated from two-sample Wilcoxon rank sum test

Table 3

Differences in outcomes between traditional and goal-directed programs, estimated in multivariate regression model

Outcome

Covariates adjusted

Effect estimate and 95% CI (new goal-directed vs. traditional program)

p value

US performance post-training

 Echocardiography performance

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing echocardiography US; ever performed echocardiography US

14.0 (12.5–15.5)

 < 0.001

 FAST performance

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing FAST US; ever performed FAST US

16.2 (13.5–19.0)

 < 0.001

 Vascular performance

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing vascular US; ever performed vascular US

11.3 (8.9–13.7)

 < 0.001

 Thoracic performance

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing thoracic US; ever performed thoracic US

8.6 (6.9–10.2)

 < 0.001

 Total performance

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence (sum of confidence scores before training: FAST, thoracic, vascular, USP, echocardiography, shock)

49.5 (43.3–55.3)

 < 0.001

Delta of confidence post-training vs. baseline

 FAST

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing FAST US; ever performed FAST US

0.4 (0.1–0.6)

0.002

 Thoracic

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing thoracic US; ever performed thoracic US

0.6 (0.3–0.8)

 < 0.001

 Vascular

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing vascular US; ever performed vascular US

0.6 (0.3–0.8)

 < 0.001

 UGP

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing USP US; ever performed USP US

–0.04 (–0.42 to 0.34)

0.850

 Echocardiography

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing echocardiography US; ever performed echocardiography US

0.4 (0.1–0.6)

0.010

 Shock

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing shock US; ever performed shock US

0.8 (0.5–1.1)

 < 0.001

Delta of confidence 1-month post-training vs. baseline

 FAST

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing FAST US; ever performed FAST US

1.0 (0.5–1.4)

 < 0.001

 Thoracic

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing thoracic US; ever performed thoracic US

1.3 (0.9–1.8)

 < 0.001

 Vascular

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing vascular US; ever performed vascular US

1.1 (0.6–1.6)

 < 0.001

 UGP

Years in ER; Prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing USP US; ever performed USP US

0.7 (0.2–1.3)

0.009

 Echocardiography

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing echocardiography US; ever performed echocardiography US

1.4 (1.0–1.9)

 < 0.001

 Shock

Years in ER; prior ER US training; number of ER US performed; number of different ER US performed; baseline confidence of performing shock US; ever performed shock US

1.3 (0.8–1.7)

 < 0.001

Number of ER US performed within 1 month after training

Years in ER; prior ER US training; number of ER US performed per month before training; number of different ER US performed; baseline confidence (sum of confidence scores before training: FAST, thoracic, vascular, USP, echocardiography, shock)

11.7 (5.6–17.8)

 < 0.001

Discussion

Bedside ultrasound has become a vital tool to aid diagnosis and treatment in the ED. Better utilization of bedside ultrasound has also been associated with improved patient outcomes and decreased costs in the ED [2, 16, 17, 18, 19, 20]. However, multiple studies have recognized a gap of POCUS utilization in ED and identified the lack of adequate training as the primary barrier for effective usage of POCUS [21, 22]. In a 2014 national survey of US critical care fellowship directors, the authors indicated even in a country like the United States, which have developed guidelines for POCUS decades ago, there were significant variations of the existing curriculums and a dire need for an improved curriculum to close the gap in POCUS training [4]. In developing countries in which resources are scarce, ED physicians face more challenges in adopting POCUS in their daily practices.

First, our study quantitatively assessed the gap of POCUS expertise in Chinese ED physicians. Although many trainees had prior experiences performing POCUS before the training, a tiny proportion of them indicated that they were independently performing ultrasound examinations, and on average, the self-confidence levels were all lower or close to a rating of two (“no confidence but willing to try”) before training. When it comes to complex bedside ultrasound procedures such as RUSH or THIRD, It is not surprising that only 7.8% of trainees had ever performed POCUS to aid diagnosis of undifferentiated shock in ED practice. At the time of the study, most Chinese ED programs relied primarily on informal bedside teaching for ED POCUS without a proper ultrasound curriculum in place. The training team is rarely staffed with enough experienced course instructors and uses outdated teaching materials.

Second, this study has demonstrated that a two and a half day ultrasound course using a new curriculum can effectively train ED clinicians in POCUS skill. Compared with the traditional training curriculum, trainees with the new curriculum had higher performance ratings even after adjustment for multiple confounders. The new curriculum was also associated with increased self-confidence at the completion of the training, which persisted when re-evaluated in 1 month after. Interestingly, the treatment effect (the new curriculum vs. the traditional) in the delta of self-confidence was of greater magnitude when measured in 1 month after training, compared with that measured at training completion. We hypothesize that the new curriculum allowed trainees with greater confidence after training completion and encouraged them to perform more POCUS procedures, which snowballed and provided additional positive feedbacks in increasing ones’ self-confidence.

Third, we designed and tailored the new curriculum to meet the greater need for effective and efficient ED POCUS training programs in low- and middle-income countries like China. Compared with North American and European countries, developing countries face more challenges of limited resources and heavy patient workload. Curriculums in developed countries usually mandate a low student to instructor ratio (no higher than 5–1), may require pathology models to include standard patients with pathological conditions, and usually take a longer time to complete (half-day for a single application) [6, 15, 23]. When developing the new curriculum, we have integrated other countries’ experiences of designing a POCUS training program in a limited resource setting [8, 21, 24, 25, 26]. We have identified the commonly utilized POCUS procedures in Chinese ED and designed the intense training sessions accordingly. Our curriculum was shorter, more focused on the selective list of procedures, and maintained relatively high students to instructors’ ratios.

We believe that the increase in performance rating, number of POCUS performed after training, and improved self-confidence can be attributable to multiple improvements we made to the curriculum. The main change of the new curriculum was the inclusion of the procedure-focused training for Fast, Thoracic, vascular, USP, echocardiography and undifferentiated shock. Trainees were trained to operate ultrasound on standard manikins and patients, perform the above standard procedures while presented with simulated clinical scenarios. We believe the simulated scenarios are more reflective of a real clinical setting and boost the retention of new skills and information with lower technique requirement on simulating resource [27].

In the new curriculum, we experimented with the concept of the flipped classroom by providing the teaching materials and instructional contents before the didactics. The utilization of flipped classroom provided the opportunity for more time spent in the didactics reviewing and coached practices so that the curriculum could achieve better results within limited training time [23]. Also, we believe inserting hands-on practice before the lecture of pathological images also helped trainees better understand the anatomical structure, so that they could relate the ultrasound maneuvers to abnormal findings on the images obtained. As noted in previous guideline [13, 23, 28], rotating the skill learning and didactics sessions also improves the efficiency of the training program. Lastly, the availabilities of interactive, 3-D images made it possible to present simulated pathological findings while trainees were practicing standard views on manikins, without having real standardized patients available for every pathological condition.

There are several limitations to our study. There was no randomization of trainees’ allocation to the old or new curriculum, which resulted in the differential distribution of baseline characteristics of trainees. Especially for trainees’ performance measurement, we were not able to collect the baseline measurement to be controlled for. However, the treatment effect of our intervention on outcomes remained robust when evaluated as a delta from baseline, or in multivariate regression models adjusted for the imbalanced baseline characteristics. We acknowledge the potential existence of residual confounding. Its relative small sample size also limits this study, and the training curriculum was only developed and evaluated in a single institution. However, our recruitment of trainees was open to hospitals nationwide, and our results confirmed a broad distribution of physicians with different seniority and experiences of POCUS. We expect our findings to generalize well in other institutions.

In our study, we have not directly measured the curriculums on all the Kirkpatrick’s four level evaluation matrices. We have focused more on the behaviors and results aspects by measuring the changes in self-confidence performing POCUS, and number of cases performed after the training. Future researches are going to provide more detailed information revealing the subjective reactions of the trainees, and the reasons why they have/have not achieved the learning goal. Those additional measurements will provide further insight to find the root cause of success and failures in teaching POCUS at the bedside in ED, and help improve the new curriculum.

Conclusion

This study suggests that a goal-directed, intensive but brief (2.5 day) ED POCUS curriculum is associated with a significant increase in trainee self-confidence and performance. The focused training on critical POCUS procedures commonly used in the ED setting has contributed to improved ED POCUS skills. Further study is needed to identify the optimal allocation of time between lectures and hands-on practices and explore other enhancements for the curriculum.

Notes

Author contributions

Di Shi was the major contributor to write the manuscript; Jihai Liu made a contribution to the study design and collected data; Jun Xu analyzed the data; Huadong Zhu and Xuezhong Yu interpreted the data. All the authors read and approved the final manuscript.

Funding

This work was supported by CAMS Fundamental research fund for Central public welfare research institute, Grant number 2018PT32004.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no competing interests.

Ethics approval

The institutional board of Peking Union Medical College Hospital (PUMCH) reviewed and approved the study, the address of the approving ethics committee is No. 1 Shuai Fu Yuan, Dong Cheng District, Beijing, China, 100,730.

Availability of data and material

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. 1.
    Heller M, Melanson SW. Applications for ultrasonography in the emergency department. Emerg Med Clin North Am. 1997;15(4):735–44.CrossRefGoogle Scholar
  2. 2.
    Atkinson PR, et al. Abdominal and cardiac evaluation with sonography in shock (ACES): an approach by emergency physicians for the use of ultrasound in patients with undifferentiated hypotension. Emerg Med J. 2009;26(2):87–91.CrossRefGoogle Scholar
  3. 3.
    Kotagal M, et al. Impact of point-of-care ultrasound training on surgical residents' confidence. J Surg Educ. 2015;72(4):e82–e8787.CrossRefGoogle Scholar
  4. 4.
    Mosier JM, et al. Critical care ultrasound training: a survey of US fellowship directors. J Crit Care. 2014;29(4):645–9.CrossRefGoogle Scholar
  5. 5.
    Whitson MR, Mayo PH. Ultrasonography in the emergency department. Crit Care. 2016;20(1):227.CrossRefGoogle Scholar
  6. 6.
    Atkinson P, et al. International Federation for Emergency Medicine point of care ultrasound curriculum. CJEM. 2015;17(2):161–70.CrossRefGoogle Scholar
  7. 7.
    Kelm DJ, et al. Longitudinal ultrasound curriculum improves long-term retention among internal medicine residents. J Grad Med Educ. 2015;7(3):454–7.CrossRefGoogle Scholar
  8. 8.
    Shah S, et al. Development of an ultrasound training curriculum in a limited resource international setting: successes and challenges of ultrasound training in rural Rwanda. Int J Emerg Med. 2008;1(3):193–6.CrossRefGoogle Scholar
  9. 9.
    Shokoohi H, et al. An experiential learning model facilitates learning of bedside ultrasound by preclinical medical students. J Surg Educ. 2016;73(2):208–14.CrossRefGoogle Scholar
  10. 10.
    So S, Patel RM, Orebaugh SL. Ultrasound imaging in medical student education: impact on learning anatomy and physical diagnosis. Anat Sci Educ. 2017;10(2):176–89.CrossRefGoogle Scholar
  11. 11.
    Hoppmann RA, et al. The evolution of an integrated ultrasound curriculum (iUSC) for medical students: 9-year experience. Crit Ultrasound J. 2015;7(1):18.CrossRefGoogle Scholar
  12. 12.
    Mateer J, et al. Model curriculum for physician training in emergency ultrasonography. Ann Emerg Med. 1994;23(1):95–102.CrossRefGoogle Scholar
  13. 13.
    American College of Emergency Physicians. American College of Emergency Physicians. ACEP emergency ultrasound guidelines-2001. Ann Emerg Med. 2001;38(4): 70-81Google Scholar
  14. 14.
    Henneberry RJ, et al. Use of point of care sonography by emergency physicians. CJEM. 2012;14(2):106–12.CrossRefGoogle Scholar
  15. 15.
    Moore CL, Gregg S, Lambert M. Performance, training, quality assurance, and reimbursement of emergency physician-performed ultrasonography at academic medical centers. J Ultrasound Med. 2004;23(4):459–66.CrossRefGoogle Scholar
  16. 16.
    Shi D, Zhang QB, Cao GH, et al. The expert consensus of bedside ultrasound assessment for undifferentiated shock. Chin J Emerg Med. 2017.26(5):498–506.Google Scholar
  17. 17.
    Laursen CB, et al. Point-of-care ultrasonography in patients admitted with respiratory symptoms: a single-blind, randomised controlled trial. Lancet Respir Med. 2014;2(8):638–46.CrossRefGoogle Scholar
  18. 18.
    Peterson MA, et al. Ultrasound for routine lumbar puncture. Acad Emerg Med. 2014;21(2):130–6.CrossRefGoogle Scholar
  19. 19.
    Stengel D, et al. Emergency ultrasound-based algorithms for diagnosing blunt abdominal trauma. Cochrane Database Syst Rev 2015;(9):Cd004446.Google Scholar
  20. 20.
    Melniker LA, et al. Randomized controlled clinical trial of point-of-care, limited ultrasonography for trauma in the emergency department: the first sonography outcomes assessment program trial. Ann Emerg Med. 2006;48(3):227–35.CrossRefGoogle Scholar
  21. 21.
    Becker DM, et al. The use of portable ultrasound devices in low- and middle-income countries: a systematic review of the literature. Trop Med Int Health. 2016;21(3):294–311.CrossRefGoogle Scholar
  22. 22.
    Henwood PC, et al. Characterizing the limited use of point-of-care ultrasound in Colombian emergency medicine residencies. Int J Emerg Med. 2014;7(1):7.CrossRefGoogle Scholar
  23. 23.
    American College of Emergency Physicians. Ultrasound guidelines: emergency, point-of-care and clinical ultrasound guidelines in medicine. Ann Emerg Med. 2017;69(5):e27–e54.CrossRefGoogle Scholar
  24. 24.
    Shah SP, et al. Impact of the introduction of ultrasound services in a limited resource setting: rural Rwanda 2008. BMC Int Health Hum Rights. 2009;9:4.CrossRefGoogle Scholar
  25. 25.
    Adler D, et al. Introduction of a portable ultrasound unit into the health services of the Lugufu refugee camp, Kigoma District, Tanzania. Int J Emerg Med. 2008;1(4):261–6.CrossRefGoogle Scholar
  26. 26.
    Shmueli H, et al. Briefly trained medical students can effectively identify rheumatic mitral valve injury using a hand-carried ultrasound. Echocardiography. 2013;30(6):621–6.CrossRefGoogle Scholar
  27. 27.
    McGaghie WC, et al. A critical review of simulation-based mastery learning with translational outcomes. Med Educ. 2014;48(4):375–85.CrossRefGoogle Scholar
  28. 28.
    Mills AM, Raja AS, Marin JR. Optimizing diagnostic imaging in the emergency department. Acad Emerg Med. 2015;22(5):625–31.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Di Shi
    • 1
  • Jihai Liu
    • 1
    Email author
  • Jun Xu
    • 1
  • Huadong Zhu
    • 1
  • Xuezhong Yu
    • 1
  1. 1.Department of Emergency MedicinePeking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical SciencesBeijingChina

Personalised recommendations