Introduction

Point-of-care ultrasound (POCUS), a form of diagnostic and therapeutic ultrasonography (US) performed by physicians treating patients, is a popular choice in pediatric emergency medicine (PEM) [1] because of its ability to provide real-time imaging that enables physicians to make clinical decisions under time-sensitive conditions and because, unlike radiography or computed tomography, it does not involve radiation exposure, which can trigger secondary malignancies [2]. Thanks to its increasing popularity, in 2015 the American College of Emergency Physicians and the American Academy of Pediatrics published a consensus statement on its use [3], and in 2016 a systematic review of POCUS use in pediatric care proposed reporting guidelines for POCUS examinations [4]. Given its popularity, training courses in pediatric POCUS are being implemented worldwide, further corroborating the efficacy of this technique [5,6,7,8,9,10,11,12,13]. However, many of the training courses target specialists such as pediatric intensivists and pediatric cardiologists [7,8,9,10, 12], and focus on cardiac and soft tissue US or US-guided nerve block [6, 9, 11]. Only one study focused on the effectiveness of comprehensive training in the use of pediatric POCUS and demonstrated an increase in the knowledge and confidence level of its users [13].

In contrast, in Japan, a large proportion of pediatricians and emergency physicians have access to US machines and can perform US examinations. Despite the preference commonly shown by pediatricians for POCUS, as witnessed by the frequent and informal courses held on its use [14], no descriptive studies of POCUS use among pediatricians in Japan have been done, nor has a standardized, comprehensive training course for pediatric POCUS use been developed, and data on its efficacy have yet to be documented. To address this omission, we implemented a comprehensive pilot training program for pediatric POCUS use by adapting the aforementioned consensus guidelines to the current clinical situation in Japan [3, 4, 15] and evaluated its effectiveness. The level of participants’ satisfaction with the course was initially assessed in a pilot study in 2015, which demonstrated a high satisfaction rate [unpublished work]. The present study aimed to continue the inquiry by analyzing participants’ assessment of self-efficacy in the course.

Materials and methods

The present before-and-after study investigated the self-efficacy level of participants in a pediatric POCUS training program conducted between August 2019 and January 2020.

Pediatric POCUS training course

Instructors were recruited from different parts of Japan based on their previous experience in teaching US use at their respective institutions. They had at least 3 years of experience in using pediatric POCUS and previously completed the pediatric emergency US course certified by the World Interactive Network Focused on Critical Ultrasound (WINFOCUS) [16] or were certified by the Japan Society of Ultrasonics in Medicine. They then established a curriculum for pediatric POCUS training by discussion. Examples of POCUS applications in our courses were chosen by adapting previously published educational guidelines [3, 4, 15] to the current clinical situation in Japan, as shown in Table 1. Because all the teaching objectives could not be covered in one course, it was decided that they should be distributed over several courses. Previous data from a questionnaire administered at a hands-on session during the 29th annual meeting of the Japanese Society of Emergency Pediatrics in 2015 [unpublished work] showed that abdominal and cardiac US were relatively common US applications for pediatricians in Japan, whereas ocular or musculoskeletal US was less frequently used. Both relatively common US applications such as cardiac and abdominal US, and less frequent but clinically important ones including musculoskeletal and ocular US, or US in shock management, were selected for the course. The course was implemented at the 7th and 8th conferences of the Japan Society of Point-of-Care Ultrasound in August 2019 and January 2020, respectively, and was held separately from the main conference. The training consisted of a half-day tutorial, including three lectures (20 min each) and four hands-on training sessions (35 min each) with a trainer-to-learner ratio of 1:4–5 (Fig. 1). Of the three lectures, one dealt with cardiac US, and of the four hands-on sessions, two dealt with cardiac US and shock management. The remaining two lectures and hands-on sessions were based on the aforementioned consensus guidelines and dealt with POCUS use in abdominal, airway/lung, musculoskeletal, and ocular examinations, and US-guided procedures, as described in Table 2 and Fig. 1.

Table 1 Educational domains
Full size table
Fig. 1
figure 1

Course flow. The present comprehensive pediatric POCUS training course was held at the 7th and 8th conferences of the Japan Society of Point-of-Care Ultrasound in August 2019 and January 2020, respectively

Full size image
Table 2 Teaching domains selected for inclusion in the pediatric POCUS training course
Full size table

LOGIQ™ e (GE Healthcare, Japan), Xario200G (Cannon Medical Systems Corporation, Japan), Noblus (Hitachi Medical Systems, Japan), and Sonosite X-Porte (FUJIFILM, SonoSite Inc., Japan) were used for scanning. For cardiac, airway/lung, and ocular US, young adult volunteers were used for scanning, and for abdominal and musculoskeletal US, a pediatric anatomical simulator (Pediatric FAST/Acute Abdomen Phantom; Kyoto Kagaku Co., Ltd.) and a hand-made model using a chicken leg were used together with volunteers. The conference organizer explained the contents of the training course to the volunteers and obtained their oral consent prior to participation. For US-guided procedures such as US-guided peripheral line placement and US-guided nerve block, simulators (peripherally inserted central venous catheter simulator [PICC simulator], Kyoto Kagaku Co., Ltd.; CAE blue phantom, Gadelius Medical K.K.) were used.

Learning materials explaining scanning techniques for obtaining appropriate images and understanding normal and abnormal images for each US application were created for each learning objective shown in Table 1, and were provided to trainees one week before the start of the course. In the lectures, the instructors explained the learning objectives for each part and demonstrated normal and abnormal images using scanning video clips. In the hands-on sessions, the trainees had opportunities to scan live patients and/or simulators to practice their scanning skills under the instructors’ supervision. During the hands-on training, the instructors used standardized materials to teach scanning techniques in each session in accordance with each learning objective in Table 1, and used video clips (cardiac, airway/lung, abdominal, ocular, and musculoskeletal US, and US-guided procedures) and scanned images of pediatric anatomical simulators (abdominal US) or chicken legs (musculoskeletal US) to illustrate abnormal images in each session. The course director monitored all the stations to ensure a standard quality of instruction.

Trainees

Physicians interested in using pediatric POCUS were recruited using marketing materials at the Japan Society of Point-of-Care Ultrasound meetings to be trainees in the half-day pediatric POCUS training course. Pre-registration was required for participation; however, prospective trainees who failed to register were allowed to register on-site.

Questionnaire

The questionnaire used in our study was created by referencing previously published studies on implementing a POCUS training course [5, 17, 18] after discussion among the instructors (Table 3). Due to the difficulty of simultaneously evaluating all components of the Kirkpatrick model for each course, the overall satisfaction level with our training course and the trainees’ self-efficacy level in each US application before and after the course were selected for assessment in this pilot study.

Table 3 Pre- and post-course survey
Full size table

Data collection

Data were collected using printed materials, and pre- and post-training evaluation forms were completed by the trainees. All assessment forms, including items on the participants’ background, such as their post-graduate year (PGY), experience participating in pediatric POCUS courses, experience using pediatric POCUS in daily practice, self-efficacy in pediatric POCUS use before and after the course, and overall satisfaction with the course were provided in Japanese (Table 3). The instructors were blinded to the written questionnaire.

Measures

The self-efficacy and satisfaction rates were assessed using a Likert scale from 0 (no confidence) to 100 (complete confidence) with 11-point intervals measuring confidence [19], and a Likert scale from 0 (no satisfaction) to 10 (full satisfaction), respectively, after the training.

Statistical analysis

The effect size of the difference in confidence level for pediatric POCUS was calculated using previous data on confidence levels obtained from a hands-on session at the 29th annual meeting of the Japanese Society of Emergency Pediatrics in 2015 [unpublished work]. With an effect size of the difference in self-assessed competency of 1.2 on a 5-point Likert scale, a sample size of 24 participants with a 5% significance level was calculated to obtain 90% power. The statistical analysis was performed using SPSS statistical software, version 18.0 (SPSS Inc. Chicago, IL, USA). Summary statistics were expressed as proportions with 95% confidence intervals (CI), and tests for association were done using the paired t test. p ≤ 0.05 was considered to indicate statistical significance.

Ethical considerations

The present study was conducted in accordance with the Declaration of Helsinki (2013) and was approved by the Institutional Review Board of our hospital (Approval number 2019-b74). The aim of this study was inscribed on the questionnaires given to the trainees and was explained orally by the course director before the start of training. Participation was voluntary, and submission of the questionnaires was understood as the participants’ consent to be enrolled.

Results

Of the 31 physicians enrolled, 25 completed all the surveys. Table 4 shows their demographic characteristics. Ten participants were in PGY 1–2, 13 were in PGY 3–5, and eight were in PGY 6 or higher. Pediatricians comprised 38.7% of the trainees (12/31), followed by interns at 22.6% (7/31) and internal medicine physicians at 16.1% (5/31). Fifteen participants (48.3%) had previous experience of attending an educational course on POCUS, and 87.1% of the trainees responded that they used US in their daily practice. Table 5 shows that the overall self-efficacy level in pediatric POCUS use after training was significantly higher than in the pre-course assessment at 86.0 (standard deviation [SD] 19.2) and 35.6 (SD 17.6), respectively (mean difference [95% confidence interval] 49.6 [39.6–61.2]) (p < 0.05). For each component—including cardiac, abdominal, airway/lung, musculoskeletal, and ocular US, US-guided procedures, and shock management—the self-efficacy levels on the post-training evaluation were significantly higher. The increase in the self-efficacy level from before to after the course tended to be higher for musculoskeletal and ocular US than for cardiac and abdominal US, and the overall satisfaction rate on the post-training evaluation was high (Table 5). There was no difference in the self-efficacy level in terms of specialty (Table 6).

Table 4 Trainee demographics
Full size table
Table 5 Self-assessment of competency in pediatric POCUS
Full size table
Table 6 Self-efficacy in pediatric POCUS by specialty
Full size table

Discussion

To the best of our knowledge, the present study is the first to assess a comprehensive training course in pediatric POCUS in Japan. A pilot training course in pediatric POCUS was created on the basis of previously published guidelines and consensus statements [3, 4, 15], which include over 20 procedures that PEM physicians are required to master to treat any of a variety of physical conditions encountered in the course of their work [20]. In a study conducted in a pediatric emergency department (ED), a 1-day training course in POCUS use for pediatric soft tissue infections demonstrated improvement in the users’ knowledge and skills when examining cases of cellulitis and subcutaneous abscess [11]. Another study demonstrated that a web-based training course in US-guided nerve block in children increased physicians’ comfort and intention levels with this procedure [6]. In addition, a 1- to 3-day training course on lung and cardiac US for pediatric intensivists and pediatric cardiologists showed a significant improvement in physicians’ knowledge and skills in performing US and interpreting the findings [7,8,9,10, 12]. However, these studies focused on a narrow variety of applications. In contrast, a study conducted in a pediatric ED in Puerto Rico demonstrated that a 2-day course in pediatric POCUS including a broad spectrum of applications, such as cardiac, pulmonary, and soft tissue US, improved pediatricians’ knowledge and confidence level in pediatric POCUS; the sample size of this study, however, was small [13]. Similarly, the present study demonstrated an improvement in the self-efficacy resulting from comprehensive training in pediatric POCUS use. Thus, our findings have the potential to promote the use of POCUS in the pediatric emergency setting in Japan.

The present study also found a significant increase in the trainees’ self-efficacy levels in performing the procedure after training (Table 5). In our study, the self-efficacy level for ocular, fracture, and soft tissue US increased considerably between before and after the course, which was consistent with the findings of previously published studies of US training courses demonstrating that the self-efficacy level was lower in US applications that trainees had not experienced than in those they had, and that the self-efficacy level increased significantly after the course [5, 21, 22]. In addition, the hands-on session contributed greatly to enhancing trainees’ efficacy level, as seen previously in a study of pediatric POCUS training [5]. In our survey, many participants expressed satisfaction with the hands-on session. An increase in the trainees’ self-efficacy level in POCUS led to an increase in the frequency of their taking the initiative to perform US examinations in their practice, which will encourage further learning and skill improvements [13, 21]. Therefore, the results of the current study demonstrated the effectiveness of our training course in facilitating pediatric POCUS use.

From the medical education perspective, the Kirkpatrick model was used to evaluate educational efficacy [23]. This model consists of levels 1 to 4, each at which the trainees’ respond to their learning experience: learning outcomes, such as increased knowledge, skills, and changes in attitude towards their training; changes in trainees’ behavior and improvements in the quality of patient care; and the results and impacts of training are assessed [24]. The question in Table 3 asking “Overall, are you satisfied with this course?” aimed to assess the participants’ satisfaction level (level 1 criteria), whereas questions such as “Are you confident of your skills in airway/lung ultrasound?” aimed to assess their self-efficacy level for each type of pediatric POCUS application (level 2 criteria). The considerable improvement in self-efficacy scores seen in all categories demonstrated that the training satisfied the level 2 (Learning) criteria. Although investigating the applicability of the POCUS techniques taught in the course to the participants’ daily practice (level 3) and determining how these techniques might improve the quality of pediatric emergency care (level 4) are necessary to evaluate the efficacy of the course in an actual clinical setting, significant yet still preliminary progress was made by including a comprehensive range of procedures. The optimal duration of training is still unknown, but courses lasting one to three days are common [25]. Therefore, future training courses incorporating assessments at levels 3 and 4 of the Kirkpatrick model may be held for one to two days. Furthermore, during the current COVID-19 pandemic, most lectures and conferences have moved online or have combined face-to-face with online tutorials [26]. A previous study showed that there was no difference between web-based learning and face-to-face education in the acquisition of knowledge in EFAST use [27]. In addition, other studies demonstrated that online learning showed non-inferiority in terms of skill acquisition in cardiac and musculoskeletal US [28, 29]. Thus, online education can be a viable alternative to traditional face-to-face learning in pediatric POCUS training.

The present study had several limitations. First, the trainees’ image acquisition and interpretation and skill retention in pediatric POCUS were not assessed due to the constraints on time and space imposed by the fact that the course was an adjunct to an academic conference. The emphasis thus fell on investigating the feasibility of a pilot study enrolling physicians with a variety of different backgrounds. A study investigating these components is currently being developed for the next training course. Second, the sample size was quite small. However, it was sufficient to demonstrate a significant improvement in the trainees’ self-efficacy in performing POCUS, as seen in the sample size calculation based on our previous data. Third, physicians who participate in professional development activities such as skills training tend to show low self-efficacy in the pre-test setting; thus, selection bias may have entered into the present study. Fourth, in the hands-on sessions, young adult volunteers were used for scanning, which may have lessened the impact on educational objectives in pediatric POCUS. However, because recruiting pediatric volunteers was not feasible, pediatric anatomical simulators were used instead. Fifth, the improvement in self-efficacy levels may have been influenced by the Hawthorne effect. During the course, all the trainees were supervised by the instructors and the course director, possibly resulting in an overestimation of their self-assessment on the post-training survey. Therefore, as the next step, implementing an instructor training course to maintain the quality of teaching skills at a high level and planning a longer course in pediatric POCUS use that includes objective measures, such as an objective structured clinical examination (OSCE), computer-based tests, and a control group, may be desirable to evaluate trainees’ skills more accurately [30].

Conclusion

The present study implemented a training course in pediatric POCUS and demonstrated a significant improvement in the trainees’ self-efficacy level. The study revealed that comprehensive training has the potential to encourage physicians to use pediatric POCUS in their daily practice.