Self-learning followed by telepresence instruction of focused cardiac ultrasound with a handheld device for medical students: a preliminary study

Abstract

Purpose

This study aimed to assess the feasibility and efficiency of self-learning with or without self-training (subjects performed scans on themselves) and telepresence instruction in focused cardiac ultrasound (FOCUS) education for medical students.

Methods

This study included 24 medical students. The participants initially completed a written pre-test and were randomized into a video lecture (participants watched a video lecture) or self-training (participants watched a video lecture and self-performed FOCUS) group. After finishing self-learning, they completed a written post-test. Then they undertook a skill pre-test and a first perception survey. Telepresence instruction was then provided. Finally, they undertook a skill post-test and a second perception survey.

Results

The written post-test total scores were significantly higher than the pre-test total scores (P < 0.001). In the skill pre-test, the scores for the video lecture and self-training groups were not significantly different (P = 0.542). The skill post-test total scores were significantly higher than the skill pre-test total scores (P = 0.008). Forty-two percent of the video lecture group participants agreed that the video lecture was effective preparation for the skill pre-test, while all participants in the same group agreed that the combination of the video lecture and telepresence instruction was effective preparation for the skill post-test.

Conclusion

This study demonstrated the feasibility and efficiency of self-learning followed by telepresence instruction on FOCUS for medical students.

Introduction

With the miniaturization of diagnostic ultrasound machines and a growing body of evidence supporting its utility, point-of-care ultrasonography (POCUS) has been extensively utilized by clinicians for decision-making and improving the quality of invasive procedures [1, 2]. Therefore, it is reasonable to suggest that training in the use of POCUS should now be included in undergraduate medical education [3,4,5,6].

Training medical students to perform POCUS is expected to facilitate the study of anatomy and physiology, improve the efficiency of physical examinations, and aid them in acquiring diagnostic and procedural skills [3, 4, 6]. Interestingly, the real-time visual information and feedback obtained with portable or handheld ultrasound devices increase their motivation to learn [3, 7]. Accessibility and lower cost make handheld ultrasound devices attractive tools for undergraduate medical education. Notably, self-training on the use of these devices is feasible, regardless of the location or timing. Students can learn at their own pace by scanning each other or by scanning themselves. Such self-training may be as effective as traditional courses [8,9,10], especially during the current COVID-19 pandemic when there are some restrictions on medical education [11]. Furthermore, it is predicted that self-training will gain more attention as handheld ultrasound devices become more widely available among medical students. As mentioned above, the format of self-training on ultrasound for medical students includes students scanning each other and scanning themselves [10]. During the COVID-19 pandemic, it has seemed easier for students to scan themselves than to scan each other without faculty supervision to enforce infection control. However, to the best of our knowledge, no studies have investigated—using objective measures—the effectiveness of self-training (in which students scanned themselves).

Tele-ultrasound with telepresence instruction has been utilized for lectures, hands-on training, and patient management [12]. In undergraduate medical education, a pilot and feasibility study has shown that medical students who were novice ultrasound users were able to obtain a cardiac view using a handheld device through telepresence instruction [13]. Poland et al. [14] demonstrated that both in-person instruction and telepresence instruction resulted in significant increases in knowledge, confidence, and performance of a focused assessment with sonography for trauma. Höhne et al. [15] demonstrated that medical students achieved an average score of 88% in the Objective Structured Assessment of Ultrasound Skills at 8 weeks after taking a telepresence course on the assessment of the abdomen, lung, and thyroid gland with a handheld ultrasound device using a flipped classroom approach. During the COVID-19 pandemic, many medical schools have converted both preclinical and clinical sessions to e-learning, small-group sessions, and distance learning [16]. Telepresence instruction on ultrasonography seems to be a training method for mitigating the loss of face-to-face ultrasound education in the current era and may be applied in the remodeling of POCUS education in future [11]. However, there is little evidence to support the use of telepresence instruction to teach medical students focused cardiac ultrasound (FOCUS), which is a core POCUS application in postgraduate and undergraduate medical education [1, 2, 5, 10].

The objectives of this study were to assess the feasibility and efficiency of self-learning with or without self-training—in which students scanned themselves—and telepresence instruction in FOCUS education.

Materials and methods

Participants and ultrasound models

Twenty-four medical students who attended Jichi Medical University between April 1 and October 31, 2021 were included in this prospective randomized study. In Japan, each medical university provides a 6-year undergraduate education program. Third- or fourth-year medical students with no or little experience in training in FOCUS skills were invited to participate in this study. They finished attending systematic lectures on cardiology and passed the written examination in the second year. Four students in the same year were recruited in each extracurricular seminar for the purpose of this study. The seminar was conducted six times in total for 24 students. In each seminar, two healthy male students were recruited to serve as ultrasound models. One served as the model for telepresence instruction, and the other for the ultrasound skill pre-test and post-test. Pre-screening with ultrasonography showed that they had good windows for obtaining cardiac views. All participants and models gave their written informed consent before inclusion in this study. The study was approved by the Ethics Committee of Jichi Medical University (Approval Case Number 20-185).

Self-learning methods

A flipped classroom approach was adopted to enhance telepresence instruction of FOCUS in this study. Self-learning methods before telepresence instruction included a pre-recorded video lecture with or without self-training using a handheld ultrasound device (Vscan Extend with Dual Probe, GE Healthcare). The principal investigator, who is an experienced doctor certified by the Japan Society of Ultrasonics in Medicine, made the video lecture material with PowerPoint (length: 48 min) and posted it online. The on-demand video lecture material consisted of the following chapters: fundamental principles of ultrasound, techniques of FOCUS and normal findings, common abnormalities, and “self-performed FOCUS”. In this study, we selected the five cardiac views: parasternal long-axis (PLAX), parasternal short-axis at the aortic level (PSAX A), parasternal short-axis at the papillary muscle level (PSAX PM), apical four-chamber (A4C), and inferior vena cava (IVC). The subcostal four-chamber (S4C) view, which is commonly included in FOCUS examinations, was also discussed in the lecture. However, the view was not adopted in skill training, considering that it seemed difficult for medical students to obtain the view in healthy models [8], and because the models were likely to feel discomfort due to repeated compression of the subcostal area during the hands-on session. M-mode and Doppler mode were not included in the study.

The self-performed FOCUS technique in addition to the standard FOCUS technique was introduced in the video lecture. Self-performed FOCUS means that examiners performed FOCUS on their own bodies. The movie in the video lecture on learning self-performed FOCUS techniques using a handheld device included positioning and probe manipulation with corresponding cardiac views (Fig. 1). The technique shown in the movie was performed by an experienced doctor who was certified by the Japan Society of Ultrasonics in Medicine. It was considered that the movie might help students conduct self-performed FOCUS using the handheld device in self-training at home.

Fig. 1

Demonstration of self-performed focused cardiac ultrasound using a handheld device during a video lecture. An experienced doctor visualized the parasternal long-axis view

Telepresence instruction

During the telepresence instruction, participants were isolated from the instructor (principal investigator) in separate rooms. Telepresence instruction was conducted using a video communication software program (Zoom; Zoom Video Communications Inc., San Jose, CA) combined with video recording and a live streaming software program (OBS Studio). The latter software program allowed simultaneous visualization of the ultrasound screen (left side) and the student’s hand position (right side) on the screen of a laptop computer. The screen on the handheld ultrasound device was displayed on the laptop screen using a Miracast receiver and a video capture device. The hand position was monitored using an external web camera connected to the laptop. The laptop screen was simultaneously displayed on the other screen of the laptop for the instructor via the video communication software program. This system allowed the instructor to remotely guide each participant in image acquisition from the healthy model. Other participants were present on-site to observe students being guided by the instructor in the performance of FOCUS either directly or with a wide monitor connected to a laptop. The observation was instructive for them, as well. Both the instructor and participants in the separate rooms could point to some structures with a pointer and draw lines or arrows on the shared screen using the function included in the communication software to enhance the bidirectional education. The setup for the telepresence instruction is shown in Fig. 2.

Fig. 2

Setup for the telepresence instruction

Telepresence instruction was provided to the four participants for 80 min in total using one model. Each participant had approximately 10 min to practice visualizing the PLAX view followed by PSAX views under tele-guidance. After the four participants finished the practice, each participant then had approximately 10 min to practice visualizing the A4C view followed by the IVC view. The goal was for all participants to be able to visualize each view such that it was acceptable for interpretation [17]. The achievement of this goal was assessed by the instructor.

Written pre-test and post-test

The written pre-test and post-test, which consisted of 20 multiple-choice questions (MCQs), were conducted before and after self-learning. The MCQ tests covered cardiac anatomy and physiology (five questions), fundamental principles of ultrasound (five questions), normal ultrasound images (five questions), and abnormal ultrasound images (five questions). The total score ranged from 0 to 20 points. Each question was shown in a PowerPoint presentation for one minute in order. Identical MCQs were used for the pre-test and post-test.

Skill pre-test and post-test

Before and after the telepresence instruction, the participants took an ultrasound skill pre-test and post-test using the other model, to evaluate their image acquisition skill in five cardiac views. Each participant was asked to visualize and maintain each view within one minute in the order of PLAX, PSAX A, PSAX PM, A4C, and SIVC. Once they realized that they had continuously visualized the best view for more than five seconds, they could finish that view within one minute and move on to the next view. If they could not achieve appropriate visualization within one minute, they had to finish the view and move on to the next view.

Using the same system used for telepresence instruction, these ultrasound movies were recorded in full and stored in the laptop. At a later date, the principal investigator clipped the last five seconds of each view and pasted it on the PowerPoint slide with the corresponding movie clip obtained from the same model by the principal investigator. The latter clip was used as the reference standard. Two raters (other experienced doctors certified by the Japan Society of Ultrasonics in Medicine) blindly graded each movie clip in random order on a scale from 1 to 5 based on a previous study (1, no meaningful image; 2, poor or insufficient for interpretation; 3, good or acceptable for interpretation; 4, excellent or minor suggestions for improvement; 5, outstanding or no suggestions for improvement) [17]. When a disagreement occurred, the median of three values rated by the two raters and the third rater was adopted as the score for each view. The total score for the five views ranged from 5 to 25 points.

Perception surveys

The participants were asked to assess the self-learning methods and the telepresence instruction on a 5-point Likert scale (1, strongly disagree; 2, disagree; 3, neutral; 4, agree; 5, strongly agree). The questionnaires were filled out anonymously. The first and second perception surveys were conducted after the skill pre-test and skill post-test, respectively. Table 1 shows the questionnaires in the surveys. Questions V1 and V2 in the first perception survey and Questions V3, V4, and V5 in the second perception survey were for the video lecture group. Questions S1 and S2 in the first perception survey and Questions S3, S4, and S5 in the second perception survey were for the self-training group.

Table 1 Questionnaire (5-point Likert scale) and the results of the first and second perception surveys

Study design

A schematic representation of the process of the study is shown in Fig. 3. Each time, four participants joined a seminar room after school on Tuesday. They initially completed the written pre-test. After the test, the instructor did not give them the question sheets or tell them the correct answers. The participants were then randomized into either “video lecture group” or “self-training group” at a ratio of 2:2 using the block randomization method with a computer-generated random sequence in Microsoft Excel (Microsoft Corporation, Redmond, WA).

Fig. 3

Schematic representation of the process of the study

Participants in the video lecture group watched the video lecture at home as pre-learning before telepresence instruction. They could watch it multiple times at their own pace. Participants in the self-training group watched the video in the same manner. In addition, they self-performed FOCUS using the handheld device based on the manual shown in the video lecture. They were encouraged to record ultrasound movie clips of the five views during self-training and to return the handheld device with the stored clips on the day of telepresence instruction; however, this was not mandatory. The participants in both groups had the opportunity to learn FOCUS by themselves using these pre-learning materials until the day of telepresence instruction.

The four participants joined the same seminar room on the weekend (Saturday afternoon or Sunday morning). At first, they completed the identical written post-test, and then checked the answers with the instructor’s comments. Next, the participants in the video lecture group received a brief instruction on how to operate the handheld ultrasound device before the skill pre-test. In both groups, each participant underwent the skill pre-test followed by the first perception survey. Then, telepresence instruction was provided for 80 min in total. Finally, in both groups, each participant underwent the skill post-test followed by the second perception survey.

Statistical analyses

The demographic data of each group were summarized as frequencies. The weighted kappa (κ) statistic was used to assess the agreement of skill test scores in each view between the two raters. (κ < 0, no agreement; κ > 0 to ≤ 0.2, slight agreement; κ > 0.2 to ≤ 0.4, fair agreement; κ > 0.4 to ≤ 0.6, moderate agreement; κ > 0.6 to ≤ 0.8, substantial agreement; and κ > 0.8 to ≤ 1, almost perfect agreement) [18]. The Wilcoxon signed-rank test was used to compare scores between the written pre-test and post-test and between the skill pre-test and post-test. The Mann–Whitney U test was used to compare the skill pre-test scores between the video lecture group and the self-training group. In the analysis of the questionnaire results, the Mann–Whitney U test was used to compare scores between Question V2 and Question S2 (between groups). The Wilcoxon signed-rank test was used to compare scores between Question V2 and Question V4 and between Question S2 and Question S4 (before and after the telepresence instruction). These statistical analyses were two-sided, with P values of < 0.05 considered statistically significant. The analyses were carried out using the STATA version 13.1 software program (StataCorp LP, College Station, TX, USA).

Results

Table 2 shows the demographics of the participants in the video lecture group (n = 12) and self-training group (n = 12). Table 3 shows the results of the written pre-test and post-test. The written post-test total scores were significantly higher than the pre-test total scores. During the telepresence instruction, all participants were able to visualize all views (PLAX, PSAX A, PSAX PM, A4C, and IVC views) that were acceptable for interpretation.

Table 2 Demographics of participants

Table 3 Results of the written pre-test and post-test

The weighted kappa values for assessing the agreement of the OSCE scores between the raters in the PLAX, PSAX A, PSAX PM, A4C, and IVC views were 0.585, 0.492, 0.556, 0.516, and 0.444, respectively. Moderate agreement was shown in each view. Table 4 shows the results of the skill pre-test and post-test. In the skill pre-test, there was no significant difference in the scores of the video lecture group and the self-training group. The skill post-test total scores were significantly higher than the skill pre-test total scores. Table 5 shows the number of video lecture group participants who had no prior experience in performing cardiac ultrasound by each skill pre-test score.

Table 4 Results of the ultrasound skill pre-test and post-test

Table 5. Number of video lecture group participants who had no prior experience in performing cardiac ultrasound by each skill pre-test score

The results of the questionnaires are shown in Table 1. All 24 participants completed the questionnaires in the first and second surveys. The scores of Question S2 were significantly higher than the scores of Question V2 (P = 0.004). The scores of Question V4 were significantly higher than the scores of Question V2 (P = 0.002). There were no significant differences in the scores of Questions S2 and S4 (P = 0.103).

Discussion

This preliminary study shows the feasibility of this flipped classroom approach, i.e., the combination of self-learning with or without self-training and telepresence instruction in FOCUS. In Questions V5 and S5 of the second perception survey, all attendees in both groups either agreed or strongly agreed that this approach deserves to be considered an effective educational method. Furthermore, all participants were able to visualize all views in the telepresence instructions and the views were therefore considered to be acceptable to understand the findings.

As a pre-learning tool, the video lecture was effective for improving knowledge. The total scores and scores for each chapter in the post-test were significantly higher in comparison to the pre-test scores. In Questions V1 and S1 of the first perception survey, all participants in both groups agreed or strongly agreed that the video lecture with or without the self-training was effective preparation for the written post-test. Moreover, the video lecture that included knobology and FOCUS techniques was also effective for the received telepresence instruction. In Question V3 of the second perception survey, all attendees in the video lecture group agreed or strongly agreed that the video lecture was effective for receiving telepresence instruction. Remarkably, as shown in Table 5, in the skill pre-test, some participants in the video lecture group who had no prior experience in the performance of cardiac ultrasound could visualize some of the cardiac views to a degree that was considered acceptable (3 points) or more than acceptable (4 or 5 points) for interpretation.

To demonstrate the effectiveness of self-training in which the students scanned themselves, we compared the skill pre-test scores between the video lecture group and the self-training group. The scores of the groups did not differ to a statistically significant extent. On the other hand, the first perception survey showed different results. All participants in the self-training group agreed or strongly agreed with Question S2 that the video lecture with the self-training was effective preparation for the skill pre-test, while for Question V2, only 42% of the participants in the video lecture group agreed or strongly agreed that the video lecture was effective preparation for skill pre-tests. The scores of Question S2 were significantly higher than the scores of Question V2. The results of the perception survey indicate that self-training may have had some positive effects on novice FOCUS trainees. Prior self-training may be useful for learning about anatomic relationships and sonographic landmarks [11], and may subsequently help reduce their cognitive load during the skill pre-test.

The detailed explanation of the self-performed FOCUS techniques combined with standard FOCUS techniques in the video lecture seemed to be appropriate for novices who were attempting to perform self-training. After the participants in the self-training group watched the videos, they performed self-training at their own discretion. We did not give them a specific assignment with regard to the acquisition of their own ultrasound images during self-training. They were encouraged to record the ultrasound movie clips in each view, but this was not mandatory. If medical students who were inexperienced in FOCUS had been given the assignment to acquire appropriate images in each view, record them, and submit them to faculty members, their motivation to learn FOCUS using this self-training method may have been higher. As a result, they may be able to reach a minimum level of competence in the performance of FOCUS and obtain greater benefit from face-to-face or telepresence instruction [8]. This hypothesis should be verified in future research using a self-training system that includes such specific assignments.

In the present study, we evaluated the effectiveness of the telepresence instruction more objectively by comparing skill pre-test and skill post-test scores. Two healthy models who had good windows for obtaining cardiac views were recruited in each seminar, with one used as model for telepresence instruction and the other model used for the skill pre-test and skill post-test. The post-test total scores were significantly higher than the pre-test total scores when both groups were combined. Significant differences were also observed in the A4C and IVC views, but not in the PLAX, PSAX A, or PSAX PM views. When interpreting the results, it has to be considered that the median pre-test scores for the A4C and IVC views (2 and 1.5 points, respectively) were lower than the median pre-test scores in the PLAX and PSAX PM views (3 and 3 points, respectively). These results indicate that for the majority of medical students with no or little experience in performing FOCUS, telepresence instruction in addition to self-learning may be especially useful for gaining competency in the ability to visualize the A4C and IVC views such that they were acceptable for interpretation (3 points) in the model students. In Question V4 of the perception surveys, all participants in the video lecture group agreed or strongly agreed that the combination of the video lecture and telepresence instruction was effective preparation for the skill post-test. In contrast, for Question V2, only 42% of the participants in the video lecture group agreed or strongly agreed that the video lecture was effective preparation for the skill pre-test. The scores for Question V4 were significantly higher than the scores for Question V2. The results also support the utility of telepresence instruction for the acquisition of FOCUS skill.

In some cases, the skill post-test scores were lower than the skill pre-test scores, in total or for individual views. The issue of reproducibility in the visualization of ultrasound images by novices has to be considered when evaluating their ultrasound skill. Otherwise, mental images formulated by scanning a model during telepresence instruction may negatively impact the visualization of images from the other model in the skill post-test.

This prospective randomized study had several limitations. First, this preliminary study had a relatively small study population and was undertaken at a single institution because it was designed to search for a suitable ultrasound training method for undergraduate medical education, at the time of the COVID-19 pandemic. Larger studies with standardized educational resources are needed to validate the findings obtained in this study. Second, the selection bias should be considered as more motivated medical students may have participated in this extracurricular course. Third, medical students with no or little experience in FOCUS skill training were invited to participate in this study as cardiac ultrasound novices. Actually, 29% of the participants had prior experience in performing cardiac ultrasound. The majority of these had informal scanning experience under the support of some faculty members, and had not received systematic education on FOCUS. Thus, they were considered to be novices. In reality, there was no significant difference in the skill pre-test scores between participants with and without prior cardiac ultrasound experience. Fourth, the agreement between the raters in the skill test results was moderate. A revision of the grading system may be needed to improve reproducibility. Fifth, the scores for questions in the anonymous perception surveys may have been influenced by interaction between participants and the principal investigator or the Hawthorne effect, which may have resulted in an overestimation of their assessments. Sixth, short-term or long-term retention of both knowledge and skills in relation to FOCUS was not evaluated in this flipped classroom approach.

Conclusion

This study demonstrated the feasibility of self-learning followed by telepresence instruction of FOCUS with a handheld device for medical students. The video lecture was effective for improving the participants’ knowledge and for receiving telepresence instruction. The study did not directly demonstrate, using any objective measures, the effectiveness of self-training in which the participants scanned themselves, but the perception surveys indicated that self-training may have had a positive effect on them. Telepresence instruction significantly improved their FOCUS performance.