Participants and ultrasound models
Twenty-four medical students who attended Jichi Medical University between April 1 and October 31, 2021 were included in this prospective randomized study. In Japan, each medical university provides a 6-year undergraduate education program. Third- or fourth-year medical students with no or little experience in training in FOCUS skills were invited to participate in this study. They finished attending systematic lectures on cardiology and passed the written examination in the second year. Four students in the same year were recruited in each extracurricular seminar for the purpose of this study. The seminar was conducted six times in total for 24 students. In each seminar, two healthy male students were recruited to serve as ultrasound models. One served as the model for telepresence instruction, and the other for the ultrasound skill pre-test and post-test. Pre-screening with ultrasonography showed that they had good windows for obtaining cardiac views. All participants and models gave their written informed consent before inclusion in this study. The study was approved by the Ethics Committee of Jichi Medical University (Approval Case Number 20-185).
A flipped classroom approach was adopted to enhance telepresence instruction of FOCUS in this study. Self-learning methods before telepresence instruction included a pre-recorded video lecture with or without self-training using a handheld ultrasound device (Vscan Extend with Dual Probe, GE Healthcare). The principal investigator, who is an experienced doctor certified by the Japan Society of Ultrasonics in Medicine, made the video lecture material with PowerPoint (length: 48 min) and posted it online. The on-demand video lecture material consisted of the following chapters: fundamental principles of ultrasound, techniques of FOCUS and normal findings, common abnormalities, and “self-performed FOCUS”. In this study, we selected the five cardiac views: parasternal long-axis (PLAX), parasternal short-axis at the aortic level (PSAX A), parasternal short-axis at the papillary muscle level (PSAX PM), apical four-chamber (A4C), and inferior vena cava (IVC). The subcostal four-chamber (S4C) view, which is commonly included in FOCUS examinations, was also discussed in the lecture. However, the view was not adopted in skill training, considering that it seemed difficult for medical students to obtain the view in healthy models , and because the models were likely to feel discomfort due to repeated compression of the subcostal area during the hands-on session. M-mode and Doppler mode were not included in the study.
The self-performed FOCUS technique in addition to the standard FOCUS technique was introduced in the video lecture. Self-performed FOCUS means that examiners performed FOCUS on their own bodies. The movie in the video lecture on learning self-performed FOCUS techniques using a handheld device included positioning and probe manipulation with corresponding cardiac views (Fig. 1). The technique shown in the movie was performed by an experienced doctor who was certified by the Japan Society of Ultrasonics in Medicine. It was considered that the movie might help students conduct self-performed FOCUS using the handheld device in self-training at home.
During the telepresence instruction, participants were isolated from the instructor (principal investigator) in separate rooms. Telepresence instruction was conducted using a video communication software program (Zoom; Zoom Video Communications Inc., San Jose, CA) combined with video recording and a live streaming software program (OBS Studio). The latter software program allowed simultaneous visualization of the ultrasound screen (left side) and the student’s hand position (right side) on the screen of a laptop computer. The screen on the handheld ultrasound device was displayed on the laptop screen using a Miracast receiver and a video capture device. The hand position was monitored using an external web camera connected to the laptop. The laptop screen was simultaneously displayed on the other screen of the laptop for the instructor via the video communication software program. This system allowed the instructor to remotely guide each participant in image acquisition from the healthy model. Other participants were present on-site to observe students being guided by the instructor in the performance of FOCUS either directly or with a wide monitor connected to a laptop. The observation was instructive for them, as well. Both the instructor and participants in the separate rooms could point to some structures with a pointer and draw lines or arrows on the shared screen using the function included in the communication software to enhance the bidirectional education. The setup for the telepresence instruction is shown in Fig. 2.
Telepresence instruction was provided to the four participants for 80 min in total using one model. Each participant had approximately 10 min to practice visualizing the PLAX view followed by PSAX views under tele-guidance. After the four participants finished the practice, each participant then had approximately 10 min to practice visualizing the A4C view followed by the IVC view. The goal was for all participants to be able to visualize each view such that it was acceptable for interpretation . The achievement of this goal was assessed by the instructor.
Written pre-test and post-test
The written pre-test and post-test, which consisted of 20 multiple-choice questions (MCQs), were conducted before and after self-learning. The MCQ tests covered cardiac anatomy and physiology (five questions), fundamental principles of ultrasound (five questions), normal ultrasound images (five questions), and abnormal ultrasound images (five questions). The total score ranged from 0 to 20 points. Each question was shown in a PowerPoint presentation for one minute in order. Identical MCQs were used for the pre-test and post-test.
Skill pre-test and post-test
Before and after the telepresence instruction, the participants took an ultrasound skill pre-test and post-test using the other model, to evaluate their image acquisition skill in five cardiac views. Each participant was asked to visualize and maintain each view within one minute in the order of PLAX, PSAX A, PSAX PM, A4C, and SIVC. Once they realized that they had continuously visualized the best view for more than five seconds, they could finish that view within one minute and move on to the next view. If they could not achieve appropriate visualization within one minute, they had to finish the view and move on to the next view.
Using the same system used for telepresence instruction, these ultrasound movies were recorded in full and stored in the laptop. At a later date, the principal investigator clipped the last five seconds of each view and pasted it on the PowerPoint slide with the corresponding movie clip obtained from the same model by the principal investigator. The latter clip was used as the reference standard. Two raters (other experienced doctors certified by the Japan Society of Ultrasonics in Medicine) blindly graded each movie clip in random order on a scale from 1 to 5 based on a previous study (1, no meaningful image; 2, poor or insufficient for interpretation; 3, good or acceptable for interpretation; 4, excellent or minor suggestions for improvement; 5, outstanding or no suggestions for improvement) . When a disagreement occurred, the median of three values rated by the two raters and the third rater was adopted as the score for each view. The total score for the five views ranged from 5 to 25 points.
The participants were asked to assess the self-learning methods and the telepresence instruction on a 5-point Likert scale (1, strongly disagree; 2, disagree; 3, neutral; 4, agree; 5, strongly agree). The questionnaires were filled out anonymously. The first and second perception surveys were conducted after the skill pre-test and skill post-test, respectively. Table 1 shows the questionnaires in the surveys. Questions V1 and V2 in the first perception survey and Questions V3, V4, and V5 in the second perception survey were for the video lecture group. Questions S1 and S2 in the first perception survey and Questions S3, S4, and S5 in the second perception survey were for the self-training group.
A schematic representation of the process of the study is shown in Fig. 3. Each time, four participants joined a seminar room after school on Tuesday. They initially completed the written pre-test. After the test, the instructor did not give them the question sheets or tell them the correct answers. The participants were then randomized into either “video lecture group” or “self-training group” at a ratio of 2:2 using the block randomization method with a computer-generated random sequence in Microsoft Excel (Microsoft Corporation, Redmond, WA).
Participants in the video lecture group watched the video lecture at home as pre-learning before telepresence instruction. They could watch it multiple times at their own pace. Participants in the self-training group watched the video in the same manner. In addition, they self-performed FOCUS using the handheld device based on the manual shown in the video lecture. They were encouraged to record ultrasound movie clips of the five views during self-training and to return the handheld device with the stored clips on the day of telepresence instruction; however, this was not mandatory. The participants in both groups had the opportunity to learn FOCUS by themselves using these pre-learning materials until the day of telepresence instruction.
The four participants joined the same seminar room on the weekend (Saturday afternoon or Sunday morning). At first, they completed the identical written post-test, and then checked the answers with the instructor’s comments. Next, the participants in the video lecture group received a brief instruction on how to operate the handheld ultrasound device before the skill pre-test. In both groups, each participant underwent the skill pre-test followed by the first perception survey. Then, telepresence instruction was provided for 80 min in total. Finally, in both groups, each participant underwent the skill post-test followed by the second perception survey.
The demographic data of each group were summarized as frequencies. The weighted kappa (κ) statistic was used to assess the agreement of skill test scores in each view between the two raters. (κ < 0, no agreement; κ > 0 to ≤ 0.2, slight agreement; κ > 0.2 to ≤ 0.4, fair agreement; κ > 0.4 to ≤ 0.6, moderate agreement; κ > 0.6 to ≤ 0.8, substantial agreement; and κ > 0.8 to ≤ 1, almost perfect agreement) . The Wilcoxon signed-rank test was used to compare scores between the written pre-test and post-test and between the skill pre-test and post-test. The Mann–Whitney U test was used to compare the skill pre-test scores between the video lecture group and the self-training group. In the analysis of the questionnaire results, the Mann–Whitney U test was used to compare scores between Question V2 and Question S2 (between groups). The Wilcoxon signed-rank test was used to compare scores between Question V2 and Question V4 and between Question S2 and Question S4 (before and after the telepresence instruction). These statistical analyses were two-sided, with P values of < 0.05 considered statistically significant. The analyses were carried out using the STATA version 13.1 software program (StataCorp LP, College Station, TX, USA).