Background

The demand for quality radiology education as part of the medical curriculum has never been greater, as it allows for the visualization and conceptualization of intangible elements of human disease and brings into clinical practice the theoretical knowledge attained through anatomy, physiology, and pathology lectures. Gaining a strong foundation of core radiology knowledge is critical for medical students, as they will need these skills in their future clinical practices regardless of their specialty [1]. Exposing students to radiology in the early stages of their medical school education has been shown to help them make informed decisions when choosing a specialty [2,3,4].

In Turkey, like most other European countries, there is no standardized approach to the scheduling of educational programs or the amount of undergraduate radiology education content in medical schools [5, 6]. Despite several frameworks proposed by the European Society of Radiology (ESR), medical schools continue to develop their approaches to medical education, and significant variations exist, sometimes within the same nation [7, 8]. In a survey reported by the ESR, radiology education was found to be underrepresented in the current undergraduate curriculum when its role in medicine was taken into account [9]. Most radiology education content is delivered in the 4th and 5th years of medical education rather than in the pre-clinical years [9]. According to the same survey, radiology is also mostly taught in modules that are integrated with other clinical sciences, making it less pragmatically useful for students to study the topics for their exams and providing an inducement for students to skip radiological topics such as radiological anatomy [9].

The COVID-19 pandemic has drastically affected our approach to education worldwide. In March 2020, 61 countries began implementing measures to close down schools and universities [10]. The shutdown of classrooms and lecture halls has resulted in a surge of investment and interest in online education and distance learning, with a focus on developing new teaching strategies [11]. Medical schools have also been following this trend; with many engaging in online curricula, students have spent almost two full academic years with restricted access to their clinical rotations and patient interactions [12]. In terms of radiology education, similar steps had been taken before the pandemic, with more time allocated to case-based learning, teaching in small groups, hands-on workshops, and trainees going into reading rooms to learn more about radiological images; all of which have been particularly diminished during the pandemic [13].

To accommodate the conditions brought upon us by the pandemic, foster international collaborations in improving the quality of undergraduate medical education, and abide by the ESR guidelines on how radiological curriculum could be improved, we have designed, implemented, and evaluated a series of online courses in Turkey called the “Principles of Radiology”. Our goal was to evaluate the effectiveness of such an online course among Turkish medical students. To the best of our knowledge, this is the first article assessing, from the students’ point of view, radiology education in Turkish medical schools with an online radiology course completed nationwide.

Methods

This observational study was approved by Baylor College of Medicine institutional review board (IRB) (protocol number: H-47,848). A written informed consent was obtained from each participant upon their application to the seminar.

Course description

An online, free of charge multidisciplinary diagnostic radiology education course was designed and implemented by radiology educators from different United States based institutions as an addendum to the ongoing Turkish radiology curriculum. The course director recruited instructors from different institutions with expertise in the discussed topics and teaching to medical students. A similar design had been conducted among medical students in the United States, Canada, and Egypt before. The goal was to teach the principles of commonly used imaging modalities with example clinical applications. The final selected imaging modalities and topics are listed in Table 1. The course content and learning objectives include all of the obligatory learning objectives of “the Turkish National Core Education Programme”. The course, also, involved some of the recommended but not obligatory learning objectives. The course language was English. The reason for this language was that both the lecturers were from the USA and all the medical schools in Turkey provide an English preparatory-year or preparatory-class. A total of 9 sessions were held over 9 weeks. Each session lasted on average for 90 min and usually included a 60-minute didactic lecture on the concepts, history, and evolution of each imaging modality followed by examples of clinical scenarios. A pre-test and post-tests were administered at the beginning and end of the lecture, respectively and fifteen minutes on average were dedicated to questions and answers.

Table 1 Pre-test and post-test scores for each session
Full size table

Online learning platform

The course sessions were delivered via Zoom Webinar (Zoom Video Communications, San Jose, CA, zoom.us), a video conferencing platform which accommodates large numbers of attendees. It also provides several features that help moderating the sessions and organizing the discussions. These features include a chat box, a question-and-answer session organizer, and added controls of who can share screen, audio, and video. A built-in polling feature was also utilized during several sessions to encourage participation and provide the speaker with information about the students’ level of knowledge about the session’s topic.

Student enrollment

In order to have a cohort that was representative of Turkish medical students, efforts were made to communicate with and assign student ambassadors in multiple medical schools in Turkey including state and private ones. The course was advertised through the student ambassadors using different channels such as social media platforms, emails, and among interest groups. An electronic registration form was distributed and interested students were asked to sign-up.

Tests and test score analysis

Before and after each session, attendees were required to take a pre-test and a post-test, respectively, consisting of 10–15 multiple choice questions via Qualtrics (Qualtrics, Provo, UT). The pre-tests covering the main learning objectives measured the students’ knowledge of the topic to be presented in that session, and the same questions were asked again at the end of the session in the post-test. A handout was also distributed with each lecture to summarize the essential material addressed in that lecture. For each session, a different student ambassador was in charge of arranging contact with the panelists in order to develop questions for the tests and information for the handout. Each participant was given an individual anonymous code, which was used to pair their pre-test and post-test scores. A paired two-sample t-test was used to compare the means between pre-test and post-test scores of each session where P < .05 was considered to be significant.

Course final evaluation survey

A course assessment survey was distributed to attendees using SurveyMonkey at the end of the course. It included 35 questions on student demographics, existing radiology education at their institution, perceived efficacy of the course in comparison to the existing radiology education, usefulness of the course for future practice, and overall enthusiasm about radiology. The survey also included questions about the relevance of session topics, each session’s rating, the usefulness of the post-session handouts, and instructor quality. All test and survey data were collected and graphed for analysis.

Results

Based on the president of Council of higher Education, there are a total of 110,331 medical students in Turkey: of which 92,159 studying in public universities and 18,152 studying in foundation universities (https://www.aa.com.tr/tr/egitim/yok-baskani-ozvar-turkiyede-110-bin-331-ogrenci-tip-fakultelerinde-okuyor/2458569). From 41 different universities, a total of 1,458 medical students registered for this online radiology course. Of those, 1,256 (86.16%) students were from 33 different state universities, and 202 (13.85%) students were from eight different private (non-profit foundations) universities. An average of 506 medical students completed both the pre-test and the post-test for each of the sessions. Mean pre-test and post-test scores were 5.10/10 (range, 3.52–6.68) and 7.58/10 (range, 5.21–8.53), respectively. For each session, a statistically significant (p < .001) increase in the mean score was observed in the post-test when compared to the pre-test. The mean increase in test scores was 46.25% (range, 26.10-83.83%) (Table 1).

The number of students who completed the end-of-course survey was 439 (30.1%). The characteristics of the participants are summarized in Table 2. Only 90 (20.50%) participants believed that there are enough online radiology courses for medical students. Of all participants, 358 (81.55%) and 340 (77.45%) found online training either extremely convenient or very convenient and extremely effective or very effective, respectively. The Zoom technology platform used during the program was found to be very friendly or extremely friendly by 410 (93.41%) students.

Table 2 Survey participants’ characteristics
Full size table

Of the 439 participants, 415 (94.53%) attended all the sessions. The number of sessions attended by the remaining 24 (5.47%) students varied between 3 and 8. Participants were asked to rate each of the nine online sessions on a scale of 1 to 4: poor {1}, fair {2}, good {3}, and excellent {4}. The weighted average of these scores varied from 3.19 to 3.65, with an average score of 3.46 points. More than 92% of the participants indicated that they strongly agree or agree that the instructor’s communication was clear and easy to understand.

Of the 439 participants, 313 (71.30%), 306 (69.70%), and 316 (71.98%) indicated that they strongly agree that the course would be useful in their clinical practice in the future, the course increased their understanding of radiology, and the course was a worthwhile experience. Also, a total of 224 (51.03%) and 172 (39.18%) participants indicated that they strongly agree and somewhat agree, respectively, that introductory principles and concepts should be presented at the beginning of the undergraduate radiology curriculum. Figure 1 summarizes responses to the eight survey questions.

Fig. 1
figure 1

Course survey results

Full size image

Participants were also asked to rate their level of confidence before and after the course on a scale of 1–4: not confident at all {1}, somewhat confident {2}, moderately confident {3}, and very confident {4}. On a weighted average, the confidence level before the course was 1.83 (range,1.68–1.93), while the level of confidence after the course was 3.09 (range, 2.95–3.29). The change in the calculated average level of confidence demonstrates a 68% increase after the course (Table 3). Almost all participants (99.32%) agreed that the program either met or exceeded their expectations. Two hundred and forty-one (54.90%) of the students indicated that the course increased their interest in radiology, and 248 (56.49%) were considering radiology as a specialty.

Table 3 Level of confidence before and after the course
Full size table

There was no statistically significant correlation between the attended sessions and year of studies in medicine or overall satisfaction with the course (p > .005).

Of 439 participants, 136 (31%) provided suggestions on ways to improve the program’s learning experience in the free text option.

Discussion

Despite the fact that imaging is increasingly employed in everyday clinical practice, medical education in radiology continues to be undervalued in curricula across institutions worldwide [14]. Traditionally, radiology rotations have been based on passive learning through didactic lectures and empirical experience attained by accompanying radiologists in the reading room [15]. Increased workload, interruptions, complicated clinical situations, and the rotation schedules of attending radiologists, which often restrict the possibility of supervised learning, are all common obstacles to this sort of learning [15,16,17]. General learners frequently prefer a radiology curriculum that is well-integrated and includes small-group sessions and case-based learning [18, 19]. The ongoing COVID-19 pandemic, as well as the educational demands of the medical student audience, necessitate a shift in traditional teaching methods. Luckily, modern learners have access to a broad range of internet resources that enable them to work from home and learn from esteemed physicians throughout the nation while adhering to social distancing standards. The digital world is a continuously developing platform that can engage students in a variety of ways, including but not limited to 3D virtual environments [20], web-based educational tools [21], and social networking platforms that allow for mass communication [22].

Our radiology concepts course was designed to meet the needs of medical students and expand their access to online education in order to take a step towards universalizing radiology knowledge. Webinars are becoming more popular because of their convenience and cost-effectiveness, and they seem to be on their way to becoming one of the most essential instructional tools of the future [23]. These online learning technologies, in contrast to traditional didactic environments, provide for greater ease of access, expanded networking options for students and radiologists, and broader transmission of education. We can raise interest in radiology as a discipline and as a prospective profession for medical students across the world by providing access to instructional tools and networking opportunities with educators worldwide.

In our Principles of Radiology course, to promote audience participation, we used Zoom’s screen share, live polling, and chat box features. Students were able to contribute to the discussions during our lectures and ask questions via live chat. At the conclusion of the webinars, our moderators monitored and transmitted the questions to the speaker(s). This allowed for more engaging lectures and created a highly welcoming environment for our audience. Furthermore, the webinar lectures were recorded and published so that our students had the option to watch them at their convenience.

Overall, we had a higher-than-expected level of participation in our online course, including 1,458 registrants, 506 attendants, and 439 survey respondents from 41 different universities across Turkey. Almost 80% of the respondents said they believed that there were not enough online radiology courses for medical students at their current institutions; 81.55% and 77.45% found online training either extremely or very convenient and extremely or very effective, respectively. The technology platforms used were found to be very or extremely friendly by 93.42% of the respondents. Finally, around 80% found the online course very or extremely convenient and effective. Above 90% percent of respondents said they strongly agree or somewhat agree that introductory principles and concepts in radiology should be taught earlier in medical education. These findings highlight the underrepresentation of radiology education in medical school curricula and the need for new virtual learning platforms in radiology for medical students.

The demographic data generated from our polled participants revealed that 56.49% were already considering radiology as a career option after medical school. However, the student sample was likely heavily biased toward individuals who were already committed to pursuing a career in radiology or rather satisfied with the offer.

The post-test results showed a substantial (46.25%; p < .001) improvement over the pre-test scores, indicating that the webinar content improved their grasp of the topics. The program received an overall positive response, with around 90% or more of responders saying that they either somewhat agree or strongly agree that it improved their comprehension of radiology, piqued their interest in radiology, and will be valuable in their clinical practice in the future.

In general, respondents very strongly agree that the training was valuable, and almost all respondents (99.32%) agreed that it met or surpassed their expectations. The curriculum boosted respondents’ confidence in their understanding of the ideas underpinning imaging modalities, indications for imaging modalities, radiology’s history, and evolution, and how to interpret an imaging study. These data indicate that this virtual program and others like it can have a strongly favorable influence on medical student education worldwide.

To encourage even more audience involvement, the remote-control option on Zoom, which can mirror the radiologist’s experience of evaluating cases with a consulting physician, might be utilized in the future. While this adjustment has been shown to benefit resident learners [24], it has the potential to benefit medical student learners as well. This could be a very efficient way of teaching small groups, although it might not be as convenient in large groups of learners. Medical students have previously shown satisfaction with learning using virtual workstations, which simulate the environment of working radiologists [14, 17]. Several students sought a more thorough comprehension of core topics prior to the webinar, such as a review of suggested books before each session. This structure would be similar to the flipped classroom concept, which allows students to examine content at their own leisure before applying it during the webinar lectures [16]. Thus, it increases audience engagement and improves the higher-order cognitive grasp of the topic by shifting the webinar delivery schedule to encourage active learning.

Even if the course is designed as an alternative addendum, each student was asked to share her opinions about the consistency and benefits of this course relative to the already ongoing curriculum.

There are some limitations of our study. First, only a small proportion of the medical students in Turkey were registered and approximately 1/3 of them took part in both proficiency tests. Second, only %30 of the students filled out the final questionnaire. Third, there was no control group. Finally, answers to the post-test questions were gathered immediately after the lessons. Thus, this could create a short-term memory false positivity even if we wanted to overcome it by assessing multiple different learning objectives in each session.

Conclusion

While webinars have been around as an instructional tool for a long time, they are currently one of the few ways to get medical education in the face of the COVID-19 pandemic, and they have the potential to determine the future of medical education. In the long run, it is unknown how these educational approaches will alter medical student behavior and attitudes regarding online learning versus traditional in-person didactics. Regardless, over the course of 3 months, we could already see the benefits of the webinars in improving radiological learning methods. In order to truly understand the effectiveness of these methods in the scope of future radiology education, it is crucial to assess medical students’ feedback and improvements in radiology knowledge gained through virtual education in the post-COVID age.