The search yielded 1479 articles of which 1102 articles remained after removing duplicates. A flowchart is provided in Fig. 1. Nineteen relevant articles were included for further analysis. Baseline characteristics of the included studies are shown in Table 2. Included studies were published between 2000 and 2013. Most studies were performed in the USA, Germany, or UK and included 26 to 687 students. Five studies used a control group of students. Imaging modalities used were CT (n = 14), MRI (n = 9), X-ray (n = 11), angiography (n = 5), and ultrasound (n = 6). Twelve studies (63 %) used a web-based e-learning program. The evaluated e-learning course was mandatory in 26 % (five studies).
Table 2 Characteristics of the included studies
Quality Assessment
The MERSQI score of the included articles is provided in Table 3. The Kirkpatrick level, which is included in the MERSQI score assessment, was 1 in 11 of the studies. This means that only the perception of students of the e-learning program was investigated. Seven studies also evaluated the effect of the e-learning program on knowledge and skills (Kirkpatrick level 2). The MERSQI score varied from 7 to 14.5 (out of 18). Most studies did not randomize groups (except for the studies of Mahnken et al. [12, 13] and Maleck et al. [13]), and none of the studies assessed the effect of e-learning on student behavior or patient outcomes.
Table 3 MERSQI score assessment. The maximum achievable score for each row is 18
Image Interaction Possibilities
In seven studies, the e-learning program offered students the possibility of viewing a stack of images [14–21]. Of which two studies used videos, i.e., scroll rated was fixed [15, 16], while in the other five studies, students could scroll through the images at their own pace [14, 17–21]. In one study, students were allowed to rotate 3D models of vessels derived from volumetric images [18]. Two studies [17, 19] additionally offered the possibility to adjust contrast settings.
Effect of Radiology E-learning on Learning Outcomes
The reported outcomes for each study are provided in Table 4. Eight studies (42 %) investigated the effect of radiology e-learning on learning outcomes, of which two studies concerned e-learning with image interaction possibilities. Learning effects were measured with pre- and post-course radiology skills tests, and all studies reported improved test results after the e-learning course. Three studies compared the test results with a control group, including one study using e-learning with image interaction possibilities [18], and found a higher improvement in knowledge in students exposed to e-learning compared to a control group [12, 13, 18].
Table 4 Reported outcomes for each study
The study of Petersson et al. [18] investigated the value of an additional 3D radiological anatomy e-learning tool for students taking vascular anatomy courses. Student test scores on 3D anatomical knowledge of peripheral vessels improved significantly after introduction of the additional e-learning tool compared to test scores of control groups from a year before who only took the traditional vascular anatomy courses. Radiological knowledge of neurovascular anatomy did not significantly improve after introducing the e-learning course.
Maleck et al. [13] randomized students in four groups: group 1 and 2 were presented with e-learning cases, respectively, with and without interactive elements; group 3 with paper-based cases with interactive elements, and a control group was not exposed to any cases. Interactive elements comprised questions guiding students through the cases and did not relate to image interaction in this case. All students had the opportunity to attend radiology lectures on a voluntary basis. Pre- and post-course tests with questions related to radiological knowledge and image interpretation skills were used. Both multiple choice questions and free-text questions were used related to radiographs projected with a slide projector. The authors found significant improvement in knowledge and X-ray image interpretation skills in both e-learning groups and in the paper-based group in contrast to the control group who showed no significant improvement. Most improvement in image interpretation skills was found in e-learning group 1, and most improvement in knowledge was found in e-learning group 2, though it was not reported if these differences were significant.
Mahnken et al. [12] provided e-learning with radiological cases and expert feedback and compared this to a group without access to the e-learning environment. Image interaction was not possible. All students followed an internship in radiology, so the control group was exposed to radiology as well but without access to the additional e-learning program. Learning effects were measured with a pre- and post-course radiology knowledge test based on the learning objectives of the internship and the e-learning content. Knowledge improvement in the e-learning group was higher, but this difference was not significant. None of the studies compared the learning effect of e-learning alone in comparison to traditional learning.
Students’ Perspectives on E-learning with Image Interaction Possibilities
Most studies reported that the possibility of image interaction was advantageous to students. For example, Ernst et al. [15] introduced a CD-ROM with stack viewing in their learning program, which was found to be a useful addition to hard copy images by 96 % of students. In the study of Arya et al. [14], an e-learning module with stack image viewing was compared to three other methods: a 3D anatomic model, a poster with tiled images, and an ultrasound station. The e-learning module was found to be most helpful in recall of anatomic principles and was perceived to be, together with the 3D anatomic model, more effective in improving the comprehension of 3D physical relationships than the other two methods. The e-learning station and the ultrasound station were thought to represent more clinically relevant material than the other two. Overall, the 3D model was found to be most valuable by students. According to a qualitative evaluation of an e-learning program using 2D images and videos of stacks, students particularly appreciate stacks of cross-sectional images for radiology learning [16]. In another study with scrollable images, 3 % of the participants suggested to add more image interaction possibilities, such as changing viewing direction [21]. One negative comment concerning image interaction was reported and concerned the lack of facilitators to assist in navigating through the scrollable images [17].
In the study of Bohl et al. [22], students did not have the opportunity to use image interaction. However, some students mentioned that it would be better to use actual software for practice which allows image interaction.
Students’ Perspectives on Radiology E-learning in General
Nearly all studies (89 %) investigated the perspectives of students on radiology e-learning in general. The quality of the images used in e-learning was rated high in six studies. Only one study with X-ray images reported that a few students (4 %) found that the image quality should be improved [23].
Seven studies described that students found that they could use the knowledge gained from the e-learning programs in clinical practice [14, 16, 17, 21–24]. On the other hand, students also experienced that the information in the e-learning course was sometimes too much, too detailed, or not relevant for the post-course assessment [14, 17, 22].
Seven studies addressed the question if e-learning could be an alternative to traditional lectures. In three studies, the e-learning program was preferred over traditional learning [13, 22, 25]. Two studies reported that students found e-learning alone not suitable but preferred e-learning as an addition to traditional lectures [16, 26]. Traditional lectures were preferred over e-learning by the majority of students in two studies [18, 27]. One of these studies mentioned that students missed the student–teacher interaction [27]. However, this study was essentially different from the other included studies as it investigated the use of e-learning in radiation protection education, and therefore, radiological images did not play a central role. The e-learning course existed of information hyperlinked to webpages and images from the Internet.
In 11 studies, students mentioned that e-learning improved their understanding of radiology and anatomy [13, 14, 19, 21–23, 26–30]. Specifically, they experienced improved recognition of pathology on radiological images [22–24]. Students also mentioned that they were more interested in radiology after the e-learning course [23, 27]. Students liked the possibility to study from home and found that e-learning made it possible to cover larger areas of knowledge in a short time frame [16, 23, 27, 28].
Teacher Perspectives
Advantages of e-learning that were experienced by authors were less administrative tasks and a decrease in costs [15, 16, 25]. Administrative tasks reduced because exams could be reviewed electronically [25]. Ernst et al. [15] experienced a decrease in costs because CD-ROMs containing both the syllabus and radiology images were used instead of a printed syllabus. In several studies, technical problems were experienced, making it temporarily impossible to access the e-learning program [20, 21, 23, 27, 28].