The participants (N = 120) were second-year medical students from a Finnish faculty of medicine attending a four-month (10-ECTS) undergraduate course in pathology that consisted of theoretical lectures, demonstrations in microscopy in a lecture hall, assistant-led demonstrations in microscopy, and participation in a set of autopsies and seminars. The demonstrations in microscopy took place during a six-week period at the beginning of the course and provided the context for the present study. The objective of the undergraduate course in pathology is to convey an understanding of how changes at tissue and cellular level impact health and disease. The microscopy instruction aims at teaching diagnostic classification based on microscopic observation. For the study of affective outcomes, there were 93 respondents. For the study of learning effectiveness, there were 54 participants.
These 54 students had participated in a pre-test (version A) and a post-test (version A). Pre-test version (B), which was equivalent to post-test version A, had a low reliability because it was too difficult for beginners, resulting in having to discard approximately half of the data. Therefore, the number of participants in the learning effectiveness study was much lower than anticipated.
Within approximately one week in the middle of the microscopy instruction period, half of the participants were asked to complete three sets of homework assignments pertaining to three different systems (of human organs) to be done in pairs. The other half of the students served as controls (no assignments). In order to assure equal opportunities for all, the controls received virtual microscopy assignments the following week. The design is presented in Figure 3. The rectangles indicate approximately one week of instruction in microscopic pathology. The purple rectangles refer to weeks of enriched instruction, whereas the white rectangles indicate weeks of instruction as usual. The arrows indicate the timing of the pre- and post-test. As mentioned earlier, there were two partially overlapping versions of the pre-test (A and B) and the post-test (A and B), but only version A proved reliable (figure 3).
The instructional intervention was based on two design principles: 1) More gradual shift from teacher-regulated instruction to self-study, 2) Elements of process-oriented instruction such as the assignment of tasks, feedback, and the use of process worksheets. It consisted of the following elements:
1. Self-paced practice in pairs on distinguishing critical abnormal features (use of teacher-prepared annotations to visualize zones of interest) and practice on making a diagnosis
2. Decision-tree to visualize the process of making a diagnosis
3. Collective feedback to students
The homework and the process worksheet were delivered by the WebMicroscope. The process worksheet, which was first introduced to the students during the pre-test, was in the form a decision-tree. The first test node in the decision-tree represented the decision regarding whether the case represented a case of neoplastic or reactive disease (see Appendix 1, Figure 4). After this critical decision, further steps were provided. A simplified version of the decision-tree was provided, so that students could mark intermediate decisions leading to the final diagnosis. A “yes” was to be marked by a cross in the circle next to each question. It is worth noting that since the undergraduate pathology curriculum covers approximately two hundred different diseases, not all of the exit nodes could be explicitly stated.
The homework tasks consisted of three parts following a simple-to-complex ordering strategy (i.e. isolated elements are presented first and only afterwards the whole with its interacting components ). In Part I, the students were presented with certain areas of the slide that were highlighted by a circle, an arrow, or another mark and asked to select the correct answer from a list of alternatives. In Part II, the students were once more presented with annotated areas, but this time asked to describe the findings. In Part III, the students were asked to suggest a diagnosis for a slide without annotations. Each assignment was to be completed before an assistant-led demonstration session. The assistant was expected to go over the assignments before commencing teaching.
After the intervention period, fully annotated digital versions of the example cases shown in the assistant-led demos were made available to the students.
Materials and procedures
The pre- and post-test materials consisted of 1) ten partially overlapping multiple-choice questions per student asking the student to identify a particular histological abnormality (i.e. a single feature), and 2) six questions asking the student to suggest a diagnosis. For the features test, one point was given for each correct answer. As for the diagnosis test, one point was given if the student had arrived at the correct general diagnosis (neoplastic or reactive). A pre-test composite index was formed of the average 11 items and a post-test score of the average of 11 items partially overlapping with the pre-test. Items included in the pre-test composite index are marked with an asterisk (*) and items in post-test with a hash (#) in Tables 1 and 2 in Appendix 2. The Kuder-Richardson coefficient of reliability (K-R 20) recommended for dichotomous variables was 0.64 for the pre-test and 0.61 for the post-test. According to Nunnally & Bernstein, the use of somewhat modest reliabilities can be justified in exploratory research . See also [15, 16].
Approximately one month later, data were gathered on student perceptions regarding the intervention vis-à-vis regular instruction. Students were asked to complete the Intrinsic Motivation Inventory (IMI: ) comparing the two instructional conditions on a seven-point Likert scale. Each item had two variants: the one referred to virtual microscopy assignments and the other to regular microscopy demonstrations. For reliabilities and sample items, please see Table 3 in Appendix 3. Students were also asked for written comments on the intervention.
Analysis of open comments
The open, written comments of the students were transcribed (two pages) and grouped under the following five rubrics: positive comments regarding the virtual microscopy experiment, negative comments regarding the experiment, feedback/suggestions, technical aspects, and feedback on the questionnaire.
The motivational items from the IMI were analyzed with paired-samples (t-tests) as each item had two variants.
The learning results were first analyzed descriptively by simply comparing the means and modes of the responses on the post-test with those of the pre-test. As the pre-test and the post-test contained a partially overlapping set of items, comparing pre-test and post-test scores on absolute terms would not have made sense. Instead, we examined a possible interaction effect using repeated-measures ANOVA. The repeated-measures within-subjects factor was time (pre-test composite index/post-test composite index) and between-subjects factor was group (experimental/control). As it was presumed that high achievers would be more likely to be able to make use of the scaffolds provided, a further analysis (ANCOVA) was conducted on high and low achievers separately. ANCOVA was used in order to determine if the experimental group would outperform the control group on the post-test when controlling for the level of perceived effort as the level of perceived effort differed in the two groups (experimental/control). There was no difference between the two groups in the performance on the pre-test.