The present study was performed among 452 first-year students studying Medicine (n = 351) and Biomedical Sciences (n = 101) at the Radboud University Nijmegen, the Netherlands, in the 2011–2012 academic year. Examination scores and responses to questionnaires were linked to each student’s identification number in order not to disclose the identity of the student. Students were informed about the study and their consent was obtained. Ethical approval was waived for this study. Nevertheless, the ethical principles of the Declaration of Helsinki were taken into account during the study design, data collection and data analysis phases.
Medicine and biomedical science study programmes
Interested students were eligible to apply for admission to the Medicine or Biomedical Science study programmes if they had obtained a diploma in pre-university education (e.g. VWO, Athenaeum or Gymnasium) with courses in biology, chemistry, physics and mathematics. Due to a numerus fixus, only a limited number of students can be accepted annually. Eligible students were accepted into the Biomedical Science programme using a lottery system based on average high school grades, with higher grades indicative of a greater chance for acceptance to the programme. For the Medicine programme, 50 % of the available positions were allocated using a similar lottery system. The other 50 % were allocated via a selection procedure in which eligible students completed a matriculation exam. Subsequently, examination scores were ranked and a top–down procedure was followed to allocate the remaining 50 % of available positions . Both study programmes include a 3-year Bachelor’s phase (i.e. undergraduate), followed by a 2- or 3-year Master’s phase (i.e. graduate) for Biomedical Sciences and Medicine, respectively. During the first year of both study programmes, the majority of the courses (60 %) are taken together.
Scores were collected during three different courses in the propaedeutic year. In chronological order, students first completed an EMQ examination (course: Principles of functional morphology), followed by an MCQ examination (course: Biochemical and physical processes), and finally a mixed EMQ and MCQ examination (course: Circulation and respiration). Grades can vary between 0 (lowest score) and 10 (highest score). Students pass a course if they obtain a grade ≥6. Students who obtain a score ≥8 are considered to be excellent.
All students were asked to complete a structured questionnaire related to sex, age, and preferences for examination format. Students were also requested to sign informed consent for participation in this study. All forms were checked for completeness by the observers who were present during the examination. The final grades per course were obtained from the Department for Evaluation, Quality and Development of Medical Education of the Radboud University Nijmegen Medical Centre. For the mixed examination, the overall score as well as the EMQ and MCQ sub-score were included for further analysis.
Multiple choice questions
Multiple-choice evaluation required students to choose the correct answer from a short list of possible answers: 3–5 alternatives that were preselected by the examiner . Correction for guessing was applied to prevent random guessing by the students and thereby obtaining higher grades .
Extended matching questions
EMQs are problem-focused questions often referring to realistic cases . They have four components: (i) a theme, (ii) a lead-in statement for the questions giving the students instructions on what to do, (iii) the questions giving students pertinent information based on which the student is to select the correct answer and (iv) a list of options or answer possibilities. In the EMQs, students were asked to select the best answer from a list of 9–26 options that were preselected by the examiner, each of which could be used once, more than once, or not at all.
The mixed examination combined EMQ and MCQ questions to test the knowledge of the students. The characteristics of both types of questions were in agreement with the EMQ only and MCQ only examinations, as described above.
Statistical analyses were performed using the Statistical Package for the Social Sciences (IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY: IBM Corp.). Quantitative data were summarized by mean and standard deviation (SD); categorical variables were presented by percentage. The difference in examination score between the MCQ, EMQ, and mixed examination was assessed using one-way analysis of variance (ANOVA). Subsequently, the average examination score over the three tests was calculated, and two new categorical variables were created. First a dummy variable was introduced to distinguish ‘poor’ (average examination score <6) from ‘moderate’ students (average examination score ≥6). The second dummy variable distinguished between ‘excellent’ and ‘other’ students (average examination score ≥8 versus <8, respectively). Using binary logistic regression analysis, we were able to determine the discriminative capacity of MCQ, EMQ and mixed examinations to detect poor (score <6) or excellent (score ≥8) students. The MCQ examination was used as the reference format in both analyses. Risk ratios (RR) were presented with their 95 % confidence intervals (CI). Finally, we created another new variable to compare the characteristics between poor, moderate and excellent students (average examination score <6, 6–8, or ≥8, respectively). Differences between the three groups of students were assessed using one-way ANOVA (continuous parameters) or Pearson’s χ
2 tests (nominal parameters). Significance was declared if p ≤ 0.05.