Abstract
Background
In the USA, medical licensing exams have attempted to test the higher levels of Bloom’s taxonomy by using multiple choice questions (MCQ) that incorporate clinical or experimental vignettes.
Aim
We wished to determine if the nature of the vignette altered the difficulty and discriminative ability of our MCQ.
Methods
We assigned our regular course MCQ to categories. Recall questions had no context. Pseudo-vignette questions had a context not needed to answer the question. Vignette 1 or vignette 2 questions required extracting information from the vignette. Difficulty (DIFF), discrimination index (DI), and point biserial correlation coefficient (PtBiS) were calculated for 1,690 questions used during our preclerkship curriculum.
Results
Recall questions were more common during the first year. Pseudo-vignette questions maintained approximately the same frequency across the courses. Vignette 1 and vignette 2 questions increased gradually. During the first year, questions were less difficult but more discriminating than during the second year. In the first year, recall questions were less difficult and discriminating than vignette questions. These differences disappeared during the second year, when DIFF, DI, and PtBiS did not differ significantly among the question formats.
Conclusions
The differences between the results of testing in first and second years may reflect experience with vignette-type questions, true learning, or a combination of both.
Similar content being viewed by others
References
Breitbach AP. Creating effective multiple choice items. Athl Ther Today. 2010;15:18–22.
Case SM and Swanson DB. 2002. Constructing written test questions for the basic and clinical sciences. 3rd ed. Retrieved March 11, 2013. Available from: http://www.nbme.org/publications/item-writing-manual.html.
Case SM, Swanson DB, Becker DF. Verbosity, window dressing, and red herrings: do they make a better test item? Acad Med. 1996;71:S28–30.
Federation of State Medical Boards of the United States, Inc and the National Board of Medical Examiners. 2013 Step 1, Content Description and General Information. Retrieved October 15, 2013 Available from: http://www.usmle.org/step-1/.
Flateby TL. A guide for writing and improving achievement tests. Office of the Provost, University of South Florida, Tampa, FL Retrieved, June 26, 2014 Available from: http://www.acad.usf.edu/Office/IE/Resources/Guide.pdf.
Fuller S, Horlen C, Cisneros R, Merz T. Pharmacy students’ reading ability and the readability of required reading materials. Am J Pharm Educ. 2007;71:111.
Haladyna TM, Downing SM. A taxonomy of multiple-choice item-writing rules. Appl Meas Educ. 1989;2:37–50.
Haladyna TM, Downing SM. Validity of a taxonomy of multiple-choice item-writing rules. Appl Meas Educ. 1989;2:51–78.
Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ. 2002;15:309–34.
Haught PA, Walls RT. Relationships of reading, MCAT, and USMLE Step 1 test results for medical students. Read Psychol. 2004;25:83–92.
Jackson JR, Brooks CM. Relationship among the MCAT reading subtest, Nelson-Denny Reading test, and medical school achievement. J Med Educ. 1985;60:478–80.
Krathwohl DR. A revision of Bloom’s taxonomy: an overview. Theor Pract. 2002;41:212–8.
Nazian SJ, Stevenson FT. The use of horizontal basic science proficiencies in a systems based curriculum. Med Sci Educ. 2013;23:2–5.
Sim S-M, Rasiah RI. Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper. Ann Acad Med Singap. 2006;35:67–71.
Tarrant MT, Ware JW. A framework for improving the quality of multiple-choice assessments. Nurse Educ. 2012;37:98–104.
University of South Florida Morsani College of Medicine Program Objectives. Retrieved October 15, 2013 Available from: http://health.usf.edu/medicine/mdprogram/core/objectives.htm.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nazian, S.J., Stevenson, F.T. Difficulty and Discriminative Ability of Various Categories of Multiple Choice Questions in Medical School Preclerkship Examinations. Med.Sci.Educ. 24, 387–393 (2014). https://doi.org/10.1007/s40670-014-0069-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-014-0069-3