Advances in Health Sciences Education

, Volume 17, Issue 3, pp 369–376

Faculty development on item writing substantially improves item quality

  • Naghma Naeem
  • Cees van der Vleuten
  • Eiad Abdelmohsen Alfaris
Article

DOI: 10.1007/s10459-011-9315-2

Cite this article as:
Naeem, N., van der Vleuten, C. & Alfaris, E.A. Adv in Health Sci Educ (2012) 17: 369. doi:10.1007/s10459-011-9315-2

Abstract

The quality of items written for in-house examinations in medical schools remains a cause of concern. Several faculty development programs are aimed at improving faculty’s item writing skills. The purpose of this study was to evaluate the effectiveness of a faculty development program in item development. An objective method was developed and used to assess improvement in faculty’s competence to develop high quality test items. This was a quasi experimental study with a pretest-midtest-posttest design. A convenience sample of 51 faculty members participated. Structured checklists were used to assess the quality of test items at each phase of the study. Group scores were analyzed using repeated measures analysis of variance. The results showed a significant increase in participants’ mean scores on Multiple Choice Questions, Short Answer Questions and Objective Structured Clinical Examination checklists from pretest to posttest (p < .0005). The effect sizes were 1.38, 3.84 and 2.20 for Multiple Choice Questions, Short Answer Questions and Objective Structured Clinical Examination, respectively. This study emphasizes that items written by faculty without faculty development are generally lacking in quality. It also provides evidence of the value of faculty development in improving the quality of items generated by faculty.

Keywords

Assessment Evaluation Faculty development Quality Test items 

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Naghma Naeem
    • 1
    • 2
  • Cees van der Vleuten
    • 1
  • Eiad Abdelmohsen Alfaris
    • 1
  1. 1.King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine (34), College of MedicineKing Saud UniversityRiyadhKingdom of Saudi Arabia
  2. 2.Department for Educational DevelopmentAga Khan UniversityKarachiPakistan