An Experimental Evaluation of Automatically Generated Multiple Choice Questions from Ontologies

  • Ghader Kurdi
  • Bijan Parsia
  • Uli Sattler
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10161)


In order to provide support for the construction of MCQs, there have been recent efforts to generate MCQs with controlled difficulty from OWL ontologies. Preliminary evaluation suggests that automatically generated questions are not field ready yet and highlight the need for further evaluations. In this study, we have presented an extensive evaluation of automatically generated MCQs. We found that even questions that adhere to guidelines are subject to the clustering of distractors. Hence, the clustering of distractors must be realised as this could affect the prediction of difficulty.



The authors would like to thank Tahani Alsubait for sharing the MCQ generator code.

Supplementary material


  1. 1.
    Alsubait, T., Parsia, B., Sattler, U.: Generating multiple choice questions from ontologies: lessons learnt. In: OWLED, Chicago, pp. 73–84 (2014)Google Scholar
  2. 2.
    Alsubait, T., Parsia, B., Sattler, U.: Generating multiple choice questions from ontologies: how far can we go? In: Lambrix, P., Hyvönen, E., Blomqvist, E., Presutti, V., Qi, G., Sattler, U., Ding, Y., Ghidini, C. (eds.) EKAW 2014. LNCS (LNAI), vol. 8982, pp. 66–79. Springer, Cham (2015). doi: 10.1007/978-3-319-17966-7_7 Google Scholar
  3. 3.
    Haladyna, T.M., Downing, S.M., Rodriguez, M.C.: A review of multiple-choice item-writing guidelines for classroom assessment. Appl. Measur. Educ. 15(3), 309–333 (2002)CrossRefGoogle Scholar
  4. 4.
    Pho, V.-M., Andre, T., Ligozat, A.-L., Grau, B., Illouz, G., Francois, T., et al.: Multiple choice question corpus analysis for distractor characterization. In: LREC, pp. 4284–4291, Reykjavik (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Computer ScienceThe University of ManchesterManchesterUK

Personalised recommendations