Skip to main content
SpringerLink
Log in

A Comparative Study on Question-Worthy Sentence Selection Strategies for Educational Question Generation

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11625))

Included in the following conference series:

Abstract

Automatic question generation, which aims at converting sentences in an article to high-quality questions, is an important task for educational practices. Recent work mainly focuses on designing effective generation architectures based on deep neural networks. However, the first and possibly the foremost step of automatic question generation has largely been ignored, i.e., identifying sentences carrying important information or knowledge that is worth asking questions about. In this work, we (i) propose a total of 9 strategies, which are grounded on heuristic question-asking assumptions, to determine sentences that are question-worthy, and (ii) compare their performance on 4 datasets by using the identified sentences as input for a well-trained question generator. Through extensive experiments, we show that (i) LexRank, a stochastic graph-based method for selecting important sentences from articles, gives robust performance across all datasets, (ii) questions collected in educational settings feature a more diverse set of source sentences than those obtained in non-educational settings, and (iii) more research efforts are needed to further improve the design of educational question generation architectures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adamson, D., Bhartiya, D., Gujral, B., Kedia, R., Singh, A., Rosé, C.P.: Automatically generating discussion questions. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 81–90. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_9

    Chapter  Google Scholar 

  2. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

  3. Bahrick, H.P., Bahrick, L.E., Bahrick, A.S., Bahrick, P.E.: Maintenance of foreign language vocabulary and the spacing effect. Psychol. Sci. 4(5), 316–321 (1993). https://doi.org/10.1111/j.1467-9280.1993.tb00571.x

    Article  Google Scholar 

  4. Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR, pp. 335–336 (1998)

    Google Scholar 

  5. Chen, G., Yang, J., Hauff, C., Houben, G.J.: LearningQ: a large-scale dataset for educational question generation. In: ICWSM (2018)

    Google Scholar 

  6. Collins-Thompson, K.: Computational assessment of text readability: a survey of current and future research. ITL-Int. J. Appl. Linguist. 165(2), 97–135 (2014)

    Article  Google Scholar 

  7. Denkowski, M., Lavie, A.: Meteor universal: language specific translation evaluation for any target language. In: SMT (2014)

    Google Scholar 

  8. Du, X., Cardie, C.: Identifying where to focus in reading comprehension for neural question generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2067–2073 (2017)

    Google Scholar 

  9. Du, X., Cardie, C.: Harvesting paragraph-level question-answer pairs from Wikipedia. In: Association for Computational Linguistics (ACL) (2018)

    Google Scholar 

  10. Du, X., Shao, J., Cardie, C.: Learning to ask: neural question generation for reading comprehension. In: ACL (2017)

    Google Scholar 

  11. Duan, N., Tang, D., Chen, P., Zhou, M.: Question generation for question answering. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 866–874 (2017)

    Google Scholar 

  12. Dunn, M., Sagun, L., Higgins, M., Guney, V.U., Cirik, V., Cho, K.: SearchQA: a new Q&A dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179 (2017)

  13. Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)

    Article  Google Scholar 

  14. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  15. Heilman, M., Smith, N.A.: Good question! statistical ranking for question generation. In: HLT-NAACL (2010)

    Google Scholar 

  16. Hermann, K.M., et al.: Teaching machines to read and comprehend. In: NIPS, pp. 1693–1701 (2015)

    Google Scholar 

  17. Hill, F., Bordes, A., Chopra, S., Weston, J.: The goldilocks principle: reading children’s books with explicit memory representations. CoRR abs/1511.02301 (2015)

    Google Scholar 

  18. Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J., et al.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies (2001)

    Google Scholar 

  19. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  20. Joshi, M., Choi, E., Weld, D.S., Zettlemoyer, L.: TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. In: ACL, July 2017

    Google Scholar 

  21. Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: RACE: large-scale reading comprehension dataset from examinations. In: EMNLP (2017)

    Google Scholar 

  22. Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. In: ACL (2004)

    Google Scholar 

  23. Mitkov, R., Ha, L.A.: Computer-aided generation of multiple-choice tests. In: HLT-NAACL (2003)

    Google Scholar 

  24. Nguyen, T., et al.: MS MARCO: a human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016)

  25. Onishi, T., Wang, H., Bansal, M., Gimpel, K., McAllester, D.A.: Who did what: a large-scale person-centered cloze dataset. In: EMNLP (2016)

    Google Scholar 

  26. Ouyang, Y., Li, W., Lu, Q., Zhang, R.: A study on position information in document summarization. In: COLING. pp. 919–927 (2010)

    Google Scholar 

  27. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: ACL (2002)

    Google Scholar 

  28. Prince, M.: Does active learning work? a review of the research. J. Eng. Educ. 93(3), 223–231 (2004)

    Article  Google Scholar 

  29. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100, 000+ questions for machine comprehension of text. In: EMNLP (2016)

    Google Scholar 

  30. Rayner, K., Foorman, B.R., Perfetti, C.A., Pesetsky, D., Seidenberg, M.S.: How psychological science informs the teaching of reading. Psychol. Sci. Public Interes. 2(2), 31–74 (2001)

    Article  Google Scholar 

  31. Richardson, M., Burges, C.J., Renshaw, E.: MCTest: a challenge dataset for the open-domain machine comprehension of text. In: EMNLP, pp. 193–203 (2013)

    Google Scholar 

  32. Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, vol. 135. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  33. Tang, D., Duan, N., Qin, T., Yan, Z., Zhou, M.: Question answering and question generation as dual tasks. arXiv preprint arXiv:1706.02027 (2017)

  34. Tsai, F.S., Tang, W., Chan, K.L.: Evaluation of novelty metrics for sentence-level novelty mining. Inf. Sci. 180(12), 2359–2374 (2010)

    Article  Google Scholar 

  35. Wang, T., Yuan, X., Trischler, A.: A joint model for question answering and question generation. arXiv preprint arXiv:1706.01450 (2017)

  36. Yang, Y., Yih, W.t., Meek, C.: WikiQA: a challenge dataset for open-domain question answering. In: EMNLP, pp. 2013–2018 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Monash University, Melbourne, Australia

    Guanliang Chen & Dragan Gasevic

  2. University of Fribourg, Fribourg, Switzerland

    Jie Yang

Authors
  1. Guanliang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dragan Gasevic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanliang Chen .

Editor information

Editors and Affiliations

  1. University of Sao Paulo, Sao Paulo, Brazil

    Seiji Isotani

  2. University of Malaga, Málaga, Spain

    Eva Millán

  3. Carnegie Mellon University, Pittsburgh, PA, USA

    Amy Ogan

  4. DePaul University, Chicago, IL, USA

    Peter Hastings

  5. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce McLaren

  6. University College London, London, UK

    Rose Luckin

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, G., Yang, J., Gasevic, D. (2019). A Comparative Study on Question-Worthy Sentence Selection Strategies for Educational Question Generation. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds) Artificial Intelligence in Education. AIED 2019. Lecture Notes in Computer Science(), vol 11625. Springer, Cham. https://doi.org/10.1007/978-3-030-23204-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-23204-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-23203-0

  • Online ISBN: 978-3-030-23204-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation