Skip to main content
SpringerLink
Log in

Residual Connection-Based Multi-step Reasoning via Commonsense Knowledge for Multiple Choice Machine Reading Comprehension

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11955))

Included in the following conference series:

Abstract

Generally, the candidate options for multiple choice machine reading comprehension (MRC) are not explicitly present in the document and need to be inferred from text or even from the world’s knowledge. Previous work endeavored to improve performance with the aid of commonsense knowledge or using multi-step reasoning strategy. However, there is no model adopt multi-step reasoning with external commonsense knowledge information to solve multiple choice MRC, and two shortcomings still remain unsolved, i.e., external knowledge may involve undesirable noise and only the latest reasoning step makes contribution to the next reasoning. To address the above issues, we propose a multi-step reasoning neural network based on the strong Co-Matching model with the aid of commonsense knowledge. Firstly, we present a sentence-level knowledge interaction (SKI) module to integrate commonsense knowledge with corresponding sentence rather than the whole MRC instance. Secondly, we present a residual connection-based multi-step reasoning (RCMR) answer module, which makes the next reasoning depending on the integration of several early reasoning steps rather than only the latest reasoning step. The comparative experimental results on MCScript show that our single model achieves a promising result comparable to SOTA single model with extra samples and specifically achieves the best result for commonsense type questions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.conceptnet.io/.

  2. 2.

    http://www.openmind.org/commonsese.

  3. 3.

    https://www.cyc.com/opencyc/.

  4. 4.

    https://wiki.dbpedia.org/.

  5. 5.

    http://www.edrdg.org/jmdict/.

  6. 6.

    https://www.wiktionary.org.

  7. 7.

    https://stanfordnlp.github.io/CoreNLP/.

  8. 8.

    https://github.com/pytorch/pytorch.

  9. 9.

    We use the code on https://github.com/shuohangwang/comatch to implement Co-Matching model. But one difference is that we add binary features to the word embedding.

  10. 10.

    The ensemble results of the HMA and the TriAN were 84.13% and 83.95% in SemEval-2018 Task 11, respectively. Table 2 shows the results of their single models.

  11. 11.

    The “commonsense” type questions accuracy of MITRE is reported in reference [27].

  12. 12.

    We use the code on https://github.com/intfloat/commonsense-rc to implement TriAN model.

References

  1. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: EMNLP, pp. 2383–2392 (2016)

    Google Scholar 

  2. Hermann, K.M., et al.: Teaching machines to read and comprehend. In: NIPS, pp. 1693–1701 (2015)

    Google Scholar 

  3. Ostermann, S., Modi, A., Roth, M., Thater, S., Pinkal, M.: MCScript: a novel dataset for assessing machine comprehension using script knowledge. In: LREC (2018)

    Google Scholar 

  4. Zhu, H., Wei, F., Qin, B., Liu, T.: Hierarchical attention flow for multiple-choice reading comprehension. In: AAAI (2018)

    Google Scholar 

  5. Parikh, S., Sai, A.B., Nema, P., Khapra, M.M.: ElimiNet: a model for eliminating options for reading comprehension with multiple choice questions. In: IJCAI, pp. 4272–4278 (2018)

    Google Scholar 

  6. Wang, S., Yu, M., Jiang, J., Chang, S.: A co-matching model for multi-choice reading comprehension. In: ACL, pp. 746–751 (2018)

    Google Scholar 

  7. Shen, Y., Huang, P.-S., Gao, J., Chen, W.: ReasoNet: learning to stop reading in machine comprehension. In: SIGKDD, pp. 1047–1055 (2017)

    Google Scholar 

  8. Tang, M., Cai, J., Zhuo, H.H.: Multi-matching network for multiple choice reading comprehension. In: AAAI (2019)

    Google Scholar 

  9. Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: an open multilingual graph of general knowledge. In: AAAI, pp. 4444–4451 (2017)

    Google Scholar 

  10. Wang, L., Sun, M., Zhao, W., Shen, K., Liu, J.: Yuanfudao at SemEval-2018 task 11: three-way attention and relational knowledge for commonsense machine comprehension. In: SemEval, pp. 758–762 (2018)

    Google Scholar 

  11. Chen, W., Quan, X., Chen, C.: Gated convolutional networks for commonsense machine comprehension. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11301, pp. 297–306. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04167-0_27

    Chapter  Google Scholar 

  12. Mihaylov, T., Frank, A.: Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge. In: ACL, pp. 821–832 (2018)

    Google Scholar 

  13. Xu, Y., Liu, J., Gao, J., Shen, Y., Liu, X.: Dynamic fusion networks for machine reading comprehension. arXiv preprint arXiv:1711.04964 (2017)

  14. Liu, X., Shen, Y., Duh, K., Gao, J.: Stochastic answer networks for machine reading comprehension. In: ACL, pp. 1694–1704 (2018)

    Google Scholar 

  15. Dhingra, B., Liu, H., Yang, Z., Cohen, W., Salakhutdinov, R.: Gated-attention readers for text comprehension. In: ACL, pp. 1832–1846 (2017)

    Google Scholar 

  16. Bond, F., Foster, R.: Linking and extending an open multilingual wordNet. In: ACL, pp. 1352–1362 (2013)

    Google Scholar 

  17. Kuo, Y.-L., et al.: Community-based game design: experiments on social games for commonsense data collection. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 15–22. ACM (2009)

    Google Scholar 

  18. Nakahara, K., Yamada, S.: Development and evaluation of a web-based game for common-sense knowledge acquisition in Japan. Unisys Technol. Rev. 30(4), 295–305 (2011)

    Google Scholar 

  19. Von Ahn, L., Kedia, M., Blum, M.: Verbosity: a game for collecting common-sense facts. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 75–78. ACM (2006)

    Google Scholar 

  20. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP, pp. 1532–1543 (2014)

    Google Scholar 

  21. Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: EMNLP, pp. 1422–1432 (2015)

    Google Scholar 

  22. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  23. Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: ACL, pp. 1556–1566 (2015)

    Google Scholar 

  24. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  25. Chen, Z., Cui, Y., Ma, W., Wang, S., Liu, T., Hu, G.: HFL-RC system at SemEval-2018 task 11: hybrid multi-aspects model for commonsense reading comprehension. arXiv preprint arXiv:1803.05655 (2018)

  26. Merkhofer, E., Henderson, J., Bloom, D., Strickhart, L., Zarrella, G.: MITRE at SemEval-2018 task 11: commonsense reasoning without commonsense knowledge. In: SemEval, pp. 1078–1082 (2018)

    Google Scholar 

  27. Ostermann, S., Roth, M., Modi, A., Thater, S., Pinkal, M.: SemEval-2018 task 11: Machine comprehension using commonsense knowledge. In: SemEval, pp. 747–757 (2018)

    Google Scholar 

  28. Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: Race: large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683 (2017)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, East China Normal University, Shanghai, 200062, People’s Republic of China

    Yixuan Sheng & Man Lan

Authors
  1. Yixuan Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Man Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Man Lan .

Editor information

Editors and Affiliations

  1. Australian National University, Canberra, ACT, Australia

    Tom Gedeon

  2. Murdoch University, Murdoch, WA, Australia

    Kok Wai Wong

  3. Kyungpook National University, Daegu, Korea (Republic of)

    Minho Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sheng, Y., Lan, M. (2019). Residual Connection-Based Multi-step Reasoning via Commonsense Knowledge for Multiple Choice Machine Reading Comprehension. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11955. Springer, Cham. https://doi.org/10.1007/978-3-030-36718-3_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36718-3_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36717-6

  • Online ISBN: 978-3-030-36718-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation