Abstract
In natural language, the similarity between the two texts is judged by their similarity score. Some of the recent NLP application such as text summarization, question answering, text generation, and text mining are depended on the machine provided text. Accuracy of response text is measured by the similar with corresponding text or human given text. Comparing by two texts and measuring the similarity defines that the two texts are lexical or semantically similar. If two texts are related to each other with the word or character, this text is lexically similar. Also, if the texts are related in meaning but not in word or character level that are semantically similar. In this research, we measure the similarity of context and question for your question answering system. Then we find the most similar answer for the corresponding question. We used universal sentence encoder for embedding and measure the similarity using cosine distance of the text. We used deep averaging network for find the best similar text. For evaluation of similarity model, we calculate the Pearson correlation value for our dataset and achieve 0.41 coefficient.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Si, S., Zheng, W., Zhou, L., Zhang, M.: Sentence similarity computation in question answering robot. IOP Conf. Ser. J. Phys. Conf. Ser. 1237, 022093 (2019)
Song, W., Feng, M., Gu, N., Wenyin, L.: Question similarity calculation for FAQ answering. In: Third International Conference on Semantics, Knowledge and Grid. 0-7695-3007-9/07 $25.00 © 2007 IEEE. https://doi.org/10.1109/SKG.2007.32
Juan, Z.M.: An effective similarity measurement for FAQ question answering system. In: 2010 International Conference on Electrical and Control Engineering
Li, Y., McLean, D., Bandar, Z.A., O’Shea, J.D., Crockett, D.: Sentence similarity based on semantic nets and corpus statistics. IEEE Trans. Knowl. Data Eng. 18(8) (2006)
Jeon, J., Bruce Croft, W., Lee, J.H.: Finding semantically similar questions based on their answers (Copyright is held by the author/owner. SIGIR’05, August 15–19, 2005, Salvador, Brazil)
Mohler, M., Mihalcea, R.: Text-to-text Semantic Similarity for Automatic Short Answer Grading. In: Proceedings of the 12th Conference of the European Chapter of the ACL, pp. 567–575, Athens, Greece, 30 March–3 April 2009. c2009 Association for Computational Linguistics
Martinez, D., MacKinlay, A., Molla-Aliod, D., Cavedon, L., Verspoor, K.: Simple similarity-based question answering strategies for biomedical text
Masum, A.K.M., Abujar, S., Tusher, S.T.H., Faisal, F., Hossain, S.A.: Sentence similarity measurement for Bengali abstractive text summarization. In: 10th ICCCNT 2019 July 6–8, 2019, IIT—Kanpur, Kanpur, India
Achananuparp, P., Hu, X., Shen, X.: The Evaluation of Sentence Similarity Measures. In: Song, I.-Y., Eder, J., Nguyen, T.M. (eds.) DaWaK 2008, LNCS 5182, pp. 305–316, 2008. © Springer-Verlag Berlin Heidelberg (2008)
Bosma, W., Marsi, E., Krahmer, E., Theune, M.: Text-to-text generation for question answering. In: van den Bosch, A., Bouma, G. (eds.) Interactive Multi-modal Question-Answering, Theory and Applications of Natural Language Processing. © Springer-Verlag Berlin Heidelberg (2011). https://doi.org/10.1007/978-3-642-17525-1_6
Cera, D., Yanga, Y., Konga, S.-y., Huaa, N., Limtiacob, N., St. Johna, R., Constanta, N., Guajardo-Cespedesa, M., Yuanc, S., Tara, C., Sunga, Y.-S., Stropea, B., Kurzweila, R.: Universal Sentence Encoder. A Google Research Mountain View, CA. Mountain View, CA. b Google Research New York, NY. Google Cambridge, MA
Cera, D., Yanga, Y., Konga, S.-y., Huaa, N., Limtiacob, N., St. Johna, R., Constanta, N., Guajardo-Cespedesa, M., Yuanc, S., Tara, C., Sunga, Y.-S., Stropea, B., Kurzweila, R.: Universal Sentence Encoder for English. A Google Research Mountain View, CA. Mountain View, CA. b Google Research New York, NY. Google Cambridge, MA
Jotheeswaran, J., Loganathan, R., MadhuSudhanan, B.: Feature reduction using principal component analysis for opinion mining. Int. J. Comput. Sci. Telecommun. 3(5) (2012)
Iyyer, M., Manjunatha, M., Boyd-Graber, J., Daumé II, H.: Deep Unordered Composition Rivals Syntactic Methods for Text Classification
Acknowledgements
We gratefully acknowledge support from DIU NLP and Machine Learning Research LAB for providing GPUs support. We thank, Dept. of CSE, Daffodil International University for providing necessary supports. And also thanks to the anonymous reviewers for their valuable comments and feedback.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Keya, M., Masum, A.K.M., Abujar, S., Akter, S., Hossain, S.A. (2021). Bengali Context–Question Similarity Using Universal Sentence Encoder. In: Hassanien, A.E., Bhattacharyya, S., Chakrabati, S., Bhattacharya, A., Dutta, S. (eds) Emerging Technologies in Data Mining and Information Security. Advances in Intelligent Systems and Computing, vol 1300. Springer, Singapore. https://doi.org/10.1007/978-981-33-4367-2_30
Download citation
DOI: https://doi.org/10.1007/978-981-33-4367-2_30
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-33-4366-5
Online ISBN: 978-981-33-4367-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)