Abstract
Question and answering system and text generation using the BERT and GPT-2 transformer is a specialized field of the information retrieval system, which a query is stated to system and relocates the correct or closet answer to a specific query asked by the natural language. The main aim of the QA system is to provide the short answer to a question rather than the list of possible relevant document and text generation is a language generation which focuses on the producing understandable test in English which can predict the next sentence or generate a text with all the raw content from previous words. The motivation for selecting this work is to provide a great relevance to find the answer, find answer to general knowledge type of question, find the answers for questions like Who? What? Where? How?, and Provide provide the shortest form of answer. The scope for the chosen work is to provide the solutions for the automation moderation in the websites to provide the exact and short information answers from the websites, like Stack Overflow, Reddit, Quora, provide the self-answering and find text. The method we are using for the QA and text generation system is a transformer architecture which consist of Encoder and Decoder which is a stack of encoder represents the BERT model and Decoder part is represented as the GPT-2 model.
Please note that the LNCS Editorial assumes that all authors have used the western naming convention, with given names preceding surnames. This determines the structure of the names in the running heads and the author index.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Annamoradnejad I, Fazli M, Habibi J (2020) Predicting subjective features from questions on QA websites using BERT. In: 2020 6th international conference on web research (ICWR)
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805
Rajpurkar Z et al (2016) Lopyrev. Squad: 100,000+ questions for machine comprehension of text. arXiv:1606.05250
Radford A, Wu J, Child R, Luan D, Amadei D, Sutskever I (2019) Language models are unsupervised multitask learners. Technical report, OpenAI
Arustselvan M, Chellatamilan T, Vijaya Kumar M (2013) A statistical language modelling approach for question answering system. Int J Sci Res Publ 3(5). 1 ISSN 2250-3153
Medved’ M, Horák A (2018) Sentence and word embedding employed in open question-answering. natural language processing centre, Faculty of Informatics, Masaryk University, Botanická 68a, 602 00, Brno, Czech Republic
Pennington J, Socher R, Manning CD (2014) GloVe: global vectors for word representation. Computer Science Department, Stanford University, Stanford, CA 94305
Wang S, Jiang J (2017) Machine comprehension using match-LSTM and answer pointer. Singapore Management University. Conference paper at ICLR, School of Information Systems
Yu L, Zhang W, Wang J, Yu Y SeqGAN: sequence generative adversarial nets with policy gradient. Shanghai Jiao Tong University, University College London. arXiv:1609.05473v6[cs.LG]. Accessed on 25 Aug 2017
Question Answering system framework: https://towardsdatascience.com/building-an-application-of-question-answering-system-from-scratch-2dfc53f760aa
Mathur A, Haider MTU (2015) Question answering system: a survey. In: 2015 international conference on smart technologies and management for computing communication controls energy and materials (ICSTM)
Vaswani A, Brain G, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A, Kaiser Ł, Polosukhin Attention is all you need. https://arxiv.org/pdf/1706.03762v5 [cs.CL] 6 Dec 2017
Schwager S, Solitario J Question and answering on SQuAD 2.0: BERT is all you need. https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1194/reports/default/15812785.pdf
GPT-2 Architecture: http://humanssingularity.com/gpt2sampling/
Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding with unsupervised learning. Technical report, OpenAI
GPT2 training pipeline: http://jalammar.github.io/illustrated-gpt2/
Sanh V, Debut L, Chaumond J, Wolf T DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. https://arxiv.org/pdf/1910.01108.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kumari, S., Pushphavati, T.P. (2023). Question Answering and Text Generation Using BERT and GPT-2 Model. In: Asari, V.K., Singh, V., Rajasekaran, R., Patel, R.B. (eds) Computational Methods and Data Engineering. Lecture Notes on Data Engineering and Communications Technologies, vol 139. Springer, Singapore. https://doi.org/10.1007/978-981-19-3015-7_8
Download citation
DOI: https://doi.org/10.1007/978-981-19-3015-7_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-3014-0
Online ISBN: 978-981-19-3015-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)