Abstract
Stance detection can be defined as the task of automatically detecting the relation between or the relative perspective of two pieces of text- a claim or headline and the corresponding article body. Stance detection is an integral part of the pipeline used for automatic fake news detection which is an open research problem in Natural Language Processing. The past year has seen a lot of developments in the field of NLP and the application of transfer learning to it. Bidirectional language models with recurrence and various transformer models have been consistently improving the state-of-the-art results on various NLP tasks. In this research work, we specifically focus on the application of embeddings from BERT and XLNet to solve the problem of stance detection. We extract the weights from the last hidden layer of the base models in both cases and use them as embeddings to train task-specific recurrent models. We also present a novel approach to tackle stance detection wherein we apply Temporal Convolutional Networks to solve the problem. Temporal Convolutional Networks are being seen as an ideal replacement for LSTM/GRUs for sequence modelling tasks. In this work, we implement models to investigate if they can be used for NLP tasks as well. We present our results with an exhaustive comparative analysis of multiple architectures trained on the Fake News Challenge (FNC) dataset.
Kushal Jain and Fenil Doshi have made equal contributions to the work and Lakshmi Kurup was our supervisor.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
S. Mohammad, S. Kiritchenko, P. Sobhani, X. Zhu, C. Cherry, SemEval-2016 Task 6: Detecting Stance in Tweets (2016). SemEval@NAACL-HLT
A. Zubiaga, G.W. Zubiaga, M. Liakata, R. Procter, P. Procter, Analysing how people orient to and spread rumours in social media by looking at conversational threads. PloS One (2016)
D. Pomerleau, D. Rao, Fake News Challenge (2017). http://www.fakenewschallenge.org/
W. Ferreira, A. Vlachos, Emergent: a novel data-set for stance classification (2016). HLT-NAACL
B. Riedel, I. Augenstein, G.P. Spithourakis, S. Riedel, A simple but tough-to-beat baseline for the fake news challenge stance detection task (2017). abs/1707.03264
G. Bhatt, A. Sharma, S. Sharma, A. Nagpal, B. Raman, A. Mittal, Combining Neural, Statistical and External Features for Fake News Stance Identification (2018). WWW
W. Largent, Talos Targets Disinformation with Fake News Challenge Victory. https://blog.talosintelligence.com/2017/06/talos-fake-news-challenge.html
A. Hanselowski, P.V. Avinesh, B. Schiller, F. Caspelherr, D. Chaudhuri, C.M. Meyer, I. Gurevych, A retrospective analysis of the fake news challenge stance-detection task (2018). COLING
J. Pennington, R. Socher, C.D. Manning, Glove: global vectors for word representation. EMNLP (2014)
T. Mikolov, K. Chen, G.S. Chen, J. Chen, Efficient Estimation of Word Representations in Vector Space (2013). abs/1301.3781
A.K. Chaudhry, Stance Detection for the Fake News Challenge: Identifying Textual Relationships with Deep Neural Nets
Q.Q. Zeng, Neural Stance Detectors for Fake News Challenge (2017)
C. Conforti, N. Collier, M.T. Pilehvar, Towards Automatic Fake News Detection: Cross-Level Stance Detection in News Articles (2019)
J. Devlin, M. Chang, K. Chang, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018). abs/1810.04805
M.E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L.S. Zettlemoyer, Deep contextualized word representations (2018). abs/1802.05365
Hugging Face pytorch-transformers. https://github.com/huggingface/pytorch-transformers
Z. Yang, Z. Dai, Y. Yang, J.G. Carbonell, R. Salakhutdinov, Q.V. Le, XLNet: Generalized Autoregressive Pretraining for Language Understanding (2019). abs/1906.08237
Z. Dai, Z. Yang, Y. Yang, J.G. Carbonell, Q.V. Le, R. Salakhutdinov, Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (2019). abs/1901.02860
I. Augenstein, T. Rocktäschel, A. Vlachos, K. Bontcheva, Stance detection with bidirectional conditional encoding. EMNLP (2016)
S. Bai, J.Z. Kolter, V. Koltun, An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018). abs/1803.01271
D. Paperno, G. Kruszewski, A. Lazaridou, Q.N. Pham, R. Bernardi, S. Pezzelle, M. Baroni, G. Boleda, R. Fernández, The LAMBADA dataset: word prediction requiring a broad discourse context (2016). abs/1606.06031
J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation. CVPR (2015)
A.F. Agarap, Deep Learning using Rectified Linear Units (ReLU) (2018). abs/1803.08375
E. Loper, S. Bird, NLTK: The Natural Language Toolkit (2002). cs.CL/0205028
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. VanderPlas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
F. Chollet, Keras, GitHub (2015). https://github.com/fchollet/keras
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in PyTorch (2017)
P. Rémy, Philipperemy/keras-tcn (2019). Retrieved from https://github.com/philipperemy/keras-tcn
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Jain, K., Doshi, F., Kurup, L. (2021). Stance Detection Using Transformer Architectures and Temporal Convolutional Networks. In: Bhatia, S.K., Tiwari, S., Ruidan, S., Trivedi, M.C., Mishra, K.K. (eds) Advances in Computer, Communication and Computational Sciences. Advances in Intelligent Systems and Computing, vol 1158. Springer, Singapore. https://doi.org/10.1007/978-981-15-4409-5_40
Download citation
DOI: https://doi.org/10.1007/978-981-15-4409-5_40
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-4408-8
Online ISBN: 978-981-15-4409-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)