Abstract
The information on the internet suffers from noise and corrupt knowledge that may arise due to human and mechanical errors. To further exacerbate this problem, an ever-increasing amount of fake news on social media, or internet in general, has created another challenge to drawing correct information from the web. This huge sea of data makes it difficult for human fact checkers and journalists to assess all the information manually. In recent years Automated Fact-Checking has emerged as a branch of natural language processing devoted to achieving this feat. In this work, we give an overview of recent approaches, emphasizing on the key challenges faced during the development of such frameworks. We test existing solutions to perform claim classification on simple-claims and introduce a new model dubbed SimpleLSTM, which outperforms baselines by 11%, 10.2% and 18.7% on FEVER-Support, FEVER-Reject and 3-Class datasets respectively. The data, metadata and code are released as open-source and will be available at https://github.com/DeFacto/SimpleLSTM.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
GridSearchCV from scikit-learn to obtain the best hyper-parameters.
- 10.
- 11.
References
Akbik, A., Blythe, D., Vollgraf, R.: Contextual string embeddings for sequence labeling. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 1638–1649 (2018)
Baly, R., Karadzhov, G., Alexandrov, D., Glass, J.R., Nakov, P.: Predicting factuality of reporting and bias of news media sources. CoRR abs/1810.01765 (2018). http://arxiv.org/abs/1810.01765
Baly, R., Mohtarami, M., Glass, J.R., MÃ rquez, L., Moschitti, A., Nakov, P.: Integrating stance detection and fact checking in a unified corpus. CoRR abs/1804.08012 (2018). http://arxiv.org/abs/1804.08012
Chen, D., Fisch, A., Weston, J., Bordes, A.: Reading wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051 (2017)
Ciampaglia, G.L., Shiralkar, P., Rocha, L.M., Bollen, J., Menczer, F., Flammini, A.: Computational fact checking from knowledge networks. PLoS ONE 10(6), e0128193 (2015)
Conforti, C., Pilehvar, M.T., Collier, N.: Towards automatic fake news detection: cross-level stance detection in news articles. In: Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pp. 40–49 (2018)
Esteves, D., Reddy, A.J., Chawla, P., Lehmann, J.: Belittling the source: trustworthiness indicators to obfuscate fake news on the web. In: Proceedings of the First Workshop on Fact Extraction and Verification (FEVER), pp. 50–59. Association for Computational Linguistics (2018). http://aclweb.org/anthology/W18-5508
Fridkin, K., Kenney, P.J., Wintersieck, A.: Liar, liar, pants on fire: How fact-checking influences citizens’ reactions to negative advertising. Polit. Commun. 32(1), 127–151 (2015). https://doi.org/10.1080/10584609.2014.914613
Gardner, M., et al.: AllenNLP: a deep semantic natural language processing platform (2017)
Gatt, A., Krahmer, E.: Survey of the state of the art in natural language generation: core tasks, applications and evaluation. J. Artif. Intell. Res. 61, 65–170 (2018)
Gerber, D., et al.: Defacto–temporal and multilingual deep fact validation. Web Semant.: Sci. Serv. Agents World Wide Web 35, 85–101 (2015)
Graves, L., Cherubini, F.: The rise of fact-checking sites in Europe (2016)
Hanselowski, A., et al.: A retrospective analysis of the fake news challenge stance detection task. arXiv preprint arXiv:1806.05180 (2018)
Hassan, N., et al.: The quest to automate fact-checking. World (2015)
Himma-Kadakas, M., et al.: Alternative facts and fake news entering journalistic content production cycle. Cosmopolitan Civil Soc.: Interdisc. J. 9(2), 25 (2017)
Lee, N., Wu, C.S., Fung, P.: Improving large-scale fact-checking using decomposable attention models and lexical tagging. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1133–1138 (2018)
Li, S., Zhao, S., Cheng, B., Yang, H.: An end-to-end multi-task learning model for fact checking. In: Proceedings of the First Workshop on Fact Extraction and Verification (FEVER), pp. 138–144. Association for Computational Linguistics (2018). http://aclweb.org/anthology/W18-5523
Mihaylova, T., et al.: Fact checking in community forums. CoRR abs/1803.03178 (2018). http://arxiv.org/abs/1803.03178
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. CoRR abs/1310.4546 (2013). http://arxiv.org/abs/1310.4546
Palau, R.M., Moens, M.F.: Argumentation mining: the detection, classification and structure of arguments in text. In: Proceedings of the 12th International Conference on Artificial Intelligence and Law, pp. 98–107. ACM (2009)
Parikh, A.P., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. In: EMNLP (2016)
Peldszus, A., Stede, M.: From argument diagrams to argumentation mining in texts: a survey. Int. J. Cogn. Inform. Nat. Intell. (IJCINI) 7(1), 1–31 (2013)
Popat, K., Mukherjee, S., Strötgen, J., Weikum, G.: Where the truth lies: explaining the credibility of emerging claims on the web and social media. In: Proceedings of the 26th International Conference on World Wide Web Companion, pp. 1003–1012. International World Wide Web Conferences Steering Committee (2017)
Popat, K., Mukherjee, S., Yates, A., Weikum, G.: Declare: debunking fake news and false claims using evidence-aware deep learning. CoRR abs/1809.06416 (2018). http://arxiv.org/abs/1809.06416
Rashkin, H., Choi, E., Jang, J.Y., Volkova, S., Choi, Y.: Truth of varying shades: analyzing language in fake news and political fact-checking. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2931–2937. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/D17-1317. http://aclweb.org/anthology/D17-1317
Reddy, A.J., Rocha, G., Esteves, D.: DeFactoNLP: fact verification using entity recognition, TFIDF vector comparison and decomposable attention. CoRR abs/1809.00509 (2018)
Taniguchi, M., Taniguchi, T., Takahashi, T., Miura, Y., Ohkuma, T.: Integrating entity linking and evidence ranking for fact extraction and verification. In: Proceedings of the First Workshop on Fact Extraction and Verification (FEVER), pp. 124–126 (2018)
Thorne, J., Vlachos, A.: Automated fact checking: task formulations, methods and future directions. arXiv preprint arXiv:1806.07687 (2018)
Thorne, J., Vlachos, A., Christodoulopoulos, C., Mittal, A.: FEVER: a large-scale dataset for fact extraction and verification. CoRR abs/1803.05355 (2018). http://arxiv.org/abs/1803.05355
Thorne, J., Vlachos, A., Christodoulopoulos, C., Mittal, A.: Fever: a large-scale dataset for fact extraction and verification. arXiv preprint arXiv:1803.05355 (2018)
Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., Mittal, A.: The fact extraction and verification (fever) shared task. arXiv preprint arXiv:1811.10971 (2018)
Tosik, M., Mallia, A., Gangopadhyay, K.: Debunking fake news one feature at a time. arXiv preprint arXiv:1808.02831 (2018)
Villata, S., Boella, G., Gabbay, D.M., van der Torre, L.: Arguing about the trustworthiness of the information sources. In: Liu, W. (ed.) ECSQARU 2011. LNCS (LNAI), vol. 6717, pp. 74–85. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22152-1_7
Vlachos, A., Riedel, S.: Fact checking: task definition and dataset construction. In: Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, pp. 18–22 (2014)
Vosoughi, S., Roy, D., Aral, S.: The spread of true and false news online. Science 359(6380), 1146–1151 (2018)
Witschge, T., Nygren, G.: Journalistic work: a profession under pressure? J. Media Bus. Stud. 6(1), 37–59 (2009)
Yang, Y., Zheng, L., Zhang, J., Cui, Q., Li, Z., Yu, P.S.: TI-CNN: convolutional neural networks for fake news detection. CoRR abs/1806.00749 (2018). http://arxiv.org/abs/1806.00749
Yin, W., Roth, D.: TwoWingOS: a two-wing optimization strategy for evidential claim verification. CoRR abs/1808.03465 (2018). http://arxiv.org/abs/1808.03465
Acknowledgement
This work was partially funded by the European Union Marie Curie ITN Cleopatra project (GA no. 812997).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Chawla, P., Esteves, D., Pujar, K., Lehmann, J. (2019). SimpleLSTM: A Deep-Learning Approach to Simple-Claims Classification. In: Moura Oliveira, P., Novais, P., Reis, L. (eds) Progress in Artificial Intelligence. EPIA 2019. Lecture Notes in Computer Science(), vol 11805. Springer, Cham. https://doi.org/10.1007/978-3-030-30244-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-30244-3_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30243-6
Online ISBN: 978-3-030-30244-3
eBook Packages: Computer ScienceComputer Science (R0)