Abstract
Most rumour detection models for social media are designed for one specific language (mostly English). There are over 40 languages on Twitter and most languages lack annotated resources to build rumour detection models. In this paper we propose a zero-shot cross-lingual transfer learning framework that can adapt a rumour detection model trained for a source language to another target language. Our framework utilises pretrained multilingual language models (e.g. multilingual BERT) and a self-training loop to iteratively bootstrap the creation of “silver labels” in the target language to adapt the model from the source language to the target language. We evaluate our methodology on English and Chinese rumour datasets and demonstrate that our model substantially outperforms competitive benchmarks in both source and target language rumour detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The following article clarifies several rumours surrounding Bill Gates: https://www.reuters.com/article/uk-factcheck-gates-idUSKBN2613CK.
- 3.
- 4.
- 5.
Reactions are replies and quotes. \(r_i\) represents all reactions that can fit the maximum sequence length (384) for the pretrained model, concatenated together as a long string.
- 6.
For XLM-RoBERTa, we have 2 [SEP] symbols between \(s_i\) and \(r_i\), following https://huggingface.co/transformers/model_doc/xlmroberta.html#transformers.XLMRobertaTokenizer.build_inputs_with_special_tokens.
- 7.
Silver labels refer to the predicted labels in the target language, while gold labels refer to the real labels in the source language.
- 8.
- 9.
- 10.
For p we search in the range of 0.94–0.96.
- 11.
Following the original paper, only a maximum of 100 users are included.
- 12.
- 13.
The monolingual student model is pretrained using Wikipedia in the target language.
References
Allport, G.W., Postman, L.: The psychology of rumor (1947)
Artetxe, M., Schwenk, H.: Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. Trans. Assoc. Comput. Linguist. 7, 597–610 (2019)
Bengio, Y.: Deep learning of representations for unsupervised and transfer learning. In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning (2012)
Bian, T., et al.: Rumor detection on social media with bi-directional graph convolutional networks. In: AAAI (2020)
Bradbury, J., Merity, S., Xiong, C., Socher, R.: Quasi-recurrent neural networks (2016)
Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL (2019)
Eisenschlos, J., Ruder, S., Czapla, P., Kardas, M., Gugger, S., Howard, J.: MultiFiT: efficient multi-lingual language model fine-tuning (2019)
French, R.M.: Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3(4), 128–135 (1999)
Gururangan, S., et al.: Don’t stop pretraining: adapt language models to domains and tasks. In: ACL (2020)
He, J., Gu, J., Shen, J., Ranzato, M.: Revisiting self-training for neural sequence generation (2019)
Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114(13), 3521–3526 (2017)
Kochkina, E., Liakata, M., Zubiaga, A.: All-in-one: multi-task learning for rumour verification. In: COLING (2018)
Liu, X., Nourbakhsh, A., Li, Q., Fang, R., Shah, S.: Real-time rumor debunking on Twitter. In: CIKM (2015)
Liu, Y., Wu, Y.F.B.: Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. In: AAAI (2018)
Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach (2019)
Long, Y., Lu, Q., Xiang, R., Li, M., Huang, C.R.: Fake news detection through multi-perspective speaker profiles. In: IJCNLP (2017)
Ma, J., et al.: Detecting rumors from microblogs with recurrent neural networks. In: IJCAI (2016)
Ma, J., Gao, W., Wei, Z., Lu, Y., Wong, K.F.: Detect rumors using time series of social context information on microblogging websites. In: CIKM (2015)
Ma, J., Gao, W., Wong, K.F.: Detect rumors in microblog posts using propagation structure via kernel learning. In: ACL (2017)
Ma, J., Gao, W., Wong, K.F.: Detect rumors on Twitter by promoting information campaigns with generative adversarial learning. In: WWW (2019)
Mendoza, M., Poblete, B., Castillo, C.: Twitter under crisis: can we trust what we rt? In: Proceedings of the first workshop on social media analytics (2010)
Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., Cherry, C.: Semeval-2016 task 6: detecting stance in tweets. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) (2016)
Mohtarami, M., Glass, J., Nakov, P.: Contrastive language adaptation for cross-lingual stance detection. In: EMNLP-IJCNLP (2019)
Pires, T., Schlinger, E., Garrette, D.: How multilingual is multilingual bert? (2019)
Ren, Y., Zhang, J.: HGAT: Hierarchical Graph Attention Network for Fake News Detection (2020)
Scudder, H.: Probability of error of some adaptive pattern-recognition machines. IEEE Trans. Inf. Theory 11(3), 363–371 (1965)
Shu, K., Mahudeswaran, D., Wang, S., Lee, D., Liu, H.: Fakenewsnet: a data repository with news content, social context and dynamic information for studying fake news on social media. Big Data 8(3), 171–188 (2020)
Shu, K., Sliva, A., Wang, S., Tang, J., Liu, H.: Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor. Newsl. 19(1), 22–36 (2017)
Tian, L., Zhang, X., Wang, Y., Liu, H.: Early detection of Rumours on Twitter via stance transfer learning. In: ECIR (2020)
Wen, W., Su, S., Yu, Z.: Cross-lingual cross-platform rumor verification pivoting on multimedia content. In: EMNLP (2018)
Wiese, G., Weissenborn, D., Neves, M.: Neural domain adaptation for biomedical question answering (2017)
Wu, L., Liu, H.: Tracing fake-news footprints: characterizing social media messages by how they propagate. In: WSDM (2018)
Xie, Q., Luong, M.T., Hovy, E., Le, Q.V.: Self-training with noisy student improves imagenet classification. In: CVPR (2020)
Xu, Y., Zhong, X., Yepes, A.J.J., Lau, J.H.: Forget me not: reducing catastrophic forgetting for domain adaptation in reading comprehension. In: IJCNN (2020)
Yang, F., Liu, Y., Yu, X., Yang, M.: Automatic detection of rumor on sina weibo. In: Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics (2012)
Yang, S., Shu, K., Wang, S., Gu, R., Wu, F., Liu, H.: Unsupervised fake news detection on social media: a generative approach. In: AAAI (2019)
Yin, D., Meng, T., Chang, K.W.: Sentibert: a transferable transformer-based architecture for compositional sentiment semantics. In: ACL (2020)
Zhou, K., Shu, C., Li, B., Lau, J.H.: Early rumour detection. In: NAACL (2019)
Zou, Y., Yu, Z., Vijaya Kumar, B., Wang, J.: Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In: ECCV (2018)
Acknowledgments
This research is supported in part by the Australian Research Council Discovery Project DP200101441.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Tian, L., Zhang, X., Lau, J.H. (2021). Rumour Detection via Zero-Shot Cross-Lingual Transfer Learning. In: Oliver, N., Pérez-Cruz, F., Kramer, S., Read, J., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2021. Lecture Notes in Computer Science(), vol 12975. Springer, Cham. https://doi.org/10.1007/978-3-030-86486-6_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-86486-6_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86485-9
Online ISBN: 978-3-030-86486-6
eBook Packages: Computer ScienceComputer Science (R0)
