Abstract
Disaster-based tweets during an emergency consist of a variety of information on people who have been hurt or killed, people who are lost or discovered, infrastructure and utilities destroyed; this information can assist governmental and humanitarian organizations in prioritizing their aid and rescue efforts. It is crucial to build a model that can categorize these tweets into distinct types due to their massive volume so as to better organize rescue and relief effort and save lives. In this study, Twitter data of 2013 Queensland flood and 2015 Nepal earthquake has been classified as disaster or non-disaster by employing three classes of models. The first model is performed using the lexical feature based on Term Frequency-Inverse Document Frequency (TF-IDF). The classification was performed using five classification algorithms such as DT, LR, SVM, RF, while Ensemble Voting was used to produce the outcome of the models. The second model uses shallow classifiers in conjunction with several features, including lexical (TF-IDF), hashtag, POS, and GloVe embedding. The third set of the model utilized deep learning algorithms including LSTM, LSTM, and GRU, using BERT (Bidirectional Encoder Representations from Transformers) for constructing semantic word embedding to learn the context. The key performance evaluation metrics such as accuracy, F1 score, recall, and precision were employed to measure and compare the three sets of models for disaster response classification on two publicly available Twitter datasets. By performing a comprehensive empirical evaluation of the tweet classification technique across different disaster kinds, the predictive performance shows that the best accuracy was achieved with DT algorithm which attained the highest performance accuracy followed by Bi-LSTM models for disaster response classification by attaining the best accuracy of 96.46% and 96.40% on the Queensland flood dataset; DT algorithm also attained 78.3% accuracy on the Nepal earthquake dataset based on the majority-voting ensemble respectively. Thus, this research contributes by investigating the integration of deep and shallow learning models effectively in a tweet classification system designed for disaster response. Examining the ways that these two methods work seamlessly offers insights into how to best utilize their complimentary advantages to increase the robustness and accuracy of locating suitable data in disaster crisis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available upon request.
References
Alam F, Joty S, Imran M (2018) Domain adaptation with adversarial training and graph embeddings. arXiv preprint arXiv:1805.05151
Alqaraleh S (2020) Classification of turkish text using machine learning: a case study using disasters tweets. International Journal of Scientific & Technology Research 9(3):4953–4956
Ashish V (2017) Attention is all you need. Adv Neural Inf Process Syst 30:1
Burel G, Alani H (2018) Crisis event extraction service (crees)-automatic detection and classification of crisis-related content on social media. In: Proceedings of the 15th ISCRAM Conference – Rochester, NY, USA May 2018
Chen Y, Xu L, Liu K, et al (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp 167–176
Cho SE, Jung K, Park HW (2013) Social media use during japan’s 2011 earthquake: how twitter transforms the locus of crisis communication. Media International Australia 149(1):28–40
Eke C, Norman AA, Shuib L, et al (2022) Random forest-based classifier for automatic sarcasm classification on twitter data using multiple features. Journal of Information Systems and Digital Technologies 4(2)
Eke CI, Norman AA, Shuib L et al (2019) A survey of user profiling: State-of-the-art, challenges, and solutions. IEEE Access 7:144907–144924
Eke CI, Norman AA, Shuib L (2021) Context-based feature technique for sarcasm identification in benchmark datasets using deep learning and bert model. IEEE Access 9:48501–48518
Eke CI, Norman AA, Shuib L (2021) Multi-feature fusion framework for sarcasm identification on twitter data: A machine learning based approach. PLoS ONE 16(6):e0252918
Gao H, Barbier G, Goolsby R (2011) Harnessing the crowdsourcing power of social media for disaster relief. IEEE Intell Syst 26(3):10–14
George A, Barathi Ganesh H, Anand Kumar M, et al (2019) Significance of global vectors representation in protein sequences analysis. In: Computer Aided Intervention and Diagnostics in Clinical and Medical Images, Springer, pp 261–269
Graves A, Schmidhuber J (2005) Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw 18(5–6):602–610
Graves A, Jaitly N, Mohamed Ar (2013) Hybrid speech recognition with deep bidirectional lstm. In: 2013 IEEE workshop on automatic speech recognition and understanding, IEEE, pp 273–278
Imran M, Castillo C, Diaz F et al (2015) Processing social media messages in mass emergency: A survey. ACM Computing Surveys (CSUR) 47(4):1–38
Kumar A, Singh JP (2019) Location reference identification from tweets during emergencies: A deep learning approach. International journal of disaster risk reduction 33:365–375
Kumar A, Singh JP, Dwivedi YK, et al (2022) A deep multi-modal neural network for informative twitter content classification during emergencies. Annals of Operations Research pp 1–32
Lamsal R, Kumar TV (2023) Twitter-based disaster response using recurrent nets. In: Research Anthology on Managing Crisis and Risk Communications. IGI Global, p 613–632
Nguyen D, Al Mannai KA, Joty S, et al (2017) Robust classification of crisis-related data on social networks using convolutional neural networks. In: Proceedings of the international AAAI conference on web and social media, pp 632–635
Nguyen TH, Grishman R (2015) Event detection and domain adaptation with convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp 365–371
Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Powers DM (2020) Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061
Prasetyo PK, Gao M, Lim EP, et al (2013) Social sensing for urban crisis management: The case of singapore haze. In: Social Informatics: 5th International Conference, SocInfo 2013, Kyoto, Japan, November 25-27, 2013, Proceedings 5, Springer, pp 478–491
Qu Y, Huang C, Zhang P, et al (2011) Microblogging after a major disaster in china: a case study of the 2010 yushu earthquake. In: Proceedings of the ACM 2011 conference on Computer supported cooperative work, pp 25–34
Ragini JR, Anand PR (2016) An empirical analysis and classification of crisis related tweets. In: 2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), IEEE, pp 1–4
Rashid NM, Shuib L, Eke CI, et al (2021) Detecting covid-19 in chest x-ray images with convolutional neural network. In: 2021 International Conference on Computer Science and Engineering (IC2SE), IEEE, pp 1–10
Salehinejad H, Valaee S, Dowdell T, et al (2018) Generalization of deep neural networks for chest pathology classification in x-rays using generative adversarial networks. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 990–994
Starbird K, Palen L, Hughes AL, et al (2010) Chatter on the red: what hazards threat reveals about the social life of microblogged information. In: Proceedings of the 2010 ACM conference on Computer supported cooperative work, pp 241–250
Tapia AH, Moore K (2014) Good enough is good enough: Overcoming disaster response organizations’ slow social media data adoption. Computer supported cooperative work (CSCW) 23:483–512
Tay Y, Tuan LA, Hui SC, et al (2018) Reasoning with sarcasm by reading in-between. arXiv preprint arXiv:1805.02856
Thomson R, Ito N, Suda H, et al (2012) Trusting tweets: The fukushima disaster and information source credibility on twitter. In: Iscram
Vieweg S (2012) Twitter communications in mass emergency: contributions to situational awareness. In: Proceedings of the ACM 2012 conference on computer supported cooperative work companion, pp 227–230
Vieweg S, Hughes AL, Starbird K, et al (2010) Microblogging during two natural hazards events: what twitter may contribute to situational awareness. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1079–1088
Yang Z, Nguyen LH, Stuve J, et al (2017) Harvey flooding rescue in social media. In: 2017 IEEE International Conference on Big Data (Big Data), IEEE, pp 2177–2185
Yang Z, Nguyen L, Zhu J, et al (2020) Coordinating disaster emergency response with heuristic reinforcement learning. In: 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), IEEE, pp 565–572
Young T, Hazarika D, Poria S et al (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75
Yu S, Su J, Luo D (2019) Improving bert-based text classification with auxiliary sentence and domain knowledge. IEEE Access 7:176600–176612
Zen H, Agiomyrgiannakis Y, Egberts N, et al (2016) Fast, compact, and high quality lstm-rnn based statistical parametric speech synthesizers for mobile devices. arXiv preprint arXiv:1606.06061
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A
Appendix A
See Tables 6, 7, 8, 9, 10, 11.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Maswadi, K., Alhazmi, A., Alshanketi, F. et al. The empirical study of tweet classification system for disaster response using shallow and deep learning models. J Ambient Intell Human Comput (2024). https://doi.org/10.1007/s12652-024-04807-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12652-024-04807-w