Skip to main content
SpringerLink
Log in

Classifying informative and non-informative tweets from the twitter by adapting image features during disaster

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

During the crisis, people post a large number of informative and non-informative tweets on Twitter. Informative tweets provide helpful information such as affected individuals, infrastructure damage, availability and resource requirements, etc. In contrast, non-informative tweets do not provide helpful information related to either humanitarian organizations or victims. Identifying informative tweets is a challenging task during the disaster. People often post images along with text on Twitter during the disaster. In addition to tweet text features, image features are also crucial for identifying informative tweets. However, existing methods use only text features but do not use image features to identify crisis-related tweets during the disaster. This paper proposes a novel approach by considering the image features along with the text features. It includes a text-based classification model, an image-based classification model and a late fusion. The text-based classification model uses the Convolutional Neural Network (CNN) and the Artificial Neural Network (ANN). CNN is used to extract text features from a tweet and the ANN is used to classify tweets based on extracted text features of CNN. The image-based classification model uses the fine-tuned VGG-16 architecture to extract the image features from the image and classify the image in a tweet. The output of the text-based classification model and the image-based classification model are combined using the late fusion technique to predict the tweet label. Extensive experiments are carried out on Twitter datasets of various crises, such as the Mexico earthquake, California Wildfires, etc., to demonstrate the effectiveness of the proposed method. The proposed method outperforms the state-of-the-art methods on various parameters to identify informative tweets during the disaster.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alam F, Imran M, Ofli F (2019) Crisisdps: Crisis data processing services. In: Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management (ISCRAM)

  2. Alam F, Ofli F, Imran M (2018) Crisismmd: Multimodal twitter datasets from natural disasters. In: AAAI Conference on Web and Social Media (ICWSM), AAAI, AAAI, Stanford, California, USA

  3. Alam F, Ofli F, Imran M (2019) Descriptive and visual summaries of disaster events using artificial intelligence techniques: case studies of hurricanes Harvey, Irma, and Maria. Behaviour & Information Technology, pp 1–31

  4. Alam F, Ofli F, Imran M (2020) Descriptive and visual summaries of disaster events using artificial intelligence techniques: case studies of hurricanes harvey, irma, and maria. Behaviour & Information Technology 39(3):288–318

    Article  Google Scholar 

  5. Basu M, Shandilya A, Khosla P, Ghosh K, Ghosh S (2019) Extracting resource needs and availabilities from microblogs for aiding post-disaster relief operations. IEEE Transactions on Computational Social Systems 6(3):604–618

    Article  Google Scholar 

  6. Breiman L (2001) Random forests. Machine learning 45(1):5–32

    Article  MATH  Google Scholar 

  7. Burel G, Alani H (2018) Crisis event extraction service (crees)-automatic detection and classification of crisis-related content on social media. In: Proc. of the 15th ISCRAM

  8. Caragea C, Silvescu A, Tapia A H (2016) Identifying informative messages in disaster events using convolutional neural networks. In: International Conference on Information Systems for Crisis Response and Management, pp 137–147

  9. Caragea C, Squicciarini A C, Stehle S, Neppalli K, Tapia A H (2014) Mapping moods: Geo-mapped sentiment analysis during hurricane sandy.. In: ISCRAM

  10. Castillo C (2016) Big crisis data: social media in disasters and time-critical situations. Cambridge University Press

  11. Chollet F, et al. (2015) Keras. https://keras.io

  12. Dunning T (1993) Accurate methods for the statistics of surprise and coincidence. Computational Linguistics 19(1):61–74. https://www.aclweb.org/anthology/J93-1003

    Google Scholar 

  13. Enenkel M, Saenz S M, Dookie D S, Braman L, Obradovich N, Kryvasheyeu Y (2018) Social media data analysis and feedback for advanced disaster risk management. CoRR abs/1802.02631 arXiv:1802.02631

  14. Fan C, Wu F, Mostafavi A (2020) A hybrid machine learning pipeline for automated mapping of events and locations from social media in disasters. IEEE Access

  15. Freund Y, Schapire R E, et al. (1996) Experiments with a new boosting algorithm. In: Icml, Bari, Italy, vol. 96, pp 148–156

  16. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  17. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  18. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, pp 630–645. Springer

  19. Houston J B, Hawthorne J, Perreault M F, Park E H, Goldstein Hode M, Halliwell M R, Turner McGowen S E, Davis R, Vaid S, McElderry J A, et al. (2015) Social media and disasters: a functional framework for social media use in disaster planning, response, and research. Disasters 39(1):1–22

    Article  Google Scholar 

  20. Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR abs/1704.04861, pp 1–9

  21. Imran M, Castillo C, Diaz F, Vieweg S (2015) Processing social media messages in mass emergency: A survey. ACM Computing Surveys (CSUR) 47(4):67

    Article  Google Scholar 

  22. Imran M, Castillo C, Lucas J, Meier P, Vieweg S (2014) Aidr: Artificial intelligence for disaster response. In: Proceedings of the 23rd International Conference on World Wide Web, pp 159–162. ACM

  23. Imran M, Mitra P, Castillo C (2016) Twitter as a lifeline: Human-annotated twitter corpora for nlp of crisis-related messages. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA), Paris, France, pp 1638–1643

  24. Kim Y (2014) Convolutional neural networks for sentence classification

  25. Kingma D P, Ba J (2015) Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

  26. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  27. Kryvasheyeu Y, Chen H, Obradovich N, Moro E, Van Hentenryck P, Fowler J, Cebrian M (2016) Rapid assessment of disaster damage using social media activity. Science advances 2(3):e1500779

    Article  Google Scholar 

  28. Luhn H P (1957) A statistical approach to mechanized encoding and searching of literary information. IBM Journal of research and development 1(4):309–317

    Article  MathSciNet  Google Scholar 

  29. Madichetty S, Sridevi M (2019) Detecting informative tweets during disaster using deep neural networks. In: 2019 11th International Conference on Communication Systems & Networks (COMSNETS), pp 709–713. IEEE

  30. Madichetty S, Sridevi M (2019) Disaster damage assessment from the tweets using the combination of statistical features and informative words. Social Network Analysis and Mining 9(1):42

    Article  Google Scholar 

  31. Madichetty S, et al. (2018) Re-ranking feature selection algorithm for detecting the availability and requirement of resources tweets during disaster. International Journal of Computational Intelligence & IoT, 1(2)

  32. McCulloch W S, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics 5(4):115–133

    Article  MathSciNet  MATH  Google Scholar 

  33. Mitchell J T, Thomas Deborah SK, Hill A A, Cutter S L (2000) Catastrophe in reel life versus real life: Perpetuating disaster myth through hollywood films. International Journal of Mass Emergencies and Disasters 18 (3):383–402

    Google Scholar 

  34. Nguyen D T, Al Mannai K A, Joty S, Sajjad H, Imran M, Mitra P (2017) Robust classification of crisis-related data on social networks using convolutional neural networks. In: Eleventh International AAAI Conference on Web and Social Media, pp 632–635

  35. Nguyen D T, Ofli F, Imran M, Mitra P (2017) Damage assessment from social media imagery data during disasters. In: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, pp 569–576. ACM

  36. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  37. Pennington J, Socher R, Manning C D (2014) Glove: Global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP). http://www.aclweb.org/anthology/D14-1162, pp 1532–1543

  38. Purohit H, Castillo C, Diaz F, Sheth A, Meier P (2014) Emergency-relief coordination on social media: Automatically matching resource requests and offers. First Monday, 19(1)

  39. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review 65(6):386

    Article  Google Scholar 

  40. Rudra K, Ganguly N, Goyal P, Ghosh S (2018) Extracting and summarizing situational information from the twitter social media during disasters. ACM Transactions on the Web (TWEB) 12(3):17

    Google Scholar 

  41. Rudra K, Ghosh S, Ganguly N, Goyal P, Ghosh S (2015) Extracting situational information from microblogs during disaster events: a classification-summarization approach. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp 583–592. ACM

  42. Rudra K, Sharma A, Ganguly N, Ghosh S (2016) Characterizing communal microblogs during disaster events. In: Advances in Social Networks Analysis and Mining (ASONAM), 2016 IEEE/ACM International Conference on, pp 96–99. IEEE

  43. Rudra K, Sharma A, Ganguly N, Imran M (2017) Classifying information from microblogs during epidemics. In: Proceedings of the 2017 International Conference on Digital Health, pp 104–108. ACM

  44. Sarter N B, Woods D D (1991) Situation awareness: A critical but ill-defined phenomenon. Int J Aviat Psychol 1(1):45–57

    Article  Google Scholar 

  45. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd international conference on learning representations, ICLR, pp 1–14

  46. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  47. Vapnik V (2013) The nature of statistical learning theory. Springer science & business media

  48. Vieweg S, Castillo C, Imran M (2014) Integrating social media communications into the rapid assessment of sudden onset disasters. In: International Conference on Social Informatics, vol 8851. Springer, pp 444–461

  49. Werbos P (1974) Beyond regression: New tools for prediction and analysis in the behavior science. Unpublished Doctoral Dissertation, Harvard University

  50. Zeiler M D (2012) Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, Tamil Nadu, 620015, India

    Sreenivasulu Madichetty & Sridevi M

Authors
  1. Sreenivasulu Madichetty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sridevi M
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sreenivasulu Madichetty.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Madichetty, S., M, S. Classifying informative and non-informative tweets from the twitter by adapting image features during disaster. Multimed Tools Appl 79, 28901–28923 (2020). https://doi.org/10.1007/s11042-020-09343-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09343-1

Keywords

Search

Navigation