Skip to main content
SpringerLink
Log in

“Generalization of convolutional network to domain adaptation network for classification of disaster images on twitter”

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Social media evolution in the recent past has drastically changed the way the affected community responds to any disaster. Within moments of happening of any disaster the activity on social media platforms swiftly jumps many folds. The affected population starts posting messages to update their family and friends about the situation and to seek help. The high volume and variety of this social media content pose challenges for the Government authorities and rescue workers to timely respond to the needs of the people. Effective computational analysis of this information can serve as an efficient tool to boost the desired response and rescue operations during such events. Recently with the significant progress in artificial intelligence systems, many machine learning methods for automatic filtration of relevant messages have been proposed by researchers. However, most of these methods follow supervised learning techniques that need abundant labeled data for training which is hard to arrange at the onset of the disaster. To overcome this constraint for a disaster in progress, our study proposes a novel deep learning convolutional network based on the concept of Unsupervised Domain Adaptation (UDA) that can classify unlabeled data for a new disaster (target domain) using the labeled data available from previous disaster (source domain). Through experiments performed on the images of seven disasters, the current work demonstrates that the proposed UDA method using the Maximum Mean Discrepancy (MMD) metric outperforms different state-of-the-art methods even without having labeled data for the new disaster.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

Can be provided.

Code availability

Can be provided and uploaded on Github.

References

  1. Alam F, Joty S, Imran M (2018) Domain adaptation with adversarial training and graph embeddings. In Proceedings of the 56th Annu meet Assoc Comput linguist proc Conf long Pap 1:1077–1087. https://doi.org/10.18653/v1/P18-1099

  2. Alam F, Ofli F, Imran M (2018) Processing social media images by combining human and machine computing during crises. Int J Hum Comput Interact 34(4):311–327. https://doi.org/10.1080/10447318.2018.1427831

    Article  Google Scholar 

  3. Alam F, Ofli F, Imran M (2018) CrisisMMD: multimodal twitter datasets from natural disasters. 12th Int AAAI Conf web Soc media. ICWSM 2018:465–473

    Google Scholar 

  4. Alam F, Ofli F, Imran M (2019) Descriptive and visual summaries of disaster events using artificial intelligence techniques: case studies of hurricanes Harvey, Irma, and Maria. Behav Inf Technol 0:1–31

  5. Basu M, Shandilya A, Ghosh K, Ghosh S (2018) Automatic matching of resource needs and availabilities in microblogs for post-disaster relief. Assoc Comput Mach (ACM) 25–26. https://doi.org/10.1145/3184558.3186911

  6. Borgwardt KM, Gretton A, Rasch MJ, Kriegel H, Schölkopf B, Smola AJ (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14):e49–e57

  7. Dong ZS, Meng L, Christenson L, Fulton L (2021) Social media information sharing for natural disaster response. Nat Hazards 107:2077–2104

    Article  Google Scholar 

  8. Dwarakanath L, Kamsin A, Rasheed RA, Anandhan A, Shuib L (2021) Automated machine learning approaches for emergency response and coordination via social Media in the Aftermath of a disaster : a review. IEEE Access 9:68917–68931. https://doi.org/10.1109/ACCESS.2021.3074819

  9. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. 32nd Int Conf Mach learn ICML 2:1180–1189. https://proceedings.mlr.press/v37/ganin15.html

  10. Ghifary M, Kleijn WB, Zhang M, Balduzzi D, Li W (2016) Deep reconstruction-classification networks for unsupervised domain adaptation. European conference on computer vision. Springer, pp 597–613. https://doi.org/10.1007/978-3-319-46493-0_36

  11. Goswami S, Chakraborty S, Ghosh S, Chakrabarti A, Chakraborty B (2018) A review on application of data mining techniques to combat natural disasters. Ain Shams Eng J 9:365–378

    Article  Google Scholar 

  12. Hiltz SR, Lee A, Imran M, Plotnick L, Power R, Turoff M (2020) International journal of disaster risk reduction exploring the usefulness and feasibility of software requirements for social media use in emergency management. Int J Disaster Risk Reduct 42:101367

    Article  Google Scholar 

  13. Imran M, Castillo C, Diaz F, Vieweg S (2018) Processing social media messages in mass emergency: Survey summary. Companion Proceedings of the The Web Conference 2:507–511. https://doi.org/10.1145/3184558.3186242

  14. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. International conference on machine learning 37:97–105. https://proceedings.mlr.press/v37/long15.html

  15. Joseph JK, Dev KA, Pradeepkumar AP, Mohan M (2018) Big data analytics and social media in disaster management. Integr Disaster Sci Manag 287–294. https://doi.org/10.1016/B978-0-12-812056-9.00016-6

  16. Kang G, Jiang L, Yang Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 4893–4902

  17. Kankanamge N, Yigitcanlar T, Goonetilleke A (2020) How engaging are disaster management related social media channels ? The case of Australian state emergency organisations. Int J Disaster Risk Reduct 48:101571

    Article  Google Scholar 

  18. Khattar A, Quadri SMK (2020) Emerging Role of Artificial Intelligence for Disaster Management Based on Microblogged Communication Int Conf Innov Comput Commun 2020. https://doi.org/10.2139/ssrn.3562973

  19. Khattar A, Quadri SMK (2021) Deep domain adaptation approach for classification of disaster images. Lect Notes Data Eng Commun Technol 57:245–259

    Article  Google Scholar 

  20. Li H, Guevara N, Herndon N, Caragea D, Neppalli K, Caragea C, Squicciarini A, Tapia AH (2015) Twitter mining for disaster response: a domain adaptation approach. J Conf Inf Syst Cris Response Manag 26(1):16–27. https://doi.org/10.1111/1468-5973.12194

  21. Li H, Caragea D, Caragea C, Herndon N (2018) Disaster response aided by tweet classification with a domain adaptation approach. J Contingencies Cris Manag 26:16–27

    Article  Google Scholar 

  22. Li X, Caragea C, Caragea D, Imran M, Ofli F (2019) Identifying disaster damage images using a domain adaptation approach. Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management 633–645

  23. Li X, Caragea D, Zhang H, Imran M (2019) Localizing and quantifying infrastructure damage using class activation mapping approaches. Soc Netw Anal Min 9:1–15

    Article  Google Scholar 

  24. Mavrodieva AV, Shaw R (2021) Social Media in Disaster Management. Disaster Risk Reduct (Methods, Approaches Pract Springer, Singapore. https://doi.org/10.1007/978-981-16-0285-6

  25. Nazer TH, Xue G, Ji Y, Liu H (2017) Intelligent disaster response via social media analysis a survey. ACM SIGKDD Explor Newsl 19:46–59

    Article  Google Scholar 

  26. Nguyen DT, Ofli F, Imran M, Mitra P (2017) Damage Assessment from Social Media Imagery Data During Disasters. In: Proc. 2017 IEEE/ACM Int Conf Adv Soc Networks Anal Min 2017. ACM, New York, NY, USA, pp 569–576

  27. Nguyen DT, Al Mannai KA, Joty S, Sajjad H, Imran M, Mitra P (2017) Robust classification of crisis-related data on social networks using convolutional neural networks. In: proc. 11th Int. Conf. Web Soc. Media, ICWSM 2017. AAAI press, pp 632–635

  28. Olteanu A, Castillo C, Diaz F, Vieweg S (2014) Crisislex: A lexicon for collecting and filtering microblogged communications in crises. Eighth international AAAI conference on weblogs and social media

  29. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  30. Pandey MN, Natarajan S (2016) How social media can contribute during disaster events? Intl Conf Adv Comput Commun Informatics 2016:1352–1356

  31. Phengsuwan J, Shah T, Thekkummal NB, Wen Z, Sun R, Pullarkatt D, Thirugnanam H, Ramesh MV, Morgan G, James P, Ranjan R (2021) Use of social media data in disaster management: a survey. Futur Internet 13:1–24

    Article  Google Scholar 

  32. Purohit H, Castillo C, Imran M, Pandev R (2018) Social-EOC: serviceability model to rank social media requests for emergency operation centers. Proc 2018 IEEE/ACM Int Conf Adv Soc networks anal mining. ASONAM 2018:119–126

    Google Scholar 

  33. Reuter C, Ludwig T, Kaufhold MA, Spielhofer T (2016) Emergency services' attitudes towards social media: a quantitative and qualitative survey across Europe. Int J Hum Comput Stud 95:96–111

    Article  Google Scholar 

  34. Robertson BW, Johnson M, Murthy D, Smith WR, Stephens KK (2019) Using a combination of human insights and ‘deep learning’ for real-time disaster communication. Prog Disaster Sci 2:100030

    Article  Google Scholar 

  35. Rudra K, Ghosh S, Ganguly N, Goyal P, Ghosh S (2015) Extracting Situational Information from Microblogs during Disaster Events. In: Proc. 24th ACM Int Conf Inf Knowl Manag ACM, New York, NY, USA, pp 583–592. https://doi.org/10.1145/2806416.2806485

  36. Said N, Ahmad K, Riegler M, Pogorelov K, Hassan L, Ahmad N, Conci N (2019) Natural disasters detection in social media and satellite imagery : a survey.

  37. Santoni V, Rufat S (2021) How fast is fast enough? Twitter usability during emergencies. Geoforum 124:20–35

    Article  Google Scholar 

  38. Shen J, Qu Y, Zhang W, Yu Y (2017) Wasserstein distance guided representation learning for domain adaptation. In: Thirty-second AAAI Conf. Artif Intell pp. 4058–4065. https://dl.acm.org/doi/abs/10.5555/3504035.3504532

  39. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. https://doi.org/10.48550/arXiv.1409.1556

  40. Sreenivasulu Madichetty SM (2020) Classifying informative and non-informative tweets from the twitter by adapting image features during disaster. Multimed tools Appl 79(39):28901–28923. https://doi.org/10.1007/s11042-020-09343-1

  41. Sun B, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Proc. Thirtieth AAAI Conf. Artif. Intell. Conf 30(1):2058–2065. https://ojs.aaai.org/index.php/AAAI/article/view/10306

  42. Vieweg S, Hughes AL, Starbird K, Palen L (2010) Microblogging during two natural hazards events: what twitter may contribute to situational awareness. Proc 28th Int Conf hum factors Comput Syst - CHI, pp 1079–1088. https://doi.org/10.1145/1753326.1753486

  43. Wiegmann M, Kersten J, Senaratne H, Potthast M, Klan F, Stein B (2021) Opportunities and risks of disaster data from social media : a systematic review of incident information. Natural Hazards and Earth System Sciences 21(5):1431–1444

  44. Wu F, Mostafavi ALI (2020) A hybrid machine learning pipeline for automated mapping of events and locations from social media in disasters. IEEE Access 8:10478–10490. https://doi.org/10.1109/ACCESS.2020.2965550

    Article  Google Scholar 

  45. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? Adv Neural Inf Process Syst 4:3320–3328

    Google Scholar 

Download references

Funding

Not Applicable.

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Natural Sciences, Jamia Millia Islamia, New Delhi, India

    Anuradha Khattar & S. M. K. Quadri

Authors
  1. Anuradha Khattar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. M. K. Quadri
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Not Applicable.

Corresponding author

Correspondence to Anuradha Khattar.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khattar, A., Quadri, S.M.K. “Generalization of convolutional network to domain adaptation network for classification of disaster images on twitter”. Multimed Tools Appl 81, 30437–30464 (2022). https://doi.org/10.1007/s11042-022-12869-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12869-1

Keywords

Search

Navigation