Skip to main content
SpringerLink
Log in

Collapsed Building Detection Using Multiple Object Tracking from Aerial Videos and Analysis of Effective Filming Techniques of Drones

  • Conference paper
  • First Online:
Information Technology in Disaster Risk Reduction (ITDRR 2022)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 672))

  • 159 Accesses

Abstract

Earthquake destroyed many buildings, especially wooden ones, in Japan. Collecting information regarding collapsed buildings during the emergency phase (i.e., 72 h after a disaster) is difficult but essential for rescue activities. This study developed an automatic model to detect collapsed buildings using multiple object tracking (MOT) from aerial videos. Roof damage and pancake collapse are destructions unique to traditional Japanese buildings. Previous studies that detected collapsed buildings using the features of debris or damage failed to discriminate between collapsed and held-up buildings when the buildings have the above Japanese feature. Therefore, this study used the deep learning MOT model to classify collapsed and held-up buildings regardless of debris appearance. The recall and precision of each track of collapsed buildings were 29.1% and 36.7%, respectively, based on cross-validation with the drone video of the 2016 Kumamoto Earthquake. Analysis between the recall and other factors indicated that the aspect ratio, speed, and appearance time of the buildings were significant features for the detection. In the relationship between recall and these factors, we deduce that the recall of track increases to 63.9% if the drone operator films aerial videos effectively. Moreover, this study analyzed effective drone filming and flying way to satisfy some conditions for detection. This result provides recommended filming guides to drone operators for future earthquakes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fire and Disaster Management Agency in Japan: Final report about the Great Hanshin-Awaji Earthquake. https://www.fdma.go.jp/disaster/info/assets/post1.pdf. Accessed 5 July 2022. (in Japanese)

  2. Hyogo Prefecture Medical Association: Actual state of affairs of human damage by the Great Hanshin-Awaji Earthquake. https://www.hyogo.med.or.jp/jmat-hyogo/day-after/siryo/. Accessed 5 July 2022. (in Japanese)

  3. Disaster response headquarters of Fukushima prefecture: No. 1778 damage report of the 2011 off the Pacific coast of Tohoku Earthquake. https://www.pref.fukushima.lg.jp/uploaded/life/620025_1725169_misc.pdf. Accessed 5 July 2022. (in Japanese)

  4. Disaster response headquarters of Kumamoto prefecture: No. 325 damage report of the 2016 Kumamoto Earthquake. https://www.pref.kumamoto.jp/uploaded/attachment/182677.pdf. Accessed 5 July 2022. (in Japanese)

  5. Fire and Disaster Management Agency in Japan: Report of review meeting about ideal way of effective initial activity of firefighter headquarter in large-scale disasters. https://www.fdma.go.jp/singi_kento/kento/items/kento004_01_houkoku.pdf. Accessed 5 July 2022. (in Japanese)

  6. Kumamoto city fire department. Activity record journal of Kumamoto city fire department in the 2016 Kumamoto Earthquake. https://www.city.kumamoto.jp/common/UploadFileDsp.aspx?c_id=5&id=19060&sub_id=1&flid=134936. Accessed 5 July 2022. (in Japanese)

  7. Tv Asahi news site. https://news.tv-asahi.co.jp/news_society/articles/000074837.html. Accessed July 2022. (in Japanese)

  8. Murakami, H.: Study on rescue and emergency activities by Kumamoto City fire department after the 2016 Kumamoto earthquake - comparison with the 2004 Niigata Chuuetsu and the 2005 West off Fukuoka Prefecture earthquakes. Report of Tono Research Institute of Earthquake Science, vol. 41, pp. 93–98 (2017). (in Japanese)

    Google Scholar 

  9. The Sankei News. https://www.sankei.com/article/20200115-7M7FDUJP2RLLDJDFXYP23H4J3E/. Accessed 5 July 2022. (in Japanese)

  10. Fire and Disaster Management Agency in Japan: Notice about promotion of drone usage of the fire brigade in disaster response. https://www.fdma.go.jp/laws/tutatsu/items/040331_drone.pdf. Accessed 5 July 2022 (in Japanese)

  11. Nazarov, E.: Emergency response management in Japan, final research report. ASIAN disaster reduction center, FY2011A program (2011)

    Google Scholar 

  12. Anand, V., Markus, G., Norman, K., Francesco, N., George, V.: Disaster damage detection through synergistic use of deep learning and 3D point cloud features derived from very high-resolution oblique aerial images, and multiple-kernel-learning. ISPRS J. Photogramm. Remote Sens. 140, 45–59 (2018)

    Article  Google Scholar 

  13. Johnny, C., Norman, K., Francesco, N.: Usability of aerial video footage for 3-D scene reconstruction and structural damage assessment. Nat. Hazards Earth Syst. Sci. 18(6), 1583–1598 (2018)

    Google Scholar 

  14. Yamazaki, F., Kubo, K., Tanabe, R., Liu, W.: Damage assessment and 3D modeling by UAV flights after the 2016 Kumamoto, Japan earthquake. In: The IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, pp. 3182–3185 (2017)

    Google Scholar 

  15. Fujita, S., Hatayama, M.: Estimation method for roof‐damaged buildings from aero-photo images during earthquakes using deep learning. Inf. Syst. Front. 25(1), 351–363 (2021)

    Google Scholar 

  16. Fujita, S., Hatayama, M.: Automatic calculation of damage rate of roofs based on image segmentation. In: 6th IFIP WG 5.15 International Conference, ITDRR 2021, pp. 3–22, Morioka, Japan, (2022)

    Google Scholar 

  17. Fujita, S., Hatayama, M.: Rapid and accurate detection of building damage investigation using an automatic method to calculate roof damage rate. IDRiM J. 12(1), 89–111 (2022)

    Article  Google Scholar 

  18. Miura, H., Aridome, T., Matsuoka, M.: Deep learning-based identification of collapsed, non-collapsed and blue tarp-covered buildings from post-disaster aerial images. Remote Sens. 12(12) (2020)

    Google Scholar 

  19. Calantropio, A., Chiabrando, F., Codastefano, M., Bourke, E.: Deep learning for automatic building damage assessment: application in post-disaster scenarios using UAV data. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. V-1-2021, 113–120 (2021)

    Google Scholar 

  20. Yamazaki, F., et al.: Development of fragility curves of Japanese buildings based on the 2016 Kumamoto earthquake. In: The 2019 Pacific Conference on Earthquake Engineering, Auckland, New Zealand (2019)

    Google Scholar 

  21. Kusaka, A., Nakamura, H., Fujiwara, H., Okano, H.: Bayesian updating of damaged building distribution in post-earthquake assessment. In: The 16th World Conference on Earthquake, 16WCEE 2017, Santiago Chile (2017)

    Google Scholar 

  22. Xie, S., et al.: Crowdsourcing rapid assessment of collapsed buildings early after the earthquake based on aerial remote sensing image: a case study of Yushu earthquake. Remote Sens. 8(9) (2016)

    Google Scholar 

  23. Khajwal, A.B., Noshadravan, A.: An uncertainty-aware framework for reliable disaster damage assessment via crowdsourcing. Int. J. Disaster Risk Reduct. 55 (2021)

    Google Scholar 

  24. Shishido, H., Kobayashi, K., Kameda, Y., Kitahara, I.: Method to generate building damage maps by combining aerial image processing and crowdsourcing. J. Disaster Res. 16(5), 827–839 (2021)

    Article  Google Scholar 

  25. Qi, J., et al.: Search and rescue rotary‐wing UAV and its application to the Lushan Ms 7.0 earthquake. J. Field Robot. 33(3), 290–321 (2016)

    Google Scholar 

  26. Pi, Y., Nath, N.D., Behzadan, A.H.: Convolutional neural networks for object detection in aerial imagery for disaster response and recovery. Adv. Eng. Inform. 43 (2020)

    Google Scholar 

  27. Zhu, X., Liang, J., Hauptmann, A.: MSNet: a multilevel instance segmentation network for natural disaster damage assessment in aerial videos. In: The IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2023–2032 (2021)

    Google Scholar 

  28. Scawthorn, C., Yanev, P.I.: Preliminary report 17 January 1995, Hyogo-ken Nambu, Japanese earthquake. Eng. Struct. 17(3), 146–157 (1995)

    Google Scholar 

  29. Okada, S., Takai, N.: Classifications of structural types and damage patterns of buildings for earthquake field investigation. In: 12th World Conference on Earthquake Engineering, Auckland, New Zealand (2000)

    Google Scholar 

  30. Scawthorn, C.: Building aspects of the 2004 Niigata Ken Chuetsu, Japan, Earthquake. Earthq. Spectra 22(1), 75–88 (2006)

    Google Scholar 

  31. Zhang, Y., et al.: Bytetrack: multi-object tracking by associating every detection box. arXiv preprint arXiv:2110.06864 (2021)

  32. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: Yolox: exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021)

  33. GitHub - ifzhang/ByteTrack: [ECCV 2022] ByteTrack: Multi-Object Tracking by Associating Every Detection Box. https://github.com/ifzhang/ByteTrack. Accessed 7 July 2022

  34. Ministry of Internal Affairs and Communications: Result summary of basic tabulation about housing and households in housiPng and land survey. https://www.stat.go.jp/data/jyutaku/2018/pdf/kihon_gaiyou.pdf. Accessed 29 July 2022. (in Japanese)

  35. DJI: Mavic 3 specification. https://www.dji.com/jp/mavic-3/specs. Accessed 29 July 2022. (in Japanese)

Download references

Acknowledgements

This work is supported by JSPS KAKENHI Grant Number JP 22J15895.

Author information

Authors and Affiliations

  1. Graduate School of Informatics, Kyoto University, 36-1 Yoshidahommachi, Sakyo-Ku, Kyoto-City, Kyoto, 606-8501, Japan

    Shono Fujita

  2. Disaster Prevention Research Institute, Kyoto University, Gokasho, Uji-City, Kyoto, 611-0011, Japan

    Michinori Hatayama

Authors
  1. Shono Fujita
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michinori Hatayama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shono Fujita .

Editor information

Editors and Affiliations

  1. University of Agder, Kristiansand, Norway

    Terje Gjøsæter

  2. University of Agder, Kristiansand, Norway

    Jaziar Radianti

  3. Tsuda University, Tokyo, Japan

    Yuko Murayama

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fujita, S., Hatayama, M. (2023). Collapsed Building Detection Using Multiple Object Tracking from Aerial Videos and Analysis of Effective Filming Techniques of Drones. In: Gjøsæter, T., Radianti, J., Murayama, Y. (eds) Information Technology in Disaster Risk Reduction. ITDRR 2022. IFIP Advances in Information and Communication Technology, vol 672. Springer, Cham. https://doi.org/10.1007/978-3-031-34207-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34207-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34206-6

  • Online ISBN: 978-3-031-34207-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation