Skip to main content
SpringerLink
Log in

A Class Distance Penalty Deep Learning Method for Post-disaster Building Damage Assessment

  • Surveying and Geo-Spatial Engineering
  • Published:
KSCE Journal of Civil Engineering Aims and scope

Abstract

Automatic building damage assessment can significantly aid rescue operations, attributed to booming deep learning and remote sensing technologies. However, the class imbalance of the dataset often skews prediction models towards the majority class in the segmentation of damaged buildings. This issue is further exacerbated when damaged buildings are categorized into multiple scales, intensifying biases within the models. Hence, this research adopts an algorithm-level method to improve the reliability of post-disaster damage assessment. It proposes a novel loss function named Ordinal Class Distance Penalty Loss (OCDPL), considering the ordinal relationship between classes and penalizing the misclassifications according to the class error distance. Two hyperparameters are also introduced to enable the model to fine-tune the contribution of ordinal relationships on the loss function. The satellite images of hurricane disasters in the xBD dataset were adopted as the case study. The results show that the proposed approach can improve F1 scores and Mean Absolute Error of overall damage level classes. Notably, the findings underscore the value of leveraging information on ordinal classes to facilitate the learning of minority classes and diminish class error distances. This aspect holds particular significance for emergency responses to widespread and severe disasters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Asgari Taghanaki S, Zheng Y, Zhou SK, Georgescu B, Sharma P, Xu D, Comaniciu D, Hamarneh G (2018) Combo loss: Handling input and output imbalance in multi-organ segmentation. arXiv Preprint, DOI: https://doi.org/10.48550/arXiv.1805.02798

  • Boin JB, Roth N, Doshi J, Llueca P, Borensztein N (2020) Multi-class segmentation under severe class imbalance: A case study in roof damage assessment. arXiv Preprint, DOI: https://doi.org/10.48550/arXiv.2010.07151

  • Bromley J, Guyon I, LeCun Y, Säckinger E, Shah R (1993) Signature verification using a “siamese” time delay neural network. International Journal of Pattern Recognition and Artificial Intelligence 7(4):669–688, DOI: https://doi.org/10.1142/S0218001493000339

    Article  Google Scholar 

  • Buda M, Maki A, Mazurowski MA (2018) A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks 106:249–259, DOI: https://doi.org/10.1016/j.neunet.2018.07.011

    Article  Google Scholar 

  • Cheng CS, Behzadan AH, Noshadravan A (2021) Deep learning for post-hurricane aerial damage assessment of buildings. Computer-Aided Civil and Infrastructure Engineering 36(6):695–710, DOI: https://doi.org/10.1111/mice.12658

    Article  Google Scholar 

  • Cooner AJ, Shao Y, Campbell JB (2016) Detection of urban damage using remote sensing and machine learning algorithms: Revisiting the 2010 Haiti earthquake. Remote Sensing 8(10):868, DOI: https://doi.org/10.3390/rs8100868

    Article  Google Scholar 

  • Dong L, Shan J (2013) A comprehensive review of earthquake-induced building damage detection with remote sensing techniques. ISPRS Journal of Photogrammetry and Remote Sensing 84:85–99, DOI: https://doi.org/10.1016/j.isprsjprs.2013.06.011

    Article  Google Scholar 

  • Duarte D, Nex F, Kerle N, Vosselman G (2018) Multi-resolution feature fusion for image classification of building damages with convolutional neural networks. Remote Sensing 10(10):1636, DOI: https://doi.org/10.3390/rs10101636

    Article  Google Scholar 

  • Fujita A, Sakurada K, Imaizumi T, Ito R, Hikosaka S, Nakamura R (2017) Damage detection from aerial images via convolutional neural networks. 2017 fifteenth IAPR international conference on machine vision applications (MVA), May 8–10, Nagoya, Japan, 5–8, DOI: https://doi.org/10.23919/MVA.2017.7986759

  • Galanis M, Rao K, Yao X, Tsai YL, Ventura J, Fricker GA (2021) DamageMap: A post-wildfire damaged buildings classifier. International Journal of Disaster Risk Reduction 65:102540, DOI: https://doi.org/10.1016/j.ijdrr.2021.102540

    Article  Google Scholar 

  • Guo H, Viktor HL (2004) Learning from unbalanced data sets with boosting and data generation: The databoost-im approach. ACM Sigkdd Explorations Newsletter 6(1):30–39, DOI: https://doi.org/10.1145/1007730.1007736

    Article  Google Scholar 

  • Gupta R, Hosfelt R, Sajeev S, Patel N, Goodman B, Doshi J, Heim E, Choset H, Gaston M (2019) xbd: A dataset for assessing building damage from satellite imagery. arXiv Preprint, DOI: https://doi.org/10.48550/arXiv.1911.09296

  • Haixiang G, Yijing L, Shang J, Mingyun G Yuanyue H, Bing G (2017) Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications 73:220–239, DOI: https://doi.org/10.1016/j.eswa.2016.12.035

    Article  Google Scholar 

  • Han W, Chen J, Wang L, Feng R Li F, Wu L, Tian T, Yan J (2021) Methods for small, weak object detection in optical high-resolution remote sensing images: A survey of advances and challenges. IEEE Geoscience and Remote Sensing Magazine 9(4):8–34, DOI: https://doi.org/10.1109/MGRS.2020.3041450

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of 2016 IEEE conference on computer vision and pattern recognition (CVPR), June 27–30, Las Vegas, NV, USA, 770–778, DOI: https://doi.org/10.1109/CVPR.2016.90

  • Hou L, Yu CP, Samaras D (2016) Squared earth mover’s distance-based loss for training deep neural networks. arXiv Preprint, DOI: https://doi.org/10.48550/arXiv.1611.05916

  • Ip F, Dohm JM, Baker VR, Doggett T, Davies AG, Castano R, Chien S, Cichy B, Greeley R, Sherwood R, Tran D (2006) Flood detection and monitoring with the autonomous sciencecraft experiment onboard EO-1. Remote Sensing of Environment 101(4):463–481, DOI: https://doi.org/10.1016/j.rse.2005.12.018

    Article  Google Scholar 

  • Jadon S (2020) A survey of loss functions for semantic segmentation. Proceedings of 2020 IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB), October 27–29, Via del Mar, Chile, 1–7, DOI: https://doi.org/10.1109/CIBCB48159.2020.9277638

  • Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. Proceedings of 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), July 21–26, Honolulu, HI, USA, 11–19, DOI: https://doi.org/10.1109/CVPRW.2017.156

  • Johnson JM, Khoshgoftaar TM (2019) Survey on deep learning with class imbalance. Journal of Big Data 6(1):1–54, DOI: https://doi.org/10.1186/s40537-019-0192-5

    Article  Google Scholar 

  • Ko B, Kwak S (2012) Survey of computer vision–based natural disaster warning systems. Optical Engineering 51(7):070901–070901, DOI: https://doi.org/10.1117/1.OE.51.7.070901

    Article  Google Scholar 

  • Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. Proceedings of 2017 IEEE conference on computer vision and pattern recognition (CVPR), July 21–26, Honolulu, HI, USA, 936–944, DOI: 10.1109/CVPR.2017.106

  • Lin TY, Goyal P, Girshick R, He K, Dollár P (2020) Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(2):318–327, DOI: https://doi.org/10.1109/TPAMI.2018.2858826

    Article  Google Scholar 

  • Lu S, Gao F, Piao C, Ma Y (2019) Dynamic weighted cross entropy for semantic segmentation with extremely imbalanced data. Proceedings of 2019 international conference on artificial intelligence and advanced manufacturing (AIAM), October 16–18, Dublin, Ireland, 230–233, DOI: https://doi.org/10.1109/AIAM48774.2019.00053

  • Masko D, Hensman P (2015) The impact of imbalanced training data for convolutional neural networks. Degree Project, KTH Royal Institute of Technology, Stockholm, Sweden

    Google Scholar 

  • Milletari F, Navab N, Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 fourth international conference on 3D vision (3DV), October 25–28, Stanford, CA, USA, 565–571, DOI: https://doi.org/10.1109/3DV.2016.79

  • Pi Y, Nath ND, Behzadan AH (2020) Convolutional neural networks for object detection in aerial imagery for disaster response and recovery. Advanced Engineering Informatics 43:101009, DOI: https://doi.org/10.1016/j.aei.2019.101009

    Article  Google Scholar 

  • Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. Proceedings of 18th international conference on medical image computing and computer-assisted intervention (MICCAI), October 5–9, Munich, Germany, 234–241, DOI: https://doi.org/10.1007/978-3-319-24574-428

  • Rubinstein R (1999) The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability 1:127–190, DOI: https://doi.org/10.1023/A:1010091220143

    Article  MathSciNet  Google Scholar 

  • Salehi SS, Erdogmus D, Gholipour A (2017) Tversky loss function for image segmentation using 3D fully convolutional deep networks. International workshop on machine learning in medical imaging (MLMI), September 7, Québec City, QC, Canada, 379–387, DOI: https://doi.org/10.1007/978-3-319-67389-9_44

  • Sudre CH, Li W, Vercauteren T, Ourselin S, Jorge Cardoso M (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Proceedings of deep learning in medical image analysis (DLMIA) and multimodal learning for clinical decision support (ML-CDS): 3rd international workshop and 7th international workshop, September 14, Québec City, QC, Canada, 240–248, DOI: https://doi.org/10.1007/978-3-319-67558-9_28

  • Sugino T, Kawase T, Onogi S, Kin T, Saito N, Nakajima Y (2021) Loss weightings for improving imbalanced brain structure segmentation using fully convolutional networks. Healthcare 9(8):938, DOI: https://doi.org/10.3390/healthcare9080938

    Article  Google Scholar 

  • Thomas J, Kareem A, Bowyer KW (2013) Automated poststorm damage classification of low-rise building roofing systems using high-resolution aerial imagery. IEEE Transactions on Geoscience and Remote Sensing 52(7):3851–3861, DOI: https://doi.org/10.1109/TGRS.2013.2277092

    Article  Google Scholar 

  • Wang H, Cui Z, Chen Y, Avidan M, Abdallah AB, Kronzer A (2018) Predicting hospital readmission via cost-sensitive deep learning. IEEE/ACM Transactions on Computational Biology and Bioinformatics 15(6):1968–1978, DOI: https://doi.org/10.1109/TCBB.2018.2827029

    Article  Google Scholar 

  • Wang Y, Cui L, Zhang C, Chen W, Xu Y, Zhang Q (2022) A two-stage seismic damage assessment method for small, dense, and imbalanced buildings in remote sensing images. Remote Sensing 14(4):1012, DOI: https://doi.org/10.3390/rs14041012

    Article  Google Scholar 

  • Weber E, Kané H (2020) Building disaster damage assessment in satellite imagery with multi-temporal fusion. arXiv Preprint, DOI: https://doi.org/10.48550/arXiv.2004.05525

  • Zagoruyko S, Komodakis N (2015) Learning to compare image patches via convolutional neural networks. Proceedings of 2015 IEEE conference on computer vision and pattern recognition (CVPR), June 7–12, Boston, MA, USA, 4353–4361, DOI: 10.1109/CVPR.2015.7299064

  • Zhao R, Qian B, Zhang X, Li Y, Wei R, Liu Y, Pan Y (2020) Rethinking dice loss for medical image segmentation. IEEE international conference on data mining (ICDM), November 17–20, Sorrento, Italy, 851–860, DOI: https://doi.org/10.1109/ICDM50108.2020.00094

  • Zhou ZH, Liu XY (2005) Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on Knowledge and Data Engineering 18(1):63–77, DOI: https://doi.org/10.1109/TKDE.2006.17

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by the National Taiwan University under grant NTU-112V1503-4 and National Science and Technology Council, Taiwan, under grant NSTC 112-2121-M-002-004.

Author information

Authors and Affiliations

  1. Dept. of Civil Engineering, National Taiwan University, Taipei City, 106319, Taiwan

    Fang Jung Tsai & Szu-Yun Lin

  2. NCREE-NTUCE Joint Artificial Intelligence Research Center, Taipei City, 106319, Taiwan

    Szu-Yun Lin

Authors
  1. Fang Jung Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Szu-Yun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Szu-Yun Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsai, F.J., Lin, SY. A Class Distance Penalty Deep Learning Method for Post-disaster Building Damage Assessment. KSCE J Civ Eng 28, 2005–2019 (2024). https://doi.org/10.1007/s12205-024-1587-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12205-024-1587-1

Keywords

Search

Navigation