One Versus All: Best Practices in Combining Multi-hazard Damage Imagery Training Datasets for Damage Detection for a Deep Learning Neural Network

Abstract

Accurate and timely damage assessment is important after any natural disaster event. Accurate damage assessments enhance the efficient distribution of resources. Building damage levels are an important outcome of damage assessment, especially in urban areas. Although at present most building damage assessments are collected manually from post-disaster satellite images or aerial photographs, efforts are now underway to automate the process. Some of these efforts use deep learning algorithms to first identify buildings and then to classify them into damage levels.

One of these efforts initiated in 2019, through the Defense Innovation Unit (DIU) and with Humanitarian Assistance and Disaster Recovery (HADR) organizations, created a multi-hazard training dataset using high-resolution satellite imagery from pre- and post-event imagery (xBD). Across 19 natural disaster events including tornados, wildfire, earthquake, hurricanes, volcanos, flood, and tsunami, buildings were identified and classified into four classes: no damage, minor damage, major damage, and destroyed. Participants in the challenge were expected to use deep learning algorithms to perform the classification. They were also provided with a base classification algorithm, in which participants were encouraged to improve. The base algorithm contained RESNET50 trained on ImageNet database and three additional convolution and max pooling layers.

This project analyzes the quality of the training dataset, discusses the pros and cons of combining training dataset across multiple natural disaster events, and provides recommendations on using the provided training dataset to optimize classification accuracy. Specifically, we will provide recommendations on creating class balance in the training dataset, in which damage labels are the most identifiable. We will also provide an assessment on which natural disasters lead to damage that is most identifiable using satellite imagery in which natural disasters lead to less accurate damage assessments. We will also examine pooling training data across natural disasters to achieve more accurate classifications.