Skip to main content
SpringerLink
Log in

Detecting building changes with off-nadir aerial images

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The tilted viewing nature of the off-nadir aerial images brings severe challenges to the building change detection (BCD) problem: the mismatch of the nearby buildings and the semantic ambiguity of the building facades. To tackle these challenges, we present a multi-task guided change detection network model, named as MTGCD-Net. The proposed model approaches the specific BCD problem by designing three auxiliary tasks, including: (1) a pixel-wise classification task to predict the roofs and facades of buildings; (2) an auxiliary task for learning the roof-to-footprint offsets of each building to account for the misalignment between building roof instances; and (3) an auxiliary task for learning the identical roof matching flow between bi-temporal aerial images to tackle the building roof mismatch problem. These auxiliary tasks provide indispensable and complementary building parsing and matching information. The predictions of the auxiliary tasks are finally fused to the main BCD branch with a multi-modal distillation module. To train and test models for the BCD problem with off-nadir aerial images, we create a new benchmark dataset, named BANDON. Extensive experiments demonstrate that our model achieves superior performance over the previous state-of-the-art competitors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fujita A, Sakurada K, Imaizumi T, et al. Damage detection from aerial images via convolutional neural networks. In: Proceedings of the 15th IAPR International Conference on Machine Vision Applications (MVA), 2017. 5–8

  2. Gupta R, Goodman B, Patel N, et al. Creating xBD: a dataset for assessing building damage from satellite imagery. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019. 10–17

  3. Dueker K J, Horton F E. Urban-change detection systems: remote-sensing inputs. Photogrammetria, 1972, 28: 89–106

    Article  Google Scholar 

  4. Chen J, Liu H, Hou J, et al. Improving building change detection in VHR remote sensing imagery by combining coarse location and co-segmentation. ISPRS Int J Geo-Inform, 2018, 7: 213

    Article  Google Scholar 

  5. Awrangjeb M. Effective generation and update of a building map database through automatic building change detection from LiDAR point cloud data. Remote Sens, 2015, 7: 14119–14150

    Article  Google Scholar 

  6. Nie G, Liao G, Zeng C. SAR image change detection method based on PPNN. Sci China Inf Sci, 2021, 64: 189304

    Article  Google Scholar 

  7. Xu B W, Xu J K, Xue N, et al. Accurate polygonal mapping of buildings in satellite imagery. 2022. ArXiv:2208.00609

  8. Ding J, Xue N, Long Y, et al. Learning RoI transformer for oriented object detection in aerial images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

  9. Ding M T, Jin Z, Tian Z, et al. Object registration for remote sensing images using robust kernel pattern vectors. Sci China Inf Sci, 2012, 55: 2611–2623

    Article  MathSciNet  MATH  Google Scholar 

  10. Kang Y, Ding Y, Li Z R, et al. A networked remote sensing system for on-road vehicle emission monitoring. Sci China Inf Sci, 2017, 60: 043201

    Article  Google Scholar 

  11. Liu F, Zhu J H, Wang W F, et al. Surface-to-air missile sites detection agent with remote sensing images. Sci China Inf Sci, 2021, 64: 194201

    Article  Google Scholar 

  12. Xu F, Hu C, Li J, et al. Special focus on deep learning in remote sensing image processing. Sci China Inf Sci, 2020, 63: 140300

    Article  Google Scholar 

  13. Daudt R C, Le Saux B, Boulch A. Fully convolutional siamese networks for change detection. In: Proceedings of the 25th IEEE International Conference on Image Processing (ICIP), 2018. 4063–4067

  14. Chen H, Qi Z, Shi Z. Remote sensing image change detection with transformers. IEEE Trans Geosci Remote Sens, 2022, 60: 1–14

    Google Scholar 

  15. Chen H, Shi Z. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sens, 2020, 12: 1662

    Article  Google Scholar 

  16. Ji S, Wei S, Lu M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set. IEEE Trans Geosci Remote Sens, 2018, 57: 574–586

    Article  Google Scholar 

  17. Sun Y, Zhang X, Huang J, et al. Fine-grained building change detection from very high-spatial-resolution remote sensing images based on deep multitask learning. IEEE Geosci Remote Sens Lett, 2022, 19: 1–5

    Google Scholar 

  18. Li W J, Meng L X, Wang J W, et al. 3D building reconstruction from monocular remote sensing images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 12548–12557

  19. Wang J, Meng L, Li W, et al. Learning to extract building footprints from off-nadir aerial images. IEEE Trans Pattern Anal Mach Intell, 2023, 45: 1294–1301

    Article  Google Scholar 

  20. van Etten A, Hogan D, Manso J, et al. The multi-temporal urban development spacenet dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. 6398–6407

  21. Yang K, Xia G S, Liu Z, et al. Asymmetric siamese networks for semantic change detection in aerial images. IEEE Trans Geosci Remote Sens, 2022, 60: 1–18

    Google Scholar 

  22. Shen L, Lu Y, Chen H, et al. S2Looking: a satellite side-looking dataset for building change detection. Remote Sens, 2021, 13: 5094

    Article  Google Scholar 

  23. Dellinger F, Delon J, Gousseau Y, et al. Change detection for high resolution satellite images, based on SIFT descriptors and an a contrario approach. In: Proceedings of IEEE Geoscience and Remote Sensing Symposium, 2014. 1281–1284

  24. Padron-Hidalgo J A, Laparra V, Longbotham N, et al. Kernel anomalous change detection for remote sensing imagery. IEEE Trans Geosci Remote Sens, 2019, 57: 7743–7755

    Article  Google Scholar 

  25. Bu S, Li Q, Han P, et al. Mask-CDNet: a mask based pixel change detection network. Neurocomputing, 2020, 378: 166–178

    Article  Google Scholar 

  26. Chen B, Chen Z, Deng L, et al. Building change detection with RGB-D map generated from UAV images. Neurocomputing, 2016, 208: 350–364

    Article  Google Scholar 

  27. Bertinetto L, Valmadre J, Henriques J F, et al. Fully-convolutional siamese networks for object tracking. In: Proceedings of European Conference on Computer Vision, 2016. 850–865

  28. Zhang C, Yue P, Tapete D, et al. A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images. ISPRS J Photogramm Remote Sens, 2020, 166: 183–200

    Article  Google Scholar 

  29. Liu Y, Pang C, Zhan Z, et al. Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model. IEEE Geosci Remote Sens Lett, 2021, 18: 811–815

    Article  Google Scholar 

  30. Daudt R C, Le Saux B, Boulch A, et al. Multitask learning for large-scale semantic change detection. Comput Vision Image Underst, 2019, 187: 102783

    Article  Google Scholar 

  31. Esfandiari M, Abdi G, Jabari S, et al. Building change detection in off-nadir images using deep learning. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2021. 1347–1350

  32. He K M, Gkioxari G, Dollár P, et al. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, 2017. 2961–2969

  33. Zheng Z, Ma A L, Zhang L P, et al. Change is everywhere: single-temporal supervised object change detection in remote sensing imagery. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. 15193–15202

  34. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 3431–3440

  35. Sudre C H, Li W Q, Vercauteren T, et al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Proceedings of Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 2017. 240–248

  36. Zhao H S, Shi J P, Qi X J, et al. Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2881–2890

  37. He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 770–778

  38. Bandara W G C, Patel V M. A transformer-based siamese network for change detection. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2022

  39. Huang X, Serge B. Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, 2017. 1501–1510

  40. Ghiasi G, Zoph B, Cubuk E, et al. Multi-task self-training for learning general representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 8856–8865

Download references

Acknowledgements

This work was supported in part by National Nature Science Foundation of China (Grant Nos. 41820104006, U22B2011, 61922065) and National Key R&D Program of China (Grant No. 2021YFB3900503).

Author information

Author notes

  1. Pang C and Wu J have the same contribution to this work.

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Chao Pang & Gui-Song Xia

  2. SenseTime Research, Beijing, 100080, China

    Jiang Wu & Can Song

  3. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, 430079, China

    Jian Ding & Gui-Song Xia

Authors
  1. Chao Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Can Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gui-Song Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gui-Song Xia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pang, C., Wu, J., Ding, J. et al. Detecting building changes with off-nadir aerial images. Sci. China Inf. Sci. 66, 140306 (2023). https://doi.org/10.1007/s11432-022-3691-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3691-4

Keywords

Search

Navigation