Skip to main content

Advertisement

SpringerLink
Log in

SOAT-UNET: a transformer-based Siamese over-attention network for change detection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The transformer plays a crucial role in building change detection (BCD) systems, which are important for observing urban development and post-disaster assessment. However, existing technologies often lack the ability to simultaneously attend to object features in bitemporal images and are not sensitive to changes in small target buildings. To address these issues, we propose SOAT-UNet, a novel transformer-based Siamese network with a multi-head over-attention block for CD tasks. Leveraging token-based space, our model extracts long-range contextual relationships and improves feature extraction for small targets. Inspired by human behavior, we generate queries (Q) from two image sets and calculate keys (K) and values (V) from another set, prioritizing regions likely to change. Experimental results demonstrate that our SOAT-UNet achieves superior CD performance compared to previous models on two existing datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Rensink, R.A.: Change detection. Annu. Rev. Psychol. 53(1), 66 (2002)

    Article  Google Scholar 

  2. Bandara, W.G.C., Patel, V.M.: Revisiting consistency regularization for semi-supervised change detection in remote sensing images. arXiv preprint arXiv:2204.08454 (2022)

  3. Tuia, D., Ratle, F., Pacifici, F., Kanevski, M.F., Emery, W.J.: Active learning methods for remote sensing image classification. IEEE Trans. Geosci. Remote Sens. 47(7), 2218–2232 (2009)

    Article  Google Scholar 

  4. Tan, B., Masek, J.G., Wolfe, R., Gao, F., Huang, C., Vermote, E.F., Sexton, J.O., Ederer, G.: Improved forest change detection with terrain illumination corrected Landsat images. Remote Sens. Environ. 136, 469–483 (2013)

    Article  Google Scholar 

  5. Daudt, R.C., Le Saux, B., Boulch, A., Gousseau, Y.: Urban change detection for multispectral earth observation using convolutional neural networks. In: IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, pp. 2115–2118 (2018). IEEE

  6. Bouziani, M., Goïta, K., He, D.-C.: Automatic change detection of buildings in urban environment from very high spatial resolution images using existing geodatabase and prior knowledge. ISPRS J. Photogramm. Remote Sens. 65(1), 143–153 (2010)

    Article  Google Scholar 

  7. Rokni, K., Ahmad, A., Selamat, A., Hazini, S.: Water feature extraction and change detection using multitemporal Landsat imagery. Remote Sens. 6(5), 4173–4189 (2014)

    Article  Google Scholar 

  8. Desclée, B., Bogaert, P., Defourny, P.: Forest change detection by statistical object-based method. Remote Sens. Environ. 102(1–2), 1–11 (2006)

    Article  Google Scholar 

  9. Brisco, B., Schmitt, A., Murnaghan, K., Kaya, S., Roth, A.: Sar polarimetric change detection for flooded vegetation. Int. J. Digit. Earth 6(2), 103–114 (2013)

    Article  Google Scholar 

  10. Wang, L., Li, R., Zhang, C., Fang, S., Duan, C., Meng, X., Atkinson, P.M.: Unetformer: a unet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery. ISPRS J. Photogram. Remote Sens. 190, 196–214 (2022)

    Article  Google Scholar 

  11. Bosc, M., Heitz, F., Armspach, J.-P., Namer, I., Gounot, D., Rumbach, L.: Automatic change detection in multimodal serial mri: application to multiple sclerosis lesion evolution. NeuroImage 20(2), 643–656 (2003)

    Article  Google Scholar 

  12. Castellana, L., D’Addabbo, A., Pasquariello, G.: A composed supervised/unsupervised approach to improve change detection from remote sensing. Pattern Recognit. Lett. 28(4), 405–413 (2007)

    Article  Google Scholar 

  13. Kuncheva, L.I., Faithfull, W.J.: Pca feature extraction for change detection in multidimensional unlabeled data. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 69–80 (2013)

    Article  Google Scholar 

  14. Chen, J., Chen, X., Cui, X., Chen, J.: Change vector analysis in posterior probability space: a new method for land cover change detection. IEEE Geosci. Remote Sens. Lett. 8(2), 317–321 (2010)

    Article  Google Scholar 

  15. Daudt, R.C., Le Saux, B., Boulch, A.: Fully convolutional Siamese networks for change detection. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 4063–4067. IEEE (2018)

  16. Wang, Q., Yuan, Z., Du, Q., Li, X.: Getnet: a general end-to-end 2-d cnn framework for hyperspectral image change detection. IEEE Trans. Geosci. Remote Sens. 57(1), 3–13 (2018)

    Article  Google Scholar 

  17. Wang, Q., Zhang, X., Chen, G., Dai, F., Gong, Y., Zhu, K.: Change detection based on faster r-cnn for high-resolution remote sensing images. Remote Sens. Lett. 9(10), 923–932 (2018)

    Article  Google Scholar 

  18. Liu, R., Kuffer, M., Persello, C.: The temporal dynamics of slums employing a cnn-based change detection approach. Remote Sens. 11(23), 2844 (2019)

    Article  Google Scholar 

  19. Zhang, Z., Vosselman, G., Gerke, M., Tuia, D., Yang, M.Y.: Change detection between multimodal remote sensing data using Siamese cnn. arXiv preprint arXiv:1807.09562 (2018)

  20. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł, Polosukhin, I.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 66 (2017)

    Google Scholar 

  21. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  22. Li, W., Xue, L., Wang, X., Li, G.: Mctnet: a multi-scale cnn-transformer network for change detection in optical remote sensing images. arXiv preprint arXiv:2210.07601 (2022)

  23. Song, X., Hua, Z., Li, J.: Remote sensing image change detection transformer network based on dual-feature mixed attention. IEEE Trans. Geosci. Remote Sens. 60, 1–16 (2022). https://doi.org/10.1109/TGRS.2022.3209972

    Article  Google Scholar 

  24. Wang, G., Li, B., Zhang, T., Zhang, S.: A network combining a transformer and a convolutional neural network for remote sensing image change detection. Remote Sens. 14(9), 2228 (2022)

    Article  Google Scholar 

  25. Yuan, J., Wang, L., Cheng, S.: Stransunet: a Siamese transunet-based remote sensing image change detection network. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 15, 9241–9253 (2022)

    Article  Google Scholar 

  26. Liu, M., Shi, Q., Li, J., Chai, Z.: Learning token-aligned representations with multimodel transformers for different-resolution change detection. IEEE Trans. Geosci. Remote Sens. 60, 1–13 (2022). https://doi.org/10.1109/TGRS.2022.3200684

    Article  Google Scholar 

  27. Li, Q., Zhong, R., Du, X., Du, Y.: Transunetcd: a hybrid transformer network for change detection in optical remote-sensing images. IEEE Trans. Geosci. Remote Sens. 60, 1–19 (2022)

    Google Scholar 

  28. Yuan, P., Zhao, Q., Zhao, X., Wang, X., Long, X., Zheng, Y.: A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images. Int. J. Digit. Earth 15(1), 1506–1525 (2022)

    Article  Google Scholar 

  29. Chen, H., Qi, Z., Shi, Z.: Remote sensing image change detection with transformers. IEEE Trans. Geosci. Remote Sens. 60, 1–14 (2021)

    Article  Google Scholar 

  30. Zhang, C., Wang, L., Cheng, S., Li, Y.: Swinsunet: pure transformer network for remote sensing image change detection. IEEE Trans. Geosci. Remote Sens. 60, 1–13 (2022)

    Google Scholar 

  31. Zhang, L., Hu, X., Zhang, M., Shu, Z., Zhou, H.: Object-level change detection with a dual correlation attention-guided detector. ISPRS J. Photogramm. Remote Sens. 177, 147–160 (2021)

    Article  Google Scholar 

  32. Song, F., Zhang, S., Lei, T., Song, Y., Peng, Z.: Mstdsnet-cd: multiscale swin transformer and deeply supervised network for change detection of the fast-growing urban regions. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2022)

    Google Scholar 

  33. Liu, M., Shi, Q., Chai, Z., Li, J.: Pa-former: learning prior-aware transformer for remote sensing building change detection. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2022)

    Google Scholar 

  34. Wang, X., Cheng, W., Feng, Y., Song, R.: Tscnet: topological structure coupling network for change detection of heterogeneous remote sensing images. Remote Sens. 15(3), 621 (2023)

    Article  Google Scholar 

  35. Zhang, M., Liu, Z., Feng, J., Liu, L., Jiao, L.: Remote sensing image change detection based on deep multi-scale multi-attention Siamese transformer network. Remote Sens. 15(3), 842 (2023)

    Article  Google Scholar 

  36. Dai, Y., Zheng, T., Xue, C., Zhou, L.: Mvit-pcd: a lightweight vit based network for Martian surface topographic change detection. IEEE Geosci. Remote Sens. Lett. 6, 66 (2023)

    Google Scholar 

  37. Chen, C.-F.R., Fan, Q., Panda, R.: Crossvit: Cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 357–366 (2021)

  38. Bandara, W.G.C., Patel, V.M.: A transformer-based Siamese network for change detection. arXiv preprint arXiv:2201.01293 (2022)

  39. Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., Shao, L.: Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 568–578 (2021)

  40. Wang, L., Li, H.: Hmcnet: hybrid efficient remote sensing images change detection network based on cross-axis attention mlp and cnn. IEEE Trans. Geosci. Remote Sens. 60, 1–14 (2022)

    Google Scholar 

  41. Chen, H., Shi, Z.: A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sens. 12(10), 1662 (2020)

  42. Zhang, C., Yue, P., Tapete, D., Jiang, L., Shangguan, B., Huang, L., Liu, G.: A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images. ISPRS J. Photogramm. Remote Sens. 166, 183–200 (2020)

    Article  Google Scholar 

  43. Fang, S., Li, K., Shao, J., Li, Z.: Snunet-cd: a densely connected Siamese network for change detection of vhr images. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)

    Google Scholar 

  44. Liu, Y., Pang, C., Zhan, Z., Zhang, X., Yang, X.: Building change detection for remote sensing images using a dual-task constrained deep Siamese convolutional network model. IEEE Geosci. Remote Sens. Lett. 18(5), 811–815 (2020)

    Article  Google Scholar 

  45. Ailimujiang, G., Jiaermuhamaiti, Y., Jumahong, H., Wang, H., Zhu, S., Nurmamaiti, P.: A transformer-based network for change detection in remote sensing using multiscale difference-enhancement. Comput. Intell. Neurosci. 6, 66 (2022)

    Google Scholar 

Download references

Funding

This work is supported by Key Research and Development Program of Shandong Province under Grant 2021CXGC011304, China University Innovation Fund under Grant 2021ZYA12004, Project of Natural Science Foundation of Shandong Province under Grant ZR2020MF066 and Youth Innovation Group Project of Shandong University under Grant 2020QNQT016.

Author information

Authors and Affiliations

  1. Institute of Marine Science and Technology, Shandong University, No. 72 Binhai Road, Qingdao, 266237, Shandong, China

    Xuhui Sun, Bowen Fu, Xiangyuan Jiang, Xiaojing Ma & Sile Ma

Authors
  1. Xuhui Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bowen Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangyuan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaojing Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sile Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XS provided ideas for article and wrote the main manuscript text, BF test model, XJ provided ideas for ablation experiments, XM checked the grammar, spelling, punctuation, and phrasing of paper, SM provided references and checked this paper. All authors reviewed the manuscript.

Corresponding authors

Correspondence to Xiaojing Ma or Sile Ma.

Ethics declarations

Conflict of interest

The authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, X., Fu, B., Jiang, X. et al. SOAT-UNET: a transformer-based Siamese over-attention network for change detection. SIViP 17, 4275–4283 (2023). https://doi.org/10.1007/s11760-023-02660-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02660-6

Keywords

Search

Navigation