Skip to main content
SpringerLink
Log in

A Weakly Supervised Convolutional Network for Change Segmentation and Classification

  • Conference paper
  • First Online:
Computer Vision – ACCV 2020 Workshops (ACCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12628))

Included in the following conference series:

Abstract

Fully supervised change detection methods require difficult to procure pixel-level labels, while weakly supervised approaches can be trained with image-level labels. However, most of these approaches require a combination of changed and unchanged image pairs for training. Thus, these methods can not directly be used for datasets where only changed image pairs are available. We present W-CDNet, a novel weakly supervised change detection network that can be trained with image-level semantic labels. Additionally, W-CDNet can be trained with two different types of datasets, either containing changed image pairs only or a mixture of changed and unchanged image pairs. Since we use image-level semantic labels for training, we simultaneously create a change mask and label the changed object for single-label images. W-CDNet employs a W-shaped siamese U-net to extract feature maps from an image pair which then get compared in order to create a raw change mask. The core part of our model, the Change Segmentation and Classification (CSC) module, learns an accurate change mask at a hidden layer by using a custom Remapping Block and then segmenting the current input image with the change mask. The segmented image is used to predict the image-level semantic label. The correct label can only be predicted if the change mask actually marks relevant change. This forces the model to learn an accurate change mask. We demonstrate the segmentation and classification performance of our approach and achieve top results on AICD and HRSCD, two public aerial imaging change detection datasets as well as on a Food Waste change detection dataset. Our code is available at: https://github.com/PhiAbs/W-CDNet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Shi, W., Min, Z., Zhang, R., Chen, S., Zhan, Z.: Change detection based on artificial intelligence: state-of-the-art and challenges. Remote Sens. 12, 1688 (2020)

    Article  Google Scholar 

  2. Ban, Y., Yousif, O.: Change detection techniques: a review. In: Ban, Y. (eds.) Multitemporal Remote Sensing. Remote Sensing and Digital Image Processing, vol. 20. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47037-5_2

  3. Liu, Z., Li, G., Mercier, G., He, Y., Pan, Q.: Change detection in heterogenous remote sensing images via homogeneous pixel transformation. IEEE Trans. Image Process. 27, 1822–1834 (2018)

    Article  MathSciNet  Google Scholar 

  4. Zhang, P., Gong, M., Su, L., Liu, J., Li, Z.: Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images. ISPRS J. Photogramm. Remote Sens. 116, 24–41 (2016)

    Article  Google Scholar 

  5. Wang, K., Gou, C.: M4CD: a robust change detection method for intelligent visual surveillance. IEEE Access 6, 15505–15520 (2018)

    Article  Google Scholar 

  6. Sakurada, K., Okatani, T.: Change detection from a street image pair using CNN features and superpixel segmentation. BMVC 61, 1–12 (2015)

    Google Scholar 

  7. Alcantarilla, P.F., Stent, S., Ros, G., Arroyo, R., Gherardi, R.: Street-view change detection with deconvolutional networks. Auton. Robots 42, 1301–1322 (2018)

    Article  Google Scholar 

  8. Daudt, R.C., Le Saux, B., Boulch, A.: Fully convolutional Siamese networks for change detection. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 4063–4067. IEEE (2018)

    Google Scholar 

  9. Khan, S.H., He, X., Porikli, F.M., Bennamoun, M., Sohel, F., Togneri, R.: Learning deep structured network for weakly supervised change detection. In: IJCAI (2017)

    Google Scholar 

  10. Jiang, X., Tang, H.: Dense high-resolution Siamese network for weakly-supervised change detection. In: 2019 6th International Conference on Systems and Informatics (ICSAI), pp. 547–552 (2019)

    Google Scholar 

  11. Minematsu, T., Shimada, A., Taniguchi, R.I.: Simple background subtraction constraint for weakly supervised background subtraction network. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–8. IEEE (2019)

    Google Scholar 

  12. Bromley, J., et al.: Signature verification using a “Siamese” time delay neural network. Int. J. Pattern Recognit. Artif. Intell. 7, 25 (1993)

    Article  Google Scholar 

  13. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  14. Zheng, S., et al.: Conditional random fields as recurrent neural networks. In: International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  15. Bourdis, N., Marraud, D., Sahbi, H.: Constrained optical flow for aerial image change detection. In: 2011 IEEE International Geoscience and Remote Sensing Symposium, pp. 4176–4179. IEEE (2011)

    Google Scholar 

  16. Daudt, R., Le Saux, B., Boulch, A., Gousseau, Y.: Multitask learning for large-scale semantic change detection. Comput. Vis. Image Underst. 187, 102783 (2019)

    Article  Google Scholar 

  17. Yu, H., Yang, W., Hua, G., Ru, H., Huang, P.: Change detection using high resolution remote sensing images based on active learning and Markov random fields. Remote Sens. 9, 1233 (2017)

    Article  Google Scholar 

  18. Lebedev, M., Vizilter, Y., Vygolov, O., Knyaz, V., Rubis, A.: Change detection in remote sensing images using conditional adversarial networks. ISPRS - Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. XLII(2), 565–571 (2018)

    Google Scholar 

  19. Guo, E., et al.: Learning to measure change: fully convolutional Siamese metric networks for scene change detection. CoRR abs/1810.09111 (2018)

    Google Scholar 

  20. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)

    Article  Google Scholar 

  21. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  22. Muchoney, D.M., Haack, B.N.: Change detection for monitoring forest defoliation. Photogramm. Eng. Remote Sens. 60, 1243–1252 (1994)

    Google Scholar 

  23. Lambin, E.F.: Change detection at multiple temporal scales: seasonal and annual variations in landscape variables. Photogramm. Eng. Remote Sens. 62, 931–938 (1996)

    Google Scholar 

  24. Collins, J.B., Woodcock, C.E.: Change detection using the Gramm-Schmidt transformation applied to mapping forest mortality. Remote Sens. Environ. 50(3), 267–279 (1994)

    Google Scholar 

  25. Jiang, H., Hu, X., Li, K., Zhang, J., Gong, J., Zhang, M.: PGA-SiamNet: pyramid feature-based attention-guided Siamese network for remote sensing orthoimagery building change detection. Remote Sens. 12, 484 (2020)

    Article  Google Scholar 

  26. Bu, S., Li, Q., Han, P., Leng, P., Li, K.: Mask-CDNet: a mask based pixel change detection network. Neurocomputing 378, 166–178 (2019)

    Google Scholar 

  27. Sakurada, K., Shibuya, M., Weimin, W.: Weakly supervised silhouette-based semantic scene change detection. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2020)

    Google Scholar 

  28. Lafferty, J.D., McCallum, A., Pereira, F.C.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282–289 (2001)

    Google Scholar 

  29. de Jong, K.L., Bosman, A.S.: Unsupervised change detection in satellite images using convolutional neural networks. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)

    Google Scholar 

  30. Chianucci, D., Savakis, A.: Unsupervised change detection using spatial transformer networks. In: 2016 IEEE Western New York Image and Signal Processing Workshop (WNYISPW), pp. 1–5. IEEE (2016)

    Google Scholar 

  31. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: Advances in Neural Information Processing Systems (NIPS 2015), vol. 28 (2015)

    Google Scholar 

  32. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)

    Google Scholar 

  33. Hahnloser, R., Sarpeshkar, R., Mahowald, M., Douglas, R., Seung, H.: Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature 405, 947–51 (2000)

    Article  Google Scholar 

  34. Caruana, R.: Multitask learning. In: Thrun, S., Pratt, L. (eds.) Learning to Learn. Springer, Boston (1998). https://doi.org/10.1007/978-1-4615-5529-2_5

  35. Yakubovskiy, P.: Segmentation models (2019). https://github.com/qubvel/segmentation_models

  36. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009 (2009)

    Google Scholar 

  37. Ji, S., Wei, S., Lu, M.: Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set. IEEE Trans. Geosci. Remote Sens. 57, 574–586 (2019)

    Article  Google Scholar 

  38. Wang, Y., Jodoin, P.M., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: CDNet 2014: an expanded change detection benchmark dataset. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2014)

    Google Scholar 

  39. Cheng, W., Zhang, Y., Lei, X., Yang, W., Xia, G.: Semantic change pattern analysis (2020)

    Google Scholar 

  40. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)

    Article  Google Scholar 

  41. Sørensen, T.: A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. Biologiske skrifter. I kommission hos E, Munksgaard (1948)

    Google Scholar 

  42. Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 24, pp. 109–117. Curran Associates, Inc. (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision Lab, ETH Zürich, Zürich, Switzerland

    Philipp Andermatt & Radu Timofte

Authors
  1. Philipp Andermatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Radu Timofte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Andermatt .

Editor information

Editors and Affiliations

  1. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  2. Seoul National University, Seoul, Korea (Republic of)

    Bohyung Han

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Andermatt, P., Timofte, R. (2021). A Weakly Supervised Convolutional Network for Change Segmentation and Classification. In: Sato, I., Han, B. (eds) Computer Vision – ACCV 2020 Workshops. ACCV 2020. Lecture Notes in Computer Science(), vol 12628. Springer, Cham. https://doi.org/10.1007/978-3-030-69756-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69756-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69755-6

  • Online ISBN: 978-3-030-69756-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation