Advertisement

End-to-End Trainable Deep Active Contour Models for Automated Image Segmentation: Delineating Buildings in Aerial Imagery

Conference paper
  • 1.1k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12357)

Abstract

The automated segmentation of buildings in remote sensing imagery is a challenging task that requires the accurate delineation of multiple building instances over typically large image areas. Manual methods are often laborious and current deep-learning-based approaches fail to delineate all building instances and do so with adequate accuracy. As a solution, we present Trainable Deep Active Contours (TDACs), an automatic image segmentation framework that intimately unites Convolutional Neural Networks (CNNs) and Active Contour Models (ACMs). The Eulerian energy functional of the ACM component includes per-pixel parameter maps that are predicted by the backbone CNN, which also initializes the ACM. Importantly, both the ACM and CNN components are fully implemented in TensorFlow and the entire TDAC architecture is end-to-end automatically differentiable and backpropagation trainable without user intervention. TDAC yields fast, accurate, and fully automatic simultaneous delineation of arbitrarily many buildings in the image. We validate the model on two publicly available aerial image datasets for building segmentation, and our results demonstrate that TDAC establishes a new state-of-the-art performance.

Keywords

Computer vision Image segmentation Active contour models Convolutional neural networks Building delineation 

Supplementary material

504453_1_En_43_MOESM1_ESM.pdf (1.8 mb)
Supplementary material 1 (pdf 1861 KB)

References

  1. 1.
    Audebert, N., Le Saux, B., Lefèvre, S.: Semantic segmentation of earth observation data using multimodal and multi-scale deep networks. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 180–196. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-54181-5_12CrossRefGoogle Scholar
  2. 2.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)CrossRefGoogle Scholar
  3. 3.
    Bischke, B., Helber, P., Folz, J., Borth, D., Dengel, A.: Multi-task learning for segmentation of building footprints with deep neural networks. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1480–1484. IEEE (2019)Google Scholar
  4. 4.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22(1), 61–79 (1997)CrossRefGoogle Scholar
  5. 5.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)CrossRefGoogle Scholar
  6. 6.
    Chen, K., et al.: Hybrid task cascade for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4974–4983 (2019)Google Scholar
  7. 7.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)CrossRefGoogle Scholar
  8. 8.
    Chen, X., Williams, B.M., Vallabhaneni, S.R., Czanner, G., Williams, R., Zheng, Y.: Learning active contour models for medical image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11632–11640 (2019)Google Scholar
  9. 9.
    Cheng, D., Liao, R., Fidler, S., Urtasun, R.: DARNet: deep active ray network for building segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7431–7439 (2019)Google Scholar
  10. 10.
    Costea, D., Marcu, A., Slusanschi, E., Leordeanu, M.: Creating roadmaps in aerial images with generative adversarial networks and smoothing-based optimization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, October 2017Google Scholar
  11. 11.
    Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., Brendel, W.: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: International Conference on Learning Representations (ICLR) (2019)Google Scholar
  12. 12.
    Gur, S., Shaharabany, T., Wolf, L.: End to end trainable active contours via differentiable rendering. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)Google Scholar
  13. 13.
    Gur, S., Wolf, L., Golgher, L., Blinder, P.: Unsupervised microvascular image segmentation using an active contours mimicking neural network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 10722–10731 (2019)Google Scholar
  14. 14.
    Hatamizadeh, A., et al.: Deep active lesion segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 98–105. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32692-0_12CrossRefGoogle Scholar
  15. 15.
    Hatamizadeh, A., Sengupta, D., Terzopoulos, D.: End-to-end deep convolutional active contours for image segmentation. arXiv preprint arXiv:1909.13359 (2019)
  16. 16.
    Hatamizadeh, A., Terzopoulos, D., Myronenko, A.: End-to-end boundary aware networks for medical image segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 187–194. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32692-0_22CrossRefGoogle Scholar
  17. 17.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  18. 18.
    Hu, P., Shuai, B., Liu, J., Wang, G.: Deep level sets for salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  19. 19.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1988)CrossRefGoogle Scholar
  20. 20.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  21. 21.
    Lankton, S., Tannenbaum, A.: Localizing region-based active contours. IEEE Trans. Image Process. 17(11), 2029–2039 (2008)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Le, T.H.N., Quach, K.G., Luu, K., Duong, C.N., Savvides, M.: Reformulating level sets as deep recurrent neural network approach to semantic segmentation. IEEE Trans. Image Process. 27(5), 2393–2407 (2018)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Li, C., Xu, C., Gui, C., Fox, M.D.: Distance regularized level set evolution and its application to image segmentation. IEEE Trans. Image Process. 19(12), 3243 (2010)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Lillesand, T., Kiefer, R.W., Chipman, J.: Remote Sensing and Image Interpretation. Wiley, Hoboken (2015)Google Scholar
  25. 25.
    Marcos, D., et al.: Learning deep structured active contours end-to-end. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8877–8885 (2018)Google Scholar
  26. 26.
    Marquez-Neila, P., Baumela, L., Alvarez, L.: A morphological approach to curvature-based evolution of curves and surfaces. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 2–17 (2013)CrossRefGoogle Scholar
  27. 27.
    Myronenko, A., Hatamizadeh, A.: Robust semantic segmentation of brain tumor regions from 3D MRIs. In: Crimi, A., Bakas, S. (eds.) BrainLes 2019. LNCS, vol. 11993, pp. 82–89. Springer, Cham (2020).  https://doi.org/10.1007/978-3-030-46643-5_8CrossRefGoogle Scholar
  28. 28.
    Rudner, T.G., et al.: Multi3Net: segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 702–709 (2019)Google Scholar
  29. 29.
    Shrivastava, N., Rai, P.K.: Remote-sensing the urban area: automatic building extraction based on multiresolution segmentation and classification. Geogr. Malays. J. Soc. Space 11(2) (2017)Google Scholar
  30. 30.
    Silberman, N., Sontag, D., Fergus, R.: Instance segmentation of indoor scenes using a coverage loss. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 616–631. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10590-1_40CrossRefGoogle Scholar
  31. 31.
    Wang, S., et al.: TorontoCity: seeing the world with a million eyes. arXiv preprint arXiv:1612.00423 (2016)
  32. 32.
    Wang, Z., Acuna, D., Ling, H., Kar, A., Fidler, S.: Object instance annotation with deep extreme level set evolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7500–7508 (2019)Google Scholar
  33. 33.
    Wu, G., et al.: Automatic building segmentation of aerial imagery using multi-constraint fully convolutional networks. Remote Sens. 10(3), 407 (2018)CrossRefGoogle Scholar
  34. 34.
    Xu, Y., Wu, L., Xie, Z., Chen, Z.: Building extraction in very high resolution remote sensing imagery using deep learning and guided filters. Remote Sens. 10(1), 144 (2018)CrossRefGoogle Scholar
  35. 35.
    Zhang, P., Ke, Y., Zhang, Z., Wang, M., Li, P., Zhang, S.: Urban land use and land cover classification using novel deep learning models based on high spatial resolution satellite imagery. Sensors 18(11), 3717 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Computer Science DepartmentUniversity of CaliforniaLos AngelesUSA

Personalised recommendations