A Comparison and Strategy of Semantic Segmentation on Remote Sensing Images

  • Junxing HuEmail author
  • Ling Li
  • Yijun Lin
  • Fengge Wu
  • Junsuo Zhao
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1074)


In recent years, with the development of aerospace technology, we use more and more images captured by satellites to obtain information. But a large number of useless raw images, limited data storage resource and poor transmission capability on satellites hinder our use of valuable images. Therefore, it is necessary to deploy an on-orbit semantic segmentation model to filter out useless images before data transmission. In this paper, we present a detailed comparison on the recent deep learning models. Considering the computing environment of satellites, we compare methods from accuracy, parameters and resource consumption on the same public dataset. And we also analyze the relation between them. Based on experimental results, we further propose a viable on-orbit semantic segmentation strategy. It will be deployed on the TianZhi-2 satellite which supports deep learning methods and will be lunched soon.


Semantic segmentation Deep learning Remote sensing images 


  1. 1.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 4, 321–331 (1988)CrossRefGoogle Scholar
  2. 2.
    Meyer, F., Beucher, S.: Morphological segmentation. J. Vis. Commun. Image Represent. 1, 21–46 (1990)CrossRefGoogle Scholar
  3. 3.
    Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in ND images. In: 2001 Proceedings of the Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 1, pp. 105–112 (2001)Google Scholar
  4. 4.
    Kampffmeyer, M., Salberg, A. B., Jenssen, R.: Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–9 (2016)Google Scholar
  5. 5.
    Volpi, M., Tuia, D.: Dense semantic labeling of subdecimeter resolution images with convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 55(2), 881–893 (2017)CrossRefGoogle Scholar
  6. 6.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  8. 8.
    Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Garcia-Rodriguez, J.: A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857 (2017)
  9. 9.
    Ball, J.E., Anderson, D.T., Chan, C.S.: Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community. J. Appl. Remote Sens. 11(4), 042609 (2017)CrossRefGoogle Scholar
  10. 10.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Cham (2015)Google Scholar
  11. 11.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)CrossRefGoogle Scholar
  12. 12.
    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2881–2890 (2017)Google Scholar
  13. 13.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)CrossRefGoogle Scholar
  14. 14.
    Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision 111(1), 98–136 (2015)CrossRefGoogle Scholar
  15. 15.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  16. 16.
    Huang, B., Lu, K., Audebert, N., Khalel, A., Tarabalka, Y., Malof, J., Lefèvre, S.: Large-scale semantic classification: outcome of the first year of Inria aerial image labeling benchmark. In: IEEE International Geoscience and Remote Sensing Symposium-CIGARSS 2018 (2018)Google Scholar
  17. 17.
    Lu, K., Sun, Y., Ong, S.H.: Dual-resolution U-Net: building extraction from aerial images. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 489–494. IEEE (2018)Google Scholar
  18. 18.
    Bischke, B., Helber, P., Folz, J., Borth, D., Dengel, A., Waterman, M.S.: Multi-task learning for segmentation of building footprints with deep neural networks. arXiv preprint arXiv:1709.05932 (2017)
  19. 19.
    Holschneider, M., Kronland-Martinet, R., Morlet, J., Tchamitchian, P.: A real-time algorithm for signal analysis with the help of the wavelet transform. In: Wavelets, pp. 286–297. Springer, Heidelberg (1990)Google Scholar
  20. 20.
    Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Advances in Neural Information Processing Systems, pp. 109–117 (2011)Google Scholar
  21. 21.
    Maggiori, E., Tarabalka, Y., Charpiat, G., Alliez, P.: Can semantic labeling methods generalize to any city? The Inria aerial image labeling benchmark. In: IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Junxing Hu
    • 1
    • 2
    Email author
  • Ling Li
    • 1
    • 2
  • Yijun Lin
    • 1
    • 2
  • Fengge Wu
    • 2
  • Junsuo Zhao
    • 2
  1. 1.University of Chinese Academy of SciencesBeijingChina
  2. 2.Institute of Software Chinese Academy of Sciences (ISCAS)BeijingChina

Personalised recommendations