Skip to main content
SpringerLink
Log in

Improved Mars Terrain Segmentation in Terms of Style Transfer and Network Architecture

  • Original Paper
  • Published:
International Journal of Aeronautical and Space Sciences Aims and scope Submit manuscript

Abstract

Mars terrain segmentation facilitates full comprehension of Martian terrain and plays an important role in the autonomous planning of Mars rover missions. However, existing methods face several issues. Since Martian terrains vary greatly over different regions, segmentation performance tends to degrade severely because of differences from the training terrain. Furthermore, it becomes necessary to utilize unannotated data due to the difficulty of annotating data as required for the Mars terrain segmentation task. Finally, mis-segmentation of objects frequently occurs. To address these challenges, three key approaches are proposed in this paper: (1) We propose a new neural style transfer method named SA-CCPL, which converts images from source domain style to target domain style without losing the image details. (2) We implement a thing-class and stuff-class perception enhancement (TSPE) module, optimizing the network’s scene-comprehension capability. (3) An ensemble of multi-scale models (EMSM) pseudo-label construction method is proposed to improve pseudo-label reliability. Experiments demonstrate that our proposed method is effective in improving the performance of Mars terrain segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data that support the findings of this study are available on request from the corresponding author, Yan Xing, upon reasonable request.

References

  1. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66. https://doi.org/10.1109/tsmc.1979.4310076

    Article  Google Scholar 

  2. Rother C, Kolmogorov V, Blake A (2004) “GrabCut” interactive foreground extraction using iterated graph cuts. ACM transactions on graphics (TOG) 23(3):309–314. https://doi.org/10.1109/tsmc.1979.4310076

    Article  Google Scholar 

  3. Achanta R et al (2012) Superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282. https://doi.org/10.1109/tpami.2012.120

    Article  Google Scholar 

  4. Kerner HR et al (2019) Novelty detection for multispectral images with application to planetary exploration. Proc AAAI Conf Artif Intell. https://doi.org/10.1609/aaai.v33i01.33019484

    Article  Google Scholar 

  5. Lai X et al (2021) Semi-supervised semantic segmentation with directional context-aware consistency. Proc IEEE/CVF Conf Comp Vis Pattern Recogn. https://doi.org/10.1109/cvpr46437.2021.00126

    Article  Google Scholar 

  6. Swan RM et al (2021) AI4mars: A dataset for terrain-aware autonomous driving on mars. Proc IEEE/CVF Conf Comp Vis Pattern Recogn. https://doi.org/10.1109/CVPRW53098.2021.00226

    Article  Google Scholar 

  7. Cordts M et al (2016) The cityscapes dataset for semantic urban scene understanding. Proc IEEE Conf Comput Vis Pattern Recognit. https://doi.org/10.1109/CVPR.2016.350

    Article  Google Scholar 

  8. Hung, Wei-Chih, et al (2018) Adversarial learning for semi-supervised semantic segmentation. arXiv preprint arXiv:1802.07934

  9. Feng Z et al (2022) DMT: Dynamic mutual training for semi-supervised learning. Pattern Recogn 130:108777. https://doi.org/10.1016/j.patcog.2022.108777

    Article  Google Scholar 

  10. Yassine Ouali, Celine Hudelot, Myriam Tami (2020) Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 12674–12684

  11. Rothrock B et al (2016) Spoc: Deep learning-based terrain classification for mars rover missions. AIAA SPACE 2016:5539. https://doi.org/10.2514/6.2016-5539

    Article  Google Scholar 

  12. Goh E, Chen J, Wilson B (2022) Mars terrain segmentation with less labels. IEEE Aerospace Conference (AERO). https://doi.org/10.1109/AERO53065.2022.9843245

    Article  Google Scholar 

  13. Zhang, Jiahang, et al (2022) S5Mars: Self-Supervised and Semi-Supervised Learning for Mars Segmentation. arXiv preprint arXiv:2207.01200. 2022

  14. Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge (2016) Image style transfer using convolutional neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2414–2423.

  15. Gatys, Leon A., et al (2016) Preserving color in neural artistic style transfer. https://doi.org/10.48550/arXiv.1606.05897

  16. Wu, Zijie, et al (2022) CCPL: Contrastive Coherence Preserving Loss for Versatile Style Transfer. Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVI. Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-19787-1_11

  17. Wang, Xiaolong, and Abhinav Gupta (2016) Generative image modeling using style and structure adversarial networks. Computer Vision–ECCV 2016: 14th European Conference. https://doi.org/10.1007/978-3-319-46493-0_20

  18. Zhu J-Y et al (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. Proc IEEE Int Conf Comp Vis. https://doi.org/10.1007/s10489-022-04389-0

    Article  Google Scholar 

  19. He, Kaiming, et al (2016) Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778

  20. Long, Jonathan, Evan Shelhamer, and Trevor Darrell (2015) Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3431–3440

  21. Ronneberger, Olaf, Philipp Fischer, and Thomas Brox (2015) U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Part III 18. pp 234-241

  22. Wang J et al (2020) Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell 43(10):3349–3364

    Article  Google Scholar 

  23. Chen, Liang-Chieh, et al (2014) Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. https://doi.org/10.48550/arXiv.1412.7062

  24. Mehta, Sachin, and Mohammad Rastegari. (2022) Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. https://doi.org/10.48550/arXiv.2110.02178

  25. Chen, Liang-Chieh, et al (2017) Rethinking atrous convolution for semantic image segmentation. https://doi.org/10.48550/arXiv.1706.05587

  26. Zhao, Hengshuang, et al (2017) Pyramid scene parsing network. Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2881–2890

  27. Xiao, Tete, et al (2018) Unified perceptual parsing for scene understanding. Proceedings of the European conference on computer vision (ECCV). pp 418–434

  28. Chen, Liang-Chieh, et al (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European conference on computer vision (ECCV). pp 801–818

  29. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59. https://doi.org/10.1016/0031-3203(95)00067-4

    Article  Google Scholar 

  30. Simonyan, Karen, and Andrew Zisserman (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  31. Yu C et al (2021) Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vision 129:3051–3068

    Article  Google Scholar 

  32. Torralba A, Russell BC, Yuen J (2010) Labelme: Online image annotation and applications. Proc IEEE 98(8):1467–1484

    Article  Google Scholar 

  33. Gonzalez, Ramon, and Karl Iagnemma (2018) Deepterramechanics: Terrain classification and slip estimation for ground robots via deep learning. arXiv preprint arXiv:1806.07379

  34. Dimastrogiovanni M, Cordes F, Reina G (2020) Terrain estimation for planetary exploration robots. Appl Sci 10(17):6044. https://doi.org/10.3390/app10176044

    Article  Google Scholar 

  35. Redmon, Joseph, and Ali Farhadi (2018) Yolov3: An incremental improvement. https://doi.org/10.48550/arXiv.1804.02767

  36. Yuan Y, Chen X, Wang J (2020) Object-contextual representations for semantic segmentation. Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, Proceedings, Part VI 16. Springer International Publishing, 2020: 173-190

  37. Lipton, Zachary, Yu-Xiang Wang, and Alexander Smola (2018) Detecting and correcting for label shift with black box predictors. International conference on machine learning. PMLR. pp 3122–3130

  38. Chunwei Ma, Zhanghexuan Ji, and Mingchen Gao (2019) Neural style transfer improves 3D cardiovascular MR image segmentation on inconsistent data. Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. Springer International Publishing. pp 128–136

  39. Kline, Timothy L (2021) Improving domain generalization in segmentation models with neural style transfer. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). pp 1324–1328

  40. Hong, Jun-Pyo, et al. (2020) Single image deblurring based on auxiliary Sobel loss function. 2020 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). pp 1–3

  41. Yang L, Zhuo W, Qi L et al (2022) ST++: Make self-training work better for semi-supervised semantic segmentation. Proc IEEE/CVF Conf Comp Vis Pattern Recogn 2022:4268–4277

    Google Scholar 

  42. Howard A, Sandler M, Chu G, et al (2019) Searching for mobilenetv3. Proceedings of the IEEE/CVF international conference on computer vision. pp 1314–1324

  43. Liu H et al (2023) RockFormer: A U-Shaped Transformer Network for Martian Rock Segmentation. IEEE Trans Geosci Remote Sens 61:1–16

    Article  Google Scholar 

  44. Atha D, Swan R M, Didier A, et al (2022) Multi-mission terrain classifier for safe rover navigation and automated science. 2022 IEEE Aerospace Conference (AERO). IEEE, pp 1–13

  45. Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., & Girdhar, R (2022) Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 1290–1299

  46. Cheng B, Schwing A, Kirillov A (2021) Per-pixel classification is not all you need for semantic segmentation. Adv Neural Inf Process Syst 34:17864–17875

    Google Scholar 

Download references

Acknowledgements

This work is supported in part by National Natural Science Foundation of China (52372423), in part by Key Laboratory of Space Flight Dynamics Technology (KJW6142210210309) and in part by Key Research and Development Projects in Zhejiang Province (No. 2022C01005, 2022C01082).

Author information

Authors and Affiliations

  1. Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province College of Information Engineering, China Jiliang University, Hangzhou, China

    Li Yang & Guangbin Huang

  2. Beijing Institute of Control Engineering, Beijing, China

    Yan Xing

Authors
  1. Li Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangbin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Xing.

Ethics declarations

Conflict of Interest

We declare that there is no conflict of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Huang, G. & Xing, Y. Improved Mars Terrain Segmentation in Terms of Style Transfer and Network Architecture. Int. J. Aeronaut. Space Sci. 25, 1121–1134 (2024). https://doi.org/10.1007/s42405-023-00702-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42405-023-00702-4

Keywords

Search

Navigation