Skip to main content
SpringerLink
Log in

ContourNet: Research on Contour Based Nighttime Semantic Segmentation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Due to the scarcity of nighttime semantic segmentation datasets and the high demand for network models, the development of semantic segmentation of nighttime scenes is still very slow. This paper proposes a new network model, ContourNet, which can model multi-level features. In addition, a separate contour network module is designed to accurately predict object contours, improving performance for objects far away, small, or with high contour continuity. A large number of experiments demonstrate that the ContourNet proposed in this paper can significantly improve the semantic segmentation ability of existing models for nighttime images, and can also improve the semantic segmentation accuracy of daytime images to a certain extent, with good generalization abilities. Specifically, after adding the contour module in this article, MIoU has increased by 5.1% on the night dataset Rebecca; MIoU has increased by 2.5% on the daytime dataset CamVid.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Google Scholar 

  2. Huang K, Shi B, Li X, Li X, Huang S, Li Y (2022) Multi-modal sensor fusion for auto driving perception: a survey. arXiv e-prints

  3. Luo S, Dai H, Shao L, Ding Y (2021) M3dssd: monocular 3d single stage object detector. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 6141–6150

  4. Yang Z, Xinjun H, Yudong Y (2021) Design and research of industrial robot control system based on machine vision. In: 2021 5th international conference on electronics, communication and aerospace technology (ICECA), pp. 209–212

  5. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 1–1

  6. Peng D, Lei Y, Hayat M, Guo Y, Li W (2022) Semantic-aware domain generalized segmentation. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 2584–2595

  7. Zhang J, Cao Y, Wu Q (2021) Vector of locally and adaptively aggregated descriptors for image feature representation. Pattern Recogn 116(4):107952

    Article  Google Scholar 

  8. Yu J, Tan M, Zhang H, Rui Y, Tao D (2022) Hierarchical deep click feature prediction for fine-grained image recognition. IEEE Trans Pattern Anal Mach Intell 44(2):563–578. https://doi.org/10.1109/TPAMI.2019.2932058

    Article  Google Scholar 

  9. Kittler J (1983) On the accuracy of the sobel edge detector. Image Vis Comput 1(1):37–42

    Article  Google Scholar 

  10. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  11. Zhang J, Fan J, Yang J, Yu J (2022) Semisupervised image classification by mutual learning of multiple self-supervised models. Int J Intell Syst 37(5):3117–3141

    Article  Google Scholar 

  12. Wei S, Wang X, Yan W, Xiang B, Zhang Z (2015) Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection. Comput Vis Pattern Recognit

  13. Xie S, Tu Z (2015) Holistically-nested edge detection. Int J Comput Vision 125(1–3):3–18

    MathSciNet  Google Scholar 

  14. Soria X, Riba E, Sappa A (2020) Dense extreme inception network: towards a robust cnn model for edge detection. In: Workshop on applications of computer vision

  15. Teichmann M, Weber M, Zöllner JM, Cipolla R, Urtasun R (2016) Multinet: real-time joint semantic reasoning for autonomous driving. In: 2018 IEEE intelligent vehicles symposium (IV) pp. 1013–1020

  16. Cheng D, Meng G, Xiang S, Pan C (2017) Fusionnet: edge aware deep convolutional networks for semantic segmentation of remote sensing harbor images. IEEE J Sel Top Appl Earth Observ Remote Sensing 99:1–15

    Google Scholar 

  17. Guo X, Yu L, Ling H (2016) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 99:1–1

    MATH  Google Scholar 

  18. Wu W, Weng J, Zhang P, Wang X, Yang W, Jiang J (2022) Uretinex-net: retinex-based deep unfolding network for low-light image enhancement. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 5891–5900

  19. Ma L, Ma T, Liu R, Fan X, Luo Z (2022) Toward fast, flexible, and robust low-light image enhancement. 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 5627–5636

  20. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778

  21. Tan X, Xu K, Cao Y, Zhang Y, Ma L, Lau R (2021) Night-time scene parsing with a large real dataset. IEEE Trans Image Process

  22. Brostow GJ, Fauqueur J, Cipolla R (2009) Semantic object classes in video: a high-definition ground truth database. Pattern Recogn Lett 30(2):88–97

    Article  Google Scholar 

  23. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Bisenet: bilateral segmentation network for real-time semantic segmentation. In: European conference on computer vision

  24. Zhao J, Liu J, Fan DP, Cao Y, Yang J, Cheng MM (2019) Egnet: edge guidance network for salient object detection. 2019 IEEE/CVF international conference on computer vision (ICCV), pp. 8778–8787

  25. Nirkin Y, Wolf L, Hassner T (2021) Hyperseg: patch-wise hypernetwork for real-time semantic segmentation. Comput Vis Pattern Recognit

  26. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR) pp. 3213–3223

  27. Zhao H, Qi X, Shen X, Shi J, Jia J (2017) ICNet for real-time semantic segmentation on high-resolution images

  28. Li H, Xiong P, Fan H, Sun J (2020) Dfanet: deep feature aggregation for real-time semantic segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  29. Hu P, Heilbron FC, Wang O, Lin ZL, Sclaroff S, Perazzi F (2020) Temporally distributed networks for fast video semantic segmentation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 8815–8824

  30. Li H, Liu C, Yang Y (2023) Layernet: a one-step layered network for semantic segmentation at night. IEEE Comput Graph Appl

  31. Fu J, Liu J, Tian H, Fang Z, Lu H (2018) Dual attention network for scene segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 3141–3149

Download references

Acknowledgements

This work was supported in part by the Open Project of the Key Lab of Enterprise Informationization and Internet of Things of Sichuan Province Grant Number 2022WZJ01, Graduate innovation fund of Sichuan University of Science and Engineering Grant Number Y2021099, and Postgraduate course construction project of Sichuan University of Science and Engineering Grant Number YZ202103.

Author information

Author notes

  1. Changjiang Liu and Hao Li have contributed equally to this work.

Authors and Affiliations

  1. School of Mathematics and Statistics, Sichuan University of Science and Engineering, Zigong, 643000, China

    Yang Yang

  2. School of Mathematics and Statistics, Key Laboratory of Higher Education of Sichuan Province for Enterprise Informationalization and Internet of Things, Sichuan University of Science and Engineering, Zigong, 643000, China

    Changjiang Liu

  3. School of Mathematics and Statistics, Sichuan University of Science and Engineering, Zigong, 643000, China

    Hao Li

Authors
  1. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changjiang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YY wrote the main manuscript, YY, LH, and LC conducted relevant experiments, LH completed the image production in the paper, LC made revisions and checks on the paper.

Corresponding author

Correspondence to Changjiang Liu.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Liu, C. & Li, H. ContourNet: Research on Contour Based Nighttime Semantic Segmentation. Neural Process Lett 55, 11089–11107 (2023). https://doi.org/10.1007/s11063-023-11366-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11366-2

Keywords

Search

Navigation