Skip to main content
SpringerLink
Log in

A Graph-Involved Lightweight Semantic Segmentation Network

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14431))

Included in the following conference series:

  • 394 Accesses

Abstract

To extract cues for pixelwise segmentation in an efficient way, this paper proposes a lightweight model that involves graph structure in the convolutional network. First, a cross-layer module is designed to adaptively aggregate hierarchical features according to the feature relations within multi-scale receptive fields. Second, a graph-involved head is presented, capturing long-range channel and feature dependencies in two sub-domains. Specifically, channel dependency is acquired in a compact spatial domain for context-aware information, while the feature dependency is obtained in the graph feature domain for category-aware representation. Afterwards, by fusing the features with long-range dependencies, the network outputs the segmentation results after a learning-free upsampling layer. Experimental results present that this model remains light while achieving competitive performances in segmentation, proving the effectiveness and efficiency of the proposed sub-modules. (https://github.com/xia-xx-cv/Graph-Lightweight-SemSeg/).

This work was supported by National Natural Science Foundation 62162029 and 62271237, Jiangxi Natural Science Foundation 20224BAB212010 and Foundation for Distinguished Young Scholars of Jiangxi Province 20224ACB212005.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Atif, N., Mazhar, S., Sarma, D., Bhuyan, M., Ahamed, S.R.: Efficient context integration through factorized pyramidal learning for ultra-lightweight semantic segmentation. arXiv preprint arXiv:2302.11785 (2023)

  2. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  3. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  4. Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., Yuille, A.: Detect what you can: detecting and representing objects using holistic models and body parts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1971–1978 (2014)

    Google Scholar 

  5. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3223 (2016)

    Google Scholar 

  6. Cui, C., et al.: PP-LCNet: a lightweight CPU convolutional neural network. arXiv preprint arXiv:2109.15099 (2021)

  7. Ding, L., Tang, H., Bruzzone, L.: LANet: local attention embedding to improve the semantic segmentation of remote sensing images. IEEE Trans. Geosci. Remote Sens. 59(1), 426–435 (2020)

    Article  Google Scholar 

  8. Emara, T., Abd El Munim, H.E., Abbas, H.M.: LiteSeg: a novel lightweight convnet for semantic segmentation. In: 2019 Digital Image Computing: Techniques and Applications (DICTA), pp. 1–7. IEEE (2019)

    Google Scholar 

  9. Fu, J., et al.: Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)

    Google Scholar 

  10. Gao, H., Wang, Z., Ji, S.: ChannelNets: compact and efficient convolutional neural networks via channel-wise convolutions. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  11. Guo, M.H., Lu, C.Z., Hou, Q., Liu, Z., Cheng, M.M., Hu, S.M.: SegNeXt: rethinking convolutional attention design for semantic segmentation. In: Proceedings of 36th Conference on Neural Information Processing Systems (NeurIPS), pp. 603–612 (2022)

    Google Scholar 

  12. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  13. Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 603–612 (2019)

    Google Scholar 

  14. Jeevan, P., Viswanathan, K., Sethi, A.: WaveMix-Lite: a resource-efficient neural network for image analysis. arXiv preprint arXiv:2205.14375 (2022)

  15. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  16. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  17. Li, H., Qiu, K., Chen, L., Mei, X., Hong, L., Tao, C.: SCAttNet: semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images. IEEE Geosci. Remote Sens. Lett. 18(5), 905–909 (2020)

    Article  Google Scholar 

  18. Li, Y., Li, X., Xiao, C., Li, H., Zhang, W.: EACNet: enhanced asymmetric convolution for real-time semantic segmentation. IEEE Signal Process. Lett. 28, 234–238 (2021)

    Article  Google Scholar 

  19. Liu, J., Zhou, Q., Qiang, Y., Kang, B., Wu, X., Zheng, B.: FDDWNet: a lightweight convolutional neural network for real-time semantic segmentation. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2373–2377. IEEE (2020)

    Google Scholar 

  20. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  21. Lu, Y., Chen, Y., Zhao, D., Chen, J.: Graph-FCN for image semantic segmentation. In: Lu, H., Tang, H., Wang, Z. (eds.) ISNN 2019. LNCS, vol. 11554, pp. 97–105. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22796-8_11

    Chapter  Google Scholar 

  22. Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., Hajishirzi, H.: ESPNet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 561–580. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_34

    Chapter  Google Scholar 

  23. Nickel, M., Kiela, D.: Poincaré embeddings for learning hierarchical representations. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 6341–6350. Curran Associates Inc., Red Hook, NY, USA (2017)

    Google Scholar 

  24. Paszke, A., Chaurasia, A., Kim, S., Culurciello, E.: ENet: a deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147 (2016)

  25. Qin, J., Wu, J., Li, M., Xiao, X., Zheng, M., Wang, X.: Multi-granularity distillation scheme towards lightweight semi-supervised semantic segmentation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) European Conference on Computer Vision, vol. 13690. pp. 481–498. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20056-4_28

  26. Romera, E., Alvarez, J.M., Bergasa, L.M., Arroyo, R.: ERFNet: efficient residual factorized convnet for real-time semantic segmentation. IEEE Trans. Intell. Transp. Syst. 19(1), 263–272 (2017)

    Article  Google Scholar 

  27. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  28. Shao, N., Cui, Y., Liu, T., Wang, S., Hu, G.: Is graph structure necessary for multi-hop question answering? In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7187–7192. Association for Computational Linguistics, November 2020. https://doi.org/10.18653/v1/2020.emnlp-main.583. https://aclanthology.org/2020.emnlp-main.583

  29. Sun, L., Cheng, S., Zheng, Y., Wu, Z., Zhang, J.: SpaNet: successive pooling attention network for semantic segmentation of remote sensing images. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 15, 4045–4057 (2022)

    Article  Google Scholar 

  30. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  31. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  32. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

    Google Scholar 

  33. Wu, T., Tang, S., Zhang, R., Cao, J., Zhang, Y.: CGNet: a light-weight context guided network for semantic segmentation. IEEE Trans. Image Process. 30, 1169–1179 (2020)

    Article  Google Scholar 

  34. Wu, Y., et al.: Bidirectional graph reasoning network for panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9080–9089 (2020)

    Google Scholar 

  35. Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.: SegFormer: simple and efficient design for semantic segmentation with transformers. Adv. Neural. Inf. Process. Syst. 34, 12077–12090 (2021)

    Google Scholar 

  36. Yang, C., et al.: Pruning parameterization with bi-level optimization for efficient semantic segmentation on the edge. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15402–15412 (2023)

    Google Scholar 

  37. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 334–349. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_20

    Chapter  Google Scholar 

  38. Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., Wang, J.: OCNet: object context network for scene parsing. arXiv preprint arXiv:1809.00916 (2018)

  39. Zhang, H., et al.: Context encoding for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7151–7160 (2018)

    Google Scholar 

  40. Zhang, L., Li, X., Arnab, A., Yang, K., Tong, Y., Torr, P.H.: Dual graph convolutional network for semantic segmentation. In: Proceedings of British Machine Vision Conference (BMVC), pp. 10.1–10.14 (2019)

    Google Scholar 

  41. Zhang, W., et al.: TopFormer: token pyramid transformer for mobile semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12083–12093 (2022)

    Google Scholar 

  42. Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)

    Google Scholar 

  43. Zhang, X., Du, B., Wu, Z., Wan, T.: LAANet: lightweight attention-guided asymmetric network for real-time semantic segmentation. Neural Comput. Appl. 34(5), 3573–3587 (2022)

    Article  Google Scholar 

  44. Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for real-time semantic segmentation on high-resolution images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 418–434. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_25

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Jiangxi University of Finance and Economics, Nanchang, 330013, China

    Xue Xia, Jiayu You & Yuming Fang

Authors
  1. Xue Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiayu You
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuming Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuming Fang .

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  2. Xiamen University, Xiamen, China

    Hanzi Wang

  3. Beijing University of Posts and Telecommunications, Beijing, China

    Zhanyu Ma

  4. Sun Yat-sen University, Guangzhou, China

    Weishi Zheng

  5. Peking University, Beijing, China

    Hongbin Zha

  6. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  8. Xiamen University, Xiamen, China

    Rongrong Ji

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xia, X., You, J., Fang, Y. (2024). A Graph-Involved Lightweight Semantic Segmentation Network. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14431. Springer, Singapore. https://doi.org/10.1007/978-981-99-8540-1_30

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8540-1_30

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8539-5

  • Online ISBN: 978-981-99-8540-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation