Skip to main content
SpringerLink
Log in

Tensor-based multi-feature affinity graph learning for natural image segmentation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In the previous image segmentation methods based on affinity graph learning, it is difficult to obtain clear contour boundaries in the process of image preprocessing, over-segmentation leads to the separation of local regions and most traditional affinity graph learning methods cluster and segment with a single feature of natural images, ignoring the associative characteristics of multiple types of features and cannot effectively utilize the useful information of multiple features of natural images. Aiming at such problems, this paper proposes a tensor-based multi-feature affinity graph learning method for natural image segmentation (TRMFAL). First, the adaptive morphological rebuilding watershed transform is applied to original natural image, the obtained superpixel image contains a lot of boundary contour information, and the fusion of local regions is relatively good; secondly, extract multi-class features from the superpixel blocks in the superpixel image, effectively utilize the features of different characteristics, and integrate them into a multi-feature data matrix according to the corresponding rules; Then, a tensor-based multi-feature affinity graph learning algorithm is proposed, in which tensor is introduced in the algorithm to effectively obtain the higher-order information of image data, and use the projection matrix to embed the original data in the low-dimensional space to decrease the dimension, while minimizing the residual error of each view feature and assign appropriate weights according to the importance of the feature information; finally, use spectral clustering performs clustering and segmentation on affinity graphs to obtain final clustering results and segmented images. In addition, an optimization iterative algorithm based on the alternating multiplier direction method is designed, which effectively solves the problem of solving the TRMFAL model. Sufficient experimental comparisons are carried out on multiple public datasets, and the results prove that the proposed method achieves the optimal clustering performance and segmentation effect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Data will be made available on reasonable request.

Notes

  1. http://vision.ucsd.edu/ leekc/ExtYaleDatabase/ExtYaleB.html.

  2. http://research.microsoft.com/en-us/projects/objectclassrecognition/.

  3. http://archive.ics.uci.edu/ml/datasets/Multiple+Features.

  4. http://mlg.ucd.ie/datasets.

References

  1. Cheng Y, Li B (2021) Image segmentation technology and its application in digital image processing. In: 2021 IEEE Asia-Pacific conference on image processing, electronics and computers (IPEC)

  2. Das A, Dhal KG, Ray S, Gálvez J (2022) Histogram-based fast and robust image clustering using stochastic fractal search and morphological reconstruction. Neural Comput Appl 34(6):4531–4554

    Google Scholar 

  3. Merényi E, Taylor J (2020) Empowering graph segmentation methods with soms and conn similarity for clustering large and complex data. Neural Comput Appl 32(24):18161–18178

    Google Scholar 

  4. Zhang G, Ge Y, Dong Z, Hao W, Yuhui Z, Chen S (2021) Deep high-resolution representation learning for cross-resolution person re-identification. IEEE Trans Image Process 30:8913–8925

    Google Scholar 

  5. Zhang G, Luo Z, Chen Y, Zheng Y, Lin W (2022) Illumination unification for person re-identification. IEEE Trans Circuits Syst Video Technol 32(10):6766–6777

    Google Scholar 

  6. Zhang Y, Liu M, He J, Pan F, Guo Y (2021) Affinity fusion graph-based framework for natural image segmentation. IEEE Trans Multimed 24:440–450

    Google Scholar 

  7. Pereyra M, McLaughlin S (2017) Fast unsupervised bayesian image segmentation with adaptive spatial regularisation. IEEE Trans Image Process 26(6):2577–2587

    MATH  MathSciNet  Google Scholar 

  8. Hettiarachchi R, Peters JF (2017) Voronoï region-based adaptive unsupervised color image segmentation. Pattern Recognit 65:119–135

    Google Scholar 

  9. Francis J, Johnson A, Madathil B, George SN (2020) A joint sparse and correlation induced subspace clustering method for segmentation of natural images. In: 2020 IEEE 17th India council international conference (INDICON)

  10. Francis J, Baburaj M, George SN (2022) An l \(1/2\) and graph regularized subspace clustering method for robust image segmentation. ACM Trans Multimed Comput Commun Appl (TOMM) 18(2):1–24

    Google Scholar 

  11. Xue X, Wang X, Zhang X, Wang J, Liu Z (2021) Image segmentation based on non-convex low rank multiple kernel clustering. In: CAAI International conference on artificial intelligence, p 420–431. Springer

  12. Zhu H, Vial R, Lu S, Peng X, Fu H, Tian Y, Cao X (2018) Yotube: searching action proposal via recurrent and static regression networks. IEEE Trans Image Process 27(6):2609–2622

    MATH  MathSciNet  Google Scholar 

  13. Xiao Y, Wei J, Wang J, Ma Q, Zhe S, Tasdizen T (2020) Graph constraint-based robust latent space low-rank and sparse subspace clustering. Neural Comput Appl 32(12):8187–8204

    Google Scholar 

  14. Xu S, Feng L, Liu S, Zhou J, Qiao H (2020) Multi-feature weighting neighborhood density clustering. Neural Comput Appl 32(13):9545–9565

    Google Scholar 

  15. Elhamifar E, Vidal R (2013) Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781

    Google Scholar 

  16. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2012) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Google Scholar 

  17. Vidal René, Favaro Paolo (2014) Low rank subspace clustering (lrsc). Pattern Recog Lett 43:47–61

    Google Scholar 

  18. Peng X, Yu Z, Yi Z, Tang H (2016) Constructing the l2-graph for robust subspace learning and subspace clustering. IEEE Trans Cybern 47(4):1053–1066

    Google Scholar 

  19. Lu C, Feng J, Lin Z, Mei T, Yan S (2018) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Mach Intell 41(2):487–501

    Google Scholar 

  20. Zhu X, Zhang S, Li Y, Zhang J, Yang L, Fang Y (2018) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng 31(8):1532–1543

    Google Scholar 

  21. Zhang X, Chen B, Sun H, Liu Z, Ren Z, Li Y (2019) Robust low-rank kernel subspace clustering based on the schatten p-norm and correntropy. IEEE Trans Knowl Data Eng 32(12):2426–2437

    MATH  Google Scholar 

  22. Xue X, Zhang X, Feng X, Sun H, Chen W, Liu Z (2020) Robust subspace clustering based on non-convex low-rank approximation and adaptive kernel. Inf Sci 513:190–205

    MATH  MathSciNet  Google Scholar 

  23. Zhang X, Xue X, Sun H, Liu Z, Guo L, Guo X (2021) Robust multiple kernel subspace clustering with block diagonal representation and low-rank consensus kernel. Knowl Based Syst 227:107243

    Google Scholar 

  24. Guo L, Zhang X, Liu Z, Xue X, Wang Q, Zheng S (2021) Robust subspace clustering based on automatic weighted multiple kernel learning. Inf Sci 573:453–474

    MathSciNet  Google Scholar 

  25. Xia R, Pan Y, Du L, Yin J (2014) Robust multi-view spectral clustering via low-rank and sparse decomposition. In: Proceedings of the AAAI conference on artificial intelligence, vol 28

  26. Cao X, Zhang C, Fu H, Liu S, Zhang H (2015) Diversity-induced multi-view subspace clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 586–594

  27. Zhao J, Xie X, Xin X, Sun S (2017) Multi-view learning overview: recent progress and new challenges. Inf Fusion 38:43–54

    Google Scholar 

  28. Lu C, Yan S, Lin Z (2016) Convex sparse spectral clustering: single-view to multi-view. IEEE Trans Image Process 25(6):2833–2843

    MATH  MathSciNet  Google Scholar 

  29. Wang S, Liu X, Zhu E, Tang C, Liu J, Hu J, Xia J, Yin J (2019) Multi-view clustering via late fusion alignment maximization. In: IJCAI, p 3778–3784

  30. Chen Y, Wang S, Zheng F, Cen Y (2020) Graph-regularized least squares regression for multi-view subspace clustering. Knowl Based Syst 194:105482

    Google Scholar 

  31. Kang Z, Shi G, Huang S, Chen W, Pu X, Zhou JT, Xu Z (2020) Multi-graph fusion for multi-view spectral clustering. Knowl Based Syst 189:105102

    Google Scholar 

  32. Zhang X, Wang J, Xue X, Sun H, Zhang J (2022) Confidence level auto-weighting robust multi-view subspace clustering. Neurocomputing 475:38–52

    Google Scholar 

  33. Chen Y, Xiao X, Peng C, Lu G, Zhou Y (2021) Low-rank tensor graph learning for multi-view subspace clustering. IEEE Trans Circuits Syst Video Technol 32(1):92–104

    Google Scholar 

  34. Zhang C, Fu H, Liu S, Liu G, Cao X (2015) Low-rank tensor constrained multiview subspace clustering. In: Proceedings of the IEEE international conference on computer vision, p 1582–1590

  35. Xie Y, Tao D, Zhang W, Liu Y, Zhang L, Qu Y (2018) On unifying multi-view self-representations for clustering by tensor multi-rank minimization. Int J Comput Vis 126(11):1157–1179

    MATH  MathSciNet  Google Scholar 

  36. Xie Y, Liu J, Qu Y, Tao D, Zhang W, Dai L, Ma L (2020) Robust kernelized multiview self-representation for subspace clustering. IEEE Trans Neural Netw Learn Syst 32(2):868–881

    MathSciNet  Google Scholar 

  37. Chen Y, Xiao X, Zhou Y (2019) Jointly learning kernel representation tensor and affinity matrix for multi-view clustering. IEEE Trans Multimed 22(8):1985–1997

    Google Scholar 

  38. Zhang X, Tan Z, Sun H, Wang Z, Qin M (2021) Orthogonal low-rank projection learning for robust image feature extraction. In: IEEE Transactions on Multimedia, p 1–1

  39. Zhou T, Zhang C, Peng X, Bhaskar H, Yang J (2019) Dual shared-specific multiview subspace clustering. IEEE Trans Cybern 50(8):3517–3530

    Google Scholar 

  40. Chen MS, Huang L, Wang CD, Huang D (2020) Multi-view clustering in latent embedding space. Proceedings AAAI Conf Artif Intell 34:3513–3520

    Google Scholar 

  41. Fang X, Han N, Wu J, Xu Y, Yang J, Wong WK, Li X (2018) Approximate low-rank projection learning for feature extraction. IEEE Trans Neural Netw Learn Syst 29(11):5228–5241

    MathSciNet  Google Scholar 

  42. Meng M, Lan M, Yu J, Wu J, Tao D (2019) Constrained discriminative projection learning for image classification. IEEE Trans Image Process 29:186–198

    MATH  MathSciNet  Google Scholar 

  43. Boykov Y, Funka-Lea G (2006) Graph cuts and efficient nd image segmentation. Int J Comput Vis 70(2):109–131

    Google Scholar 

  44. Grady L (2006) Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell 28(11):1768–1783

  45. Vincent L, Soille P (1991) Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 13(06):583–598

    Google Scholar 

  46. Lei T, Jia X, Liu T, Liu S, Meng H, Nandi AK (2019) Adaptive morphological reconstruction for seeded image segmentation. IEEE Trans Image Process 28(11):5510–5523

    MATH  MathSciNet  Google Scholar 

  47. Kilmer ME, Martin CD (2011) Factorization strategies for third-order tensors. Linear Algebra Appl 435(3):641–658

    MATH  MathSciNet  Google Scholar 

  48. Zhang C, Fu H, Hu Q, Cao X, Xie Y, Tao D, Xu D (2018) Generalized latent multi-view subspace clustering. IEEE transactions on pattern analysis and machine intelligence 42(1):86–99

    Google Scholar 

  49. Cheng B, Liu G, Wang J, Huang Z, Yan S (2011) Multi-task low-rank affinity pursuit for image segmentation. In: 2011 International conference on computer vision, p 2439–2446. IEEE

  50. Beck A (2015) On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes. SIAM J Optim 25(1):185–209

    MATH  MathSciNet  Google Scholar 

  51. Park H (1991) A parallel algorithm for the unbalanced orthogonal procrustes problem. Parallel Comput 17(8):913–923

    MATH  Google Scholar 

  52. Ng A, Jordan M, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Advances Neural Inf Process Syst 14:849–856

    Google Scholar 

  53. Wang H, Yang Y, Liu B (2019) Gmc: graph-based multi-view clustering. IEEE Transactions on Knowl Data Eng 32(6):1116–1129

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. 62102331, No. 62176125, No. 61772272), in part by the Natural Science Foundation of Sichuan Province (Grant No. 2022NSFSC0839), and in part by the Southwest University of Science and Technology Doctoral Fund Project (Grant No. 22zx7110).

Author information

Author notes

  1. Xiaoqian Zhang have contributed equally.

Authors and Affiliations

  1. School of Computer Science and Technology, Southwest University of Science and Technology, 59 Qinglong Road, Mianyang, 621010, Sichuan, China

    Xiao Wang, Xiaoqian Zhang, Jinghao Li & Shuai Zhao

  2. School of Computer Science and Engineering, Nanjing University of Science and Technology, 200 Xiaolingwei Street, Nanjing, 210094, Jiangsu, China

    Huaijiang Sun

Authors
  1. Xiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoqian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinghao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuai Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huaijiang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqian Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Zhang, X., Li, J. et al. Tensor-based multi-feature affinity graph learning for natural image segmentation. Neural Comput & Applic 35, 10997–11012 (2023). https://doi.org/10.1007/s00521-023-08279-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08279-5

Keywords

Search

Navigation