Abstract
Low-rank decomposition (LRD) is an effective scheme to explore the affinity among superpixels in the image and video segmentation. However, the superpixel feature collected based on colour, shape, and texture may be rough, incompatible, and even conflicting if multiple features extracted in various manners are vectored and stacked straight together. It poses poor correlation, inconsistence on intra-category superpixels, and similarities on inter-category superpixels. This paper proposes a correlation-enhanced superpixel for video segmentation in the framework of LRD. Our algorithm mainly consists of two steps, feature analysis to establish the initial affinity among superpixels, followed by construction of a correlation-enhanced superpixel. This work is very helpful to perform LRD effectively and find the affinity accurately and quickly. Experiments conducted on datasets validate the proposed method. Comparisons with the state-of-the-art algorithms show higher speed and more precise in video segmentation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2010) Slic superpixels. In Technical report, EPFL
Brox T, Malik J (2010) Object segmentation by long term analysis of point trajectories. In: Proceedings of European conference on computer vision. https://doi.org/10.1007/978-3-642-15555-0_21
Chen L, Papandreou G, Kokkinos I, Murphy K et al (2016) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848
Chen L, Zhu Y, Papandreou G, et al (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. Preprint arXiv:1802.02611
Cheng B, Liu G, Wang J, et al (2011). Multi-task low-rank affinity pursuit for image segmentation. In Proceedings of IEEE international conference on computer vision, pp 2439–2446
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 886–893
Duta I, Uijlings J, Nguyen T, et al (2016) Histograms of motion gradients for real-time video classification. In: International workshop on content-based multi-media indexing. https://doi.org/10.1109/cbmi.2016.7500260
Farnoush Z, Borislav A, Jan S (2018). Superpixel-based road segmentation for real-time systems using CNN. In: Proceedings of the 13th international joint conference on computer vision, imaging and computer graphics theory and applications (VISIGRAPP), pp 257–265
Felzenszwalb P, Huttenlocher D (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181
Galasso F, Cipolla R, Schiele B (2012) Video segmentation with superpixels. In: Proceedings of the Asian conference on computer vision, pp 760–774
Galasso N, Nagaraja J, Cardenas T, Brox B, Schiele A (2013) Unified video segmentation benchmark: annotation, metrics and analysis. In: International conference on computer vision. https://doi.org/10.1109/iccv.2013.438
Grundmann M, Kwatra V, Han M, et al (2010) Efficient hierarchical graph-based video segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition, pp 2141–2148
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. arXiv:1512.03385. https://doi.org/10.1109/cvpr.2016.90
Konstantinos G (2008) The bhattacharyya. Measure, Version 1.0, March 20
Li C, Lin L, Zuo W, Wang W, Tang J,Yan S (2015) SOLD: sub-optimal low-rank decomposition for efficient video segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp 5519–5527. https://doi.org/10.1109/cvpr.2015.7299191
Li T, Bin Cheng B, Ni B et al (2016a) Multitask low-rank affinity graph for image segmentation and image annotation. ACM Trans Intell Syst Technol 7(4):1–18
Li C, Lin L, Zuo W, Wang W, Tang J (2016b) An approach to streaming video segmentation with sub-optimal low-rank decomposition. IEEE Trans Image Process T-IP 25(5):1947–1960
Liu R, Lin Z, Torre F, Su Z (2012) Fixed-rank representation for unsupervised visual learning. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 598–605
Liu G, Lin Z, Yan S et al (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184
Luc P, Couprie C, Chintala S, et al (2016) Semantic segmentation using adversarial networks. In: NIPS-2016 NIPS workshop on adversarial training, Barcelona, Spain. arXiv:1611.08408
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Med Image Comput Comput-Assist Interv (MICCAI) 9351:234–241
Das A, Ghosh S, Sarkhel R, et al. (2018) Combining multi-level contexts of superpixel using convolutional neural networks to perform natural scene labeling. arXiv:1803.05200
Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans PAMI 22(8):888–905
Wang L, Dong M (2012) Multi-level low-rank approximation based spectral clustering for image segmentation. Pattern Recognit Lett 33(16):2206–2215
Wang Y, Jiang Y, Wu Y et al (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161
Wen Z, Yin W, Zhang Y (2010) Solving a low-rank factorization model for matrix completion by a non-linear successive over-relaxation algorithm. Rice CAAM Tech Report TR10-07
Xu C, Corso J (2012). Evaluation of super-voxel methods for early video processing. In: Proceedings of IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.2012.6247802
Xu H, Zhou W, Wang Y, Wang W, Mo Y (2017) Matrix separation based on lmafit-seed. Comput J 60(11):1609–1618
Yin M, Gao J, Lin Z (2016) Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell 38(3):504–517
Zhang T, Ghanem B, Liu S, et al (2013) Low-rank sparse coding for image classification. In Proceedings of IEEE international conference on computer vision, pp 281–288
Acknowledgements
This work was supported by National Natural Science Foundation of China (Nos. 61602397, 61841103), The Natural Science Foundation of Hunan Province (2017JJ2251, 2017JJ3315), and Chinese Scholar-ship Council of the Ministry of Education.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xu, H., Hancock, E.R. & Zhou, W. The low-rank decomposition of correlation-enhanced superpixels for video segmentation. Soft Comput 23, 13055–13065 (2019). https://doi.org/10.1007/s00500-019-03849-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-019-03849-z