Skip to main content
SpringerLink
Log in

Weighted superpixel segmentation

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Image boundaries and regularity are two important factors in superpixel segmentation. Balancing the influence of image boundaries and regularity is key to producing superpixels. In this paper, we present a novel superpixel segmentation algorithm, called weighted superpixel segmentation (WSS), which is capable of generating superpixels with high boundary adherence and regular shape in a linear time. In WSS, superpixels are generated according to a distance metric defined by the combination of a weight function term, color distance term and plane distance term. Unlike other superpixel algorithms, the weight function is calculated for each pixel to determine the weight of the color distance term and plane distance term in the distance metric. To increase superpixel regularity, superpixel seeds are initialized in a hexagonal manner. Then, we use the distance metric to obtain the initial superpixels. Determining the seed search range is an essential factor to improve algorithm accuracy; thus, a dynamic circle search range is designed in our algorithm that can provide better superpixel results. Finally, a merging strategy is applied to obtain the final superpixels and ensure that the number of superpixels agrees with expectations. Experimental results demonstrate that WSS performs as well as or even better than the existing methods in terms of several commonly used evaluation metrics in superpixel segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Alexe, B., Deselaers, T., Ferrari, V.: Measuring the objectness of image windows. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2189–2202 (2012)

    Article  Google Scholar 

  3. Bódis-Szomorú, A., Riemenschneider, H., Van Gool, L.: Superpixel meshes for fast edge-preserving surface reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2011–2020 (2015)

  4. Chen, J., Li, Z., Huang, B.: Linear spectral clustering superpixel. IEEE Trans. Image Process. 26(7), 3317–3330 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Cheng, J., Liu, J., Xu, Y., Yin, F., Wong, D.W.K., Tan, N.M., Tao, D., Cheng, C.Y., Aung, T., Wong, T.Y.: Superpixel classification based optic disc and optic cup segmentation for glaucoma screening. IEEE Trans. Med. Imaging 32(6), 1019–1032 (2013)

    Article  Google Scholar 

  6. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  7. Cour, T., Shi, J.: Recognizing objects by piecing together the segmentation puzzle. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  8. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)

    Article  Google Scholar 

  9. He, X., Zemel, R.S., Ray, D.: Learning and incorporating top-down cues in image segmentation. In: European conference on computer vision, pp. 338–351 (2006)

  10. Jampani, V., Sun, D., Liu, M.Y., Yang, M.H., Kautz, J.: Superpixel sampling networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 352–368 (2018)

  11. Levinshtein, A., Stere, A., Kutulakos, K.N., Fleet, D.J., Dickinson, S.J., Siddiqi, K.: Turbopixels: fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2290–2297 (2009)

    Article  Google Scholar 

  12. Li, Z., Wu, X.M., Chang, S.F.: Segmentation using superpixels: a bipartite graph partitioning approach. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp. 789–796 (2012)

  13. Lim, J., Han, B.: Generalized background subtraction using superpixels with label integrated motion estimation. In: European Conference on Computer Vision, pp. 173–187. Springer (2014)

  14. Liu, B., Hu, H., Wang, H., Wang, K., Liu, X., Yu, W.: Superpixel-based classification with an adaptive number of classes for polarimetric SAR images. IEEE Trans. Geosci. Remote Sens. 51(2), 907–924 (2013)

    Article  Google Scholar 

  15. Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., Pietikäinen, M.: Deep learning for generic object detection: A survey. arXiv preprint arXiv:1809.02165 (2018)

  16. Liu, M.Y., Tuzel, O., Ramalingam, S., Chellappa, R.: Entropy rate superpixel segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2104 (2011)

  17. Ma, B., Hu, H., Shen, J., Liu, Y., Shao, L.: Generalized pooling for robust object tracking. IEEE Trans. Image Process. 25(9), 4199–4208 (2016)

    MathSciNet  MATH  Google Scholar 

  18. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)

  19. Moore, A.P., Prince, S.J., Warrell, J.: Lattice cut- constructing superpixels using layer constraints. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2124 (2010)

  20. Mori, G., Ren, X., Efros, A.A., Malik, J.: Recovering human body configurations: combining segmentation and recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 326–333 (2004)

  21. Nwogu, I., Corso, J.J.: 2: beyond pairwise belief propagation labeling by approximating Kikuchi free energies. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  22. Pan, X., Zhou, Y., Li, F., Zhang, C.: Superpixels of rgb-d images for indoor scenes based on weighted geodesic driven metric. IEEE Trans. Vis. Comput. Graph. 23(10), 2342–2356 (2017)

    Article  Google Scholar 

  23. Peng, B., Zhang, L., Zhang, D.: Automatic image segmentation by dynamic region merging. IEEE Trans. Image Process. 20(12), 3592–3605 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  24. Perazzi, F., Krähenbühl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–740 (2012)

  25. Ren, X., Malik, J.: Learning a classification model for segmentation. In: Proceedings of IEEE International Conference on Computer Vision, vol 1, pp 10–17 (2003)

  26. Sankaranarayanan, S., Balaji, Y., Jain, A., Nam Lim, S., Chellappa, R.: Learning from synthetic data: addressing domain shift for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3752–3761 (2018)

  27. Shen, J., Du, Y., Wang, W., Li, X.: Lazy random walks for superpixel segmentation. IEEE Trans. Image Process. 23(4), 1451–1462 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  28. Shen, J., Hao, X., Liang, Z., Liu, Y., Wang, W., Shao, L.: Real-time superpixel segmentation by DBSCAN clustering algorithm. IEEE Trans. Image Process. 25(12), 5933–5942 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  29. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  30. Tighe, J., Lazebnik, S.: Superparsing: scalable nonparametric image parsing with superpixels. In: European Conference on Computer Vision, pp. 352–365 (2010)

  31. Tu, W.C., Liu, M.Y., Jampani, V., Sun, D., Chien, S.Y., Yang, M.H., Kautz, J.: Learning superpixels with segmentation-aware affinity loss. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 568–576 (2018)

  32. Van den Bergh, M., Boix, X., Roig, G., de Capitani, B., Van Gool, L.: Seeds: superpixels extracted via energy-driven sampling. In: European Conference on Computer Vision, pp. 13–26 (2012)

  33. Veksler, O., Boykov, Y., Mehrani, P.: Superpixels and supervoxels in an energy optimization framework. In: European Conference on Computer Vision, pp. 211–224 (2010)

  34. Wang, S., Lu, H., Yang, F., Yang, M.H.: Superpixel tracking. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1323–1330 (2011)

  35. Wang, W., Shen, J., Shao, L.: Consistent video saliency using local gradient flow optimization and global refinement. IEEE Trans. Image Process. 24(11), 4185–4196 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wang, W., Shen, J., Shao, L., Porikli, F.: Correspondence driven saliency transfer. IEEE Trans. Image Process. 25(11), 5025–5034 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wang, Z., Feng, J., Yan, S., Xi, H.: Image classification via object-aware holistic superpixel selection. IEEE Trans. Image Process. 22(11), 4341–4352 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  38. Yang, M., Yu, K., Zhang, C., Li, Z., Yang, K.: Denseaspp for semantic segmentation in street scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3684–3692 (2018)

  39. Zhang, Y., Li, X., Gao, X., Zhang, C.: A simple algorithm of superpixel segmentation with boundary constraint. IEEE Trans. Circuits Syst. Video Technol. 27(7), 1502–1514 (2017)

    Google Scholar 

  40. Zhou, Y., Ju, L., Wang, S.: Multiscale superpixels and supervoxels based on hierarchical edge-weighted centroidal voronoi tessellation. IEEE Trans. Image Process. 24(11), 3834–3845 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  41. Zhou, Y., Pan, X., Wang, W., Yin, Y., Zhang, C.: Superpixels by bilateral geodesic distance. IEEE Trans. Circuits Syst. Video Technol. 27(11), 2281–2293 (2017)

    Article  Google Scholar 

  42. Zhou, Y., Zhu, Y., Ye, Q., Qiu, Q., Jiao, J.: Weakly supervised instance segmentation using class peak response. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3791–3800 (2018)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University, Jinan, 250101, China

    Xin Qian, Xuemei Li & Caiming Zhang

  2. Shandong Co-Innovation Center of Future Intelligent Computing, Yantai, 264025, China

    Xuemei Li & Caiming Zhang

  3. Shandong Province Key Lab of Digital Media Technology, Shandong University of Finance and Economics, Jinan, 250061, China

    Caiming Zhang

Authors
  1. Xin Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuemei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Caiming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuemei Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported partly by the NSFC Joint Fund with Zhejiang Integration of Informatization and Industrialization under Key Project under Grant No. U1609218 and NSFC under Grant Nos. 61572292, 61332015, 61602277, 61873117.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qian, X., Li, X. & Zhang, C. Weighted superpixel segmentation. Vis Comput 35, 985–996 (2019). https://doi.org/10.1007/s00371-019-01682-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01682-x

Keywords

Search

Navigation