Multimedia Tools and Applications

, Volume 76, Issue 8, pp 10443–10463 | Cite as

Extended quantum cuts for unsupervised salient object extraction

  • Çağlar Aytekin
  • Ezgi Can Ozan
  • Serkan Kiranyaz
  • Moncef Gabbouj


In this manuscript, an unsupervised salient object extraction algorithm is proposed for RGB and RGB-Depth images. Saliency estimation is formulated as a foreground detection problem. To this end, Quantum-Cuts (QCUT), a recently proposed spectral foreground detection method is investigated and extended to formulate the saliency estimation problem more efficiently. The contributions of this work are as follows: (1) a new proof for QCUT from spectral graph theory point of view is provided, (2) a detailed analysis of QCUT and comparison to well-known graph clustering methods are conducted, (3) QCUT is utilized in a multiresolution framework, (4) a novel affinity matrix construction scheme is proposed for better encoding of saliency cues into the graph representation and (5) a multispectral analysis for a richer set of salient object proposals is investigated. With the above improvements, we propose Extended Quantum Cuts, which consistently achieves an exquisite performance over all benchmark saliency detection datasets, containing around 18 k images in total. Finally, the proposed approach also outperforms the state-of-the-art on a recently announced RGB-Depth saliency dataset.


Visual saliency Salient object detection Quantum mechanics Spectral graph theory Schrödinger’s equation 


  1. 1.
    Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  2. 2.
    Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2281CrossRefGoogle Scholar
  3. 3.
    Alpert S, Galun M, Basri R, Brandt A (2007) Image segmentation by probabilistic bottom-up aggregation and cue integration. In CVPR, pp 1–8Google Scholar
  4. 4.
    Aytekin C, Kiranyaz S, Gabbouj M (2013) Quantum mechanics in computer vision: automatic object extraction. Int Conf Image Process:2489–2493Google Scholar
  5. 5.
    Aytekin C, Kiranyaz S, Gabbouj M (2014) Automatic object segmentation by quantum cuts. 22nd International Conference on Pattern Recognition, Stockholm, 24–28, pp 112–117Google Scholar
  6. 6.
    Borji A (2014) What is a salient object? A dataset and a baseline model for salient object detection. In IEEE TIPGoogle Scholar
  7. 7.
    Borji A, Cheng MM, Jiang H, Li J et al (2015) Salient object detection: a benchmark. arXiv preprint arXiv:1501.02741Google Scholar
  8. 8.
    Casares M, Velipasalar S, Pinto A (2010) Light-weight salient foreground detection for embedded smart cameras. Comput Vis Image Underst 114(11):1223–1237CrossRefGoogle Scholar
  9. 9.
    Cheng M.-M, Mitra NJ, Huang X, Hu S.-M; SalientShape: Group Saliency in Image Collections (2013) The visual computerGoogle Scholar
  10. 10.
    Cheng M.-M, Warrell J, Lin W.-Y, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. Int Conf Comput VisGoogle Scholar
  11. 11.
    Cheng MM, Zhang G, Mitra N, Huang X, Hu S (2011) Global contrast based salient region detection. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  12. 12.
    Gao D, Mahadevan V, Vasconcelos N (2007) The discriminant center-surround hypothesis for bottom-up saliency. Conf Neural Inf Process SystGoogle Scholar
  13. 13.
    Goferman S, Manor L, Tal A (2010) Context-aware saliency detection. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  14. 14.
    Hadizadeh H, Bajic IV (2014) Saliency-aware video compression. IEEE Trans Image Process 23(1):19–33MathSciNetCrossRefGoogle Scholar
  15. 15.
    Han J, Sun L, Hu X, Han J, Shao L (2014) Spatial and temporal visual attention prediction in videos using eye movement data. Neurocomputing 145:140–153CrossRefGoogle Scholar
  16. 16.
    Harel J, Koch C, Perona P (2006) Graph-based visual saliency. Conf Neural Inf Process SystGoogle Scholar
  17. 17.
    Hou X, Zhang L (2007) Thumbnail Generation based on Global Saliency. Adv Neurodyn: 999–1003Google Scholar
  18. 18.
    Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  19. 19.
    Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259CrossRefGoogle Scholar
  20. 20.
    Jiang Z, Davis LS (2013) Submodular salient region detection. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  21. 21.
    Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. In British Machine Vision Conference (BMVC)Google Scholar
  22. 22.
    Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  23. 23.
    Jiang B, Zhang L, Lu H, Yang C, Yang M-H (2013) Saliency detection via absorbing Markov chain. 14th International Conference on Computer VisionGoogle Scholar
  24. 24.
    Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. Int Conf Comput VisGoogle Scholar
  25. 25.
    Kiranyaz S, Ferreira M, Gabbouj M (2006) Automatic object extraction over multiscale edge field for multimedia retrieval. IEEE Trans Image Process:15(12)Google Scholar
  26. 26.
    Klein D, Frintrop S (2011) Center-surround divergence of feature statistics for salient object detection. Int Conf Comput VisGoogle Scholar
  27. 27.
    Liboff RL (2003) Introductory quantum mechanics, 4th edn. Addison Wesley, San FranciscoMATHGoogle Scholar
  28. 28.
    Liu T, Sun J, Zheng N, Tang X, Shum H (2007) Learning to detect a salient object. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  29. 29.
    Luxburg UV (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416MathSciNetCrossRefGoogle Scholar
  30. 30.
    Margolin R, Manor LZ, Tal A (2012) Saliency for image manipulation. Vis Comput 29(5):381–392CrossRefGoogle Scholar
  31. 31.
    Peng H, Li B, Xiong W, Hu W, Ji R RGBD salient object detection: a benchmark and algorithms. In Proceedings of the 13th European Conference on Computer Vision (ECCV2014)Google Scholar
  32. 32.
    Perazzi F, Krahenbuhl P, Pritch Y, Hornung A (2012) Saliency filters: contrast based filtering for salient region detection. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  33. 33.
    Rigas I, Economou G, Fotopoulos S (2015) Efficient modelling of visual saliency based on local sparse representation and the use of hamming distance. Comput Vis Image Underst 134:33–45CrossRefGoogle Scholar
  34. 34.
    Rother C, Kolmogorov V, Blake A (2004) GrabCut: interactive foreground extraction using iterated graph cuts. In SIGGRAPHGoogle Scholar
  35. 35.
    Rudoy D, Goldman DB, Shechtman E, Manor LZ (2013) Learning video saliency from human gaze using candidate selection. IEEE Conference on Computer Vision and Pattern Recognition, pp 1147–1154Google Scholar
  36. 36.
    Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905CrossRefGoogle Scholar
  37. 37.
  38. 38.
    Tong Y, Cheikh FA, Guraya FFE, Konik H, Tremeau A (2011) A spatiotemporal saliency model for video surveillance. Cogn Comput 3(1):241–263CrossRefGoogle Scholar
  39. 39.
    Valenti R, Sebe N, Gevers T (2009) Image saliency by isocentric curvedness and color. Int Conf Comput VisGoogle Scholar
  40. 40.
    Wang L, Xue J, Zheng N, Hua G (2011) Automatic salient object extraction with contextual cue. Int Conf Comput VisGoogle Scholar
  41. 41.
    Wei YC, Cheng CK (1991) Ratio cut partitioning for hierarchical designs 10(7):911–921Google Scholar
  42. 42.
    Wei Y, Wen F, Zhu W, Sun J (2012) Geodesic saliency using background priors. Eur Conf Comput VisGoogle Scholar
  43. 43.
    Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. Comput Vis Pattern RecogGoogle Scholar
  44. 44.
    Yang C, Zhang L, Lu H, Ruan X, Yang M.-H (2013) Saliency detection via graph-based manifold ranking. Comput Vis Pattern RecogGoogle Scholar
  45. 45.
    Zhu W, Liang S, Wei Y, J Sun (2014) Saliency optimization from robust background detection. IEEE Conf Comput Vis Pattern RecogGoogle Scholar
  46. 46.
    Zou WB, Kpalma K, Liu Z, Ronsin J (2013) Segmentation driven low-rank matrix recovery for saliency detection, BMVCGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Çağlar Aytekin
    • 1
  • Ezgi Can Ozan
    • 1
  • Serkan Kiranyaz
    • 2
  • Moncef Gabbouj
    • 1
  1. 1.Department of Signal ProcessingTampere University of TechnologyTampereFinland
  2. 2.Department of Electrical EngineeringQatar UniversityDohaQatar

Personalised recommendations