The Visual Computer

, Volume 33, Issue 5, pp 665–685 | Cite as

An evolutionary learning based fuzzy theoretic approach for salient object detection

  • Aditi KapoorEmail author
  • K. K. Biswas
  • M. Hanmandlu
Original Article


Human attention tends to get focused on the most prominent components of a scene which are in sharp contrast with the background. These are termed as salient regions. The human brain perceives an object to be salient based on various features like the relative intensity, spread of the region, color contrast with the background, size and position within an image. Since these features vary widely, no crisp thresholds can be specified for an automatic salient region detector. In this paper we present a rule based system which uses a set of fuzzy features to mark out the salient region in an image. A genetic algorithm based evolutionary system is used to learn the rules from the training images. Extensive comparisons with the state-of-the-art methods in terms of precision, recall and F-measure are made on two different publicly available datasets to prove the effectiveness of this approach. The application of the proposed salient object detection approach is shown in non-photorealistic rendering, perception based image compression and context aware retargeting applications with promising results.


Salient Fuzzy Genetic algorithm  Color features 


  1. 1.
    MacEvoy B (2005) CIELAB \({\rm a}^{*} {\rm b}^{*}\) plane. Accessed 5th March 2016
  2. 2.
    Achanta, R., Estrada, F.J., Wils, P., Susstrunk, S.: Salient region detection and segmentation. In: ICVS, pp. 66–75 (2008)Google Scholar
  3. 3.
    Achanta, R., Susstrunk, S.: Saliency detection for content-aware image resizing. In: ICIP, pp. 1005–1008 (2009)Google Scholar
  4. 4.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: CVPR, pp. 1597–1604 (2009)Google Scholar
  5. 5.
    Achanta, R., Susstrunk, S.: Saliency detection using maximum symmetric surround. In: ICIP, pp. 2653–2656 (2010)Google Scholar
  6. 6.
    Alexe, B., Deselaers, T., Ferrari, V.: What is an object? In: CVPR (2010)Google Scholar
  7. 7.
    Aytekin, C., Kiranyaz, S., Gabbouj, M.: Automatic object segmentation by quantum cuts. In: International Conference on Pattern Recognition, pp. 112–117 (2014)Google Scholar
  8. 8.
    Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark, CoRR (2015). arXiv:1501.02741
  9. 9.
    Bruce, N.: Features that draw visual attention: an information theoretic perspective. Neurocomputing 65–66, 125–133 (2005)CrossRefGoogle Scholar
  10. 10.
    Bruce, N.D.B., Tsotsos, J.K.: Saliency based on information maximization. In: NIPS, vol. 18, pp. 155–162 (2006)Google Scholar
  11. 11.
    Bruce, N.D.B., Tsotsos, J.K.: Saliency, attention, and visual search: an information theoretic approach. J. Vis. 9(3), 1–24 (2009)CrossRefGoogle Scholar
  12. 12.
    Chang, K.Y., Liu, T.L., Chen, H.T., Lai, S.H.: Fusing generic objectness and visual saliency for salient object detection. In: ICCV, pp. 914–921 (2011)Google Scholar
  13. 13.
    Chen, L., Fan, X., Ma, W., Zhang, H., Zhou, H.: A visual attention model for adapting images on small displays. In: MMM03, pp. 353–364 (2003)Google Scholar
  14. 14.
    Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H.S., Hu, S.-M.: Global contrast based salient region detection. In: IEEE TPAMI (2015)Google Scholar
  15. 15.
    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (IEEE CVPR), pp. 409–416 (2011)Google Scholar
  16. 16.
    Cheng, M.M., Warrell, C.H., Lin, W.Y., Zheng, S., Vineet, V., Crook, N.: Efficient salient region detection with soft image abstraction. In: IEEE ICCV, pp. 1529–1536 (2013)Google Scholar
  17. 17.
    Cheng, M.M., Hu, S.M., Mitra, N.J., Huang, X.: SalientShape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014). doi: 10.1007/s00371-013-0867-4 CrossRefGoogle Scholar
  18. 18.
    DeCarlo, D., Santella, A.: Stylization and abstraction of photographs. In: 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH, pp. 769–776 (2002)Google Scholar
  19. 19.
    Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: CVPR, pp. 1657–1664 (2011)Google Scholar
  20. 20.
    Duan, L., Wu, C., Miao, J., Qing, L., Fu, Y.: Visual saliency detection by spatially weighted dissimilarity. In: CVPR, pp. 473–480 (2011)Google Scholar
  21. 21.
    Erdem, E., Erdem, A.: Visual saliency estimation by nonlinearly integrating features using region covariances. J. Vis. 13(4):11, 1–20 (2013)Google Scholar
  22. 22.
    Feng, J., Wei, Y., Tao, L., Zhang, C., Sun, J.: Salient object detection by composition. In: ICCV, pp. 1028–1035 (2011)Google Scholar
  23. 23.
    Fu, K., Gong, C., Yang, J., Zhou, Y.: Salient object detection via color contrast and color distribution. In: 2012 IEEE 11th Asian Conference on Computer Vision (ACCV), IEEE (2012)Google Scholar
  24. 24.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context aware saliency detection. In: CVPR, pp. 2376–2383 (2010)Google Scholar
  25. 25.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE TPAMI 34(10), 1915–1926 (2012)Google Scholar
  26. 26.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS, pp. 545–552 (2007)Google Scholar
  27. 27.
    Heckbert, P.: Color image quantization for frame buffer display. Comput. Graph. 16(3), 207–307 (1982)Google Scholar
  28. 28.
    Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. . IEEE TPAMI 34(1), 194–201 (2012)Google Scholar
  29. 29.
    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: CVPR, pp. 1–8 (2007)Google Scholar
  30. 30.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis? In: IEEE TPAMI (1998)Google Scholar
  31. 31.
    Jang, J.S.R., Sun, C.T., Mizutani, E.: Fuzzy set theory. In: Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence, pp. 37–86. Pearson Prentice Hall, USA (1997). ISBN 81-317-1109-9Google Scholar
  32. 32.
    Jiang, B., Zhang, L., Lu, H., Yang, C., Yang, M.H.: Saliency detection via absorbing markov chain. In: ICCV (2013)Google Scholar
  33. 33.
    Jiang, H., Wang, J., Yuan, Z., et al.: Automatic salient object segmentation based on context and shape prior. BMVC 3(4), 110.1–110.12 (2011)Google Scholar
  34. 34.
    Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: IEEE CVPR, pp. 2083–2090 (2013)Google Scholar
  35. 35.
    Kanan, C., Cottrell, G.W.: Robust classification of objects, faces, and flowers using natural image statistics. In: CVPR, pp. 2472–2479 (2010)Google Scholar
  36. 36.
    Kapoor, A., Biswas, K.K.: A case-based reasoning approach for detection of salient regions in images. In: ICVGIP, pp. 48–55 (2010)Google Scholar
  37. 37.
    Kapoor, A., Biswas, K.K., Hanmandlu, M.: Salient object detection using a fuzzy theoretic approach. In: ICVGIP, Article No. 9 (2012)Google Scholar
  38. 38.
    Li, N., Ye, J., Ji, H., Ling, H., Ju, Y.: Saliency detection on light field. In: CVPR (2014)Google Scholar
  39. 39.
    Li, X., Lu, H., Zhang, L., Ruan, X., Yang, M.H.: Saliency detection via dense and sparse reconstruction. In: ICCV (2013)Google Scholar
  40. 40.
    Liu, T., Sun, J., Zheng, J., Tang, X., Shum, H.: Learning to detect a salient object. In: CVPR, pp. 1–8 (2007)Google Scholar
  41. 41.
    Ma, M., Zhang, H.: Contrast-based image attention analysis by using fuzzy growing. In: Proc. of the 11th ACM Int. conference on Multimedia (2003)Google Scholar
  42. 42.
    Marchesotti, L., Cifarelli, C., Csurka, G.: A framework for visual saliency detection with applications to image thumbnailing. In: ICCV, pp. 2232–2239 (2009)Google Scholar
  43. 43.
    Margolin, R., Tal, A., Zelnik-Manor, L.: What makes a patch distinct? In: CVPR (2013)Google Scholar
  44. 44.
    Margolin, R., Zelnik-Manor, L., Tal, A.: Saliency for image manipulation. Vis. Comput. 29(5), 381–392 (2013)CrossRefGoogle Scholar
  45. 45.
    Murray, N., Vanrell, M., Otazu, X., Parraga, C.A.: Saliency estimation using a non-parametric low-level vision model. In: CVPR, pp. 433–440 (2011)Google Scholar
  46. 46.
    Nilsson, M., Nordberg, J., Claesson, T.: Face detection using local smqt features and split up snow classifier. In: ICASSP, pp. 1222–1239 (2007)Google Scholar
  47. 47.
    Perazzi, F., Krahenbuhl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: IEEE CVPR, pp. 733–740 (2012)Google Scholar
  48. 48.
    Rahtu, E., Kannala, J., Salo, M., Heikkila, J.: Segmenting salient objects from images and videos. In: ECCV (2010)Google Scholar
  49. 49.
    Roth, D., Yang, M., Ahuja, N.: A snow-based face detector. In: NIPS, pp. 855–861 (2000)Google Scholar
  50. 50.
    Rong, M.: Perception-based multi-quality image compression for efficient transmission. In: ISPACS, pp. 817–820 (2006)Google Scholar
  51. 51.
    Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J. Vis. 9(12), 15.1–27 (2009)Google Scholar
  52. 52.
    Siva, P., Russell, C., Xiang, T., Agapito, L.: Looking beyond the image: unsupervised learning for object saliency and detection. In: CVPR (2013)Google Scholar
  53. 53.
    Tamara, L.: Finding iconic images. In: Internet Vision Workshop at CVPR (2009)Google Scholar
  54. 54.
    Tavakoli, H.R., Rahtu, E., Heikkila, J.: Fast and efficient saliency detection using sparse sampling and kernel density estimation. In: Image Analysis—17th Scandinavian Conference, SCIA’11, Ystad, pp. 666–675 (2011)Google Scholar
  55. 55.
    Wandell, B.: Foundations of Vision. Sinauer, Sunderland (1995)Google Scholar
  56. 56.
    Watson, A.B.: Image compression using the discrete cosine transform. Math. J. 4(1), 81–88 (1994)MathSciNetGoogle Scholar
  57. 57.
    Wei, L.S., Sang, N., Wang, Y., Wang, D., Wang, F.: Variable resolution image compression based on a model of visual attention. In: Proc. of SPIE, vol. 7495 74950P-1 (2009)Google Scholar
  58. 58.
    Wei, W., Wen, F., Zhu, W., Sun, J.: Geodesic saliency using background priors. In: Computer Vision ECCV’12, pp. 29–42. Springer, New York (2012)Google Scholar
  59. 59.
    Xie, Y., Lu, H., Yang, M.-H.: Bayesian saliency via low and mid level cues. IEEE TIP 22(5), 1689–1698 (2013)Google Scholar
  60. 60.
    Xu, M., Zhang, H.: Saliency detection with color contrast based on boundary information and neighbors. Vis. Comput. (2014)Google Scholar
  61. 61.
    Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  62. 62.
    Yang, C., Zhang, L., Lu, H.: Graph-regularized saliency detection with convex-hull-based center prior. IEEE Signal Process. Lett. 20(7), 637–640 (2013)CrossRefGoogle Scholar
  63. 63.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: CVPR (2013)Google Scholar
  64. 64.
    Yu, S.X., Lisin, D.A.: Image compression based on visual saliency at individual scales. In: Advances in Visual Computing, 5th International Symposium, ISVC Proceedings, Part I, pp. 157–166 (2009)Google Scholar
  65. 65.
    Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatio temporal cues. In: Proceedings of the 14th Annual ACM International Conference on Multimedia, ACM’06, pp. 815–824 (2006)Google Scholar
  66. 66.
    Zhang, J., Sclaroff, S.: Saliency detection: a boolean map approach. In: ICCV, pp. 153–160 (2013)Google Scholar
  67. 67.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: A Bayesian framework for saliency using natural statistics. J. Vis. 8(7):32, 1–20 (2008)Google Scholar
  68. 68.
    Zhou, L., Fu, K., Li, y, Qiao, Y., JiangHe, X., Yang, J.: Bayesian salient object detection based on saliency driven clustering. Signal Process. Image Commun. 29, 434–447 (2014)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.School of Information TechnologyIndian Institute of Technology DelhiNew DelhiIndia
  2. 2.Department of Computer Science and EngineeringIndian Institute of Technology DelhiNew DelhiIndia
  3. 3.Department of Electrical EngineeringIndian Institute of Technology DelhiNew DelhiIndia

Personalised recommendations