Skip to main content
SpringerLink
Log in

Visual Attention Prediction Using Saliency Determination of Scene Understanding for Social Robots

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

In this paper, the biological ability of visual attention is modeled for social robots to understand scenes and circumstance. Visual attention is determined by evaluating visual stimuli and prior knowledge in the intelligent saliency searching. Visual stimuli are measured using information entropy and biological color sensitivities, where the information entropy evaluates information qualities and the color sensitivity assesses biological attraction of a presented scene. We also learn and utilize the prior knowledge of people’s focus in the prediction of visual attention. The performance of the proposed technique is studied on different sorts of natural scenes and evaluated with fixation data of actual eye-tracking database. The experimental results proved the effectiveness of the proposed technique in discovering salient regions and predicting visual attention. The robustness of the proposed technique to transformation and illumination variance is also investigated. Social robots equipped with the proposed technique can autonomously determine their attention to a scene autonomously so as to behave naturally in the human robot interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Sha’ashua A, Ullman S (1988) Structural saliency: The detection of globally salient structures using a locally connected network. In: ICCV88

    Google Scholar 

  2. Rosin P (1997) Edges: saliency measures and automatic thresholding. Mach Vis Appl 9(4):139–159

    Article  Google Scholar 

  3. Sun Y, Fisher R (2003) Object-based visual attention for computer vision. Artif Intell 146(1):77–123

    Article  MathSciNet  MATH  Google Scholar 

  4. Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. In: IEEE conference on computer vision and pattern recognition (CVPR07). IEEE Computer Society, Citeseer, pp 1–8

    Chapter  Google Scholar 

  5. Gao D, Vasconcelos N (2005) Discriminant saliency for visual recognition from cluttered scenes. Adv Neural Inf Process Syst 17:481–488

    Google Scholar 

  6. Gao D, Mahadevan V, Vasconcelos N (2007) The discriminant center-surround hypothesis for bottom-up saliency. In: Neural information processing systems (NIPS), Vancouver, Canada

    Google Scholar 

  7. Kadir T, Brady M (2001) Saliency, scale and image description. Int J Comput Vis 45(2):83–105

    Article  MATH  Google Scholar 

  8. Chan A, Mahadevan V, Vasconcelos N (000) Generalized Stauffer–Grimson background subtraction for dynamic scenes, Mach. Vis. Appl. (2010) 1–16

  9. Wixson L (2002) Detecting salient motion by accumulating directionally-consistent flow. IEEE Trans Pattern Anal Mach Intell 22(8):774–780

    Article  Google Scholar 

  10. Li Z (2002) A saliency map in primary visual cortex. Trends Cogn Sci 6(1):9–16

    Article  Google Scholar 

  11. Itti L, Koch C, Niebur E (2002) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  12. Parkhurst D, Law K, Niebur E (2002) Modeling the role of salience in the allocation of overt visual attention. Vis Res 42(1):107–123

    Article  Google Scholar 

  13. Park S, Shin J, Lee M (2010) Biologically inspired saliency map model for bottom-up visual attention. In: Biologically motivated computer vision. Springer, Berlin, pp 113–145

    Google Scholar 

  14. Borji A, Ahmadabadi M, Araabi B (2009) Cost-sensitive learning of top-down modulation for attentional control, Mach Vis Appl 1–16

  15. Oliva A, Torralba A, Castelhano M, Henderson J (2003) Top-down control of visual attention in object detection. In: 2003 international conference on image processing. ICIP 2003. Proceedings

    Google Scholar 

  16. Ramanathan S, Katti H, Sebe N, Kankanhalli M, Chua T (2010) An eye fixation database for saliency detection in images. In: Computer vision–ECCV 2010, pp 30–43

    Chapter  Google Scholar 

  17. Judd T, Ehinger K, Durand F, Torralba A (2010) Learning to predict where humans look. In: IEEE 12th international conference on computer vision, 2009 IEEE, New York, pp 2106–2113

    Google Scholar 

  18. Kandel E, Schwartz J, Jessell T, Mack S, Dodd J (1985) Principles of neural science. Elsevier, New York

    Google Scholar 

  19. Sharpe L, Stockman A, Jagla W, Jagle H (2005) A luminous efficiency function, V∗ λ, for daylight adaptation. J Vis 5:948–968

    Article  Google Scholar 

  20. Smith T, Guild J (1931) The cie colorimetric standards and their use. Trans Opt Soc 33:73

    Article  Google Scholar 

  21. Stokes M, Anderson M, Chandrasekar S, Motta R A standard default color space for the Internet–sRGB. Microsoft and Hewlett-Packard Joint Report

  22. Wright W (1929) A re-determination of the trichromatic coefficients of the spectral colours. Trans Opt Soc 30:141

    Article  Google Scholar 

  23. Ge SS, Yang Y, Lee T (2008) Hand gesture recognition and tracking based on distributed locally linear embedding. Image Vis Comput 26(12):1607–1620

    Article  Google Scholar 

  24. Sun J, Zhang X, Cui J, Zhou L (2006) Image retrieval based on color distribution entropy. Pattern Recognit Lett 27(10):1122–1126

    Article  Google Scholar 

  25. Rao A, Srihari RK, Zhang Z (1999) Spatial color histograms of content-based image retrieval. In: 11th IEEE international conference on tools with artifical intelligence, 1999. Proceedings. IEEE Press, New York, pp 183–186

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Social Robotics Laboratory, Interactive Digital Media Institute and Department of Electrical and Computer Engineering, National University of Singapore, Singapore, 117576, Singapore

    Hongsheng He, Shuzhi Sam Ge & Zhengchen Zhang

  2. Robotics Institute and School of Computer Science and Engineering, University of Electronic Science and Technology of China (UESTC), Chengdu, 611731, P.R. China

    Shuzhi Sam Ge

Authors
  1. Hongsheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuzhi Sam Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengchen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuzhi Sam Ge.

Rights and permissions

Reprints and permissions

About this article

Cite this article

He, H., Ge, S.S. & Zhang, Z. Visual Attention Prediction Using Saliency Determination of Scene Understanding for Social Robots. Int J of Soc Robotics 3, 457–468 (2011). https://doi.org/10.1007/s12369-011-0105-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-011-0105-z

Keywords

Search

Navigation