Skip to main content
SpringerLink
Log in

Guiding attention of faces through graph based visual saliency (GBVS)

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

In a general scenario, while attending a scene containing multiple faces or looking towards a group photograph, our attention does not go equal towards all the faces. It means, we are naturally biased towards some faces. This biasness happens due to availability of dominant perceptual features in those faces. In visual saliency terminology it can be called as ‘salient face’. Human’s focus their gaze towards a face which carries the ‘dominating look’ in the crowd. This happens due to comparative saliency of the faces. Saliency of a face is determined by its feature dissimilarity with the surrounding faces. In this context there is a big role of human psychology and its cognitive science too. Therefore, enormous researches have been carried out towards modeling the computer vision system like human’s vision. This paper proposed a graphical based bottom up approach to point up the salient face in the crowd or in an image having multiple faces. In this novel method, visual saliencies of faces have been calculated based on the intensity values, facial areas and their relative spatial distances. Experiment has been conducted on gray scale images. In order to verify this experiment, three level of validation has been done. In the first level, our results have been verified with the prepared ground truth. In the second level, intensity scores of proposed saliency maps have been cross verified with the saliency score. In the third level, saliency map is validated with some standard parameters. The results are found to be interesting and in some aspects saliency predictions are like human vision system. The evaluation made with the proposed approach shows moderately boost up results and hence, this idea can be useful in the future modeling of intelligent vision (robot vision) system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  • Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009. IEEE, pp 1597–1604

  • Bailey DG, Hodgson RM (1985) Range filters: localintensity subrange filters and their properties. Image Vis Comput 3(3):99–110

    Article  Google Scholar 

  • Bakouie F, Pishnamazi M, Zeraati R, Gharibzadeh S (2017) Scale-freeness of dominant and piecemeal perceptions during binocular rivalry. Cognit Neurodyn 11(4):319–326

    Article  Google Scholar 

  • Borji A, Itti L (2012) Exploiting local and global patch rarities for saliency detection. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 478–485

  • Cheng MM, Mitra NJ, Huang X, Hu SM (2014) Salientshape: group saliency in image collections. Vis Comput 30(4):443–453

    Article  Google Scholar 

  • Cheng MM, Mitra NJ, Huang X, Torr PH, Hu SM (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  PubMed  Google Scholar 

  • Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  • Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61(1):38–59

    Article  Google Scholar 

  • Cordiner A, Ogunbona P, Li W (2009) Face detection using generalised integral image features. In: 2009 16th IEEE international conference on image processing (ICIP). IEEE, pp 1229–1232

  • Di Maio V, Santillo S, Sorgente A, Vanacore P, Ventriglia F (2018) Influence of active synaptic pools on the single synaptic event. Cognit Neurodyn 12(4):391–402

    Article  Google Scholar 

  • Frintrop S, Rome E, Christensen HI (2010) Computational visual attention systems and their cognitive foundations: a survey. ACM Trans Appl Percept (TAP) 7(1):6

    Google Scholar 

  • Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926

    Article  PubMed  Google Scholar 

  • Gottumukkal R, Asari VK (2004) An improved face recognition technique based on modular PCA approach. Pattern Recognit Lett 25(4):429–436

    Article  Google Scholar 

  • Harel J, Koch C, Perona P (2006) Graph-based visual saliency. In: NIPS, vol 1, no 2, p 5

  • Heisele B, Ho P, Wu J, Poggio T (2003) Face recognition: component-based versus global approaches. Comput Vis Image Underst 91(1):6–21

    Article  Google Scholar 

  • Hu Y, Rajan D, Chia LT (2005) Adaptive local context suppression of multiple cues for salient visual attention detection. In: 2005 IEEE international conference on multimedia and expo. IEEE, p 4

  • Itti L (2000) Models of bottom-up and top-down visual attention. Doctoral dissertation, California Institute of Technology

  • Itti L, Koch C (2000) A saliency-based search mechanism for overt and covert shifts of visual attention. Vis Res 40(10):1489–1506

    Article  CAS  PubMed  Google Scholar 

  • Itti L, Koch C (2001) Computational modelling of visual attention. Nat Rev Neurosci 2(3):194–203

    Article  CAS  PubMed  Google Scholar 

  • Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  • Jafri R, Arabnia HR (2009) A survey of face recognition techniques. JIPS 5(2):41–68

    Google Scholar 

  • Jiang M, Xu J, Zhao Q (2014) Saliency in crowd. In: European conference on computer vision. Springer, Cham, pp 17–32

  • Kant Kumar R, Garain J, Ranjan Kisku D, Sanyal G (2018) Estimating attention of faces due to its growing level of emotions. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1952–1960

  • Kirby M, Sirovich L (1990) Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Trans Pattern Anal Mach Intell 12(1):103–108

    Article  Google Scholar 

  • Koch C, Ullman S (1987) Shifts in selective visual attention: towards the underlying neural circuitry. In: Vaina LM (ed) Matters of intelligence. Springer, Dordrecht, pp 115–141

    Chapter  Google Scholar 

  • Kucerova J (2011) Saliency map augmentation with facial detection. In: Proceedings of the 15th Central European seminar on computer graphics

  • Kumar RKGarain J, Kisku DR, Sanyal G (2015) A novel approach to enlighten the effect of neighbor faces during attending a face in the crowd. In: TENCON 2015-2015 IEEE Region 10 conference. IEEE, pp 1–4

  • Kumar RK, Garain J, Sanyal G, Kisku DR (2015) Estimating normalized attention of viewers on account of relative visual saliency of faces (NRVS). Int J Softw Eng Appl 9(7):85–92

    Google Scholar 

  • Kumar RK, Garain J, Kisku DR, Sanyal G (2018) Constraint saliency based intelligent camera for enhancing viewers attention towards intended face. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2018.01.002

    Article  Google Scholar 

  • Langton SR, Law AS, Burton AM, Schweinberger SR (2008) Attention capture by faces. Cognition 107(1):330–342

    Article  PubMed  Google Scholar 

  • Lawrence S, Giles CL, Tsoi AC, Back AD (1997) Face recognition: a convolutional neural-network approach. IEEE Trans Neural Netw 8(1):98–113

    Article  CAS  PubMed  Google Scholar 

  • Li J, Tian Y, Huang T (2014) Visual saliency with statistical priors. Int J Comput Vis 107(3):239–253

    Article  Google Scholar 

  • Martinez-Conde S, Macknik SL, Hubel DH (2004) The role of fixational eye movements in visual perception. Nat Rev Neurosci 5(3):229–240

    Article  CAS  PubMed  Google Scholar 

  • Mateos DM, Erra RG, Wennberg R, Velazquez JP (2018) Measures of entropy and complexity in altered states of consciousness. Cognit Neurodyn 12(1):73–84

    Article  CAS  Google Scholar 

  • Min X, Zhai G, Gu K, Liu J, Wang S, Zhang X, Yang X (2017a) Visual attention analysis and prediction on human faces. Inf Sci 420:417–430

    Article  Google Scholar 

  • Min X, Zhai G, Gu K, Yang X (2017b) Fixation prediction through multimodal analysis. ACM Trans Multimedia Comput Commun Appl (TOMM) 13(1):6

    Google Scholar 

  • Nefian AV, Hayes MH (1998) Hidden Markov models for face recognition. In: Proceedings of the 1998 IEEE international conference on acoustics, speech and signal processing, 1998, vol 5. IEEE, pp 2721–2724

  • Pal R, Mukherjee A, Mitra P, Mukherjee J (2010) Modelling visual saliency using degree centrality. IET Comput Vis 4(3):218–229

    Article  Google Scholar 

  • Pal R (2014) Computational models of visual attention: a survey. In: Srivastava R, Singh SK, Shukla KK (eds) Research developments in computer vision and image processing: methodologies and applications, 1st edn. IGI Global, USA, pp 54–76

    Chapter  Google Scholar 

  • Peters RJ, Itti L (2007) Beyond bottom-up: Incorporating task-dependent influences into a computational model of spatial attention. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8

  • Riche N, Mancas M, Duvinage M, Mibulumukini M, Gosselin B, Dutoit T (2013) A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Process Image Commun 28(6):642–658

    Article  Google Scholar 

  • Roy S, Mitra P (2016) Visual saliency detection: a Kalman filter based approach. arXiv preprint arXiv:1604.04825

  • Salah AA, Alpaydin E, Akarun L (2002) A selective attention-based method for visual pattern recognition with application to handwritten digit recognition and face recognition. IEEE Trans Pattern Anal Mach Intell 24(3):420–425

    Article  Google Scholar 

  • Tang C, Wang P, Zhang C, Li W (2017) Salient object detection via weighted low rank matrix recovery. IEEE Signal Process Lett 24(4):490–494

    Article  Google Scholar 

  • Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  • Tolba AS, El-Baz AH, El-Harby AA (2006) Face recognition: a literature review. Int J Signal Process 2(2):88–103

    Google Scholar 

  • Urakawa T, Bunya M, Araki O (2017) Involvement of the visual change detection process in facilitating perceptual alternation in the bistable image. Cognit Neurodyn 11(4):307–318

    Article  Google Scholar 

  • Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp I–I

  • Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  • Võ MLH, Smith TJ, Mital PK, Henderson JM (2012) Do the eyes really have it? Dynamic allocation of attention when viewing moving faces. J Vis 12(13):3

    Article  PubMed  Google Scholar 

  • Wang J, Yin L, Moore J (2007) Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction. ACM Trans Multimed Comput Commun Appl (TOMM) 3(4):3

    Google Scholar 

  • Wang Q, Zheng W, Piramuthu R (2016) Grab: visual saliency via novel graph model and background priors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 535–543

  • Wei H, Dai D, Bu Y (2017) A plausible neural circuit for decision making and its formation based on reinforcement learning. Cognit Neurodyn 11(3):259–281

    Article  Google Scholar 

  • Yang P (2011) Facial expression recognition and expression intensity estimation. Doctoral dissertation, Rutgers University-Graduate School-New Brunswick

  • Zanca D, Gori M (2017) Variational laws of visual attention for dynamic scenes. In: Proceedings of the 2017 neural information processing systems, 2017, NIPS 2017, pp 3823–3832

  • Zhang L, Yang C, Lu H, Ruan X, Yang MH (2017a) Ranking saliency. IEEE Trans Pattern Anal Mach Intell 39(9):1892–1904

    Article  PubMed  Google Scholar 

  • Zhang J, Ehinger KA, Wei H, Zhang K, Yang J (2017b) A novel graph-based optimization framework for salient object detection. Pattern Recognit 64:39–50

    Article  Google Scholar 

  • Zhao W, Chellappa R, Phillips PJ, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv (CSUR) 35(4):399–458

    Article  Google Scholar 

  • Zhao Q, Ge SS, Ye M, Liu S, He W (2016) Learning saliency features for face detection and recognition using multi-task network. Int J Soc Robot 8(5):709–720

    Article  Google Scholar 

  • Zhu X, Tang C, Wang P, Xu H, Wang M, Chen J, Tian J (2018) Saliency detection via affinity graph learning and weighted manifold ranking. Neurocomputing 312:239–250

    Article  Google Scholar 

  • Zommara NM, Takahashi M, Ounjai K, Lauwereyns J (2018) A gaze bias with coarse spatial indexing during a gambling task. Cognit Neurodyn 12(2):171–181

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Durgapur, Durgapur, West Bengal, 713209, India

    Ravi Kant Kumar, Jogendra Garain, Dakshina Ranjan Kisku & Goutam Sanyal

Authors
  1. Ravi Kant Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jogendra Garain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dakshina Ranjan Kisku
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Goutam Sanyal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ravi Kant Kumar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, R.K., Garain, J., Kisku, D.R. et al. Guiding attention of faces through graph based visual saliency (GBVS). Cogn Neurodyn 13, 125–149 (2019). https://doi.org/10.1007/s11571-018-9515-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11571-018-9515-z

Keywords

Search

Navigation