Dynamic Saliency Models and Human Attention: A Comparative Study on Videos

  • Nicolas Riche
  • Matei Mancas
  • Dubravko Culibrk
  • Vladimir Crnojevic
  • Bernard Gosselin
  • Thierry Dutoit
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7726)

Abstract

Significant progress has been made in terms of computational models of bottom-up visual attention (saliency). However, efficient ways of comparing these models for still images remain an open research question. The problem is even more challenging when dealing with videos and dynamic saliency. The paper proposes a framework for dynamic-saliency model evaluation, based on a new database of diverse videos for which eye-tracking data has been collected. In addition, we present evaluation results obtained for 4 state-of-the-art dynamic-saliency models, two of which have not been verified on eye-tracking data before.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Itti, L., Koch, C.: Computational modelling of visual attention. Nature Reviews Neuroscience 2, 194–203 (2001)CrossRefGoogle Scholar
  2. 2.
    Olveczky, B.P., Baccus, S.A., Meister, M.: Segregation of object and background motion in the retina. Nature 423, 401–408 (2003)CrossRefGoogle Scholar
  3. 3.
    James, W.: The Principles of Psychology, vol. 1. Dover Publications (1950)Google Scholar
  4. 4.
    Styles, E.A.: Attention, Perception, and Memory: An Integrated Introduction. Taylor & Francis Routledge, New York (2005)Google Scholar
  5. 5.
    Wolfe, J.M.: Visual attention. In: Seeing, pp. 335–386. Academic Press (2000)Google Scholar
  6. 6.
    Marques, O., Mayron, L.M., Borba, G.B., Gamba, H.R.: An attention-driven model for grouping similar images with image retrieval applications. EURASIP Journal on Advances in Signal Processing 2007 (2007)Google Scholar
  7. 7.
    Siagian, C., Itti, L.: Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 300–312 (2007)CrossRefGoogle Scholar
  8. 8.
    Siagian, C., Itti, L.: Biologically inspired mobile robot vision localization. IEEE Transactions on Robotics 25, 861–873 (2009)CrossRefGoogle Scholar
  9. 9.
    Ma, Y.F., Hua, X.S., Lu, L., Zhang, H.J.: A generic framework of user attention model and its application in video summarization. IEEE Transactions on Multimedia 7, 907–919 (2005)CrossRefGoogle Scholar
  10. 10.
    Culibrk, D., Mirkovic, M., Zlokolica, V., Pokric, M., Crnojevic, V., Kukolj, D.: Salient motion features for video quality assessment. IEEE Transactions on Image Processing, 1 (2010)Google Scholar
  11. 11.
    Itti, L.: Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing 13, 1304–1318 (2004)CrossRefGoogle Scholar
  12. 12.
    Bruce, N., Tsotsos, J.: Saliency based on information maximization. Advances in Neural Information Processing Systems 18, 155 (2006)Google Scholar
  13. 13.
    Seo, H., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. Journal of Vision 9, 1–12 (2009)CrossRefGoogle Scholar
  14. 14.
    Zhang, L., Tong, M., Marks, T., Shan, H., Cottrell, G.: Sun: A bayesian framework for saliency using natural statistics. Journal of Vision 8, 1–20 (2008)Google Scholar
  15. 15.
    Mancas, M., Riche, N., Leroy, J., Gosselin, B.: Abnormal motion selection in crowds using bottom-up saliency. In: IEEE ICIP (2011)Google Scholar
  16. 16.
    Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 171–177 (2010)CrossRefGoogle Scholar
  17. 17.
    Mancas, M., Riche, N.: Attention website, http://tcts.fpms.ac.be/attention
  18. 18.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 1254–1259 (1998)CrossRefGoogle Scholar
  19. 19.
    Li, L., Huang, W., Gu, I., Tian, Q.: Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans. Image Processing 13, 1459–1472 (2004)CrossRefGoogle Scholar
  20. 20.
    Rosin, L.: Thresholding for change detection. In: Proc. of the Sixth International Conference on Computer Vision, ICCV 1998 (1998)Google Scholar
  21. 21.
    Hodge, V.J., Austin, J.: A survey of outlier detection methodologies. Artificial Intelligence Review 22, 85–126 (2004)MATHCrossRefGoogle Scholar
  22. 22.
    Farnebäck, G.: Two-Frame Motion Estimation Based on Polynomial Expansion. In: Bigun, J., Gustavsson, T. (eds.) SCIA 2003. LNCS, vol. 2749, pp. 363–370. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  23. 23.
    Itti, L.: Crcns-orig video and eye tracking database, http://crcns.org/data-sets/eye/eye-1
  24. 24.
    Henderson, J.M.: Diem video and eye tracking database, http://thediemproject.wordpress.com/
  25. 25.
    Hadizadeh, H., Enriquez, M.J., Bajic, I.V.: Eye tracking database for a set of standard video sequences. IEEE Trans. Image Processing (2011) (accepted for publication)Google Scholar
  26. 26.
    Dorr, M., Martinetz, T., Gegenfurtner, K.R., Barth, E.: Variability of eye movements when viewing dynamic natural scenes. Journal of Vision 10 (2010)Google Scholar
  27. 27.
    Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 171–177 (2010)CrossRefGoogle Scholar
  28. 28.
    Machine, S.: Facelab commercial eye tracking system, http://www.seeingmachines.com/product/facelab/
  29. 29.
    Peters, R.J., Iyer, A., Itti, L., Koch, C.: Components of bottom-up gaze allocation in natural images. Vision Research 45, 2397–2416 (2005)CrossRefGoogle Scholar
  30. 30.
    Green, D.M., Swets, J.A.: Signal detection theory and psychophysics. Wiley, New York (1966)Google Scholar
  31. 31.
    Borji, A.: Evaluation measures for saliency maps: Cc and nss, https://sites.google.com/site/saliencyevaluation/evaluation-measures
  32. 32.
    Lau, B.: Evaluation measures for saliency maps: Auc roc, http://www.subcortex.net/research/code/area_under_roc_curve
  33. 33.
    Gurland, J., Tripathi, R.C.: A simple approximation for unbiased estimation of the standard deviation. 25, 30–32 (1971)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Nicolas Riche
    • 1
    • 2
  • Matei Mancas
    • 1
    • 2
  • Dubravko Culibrk
    • 1
    • 2
  • Vladimir Crnojevic
    • 1
    • 2
  • Bernard Gosselin
    • 1
    • 2
  • Thierry Dutoit
    • 1
    • 2
  1. 1.University of MonsBelgium
  2. 2.University of Novi SadSerbia

Personalised recommendations