An Eye Fixation Database for Saliency Detection in Images

  • Subramanian Ramanathan
  • Harish Katti
  • Nicu Sebe
  • Mohan Kankanhalli
  • Tat-Seng Chua
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6314)

Abstract

To learn the preferential visual attention given by humans to specific image content, we present NUSEF- an eye fixation database compiled from a pool of 758 images and 75 subjects. Eye fixations are an excellent modality to learn semantics-driven human understanding of images, which is vastly different from feature-driven approaches employed by saliency computation algorithms. The database comprises fixation patterns acquired using an eye-tracker, as subjects free-viewed images corresponding to many semantic categories such as faces (human and mammal), nudes and actions (look, read and shoot). The consistent presence of fixation clusters around specific image regions confirms that visual attention is not subjective, but is directed towards salient objects and object-interactions.

We then show how the fixation clusters can be exploited for enhancing image understanding, by using our eye fixation database in an active image segmentation application. Apart from proposing a mechanism to automatically determine characteristic fixation seeds for segmentation, we show that the use of fixation seeds generated from multiple fixation clusters on the salient object can lead to a 10% improvement in segmentation performance over the state-of-the-art.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  2. 2.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vision 81(1), 2–23 (2009)CrossRefGoogle Scholar
  3. 3.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vision 42(3), 145–175 (2001)MATHCrossRefGoogle Scholar
  4. 4.
    Van Gemert, J.C., Geusebroek, J.M., Veenman, C.J., Snoek, C.G.M., Smeulders, A.W.M.: Robust scene categorization by learning image statistics in context. In: CVPR-SLAM Workshop (2006)Google Scholar
  5. 5.
    Zheng, Y.T., Neo, S.Y., Chua, T.S., Tian, Q.: Visual synset: a higher-level visual representation for object-based image retrieval. The Visual Computer 25(1), 13–23 (2009)CrossRefGoogle Scholar
  6. 6.
    Uijlings, J.R.R., Smeulders, A.W.M., Scha, R.J.H.: Real-time bag of words, approximately. In: CIVR (2009)Google Scholar
  7. 7.
    Spain, M., Perona, P.: Some objects are more equal than others: Measuring and predicting importance. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 523–536. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: ICCV (2009)Google Scholar
  9. 9.
    Peters, R.J., Iyer, A., Itti, L., Koch, C.: Components of bottom-up gaze allocation in natural images. Vision Research 45(8), 2397–2416 (2005)CrossRefGoogle Scholar
  10. 10.
    Valenti, R., Sebe, N., Gevers, T.: Image saliency by isocentric curvedness and color. In: ICCV (2009)Google Scholar
  11. 11.
    Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object. In: CVPR (2007)Google Scholar
  12. 12.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision 8(7), 1–20 (2008)CrossRefGoogle Scholar
  13. 13.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. PAMI 20(11), 1254–1259 (1998)Google Scholar
  14. 14.
    Bruce, N., Tsotsos, J.: Saliency, attention, and visual search: An information theoretic approach. J. of Vision 9(3), 1–24 (2009)CrossRefGoogle Scholar
  15. 15.
    Subramanian, R., Harish, K., Raymond, H., Chua, T.S., Kankanhalli, M.: Automated localization of affective objects and actions in images via caption text-cum-eye gaze analysis. In: ACM MM, pp. 729–732 (2009)Google Scholar
  16. 16.
    Einhuser, W., Spain, M., Perona, P.: Objects predict fixations better than early saliency. J. Vis. 8(14), 1–26 (2008)CrossRefGoogle Scholar
  17. 17.
    Mishra, A., Aloimonos, Y., Fah, C.L.: Active segmentation with fixation. In: ICCV (2009)Google Scholar
  18. 18.
    Lang, P., Bradley, M., Cuthbert, B.: (iaps): Affective ratings of pictures and instruction manual. Technical report, University of Florida (2008)Google Scholar
  19. 19.
    Bagon, S., Boiman, O., Irani, M.: What is a good image segment? A unified approach to segment extraction. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 30–44. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: CVPR (2007)Google Scholar
  21. 21.
    Arbelaez, P., Cohen, L.: Constraine image segmentation from hierarchical boundaries. In: CVPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Subramanian Ramanathan
    • 1
  • Harish Katti
    • 2
  • Nicu Sebe
    • 1
  • Mohan Kankanhalli
    • 2
  • Tat-Seng Chua
    • 2
  1. 1.Department of Information Engineering and Computer ScienceUniversity of TrentoItaly
  2. 2.School of ComputingNational University of Singapore (NUS)Singapore

Personalised recommendations