Advertisement

Image and Video Region Saliency Based on Space and Motion

  • Jian Li
  • Martin Levine
  • Xiangjing An
  • Zhenping Sun
  • Hangen He
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 83)

Abstract

This paper proposes a new bottom-up paradigm for detecting visual saliency in images and videos, which is based on scale space analysis of the log amplitude spectrum of natural images and videos. A salient region is assumed to be any region exhibiting a distinct pattern whose intensity, color, texture and motion is different from the rest of the image. Thus patterns which appear frequently as well as uniform regions are suppressed to produce salient region pop-out. We show that the convolution of the image log amplitude spectrum with a low-pass Gaussian kernel (at the proper scale) is equivalent to such an image saliency detector. A saliency map can then be obtained by reconstructing the 2-D signal using the original phase spectrum and an appropriately filtered log amplitude spectrum to produce pop-out. The optimal scale for each image feature channel (intensity, color, motion) is determined by minimizing the entropy of its saliency map. This produces four maps which are then fused by a weighted linear combination. Significantly, the approach does not require the setting of any parameters. We demonstrate experimentally that the proposed model has the ability to highlight small and large salient regions and to inhibit repeating patterns in both images and videos.

Keywords

Amplitude Spectrum Natural Image Neural Information Processing System Salient Region Saliency Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Laurent, I., Christof, K.: Computational modelling of visual attention. Nature Reviews Neuroscience 2(3), 194–203 (2001)CrossRefGoogle Scholar
  2. 2.
    Yantis, S.: How visual salience wins the battle for awareness. Nature Neuroscience 8(8), 975–977 (2005)CrossRefGoogle Scholar
  3. 3.
    Bernhard, S., et al.: A Nonparametric Approach to Bottom-Up Visual Saliency. Advances in Neural Information Processing Systems 19, 689–696 (2007)Google Scholar
  4. 4.
    Vijay, M., Dashan, G.L., et al.: The discriminant center-surround hypothesis for bottom-up saliency. In: Advances in Neural Information Processing Systems, vol. 20, pp. 497–504 (2008)Google Scholar
  5. 5.
    Chenlei, G., Liming, Z.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE transactions on image processing 19(1), 185–198 (2010)CrossRefGoogle Scholar
  6. 6.
    Christof, K., Laurent, I., et al.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  7. 7.
    Beck, D.M., Kastner, S.: Stimulus context modulates competition in human extrastriate cortex. Nature Neuroscience 8(8), 1110–1116 (2005)CrossRefGoogle Scholar
  8. 8.
    Neil, B., John, T.: Saliency Based on Information Maximization. In: Advances in Neural Information Processing Systems, vol. 18, pp. 155–162 (2006)Google Scholar
  9. 9.
    Christof, K., Jonathan, H., et al.: Graph-Based Visual Saliency. In: Advances in Neural Information Processing Systems, vol. 19, pp. 545–552 (2007)Google Scholar
  10. 10.
    Wolfgang, E., Christof, K., et al.: Predicting human gaze using low-level saliency combined with face detection. In: Advances in Neural Information Processing Systems, vol. 20, pp. 241–248 (2008)Google Scholar
  11. 11.
    Xiaodi, H., Liqing, Z.: Dynamic visual attention: searching for coding length increments. In: Advances in Neural Information Processing Systems, vol. 21, pp. 681–688 (2009)Google Scholar
  12. 12.
    Xiaodi, H., Liqing, Z.: Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2007, pp. 1–8 (2007)Google Scholar
  13. 13.
    Chenlei, G., Qi, M., et al.: Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2008, pp. 1–8 (2008)Google Scholar
  14. 14.
    Hemami, S., Estrada, F., et al.: Frequency-tuned Salient Region Detection. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2009, pp. 1597–1604 (2009)Google Scholar
  15. 15.
    Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognitive Psychology 12(1), 97–136 (1980)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jian Li
    • 1
  • Martin Levine
    • 2
  • Xiangjing An
    • 1
  • Zhenping Sun
    • 1
  • Hangen He
    • 1
  1. 1.Institute of AutomationNatl. Univ. of Defense TechnologyP.R. China
  2. 2.Center for Intelligent MachinesMcGill Univ.Canada

Personalised recommendations