Image and Video Region Saliency Based on Space and Motion
This paper proposes a new bottom-up paradigm for detecting visual saliency in images and videos, which is based on scale space analysis of the log amplitude spectrum of natural images and videos. A salient region is assumed to be any region exhibiting a distinct pattern whose intensity, color, texture and motion is different from the rest of the image. Thus patterns which appear frequently as well as uniform regions are suppressed to produce salient region pop-out. We show that the convolution of the image log amplitude spectrum with a low-pass Gaussian kernel (at the proper scale) is equivalent to such an image saliency detector. A saliency map can then be obtained by reconstructing the 2-D signal using the original phase spectrum and an appropriately filtered log amplitude spectrum to produce pop-out. The optimal scale for each image feature channel (intensity, color, motion) is determined by minimizing the entropy of its saliency map. This produces four maps which are then fused by a weighted linear combination. Significantly, the approach does not require the setting of any parameters. We demonstrate experimentally that the proposed model has the ability to highlight small and large salient regions and to inhibit repeating patterns in both images and videos.
KeywordsAmplitude Spectrum Natural Image Neural Information Processing System Salient Region Saliency Detection
Unable to display preview. Download preview PDF.
- 3.Bernhard, S., et al.: A Nonparametric Approach to Bottom-Up Visual Saliency. Advances in Neural Information Processing Systems 19, 689–696 (2007)Google Scholar
- 4.Vijay, M., Dashan, G.L., et al.: The discriminant center-surround hypothesis for bottom-up saliency. In: Advances in Neural Information Processing Systems, vol. 20, pp. 497–504 (2008)Google Scholar
- 8.Neil, B., John, T.: Saliency Based on Information Maximization. In: Advances in Neural Information Processing Systems, vol. 18, pp. 155–162 (2006)Google Scholar
- 9.Christof, K., Jonathan, H., et al.: Graph-Based Visual Saliency. In: Advances in Neural Information Processing Systems, vol. 19, pp. 545–552 (2007)Google Scholar
- 10.Wolfgang, E., Christof, K., et al.: Predicting human gaze using low-level saliency combined with face detection. In: Advances in Neural Information Processing Systems, vol. 20, pp. 241–248 (2008)Google Scholar
- 11.Xiaodi, H., Liqing, Z.: Dynamic visual attention: searching for coding length increments. In: Advances in Neural Information Processing Systems, vol. 21, pp. 681–688 (2009)Google Scholar
- 12.Xiaodi, H., Liqing, Z.: Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2007, pp. 1–8 (2007)Google Scholar
- 13.Chenlei, G., Qi, M., et al.: Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2008, pp. 1–8 (2008)Google Scholar
- 14.Hemami, S., Estrada, F., et al.: Frequency-tuned Salient Region Detection. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2009, pp. 1597–1604 (2009)Google Scholar