Advertisement

International Journal of Computer Vision

, Volume 45, Issue 2, pp 83–105 | Cite as

Saliency, Scale and Image Description

  • Timor Kadir
  • Michael Brady
Article

Abstract

Many computer vision problems can be considered to consist of two main tasks: the extraction of image content descriptions and their subsequent matching. The appropriate choice of type and level of description is of course task dependent, yet it is generally accepted that the low-level or so called early vision layers in the Human Visual System are context independent.

This paper concentrates on the use of low-level approaches for solving computer vision problems and discusses three inter-related aspects of this: saliency; scale selection and content description. In contrast to many previous approaches which separate these tasks, we argue that these three aspects are intrinsically related. Based on this observation, a multiscale algorithm for the selection of salient regions of an image is introduced and its application to matching type problems such as tracking, object recognition and image retrieval is demonstrated.

visual saliency scale selection image content descriptors feature extraction salient features image database entropy scale-space 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alvarez, L., Lions, P., and Morel, J. 1992. Image selective smoothing and edge detection by nonlinear diffusion. II. SIAM Journal on Numerical Analysis, 29(3):845–866.Google Scholar
  2. Bergholm, F. 1986. Edge focusing. In Proc.Int.Conf.on Pattern Recognition, Paris, France, pp. 597–600.Google Scholar
  3. Blake, A. and Isard, M. 1997. The CONDENSATION algorithm-conditional density propagation and applications to visual tracking. In Advances in Neural Information Processing Systems,Vol. 9, M.C. Mozer, M.I. Jordan, and T. Petsche (Eds.). MIT Press: Cambridge, MA.Google Scholar
  4. Burt, P.J. and Adelson, E.H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans.Communication, 31(4):532–540.Google Scholar
  5. Chomat, O., deVerdiere, V.C., Hall, D., and Crowley, J.L. 2000. Local scale selection for Gaussian based description techniques. In Proc.European Conf.Computer Vision, pp. 117–133.Google Scholar
  6. Coifman, R. and Wickerhauser, M. 1992. Entropy-based algorithms for best basis selection. IEEE Trans.on Information Theory, 38(2):713–718.Google Scholar
  7. Deriche, R. and Blaszka, T. 1993. Recovering and characterizing image features using an efficient model based approach. In Proc.Int.Conf.on Computer Vision and Pattern Recognition, pp. 530–535.Google Scholar
  8. Dufournaud, Y., Schmid, C., and Horaud, R. 2000. Matching images with different resolutions. In Proc.Int.Conf.on Computer Vision and Pattern Recognition, pp. 612–618.Google Scholar
  9. Gilles, S. 1998. Robust description and matching of images. Ph.D. Thesis, University of Oxford.Google Scholar
  10. Harris, C. and Stephens, M. 1988. A combined corner and edge detector. In Proc.4th Alvey Vision Conf,Manchester, pp. 189–192.Google Scholar
  11. Jägersand, M. 1995. Saliency maps and attention selection in scale and spatial coordinates: An information theoretic approach. In Proc.Int.Conf.on Computer Vision. MIT Press: Cambridge, MA, pp. 195–202.Google Scholar
  12. Julesz, B. 1995. Dialogues on Perception. MIT Press: Cambridge, MA.Google Scholar
  13. Koenderink, J.J. 1984. The structure of images. Biological Cybernetics, 50:363–370.Google Scholar
  14. Koenderink, J.J. and van Doorn, A.J. 1987. Representation of local geometry in the visual system. Biological Cybernetics, 63: 291–297.Google Scholar
  15. Leclerc, Y.G. 1989. Constructing simple stable descriptions for image partitioning. Int.Journal of Computer Vision, 3:73–102.Google Scholar
  16. Lindeberg, T. 1993. On scale selection for differential operators. In Proc.8th Scandinavian Conf.on Image Analysis, Tromso, Norway, pp. 857–866.Google Scholar
  17. Lindeberg, T. 1994. Junction detection with automatic selection of detection scales and localization scales. In Proc.Int.Conf.on Image Processing, pp. 924–928.Google Scholar
  18. Lindeberg, T. and ter Haar Romeny, B.M. 1994. Linear Scale-Space: I.Basic Theory, II.Early Visual Operations. Kluwer Academic Publishers: Dordrecht, The Netherlands, pp. 1–77.Google Scholar
  19. Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press: San Diego.Google Scholar
  20. Marr, D. 1982. Vision: A Computational Investigation into the Human Representation ond Processing of Visual Information. W.H. Freeman: San Francisco.Google Scholar
  21. Marr, D. and Hildreth, E. 1979. Theory of edge detection. In Proceedings Royal Society of London Bulletin, 204:301–328.Google Scholar
  22. Milanese, R. 1993. Detecting salient regions in an image: From biological evidence to computer implementation. Ph.D. Thesis, University of Geneva.Google Scholar
  23. Mokhtarian, F. and Suomela, R. 1998. Robust image corner detection through curvature scale space. IEEE Trans.Pattern Analysis and Machine Intelligence, 20(12):1376–1381.Google Scholar
  24. Neisser, U. 1964.Visual search. Scientific American, 210(6):94–102.Google Scholar
  25. Nene, S., Nayar, S., and Murase, H. 1996. Columbia image object library. Technical Report, Department of Computer Science, Columbia University.Google Scholar
  26. Perona, P. and Malik, J. 1988. Scale space and edge detection using anisotropic diffusion. In Proc.Int.Conf.on Computer Vision and Pattern Recognition, pp. 16–22.Google Scholar
  27. Schiele, B. 1997. Object recognition using multidimensional receptive field histograms. Ph.D. Thesis, I.N.P. de Grenoble.Google Scholar
  28. Schmid, C. and Mohr, R. 1997. Local greyvalue invariants for image retrieval. IEEE Trans.Pattern Analysis and Machine Intelligence, 19(5):530–535.Google Scholar
  29. Schmid, C., Mohr, R., and Bauckhage, C. 1998. Comparing and evaluating interest points. In Proc.Int.Conf.on Computer Vision, pp. 230–235.Google Scholar
  30. Sha'ashua, A. and Ullman, S. 1988. Structural saliency: The detection of globally salient structures using a locally connected network. In Proc.Int.Conf.on Computer Vision, Tampa, FL, pp. 321–327.Google Scholar
  31. Starck, J. and Murtagh, F. 1999. Multiscale entropy filtering. EURASIP Signal Processing, pp. 147–165.Google Scholar
  32. Swain, M.J. 1990. Color indexing. Ph.D. Thesis, University of Rochester.Google Scholar
  33. ter Haar Romeny, B.M. 1996. Introduction to scale-space theory: Multiscale geometric image analysis. Ph.D. Thesis, Utrecht University.Google Scholar
  34. Treisman, A. 1985. Preattentive processing in vision. Computer Vision, Graphics, and Image Processing, 31(2):156–177.Google Scholar
  35. Walker, K.N., Cootes, T.F., and Taylor, C.J. 1998a. Locating salient facial features using image invariants. In Int.Conf.on Automatic Face and Gesture Recognition, Nara, Japan.Google Scholar
  36. Walker, K.N., Cootes, T.F., and Taylor, C.J. 1998b. Locating salient object features. In Proc.British Machine Vision Conference, Southampton, UK, pp. 557–566.Google Scholar
  37. Weickert, J. 1997. A review of nonlinear diffusion filtering. Lecture Notes in Computer Science, 1252:3–28.Google Scholar
  38. Winter, A., Maitre, H., Cambou, N., and Legrand, E. 1997. Entropy and multiscale analysis: An original feature extraction algorithm for aerial and satellite images. In Proc.Int.Conf.on Acoustics, Speech, and Signal Processing. IEEE.Google Scholar
  39. Witkin, A. 1983. Scale-space filtering. In Proc.Int.Joint Conf.on Artificial Intelligence, Karlsruhe, Germany.Google Scholar
  40. Zheng, B.Y., Qian, W., and Clarke, L.P. 1996. Digital mammography-Mixed feature neural network with spectral entropy decision for detection of microcalcifications. IEEE Trans.on Medical Imaging, 15(5):589–597.Google Scholar
  41. Zhu, S.C. and Yuille, A. 1996. Region competition: Unifying snakes, region growing, and bayes/mdl for multiband image segmentation. IEEE Pattern Analysis and Machine Intelligence, 18(9):884–900.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Timor Kadir
    • 1
  • Michael Brady
    • 1
  1. 1.Robotics Research Laboratory, Department of Engineering ScienceUniversity of OxfordOxfordUK

Personalised recommendations