International Journal of Computer Vision

, Volume 30, Issue 2, pp 79–116 | Cite as

Feature Detection with Automatic Scale Selection

  • Tony Lindeberg
Article

Abstract

The fact that objects in the world appear in different ways depending on the scale of observation has important implications if one aims at describing them. It shows that the notion of scale is of utmost importance when processing unknown measurement data by automatic methods. In their seminal works, Witkin (1983) and Koenderink (1984) proposed to approach this problem by representing image structures at different scales in a so-called scale-space representation. Traditional scale-space theory building on this work, however, does not address the problem of how to select local appropriate scales for further analysis. This article proposes a systematic methodology for dealing with this problem. A framework is presented for generating hypotheses about interesting scale levels in image data, based on a general principle stating that local extrema over scales of different combinations of γ-normalized derivatives are likely candidates to correspond to interesting structures. Specifically, it is shown how this idea can be used as a major mechanism in algorithms for automatic scale selection, which adapt the local scales of processing to the local image structure.

Support for the proposed approach is given in terms of a general theoretical investigation of the behaviour of the scale selection method under rescalings of the input pattern and by integration with different types of early visual modules, including experiments on real-world and synthetic data. Support is also given by a detailed analysis of how different types of feature detectors perform when integrated with a scale selection mechanism and then applied to characteristic model patterns. Specifically, it is described in detail how the proposed methodology applies to the problems of blob detection, junction detection, edge detection, ridge detection and local frequency estimation.

In many computer vision applications, the poor performance of the low-level vision modules constitutes a major bottleneck. It is argued that the inclusion of mechanisms for automatic scale selection is essential if we are to construct vision systems to automatically analyse complex unknown environments.

scale scale selection normalized derivative feature detection blob detection corner detection frequency estimation Gaussian derivative scale-space multi-scale representation computer vision 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abramowitz, M. and Stegun, I.A. (Eds.). 1964. Handbook of Mathematical Functions. Applied Mathematics Series. National Bureau of Standards, 55th edition.Google Scholar
  2. Almansa, A. and Lindeberg, T. 1996. Enhancement of fingerprint images by shape-adapted scale-space operators. In Gaussian Scale-Space Theory: Proc. Ph.D. School on Scale-Space Theory, Copenhagen, Denmark. J. Sporring, M. Nielsen, L. Florack, and P. Johansen (Eds.), Kluwer Academic Publishers.Google Scholar
  3. Almansa, A. and Lindeberg, T. 1998. Fingerprint enhancement by shape adaptation of scale-space operators with automatic scale selection. Technical Report ISRN KTH NA/P-98/03-SE., Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden.Google Scholar
  4. Babaud, J., Witkin, A.P., Baudin, M., and Duda, R.O. 1986. Uniqueness of the Gaussian kernel for scale-space filtering. IEEE Trans. Pattern Analysis and Machine Intell., 8(1):26-33.Google Scholar
  5. Blom, J. 1992. Topological and geometrical aspects of image structure. Ph.D. Thesis. Dept. Med. Phys. Physics, Univ. Utrecht, NL-3508 Utrecht, Netherlands.Google Scholar
  6. Blostein, D. and Ahuja, N. 1989. A multiscale region detector. Computer Vision, Graphics, and Image Processing, 45:22-41.Google Scholar
  7. Bretzner, L. and Lindeberg, T. 1996. Feature tracking with automatic selection of spatial scales. Technical Report ISRN KTH/NA/P-96/21-SE, Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden. Revised version in Computer Vision and Image Understanding.Google Scholar
  8. Bretzner, L. and Lindeberg, T. 1997. On the handling of spatial and temporal scales in feature tracking. In Proc. 1st Int. Conf. on Scale-Space Theory in Computer Vision, Utrecht, The Netherlands. ter Haar Romeny et al. (Ed.), LNCS, Springer Verlag: New York, Vol. 1252, pp. 128-139.Google Scholar
  9. Bretzner, L. and Lindeberg, T. 1998. Feature tracking with automatic selection of spatial scales. Computer Vision and Image Understanding, 71(3):385-392.Google Scholar
  10. Brunnström, K., Lindeberg, T., and Eklundh, J.-O. 1992. Active detection and classification of junctions by foveation with a headeye system guided by the scale-space primal sketch. In Proc. 2nd European Conf. on Computer Vision, Santa Margherita Ligure, Italy. G. Sandini (Ed.), Vol. 588 of Lecture Notes in Computer Science, Springer-Verlag, pp. 701-709.Google Scholar
  11. Burt, P.J. 1981. Fast filter transforms for image processing. Computer Vision, Graphics, and Image Processing, 16:20-51.Google Scholar
  12. Crowley, J.L. 1981. A representation for visual information. Ph.D. Thesis, Carnegie-Mellon University, Robotics Institute, Pittsburgh, Pennsylvania.Google Scholar
  13. Crowley, J.L. and Parker, A.C. 1984. A representation for shape based on peaks and ridges in the difference of low-pass transform. IEEE Trans. Pattern Analysis and Machine Intell., 6(2): 156-170.Google Scholar
  14. Deriche, R. and Giraudon, G. 1990. Accurate corner detection: An analytical study. In Proc. 3rd Int. Conf. on Computer Vision, Osaka, Japan, pp. 66-70.Google Scholar
  15. Dreschler, L. and Nagel, H.-H. 1982. Volumetric model and 3Dtrajectory of a moving car derived from monocular TV-frame sequences of a street scene. Computer Vision, Graphics, and Image Processing, 20(3):199-228.Google Scholar
  16. Field, D.J. 1987. Relations between the statistics of natural images and the response properties of cortical cells. J. of the Optical Society of America, 4:2379-2394.Google Scholar
  17. Florack, L.M.J. 1993. The syntactical structure of scalar images. Ph.D. Thesis. Dept. Med. Phys. Physics, Univ. Utrecht, NL-3508 Utrecht, Netherlands.Google Scholar
  18. Florack, L.M.J., ter Haar Romeny, B.M., Koenderink, J.J., and Viergever, M.A. 1992. Scale and the differential structure of images. Image and Vision Computing, 10(6):376-388.Google Scholar
  19. Florack, L.M.J., ter Haar Romeny, B.M., Koenderink, J.J., and Viergever, M.A. 1994. Linear scale-space. J. of Mathematical Imaging and Vision, 4(4):325-351.Google Scholar
  20. Förstner, W.A. and Gülch, E. 1987. A fast operator for detection and precise location of distinct points, corners and centers of circular features. In Proc. Intercommission Workshop of the Int. Soc. for Photogrammetry and Remote Sensing, Interlaken, Switzerland.Google Scholar
  21. Gårding, J. and Lindeberg, T. 1996. Direct computation of shape cues using scale-adapted spatial derivative operators. Int. J. of Computer Vision, 17(2):163-191.Google Scholar
  22. ter Haar Romeny, B. (Ed.). 1994. Geometry-Driven Diffusion in Computer Vision. Kluwer Academic Publishers, Netherlands.Google Scholar
  23. Johansen, P., Skelboe, S., Grue, K., and Andersen, J.D. 1986. Representing signals by their top points in scale-space. In Proc. 8th Int. Conf. on Pattern Recognition, Paris, France, pp. 215- 217.Google Scholar
  24. Kitchen, L. and Rosenfeld, A. 1982. Gray-level corner detection. Pattern Recognition Letters, 1(2):95-102.Google Scholar
  25. Koenderink, J.J. 1984. The structure of images. Biological Cybernetics, 50:363-370.Google Scholar
  26. Koenderink, J.J. and Richards, W. 1988. Two-dimensional curvature operators. J. of the Optical Society of America, 5:7:1136- 1141.Google Scholar
  27. Koenderink, J.J. and van Doorn, A.J. 1990. Receptive field families. Biological Cybernetics, 63:291-298.Google Scholar
  28. Koenderink, J.J. and van Doorn, A.J. 1992. Generic neighborhood operators. IEEE Trans. Pattern Analysis and Machine Intell., 14(6):597-605.Google Scholar
  29. Korn, A.F. 1988. Toward a symbolic representation of intensity changes in images. IEEE Trans. Pattern Analysis and Machine Intell., 10(5):610-625.Google Scholar
  30. Lindeberg, T. 1990. “Scale-space for discrete signals. IEEE Trans. Pattern Analysis and Machine Intell., 12(3):234-254.Google Scholar
  31. Lindeberg, T. 1991. Discrete scale-space theory and the scale-space primal sketch. Ph.D. Thesis. ISRN KTH/NA/P-91/08-SE. Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden.Google Scholar
  32. Lindeberg, T. 1993. Detecting salient blob-like image structures and their scales with a scale-space primal sketch: A method for focus-of-attention. Int. J. of Computer Vision, 11(3):283-318.Google Scholar
  33. Lindeberg, T. 1993. On scale selection for differential operators. In Proc. 8th Scandinavian Conf. on Image Analysis, Tromsø, Norway, K. Heia K.A. Høgdra, B. Braathen (Ed.), pp. 857-866.Google Scholar
  34. Lindeberg, T. 1994a. Scale-Space Theory in Computer Vision. Kluwer Academic Publishers: Netherlands.Google Scholar
  35. Lindeberg, T. 1994b. Scale selection for differential operators. Technical Report ISRN KTH/NA/P-94/03-SE, Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden.Google Scholar
  36. Lindeberg, T. 1994c. On the axiomatic foundations of linear scalespace: Combining semi-group structure with causality vs. scale invariance. Technical Report ISRN KTH/NA/P-94/20-SE, Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden. Revised version in J. Sporring and M. Nielsen and L. Florack and P. Johansen (Eds.) Gaussian Scale-Space Theory: Proc. Ph.D. School on Scale-Space Theory, Copenhagen, Denmark, Kluwer Academic Publishers.Google Scholar
  37. Lindeberg, T. 1994d. Junction detection with automatic selection of detection scales and localization scales. In Proc. 1st International Conference on Image Processing, Austin, Texas, Vol. 1, pp. 924- 928.Google Scholar
  38. Lindeberg, T. 1995a. Direct estimation of affine deformations of brightness patterns using visual front-end operators with automatic scale selection. In Proc. 5th International Conference on Computer Vision, Cambridge, MA, pp. 134-141.Google Scholar
  39. Lindeberg, T. 1995b. On scale selection in subsampled (hybrid) multi-scale representations. Draft manuscript.Google Scholar
  40. Lindeberg, T. 1996a. Feature detection with automatic scale selection. Technical Report ISRN KTH/NA/P-96/18-SE, Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden.Google Scholar
  41. Lindeberg, T. 1996b. Edge detection and ridge detection with automatic scale selection. Technical Report ISRN KTH/NA/P-96/06-SE, Dept. of Numerical Analysis and Computing Science, KTH, Stockholm, Sweden. Revised version in Intl. J. of ComputerVision, 30(2), 1998.Google Scholar
  42. Lindeberg, T. 1996c. Edge detection and ridge detection with automatic scale selection. In Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, 1996, San Francisco, California, pp. 465-470.Google Scholar
  43. Lindeberg, T. 1996d. A scale selection principle for estimating image deformations. Technical Report ISRN KTH/NA/P-96/16-SE, Dept. of Numerical Analysis and Computing Science, KTH. Image and Vision Computing, in press.Google Scholar
  44. Lindeberg, T. 1996e. On the axiomatic foundations of linear scalespace, In Gaussian Scale-Space Theory: Proc. Ph.D. School on Scale-Space Theory, Copenhagen, Denmark, J. Sporring, M. Nielsen, L. Florack, and P. Johansen (Eds.), Kluwer Academic Publishers.Google Scholar
  45. Lindeberg, T. 1997. On automatic selection of temporal scales in time-casual scale-space. In Proc. AFPAC'97: Algebraic Frames for the Perception-Action Cycle, G. Sommer and J. J. Koenderink (Eds.), Vol. 1315 of Lecture Notes in Computer Science, Springer Verlag, Berlin, pp. 94-113.Google Scholar
  46. Lindeberg, T. and Gārding, J. 1993. Shape from texture from a multiscale perspective. In Proc. 4th Int. Conf. on Computer Vision, Berlin, Germany, H.-H. Nagel et. al., (Eds.), pp. 683-691.Google Scholar
  47. Lindeberg, T. and Gārding, J. 1997. Shape-adapted smoothing in estimation of 3D depth cues from affine distortions of local 2D structure. Image and Vision Computing, 15:415-434.Google Scholar
  48. Lindeberg, T. and Li, M. 1995. Segmentation and classification of edges using minimum description length approximation and complementary junction cues. In Proc. 9th Scandinavian Conference on Image Analysis, Uppsala, Sweden, G. Borgefors (Ed.), pp. 767-776.Google Scholar
  49. Lindeberg, T. and Li, M. 1997. Segmentation and classification of edges using minimum description length approximation and complementary junction cues. Computer Vision and Image Understanding, 67(1):88-98.Google Scholar
  50. Lindeberg, T. and Olofsson, G. 1995. The aspect feature graph in recognition by parts. Draft manuscript.Google Scholar
  51. Mallat, S.G. and Hwang, W.L. 1992. Singularity detection and processing with wavelets. IEEE Trans. Information Theory, 38(2):617-643.Google Scholar
  52. Mallat, S.G. and Zhong, S. 1992. Characterization of signals from multi-scale edges. IEEE Trans. Pattern Analysis and Machine Intell., 14(7):710-723.Google Scholar
  53. Marr, D. 1982. Vision. W.H. Freeman: New York.Google Scholar
  54. Noble, J.A. 1988. Finding corners. Image and Vision Computing, 6(2):121-128.Google Scholar
  55. Pauwels, E.J., Fiddelaers, P., Moons, T., and van Gool, L. J. 1995. An extended class of scale-invariant and recursive scale-space filters. IEEE Trans. Pattern Analysis and Machine Intell., 17(7): 691-701.Google Scholar
  56. Tsotsos, J.K., Culhane, S.M., Wai, W.Y.K., Lai, Y., Davis, N., and Nufflo, F. 1995. Modeling visual attention via selective tuning, Artificial Intelligence, 78:507-545.Google Scholar
  57. Voorhees, H. and Poggio, T. 1987. Detecting textons and texture boundaries in natural images. In Proc. 1st Int. Conf. on Computer Vision, London, England.Google Scholar
  58. Wiltschi, K., Pinz, A., and Lindeberg, T. 1997. Classification of carbide distributions using scale selection and directional distributions, In Proc. 4th International Conference on Image Processing, Santa Barbara, CA.Google Scholar
  59. Witkin, A.P. 1983. Scale-space filtering. In Proc. 8th Int. Joint Conf. Art. Intell., Karlsruhe, West Germany, pp. 1019-1022.Google Scholar
  60. Young, R.A. 1985. The Gaussian derivative theory of spatial vision: Analysis of cortical cell receptive field line-weighting pro-files. Technical Report GMR-4920, Computer Science Department, General Motors Research Lab., Warren, Michigan.Google Scholar
  61. Young, R.A. 1987. The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spatial Vision, 2:273-293.Google Scholar
  62. Yuille, A.L. and Poggio, T.A. 1986. Scaling theorems for zerocrossings. IEEE Trans. Pattern Analysis and Machine Intell., 8: 15-25.Google Scholar
  63. Zhang, W. and Bergholm, F. 1993. An extension of Marr's signature based edge classification and other methods for determination of diffuseness and height of edges, as well as line width. In Proc. 4th Int. Conf. on Computer Vision, Berlin, Germany, H.-H. Nagel et al. (Eds.), IEEE Computer Society Press, pp. 183-191.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Tony Lindeberg
    • 1
  1. 1.Computational Vision and Active Perception Laboratory (CVAP), Department of Numerical Analysis and Computing ScienceKTH (Royal Institute of Technology)StockholmSweden. E-mail

Personalised recommendations