Skip to main content
SpringerLink
Log in

Interest Point Detection and Scale Selection in Space-Time

  • Conference paper
  • First Online:
Scale Space Methods in Computer Vision (Scale-Space 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2695))

Included in the following conference series:

Abstract

Several types of interest point detectors have been proposed for spatial images. This paper investigates how this notion can be generalised to the detection of interesting events in space-time data. Moreover, we develop a mechanism for spatio-temporal scale selection and detect events at scales corresponding to their extent in both space and time. To detect spatio-temporal events, we build on the idea of the Harris and Förstner interest point operators and detect regions in space-time where the image structures have significant local variations in both space and time. In this way, events that correspond to curved space-time structures are emphasised, while structures with locally constant motion are disregarded.

To construct this operator, we start from a multi-scale windowed second moment matrix in space-time, and combine the determinant and the trace in a similar way as for the spatial Harris operator. All spacetime maxima of this operator are then adapted to characteristic scales by maximising a scale-normalised space-time Laplacian operator over both spatial scales and temporal scales. The motivation for performing temporal scale selection as a complement to previous approaches of spatial scale selection is to be able to robustly capture spatio-temporal events of different temporal extent. It is shown that the resulting approach is truly scale invariant with respect to both spatial scales and temporal scales. The proposed concept is tested on synthetic and real image sequences. It is shown that the operator responds to distinct and stable points in space-time that often correspond to interesting events. The potential applications of the method are discussed.

The support from the Swedish Research Council and from the Royal Swedish Academy of Sciences as well as the Knut and Alice Wallenberg Foundation is gratefully acknowledged.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Almansa, A. and Lindeberg, T. (2000). Fingerprint enhancement by shape adaptation of scale-space operators with automatic scale-selection, IEEE Transactions on Image Processing 9(12): 2027–2042.

    Article  MATH  MathSciNet  Google Scholar 

  • Barron, J., Fleet, D. and Beauchemin, S. (1994). Performance of optical flow techniques, International Journal of Computer Vision 12(1): 43–77.

    Article  Google Scholar 

  • Bigün, J., Granlund, G. and Wiklund, J. (1991). Multidimensional orientation estimation with applications to texture analysis and optical flow, IEEE Transactions on Pattern Analysis and Machine Intelligence 13(8): 775–790.

    Article  Google Scholar 

  • Black, M. and Jepson, A. (1998). Eigen tracking: Robust matching and tracking of articulated objects using view-based representation, International Journal of Computer Vision 26(1): 63–84.

    Article  Google Scholar 

  • Blake, A. and Isard, M. (1998). Condensation — conditional density propagation for visual tracking, IJCV 29(1): 5–28.

    Article  Google Scholar 

  • Chomat, O., de Verdiere, V., Hall, D. and Crowley, J. (2000a). Local scale selection for Gaussian based description techniques, Proc. Sixth European Conference on Computer Vision, Vol. 1842 of Lecture Notes in Computer Science, Springer Verlag, Berlin, Dublin, Ireland, pp. 117–133.

    Google Scholar 

  • Chomat, O., Martin, J. and Crowley, J. (2000b). A probabilistic sensor for the perception and recognition of activities, Proc. Sixth European Conference on Computer Vision, Dublin, Ireland, pp. I:487–503.

    Google Scholar 

  • Fleet, D., Black, M. and Jepson, A. (1998). Motion feature detection using steerable flow fields, Proc. Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 274–281.

    Google Scholar 

  • Florack, L. M. J. (1997). Image Structure, Kluwer Academic Publishers, Dordrecht, Netherlands.

    Google Scholar 

  • Förstner, W. A. and Gülch, E. (1987). A fast operator for detection and precise location of distinct points, corners and centers of circular features, Proc. Intercommission Workshop of the Int. Soc. for Photogrammetry and Remote Sensing, Interlaken, Switzerland.

    Google Scholar 

  • Garding, J. and Lindeberg, T. (1996). Direct computation of shape cues using scale-adapted spatial derivative operators, International Journal of Computer Vision 17(2): 163–191.

    Article  Google Scholar 

  • Hall, D., de Verdiere, V. and Crowley, J. (2000). Object recognition using coloured receptive fields, Proc. Sixth European Conference on Computer Vision, Vol. 1842 of Lecture Notes in Computer Science, Springer Verlag, Berlin, Dublin, Ireland, pp. 164–177.

    Google Scholar 

  • Harris, C. and Stephens, M. (1988). A combined corner and edge detector, Alvey Vision Conference, pp. 147–152.

    Google Scholar 

  • Koenderink, J. J. (1988). Scale-time, Biological Cybernetics 58: 159–162.

    Article  MATH  MathSciNet  Google Scholar 

  • Koenderink, J. J. and van Doorn, A. J. (1992). Generic neighborhood operators, IEEE Transactions on Pattern Analysis and Machine Intelligence 14(6): 597–605.

    Article  Google Scholar 

  • Laptev, I. and Lindeberg, T. (2002). Velocity-adaptation of spatio-temporal receptive fields for direct recognition of activities: An experimental study, in D. Suter (ed.), Proc. ECCV’02 workshop on Statistical Methods in Video Processing, Copenhagen, Denmark, pp. 61–66.

    Google Scholar 

  • Lindeberg, T. (1994). Scale-Space Theory in Computer Vision, Kluwer Academic Publishers, Boston.

    Google Scholar 

  • Lindeberg, T. (1997). On automatic selection of temporal scales in time-causal scale-space, AFPAC’97: Algebraic Frames for the Perception-Action Cycle, Vol. 1315 of Lecture Notes in Computer Science, Springer Verlag, Berlin, pp. 94–113.

    Chapter  Google Scholar 

  • Lindeberg, T. (1998). Feature detection with automatic scale selection, International Journal of Computer Vision 30(2): 77–116.

    Google Scholar 

  • Lindeberg, T. and Bretzner, L. (2003). Real-time scale selection in hybrid multi-scale representations, Proc. Scale-Space’03, LNCS, Springer Verlag, these proceedings.

    Google Scholar 

  • Lindeberg, T. and Fagerström, D. (1996). Scale-space with causal time direction, Proc. Fourth European Conference on Computer Vision, Vol. 1064 of Lecture Notes in Computer Science, Springer Verlag, Berlin, Cambridge, UK, pp. I:229–240.

    Google Scholar 

  • Lowe, D. (1999). Object recognition from local scale-invariant features, Proc. Seventh International Conference on Computer Vision, Corfu, Greece, pp. 1150–1157.

    Google Scholar 

  • Mikolajczyk, K. and Schmid, C. (2001). Indexing based on scale invariant interest points, Proc. Eighth International Conference on Computer Vision, Vancouver, Canada, pp. I:525–531.

    Google Scholar 

  • Mikolajczyk, K. and Schmid, C. (2002). An affine invariant interest point detector, Proc. Seventh European Conference on Computer Vision, Vol. 2350 of Lecture Notes in Computer Science, Springer Verlag, Berlin, Copenhagen, Denmark, pp. I:128–142.

    Google Scholar 

  • Niyogi, S. A. (1995). Detecting kinetic occlusion, Proc. Fifth International Conference on Computer Vision, Cambridge, MA, pp. 1044–1049.

    Google Scholar 

  • Schmid, C. and Mohr, R. (1997). Local grayvalue invariants for image retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence 19(5): 530–535.

    Article  Google Scholar 

  • Schmid, C., Mohr, R. and Bauckhage, C. (2000). Evaluation of interest point detectors, International Journal of Computer Vision 37(2): 151–172.

    Article  MATH  Google Scholar 

  • Smith, S. and Brady, J. (1995). ASSET-2: Real-time motion segmentation and shape tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 17(8): 814–820.

    Article  Google Scholar 

  • Tell, D. and Carlsson, S. (2002). Combining topology and appearance for wide baseline matching, Proc. Seventh European Conference on Computer Vision, Vol. 2350 of Lecture Notes in Computer Science, Springer Verlag, Berlin, Copenhagen, Denmark, pp. I:68–83.

    Google Scholar 

  • Tuytelaars, T. and Van Gool, L. (2000). Wide baseline stereo matching based on local, affinely invariant regions, British Machine Vision Conference, pp. 412–425.

    Google Scholar 

  • Witkin, A. P. (1983). Scale-space filtering, Proc. 8th Int. Joint Conf. Art. Intell., Karlsruhe, Germany, pp. 1019–1022.

    Google Scholar 

  • Zelnik-Manor, L. and Irani, M. (2001). Event-based analysis of video, Proc. Computer Vision and Pattern Recognition, Kauai Marriott, Hawaii, pp. II:123–130.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computational Vision and Active Perception Laboratory (CVAP) Dept. of Numerical Analysis and Computing Science, KTH, S-100 44, Stockholm, Sweden

    Ivan Laptev & Tony Lindeberg

Authors
  1. Ivan Laptev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tony Lindeberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Imaging Sciences, 5th Floor Thomas Guy House, Guy’s Campus, London, SE1 9RT, UK

    Lewis D. Griffin

  2. IT University of Copenhagen, Glentevej 67-69, Copenhagen NV, Denmark

    Martin Lillholm

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Laptev, I., Lindeberg, T. (2003). Interest Point Detection and Scale Selection in Space-Time. In: Griffin, L.D., Lillholm, M. (eds) Scale Space Methods in Computer Vision. Scale-Space 2003. Lecture Notes in Computer Science, vol 2695. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44935-3_26

Download citation

  • DOI: https://doi.org/10.1007/3-540-44935-3_26

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40368-5

  • Online ISBN: 978-3-540-44935-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation