Skip to main content
SpringerLink
Log in

Feature tracking and matching in video using programmable graphics hardware

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper describes novel implementations of the KLT feature tracking and SIFT feature extraction algorithms that run on the graphics processing unit (GPU) and is suitable for video analysis in real-time vision systems. While significant acceleration over standard CPU implementations is obtained by exploiting parallelism provided by modern programmable graphics hardware, the CPU is freed up to run other computations in parallel. Our GPU-based KLT implementation tracks about a thousand features in real-time at 30 Hz on 1,024 × 768 resolution video which is a 20 times improvement over the CPU. The GPU-based SIFT implementation extracts about 800 features from 640 × 480 video at 10 Hz which is approximately 10 times faster than an optimized CPU implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bramberger, M., Rinner, B., Schwabach, H.: An embedded smart camera on a scalable heterogeneous multi-DSP system. In: Proceedings of the European DSP Education and Research Symposium (EDERS 2004) (2004)

  2. Klupsch, S., Ernst, M., Huss, S.A., Rumpf, M., Strzodka, R.: Real time image processing based on reconfigurable hardware acceleration. In: Proceedings of IEEE Workshop Heterogeneous Reconfigurable Systems on Chip (2002)

  3. Fung, J., Mann, S.: OpenVIDIA: parallel GPU computer vision. ACM MULTIMEDIA 2005, pp. 849–852 (2005)

  4. Fung, J., Mann, S.: Computer vision signal processing on graphics processing units. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004), Montreal pp. V-93–V-96 (2004)

  5. Gong, M., Langille, A., Gong, M.: Real-time image processing using graphics hardware: a performance study. In: International Conference on Image Analysis and Recognition, pp. 1217–1225 (2005)

  6. Tomasi, C., Kanade, T.: Detection and Tracking of Point Features. Tech. Rept. CMU-CS-91132, Carnegie Mellon University (1991)

  7. Lukas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

  8. Birchfield, S.: Derivation of Kanade-Lucas-Tomasi tracking equation. unpublished notes (1997)

  9. Birchfield, S.: KLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker. http://www.ces.clemson.edu/~stb/klt (2005)

  10. Lowe D.G. (2004). Distinctive image features from scale-invariant keypoints. IJCV 60(2): 91–110

    Article  Google Scholar 

  11. Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Conference on Computer Vision and Pattern Recognition (CVPR) pp. 211–217 (2003)

  12. Zach, C., Bischof, H., Karner, K.: Hierarchical disparity estimation with programmable 3D hardware. In: WSCG (International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision). Short Communications, pp. 275–282, Plzen, Slowakei (2004)

  13. Woetzel, J., Koch, R.: Real-time multi-stereo depth estimation on GPU with approximative discontinuity handling. In: European Conf. on Visual Media Production (2004)

  14. Labatut P., Keriven R. and Pons J.-P. (2006). A GPU implementation of level set multiview stereo. Int. Conf. Comput. Sci. 4: 212–219

    Google Scholar 

  15. Yang R. and Welch G. (2002). Fast image segmentation and smoothing using commodity graphics hardware. J. Graph. Tools 7(4): 91–100

    Google Scholar 

  16. Strzodka R., Droske M. and Rumpf M. (2004). Image registration by a regularized gradient flow—a streaming implementation in DX9 graphics hardware. Computing 73(4): 373–389

    Article  MathSciNet  MATH  Google Scholar 

  17. Griesser, A., Roeck, S.D., Neubeck, A., Gool, L.J.V.: GPU-based foreground-background segmentation using an extended colinearity criterion. In: Vision, Modeling, and Visualization (VMV) (2005)

  18. Pollefeys M., Gool L.J.V., Vergauwen M., Verbiest F., Cornelis K., Tops J. and Koch R. (2004). Visual Modeling with a Hand-Held Camera. IJCV 59(3): 207–232

    Article  Google Scholar 

  19. Akbarzadeh, A., Frahm, J.-M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nistr, D., Pollefeys, M.: Towards urban 3D reconstruction from video, Invited paper. In: 3rd International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT) (2006)

  20. GPGPU: General-Purpose Computation on GPUs. http://www.gpgpu.org (2004)

  21. Bjorke, K.A.: NVIDIA Corporation. Image processing using parallel GPU units. Proceedings of SPIE, vol. 6065 (2006)

  22. Pharr M. and Fernando R. (2005). GPU Gems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation. Addison-Wesley Prof, Reading

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, CB# 3175 Sitterson Hall, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA

    Sudipta N. Sinha, Jan-Michael Frahm & Marc Pollefeys

  2. Real-time Vision and Modeling Department, Siemens Corporate Research, 755 College Road East, Princeton, NJ, 08540, USA

    Yakup Genc

Authors
  1. Sudipta N. Sinha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan-Michael Frahm
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marc Pollefeys
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yakup Genc
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudipta N. Sinha.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sinha, S.N., Frahm, JM., Pollefeys, M. et al. Feature tracking and matching in video using programmable graphics hardware. Machine Vision and Applications 22, 207–217 (2011). https://doi.org/10.1007/s00138-007-0105-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-007-0105-z

Keywords

Search

Navigation