Machine Vision and Applications

, Volume 22, Issue 1, pp 207–217 | Cite as

Feature tracking and matching in video using programmable graphics hardware

  • Sudipta N. Sinha
  • Jan-Michael Frahm
  • Marc Pollefeys
  • Yakup Genc
Original Paper


This paper describes novel implementations of the KLT feature tracking and SIFT feature extraction algorithms that run on the graphics processing unit (GPU) and is suitable for video analysis in real-time vision systems. While significant acceleration over standard CPU implementations is obtained by exploiting parallelism provided by modern programmable graphics hardware, the CPU is freed up to run other computations in parallel. Our GPU-based KLT implementation tracks about a thousand features in real-time at 30 Hz on 1,024 × 768 resolution video which is a 20 times improvement over the CPU. The GPU-based SIFT implementation extracts about 800 features from 640 × 480 video at 10 Hz which is approximately 10 times faster than an optimized CPU implementation.


Visual tracking Vehicle tracking Video surveillance Visual inspection Vision system Robot navigation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bramberger, M., Rinner, B., Schwabach, H.: An embedded smart camera on a scalable heterogeneous multi-DSP system. In: Proceedings of the European DSP Education and Research Symposium (EDERS 2004) (2004)Google Scholar
  2. 2.
    Klupsch, S., Ernst, M., Huss, S.A., Rumpf, M., Strzodka, R.: Real time image processing based on reconfigurable hardware acceleration. In: Proceedings of IEEE Workshop Heterogeneous Reconfigurable Systems on Chip (2002)Google Scholar
  3. 3.
    Fung, J., Mann, S.: OpenVIDIA: parallel GPU computer vision. ACM MULTIMEDIA 2005, pp. 849–852 (2005)Google Scholar
  4. 4.
    Fung, J., Mann, S.: Computer vision signal processing on graphics processing units. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004), Montreal pp. V-93–V-96 (2004)Google Scholar
  5. 5.
    Gong, M., Langille, A., Gong, M.: Real-time image processing using graphics hardware: a performance study. In: International Conference on Image Analysis and Recognition, pp. 1217–1225 (2005)Google Scholar
  6. 6.
    Tomasi, C., Kanade, T.: Detection and Tracking of Point Features. Tech. Rept. CMU-CS-91132, Carnegie Mellon University (1991)Google Scholar
  7. 7.
    Lukas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  8. 8.
    Birchfield, S.: Derivation of Kanade-Lucas-Tomasi tracking equation. unpublished notes (1997)Google Scholar
  9. 9.
    Birchfield, S.: KLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker. (2005)
  10. 10.
    Lowe D.G. (2004). Distinctive image features from scale-invariant keypoints. IJCV 60(2): 91–110 CrossRefGoogle Scholar
  11. 11.
    Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Conference on Computer Vision and Pattern Recognition (CVPR) pp. 211–217 (2003)Google Scholar
  12. 12.
    Zach, C., Bischof, H., Karner, K.: Hierarchical disparity estimation with programmable 3D hardware. In: WSCG (International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision). Short Communications, pp. 275–282, Plzen, Slowakei (2004)Google Scholar
  13. 13.
    Woetzel, J., Koch, R.: Real-time multi-stereo depth estimation on GPU with approximative discontinuity handling. In: European Conf. on Visual Media Production (2004)Google Scholar
  14. 14.
    Labatut P., Keriven R. and Pons J.-P. (2006). A GPU implementation of level set multiview stereo. Int. Conf. Comput. Sci. 4: 212–219 Google Scholar
  15. 15.
    Yang R. and Welch G. (2002). Fast image segmentation and smoothing using commodity graphics hardware. J. Graph. Tools 7(4): 91–100 Google Scholar
  16. 16.
    Strzodka R., Droske M. and Rumpf M. (2004). Image registration by a regularized gradient flow—a streaming implementation in DX9 graphics hardware. Computing 73(4): 373–389 MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Griesser, A., Roeck, S.D., Neubeck, A., Gool, L.J.V.: GPU-based foreground-background segmentation using an extended colinearity criterion. In: Vision, Modeling, and Visualization (VMV) (2005)Google Scholar
  18. 18.
    Pollefeys M., Gool L.J.V., Vergauwen M., Verbiest F., Cornelis K., Tops J. and Koch R. (2004). Visual Modeling with a Hand-Held Camera. IJCV 59(3): 207–232 CrossRefGoogle Scholar
  19. 19.
    Akbarzadeh, A., Frahm, J.-M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nistr, D., Pollefeys, M.: Towards urban 3D reconstruction from video, Invited paper. In: 3rd International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT) (2006)Google Scholar
  20. 20.
    GPGPU: General-Purpose Computation on GPUs. (2004)
  21. 21.
    Bjorke, K.A.: NVIDIA Corporation. Image processing using parallel GPU units. Proceedings of SPIE, vol. 6065 (2006)Google Scholar
  22. 22.
    Pharr M. and Fernando R. (2005). GPU Gems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation. Addison-Wesley Prof, Reading Google Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  • Sudipta N. Sinha
    • 1
  • Jan-Michael Frahm
    • 1
  • Marc Pollefeys
    • 1
  • Yakup Genc
    • 2
  1. 1.Department of Computer Science, CB# 3175 Sitterson HallUniversity of North Carolina at Chapel HillChapel HillUSA
  2. 2.Real-time Vision and Modeling DepartmentSiemens Corporate ResearchPrincetonUSA

Personalised recommendations