Abstract
This paper describes novel implementations of the KLT feature tracking and SIFT feature extraction algorithms that run on the graphics processing unit (GPU) and is suitable for video analysis in real-time vision systems. While significant acceleration over standard CPU implementations is obtained by exploiting parallelism provided by modern programmable graphics hardware, the CPU is freed up to run other computations in parallel. Our GPU-based KLT implementation tracks about a thousand features in real-time at 30 Hz on 1,024 × 768 resolution video which is a 20 times improvement over the CPU. The GPU-based SIFT implementation extracts about 800 features from 640 × 480 video at 10 Hz which is approximately 10 times faster than an optimized CPU implementation.
Similar content being viewed by others
References
Bramberger, M., Rinner, B., Schwabach, H.: An embedded smart camera on a scalable heterogeneous multi-DSP system. In: Proceedings of the European DSP Education and Research Symposium (EDERS 2004) (2004)
Klupsch, S., Ernst, M., Huss, S.A., Rumpf, M., Strzodka, R.: Real time image processing based on reconfigurable hardware acceleration. In: Proceedings of IEEE Workshop Heterogeneous Reconfigurable Systems on Chip (2002)
Fung, J., Mann, S.: OpenVIDIA: parallel GPU computer vision. ACM MULTIMEDIA 2005, pp. 849–852 (2005)
Fung, J., Mann, S.: Computer vision signal processing on graphics processing units. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004), Montreal pp. V-93–V-96 (2004)
Gong, M., Langille, A., Gong, M.: Real-time image processing using graphics hardware: a performance study. In: International Conference on Image Analysis and Recognition, pp. 1217–1225 (2005)
Tomasi, C., Kanade, T.: Detection and Tracking of Point Features. Tech. Rept. CMU-CS-91132, Carnegie Mellon University (1991)
Lukas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)
Birchfield, S.: Derivation of Kanade-Lucas-Tomasi tracking equation. unpublished notes (1997)
Birchfield, S.: KLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker. http://www.ces.clemson.edu/~stb/klt (2005)
Lowe D.G. (2004). Distinctive image features from scale-invariant keypoints. IJCV 60(2): 91–110
Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Conference on Computer Vision and Pattern Recognition (CVPR) pp. 211–217 (2003)
Zach, C., Bischof, H., Karner, K.: Hierarchical disparity estimation with programmable 3D hardware. In: WSCG (International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision). Short Communications, pp. 275–282, Plzen, Slowakei (2004)
Woetzel, J., Koch, R.: Real-time multi-stereo depth estimation on GPU with approximative discontinuity handling. In: European Conf. on Visual Media Production (2004)
Labatut P., Keriven R. and Pons J.-P. (2006). A GPU implementation of level set multiview stereo. Int. Conf. Comput. Sci. 4: 212–219
Yang R. and Welch G. (2002). Fast image segmentation and smoothing using commodity graphics hardware. J. Graph. Tools 7(4): 91–100
Strzodka R., Droske M. and Rumpf M. (2004). Image registration by a regularized gradient flow—a streaming implementation in DX9 graphics hardware. Computing 73(4): 373–389
Griesser, A., Roeck, S.D., Neubeck, A., Gool, L.J.V.: GPU-based foreground-background segmentation using an extended colinearity criterion. In: Vision, Modeling, and Visualization (VMV) (2005)
Pollefeys M., Gool L.J.V., Vergauwen M., Verbiest F., Cornelis K., Tops J. and Koch R. (2004). Visual Modeling with a Hand-Held Camera. IJCV 59(3): 207–232
Akbarzadeh, A., Frahm, J.-M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nistr, D., Pollefeys, M.: Towards urban 3D reconstruction from video, Invited paper. In: 3rd International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT) (2006)
GPGPU: General-Purpose Computation on GPUs. http://www.gpgpu.org (2004)
Bjorke, K.A.: NVIDIA Corporation. Image processing using parallel GPU units. Proceedings of SPIE, vol. 6065 (2006)
Pharr M. and Fernando R. (2005). GPU Gems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation. Addison-Wesley Prof, Reading
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sinha, S.N., Frahm, JM., Pollefeys, M. et al. Feature tracking and matching in video using programmable graphics hardware. Machine Vision and Applications 22, 207–217 (2011). https://doi.org/10.1007/s00138-007-0105-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-007-0105-z