Abstract
Feature vectors can be anything from simple surface normals to more complex feature descriptors. Feature extraction is important to solve various computer vision problems: e.g. registration, object recognition and scene understanding. Most of these techniques cannot be computed online due to their complexity and the context where they are applied. Therefore, computing these features in real-time for many points in the scene is impossible. In this work, a hardware-based implementation of 3D feature extraction and 3D object recognition is proposed to accelerate these methods and therefore the entire pipeline of RGBD based computer vision systems where such features are typically used. The use of a GPU as a general purpose processor can achieve considerable speed-ups compared with a CPU implementation. In this work, advantageous results are obtained using the GPU to accelerate the computation of a 3D descriptor based on the calculation of 3D semi-local surface patches of partial views. This allows descriptor computation at several points of a scene in real-time. Benefits of the accelerated descriptor have been demonstrated in object recognition tasks. Source code will be made publicly available as contribution to the Open Source Point Cloud Library.
Similar content being viewed by others
Notes
Kinect for XBox 360: http://www.xbox.com/kinect Microsoft.
This variable contains the thread index and is used to compute memory addresses and to make control decisions.
This variable contains the block index where the thread is being executed.
This variable contains the dimensions of the block.
The Point Cloud Library (or PCL) is a large scale, open project [27] for 2D/3D image and point cloud processing.
References
Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Full DoF tracking of a hand interacting with an object by modeling occlusions and physical constraints. In: Proceedings of the 2011 International Conference on Computer Vision. ICCV11, IEEE Computer Society, Washington DC, pp 2088–2095 (2011)
Izadi, S., Newcombe, R.A., Kim, D., Hilliges, O., Molyneaux, D., Hodges, S., Kohli, P., Shotton, J., Davison, A.J., Fitzgibbon, A.W.: Kinectfusion: real-time dynamic 3D surface reconstruction and interaction. In: SIGGRAPH Talks (2011)
Kerl, C., Sturm, J., Cremers, D.: Dense visual slam for RGB-D cameras. In: Proceedings of the International Conference on Intelligent Robot Systems (IROS) (2013)
Griffin, W., Wang, Y., Berrios, D., Olano, M.: Real-time GPU surface curvature estimation on deforming meshes and volumetric data sets. IEEE Trans. Vis. Comput. Graphics 18(10), 1603–1613 (2012)
Prisacariu, V., Reid, I.: fastHOG—a real-time GPU implementation of HOG. Technical Report 2310/09, Department of Engineering Science, Oxford University (2009)
Himmelsbach, M., Luettel, T., Wuensche, H.J.: Real-time object classification in 3D point clouds using point feature histograms. In: Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS’09, IEEE Press, Piscataway, pp 994–1000 (2009)
Olesen, S., Lyder, S., Kraft, D., Krger, N., Jessen, J.: Real-time extraction of surface patches with associated uncertainties by means of kinect cameras. J. Real Time Image Process, 1–14 (2012)
Blum, M., Springenberg, J.T., Wülfing, J., Riedmiller, M.: A learned feature descriptor for object recognition in rgb-d data. In: ICRA, pp 1298–1303 (2012)
Lee, W., Park, N., Woo, W.: Depth-assisted real-time 3D object detection for augmented reality. ICAT’11, vol. 2, pp. 126–132 (2011)
Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999)
Stein, F., Medioni, G.: Structural indexing: efficient 3D object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 125–145 (1992)
Chua, C.S., Jarvis, R.: Point signatures: a new representation for 3D object recognition. Int. J. Comput. Vis. 25(1), 63–85 (1997)
Ruiz-Correa, S., Shapiro, L., Melia, M.: A new signature-based method for efficient 3D object recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1. CVPR 2001, pp. I-769–I-776 (2001)
Yamany, S.M., Farag, A.A.: Surface signatures: an orientation independent free-form surface representation scheme for the purpose of objects registration and matching. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1105–1120 (2002)
Ashbrook, A., Fisher, R.: Finding surface correspondence for object recognition and registration. In: Proceedings of the 5th European Conference on Computer Vision, vol. II, Freiburg, pp. 674–680 (1998)
Chen, H., Bhanu, B.: 3D free-form object recognition in range images using local surface patches. Pattern Recognit. Lett. 28(10), 1252–1262 (2007)
Taati, B., Greenspan, M.: Local shape descriptor selection for object recognition in range data. Comput. Vis. Image Underst. 115(5), 681–694 (2011)
Shang, L., Greenspan, M.A.: Real-time object recognition in sparse range images using error surface embedding. Int. J.Comput. Vis. 89(2–3), 211–228 (2010)
Campbell, R.J., Flynn, P.J.: A survey of free-form object representation and recognition techniques. Comput. Vis. Image Underst. 81, 166–210 (2001)
Mian, A.S., Bennamoun, M., Owens, R.A.: Automatic correspondence for 3D modeling: an extensive review. Int. J. Shape Model. 11(2), 253–291 (2005)
Mian, A.S., Bennamoun, M., Owens, R.: Three-dimensional model-based object recognition and segmentation in cluttered scenes. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1584–1601 (2006)
Mian, A.S., Bennamoun, M., Owens, R.A.: A novel representation and feature matching algorithm for automatic pairwise registration of range images. Int. J. Comput. Vis. 66(1), 19–40 (2006)
Hetzel, G., Leibe, B., Levi, P., Schiele, B.: 3D object recognition from range images using local feature histograms. In: CVPR (2). IEEE Computer Society, pp. 394–399 (2001)
Tombari, F., Salti, S., di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV (3). Volume 6313 of Lecture Notes in Computer Science. Springer, Berlin, pp. 356–369 (2010)
Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)
NVIDIA. CUDA Programming Guide 5.0. (2012)
Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai (2011)
Zhang, Z.: Microsoft kinect sensor and its effect. IEEE MultiMedia 19(2), 4–10 (2012)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision. ICCV ’98, IEEE Computer Society, Washington, DC, pp. 839–846 (1998)
Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications—M2SFA2 2008, Marseille (2008)
Wasza, J., Bauer, S., Hornegger, J.: Real-time preprocessing for dense 3D range imaging on the GPU: defect interpolation, bilateral temporal averaging and guided filtering. In: ICCV Workshops, pp. 1221–1227 (2011)
Holz, D., Behnke, S.: Fast range image segmentation and smoothing using approximate surface reconstruction and region growing. In: Proceedings of the 12th International Conference on Intelligent Autonomous Systems (IAS), Jeju Island (2012)
Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer graphics: principles and practice, 2nd edn. Addison-Wesley Longman Publishing Co., Inc., Boston (1990)
Wang, L., Huang, M., El-Ghazawi, T.A.: Exploiting concurrent kernel execution on graphic processing units. In: Smari, W.W., McIntire, J.P. (eds.) HPCS, IEEE, pp. 24–32 (2011)
NVIDIA. Visual profiler (2012)
Harris, M.: Optimizing parallel reduction in CUDA. NVIDIA Dev. Technology (2008)
Acknowledgments
This work was partially funded by the Valencian Government BEFPI/2012/056, and by the European Network of Excellence on High Performance and Embedded Architecture and Compilation (HiPEAC). Experiments were made possible with a generous donation of hardware from NVDIA.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Orts-Escolano, S., Morell, V., Garcia-Rodriguez, J. et al. Real-time 3D semi-local surface patch extraction using GPGPU. J Real-Time Image Proc 10, 647–666 (2015). https://doi.org/10.1007/s11554-013-0385-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-013-0385-7