Skip to main content
SpringerLink
Log in

Real-time 3D semi-local surface patch extraction using GPGPU

Application to 3D object recognition

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Feature vectors can be anything from simple surface normals to more complex feature descriptors. Feature extraction is important to solve various computer vision problems: e.g. registration, object recognition and scene understanding. Most of these techniques cannot be computed online due to their complexity and the context where they are applied. Therefore, computing these features in real-time for many points in the scene is impossible. In this work, a hardware-based implementation of 3D feature extraction and 3D object recognition is proposed to accelerate these methods and therefore the entire pipeline of RGBD based computer vision systems where such features are typically used. The use of a GPU as a general purpose processor can achieve considerable speed-ups compared with a CPU implementation. In this work, advantageous results are obtained using the GPU to accelerate the computation of a 3D descriptor based on the calculation of 3D semi-local surface patches of partial views. This allows descriptor computation at several points of a scene in real-time. Benefits of the accelerated descriptor have been demonstrated in object recognition tasks. Source code will be made publicly available as contribution to the Open Source Point Cloud Library.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. Kinect for XBox 360: http://www.xbox.com/kinect Microsoft.

  2. This variable contains the thread index and is used to compute memory addresses and to make control decisions.

  3. This variable contains the block index where the thread is being executed.

  4. This variable contains the dimensions of the block.

  5. The Point Cloud Library (or PCL) is a large scale, open project [27] for 2D/3D image and point cloud processing.

References

  1. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Full DoF tracking of a hand interacting with an object by modeling occlusions and physical constraints. In: Proceedings of the 2011 International Conference on Computer Vision. ICCV11, IEEE Computer Society, Washington DC, pp 2088–2095 (2011)

  2. Izadi, S., Newcombe, R.A., Kim, D., Hilliges, O., Molyneaux, D., Hodges, S., Kohli, P., Shotton, J., Davison, A.J., Fitzgibbon, A.W.: Kinectfusion: real-time dynamic 3D surface reconstruction and interaction. In: SIGGRAPH Talks (2011)

  3. Kerl, C., Sturm, J., Cremers, D.: Dense visual slam for RGB-D cameras. In: Proceedings of the International Conference on Intelligent Robot Systems (IROS) (2013)

  4. Griffin, W., Wang, Y., Berrios, D., Olano, M.: Real-time GPU surface curvature estimation on deforming meshes and volumetric data sets. IEEE Trans. Vis. Comput. Graphics 18(10), 1603–1613 (2012)

    Google Scholar 

  5. Prisacariu, V., Reid, I.: fastHOG—a real-time GPU implementation of HOG. Technical Report 2310/09, Department of Engineering Science, Oxford University (2009)

  6. Himmelsbach, M., Luettel, T., Wuensche, H.J.: Real-time object classification in 3D point clouds using point feature histograms. In: Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS’09, IEEE Press, Piscataway, pp 994–1000 (2009)

  7. Olesen, S., Lyder, S., Kraft, D., Krger, N., Jessen, J.: Real-time extraction of surface patches with associated uncertainties by means of kinect cameras. J. Real Time Image Process, 1–14 (2012)

  8. Blum, M., Springenberg, J.T., Wülfing, J., Riedmiller, M.: A learned feature descriptor for object recognition in rgb-d data. In: ICRA, pp 1298–1303 (2012)

  9. Lee, W., Park, N., Woo, W.: Depth-assisted real-time 3D object detection for augmented reality. ICAT’11, vol. 2, pp. 126–132 (2011)

  10. Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999)

    Google Scholar 

  11. Stein, F., Medioni, G.: Structural indexing: efficient 3D object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 125–145 (1992)

    Google Scholar 

  12. Chua, C.S., Jarvis, R.: Point signatures: a new representation for 3D object recognition. Int. J. Comput. Vis. 25(1), 63–85 (1997)

    Google Scholar 

  13. Ruiz-Correa, S., Shapiro, L., Melia, M.: A new signature-based method for efficient 3D object recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1. CVPR 2001, pp. I-769–I-776 (2001)

  14. Yamany, S.M., Farag, A.A.: Surface signatures: an orientation independent free-form surface representation scheme for the purpose of objects registration and matching. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1105–1120 (2002)

    Google Scholar 

  15. Ashbrook, A., Fisher, R.: Finding surface correspondence for object recognition and registration. In: Proceedings of the 5th European Conference on Computer Vision, vol. II, Freiburg, pp. 674–680 (1998)

  16. Chen, H., Bhanu, B.: 3D free-form object recognition in range images using local surface patches. Pattern Recognit. Lett. 28(10), 1252–1262 (2007)

    Google Scholar 

  17. Taati, B., Greenspan, M.: Local shape descriptor selection for object recognition in range data. Comput. Vis. Image Underst. 115(5), 681–694 (2011)

    Google Scholar 

  18. Shang, L., Greenspan, M.A.: Real-time object recognition in sparse range images using error surface embedding. Int. J.Comput. Vis. 89(2–3), 211–228 (2010)

    Google Scholar 

  19. Campbell, R.J., Flynn, P.J.: A survey of free-form object representation and recognition techniques. Comput. Vis. Image Underst. 81, 166–210 (2001)

    Google Scholar 

  20. Mian, A.S., Bennamoun, M., Owens, R.A.: Automatic correspondence for 3D modeling: an extensive review. Int. J. Shape Model. 11(2), 253–291 (2005)

    Google Scholar 

  21. Mian, A.S., Bennamoun, M., Owens, R.: Three-dimensional model-based object recognition and segmentation in cluttered scenes. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1584–1601 (2006)

    Google Scholar 

  22. Mian, A.S., Bennamoun, M., Owens, R.A.: A novel representation and feature matching algorithm for automatic pairwise registration of range images. Int. J. Comput. Vis. 66(1), 19–40 (2006)

    Google Scholar 

  23. Hetzel, G., Leibe, B., Levi, P., Schiele, B.: 3D object recognition from range images using local feature histograms. In: CVPR (2). IEEE Computer Society, pp. 394–399 (2001)

  24. Tombari, F., Salti, S., di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV (3). Volume 6313 of Lecture Notes in Computer Science. Springer, Berlin, pp. 356–369 (2010)

  25. Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)

    Google Scholar 

  26. NVIDIA. CUDA Programming Guide 5.0. (2012)

  27. Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai (2011)

  28. Zhang, Z.: Microsoft kinect sensor and its effect. IEEE MultiMedia 19(2), 4–10 (2012)

    Google Scholar 

  29. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision. ICCV ’98, IEEE Computer Society, Washington, DC, pp. 839–846 (1998)

  30. Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications—M2SFA2 2008, Marseille (2008)

  31. Wasza, J., Bauer, S., Hornegger, J.: Real-time preprocessing for dense 3D range imaging on the GPU: defect interpolation, bilateral temporal averaging and guided filtering. In: ICCV Workshops, pp. 1221–1227 (2011)

  32. Holz, D., Behnke, S.: Fast range image segmentation and smoothing using approximate surface reconstruction and region growing. In: Proceedings of the 12th International Conference on Intelligent Autonomous Systems (IAS), Jeju Island (2012)

  33. Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer graphics: principles and practice, 2nd edn. Addison-Wesley Longman Publishing Co., Inc., Boston (1990)

  34. Wang, L., Huang, M., El-Ghazawi, T.A.: Exploiting concurrent kernel execution on graphic processing units. In: Smari, W.W., McIntire, J.P. (eds.) HPCS, IEEE, pp. 24–32 (2011)

  35. NVIDIA. Visual profiler (2012)

  36. Harris, M.: Optimizing parallel reduction in CUDA. NVIDIA Dev. Technology (2008)

Download references

Acknowledgments

This work was partially funded by the Valencian Government BEFPI/2012/056, and by the European Network of Excellence on High Performance and Embedded Architecture and Compilation (HiPEAC). Experiments were made possible with a generous donation of hardware from NVDIA.

Author information

Authors and Affiliations

  1. Computer Technology Department, University of Alicante, Alicante, Spain

    Sergio Orts-Escolano, Vicente Morell, Jose Garcia-Rodriguez & Miguel Cazorla

  2. Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, UK

    Robert B. Fisher

Authors
  1. Sergio Orts-Escolano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vicente Morell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose Garcia-Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miguel Cazorla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert B. Fisher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergio Orts-Escolano.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Orts-Escolano, S., Morell, V., Garcia-Rodriguez, J. et al. Real-time 3D semi-local surface patch extraction using GPGPU. J Real-Time Image Proc 10, 647–666 (2015). https://doi.org/10.1007/s11554-013-0385-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0385-7

Keywords

Search

Navigation