Point-Based Object Recognition in RGB-D Images

  • Artur Wilkowski
  • Tomasz Kornuta
  • Włodzimierz Kasprzak
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 323)


To operate autonomously a robot system needs among others to perceive the environment and to recognize the scene objects. In particular, nowadays an RGB-D sensor can be applied for vision-based perception. In this paper, two data-driven RGB-D image analysis steps, required for a reliable 3D object recognition process, are studied and appropriate algorithmic solutions are proposed. Clusters of 3D point features are detected in order to represent 3D object hypotheses. Particular clusters act as initial rough object hypotheses, allowing to constrain the subsequent model-based search for more distinctive object features in the image, like surface patches, textures and edges. In parallel, a 3D surface-based occupancy map is created, that delivers surface segments for the object recognition process. Test results are reported on various approaches to point feature detection and description, and point cloud processing.


Point Cloud Visual Odometry Robot Operating System Object Hypothesis Object Recognition Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ROS: Robot operating system (November 2013),
  2. 2.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: Rgb-d mapping: Using kinect-style depth cameras for dense 3d modeling of indoor environments. Int. J. Rob. Res. 31(5), 647–663 (2012)CrossRefGoogle Scholar
  3. 3.
    Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the rgb-d slam system. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1691–1696 (May 2012)Google Scholar
  4. 4.
    Mörwald, T., Prankl, J., Richtsfeld, A., Zillich, M., Vincze, M.: Blort-the blocks world robotic vision toolbox. In: Best Practice in 3D Perception and Modeling for Mobile Manipulation, ICRA Workshop (2010)Google Scholar
  5. 5.
    Collet Romea, A., Martinez Torres, M., Srinivasa, S.: The moped framework: Object recognition and pose estimation for manipulation. International Journal of Robotics Research 30(10), 1284–1306 (2011)CrossRefGoogle Scholar
  6. 6.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  7. 7.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  8. 8.
    Schnabel, R., Wahl, R., Klein, R.: Efficient ransac for point-cloud shape detection. Computer Graphics Forum 26(2), 214–226 (2007)CrossRefGoogle Scholar
  9. 9.
    Segal, A., Haehnel, D., Thrun, S.: Generalized-icp. In: Proc. of Robotics: Science and Systems (RSS) (2009)Google Scholar
  10. 10.
    Kasprzak, W.: Integration of different computational models in a computer vision framework. In: 2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM), pp. 13–18 (October 2010)Google Scholar
  11. 11.
    Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: Kinectfusion: Real-time 3d reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST 2011, pp. 559–568. ACM, New York (2011)Google Scholar
  12. 12.
    Dryanovski, I., Valenti, R., Xiao, J.: Fast visual odometry and mapping from rgb-d data. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 2305–2310 (May 2013)Google Scholar
  13. 13.
    Marder-Eppstein, E., Berger, E., Foote, T., Gerkey, B., Konolige, K.: The office marathon: Robust navigation in an indoor office environment. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 300–307 (May 2010)Google Scholar
  14. 14.
    Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in hand 3d object modeling. Technical Report UW-CSE-10-09-01, University of Washington (2010)Google Scholar
  15. 15.
    Triebel, R., Pfaff, P., Burgard, W.: Multi-level surface maps for outdoor terrain mapping and loop closing. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2276–2282 (October 2006)Google Scholar
  16. 16.
    Wurm, K.M., Hornung, A., Bennewitz, M., Stachniss, C., Burgard, W.: Octomap: A probabilistic, flexible, and compact 3d map representation for robotic systems. In: Proc. of the ICRA 2010 Worksshop (2010)Google Scholar
  17. 17.
    Whelan, T., Johannsson, H., Kaess, M., Leonard, J., McDonald, J.: Robust real-time visual odometry for dense rgb-d mapping. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 5724–5731 (May 2013)Google Scholar
  18. 18.
    Rusu, R., Blodow, N., Marton, Z., Beetz, M.: Aligning point cloud views using persistent feature histograms. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2008, pp. 3384–3391 (September 2008)Google Scholar
  19. 19.
    Leutenegger, S., Chli, M., Siegwart, R.: Brisk: Binary robust invariant scalable keypoints. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2548–2555 (November 2011)Google Scholar
  20. 20.
    Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: International Conference on Computer Vision Theory and Application, VISSAPP 2009, pp. 331–340. INSTICC Press (2009)Google Scholar
  21. 21.
    Kasprzak, W., Kornuta, T., Zieliński, C.: A virtual receptor in a robot control framework. In: Szewczyk, R., Zieliński, C., Kaliczyńska, M. (eds.) Recent Advances in Automation, Robotics and Measuring Techniques. AISC, vol. 267, pp. 399–408. Springer, Heidelberg (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Artur Wilkowski
    • 1
  • Tomasz Kornuta
    • 2
  • Włodzimierz Kasprzak
    • 1
  1. 1.Industrial Research Institute for Automation and MeasurementsWarsawPoland
  2. 2.Institute of Control and Computation EngineeringWarsaw University of TechnologyWarsawPoland

Personalised recommendations