Perception Subsystem for Object Recognition and Pose Estimation in RGB-D Images

  • Tomasz Kornuta
  • Michał Laszkowski
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 440)


RGB-D sensors have become key components of all kind of robotic systems. In this paper we present a perception subsystem for object recognition and pose estimation in RGB-D images. The system is able to recognize many objects at once, disregarding whether they belong to one or many classes. Next to the detailed description of the principle of the system operation we present several off-line and on-line experiments validating the system, including verification in the task of picking up recognized objects with IRp-6 manipulator.


RGB-D image SIFT Object recognition Pose estimation Object picking 



This project was funded by the National Science Centre according to the decision number DEC-2012/05/D/ST6/03097. Tomasz Kornuta is supported by the IBM Research, Almaden through the IBM PostDoc/LTS Programme.


  1. 1.
    Dziergwa, M., Kaczmarek, P., Kędzierski, J.: RGB-D sensors in social robotics. J. Autom. Mobile Robot. Intell. Syst. 9(1), 18–27 (2015)CrossRefGoogle Scholar
  2. 2.
    Ramey, A., González-Pacheco, V., Salichs, M.A.: Integration of a low-cost RGB-D sensor in a social robot for gesture recognition. In: Proceedings of the 6th International Conference on Human-robot Interaction, pp. 229–230. ACM (2011)Google Scholar
  3. 3.
    Stefańczyk, M., Banachowicz, K., Walęcki, M., Winiarski, T.: 3D camera and lidar utilization for mobile robot navigation. J. Autom. Mobile Robot. Intell. Syst. 7(4), 27–33 (2013)Google Scholar
  4. 4.
    Belter, D., Nowicki, M., Skrzypczyński, P., Walas, K., Wietrzykowski, J.: Lightweight RGB-D slam system for search and rescue robots. In: Progress in Automation, Robotics and Measuring Techniques, pp. 11–21. Springer (2015)Google Scholar
  5. 5.
    Walas, K.: Terrain classification and negotiation with a walking robot. J. Intell. Robot. Syst. 78(3–4), 401–423 (2015)CrossRefGoogle Scholar
  6. 6.
    Großmann, B., Siam, M., Krüger, V.: Comparative evaluation of 3D pose estimation of industrial objects in RGB pointclouds. In: Computer Vision Systems, pp. 329–342. Springer (2015)Google Scholar
  7. 7.
    Ren, X., Fox, D., Konolige, K.: Change their perception: RGB-D for 3-D modeling and recognition. IEEE Robot. Autom. Mag. 20(4), 49–59 (2013)CrossRefGoogle Scholar
  8. 8.
    Maitin-Shepard, J., Cusumano-Towner, M., Lei, J., Abbeel, P.: Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 2308–2315 (2010)Google Scholar
  9. 9.
    Doumanoglou, A., Kargakos, A., Kim, T.K., Malassiotis, S.: Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 987–993. IEEE (2014)Google Scholar
  10. 10.
    Lenz, I., Knepper, R., Saxena, A.: DeepMPC: learning deep latent features for model predictive control. In: Proceedings of Robotics: Science and Systems, Rome, Italy (July 2015)Google Scholar
  11. 11.
    Rusu, R., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: IEEE International Conference on Robotics and Automation. ICRA’09, pp. 3212–3217. IEEE (2009)Google Scholar
  12. 12.
    Harasymowicz-Boggio, B., Chechliński, Ł., Siemiątkowska, B.: Nature-inspired, parallel object recognition. In: Progress in Automation, Robotics and Measuring Techniques, pp. 53–62. Springer (2015)Google Scholar
  13. 13.
    Bo, L., Ren, X., Fox, D.: Unsupervised feature learning for RGB-D based object recognition. In: Experimental Robotics, pp. 387–402. Springer (2013)Google Scholar
  14. 14.
    Stefańczyk, M., Laszkowski, M., Kornuta, T.: WUT visual perception dataset—a dataset for registration and recognition of objects. In: Szewczyk, R., Zieliński, C., Kaliczyńska, M. (eds.) Recent Advances in Automation, Robotics and Measuring Techniques. Advances in Intelligent Systems and Computing (AISC). Springer (2016) (Under review)Google Scholar
  15. 15.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  16. 16.
    Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISAPP (1), pp. 331–340 (2009)Google Scholar
  17. 17.
    Chen, H., Bhanu, B.: 3D free-form object recognition in range images using local surface patches. Pattern Recognit. Lett. 28(10), 1252–1262 (2007)CrossRefGoogle Scholar
  18. 18.
    Aldoma, A., Tombari, F., Rusu, R.B., Vincze, M.: OUR-CVFH-Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6DOF Pose Estimation. Springer (2012)Google Scholar
  19. 19.
    Seredynski, D., Winiarski, T., Banachowicz, K., Zielinski, C.: Grasp planning taking into account the external wrenches acting on the grasped object. In: 2015 10th International Workshop on Robot Motion and Control (RoMoCo), pp. 40–45. IEEE (2015)Google Scholar
  20. 20.
    Walcki, M., Banachowicz, K., Winiarski, T.: Research oriented motor controllers for robotic applications. In: Kozłowski, K. (ed.) Robot Motion and Control 2011 (LNCiS) Lecture Notes in Control and Information Sciences, vol. 422, pp. 193–203. Springer Verlag London Limited (2012)Google Scholar
  21. 21.
    Winiarski, T., Banachowicz, K.: Automated generation of component system for the calibration of the service robot kinematic parameters. In: 20th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR’2015, pp. 1098–1103. IEEE (2015)Google Scholar
  22. 22.
    Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: an open-source robot operating system. In: Proceedings of the Open-Source Software Workshop at the International Conference on Robotics and Automation (ICRA) (2009)Google Scholar
  23. 23.
    Bruyninckx, H.: The real-time motion control core of the OROCOS project. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2766–2771. IEEE (Sept 2003)Google Scholar
  24. 24.
    Stefańczyk, M., Kornuta, T.: Handling of asynchronous data flow in robot perception subsystems. In: Simulation, Modeling, and Programming for Autonomous Robots. Lecture Notes in Computer Science, vol. 8810, pp. 509–520. Springer (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.IBM Research, AlmadenSan JoseUSA
  2. 2.Institute of Control and Computation EngineeringWarsaw University of TechnologyWarsawPoland

Personalised recommendations