Advertisement

Model-Based 3D Object Recognition in RGB-D Images

  • Maciej Stefańczyk
  • Włodzimierz Kasprzak
Chapter
Part of the Intelligent Systems Reference Library book series (ISRL, volume 145)

Abstract

A computational framework for 3D object recognition in RGB-D images is presented. The focus is on computer vision applications in indoor autonomous robotics, where objects need to be recognized either for the purpose of being grasped and manipulated by the robot, or where the entire scene must be recognized to allow high-level cognitive tasks to be performed. The framework integrates solutions for generic (i.e. type-based) object representation (e.g. semantic networks), trainable transformations between abstraction levels (e.g. by neural networks), reasoning under uncertain and partial data (e.g. Dynamic Bayesian Networks, Fuzzy Logic), optimized model-to-data matching (e.g. constraint optimization problems) and efficient search strategies (switching between data- and model-driven inference steps). The computational implementation of the object model and the object recognition strategy is presented in more details. Testing scenarios deal with the recognition of cups and bottles or household furniture. Conducted experiments and the chosen applications confirmed, that this approach is valid and may easily be adapted to multiple scenarios.

Notes

Acknowledgements

The manuscript preparation was supported by statutory funds of the Warsaw University of Technology.

References

  1. 1.
    Ren, X., Fox, D., Konolige, K.: Change their perception: RGB-D for 3-D modeling and recognition. IEEE Robot. Autom. Mag. 20(4), 49–59 (2013)CrossRefGoogle Scholar
  2. 2.
    Richtsfeld, A., Mörwald, T., Prankl, J., Zillich, M., Vincze, M.: Learning of perceptual grouping for object segmentation on RGB-D data. J. Vis. Commun. Image Represent. 25(1), 64–73 (2014)CrossRefGoogle Scholar
  3. 3.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Rob. Res. 31(5), 647–663 (2012).  https://doi.org/10.1177/0278364911434148
  4. 4.
    Newcombe, R.A., Davison, A.J., Izadi, S., Kohli, P., Hilliges, O., Shotton, J., Molyneaux, D., Hodges, S., Kim, D., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 127–136. IEEE (2011)Google Scholar
  5. 5.
    Wilkowski, A., Kornuta, T., Stefańczyk, M., Kasprzak, W.: Efficient generation of 3D surfel maps using RGB-D sensors. Int. J. Appl. Math. Comput. Sci. (AMCS) 26(1), 99–122 (2016).  https://doi.org/10.1515/amcs-2016-0007
  6. 6.
    Hanson, A., Riseman, E.: The VISIONS image-understanding system. Adv. Comput. Vis. 1, 1–114 (1988)Google Scholar
  7. 7.
    Hwang, V.S.S., Davis, L.S., Matsuyama, T.: Hypothesis integration in image understanding systems. Comput. Vis. Graph. Image Process. 36(2–3), 321–371 (1986)CrossRefGoogle Scholar
  8. 8.
    Niemann, H., Sagerer, G.F., Schroder, S., Kummert, F.: ERNEST: a semantic network system for pattern understanding. IEEE Trans. Pattern Anal. Mach. Intell. 12(9), 883–905 (1990)CrossRefGoogle Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)Google Scholar
  10. 10.
    Socher, R., Huval, B., Bhat, B., Manning, C., Ng, A.: Convolutional-recursive deep learning for 3D object classification. Adv. Neural Inf. Process. Syst. 25, 656–664 (2012)Google Scholar
  11. 11.
    Behnke, S.: Hierarchical Neural Networks for Image Interpretation. Lecture Notes in Computer Science, vol. 2766. Springer, Berlin (2003)Google Scholar
  12. 12.
    Lin, D., Fidler, S., Urtasun, R.: Holistic scene understanding for 3d object detection with rgbd cameras. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, Australia, December 1–8, 2013, pp. 1417–1424. IEEE Computer Society, ISBN 978-1-4799-2839-2 (2013)Google Scholar
  13. 13.
    Kasprzak, W.: A linguistic approach to 3-D object recognition. Comput. Graph. 11(4), 427–443 (1987)CrossRefGoogle Scholar
  14. 14.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  15. 15.
    Russel, S., Norvig, P.: Artificial Intelligence. A Modern Approach, 3rd edn. Prentice Hall, Upper Saddle River (2011)Google Scholar
  16. 16.
    Tsardoulias, E., Zieliński, C., Kasprzak, W., Reppou, S.: Merging robotics and AAL ontologies: the RAPP methodology. In: Progress in Automation, Robotics and Measuring Techniques. Advances in Intelligent Systems and Computing, vol. 351, pp. 285–297. Springer International Publishing (2015)Google Scholar
  17. 17.
    Zieliński, C., et al.: Variable structure robot control systems–the RAPP approach. Robot. Auton. Syst. 94, 226–244 (2017).  https://doi.org/10.1016/j.robot.2017.05.002
  18. 18.
    Collet, A., Martinez, M., Srinivasa, S.S.: The MOPED framework: object recognition and pose estimation for manipulation. Int. J. Robot. Res. 30(10), 1284–1306 (2011)CrossRefGoogle Scholar
  19. 19.
    Hinterstoisser, S., Holzer, S., Cagniart, C., Ilic, S., Konolige, K., Navab, N., Lepetit, V.: Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 858–865. IEEE (2011)Google Scholar
  20. 20.
    O’Hara, S., Draper, B.A.: Introduction to the bag of features paradigm for image classification and retrieval (2011). arXiv preprint arXiv:1101.3354
  21. 21.
    Lysenkov, I., Rabaud, V.: Pose estimation of rigid transparent objects in transparent clutter. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 162–169. IEEE (2013)Google Scholar
  22. 22.
    Pangercic, D., Haltakov, V., Beetz, M.: Fast and robust object detection in household environments using vocabulary trees with sift descriptors. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Workshop on Active Semantic Perception and Object Search in the Real World, San Francisco, CA, USA. Citeseer (2011)Google Scholar
  23. 23.
    Kasprzak, W., Kornuta, T., Zieliński, C.: A virtual receptor in a robot control framework. In: Szewczyk, R., Zieliński, C., Kaliczyńska, M. (eds.) Recent Advances in Automation, Robotics and Measuring Techniques. Advances in Intelligent Systems and Computing (AISC), vol. 267, pp. 399–408. Springer, Berlin (2014)CrossRefGoogle Scholar
  24. 24.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  25. 25.
    Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Henry Holt and Co., Inc., New York (1982)Google Scholar
  26. 26.
    Lowe, D.G.: Three-dimensional object recognition from single two-dimensional images. Artif. Intell. 31(3), 355–395 (1987)CrossRefGoogle Scholar
  27. 27.
    Forsyth, D.A., Ponce, J.: Computer Vision: A Modern Approach. Prentice Hall Professional Technical Reference (2002)Google Scholar
  28. 28.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, Upper Saddle River (2002)Google Scholar
  29. 29.
    Stefańczyk, M., Kasprzak, W.: Multimodal segmentation of dense depth maps and associated color information. In: Bolc, L., Tadeusiewicz, R., Chmielewski, L., Wojciechowski, K. (eds.) Proceedings of the International Conference on Computer Vision and Graphics. Lecture Notes in Computer Science, vol. 7594, pp. 626–632. Springer, Berlin (2012)Google Scholar
  30. 30.
    Richtsfeld, A., Morwald, T., Prankl, J., Zillich, M., Vincze, M.: Segmentation of unknown objects in indoor environments. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4791–4796. IEEE (2012)Google Scholar
  31. 31.
    Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robot. Autom. Mag. 11(4), 110–122 (2004)CrossRefGoogle Scholar
  32. 32.
    Łępicka, M., Kornuta, T., Stefańczyk, M.: Utilization of colour in ICP-based point cloud registration. In: Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015. Advances in Intelligent Systems and Computing, pp. 821–830. Springer, Berlin (2016)Google Scholar
  33. 33.
    Naylor, B.: Computational representations of geometry. In: Representations of Geometry for Computer Graphics, SIGGRAPH ’94 Course Notes (1994)Google Scholar
  34. 34.
    Jaklic, A., Leonardis, A., Solina, F.: Segmentation and Recovery of Superquadrics, vol. 20. Springer Science & Business Media (2013)Google Scholar
  35. 35.
    Foote, T.: tf: the transform library. In: 2013 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6. IEEE (2013)Google Scholar
  36. 36.
    Kasprzak, W.: Integration of different computational models in a computer vision framework. In: 2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM), pp. 13–18 (2010).  https://doi.org/10.1109/CISIM.2010.5643697
  37. 37.
    Stefańczyk, M., Pietruch, R.: Hypothesis generation in generic, model-based object recognition system. In: Szewczyk, R., Zieliński, C., Kaliczyńska, M. (eds.) Recent Advances in Automation, Robotics and Measuring Techniques. Advances in Intelligent Systems and Computing (AISC), vol. 440, pp. 717–727. Springer, Berlin (2016).  https://doi.org/10.1007/978-3-319-29357-8_62
  38. 38.
    Wilkowski, A., Stefańczyk, M.: Detection and recognition of compound 3D models by hypothesis generation. In: Szewczyk, R., Zieliński, C., Kaliczyńska, M. (eds.) Recent Advances in Automation, Robotics and Measuring Techniques. Advances in Intelligent Systems and Computing (AISC), vol. 440, pp. 659–668. Springer, Berlin (2016).  https://doi.org/10.1007/978-3-319-29357-8_57
  39. 39.
    Stefańczyk, M., Laszkowski, M., Kornuta, T.: WUT visual perception dataset-a dataset for registration and recognition of objects. In: Challenges in Automation, Robotics and Measurement Techniques. Advances in Intelligent Systems and Computing (AISC), vol. 440, pp. 635–645. Springer, Berlin (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Institute of Control and Computation EngineeringWarsaw University of TechnologyWarsawPoland

Personalised recommendations