A Layered Learning Approach to 3D Multimodal People Detection Using Low-Cost Sensors in a Mobile Robot

  • Loreto SusperregiEmail author
  • Basilio Sierra
  • Jose María Martínez-Otzeta
  • Elena Lazkano
  • Ander Ansuategui
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 153)


In this paper we propose a novel approach for low cost multimodal detection of humans with mobile service robots. Detecting people is a key capability for robots that operate in populated environments. The main objective of this article is to illustrate the implementation of machine learning paradigms with computer vision techniques to improve human detection using 3D vision and a thermal sensor. Experimental results carried out in a manufacturing shop-floor show that the percentage of wrong classified using only Kinect is drastically reduced with the classification algorithms and with the combination of the three information sources.


Computer Vision Machine Learning Robotics 3D People Detection Multimodal People Detection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
  3. 3.
    Bellotto, N., Hu, H.: Multisensor data fusion for joint people tracking and identification with a service robot. In: 2007 IEEE International Conference on Robotics and Biomimetics ROBIO, pp. 1494–1499 (2007)Google Scholar
  4. 4.
    Cielniak, G.: People tracking by mobile robots using thermal and colour vision. Ph.D. thesis, Department of Technology Orebro University (2007)Google Scholar
  5. 5.
    Gundimada, S., Asari, V.K., Gudur, N.: Face recognition in multi-sensor images based on a novel modular feature selection technique. Inf. Fusion 11, 124–132 (2010)CrossRefGoogle Scholar
  6. 6.
    Hofmann, M., Kaiser, M., Aliakbarpour, H., Rigoll, G.: Fusion of multi-modal sensors in a voxel occupancy grid for tracking and behaviour analysis. In: Proc. 12th Intern. Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Delft, The Netherlands (2011)Google Scholar
  7. 7.
    Martinez-Otzeta, J.M., Ibarguren, A., Ansuategui, A., Susperregi, L.: Laser Based People Following Behaviour in an Emergency Environment. In: Xie, M., Xiong, Y., Xiong, C., Liu, H., Hu, Z. (eds.) ICIRA 2009. LNCS, vol. 5928, pp. 33–42. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Mozos, O.M., Kurazume, R., Hasegawa, T.: Multi-part people detection using 2D range data. International Journal of Social Robotics 2(1), 31–40 (2010)CrossRefGoogle Scholar
  9. 9.
    Schiele: Visual People Detection - Different Models, Comparison and Discussion. In: IEEE International Conference on Robotics and Automation (ICRA) (2009)Google Scholar
  10. 10.
    Sierra, B., Serrano, N., Larrañaga, P., Plasencia, E.J., Inza, I., Jiménez, J.J., Revuelta, P., Mora, M.L.: Using bayesian networks in the construction of a bi-level multi-classifier. A case study using intensive care unit patients data. Artificial Intelligence in Medicine 22(3), 233–248 (2001)CrossRefGoogle Scholar
  11. 11.
    Treptow, A., Cielniak, G., Duckett, T.: Active people recognition using thermal and grey images on a mobile security robot. In: Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2005), Edmonton, Canada (2005)Google Scholar
  12. 12.
    Zin, T.T., Takahashi, H., Toriu, T., Hama, H.: Fusion of infrared and visible images for robust person detection. Image Fusion (2011), InTech

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Loreto Susperregi
    • 1
    Email author
  • Basilio Sierra
    • 2
  • Jose María Martínez-Otzeta
    • 1
  • Elena Lazkano
    • 2
  • Ander Ansuategui
    • 1
  1. 1.TEKNIKER-IK4BadajozSpain
  2. 2.University of Basque CountrySan SebastianSpain

Personalised recommendations