Onboard Robust Person Detection and Tracking for Domestic Service Robots

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 418)


Domestic assistance for the elderly and impaired people is one of the biggest upcoming challenges of our society. Consequently, in-home care through domestic service robots is identified as one of the most important application area of robotics research. Assistive tasks may range from visitor reception at the door to catering for owner’s small daily necessities within a house. Since most of these tasks require the robot to interact directly with humans, a predominant robot functionality is to detect and track humans in real time: either the owner of the robot or visitors at home or both. In this article we present a robust method for such a functionality that combines depth-based segmentation and visual detection. The robustness of our method lies in its capability to not only identify partially occluded humans (e.g., with only torso visible) but also to do so in varying lighting conditions. We thoroughly validate our method through extensive experiments on real robot datasets and comparisons with the ground truth. The datasets were collected on a home-like environment set up within the context of RoboCup@Home and RoCKIn@Home competitions.


Person detection Tracking Domestic environment  RoboCup Service robotics Benchmarking 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
  3. 3.
    Ahmad, A., Xavier, J., Santos-Victor, J., Lima, P.: 3D to 2D bijection for spherical objects under equidistant fisheye projection. Computer Vision and Image Understanding 125, 172–183 (2014)CrossRefGoogle Scholar
  4. 4.
    Camplani, M., Salgado, L.: Background foreground segmentation with RGB-D kinect data: An efficient combination of classifiers. Journal of Visual Communication and Image Representation 25(1), 122–136 (2014)CrossRefGoogle Scholar
  5. 5.
    Cruz, L., Lucio, D., Velho, L.: Kinect and rgbd images: Challenges and applications. In: 25th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI-T), pp. 36–49. IEEE (2012)Google Scholar
  6. 6.
    Cucchiara, R., Prati, A., Vezzani, R.: A multi-camera vision system for fall detection and alarm generation. Expert Systems 24(5), 334–345 (2007)CrossRefGoogle Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  8. 8.
    Horprasert, T., Harwood, D., Davis, L.: A robust background subtraction and shadow detection. In: Proc. ACCV, pp. 983–988 (2000)Google Scholar
  9. 9.
    Kaushik, A., Celler, B.: Characterization of pir detector for monitoring occupancy patterns and functional health status of elderly people living alone at home. Technology and Health Care 15(4), 273–288 (2007)Google Scholar
  10. 10.
    Kulyukin, V., Gharpure, C., Nicholson, J., Pavithran, S.: RFID in robot-assisted indoor navigation for the visually impaired. In: 2004 IEEE/RSJ International Conference on Proceedings of the Intelligent Robots and Systems, (IROS 2004), vol. 2, pp. 1979–1984, September 2004Google Scholar
  11. 11.
    Noury, N., Herve, T., Rialle, V., Virone, G., Mercier, E., Morey, G., Moro, A., Porcheron, T.: Monitoring behavior in home using a smart fall sensor and position sensors. In: 1st Annual International, Conference on Microtechnologies in Medicine and Biology, pp. 607–610 (2000)Google Scholar
  12. 12.
    Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)Google Scholar
  13. 13.
    Satake, J., Miura, J.: Robust stereo-based person detection and tracking for a person following robot. In: ICRA Workshop on People Detection and Tracking (2009)Google Scholar
  14. 14.
    Scheutz, M., McRaven, J., Cserey, G.: Fast, reliable, adaptive, bimodal people tracking for indoor environments. In: 2004 IEEE/RSJ International Conference on Proceedings of the Intelligent Robots and Systems, (IROS 2004), vol. 2, pp. 1347–1352, September 2004Google Scholar
  15. 15.
    Srichumroenrattana, N., Lursinsap, C., Lipikorn, R.: 2D face image depth ordering using adaptive hillcrest-valley classification and Otsu. In: 2010 IEEE 10th International Conference on Signal Processing (ICSP), pp. 645–648, October 2010Google Scholar
  16. 16.
    Vezzani, R., Grana, C., Cucchiara, R.: Probabilistic people tracking with appearance models and occlusion classification: The ad-hoc system. Pattern Recognition Letters 32(6), 867–877 (2011)CrossRefGoogle Scholar
  17. 17.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I-511 (2001)Google Scholar
  18. 18.
    Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Ninth IEEE International Conference on Proceedings of the Computer Vision, pp. 734–741. IEEE (2003)Google Scholar
  19. 19.
    Wen-Hau, L., Wu, C., Fu, L.: Inhabitants tracking system in a cluttered home environment via floor load sensors. IEEE Transactions on Automation Science and Engineering 5(1), 10–20 (2008)CrossRefGoogle Scholar
  20. 20.
    Xia, L., Chen, C., Aggarwal, J.: Human detection using depth information by kinect. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 15–22. IEEE (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Robotics and Cybernetics Research GroupCenter for Automation and RoboticsMadridSpain
  2. 2.Max Planck Institute for Biological CyberneticsTübingenGermany
  3. 3.Institute for Systems and Robotics, Instituto Superior TcnicoUniversidade de LisboaLisboaPortugal

Personalised recommendations