RGB-D Human Detection and Tracking for Industrial Environments

  • Matteo Munaro
  • Christopher Lewis
  • David Chambers
  • Paul Hvass
  • Emanuele Menegatti
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 302)


Reliably detecting and tracking movements of nearby workers on the factory floor are crucial to the safety of advanced manufacturing automation in which humans and robots share the same workspace. In this work, we address the problem of multiple people detection and tracking in industrial environments by proposing algorithms which exploit both color and depth data to robustly track people in real time. For people detection, a cascade organization of these algorithms is proposed, while tracking is performed based on a particle filter which can interpolate sparse detection results by exploiting color histograms of people. Tracking results of different combinations of the proposed methods are evaluated on a novel dataset collected with a consumer RGB-D sensor in an industrial-like environment. Our techniques obtain good tracking performances even in an industrial setting and reach more than 30 Hz update rate. All these algorithms have been released as open source as part of the ROS-Industrial project.


Human detection and tracking ROS-Industrial RGB-D Open source 



We would like to thank Open Perception Inc., the Southwest Research Institute (SwRI) and the National Institute of Standards and Technology (NIST) for funding this research and Radu Bogdan Rusu for the fruitful discussions on this topic.


  1. 1.
    F. Basso, M. Munaro, S. Michieletto, E. Pagello, and E. Menegatti. Fast and robust multi-people tracking from rgb-d data for a mobile robot. In 12th Intelligent Autonomous Systems Conference (IAS-12), pages 265–276, Jeju Island, Korea, June 2012.Google Scholar
  2. 2.
    G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.Google Scholar
  3. 3.
    Michael D. Breitenstein, Fabian Reichlin, Bastian Leibe, Esther Koller-Meier, and Luc Van Gool. Robust tracking-by-detection using a detector confidence particle filter. In International Conference on Computer Vision (ICCV) 2009, volume 1, pages 1515–1522, October 2009.Google Scholar
  4. 4.
    Alexander Carballo, Akihisa Ohya, and Shin’ichi Yuta. Reliable people detection using range and intensity data from multiple layers of laser range finders on a mobile robot. International Journal of Social Robotics, 3(2):167–186, 2011.Google Scholar
  5. 5.
    David R. Chambers, Clay Flannigan, and Benjamin Wheeler. High-accuracy real-time pedestrian detection system using 2d and 3d features. Proc. SPIE, 8384:83840G–83840G-11, 2012.Google Scholar
  6. 6.
    W. Choi, C. Pantofaru, and S. Savarese. Detecting and tracking people using an rgb-d camera via multiple detector fusion. In International Conference on Computer Vision (ICCV) Workshops 2011, pages 1076–1083, 2011.Google Scholar
  7. 7.
    W. Choi, C. Pantofaru, and S. Savarese. A general framework for tracking multiple people from a moving camera. Pattern Analysis and Machine Intelligence (PAMI), 35(7):1577–1591, 2012.Google Scholar
  8. 8.
    N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition (CVPR) 2005, volume 1, pages 886–893, June 2005.Google Scholar
  9. 9.
    A. Ess, B. Leibe, K. Schindler, and L. Van Gool. A mobile vision system for robust multi-person tracking. In Computer Vision and Pattern Recognition (CVPR) 2008, pages 1–8, 2008.Google Scholar
  10. 10.
    A. Ess, B. Leibe, K. Schindler, and L. Van Gool. Moving obstacle detection in highly dynamic scenes. In International Conference on Robotics and Automation (ICRA) 2009, pages 4451–4458, 2009.Google Scholar
  11. 11.
    Mark Everingham, Luc Gool, Christopher K. Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88:303–338, June 2010.Google Scholar
  12. 12.
    Matthias Luber, Luciano Spinello, and Kai O. Arras. People tracking in rgb-d data with on-line boosted target models. In International Conference On Intelligent Robots and Systems (IROS) 2011, pages 3844–3849, 2011.Google Scholar
  13. 13.
    D. Mitzel and B. Leibe. Real-time multi-person tracking with detector assisted structure propagation. In International Conference on Computer Vision (ICCV) Workshops 2011, pages 974–981. IEEE, 2011.Google Scholar
  14. 14.
    Oscar Mozos, Ryo Kurazume, and Tsutomu Hasegawa. Multi-part people detection using 2d range data. International Journal of Social Robotics, 2:31–40, 2010.Google Scholar
  15. 15.
    M. Munaro, F. Basso, and E. Menegatti. Tracking people within groups with rgb-d data. In Proc. of the International Conference on Intelligent Robots and Systems (IROS), pages 2101–2107, Algarve, Portugal, October 2012.Google Scholar
  16. 16.
    M. Munaro and E. Menegatti. Fast rgb-d people tracking for service robots. Autonomous Robots Journal, 2014.Google Scholar
  17. 17.
    Luis E. Navarro-Serment, Christoph Mertz, and Martial Hebert. Pedestrian detection and tracking using three-dimensional ladar data. In The International Journal of Robotics Research, Special Issue on the Seventh International Conference on Field and Service Robots, pages 103–112, 2009.Google Scholar
  18. 18.
    C. Papageorgiou, T. Evgeniou, and T. Poggio. A trainable pedestrian detection system. In In Proceedings of IEEE Intelligent Vehicles Symposium ’98, pages 241–246, 1998.Google Scholar
  19. 19.
    Morgan Quigley, Brian Gerkey, Ken Conley, Josh Faust, Tully Foote, Jeremy Leibs, Eric Berger, Rob Wheeler, and Andrew Ng. Ros: an open-source robot operating system. In International Conference on Robotics and Automation (ICRA), 2009.Google Scholar
  20. 20.
    Radu Bogdan Rusu and Steve Cousins. 3D is here: Point Cloud Library (PCL). In International Conference on Robotics and Automation (ICRA) 2011, pages 1–4, Shanghai, China, May 9–13 2011.Google Scholar
  21. 21.
    Luciano Spinello and Kai O. Arras. People detection in rgb-d data. In International Conference On Intelligent Robots and Systems (IROS) 2011, pages 3838–3843, 2011.Google Scholar
  22. 22.
    Luciano Spinello, Kai O. Arras, Rudolph Triebel, and Roland Siegwart. A layered approach to people detection in 3d range data. In Conference on Artificial Intelligence AAAI’10, PGAI Track, Atlanta, USA, 2010.Google Scholar
  23. 23.
    Luciano Spinello, Matthias Luber, and Kai O. Arras. Tracking people in 3d using a bottom-up top-down people detector. In International Conference on Robotics and Automation (ICRA) 2011, pages 1304–1310, Shanghai, 2011.Google Scholar
  24. 24.
    Paul Viola and Michael Jones. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition (CVPR) 2001, volume 1, pages 511–518, 2001.Google Scholar
  25. 25.
    Hao Zhang, C. Reardon, and L.E. Parker. Real-time multiple human perception with color-depth cameras on a mobile robot. IEEE Transactions on Cybernetics - Part B, 43(5):1429–1441, Oct 2013.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Matteo Munaro
    • 1
  • Christopher Lewis
    • 2
  • David Chambers
    • 2
  • Paul Hvass
    • 2
  • Emanuele Menegatti
    • 1
  1. 1.Department of Information EngineeringUniversity of PadovaPadovaItaly
  2. 2.Southwest Research InstituteSan AntonioUSA

Personalised recommendations