Skip to main content
SpringerLink
Log in
Book cover

International Conference on Intelligent Autonomous Systems

IAS 2016: Intelligent Autonomous Systems 14 pp 485–497Cite as

A Powerful and Cost-Efficient Human Perception System for Camera Networks and Mobile Robotics

  • Conference paper
  • First Online:

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 531))

Abstract

In this work, we present a software library which enables the efficient use of the Kinect One, a time-of-flight RGB-D sensor, with the nVidia Jetson TK1, an ARM-based embedded system, for the purpose of people detection. Our software exploits nVidia CUDA to process all data necessary for robust people detection algorithm and other perception algorithms by parallelizing the generation of the 3D point cloud and many pixel-wise operations on both the raw depth and the infrared images coming from the Kinect One sensor. The library developed has been released as open-source and the whole system has been tested as a people detection node in an open source multi-node RGB-D tracking framework (OpenPTrack). The results gathered show that the proposed system can be effectively used as a people detection node, outperforming the state-of-the-art in terms of people detection frame rate not only with the nVidia Jetson, but also with non-embedded computers.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/OpenPTrack/libfreenect2

  2. 2.

    https://github.com/OpenPTrack/kinect2_bridge

  3. 3.

    https://github.com/OpenKinect/libfreenect2

  4. 4.

    https://github.com/OpenPTrack/libfreenect2/tree/jetson-dev https://github.com/OpenPTrack/kinect2_bridge/tree/jetson-dev

References

  1. Zennaro, S., Munaro, M., Milani, S., Zanuttigh, P., Bernardi, A., Ghidoni, S., Menegatti, E.: Performance evaluation of the 1st and 2nd generation kinect for multimedia applications. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2015)

    Google Scholar 

  2. Gokturk, S.B., Yalcin, H., Bamji, C.: A time-of-flight depth sensor-system description, issues and solutions. In: Conference on Computer Vision and Pattern Recognition Workshop, 2004. CVPRW’04, pp. 35–35. IEEE (2004)

    Google Scholar 

  3. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 5 (2009)

    Google Scholar 

  4. Munaro, M., Basso, F., Menegatti, E.: Openptrack: open source multi-camera calibration and people tracking for RGB-D camera networks. Robot. Auton. Syst. (2015)

    Google Scholar 

  5. Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)

    Article  Google Scholar 

  6. Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the RGB-D slam system. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1691–1696. IEEE (2012)

    Google Scholar 

  7. Munaro, M., Menegatti, E.: Fast RGB-D people tracking for service robots. Auton. Robots 37(3), 227–242 (2014)

    Article  Google Scholar 

  8. Munaro, M., Basso, F., Michieletto, S., Pagello, E., Menegatti, E.: A software architecture for RGB-D people tracking based on ROS framework for a mobile robot. In: Frontiers of Intelligent Autonomous Systems, pp. 53–68. Springer (2013)

    Google Scholar 

  9. Fleuret, F., Shitrit, H.B., Fua, P.: Re-identification for improved people tracking. In: Person Re-identification, pp. 309–330. Springer (2014)

    Google Scholar 

  10. Munaro, M., Ghidoni, S., Dizmen, D.T., Menegatti, E.: A feature-based approach to people re-identification using skeleton keypoints. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5644–5651. IEEE (2014)

    Google Scholar 

  11. Nanni, L., Munaro, M., Ghidoni, S., Menegatti, E., Brahnam, S.: Ensemble of different approaches for a reliable person re-identification system. Appl. Comput. Inf. (2015)

    Google Scholar 

  12. Carraro, M., Antonello, M., Tonin, L., Menegatti, E.: An open source robotic platform for ambient assisted living. In: Artificial Intelligence and Robotics (AIRO) (2015)

    Google Scholar 

  13. Fischinger, D., Einramhof, P., Papoutsakis, K., Wohlkinger, W., Mayer, P., Panek, P., Hofmann, S., Koertner, T., Weiss, A., Argyros, A., et al.: Hobbit, a care robot supporting independent living at home: first prototype and lessons learned. Robot. Auton. Syst. (2014)

    Google Scholar 

  14. Ghidoni, S., Anzalone, S.M., Munaro, M., Michieletto, S., Menegatti, E.: A distributed perception infrastructure for robot assisted living. Robot. Auton. Syst. 62(9), 1316–1328 (2014)

    Article  Google Scholar 

  15. Basso, F., Pretto, A., Menegatti, E.: Unsupervised intrinsic and extrinsic calibration of a camera-depth sensor couple. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 6244–6249. IEEE (2014)

    Google Scholar 

  16. Nickolls, J., Buck, I., Garland, M., Skadron, K.: Scalable parallel programming with CUDA. Queue 6(2), 40–53 (2008)

    Article  Google Scholar 

  17. Xiang, L.: libfreenect2 CUDA library. https://github.com/xlz/libfreenect2 (2015). [Online; accessed 2016-02-03]

  18. Wang, X.: Intelligent multi-camera video surveillance: a review. Pattern Recognit. Lett. 34(1), 3–19 (2013)

    Article  Google Scholar 

  19. Vezzani, R., Baltieri, D., Cucchiara, R.: Pathnodes integration of standalone particle filters for people tracking on distributed surveillance systems. In: Image Analysis and Processing–ICIAP 2009, pp. 404–413. Springer (2009)

    Google Scholar 

  20. Kandhalu, A., Rowe, A., Rajkumar, R., Huang, C., Yeh, C.-C.: Real-time video surveillance over IEEE 802.11 mesh networks. In: Real-Time and Embedded Technology and Applications Symposium, 2009. RTAS 2009. 15th IEEE, pp. 205–214. IEEE (2009)

    Google Scholar 

  21. Jafari, O.H., Mitzel, D., Leibe, B.: Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5636–5643. IEEE (2014)

    Google Scholar 

  22. Munaro, M., Basso, F., Menegatti, E.: Tracking people within groups with RGB-D data. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2101–2107. IEEE (2012)

    Google Scholar 

  23. Munaro, M., Lewis, C., Chambers, D., Hvass, P., Menegatti, E.: RGB-D human detection and tracking for industrial environments. In: Intelligent Autonomous Systems, vol. 13, pp. 1655–1668. Springer (2016)

    Google Scholar 

  24. Sanders, J., Kandrot, E.: CUDA by example: an introduction to general-purpose GPU programming. Addison-Wesley Professional (2010)

    Google Scholar 

  25. Ryoo, S., Rodrigues, C.I., Baghsorkhi, S.S., Stone, S.S., Kirk, D.B., Hwu, W.-m.W.: Optimization principles and application performance evaluation of a multithreaded GPU using CUDA. In: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 73–82. ACM (2008)

    Google Scholar 

  26. Aldoma, A., Tombari, F., Prankl, J., Richtsfeld, A., Di Stefano, L., Vincze, M.: Multimodal cue integration through hypotheses verification for RGB-D object recognition and 6DOF pose estimation. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 2104–2111. IEEE (2013)

    Google Scholar 

  27. Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robot. Res. 31(5), 647–663 (2012)

    Article  Google Scholar 

  28. Carraro, M., Munaro, M., Menegatti, E.: Cost-efficient RGB-D smart camera for people detection and tracking. J. Electron. Imaging 25(4), 041007–041007 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

Portions of this work have been supported by NVidia, OpenPerception and the REMAP center at UCLA. The authors would like to thank Randy Illum and Jeff Burke at UCLA for the extensive testing of the developed software.

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Padova, Via Gradenigo 6/A, 35131, Padova, Italy

    Marco Carraro, Matteo Munaro & Emanuele Menegatti

Authors
  1. Marco Carraro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matteo Munaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emanuele Menegatti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Marco Carraro , Matteo Munaro or Emanuele Menegatti .

Editor information

Editors and Affiliations

  1. Shanghai Jiao Tong University , Shanghai, China

    Weidong Chen

  2. Osaka University , Osaka, Japan

    Koh Hosoda

  3. University of Padua , Padua, Italy

    Emanuele Menegatti

  4. Osaka University , Osaka, Japan

    Masahiro Shimizu

  5. Shanghai Jiao Tong University , Shanghai, China

    Hesheng Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Carraro, M., Munaro, M., Menegatti, E. (2017). A Powerful and Cost-Efficient Human Perception System for Camera Networks and Mobile Robotics. In: Chen, W., Hosoda, K., Menegatti, E., Shimizu, M., Wang, H. (eds) Intelligent Autonomous Systems 14. IAS 2016. Advances in Intelligent Systems and Computing, vol 531. Springer, Cham. https://doi.org/10.1007/978-3-319-48036-7_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48036-7_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48035-0

  • Online ISBN: 978-3-319-48036-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation