Skip to main content
SpringerLink
Log in

From Tele-Operated Robots to Social Robots with Autonomous Behaviors

  • Conference paper
  • First Online:
Robotics (SBR 2014 2014, ROBOCONTROL 2014, LARS 2014)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 507))

Abstract

This paper presents a perception/interface device for Telepresence Mobile Robots using a Kinect sensor. Firstly, using the Kinect RGB camera (Webcam) and image processing techniques, it is possible to detect a human face, allowing the robot to track the face, getting closer of a person, moving forward and rotating to get a better pose (position and orientation) to interact with him/her. Then it is possible to recognize hand gestures using the Kinect 3D sensor (Depth camera). The proposed gesture recognition method tracks the hands positions and movements, when moving it forward towards the robot, and then recognizing a set of predefined gestures/commands. Finally, the 3D perception provided by the Kinect also allows us to detect obstacles, avoiding collisions and safely moving the mobile robot base, also allowing to search for someone in the environment. Practical experiments are presented demonstrating the obtained results: (i) in the searching and tracking of human faces; (ii) in the robot positioning related to the user we want to interact with; and also (iii) in the human gesture recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Viola-Jones detectors from OpenCV (http://opencv.org) with three training files: haarcascade_frontalface_alt2.xml and haarcascade_frontalface_default.xml.

  2. 2.

    Viola-Jones detectors from OpenCV (http://opencv.org) with three training files: haarcascade_frontalface_alt2.xml, haarcascade_frontalface_default.xml, and haarcascade_profileface.xml.

  3. 3.

    See the videos at the link: https://www.youtube.com/channel/UC13inNWCkmGwoAtpbqpJucw.

References

  1. Alvarenga, M.L.T., Correa, D.S.O., Osório, F.S.: Redes neurais artificiais aplicadas no reconhecimento de gestos usando o kinect. In: Computer on the Beach, p. 10 (2012)

    Google Scholar 

  2. Bartneck, C., Forlizzi, J.: A design-centred framework for social human-robot interaction. In: 13th IEEE International Workshop on Robot and Human Interactive Communication, ROMAN 2004, pp. 591–594, Sept 2004

    Google Scholar 

  3. Billinghurst, M.: Gesture based interaction (chapt. 14). In: Human Input to Computer Systems: Theories, Techniques and Technology (2011)

    Google Scholar 

  4. Chen, Q., Georganas, N.D., Petriu, E.M.: Real-time vision-based hand gesture recognition using haar-like features. In: Instrumentation and Measurement Technology Conference Proceedings, IMTC 2007, pp. 1–6. IEEE, May 2007

    Google Scholar 

  5. Clodic, A., Fleury, S., Alami, R., Herrb, M., Chatila, R.: Supervision and interaction. In: Proceedings of the 12th International Conference on Advanced Robotics, ICAR 2005, pp. 725–732, July 2005

    Google Scholar 

  6. Correa, D.S.O., Sciotti, D.F., Prado, M.G., Sales, D.O., Wolf, D.F., Osorio, F.S.: Mobile robots navigation in indoor environments using kinect sensor. In: 2012 Second Brazilian Conference on Critical Embedded Systems (CBSEC), pp. 36–41, May 2012

    Google Scholar 

  7. DoubleRobotics: Work from anywhere (telepresence). http://www.doublerobotics.com. Accessed: 23 April 2014

  8. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Pearson/Prentice Hall, Upper Saddle River, N. J. (2008)

    Google Scholar 

  9. Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 532–550 (1987)

    Article  Google Scholar 

  10. Hasanuzzaman, M., Zhang, T., Ampornaramveth, V., Gotoda, H., Shirai, Y., Ueno, H.: Adaptive visual gesture recognition for human-robot interaction using a knowledge-based software platform. Robot. Auton. Syst. 55(8), 643–657 (2007). http://www.sciencedirect.com/science/article/pii/S0921889007000383

    Article  Google Scholar 

  11. IJsselsteijn, W.A.: History of telepresence. In: Schreer, O., Kauff, P., Sikora, T. (eds.) 3D Communication: Algorithms, Concepts and Real-time Systems in Human Centred Communication, pp. 7–22. Wiley, Chichester (2005)

    Google Scholar 

  12. iRobot: irobot \({\rm ava}\textregistered \) 500 - video collaboration robot.http://www.irobot.com/en/us/learn/commercial. Accessed: 23 April 2014

  13. Jensen, B., Tomatis, N., Mayor, L., Drygajlo, A., Siegwart, R.: Robots meet humans-interaction in public spaces. IEEE Trans. Industr. Electron. 52(6), 1530–1546 (2005)

    Article  Google Scholar 

  14. Kanda, T., Glas, D.F., Shiomi, M., Hagita, N.: Abstracting people’s trajectories for social robots to proactively approach customers. IEEE Trans. Robot. 25(6), 1382–1396 (2009)

    Article  Google Scholar 

  15. Markoff, J.: The boss is robotic, and rolling up behind you. The New York Times - Science, 4 Sep 2010

    Google Scholar 

  16. Michalowski, M.P., Šabanović, S., DiSalvo, C., Busquets, D., Hiatt, L.M., Melchior, N.A., Simmons, R.: Socially distributed perception: Grace plays social tag at aaai 2005. Auton. Robots 22(4), 385–397 (2007). http://dx.doi.org/10.1007/s10514-006-9015-6

    Article  Google Scholar 

  17. Minsky, M.: Telepresence. OMNI Magazine, June 1980

    Google Scholar 

  18. Mumm, J., Mutlu, B.: Human-robot proxemics: physical and psychological distancing in human-robot interaction. In: Proceedings of the 6th International Conference on Human-Robot Interaction, pp. 331–338. ACM (2011)

    Google Scholar 

  19. NASA: Robonaut telepresence. http://robonaut.jsc.nasa.gov/R1/sub/telepresence.asp. Accessed 23 April 2014

  20. Sales, D., Correa, D., Osório, F.S., Wolf, D.F.: 3D vision-based autonomous navigation system using ANN and kinect sensor. In: Jayne, C., Yue, S., Iliadis, L. (eds.) EANN 2012. CCIS, vol. 311, pp. 305–314. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  21. Sales, D.O., Correa, D.O., Fernandes, L.C., Wolf, D.F., Osório, F.S.: Adaptive finite state machine based visual autonomous navigation system. Eng. Appl. Artif. Intell. 29, 152–162 (2014). http://www.sciencedirect.com/science/article/pii/S0952197613002406

    Article  Google Scholar 

  22. Simmons, R., Goldberg, D., Goode, A., Montemerlo, M., Roy, N., Schultz, A.C., Abramson, M., Horswill, I., Kortenkamp, D., Maxwell, B.: Grace: An autonomous robot for the aaai robot challenge. Technical report, DTIC Document (2003)

    Google Scholar 

  23. Snowden, E.: Ted talk edward snowden (telepresence - live from ted2014). https://www.ted.com/speakers/edward_snowden. Accessed: 23 April 2014

  24. Soille, P.: Morphological Image Analysis: Principles and Applications, 2nd edn. Springer-Verlag New York Inc., Secaucus (2003)

    MATH  Google Scholar 

  25. Technologies, S.: Beam+ and beam pro (telepresence). https://www.suitabletech.com. Accessed: 23 April 2014

  26. Thrun, S., Beetz, M., Bennewitz, M., Burgard, W., Cremers, A.B., Dellaert, F., Fox, D., Hahnel, D., Rosenberg, C., Roy, N., Schulte, J., Schulz, D.: Probabilistic algorithms and the interactive museum tour-guide robot minerva. Int. J. Robot. Res. 19(11), 972–999 (2000)

    Article  Google Scholar 

  27. Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57(2), 137–154 (2001)

    Article  Google Scholar 

  28. Waldherr, S., Romero, R., Thrun, S.: A gesture based interface for human-robot interaction. Auton. Robots 9(2), 151–173 (2000)

    Article  Google Scholar 

  29. Welch, G., Bishop, G.: An introduction to the kalman filter. Technical report, Chapel Hill, NC, USA (1995)

    Google Scholar 

  30. Wortham, J.: With kinect controller, hackers take liberties. The New York Times 21 (November 2010)

    Google Scholar 

Download references

Acknowledgment

The authors acknowledge the support granted by the Mobile Robotics Laboratory (LRM Lab.) and the Center for Robotics (CRob/SC) of University of São Paulo at São Carlos.

Rafael Alceste Berri thanks CAPES/DS by the financial support for the graduate program in computer science.

Author information

Authors and Affiliations

  1. Mobile Robotics Lab (LRM) and Center for Robotics (CRob/USP-SC), University of São Paulo-ICMC, Trabalhador São-carlense. 400, São Carlos, São Paulo, 13566-590, Brazil

    Rafael Berri, Denis Wolf & Fernando Osório

Authors
  1. Rafael Berri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Denis Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernando Osório
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Osório .

Editor information

Editors and Affiliations

  1. Universidade de São Paulo, São Carlos, São Paulo, Brazil

    Fernando S. Osório

  2. Universidade de São Paulo, São Carlos, São Paulo, Brazil

    Denis Fernando Wolf

  3. Universidade de São Paulo, São Carlos, São Paulo, Brazil

    Kalinka Castelo Branco

  4. Universidade de São Paulo, São Carlos, São Paulo, Brazil

    Valdir Grassi Jr.

  5. Universidade de São Paulo, São Carlos, São Paulo, Brazil

    Marcelo Becker

  6. Universidade de São Paulo, São Carlos, São Paulo, Brazil

    Roseli Romero

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Berri, R., Wolf, D., Osório, F. (2015). From Tele-Operated Robots to Social Robots with Autonomous Behaviors. In: Osório, F., Wolf, D., Castelo Branco, K., Grassi Jr., V., Becker, M., Romero, R. (eds) Robotics. SBR 2014 ROBOCONTROL LARS 2014 2014 2014. Communications in Computer and Information Science, vol 507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48134-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-48134-9_3

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-48133-2

  • Online ISBN: 978-3-662-48134-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation