Skip to main content
SpringerLink
Log in

SocRob@Home

Integrating AI Components in a Domestic Robot System

  • Technical Contribution
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

This paper describes the SocRob@Home robot system, consisting of a mobile robot (MBOT) equipped with several sensors and actuators, including a manipulator arm, and several software modules that provide the skills and capability to perform domestic tasks while interacting with humans in a domestic environment. We describe the whole system holistically, explaining how it integrates the contributing modules, and then we focus on the most relevant sub-systems, pointing out the original contributions of our research and development on the system in the last 5 years. The robot system includes metric and semantic mapping, several navigation modes (way-point navigation, person following and multi-sensor obstacle detection and avoidance), vision-based object detection, recognition, servoing and grasping, speech understanding, task planning and task execution. The robot system is mostly activated by speech commands from a human, and these commands, after being interpreted, are executed by the robot sub-systems, coordinated by a task executor. Lessons learned during the development and use of this system, which are useful as guidelines for the development of similar robot systems, are provided. MBOT’s performance is assessed using the task benchmarks scoring system of the European Robotics League competitions on Consumer Service robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://monarch-fp7.eu.

  2. https://irsgroup.isr.tecnico.ulisboa.pt/harode/harode/.

  3. http://gaips.inesc-id.pt/inside/.

  4. https://www.robocup.org/domains/3.

  5. http://rockinrobotchallenge.eu/home.php.

  6. https://www.eu-robotics.net/robotics_league/erl-consumer/about/index.html.

  7. http://socrob.isr.tecnico.ulisboa.pt.

  8. http://wiki.ros.org/openslam_gmapping.

  9. http://wiki.ros.org/move_base.

  10. http://wiki.ros.org/amcl.

  11. \(\varGamma\) is set to an arbitrarily small ball around the goal.

  12. https://github.com/socrob/mbot_natural_language_processing.

References

  1. Bellotto N, Hanheide M, Dondrup C (2014) BayesTracking: C++ framework for Bayesian filter tracking (UKF, EKF, Particles). https://github.com/LCAS/bayestracking. https://doi.org/10.5281/zenodo.10318

  2. Bellotto N, Hu H (2010) Computationally efficient solutions for tracking people with a mobile robot: an experimental evaluation of Bayesian filters. Auton Robots 28(4):425–438

    Article  Google Scholar 

  3. Brock O, Khatib O (1999) High-speed navigation using the global dynamic window approach. Proc IEEE Int Conf Robotics Autom 1:341–346 (IEEE)

    Article  Google Scholar 

  4. Brzozowska E, Lima O, Ventura R (2019) A generic optimization based cartesian controller for robotic mobile manipulation. In: Proceedings of the IEEE international conference on robotics and automation (ICRA-19). (accepted)

  5. Cartucho J, Ventura R, Veloso M (2018) Robust object recognition through symbiotic deep learning in mobile robots. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2336–2341 (IEEE)

  6. Cashmore M, Fox M, Long D, Magazzeni D, Ridder B, Carrera A, Palomeras N, Hurtos N, Carreras M (2015) Rosplan: planning in the robot operating system. In: ICAPS, pp 333–341

  7. Central Lab Facilities (CITEC) and SocRob project at ISR/IST: bayes\_people\_tracker. (2018). https://github.com/socrob/bayes_people_tracker

  8. Chaumette F, Hutchinson S (2006) Visual servo control. I. Basic approaches. IEEE Robot Autom Mag 13(4):82–90

    Article  Google Scholar 

  9. Chitta S, Sucan I, Cousins S (2012) Moveit![ROS topics]. IEEE Robot Autom Mag 19(1):18–19

    Article  Google Scholar 

  10. De Raedt L, Kimmig A (2015) Probabilistic (logic) programming concepts. Mach Learn 100(1):5–47. https://doi.org/10.1007/s10994-015-5494-z

    Article  MathSciNet  MATH  Google Scholar 

  11. Forlizzi J, DiSalvo C (2006) Service robots in the domestic environment: a study of the roomba vacuum in the home. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction, pp 258–265. ACM

  12. Fox D, Burgard W, Thrun S (1997) The dynamic window approach to collision avoidance. IEEE Robot Autom Mag 4(1):23–33

    Article  Google Scholar 

  13. Girshick R (2015) Fast R-CNN. In: IEEE Int’l Conf. computer vision (ICCV), pp 1440–1448

  14. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conf. computer vision and pattern recognition (CVPR), pp 580–587

  15. Gonçalves J, Lima PU (2019) Grasp planning with incomplete knowledge about the object to be grasped. In: Proc. of 19th IEEE Int. Conf. on autonomous robot systems and competitions (ICARSC). IEEE

  16. Han J, Jo M, Park S, Kim S (2005) The educational use of home robots for children. In: Robot and human interactive communication, 2005. ROMAN 2005. IEEE International Workshop on, pp 378–383. IEEE

  17. Hess W, Kohler D, Rapp H, Andor D (2016) Real-time loop closure in 2D lidar slam. In: 2016 IEEE international conference on robotics and automation (ICRA), pp 1271–1278

  18. Hirai K, Hirose M, Haikawa Y, Takenaka T (1998) The development of Honda humanoid robot. Robot Autom 2018, Proc 1998 IEEE Int Conf 2:1321–1326 (IEEE)

    Google Scholar 

  19. Katz M, Hoffmann J (2014) Mercury planner: pushing the limits of partial delete relaxation. In: Proceedings of the 8th international planning competition (IPC-2014)

  20. Kinarullathil1 M, Martins PH, Azevedo C, Lima O, Lawless G, Lima PU, Custódio L, Ventura R (2018) From user spoken commands to robot task plans: a case study in RoboCup@Home. In: Proc. of language and robotics workshop of IROS2018. IEEE

  21. Kraetzschmar G, Basiri M, Lima PU, Ahmad A, Amigoni F, Awaad I, Berghofer J, Bonarini RBA, Dwiputra R, Fontana G, Hegger F, Hochgeschwender N, Iocchi L, Matteucci M, Nardi D, Miraldo P, Schiaffonati V, Schneider S, Bastianelli E, Resende P, Mendes J (2018) European Robotics League for consumer service robots rulebook, 9 October. Available online at: https://www.eu-robotics.net/robotics_league/upload/documents-2018/ERL_Consumer_10092018.pdf

  22. Lima O, Ventura R, Awaad I (2018) Integrating classical planning and real robots in industrial and service robotics domains. PlanRob, ICAPS Workshop

  23. Lima PU (2018) A probabilistic approach to benchmarking and performance evaluation of robot systems. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 7231–7236. IEEE

  24. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: European conference on computer vision, pp 740–755. Springer

  25. Martins PH, Custódio L, Ventura R (2018) A deep learning approach for understanding natural language commands for mobile service robots. CoRR abs/1807.03053. arXiv:1807.03053

  26. Matamoros M, Seib V, Memmesheimer R, Paulus D (2018) Robocup@Home: summarizing achievements in over 11 years of competition. In: Proceedings of 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC 2018), pp 186–191, April, Torres Vedras, Portugal

  27. McDermott DM (2000) The 1998 AI planning systems competition. AI Mag 21(2):35

    Google Scholar 

  28. Melo FS, Sardinha A, Belo D, Couto M, Faria M, Farias A, Gambôa H, Jesus C, Kinarullathil M, Lima P, Luz L, Mateus A, Melo I, Moreno P, Osório D, Paiva A, Pimentel J, Rodrigues J, Sequeira P, Solera-Ureña R, Vasco M, Veloso M, Ventura R (2018) Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy. Artif Intell Med 96:198–216

    Article  Google Scholar 

  29. Messias J, Ventura R, Lima P, Sequeira J, Alvito P, Marques C, Carriço P (2014) A robotic platform for edutainment activities in a pediatric hospital. In: Proceedings of the IEEE international conference on autonomous robot systems and competitions (ICARSC), pp 193–198

  30. Montemerlo M, Thrun S (2007) Fastslam 2.0. FastSLAM: s scalable method for the simultaneous localization and mapping problem in robotics. Springer, New York, pp 63–90

    MATH  Google Scholar 

  31. Pangercic D, Pitzer B, Tenorth M, Beetz M (2012) Semantic object maps for robotic housework-representation, acquisition and use. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, pp 4644–4651. IEEE

  32. Redmon J (2013–2016) Darknet: open source neural networks in c. http://pjreddie.com/darknet/

  33. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767

  34. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of Neural Information Processing Systems (NIPS 2015), pp 91–99, Montreal, Canada, December

  35. Rusu RB, Blodow N, Marton ZC, Beetz M (2009) Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments. In: Intelligent robots and systems, 2009. IROS 2009. IEEE/RSJ international conference on, pp 1–6. IEEE

  36. Sethian JA (1999) Fast marching methods. SIAM Rev 41(2):199–235

    Article  MathSciNet  Google Scholar 

  37. Spaan MT, Veiga TS, Lima PU (2015) Decision-theoretic planning under uncertainty with information rewards for active cooperative perception. Auton Agents Multi-agent Syst 29(6):1157–1185. https://doi.org/10.1007/s10458-014-9279-8

    Article  Google Scholar 

  38. Stückler J, Holz D, Behnke S (2012) Demonstrating everyday manipulation skills in robocup@home. Robot Autom Mag 19(2):34–42

    Article  Google Scholar 

  39. Talebpour Z, Geetha Viswanathan D, Ventura R, Englebienne G, Martinoli A (2016) Incorporating perception uncertainty in human-aware navigation: a comparative study. In: IEEE international symposium on robot and human interactive communication (RO-MAN)

  40. Veiga TS, Silva M, Ventura R, Lima PU (2019) An hierarchical approach to active semantic mapping using probabilistic logic and information reward POMDP. In: Proc. of 29th international conference on automated planning and scheduling (ICAPS)

  41. Ventura R, Ahmad A (2015) RoboCup 2014: Robot World Cup XVIII, chap. Towards optimal robot navigation in domestic spaces, pp 318–331. LNAI 8992. Springer. https://doi.org/10.1007/978-3-319-18615-3_26

    Chapter  Google Scholar 

  42. Wasik A, Pereira JN, Ventura R, Lima PU, Martinoli A (2016) Graph-based distributed control for adaptive multi-robot patrolling using local formation reconfiguration. In: IEEE/RSJ international conference on intelligent robots and systems (IROS)

Download references

Author information

Authors and Affiliations

  1. Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal

    Pedro U. Lima, Carlos Azevedo, Emilia Brzozowska, João Cartucho, Tiago J. Dias, João Gonçalves, Mithun Kinarullathil, Guilherme Lawless, Oscar Lima, Rute Luz, Pedro Miraldo, Enrico Piazza, Miguel Silva, Tiago Veiga & Rodrigo Ventura

Authors
  1. Pedro U. Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Azevedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emilia Brzozowska
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. João Cartucho
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tiago J. Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. João Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mithun Kinarullathil
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Guilherme Lawless
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Oscar Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Rute Luz
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Pedro Miraldo
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Enrico Piazza
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Miguel Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Tiago Veiga
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Rodrigo Ventura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro U. Lima.

Additional information

This work was partially supported by ISR/LARSyS Strategic Funding through the FCT project PEst-OE/EEI/LA0009/2013 and by the FCT project HARODE PTDC/EEI-SII/4698/2014.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lima, P.U., Azevedo, C., Brzozowska, E. et al. SocRob@Home. Künstl Intell 33, 343–356 (2019). https://doi.org/10.1007/s13218-019-00618-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-019-00618-w

Keywords

Search

Navigation