This paper describes the SocRob@Home robot system, consisting of a mobile robot (MBOT) equipped with several sensors and actuators, including a manipulator arm, and several software modules that provide the skills and capability to perform domestic tasks while interacting with humans in a domestic environment. We describe the whole system holistically, explaining how it integrates the contributing modules, and then we focus on the most relevant sub-systems, pointing out the original contributions of our research and development on the system in the last 5 years. The robot system includes metric and semantic mapping, several navigation modes (way-point navigation, person following and multi-sensor obstacle detection and avoidance), vision-based object detection, recognition, servoing and grasping, speech understanding, task planning and task execution. The robot system is mostly activated by speech commands from a human, and these commands, after being interpreted, are executed by the robot sub-systems, coordinated by a task executor. Lessons learned during the development and use of this system, which are useful as guidelines for the development of similar robot systems, are provided. MBOT’s performance is assessed using the task benchmarks scoring system of the European Robotics League competitions on Consumer Service robots.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
\(\varGamma\) is set to an arbitrarily small ball around the goal.
Bellotto N, Hu H (2010) Computationally efficient solutions for tracking people with a mobile robot: an experimental evaluation of Bayesian filters. Auton Robots 28(4):425–438
Brock O, Khatib O (1999) High-speed navigation using the global dynamic window approach. Proc IEEE Int Conf Robotics Autom 1:341–346 (IEEE)
Brzozowska E, Lima O, Ventura R (2019) A generic optimization based cartesian controller for robotic mobile manipulation. In: Proceedings of the IEEE international conference on robotics and automation (ICRA-19). (accepted)
Cartucho J, Ventura R, Veloso M (2018) Robust object recognition through symbiotic deep learning in mobile robots. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2336–2341 (IEEE)
Cashmore M, Fox M, Long D, Magazzeni D, Ridder B, Carrera A, Palomeras N, Hurtos N, Carreras M (2015) Rosplan: planning in the robot operating system. In: ICAPS, pp 333–341
Central Lab Facilities (CITEC) and SocRob project at ISR/IST: bayes\_people\_tracker. (2018). https://github.com/socrob/bayes_people_tracker
Chaumette F, Hutchinson S (2006) Visual servo control. I. Basic approaches. IEEE Robot Autom Mag 13(4):82–90
Chitta S, Sucan I, Cousins S (2012) Moveit![ROS topics]. IEEE Robot Autom Mag 19(1):18–19
De Raedt L, Kimmig A (2015) Probabilistic (logic) programming concepts. Mach Learn 100(1):5–47. https://doi.org/10.1007/s10994-015-5494-z
Forlizzi J, DiSalvo C (2006) Service robots in the domestic environment: a study of the roomba vacuum in the home. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction, pp 258–265. ACM
Fox D, Burgard W, Thrun S (1997) The dynamic window approach to collision avoidance. IEEE Robot Autom Mag 4(1):23–33
Girshick R (2015) Fast R-CNN. In: IEEE Int’l Conf. computer vision (ICCV), pp 1440–1448
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conf. computer vision and pattern recognition (CVPR), pp 580–587
Gonçalves J, Lima PU (2019) Grasp planning with incomplete knowledge about the object to be grasped. In: Proc. of 19th IEEE Int. Conf. on autonomous robot systems and competitions (ICARSC). IEEE
Han J, Jo M, Park S, Kim S (2005) The educational use of home robots for children. In: Robot and human interactive communication, 2005. ROMAN 2005. IEEE International Workshop on, pp 378–383. IEEE
Hess W, Kohler D, Rapp H, Andor D (2016) Real-time loop closure in 2D lidar slam. In: 2016 IEEE international conference on robotics and automation (ICRA), pp 1271–1278
Hirai K, Hirose M, Haikawa Y, Takenaka T (1998) The development of Honda humanoid robot. Robot Autom 2018, Proc 1998 IEEE Int Conf 2:1321–1326 (IEEE)
Katz M, Hoffmann J (2014) Mercury planner: pushing the limits of partial delete relaxation. In: Proceedings of the 8th international planning competition (IPC-2014)
Kinarullathil1 M, Martins PH, Azevedo C, Lima O, Lawless G, Lima PU, Custódio L, Ventura R (2018) From user spoken commands to robot task plans: a case study in RoboCup@Home. In: Proc. of language and robotics workshop of IROS2018. IEEE
Kraetzschmar G, Basiri M, Lima PU, Ahmad A, Amigoni F, Awaad I, Berghofer J, Bonarini RBA, Dwiputra R, Fontana G, Hegger F, Hochgeschwender N, Iocchi L, Matteucci M, Nardi D, Miraldo P, Schiaffonati V, Schneider S, Bastianelli E, Resende P, Mendes J (2018) European Robotics League for consumer service robots rulebook, 9 October. Available online at: https://www.eu-robotics.net/robotics_league/upload/documents-2018/ERL_Consumer_10092018.pdf
Lima O, Ventura R, Awaad I (2018) Integrating classical planning and real robots in industrial and service robotics domains. PlanRob, ICAPS Workshop
Lima PU (2018) A probabilistic approach to benchmarking and performance evaluation of robot systems. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 7231–7236. IEEE
Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: European conference on computer vision, pp 740–755. Springer
Martins PH, Custódio L, Ventura R (2018) A deep learning approach for understanding natural language commands for mobile service robots. CoRR abs/1807.03053. arXiv:1807.03053
Matamoros M, Seib V, Memmesheimer R, Paulus D (2018) Robocup@Home: summarizing achievements in over 11 years of competition. In: Proceedings of 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC 2018), pp 186–191, April, Torres Vedras, Portugal
McDermott DM (2000) The 1998 AI planning systems competition. AI Mag 21(2):35
Melo FS, Sardinha A, Belo D, Couto M, Faria M, Farias A, Gambôa H, Jesus C, Kinarullathil M, Lima P, Luz L, Mateus A, Melo I, Moreno P, Osório D, Paiva A, Pimentel J, Rodrigues J, Sequeira P, Solera-Ureña R, Vasco M, Veloso M, Ventura R (2018) Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy. Artif Intell Med 96:198–216
Messias J, Ventura R, Lima P, Sequeira J, Alvito P, Marques C, Carriço P (2014) A robotic platform for edutainment activities in a pediatric hospital. In: Proceedings of the IEEE international conference on autonomous robot systems and competitions (ICARSC), pp 193–198
Montemerlo M, Thrun S (2007) Fastslam 2.0. FastSLAM: s scalable method for the simultaneous localization and mapping problem in robotics. Springer, New York, pp 63–90
Pangercic D, Pitzer B, Tenorth M, Beetz M (2012) Semantic object maps for robotic housework-representation, acquisition and use. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, pp 4644–4651. IEEE
Redmon J (2013–2016) Darknet: open source neural networks in c. http://pjreddie.com/darknet/
Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of Neural Information Processing Systems (NIPS 2015), pp 91–99, Montreal, Canada, December
Rusu RB, Blodow N, Marton ZC, Beetz M (2009) Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments. In: Intelligent robots and systems, 2009. IROS 2009. IEEE/RSJ international conference on, pp 1–6. IEEE
Sethian JA (1999) Fast marching methods. SIAM Rev 41(2):199–235
Spaan MT, Veiga TS, Lima PU (2015) Decision-theoretic planning under uncertainty with information rewards for active cooperative perception. Auton Agents Multi-agent Syst 29(6):1157–1185. https://doi.org/10.1007/s10458-014-9279-8
Stückler J, Holz D, Behnke S (2012) Demonstrating everyday manipulation skills in robocup@home. Robot Autom Mag 19(2):34–42
Talebpour Z, Geetha Viswanathan D, Ventura R, Englebienne G, Martinoli A (2016) Incorporating perception uncertainty in human-aware navigation: a comparative study. In: IEEE international symposium on robot and human interactive communication (RO-MAN)
Veiga TS, Silva M, Ventura R, Lima PU (2019) An hierarchical approach to active semantic mapping using probabilistic logic and information reward POMDP. In: Proc. of 29th international conference on automated planning and scheduling (ICAPS)
Ventura R, Ahmad A (2015) RoboCup 2014: Robot World Cup XVIII, chap. Towards optimal robot navigation in domestic spaces, pp 318–331. LNAI 8992. Springer. https://doi.org/10.1007/978-3-319-18615-3_26
Wasik A, Pereira JN, Ventura R, Lima PU, Martinoli A (2016) Graph-based distributed control for adaptive multi-robot patrolling using local formation reconfiguration. In: IEEE/RSJ international conference on intelligent robots and systems (IROS)
This work was partially supported by ISR/LARSyS Strategic Funding through the FCT project PEst-OE/EEI/LA0009/2013 and by the FCT project HARODE PTDC/EEI-SII/4698/2014.
About this article
Cite this article
Lima, P.U., Azevedo, C., Brzozowska, E. et al. SocRob@Home. Künstl Intell 33, 343–356 (2019). https://doi.org/10.1007/s13218-019-00618-w
- System-level AI for service robots
- Reasoning with uncertain knowledge
- Semantic mapping
- Speech understanding
- Mobile manipulation