Journal of Intelligent & Robotic Systems

, Volume 94, Issue 2, pp 455–470 | Cite as

Robot Navigation in Domestic Environments: Experiences Using RGB-D Sensors in Real Homes

  • Paloma de la PuenteEmail author
  • Markus Bajones
  • Christian Reuther
  • Daniel Wolf
  • David Fischinger
  • Markus Vincze


Future home and service robots will require advanced navigation and interaction capabilities. In particular, domestic environments present open challenges that are hard to identify by conducting controlled tests in home-like settings: there is the need to test and evaluate navigation in the actual homes of users. This paper presents the experiences of operating a mobile robot with manipulation capabilities and an open set of tasks in extensive trials with real users, in their own homes. The main difficulties encountered are the requirement to move safely in cluttered 3D environments, the problems related to navigation in narrow spaces, and the need for an adaptive rather than fixed way to approach the users. We describe our solutions based on RGB-D perception and evaluate the integrated system for navigation in real home environments, pointing out remaining challenges towards more advanced commercial solutions.


Navigation Real home environments RGB-D sensors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



We would like to acknowledge the contribution of all the project partners who made this work possible. Furthermore, Fig. 2. c) and d) were taken by Ammar Qammaz (Institute of Computer Science, FORTH) and Fig. 18. was obtained thanks to Susanne Frennert and Håkan Eftring (Certec, Lund University), who also recorded Online Resource 3. Lara Lammer (TU Wien) and Markus Weninger (AAF) additionally provided significant multimedia material. Astrid Weiss shared great discussions and advice. We are also very grateful to our elderly users. This work has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement No 288146.

Supplementary material

(MP4 16.8 MB)

(MP4 17.8 MB)

(MP4 18.9 MB)


  1. 1.
    Vaussard, F., Fink, J., Bauwens, V., Rétornaz, P., Hamel, D., Dillenbourg, P., Mondada, F.: Lessons learned from robotic vacuum cleaners entering the home ecosystem. Robot. Auton. Syst. 62(3), 376–391 (2014)CrossRefGoogle Scholar
  2. 2.
    Gross, H.M., Mueller, S., Schroeter, C., Volkhardt, M., Scheidig, A., Debes, K., Richter, K., Doering, N.: Robot companion for domestic health assistance Implementation, test and case study under everyday conditions in private apartments. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), 5992–5999 (2015)Google Scholar
  3. 3.
    Ruiz-Sarmiento, J.R., Galindo, C., González-Jiménez, J.: Robot@home, a robotic dataset for semantic mapping of home environments. Int. J. Robot. Res. 36(2), 131–141 (2017)CrossRefGoogle Scholar
  4. 4.
    Xu, Y., Qian, H., Wu, X.: Household service robotics. Intelligent systems. Academic Press, Cambridge (2015)Google Scholar
  5. 5.
    Fischinger, D., Einramhof, P., Papoutsakis, K., Wohlkinger, W., Mayer, P., Panek, P., Hofmann, S., Koertner, T., Weiss, A., Argyros, A., Hobbit, M. Vincze.: A care robot supporting independent living at home: First prototype and lessons learned. Robot. Auton. Syst. 75(Part A), 60–78 (2016)CrossRefGoogle Scholar
  6. 6.
    Coradeschi, S., Cesta, A., Cortellessa, G., Coraci, L., Galindo, C., Gonzalez, J., Karlsson, L., Forsberg, A., Frennert, S., Furfari, F., Loutfi, A., Orlandini, A., Palumbo, F., Pecora, F., von Rump, S., Stimec, A., Ullberg, J., Otslund, B.: Giraffplus: A system for monitoring activities and physiological parameters and promoting social interaction for elderly. In: Human-Computer systems interaction: backgrounds and applications 3, vol. 300, pp. 261–271. Springer International Publishing (2014)Google Scholar
  7. 7.
    Görer, B.R., Salah, A.A., Akın, H.L.: A robotic fitness coach for the elderly. In: Proc. of ambient intelligence: 4th international joint conference, (AmI), pp. 124–139. Springer International Publishing (2013)Google Scholar
  8. 8.
    Bajones, M., Fischinger, D., Weiss, A., de la Puente, P., Wolf, D., Vincze, M., Körtner, T., Weninger, M., Papoutsakis, D.M.K., Qammaz, A., Panteleris, P., Foukarakis, M., Adami, I., Ioannidi, D., Leonidis, A., Antona, M., Argyros, A., Mayer, P., Panek, P., Eftring, H., Frennert, S.: A year of field trials with the Hobbit Robot. Submitted to ACM Transactions on Human-Robot Interaction (2017)Google Scholar
  9. 9.
    Breuer, T., Giorgana Macedo, G.R., Hartanto, R., Hochgeschwender, N., Holz, D., Hegger, F., Jin, Z., Müller, C., Paulus, J., Reckhaus, M., Álvarez Ruiz, J.A., Plöger, P.G., Kraetzschmar, G.K.: Johnny: An autonomous service robot for domestic environments. J. Intell. Robot. Syst. 66(1), 245–272 (2012)CrossRefGoogle Scholar
  10. 10.
    Marginean, M.T.: Distributed system for domestic robot operation using computer vision. PhD thesis (2016)Google Scholar
  11. 11.
    Forlizzi, J., DiSalvo, C.: Service robots in the domestic environment A study of the roomba vacuum in the home. In: Proc. of ACM SIGCHI/SIGART conference on human-robot interaction, pp. 258–265 (2006)Google Scholar
  12. 12.
    Schroeter, C., Mueller, S., Volkhardt, M., Einhorn, E., Huijnen, C., van den Heuvel, H., van Berlo, A., Bley, A., Gross, H.-M.: Realization and user evaluation of a companion robot for people with mild cognitive impairments. In: Proc. of IEEE international conference on robotics and automation (ICRA), pp. 1153–1159 (2013)Google Scholar
  13. 13.
    Zsiga, K., Tóth, A., Pilissy, T., Péter, O., Dénes, Z., Fazekas, G.: Evaluation of a companion robot based on field tests with single older adults in their homes. Assist. Technol. 19, 1–8 (2017). Google Scholar
  14. 14.
    Iocchi, L., Ruiz-del Solar, J., van der Zant, T.: Domestic service robots in the real world. J. Intell. Robot. Syst. 66(1), 183–186 (2012)CrossRefGoogle Scholar
  15. 15.
    de la Puente, P., Bajones, M., Einramhof, P., Wolf, D., Fischinger, D., Vincze, M.: RGB-d sensor setup for multiple tasks of home robots and experimental results. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2587–2594 (2014)Google Scholar
  16. 16.
    Bajones, M., Fischinger, D., Weiss, A., et al.: Hobbit: providing fall detection and prevention for the elderly in the real world. J. Robot. 2018, 20pp (2018). Google Scholar
  17. 17.
    de la Puente, P., Bajones, M., Reuther, C., Fischinger, D., Wolf, D., Vincze, M.: Experiences with RGB-d based navigation in real home robotic trials. In: OAGM-ARW joint workshop 2016 on computer vision and robotics, pp. 153–165 (2016)Google Scholar
  18. 18.
    Gates, B.: A robot in every home. Scientific American Magazine 296(1), 58–65 (2007)CrossRefGoogle Scholar
  19. 19.
    Fortunati, L.: Moving robots from industrial sectors to domestic spheres: a foreword. In: Esposito, A., Jain, L. (eds.) Toward Robotic Socially Believable Behaving Systems - Volume II. Intelligent Systems Reference Library, vol. 106. Springer, Cham (2016)Google Scholar
  20. 20.
    Bogue, R.: Domestic robots: Has their time finally come?. Industrial Robot: An International Journal 44(2), 129–136 (2017)CrossRefGoogle Scholar
  21. 21.
    Kostavelis, I., Giakoumis, D., Malassiotis, S., Tzovaras, D.: Human aware robot navigation in semantically annotated domestic environments. In: Proc. of universal access in human-computer interaction, pp. 414–423, Springer International Publishing (2016)Google Scholar
  22. 22.
    Kirsch, A.: Heuristic decision-making for human-aware navigation in domestic environments. In: Proc. of global conference on artificial intelligence (GCAI), vol. 41 of EPiC Series in Computing, pp. 200–213 (2016)Google Scholar
  23. 23.
    Mount, J.: M Milford. 2d visual place recognition for domestic service robots at night. In: Proc. of IEEE international conference on robotics and automation (ICRA),pp. 4822–4829 (2016)Google Scholar
  24. 24.
    Ruiz-Sarmiento, J.R., Galindo, C., Gonzalez-Jimenez, J.: Joint categorization of objects and rooms for mobile robots. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2523–2528 (2015)Google Scholar
  25. 25.
    Uršič, P., Leonardis, A., Skočaj, D., Kristan, M.: Learning part-based spatial models for laser-vision-based room categorization. Int. J. Robot. Res. 36(4), 379–402 (2015)Google Scholar
  26. 26.
    Sprute, D., Rasch, R., Tönnies, K., König, M.: A framework for interactive teaching of virtual borders to mobile robots. In: Proc. of IEEE/RSJ international symposium on robot and human interactive communication (RO-MAN), pp. 1175–1181 (2017)Google Scholar
  27. 27.
    Holz, D., Kraetzschmar, G.K., Rome, E.: Robocup 2009. Chapter Robust and Computationally Efficient Navigation in Domestic Environments, pp 104–115. Springer, Berlin (2010)Google Scholar
  28. 28.
    Soroka, A.J., Qiu, R., Noyvirt, A., Ji, Z.: Challenges for service robots operating in non-industrial environments. In: Proc. of IEEE international conference on industrial informatics (INDIN), pp. 1152–1157 (2012)Google Scholar
  29. 29.
    Prassler, E., Bischoff, R., Burgard, W., Haschke, R., Hägele, M., Lawitzky, G., Nebel, B., Plöger, P., Reiser, U., Zöllner, M.: Towards service robots for everyday environments. recent advances in designing service robots for complex tasks in everyday environments. Springer Tracts in Advanced Robotics, Springer, Berlin (2012)CrossRefGoogle Scholar
  30. 30.
    de Graaf, M.M.A., Ben Allouch, S., van Dijk, J.A.G.M.: Long-term evaluation of a social robot in real homes. Interact. Stud. 17(3), 462–491 (2016)CrossRefGoogle Scholar
  31. 31.
    Mast, M., Burmester, M., Berner, E., Facal, D., Pigini, L., Blasi, L.: Semi-autonomous teleoperated learning in-home service robots for elderly care A qualitative study on needs and perceptions of elderly people, family caregivers, and professional caregivers. In: Proc. of the 20th international conference on robotics and mechatronics, pp. 1–6 (2010)Google Scholar
  32. 32.
    Papadopoulos, F., Küster, D., Corrigan, L.J., Kappas, A., Castellano, G.: Do relative positions and proxemics affect the engagement in a human-robot collaborative scenario?. Interact. Stud. 17(3), 321–347 (2016)CrossRefGoogle Scholar
  33. 33.
    Mead, R., Matarić, M.J.: Perceptual models of human-robot proxemics. In: Hsieh, M., Khatib, O., Kumar, V. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 109. Springer, Cham (2016)Google Scholar
  34. 34.
    Rossi, S., Ferland, F., Tapus, A.: User profiling and behavioral adaptation for HRI: a survey. Pattern Recogn. Lett. 99(Supplement C), 3–12 (2017)CrossRefGoogle Scholar
  35. 35.
    Koay, K.L., Syrdal, D.S., Ashgari-Oskoei, M., Walters, M.L., Dautenhahn, K.: Social roles and baseline proxemic preferences for a domestic service robot. Int. J. Soc. Robot. 6(4), 469–488 (2014)CrossRefGoogle Scholar
  36. 36.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng. A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)Google Scholar
  37. 37.
    Einhorn, E., Langner, T., Stricker, R., Martin, C., Gross, H.: Mira - middleware for robotic applications. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2591–2598 (2012)Google Scholar
  38. 38.
    Bohren, J., Cousins, S.: The smach high-level executive [ros news]. IEEE Robot. Autom. Mag. 17(4), 18–20 (2010)CrossRefGoogle Scholar
  39. 39.
    Hawes, N., Burbridge, C., Jovan, F., Kunze, L., Lacerda, B., Mudrová, L., Young, J., Wyatt, J., Hebesberger, D., Körtner, T., Ambrus, R., Bore, N., Folkesson, J., Jensfelt, P., Beyer, L., Hermans, A., Leibe, B., Aldoma, A., Fäulhammer, T., Zillich, M., Vincze, M., Chinellato, E., Al-Omari, M., Duckworth, P., Gatsoulis, Y., Hogg, D., Cohn, A., Dondrup, C., Pulido, J., Krajník, T., Santos, J., Duckett, T., Hanheide, M.: The STRANDS, project: Long-term autonomy in everyday environments. IEEE Robot. Autom. Mag. 24(3), 146–156 (2016)CrossRefGoogle Scholar
  40. 40.
    Hendrich, N., Bistry, H., Jianwei, Z.: PEIS, MIRA, and ROS: Three frameworks, one service robot -A tale of integration. In: 2014 IEEE international conference on robotics and biomimetics (ROBIO), pp. 1749–1756 (2014)Google Scholar
  41. 41.
    Grisetti, G., Stachniss, C., Burgard, W.: Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Trans. Robot. 23(1), 34–46 (2007)CrossRefGoogle Scholar
  42. 42.
    Trevor, A.J.B., Cosgun, A., Kumar, J., Christensen, H.I.: Interactive map labeling for service robots. In: IROS Workshop on Active Semantic Perception (2012)Google Scholar
  43. 43.
    González-Jiménez, J., Galindo, C., Melendez-Fernandez, F., Ruiz-Sarmiento, J.R.: Building and exploiting maps in a telepresence robotic application. In: 10th international conference on informatics in control, automation and robotics (ICINCO), pp. 322–328 (2013)Google Scholar
  44. 44.
    Panteleris, P., Argyros, A.: Vision-based SLAM And moving objects tracking for the perceptual support of a smart walker platform. In: Computer Vision - ECCV 2014 Workshops, vol. 8927 of Lecture Notes in Computer Science, pp. 407–423, Springer International Publishing (2015)Google Scholar
  45. 45.
    Qammaz, A., Kyriazis, N., Argyros, A.A.: Boosting the performance of model-based 3d tracking by employing low level motion cues. In: Proc. of the British machine vision conference (BMVC), pp. 144.1–144.11 (2015)Google Scholar
  46. 46.
    Fischinger, D., Weiss, A., Vincze, M.: Learning grasps with topographic features. Int. J. Robot. Res. 34(9), 1167–1194 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.Centre for Automation and Robotics, UPM-CSICUniversidad Politécnica de MadridMadridSpain
  2. 2.Vienna University of Technology (TU Wien)ViennaAustria
  3. 3.MetralabsIlmenauGermany
  4. 4.Aeolus Robotics, Inc.ViennaAustria

Personalised recommendations