Advertisement

A Robotic Home Assistant with Memory Aid Functionality

  • Iris WieserEmail author
  • Sibel Toprak
  • Andreas Grenzing
  • Tobias Hinz
  • Sayantan Auddy
  • Ethem Can Karaoğuz
  • Abhilash Chandran
  • Melanie Remmels
  • Ahmed El Shinawi
  • Josip Josifovski
  • Leena Chennuru Vankadara
  • Faiz Ul Wahab
  • Alireza M. Bahnemiri
  • Debasish Sahu
  • Stefan Heinrich
  • Nicolás Navarro-Guerrero
  • Erik Strahl
  • Johannes Twiefel
  • Stefan Wermter
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9904)

Abstract

We present the robotic system IRMA (Interactive Robotic Memory Aid) that assists humans in their search for misplaced belongings within a natural home-like environment. Our stand-alone system integrates state-of-the-art approaches in a novel manner to achieve a seamless and intuitive human-robot interaction. IRMA directs its gaze toward the speaker and understands the person’s verbal instructions independent of specific grammatical constructions. It determines the positions of relevant objects and navigates collision-free within the environment. In addition, IRMA produces natural language descriptions for the objects’ positions by using furniture as reference points. To evaluate IRMA’s usefulness, a user study with 20 participants has been conducted. IRMA achieves an overall user satisfaction score of 4.05 and a perceived accuracy rating of 4.15 on a scale from 1–5 with 5 being the best.

Keywords

Robotic home assistant Human-robot interaction Social robotics Memory service system Speech recognition Natural language understanding Object detection Person detection 

Notes

Acknowledgments

The authors gratefully acknowledge partial support from the German Research Foundation DFG under project CML (TRR 169), the European Union under project SECURE (No 642667), and the Hamburg Landesforschungsförderungsprojekt.

References

  1. 1.
    Barla, A., Odone, F., Verri, A.: Histogram intersection kernel for image classification. In: International Conference on Image Processing (ICIP), vol. 3, pp. 513–516. IEEE (2003)Google Scholar
  2. 2.
    Bartneck, C., Kulić, D., Croft, E., Zoghbi, S.: Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int. J. Social Robot. 1(1), 71–81 (2008)CrossRefGoogle Scholar
  3. 3.
    Beer, J.M., Smarr, C.A., Chen, T.L., Prakash, A., Mitzner, T.L., Kemp, C.C., Rogers, W.A.: The domesticated robot: design guidelines for assisting older adults to age in place. In: Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2012, pp. 335–342. ACM/IEEE (2012)Google Scholar
  4. 4.
    Bohren, J., Cousins, S.: The SMACH high-level executive. IEEE Robot. Autom. Mag. 17(4), 18–20 (2010)CrossRefGoogle Scholar
  5. 5.
    Bohren, J., Rusu, R.B., Jones, E.G., Marder-Eppstein, E., Pantofaru, C., Wise, M., Mösenlechner, L., Meeussen, W., Holzer, S.: Towards autonomous robotic butlers: lessons learned with the PR2. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5568–5575. IEEE (2011)Google Scholar
  6. 6.
    Brooke, J.: SUS - a quick and dirty usability scale. In: Usability Evaluation in Industry, pp. 189–194. Taylor & Francis (1996)Google Scholar
  7. 7.
    Brooke, J.: SUS: a retrospective. J. Usability Stud. 8(2), 29–40 (2013)Google Scholar
  8. 8.
    Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell. 17(8), 790–799 (1995)CrossRefGoogle Scholar
  9. 9.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  10. 10.
    Deyle, T., Reynolds, M.S., Kemp, C.C.: Finding and navigating to household objects with UHF RFID tags by optimizing RF signal strength. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2579–2586, September 2014Google Scholar
  11. 11.
    Fasola, J., Mataric, M.: A socially assistive robot exercise coach for the elderly. J. Hum.-Robot Interact. 2(2), 3–32 (2013)CrossRefGoogle Scholar
  12. 12.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 524–531. IEEE (2005)Google Scholar
  13. 13.
    Foster, M.E., Gaschler, A., Giuliani, M., Isard, A., Pateraki, M., Petrick, R.P.: Two people walk into a bar: dynamic multi-party social interaction with a robot agent. In: ACM International Conference on Multimodal Interaction, pp. 3–10. ICMI, ACM (2012)Google Scholar
  14. 14.
    Fox, D.: Adapting the sample size in particle filters through KLD-sampling. Int. J. Robot. Res. 22(12), 985–1003 (2003)CrossRefGoogle Scholar
  15. 15.
    Graf, B., Reiser, U., Hägele, M., Mauz, K., Klein, P.: Robotic home assistant care-O-bot 3 - product vision and innovation platform. In: IEEE Workshop on Advanced Robotics and its Social Impacts, pp. 139–144. IEEE (2009)Google Scholar
  16. 16.
    Guo, B., Imai, M.: Home-explorer: search, localize and manage the physical artifacts indoors. In: International Conference on Advanced Information Networking and Applications (AINA), pp. 378–385. IEEE (2007)Google Scholar
  17. 17.
    Hinaut, X., Petit, M., Pointeau, G., Dominey, P.F.: Exploring the acquisition and production of grammatical constructions through human-robot interaction with echo state networks. Front. Neurorobotics 8, 16 (2014)CrossRefGoogle Scholar
  18. 18.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 2169–2178. IEEE (2006)Google Scholar
  19. 19.
    Levenshtein, V.: Binary codes capable of correcting deletions, insertions and reversals. Sov. Phys. Dokl. 10, 707 (1966)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Likert, R.: A Technique for the Measurement of Attitudes. Archives of Psychology (1932)Google Scholar
  21. 21.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    Murray, J., Wermter, S., Erwin, H.: Auditory robotic tracking of sound sources using hybrid cross-correlation and recurrent networks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3554–3559. IEEE (2005)Google Scholar
  23. 23.
    Palmer, M., Gildea, D., Xue, N.: Semantic role labeling. Synth. Lect. Hum. Lang. Technol. 3(1), 1–103 (2010)CrossRefGoogle Scholar
  24. 24.
    Parisi, G.I., Bauer, J., Strahl, E., Wermter, S.: A multi-modal approach for assistive humanoid robots. In: Workshop on Multimodal and Semantics for Robotics Systems (MuSRobS), vol. 1540, pp. 10–15. CEUR Workshop Proceedings (2015)Google Scholar
  25. 25.
    Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 6 (2009)Google Scholar
  26. 26.
    Rosenthal-von der Pütten, A.M., Krämer, N.C.: How design characteristics of robots determine evaluation and uncanny valley related responses. Comput. Hum. Behav. 36, 422–439 (2014)CrossRefGoogle Scholar
  27. 27.
    Twiefel, J., Baumann, T., Heinrich, S., Wermter, S.: Improving domain-independent cloud-based speech recognition with domain-dependent phonetic post-processing. In: AAAI Conference on Artificial Intelligence, vol. 28, pp. 1529–1535. AAAI Press (2014)Google Scholar
  28. 28.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 511–518. IEEE (2001)Google Scholar
  29. 29.
    Wieser, I., Toprak, S., Grenzing, A., Hinz, T., Auddy, S., Karaoğuz, E.C., Chandran, A., Remmels, M., El Shinawi, A., Josifovski, J., Vankadara, L.C., Ul Wahab, F., Bahnemiri, A.M., Sahu, D., Heinrich, S., Navarro-Guerrero, N., Strahl, E., Twiefel, J., Wermter, S.: A Robotic Home Assistant with Memory Aid Functionality, video. IEEE RO-MAN 2016. https://www.informatik.uni-hamburg.de/wtm/videos/VideoSubmission_UniHamburgWTM_RO-MAN2016.mp.4. Accepted May 2016
  30. 30.
    Wieser, I., Toprak, S., Grenzing, A., Hinz, T., Auddy, S., Karaoğuz, E.C., Chandran, A., Remmels, M., El Shinawi, A., Josifovski, J., Vankadara, L.C., Ul Wahab, F., Bahnemiri, A.M., Sahu, D., Heinrich, S., Navarro-Guerrero, N., Strahl, E., Twiefel, J., Wermter, S.: Dataset for “A Robotic Home Assistant with Memory Aid Functionality”, May 2016. https://figshare.com/s/d949d3410df8db468f77

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Iris Wieser
    • 1
    Email author
  • Sibel Toprak
    • 1
  • Andreas Grenzing
    • 1
  • Tobias Hinz
    • 1
  • Sayantan Auddy
    • 1
  • Ethem Can Karaoğuz
    • 1
  • Abhilash Chandran
    • 1
  • Melanie Remmels
    • 1
  • Ahmed El Shinawi
    • 1
  • Josip Josifovski
    • 1
  • Leena Chennuru Vankadara
    • 1
  • Faiz Ul Wahab
    • 1
  • Alireza M. Bahnemiri
    • 1
  • Debasish Sahu
    • 1
  • Stefan Heinrich
    • 1
  • Nicolás Navarro-Guerrero
    • 1
  • Erik Strahl
    • 1
  • Johannes Twiefel
    • 1
  • Stefan Wermter
    • 1
  1. 1.Department of Informatics, Knowledge Technology (WTM)University of HamburgHamburgGermany

Personalised recommendations