Experimental Evaluation of a Multi-modal User Interface for a Robotic Service

  • Alessandro Di NuovoEmail author
  • Ning Wang
  • Frank Broz
  • Tony Belpaeme
  • Ray  Jones
  • Angelo Cangelosi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9716)


This paper reports the experimental evaluation of a Multi-Modal User Interface (MMUI) designed to enhance the user experience in terms of service usability and to increase acceptability of assistive robot systems by elderly users. The MMUI system offers users two main modalities to send commands: they are a GUI, usually running on the tablet attached to the robot, and a SUI, with a wearable microphone on the user. The study involved fifteen participants, aged between 70 and 89 years old, who were invited to interact with a robotic platform customized for providing every-day care and services to the elderly. The experimental task for the participants was to order a meal from three different menus using any interaction modality they liked. Quantitative and qualitative data analyses demonstrate a positive evaluation by users and show that the multi-modal means of interaction can help to make elderly-robot interaction more flexible and natural.


Human-robot interaction and interfaces Service robotics Socially Assistive Robotics 


  1. 1.
    Robot-Era project: Implementation and integration of advanced robotic systems and intelligent environments in real scenarios for the ageing population, FP7 - ICT - Challenge 5: ICT for Health, Ageing Well, Inclusion and Governance. Grant agreement number 288899.
  2. 2.
    Al-Razgan, M.S., Al-Khalifa, H.S., Al-Shahrani, M.D., AlAjmi, H.H.: Touch-Based mobile phone interface guidelines and design recommendations for elderly people: a survey of the literature. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds.) ICONIP 2012, Part IV. LNCS, vol. 7666, pp. 568–574. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  3. 3.
    Arch, A., Abou-Zahra, S., Henry, S.L.: Older users online: WAI guidelines address the web experiences of older users. User Experience Mag. 8(1), 18–19 (2009)Google Scholar
  4. 4.
    Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 24(6), 574–594 (2008)CrossRefGoogle Scholar
  5. 5.
    Bemelmans, R., Gelderblom, G.J., Jonker, P., de Witte, L.: Socially assistive robots in elderly care: a systematic review into effects and effectiveness. J. Am. Med. Directors Assoc. 13(2), 114–120 (2012)CrossRefGoogle Scholar
  6. 6.
    Bohus, D., Rudnicky, A.I.: The ravenclaw dialog management framework: architecture and systems. Comput. Speech Lang. 23(3), 332–361 (2009)CrossRefGoogle Scholar
  7. 7.
    Brooke, J.: Sus-a quick and dirty usability scale. Usability Eval. Ind. 189, 194 (1996)Google Scholar
  8. 8.
    Broz, F., Di Nuovo, A., Belpaeme, T., Cangelosi, A.: Talking about task progress: towards integrating task planning and dialog for assistive robotic services. Paladyn, J. Behav. Robot. 6(1), 111–118 (2015)Google Scholar
  9. 9.
    Broz, F., Nuovo, A.D., Belpaeme, T., Cangelosi, A.: Multimodal robot feedback for eldercare. In: Workshop on Robot Feedback in Human-Robot Interaction: How to Make a Robot Readable for a Human Interaction Partner at Ro-MAN 2012 (2012)Google Scholar
  10. 10.
    Casiddu, N., Cavallo, F., Divano, A., Mannari, I., Micheli, E., Porfirione, C., Zallio, M., Aquilano, M., Dario, P.: Robot interface design of domestic and condominium robot for ageing population. In: ForITAAL, October 2013Google Scholar
  11. 11.
    Cavallo, F., Limosani, R., Manzi, A., Bonaccorsi, M., Esposito, R., Rocco, M., Pecora, F., Teti, G., Saffiotti, A., Dario, P.: Development of a socially believable multi-robot solution from town to home. Cogn. Comput. 6(4), 954–967 (2014)CrossRefGoogle Scholar
  12. 12.
    Consortium, W.W.W.: Web accessibility for older users.
  13. 13.
    Di Nuovo, A., Broz, F., Belpaeme, T., Cangelosi, A., Cavallo, F., Dario, P., Esposito, R.: A web based multi-modal interface for elderly users of the robot-era multi-robot services. In: Proceedings of IEEE Conference on System, Man and Cybernetics, pp. 1–6. IEEE, October 2014Google Scholar
  14. 14.
    Dumas, B., Lalanne, D., Oviatt, S.: Multimodal interfaces: a survey of principles, models and frameworks. In: Lalanne, D., Kohlas, J. (eds.) Human Machine Interaction. LNCS, vol. 5440, pp. 3–26. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Jaimes, A., Sebe, N.: Multimodal human-computer interaction: a survey. Comput. Vis. Image Underst. 108(1), 116–134 (2007)CrossRefGoogle Scholar
  16. 16.
    Jian, C., Shi, H., Schafmeister, F., Rachuy, C., Sasse, N., Schmidt, H., Hoemberg, V., von Steinbüchel, N.: Touch and speech: multimodal interaction for elderly persons. In: Gabriel, J., Schier, J., Van Huffel, S., Conchon, E., Correia, C., Fred, A., Gamboa, H. (eds.) BIOSTEC 2012. CCIS, vol. 357, pp. 385–400. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  17. 17.
    Kline, P.: The Handbook of Psychological Testing, 2nd edn. Routledge, London (2000)Google Scholar
  18. 18.
    Kristoffersson, A., Coradeschi, S., Loutfi, A.: A review of mobile robotic telepresence. Adv. Hum. Comput. Interact. 2013, 3 (2013)CrossRefGoogle Scholar
  19. 19.
    Mayer, P., Beck, C., Panek, P.: Examples of multimodal user interfaces for socially assistive robots in ambient assisted living environments. In: 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom), pp. 401–406. IEEE (2012)Google Scholar
  20. 20.
    Panek, P., Edelmayer, G., Mayer, P., Beck, C., Rauhala, M.: User acceptance of a mobile LED projector on a socially assistive robot. In: Wichert, R., Eberhardt, B. (eds.) Ambient Assisted Living. ATSC, vol. 2, pp. 77–92. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  21. 21.
    Petzold, M., Barbabella, F., Bobeth, J., Kern, D., Mayer, C., Morandell, M.: Towards an ambient assisted living user interaction taxonomy. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 49–54. ACM (2013)Google Scholar
  22. 22.
    Tapus, A., Maja, M., Scassellatti, B.: The grand challenges in socially assistive robotics. IEEE Robot. Autom. Mag. 14(1), 1–7 (2007)CrossRefGoogle Scholar
  23. 23.
    Wang, N., Broz, F., Di Nuovo, A., Belpaeme, T., Cangelosi, A.: A user-centric design of service robots speech interface for the elderly. In: Proceedings of the International Conference on Non-Linear Speech Processing, NOLISP 2015 (2015)Google Scholar
  24. 24.
    Werner, F., Werner, K., Oberzaucher, J.: Tablets for seniors – an evaluation of a current model (iPad). In: Wichert, R., Eberhardt, B. (eds.) Ambient Assisted Living. ATSC, vol. 2, pp. 177–184. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  25. 25.
    Zaphiris, P., Kurniawan, S., Ghiawadwala, M.: A systematic approach to the development of research-based web design guidelines for older people. Univ. Access Inf. Soc. 6(1), 135–136 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Alessandro Di Nuovo
    • 1
    Email author
  • Ning Wang
    • 2
  • Frank Broz
    • 3
  • Tony Belpaeme
    • 2
  • Ray  Jones
    • 4
  • Angelo Cangelosi
    • 2
  1. 1.Department of ComputingSheffield Hallam UniversitySheffieldUK
  2. 2.Centre for Robotics and Neural SystemsPlymouth UniversityPlymouthUK
  3. 3.School of Mathematical and Computer SciencesHeriot-Watt UniversityEdinburghUK
  4. 4.School of Nursing and MidwiferyPlymouth UniversityPlymouthUK

Personalised recommendations