Skip to main content
SpringerLink
Log in

Current Challenges for UX Evaluation of Human-Robot Interaction

  • Conference paper
  • First Online:
Advances in Ergonomics of Manufacturing: Managing the Enterprise of the Future

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 490))

Abstract

The development of socially interactive robots is expected to have an increasing importance in everyday life for a growing number of people. For social robots to provide long-term added value to people’s lives, it is of major importance to stress the need for developing a positive user experience (UX). The human-centered view emphasizes various aspects including acceptance, usability, and credibility, as they emerge in the interaction between humans and robots. In current human-robot interaction (HRI) research, UX is reckoned to be important, but is often taken for granted. However, a positive user experience does not appear by itself but has to be systematically designed and evaluated. In this paper, we focus on the role and relevance of UX in HRI and present three challenges related to the evaluation of UX in interaction with robots, addressing the need for interdisciplinary research in order to achieve long-term success of socially interactive robots.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Briefly stated, formative evaluation is typically performed during the early development of a system, while summative evaluation is performed at the end of a design process.

References

  1. Dautenhahn, K.: Socially intelligent robots: dimensions of human-robot interaction. Phil. Trans. R. Soc. B 362(1480), 679–704 (2007)

    Article  Google Scholar 

  2. Weiss, A., Bernhaupt, R., Tscheligi, M.: The USUS evaluation framework for user-centered HRI. In: Dautenhahn, K., Saunders, J. (eds.) New Frontiers in Human–Robot Interaction, pp. 89–110). John Benjamins Publishing Co, Amsterdam (2011)

    Google Scholar 

  3. Thrun, S.: Toward a framework for human-robot interaction. Hum. Comput. Interact. 19(1–2), 9–24 (2004)

    Article  Google Scholar 

  4. Bartneck, C., Kulić, D., Croft, E., Zoghbi, S.: Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int. J. Soc. Robot. 1(1), 71–81 (2009)

    Article  Google Scholar 

  5. Dautenhahn, K., Saunders, J. (eds.): New Frontiers in Human-Robot Interaction. Advances in Interaction Studies. J. Benjamins Pub, Amsterdam (2011)

    Google Scholar 

  6. Dautenhahn, K.: Methodology and themes of human-robot interaction: a growing research field. Int. J. Adv. Robot. Syst. 4(1), 103–108 (2007)

    Google Scholar 

  7. Dautenhahn, K.: Human-robot interaction. In: Soegaard, M., Dam, R.F. (eds.) The Encyclopedia of Human-Computer Interaction. The Interaction Design Foundation, Aarhus, Denmark (2013). Available at: http://www.interaction-design.org/encyclopedia/human-robot_interaction.html

  8. Brady, K., Tarn, T., Xi, N., Love, L., Lloyd, P., Davis, H., Burks, B.: Remote systems for waste retrieval from the Oak Ridge national laboratory gunite tanks. Int. J. Robot. Res. 17, 450–460 (1998)

    Article  Google Scholar 

  9. Thrun, S., Hähnel, D., Ferguson, D., Montemerlo, M., Triebel, R., Burgard, W., Bakery, C., Omohundroy, Z., Thayery, S., Whittakery, W.: A system for volumetric robotic mapping of abandoned mines. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). IEEE, Piscataway, NJ (2003)

    Google Scholar 

  10. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  MATH  Google Scholar 

  11. Hartson, H.R., Pyla, P.S.: The UX Book: process and guidelines for ensuring a quality user experience. Elsevier, Amsterdam (2012)

    Google Scholar 

  12. Hassenzahl, M.: User experience and experience design. In: Soegaard, M., Dam, R.F. (eds.) The Encyclopedia of Human-Computer Interaction, 2nd edn. The Interaction Design Foundation, Aarhus, Denmark (2013). Available at: https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/user-experience-and-experience-design

  13. Hassenzahl, M., Tractinsky, N.: User experience—a research agenda. Behav. Inform. Technol. 25(2), 91–97 (2006)

    Article  Google Scholar 

  14. ISO 9241-210: Human-Centred Design for Interactive Systems, International Organization for Standardization, Geneva, Switzerland (2010)

    Google Scholar 

  15. ISO 9241-11: Guidance on Usability, International Organization for Standardization, Geneva, Switzerland (1998)

    Google Scholar 

  16. Hassenzahl, M., Roto, V.: Being and doing: a perspective on user experience and its measurement. Interfaces 72, 10–12 (2007)

    Google Scholar 

  17. Anderson, J., McRee, J., Wilson, R., the Effective UI Team: Effective UI. O’Reilly, Sebastopol, CA (2010)

    Google Scholar 

  18. de Graaf, M.M.A., Allouch, S.B.: Exploring influencing variables for the acceptance of social robots. Robot. Auton. Syst. 6(12), 1476–1486 (2013)

    Article  Google Scholar 

  19. Alenljung, B., Lindblom, J.: User experience of socially interactive robots: its role and relevance. In: Vallverdú, J. (ed.) Handbook of Research on Synthesizing Human Emotion in Intelligent Systems and Robotics, pp. 352–364. IGI Global, Hershey (2015)

    Google Scholar 

  20. Kim, Y., Mutlu, B.: How social distance shapes human–robot interaction. Int. J. Hum. Comput. Stud. 72, 783–795 (2014)

    Article  Google Scholar 

  21. Jokinen, K., Wilcock, G.: Modelling user experience in human-robot interactions. In: Böck, R., Bonin, F., Campbell, N. Poppe, R. (eds.) Second International Workshop, MA3HMI 2014, pp. 45–56 (2015)

    Google Scholar 

  22. Moon, A.J., Zheng, M., Troniak, D.M., Blumer, B., Gleeson, B., MacLean, K., Pan, M.K.X.J., Croft, E.A.: Meet me where I’m gazing: how shared attention gaze affects human-robot handover timing. Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction, pp. 334–341. ACM, Bielefeld (2014)

    Chapter  Google Scholar 

  23. Juarez, A., Bartneck, C., Feijs, L.: Using semantic technologies to describe robotic embodiments. In: Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 425–432. Lausanne, Switzerland (2011)

    Google Scholar 

  24. Callejas, Z., Griol, D., López-Cózar, R.: A framework for the assessment of synthetic personalities according to user perception. Int. J. Hum. Comput. Stud. 72(7), 567–583 (2014)

    Article  Google Scholar 

  25. Karreman, D., Ludden, G., Evers, V.: Visiting cultural heritage with a tour guide robot: a user evaluation study in-the-wild. In: Tapus, A., André, E., Martin, J.C., Ferland, F., Ammi, M. (eds.) Social Robotics. LNCS, vol. 9388, pp. 317–326. Springer International Publishing, Switzerland (2015)

    Google Scholar 

  26. dos Santos, T.F., de Castro, D.G., Masiero, A.A., Aquino Junior, P.T.: Behavioral persona for human-robot interaction: a study based on pet robot. Proceedings of the 16th International Conference. HCI International, pp. 687–696. Heraklion, Crete (2014)

    Google Scholar 

  27. Xu, Q., Ng, J., Cheong, Y.L., Tan, O., Wong, J.B., Tay, T.C., Park, T. (2012). The role of social context in human-robot interaction. Southeast Asian Network of Ergonomics Societies Conference (SEANES). Langkawi, Kedah

    Google Scholar 

  28. Beňuš, Š.: Social aspects of entrainment in spoken interaction. Cog. Comp. 6(4), 802–813 (2014)

    Article  Google Scholar 

  29. Cohen, I., Looije, R., Neerincx, M.A.: Child’s perception of robot’s emotions: effects of platform, context and experience. Int. J. Soc. Robot. 6, 507–518 (2014)

    Article  Google Scholar 

  30. Holthaus, P., Wachsmuth, S.: The receptionist robot. In: Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction. ACM, pp. 329. Bielefeld, Germany (2014)

    Google Scholar 

  31. Anzalone, S.M., Boucenna, S., Ivaldi, S., Chetouani, M.: Evaluating the engagement with social robots. Int. J. Soc. Robot. 7(4), 465–478 (2015)

    Article  Google Scholar 

  32. Bevilacqua, R., Felici, E., Marcellini, F., Glende, S., Klemcke, S., Conrad, I., Esposito, R., Cavallo, F., Dario, P.: Robot-era project: preliminary results on the system usability. In: Marcus, A. (ed.) Design, User Experience, and Usability: Interactive Experience Design: 4th International Conference, DUXU 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, Proceedings, Part III, pp. 553-561, Springer International Publishing, Cham, 2–7 Aug 2015

    Google Scholar 

  33. Cesta, A., Cortellessa, G., Orlandini, A., Tiberio, L.: Long-term evaluation of a telepresence robot for the elderly: methodology and ecological case study. Int. J. Soc. Robot. 1–20 (2016)

    Google Scholar 

  34. Johnson, D.O., Cuijpers, R.H., Juola, J.F., Torta, E., Simonov, M., Frisiello, A., Bazzani, M., Yan, W., Weber, C., Wermter, S., Meins, N., Oberzaucher, J., Panek, P., Edelmayer, G., Mayer, P., Beck, C.: Socially assistive robots: a comprehensive approach to extending independent living. Int. J. Soc. Robot. 6, 195–211 (2014)

    Article  Google Scholar 

  35. Lohse, M., Siepmann, F., Wachsmuth, S.: A modeling framework for user-driven iterative design of autonomous systems. Int. J. Soc. Robot. 6, 121–139 (2014)

    Article  Google Scholar 

  36. Goodman, E., Kuniavsky, M., Moed, A.: Observing the user experience: a practitioner’s guide to user research. Morgan Kaufmann Publishers, San Francisco (2013)

    Google Scholar 

  37. Young, J.E., Sung, J., Voida, A., Sharlin, E., Igarashi, T., Christensen, H.I., Grinter, R.E.: Evaluating human-robot interaction—focusing on the holistic interaction experience. Int. J. Soc. Robot. 3(1), 53–67 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

The authors wish to thank Beatrice Alenljung and David Vernon for input and valuable comments. This work is financially supported by the Knowledge Foundation, SIDUS project AIR (Action and intention recognition in human interaction with autonomous systems).

Author information

Authors and Affiliations

  1. University of Skövde, Box 408 541 28, Skövde, Sweden

    Jessica Lindblom & Rebecca Andreasson

Authors
  1. Jessica Lindblom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rebecca Andreasson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jessica Lindblom .

Editor information

Editors and Affiliations

  1. RWTH Aachen University , Aachen, Germany

    Christopher Schlick

  2. Poznan University of Technology , Poznan, Poland

    Stefan Trzcieliński

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Lindblom, J., Andreasson, R. (2016). Current Challenges for UX Evaluation of Human-Robot Interaction. In: Schlick, C., Trzcieliński, S. (eds) Advances in Ergonomics of Manufacturing: Managing the Enterprise of the Future. Advances in Intelligent Systems and Computing, vol 490. Springer, Cham. https://doi.org/10.1007/978-3-319-41697-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41697-7_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41696-0

  • Online ISBN: 978-3-319-41697-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation