Skip to main content
Log in

A conceptual framework to evaluate human-robot collaboration

  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript


Human-Robot Collaboration (HRC) is a form of direct interaction between humans and robots. The objective of this type of interaction is to perform a task by combining the skills of both humans and robots. HRC is characterized by several aspects, related both to robots and humans. Many works have focused on the study of specific aspects related to HRC, e.g., safety, task organization. However, a major issue is to find a general framework to evaluate the collaboration between humans and robots considering all the aspects of the interaction. The goals of this paper are the following: (i) highlighting the different latent dimensions that characterize the HRC problem and (ii) constructing a conceptual framework to evaluate and compare different HRC configuration profiles. The description of the methodology is supported by some practical examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others


  1. ABB: (2020).

  2. Ahmad M, Mubin O, Orlando J (2017) A systematic review of adaptivity in human-robot interaction. Multimodal Technologies and Interaction 1(3):14.

    Article  Google Scholar 

  3. Aliev K, Antonelli D (2019) Analysis of cooperative industrial task execution by mobile and manipulator robots. In: Trojanowska J, Ciszak O, Machado JM, Pavlenko I (eds) Advances in manufacturing II. Springer International Publishing, Cham, pp 248–260

  4. Arai T, Kato R, Fujita M (2010) Assessment of operator stress induced by robot collaboration in assembly. CIRP Ann 59(1):5–8

    Article  Google Scholar 

  5. Argall B, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robot Auton Syst 57(5):469–483.

    Article  Google Scholar 

  6. Astrom KJ, Wittenmark B (1994) Adaptive control, 2nd edn. Addison-Wesley Longman Publishing Co., Inc., Boston

    MATH  Google Scholar 

  7. Bandari YK, Williams SW, Ding J, Martina F (2015) Additive manufacture of large structures: robotic or cnc systems?. In: Proceedings of the 26th international solid freeform fabrication symposium, Austin, TX, USA, pp 12–14

  8. Bangor A, Kortum PT, Miller JT (2008) An empirical evaluation of the system usability scale. International Journal of Human–Computer Interaction 24(6):574–594.

    Article  Google Scholar 

  9. Bauer A, Wollherr D, Buss M (2008) Human–robot collaboration: a survey. International Journal of Humanoid Robotics 05:47–66.

    Article  Google Scholar 

  10. Beer JM, Fisk AD, Rogers WA (2014) Toward a framework for levels of robot autonomy in human-robot interaction. Journal of Human-Robot Interaction 3(2):74–99.

    Article  Google Scholar 

  11. Biggs G, MacDonald B (2003) A survey of robot programming systems. In: Proceedings of the Australasian conference on robotics and automation, pp 1–3

  12. Bluethmann W, Ambrose R, Diftler M, Askew S, Huber E, Goza M, Rehnmark F, Lovchik C, Magruder D (2003) Robonaut: a robot designed to work with humans in space. Auton Robot 14(2):179–197.

    Article  MATH  Google Scholar 

  13. Bradshaw JM, Feltovich PJ, Jung H, Kulkarni S, Taysom W, Uszok A (2004) Dimensions of adjustable autonomy and mixed-initiative interaction. In: Nickles M, Rovatsos M, Weiss G (eds) Agents and computational autonomy, lecture notes in computer science. Springer, Berlin , pp 17–39

  14. Bröhl C, Nelles J, Brandl C, Mertens A, Schlick CM (2016) TAM reloaded: a technology acceptance model for human-robot cooperation in production systems. In: Stephanidis C (ed) HCI international 2016 – Posters’ Extended Abstracts, vol 617. Springer International Publishing, Cham, pp 97–103

  15. Brooke J (1996) SUS - a quick and dirty usability scale. In: Jordan P, Thomas B, Weerdmeester B, McClelland I (eds) Usability evaluation in industry, chap. 21. CRC Press, London, pp 189–194

  16. Bruno G, Antonelli D (2018) Dynamic task classification and assignment for the management of human-robot collaborative teams in workcells. Int J Adv Manufact Technol 98(9-12):2415–2427

    Article  Google Scholar 

  17. BS 4778-3.1:1991: Quality vocabulary. availability, reliability and maintainability terms. Guide to concepts and related definitions. Standard BS 4778-3.1:1991, British Standards Institution, London, UK (1991)

  18. BS 8611:2016: Robots and robotic devices. guide to the ethical design and application of robots and robotic systems. Standard BS 8611:2016, British Standards Institution, London, UK (2016)

  19. Campana JR, Quaresma M (2017) The importance of specific usability guidelines for robot user interfaces. In: Marcus A, Wang W (eds) Design, user experience, and usability: designing pleasurable experiences, lecture notes in computer science. Springer International Publishing, Cham, pp 471–483.

  20. Charalambous G, Fletcher S, Webb P (2015) Identifying the key organisational human factors for introducing human-robot collaboration in industry: an exploratory study. Int J Adv Manufact Technol 81(9-12):2143–2155

    Article  Google Scholar 

  21. Charalambous G, Fletcher S, Webb P (2016) The development of a scale to evaluate trust in industrial human-robot collaboration. International Journal of Social Robotics 8(2):193–209

    Article  Google Scholar 

  22. Charalambous G, Fletcher SR, Webb P (2017) The development of a human factors readiness level tool for implementing industrial human-robot collaboration. Int J Adv Manufact Technol 91(5-8):2465–2475

    Article  Google Scholar 

  23. Colgate JE, Wannasuphoprasit W, Peshkin MA (1996) Cobots: robots for collaboration with human operators. In: Proceedings of the 1996 ASME dynamic systems and control division, DSC, 1996-8, vol 58. ASME, New York, pp 433–439

  24. Corrales JA, Candelas FA, Torres F (2011) Safe human–robot interaction based on dynamic sphere-swept line bounding volumes. Robot Comput Integr Manuf 27(1):177–185.

    Article  Google Scholar 

  25. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13(3):319–340.

    Article  Google Scholar 

  26. De Santis A, Siciliano B, De Luca A, Bicchi A (2008) An atlas of physical human–robot interaction. Mech Mach Theory 43(3):253–270.

    Article  MATH  Google Scholar 

  27. Dedeke A (2017) Cybersecurity framework adoption: using capability levels for implementation tiers and profiles. IEEE Security Privacy 15(5):47–54.

    Article  Google Scholar 

  28. Dubowsky S, Genot F, Godding S, Kozono H, Skwersky A, Yu H, Yu LS (2000) PAMM- a robotic aid to the elderly for mobility assistance and monitoring: a helping-hand for the elderly. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International conference on robotics and automation. Symposia Proceedings (Cat. No.00CH37065), vol 1. IEEE, pp 570–576.

  29. Eimontaite I, Gwilt I, Cameron D, Aitken JM, Rolph J, Mokaram S, Law J (2019) Language-free graphical signage improves human performance and reduces anxiety when working collaboratively with robots. Int J Adv Manufact Technol 100(1-4):55–73

    Article  Google Scholar 

  30. Endsley MR, Kaber DB (1999) Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics 42(3):462–492.

    Article  Google Scholar 

  31. Fondazione Ergo: EAWS website (2019).

  32. Franceschini F, Galetto M, Maisano D (2019) Designing performance measurement systems: theory and practice of key performance indicators. Management for professionals. Springer International Publishing, Cham.

    Book  Google Scholar 

  33. Gervasi R, Mastrogiacomo L, Franceschini F (2019) Towards the definition of a human-robot collaboration scale. In: Bini M, Amenta P, D’Ambra A, Camminatiello I (eds) Statistical methods for service quality evaluation - book of short papers of IES 2019, Rome, Italy, July 4-5. Cuzzolin, Italy, pp 75–80

  34. Goodrich MA, Schultz AC (2007) Human-robot interaction: a survey Foundations and trends in human-computer interaction, vol 1. Now, Boston

    Book  Google Scholar 

  35. Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N (eds) Advances in psychology, human mental workload, vol 52, pp 139–183, North-Holland.

  36. Huang HM, Messina E, Wade R, English R, Novak B, Albus J (2004) Autonomy measures for robots. In: Proceedings of the international mechanical engineering congress. ASME, Anaheim, pp 1241–1247.

  37. Ikuta K, Ishii H, Nokata M (2003) Safety evaluation method of design and control for human-care robots. The International Journal of Robotics Research 22(5):281–297.

    Article  Google Scholar 

  38. ISO 10218-2:2011: robots and robotic devices – safety requirements for industrial robots – Part 2: robot systems and integration. Standard ISO 10218-2:2011, International Organization for Standardization, Geneva, CH (2011).

  39. ISO 12100:2010: Safety of machinery – general principles for design – risk assessment and risk reduction. Standard ISO 12100:2010, International Organization for Standardization, Geneva, CH (2010).

  40. ISO 26800:2011: Ergonomics - general approach, principles and concepts. Standard ISO 26800:2011, International Organization for Standardization, Geneva, CH (2011).

  41. ISO 9241-11:2018: Ergonomics of human-system interaction - part 11: usability: definitions and concepts. Standard ISO 9241-11:2018, International Organization for Standardization, Geneva, CH (2018).

  42. ISO/TR 14121-2:2012: Safety of machinery – risk assessment – part 2: practical guidance and examples of methods. Standard ISO/TR 14121-2:2012, International Organization for Standardization, Geneva, CH (2012).

  43. ISO/TS 15066:2016: Robots and robotic devices – collaborative robots. Standard ISO/TS 15066:2016, International Organization for Standardization, Geneva, CH (2016).

  44. Kozar O (2010) Towards better group work: seeing the difference between cooperation and collaboration. In: English teaching forum. ERIC, vol 48, pp 16–23

  45. Kragic D, Gustafson J, Karaoguz H, Jensfelt P, Krug R (2018) Interactive, collaborative robots: challenges and opportunities. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence. International Joint Conferences on Artificial Intelligence Organization, Stockholm, pp 18–25

  46. Krüger J, Lien TK, Verl A (2009) Cooperation of human and machines in assembly lines. CIRP Ann 58(2):628–646.

    Article  Google Scholar 

  47. Krüger J, Nickolay B, Heyer P, Seliger G (2005) Image based 3d surveillance for flexible man-robot-cooperation. CIRP Ann 54(1):19–22.

    Article  Google Scholar 

  48. Krüger M, Wiebel CB, Wersing H (2017) From tools towards cooperative assistants. In: Proceedings of the 5th international conference on human agent interaction - HAI ’17. ACM Press, Bielefeld, pp 287–294.

  49. KUKA: (2020).

  50. Kurose JF, Ross KW (2013) Computer networking: a top-down approach, 6th edn. Addison-Wesley, Upper Saddle River

    Google Scholar 

  51. Lindblom J, Wang W (2018) Towards an evaluation framework of safety, trust, and operator experience in different demonstrators of human-robot collaboration. In: Case K, Thorvald P (eds) Advances in manufacturing technology XXXII, No 8. IOS Press, pp 145–150.

  52. Mastrogiacomo L, Barravecchia F, Franceschini F (2018) Definition of a conceptual scale of servitization: proposal and preliminary results CIRP. Journal of Manufacturing Science and Technology.

  53. Mateus JC, Claeys D, Limère V, Cottyn J, Aghezzaf EH (2019) A structured methodology for the design of a human-robot collaborative assembly workplace. The International Journal of Advanced Manufacturing Technology 102(5-8):2663–2681

    Article  Google Scholar 

  54. Maurtua I, Ibarguren A, Kildal J, Susperregi L, Sierra B (2017) Human-robot collaboration in industrial applications: safety, interaction and trust. Int J Adv Robot Syst 14(4):1–10. 10.1177/1729881417716010

    Article  Google Scholar 

  55. McInnerney JM, Roberts TS (2004) Collaborative or cooperative learning? In: Roberts TS (ed) Online collaborative learning: theory and practice, chap. 10. Information Science Publishing, Hershey, pp 203–214

  56. Nelles J, Kwee-Meier ST, Mertens A (2019) Evaluation metrics regarding human well-being and system performance in human-robot interaction – a literature review. In: Bagnara S, Tartaglia R, Albolino S, Alexander T, Fujita Y (eds) Proceedings of the 20th congress of the international ergonomics association (IEA 2018), vol. 825. Springer International Publishing, Cham, pp 124–135

  57. Neto P, Simão M, Mendes N, Safeea M (2019) Gesture-based human-robot interaction for human assistance in manufacturing. The International Journal of Advanced Manufacturing Technology 101(1-4):119–135

    Article  Google Scholar 

  58. NIST: Framework for improving critical infrastructure cybersecurity. Tech. rep., National Institute of Standards and Technology, Gaithersburg, MD, USA (2018).

  59. Papanastasiou S, Kousi N, Karagiannis P, Gkournelos C, Papavasileiou A, Dimoulas K, Baris K, Koukas S, Michalos G, Makris S (2019) Towards seamless human robot collaboration: integrating multimodal interaction. The International Journal of Advanced Manufacturing Technology

  60. Pineau J, Montemerlo M, Pollack M, Roy N, Thrun S (2003) Towards robotic assistants in nursing homes: challenges and results. Robot Auton Syst 42(3):271–281.

    Article  MATH  Google Scholar 

  61. Priyadarshini I (2018) Cyber security risks in robotics. In: USA IRMA (ed) Cyber security and threats: concepts, methodologies, tools, and applications, chap. 61. Hershey, IGI Global, pp 1235–1250.

  62. Raibulet C (2008) Facets of Adaptivity. In: Morrison R., Balasubramaniam D., Falkner K. (eds) Software architecture, lecture notes in computer science. Springer, Berlin, pp 342–345.

  63. Rethink Robotics: (2020). url

  64. Roschelle J, Teasley SD (1995) The construction of shared knowledge in collaborative problem solving. In: O’Malley C (ed) Computer supported collaborative learning. Springer, Berlin, pp 69–97

  65. Rozo L, Calinon S, Caldwell DG, Jimenez P, Torras C (2016) Learning physical collaborative robot behaviors from human demonstrations. IEEE Trans Robot 32(3):513–527

    Article  Google Scholar 

  66. Sauppè A, Mutlu B (2015) The social impact of a robot co-worker in industrial settings. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems - CHI ’15. ACM Press, Seoul, pp 3613–3622

  67. Schaub K, Caragnano G, Britzke B, Bruder R (2013) The European assembly worksheet. Theoretical Issues in Ergonomics Science 14(6):616–639.

    Article  Google Scholar 

  68. Schmidtler J, Körber M, Bengler K (2016) A trouble shared is a trouble halved — usability measures for human-robot collaboration. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 000217–000222.

  69. Scholtz J (2003) Theory and evaluation of human robot interactions. In: Proceedings of the 36th annual Hawaii international conference on system sciences, 2003.

  70. Sheridan TB, Verplank WL (1978) Human and computer control of undersea teleoperators. Tech. rep. Massachusetts Institute of Technology Cambridge Man-Machine Systems Lab

  71. Smith KA (1995) Cooperative learning: effective teamwork for engineering classrooms. In: Proceedings frontiers in education 1995 25th annual conference. Engineering Education for the 21st Century, vol 1. IEEE, pp 2b5.13–2b5.18

  72. Spenko M, Yu H, Dubowsky S (2006) Robotic personal aids for mobility and monitoring for the elderly. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(3):344–351.

    Article  Google Scholar 

  73. Stanton NA (2006) Hierarchical task analysis: developments, applications, and extensions. Applied Ergonomics 37(1):55–79.

    Article  Google Scholar 

  74. Tan JTC, Duan F, Zhang Y, Watanabe K, Kato R, Arai T (2009) Human-robot collaboration in cellular manufacturing: design and development. In: 2009 IEEE/RSJ international conference on intelligent robots and systems. IEEE, St. Louis, pp 29–34

  75. Thrun S (2004) Toward a framework for human-robot interaction. Human-Computer Interaction 19(1):9–24.

    Article  Google Scholar 

  76. Tsarouchi P, Makris S, Chryssolouris G (2016) Human-robot interaction review and challenges on task planning and programming. Int J Comput Integr Manuf 29(8):916–931

    Article  Google Scholar 

  77. Universal Robots: Collaborative robotic automation | Cobots from Universal Robots (2020).

  78. Venkatesh V, Bala H (2008) Technology acceptance model 3 and a research agenda on interventions. Decis Sci 39(2):273–315.

    Article  Google Scholar 

  79. Venkatesh V, Davis FD (2000) A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci 46(2):186–204.

    Article  Google Scholar 

  80. Veruggio G (2006) The EURON roboethics roadmap. In: 2006 6Th IEEE-RAS international conference on humanoid robots. IEEE, University of Genova, Genova, pp 612–617

  81. Wang L (2015) Collaborative robot monitoring and control for enhanced sustainability. The International Journal of Advanced Manufacturing Technology 81(9):1433–1445.

    Article  Google Scholar 

  82. Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M, Chen Y, Lv H, Ji H (2019) 2.5DHANDS: a gesture-based MR remote collaborative platform. The International Journal of Advanced Manufacturing Technology 102(5-8):1339–1353

    Article  Google Scholar 

  83. Wang Y, Zhang F (eds) (2017) Trends in control and decision-making for human–robot collaboration systems. Springer International Publishing, Cham

  84. Yanco HA, Drury J (2004) Classifying human-robot interaction: an updated taxonomy. In: 2004 IEEE international conference on systems, man and cybernetics, vol 3, pp 2841–2846.

  85. Yu H, Spenko M, Dubowsky S (2003) An adaptive shared control system for an intelligent mobility aid for the elderly. Auton Robot 15(1):53–66.

    Article  Google Scholar 

Download references


This work has been partially supported by “Ministero dell’Istruzione, dell’Università e della Ricerca” Award “TESUN-83486178370409 finanziamento dipartimenti di eccellenza CAP. 1694 TIT. 232 ART. 6”.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Riccardo Gervasi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: EAWS structure

Appendix: EAWS structure

In this section, the structure of EAWS [67], a tool for the evaluation of physical ergonomics, is reported in more detail. Moreover, the evaluation of physical ergonomics through EAWS for the assembly task example, introduced in Section 5.1, is shown in detail. EAWS is divided in two macro-sections: Whole body and Upper limbs. The whole body macro-section is composed of four sections:

  • Extra points (Fig. 7), which contains additional types physical work load

  • Body postures (Fig. 8), which addresses static working postures and high frequent movements

  • Action forces (Fig. 9), which concerns body forces and forces of the hand–finger system

  • Manual materials handling (Fig. 10), which addresses the handling of loads of more than 2–3 kg

Fig. 7
figure 7

Overall result and Extra points section of EAWS [67]. The evaluations for the assembly task are provided in red

Fig. 8
figure 8

Body postures section of EAWS [67]. The evaluations for the assembly task are provided in red

Fig. 9
figure 9

Action forces section of EAWS [67]. The evaluations for the assembly task are provided in red

Fig. 10
figure 10

Manual materials handling section of EAWS [67]. The evaluations for the assembly task are provided in red

The Upper limbs macro-section has only one section: Upper limb load in repetitive tasks (Fig. 11), which covers gripping modes, forces, postures of the upper limbs in repetitive task.

Fig. 11
figure 11

Upper limbs macro-section evaluation for assembly task. The evaluations for the assembly task are provided in red

Moreover, Figs. 78910, and 11 contain the evaluation of each EAWS section for the assembly task example, introduced in Section 5.1. Manual materials handling section was not taken into account, due to the absence of handling of loads exceeding 2–3 kg. Adding up the scores, the whole body macro-section obtained 12 points, while the upper limbs macro-section 2.8 points (Fig. 7). Therefore, the final score of the EAWS is 12, as it is the maximum between the scores of the two macro-sections. The final EAWS evaluation is “Green”, since the final score is less than 25.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gervasi, R., Mastrogiacomo, L. & Franceschini, F. A conceptual framework to evaluate human-robot collaboration. Int J Adv Manuf Technol 108, 841–865 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: