Skip to main content

Human–Robot Collaboration in Manufacturing: A Multi-agent View

  • Chapter
  • First Online:
Advanced Human-Robot Collaboration in Manufacturing

Abstract

Recent years have witnessed a growing interest in reintroducing the advantages of the human workforce into industrial production processes while keeping the benefits of machines already proven in production automation. Both industry and academia exhibit intense interest in the topic of combining human and robotic resources in collaborative production environments. Nevertheless, the domain of human–robot collaboration is still undergoing intense evolution—nothing proves it more than the diversity of fundamental approaches, world-views, and gaps in standardisation that all hint at the fact that even the overall understanding of the domain is yet to be consolidated. It is, thus, not realistic to expect that a single comprehensive morphological work would reconcile today’s multitude of views on human–robot collaboration, and this is not the goal of this introductory chapter either. Instead, the chapter gives an overview of the domain relying on a single selected paradigm, namely, multi-agent systems. This choice is based on the assumption that this branch of distributed artificial intelligence, having matured over several decades of research and application, provides feasible perspectives and terminological waypoints for collaborative settings under the structured circumstances of industrial production. The chapter aims to outline structural properties and mechanisms of collaborative systems from an agent-oriented point of view, and aims to provide a reference of terms and concepts which make many different views of the domain comparable. Further chapters of this book, as well as numerous application examples, known industrial solutions and standards, are positioned within this framework to connect theoretical waypoints and practical findings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. J. Bütepage, H. Kjellström, D. Kragic, Anticipating many futures: Online human motion prediction and generation for human–robot interaction, In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA) (Brisbane, Australia, 2018), pp. 4563–4570

    Google Scholar 

  2. M.M. Cummings, Man versus machine or man + machine? IEEE Intell. Syst. 29(5), 62–69 (2014)

    Article  Google Scholar 

  3. G. Charalambous, S. Fletcher, P. Webb, Identifying the key organisational human factors for introducing human–robot collaboration in industry: an exploratory study. Int J Adv Manuf Technol 81(9–12), 2143–2155 (2015)

    Article  Google Scholar 

  4. L. Monostori, J. Váncza, S.R. Kumara, Agent-based systems for manufacturing. CIRP Ann. 55(2), 697–720 (2006)

    Article  Google Scholar 

  5. J. Váncza, Agent theory, in CIRP Encyclopedia of Production Engineering, ed. by L. Laperrière, G. Reinhart (Springer, 2014), pp. 33–40

    Google Scholar 

  6. L. Wang, A. Haghighi, Combined strength of holons, agents and function blocks in cyber-physical systems. J. Manuf. Syst. 40, 25–34 (2016)

    Article  Google Scholar 

  7. L. Wang, B. Schmidt, M. Givehchi, G. Adamson, Robotic assembly planning and control with enhanced adaptability through function blocks. Int J Adv Manuf Technol 77(1–4), 705–715 (2015)

    Google Scholar 

  8. A. Coman, D.W. Aha, Cognitive support for rebel agents: social awareness and counternarrative intelligence. in Proceedings of the Fifth Conference on Advances in Cognitive Systems (Troy, NY, USA, 2017), pp. 1–6

    Google Scholar 

  9. S.E. Hashemi-Petroodi, S. Thevenin, S. Kovalev, A. Dolgui, Operations management issues in design and control of hybrid human–robot collaborative manufacturing systems: a survey. Annu. Rev. Control. 49, 264–276 (2020)

    Article  Google Scholar 

  10. L. Monostori, B. Kádár, T. Bauernhansl, S. Kondoh, S. Kumara, G. Reinhart, O. Sauer, G. Schuh, W. Sihn, K. Ueda, Cyber-physical systems in manufacturing. CIRP Ann. 65(2), 621–641 (2016)

    Article  Google Scholar 

  11. G. Weiss (ed.), Multiagent systems: a modern approach to distributed artificial intelligence, in Intelligent Robotics and Autonomous Agents series (The MIT Press, 2000). ISBN 978–0262731317

    Google Scholar 

  12. A.S. Rao, M.P. Georgeff, A model-theoretic approach to the verification of situated reasoning systems, in Proceedings of the 13th International Joint Conference on Artificial Intelligence (Chambéry, France, 1993), pp. 318–324

    Google Scholar 

  13. H.A. Yanco, J. Drury, Classifying human–robot interaction: an updated taxonomy, in Proceedings of the 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), vol. 3 (The Hauge, Netherlands, 2004), pp. 2841–2846

    Google Scholar 

  14. M. Cummings, Informing autonomous system design through the lens of skill-, rule-, and knowledge-based behaviors. J Cogn Eng Decis Mak 12(1), 58–61 (2018)

    Article  Google Scholar 

  15. J. Rasmussen, Skills, rules, and knowledge: signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans. Syst. Man Cybern. 13(3), 257–266 (1983)

    Article  Google Scholar 

  16. N. Nikolakis, K. Sipsas, S. Makris, Cockpit: a portal for symbiotic human–robot collaborative assembly. Ibid. Chapter 9

    Google Scholar 

  17. G. Horváth, Cs. Kardos, A. Kovács, B. Pataki, S. Tóth, Worker assistance suite for efficient human–robot collaboration. Ibid. Chapter 15

    Google Scholar 

  18. M.R. Pedersen, L. Nalpantidis, R.S. Andersen, C. Schou, S. Bøgh, V. Krüger, O. Madsen, Robot skills for manufacturing: from concept to industrial deployment, Robot. Comput.-Integr. Manuf. 37, 282–291 (2016)

    Google Scholar 

  19. S.Ch. Akkaladevil, M. Plasch, M. Hofmann, L. Hiesmair, M. Ikeda, N. Chitturi, A. Pichler, Programming-free approaches for human–robot collaboration in assembly tasks. Ibid. Chapter 12

    Google Scholar 

  20. T. Gašpar, M. Deniša, P. Radanovič, B. Ridge, T. Rajeeth Savarimuthu, A. Kramberger, M. Priggemeyer, J. Roßmann, F. Wörgötter, T. Ivanovska, S. Parizi, Ž. Gosar, I. Kovač, A. Ude, Smart hardware integration with advanced robot programming technologies for efficient reconfiguration of robot workcells, Robot. Comput.-Integr. Manuf. 66, 2020, 101979

    Google Scholar 

  21. Y. Liu, Z. Li, H. Liu, Z. Kan, Skill transfer learning for autonomous robots and human–robot cooperation: a survey. Robot Auton Syst 128, 103515 (2020)

    Article  Google Scholar 

  22. C.D. Buckingham, A. Ahmed, A. Adams, Designing multiple user perspectives and functionality for clinical decision support systems, in Proceedings of the 2013 Federated Conference on Computer Science and Information Systems (FedCSIS 2013) (Kraków, Poland, 2013), pp. 211–218

    Google Scholar 

  23. C. Schlenoff, E. Prestes, R. Madhavan, P. Goncalves, H. Li, S. Balakirsky, T. Kramer, E. Miguelanez, An IEEE standard ontology for robotics and automation, in Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (Vilamoura, Algarve, Portugal, 2012), pp. 1337–1342

    Google Scholar 

  24. N. Nikolakis, K. Alexopoulos, K. Sipsas, Resource availability and capability monitoring. Ibid. Chapter 7

    Google Scholar 

  25. K. Sipsas, N. Nikolakis, S. Makris, Dynamic assembly planning and task assignment. Ibid. Chapter 8

    Google Scholar 

  26. R.X. Gao, L. Wang, P. Wang, J. Zhang, H. Liu, Human motion tracking, recognition, and prediction for robot control. Ibid. Chapter 11

    Google Scholar 

  27. A.S. Rao, M.P. Georgeff, Modeling rational agents within a BDI-architecture, in Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning (Cambridge, MA, USA, 1991), pp. 473–484

    Google Scholar 

  28. M.E. Bratman, Intention, Plans, and Practical Reason (Harvard University Press, Cambridge, MA, USA, 1987). ISBN 0674458184, https://doi.org/10.2307/2185304

  29. H-M. Huang, E. Messina, J. Albus, Toward a generic model for autonomy levels for unmanned systems (ALFUS). Technical report, National Institute of Standards and Technology Gaithersburg MD (2003)

    Google Scholar 

  30. K.S. Barber, C.E. Martin, Agent autonomy: specification, measurement, and dynamic adjustment, in Proceedings of the Autonomy Control Software Workshop at Autonomous Agents (Agents’99) (Seattle, WA, USA, 1999), pp. 8–15

    Google Scholar 

  31. S.A. Mostafa, M.S. Ahmad, A. Mustapha, Adjustable autonomy: a systematic literature review. Artif. Intell. Rev. 51(2), 149–186 (2019)

    Article  Google Scholar 

  32. M. Sierhuis, J.M. Bradshaw, A. Acquisti, R. van Hoof, R. Jeffers, A. Uszok, Human–agent teamwork and adjustable autonomy in practice, in Proceedings of the Seventh International Symposium on Artificial Intelligence, Robotics and Automation in Space (I-SAIRAS 2003) (Nara, Japan, 2003), paper-ID AM-26, pp. 1–9

    Google Scholar 

  33. G. Klein, D.D. Woods, J.M. Bradshaw, R.R. Hoffman, P.J. Feltovich, Ten challenges for making automation a ‘team player’ in joint human–agent activity. IEEE Intell. Syst. 19(6), 91–95 (2004)

    Article  Google Scholar 

  34. X.V. Wang, L. Wang, Safety strategy and framework for human–robot collaboration. Ibid. Chapter 3

    Google Scholar 

  35. B. Dunin-Kęplicz, R. Verbrugge, Teamwork in Multi-agent Systems: A Formal Approach (Wiley, 2011). ISBN 9781119957607

    Google Scholar 

  36. K. Zhou, J.C. Doyle, K. Glover, Robust and Optimal Control (Prentice Hall, 1996). ISBN 9780134565675

    Google Scholar 

  37. R.E. Kalman, A new approach to linear filtering and prediction problems. Tran. ASME–J. Basic Eng 82, 35–45 (1960)

    Google Scholar 

  38. L. Wang, R.X. Gao, J. Váncza, J. Krüger, X.V. Wang, S. Makris, G. Chryssolouris, Symbiotic human–robot collaborative assembly. CIRP Ann. 68(2), 701–726 (2019)

    Article  Google Scholar 

  39. H. Liu, L. Wang, Latest developments of gesture recognition for human–robot collaboration. Ibid. Chapter 2

    Google Scholar 

  40. P. Wang, H. Liu, L. Wang, R.X. Gao, Deep learning-based human motion recognition for predictive context-aware human–robot collaboration. CIRP Ann. 67(1), 17–20 (2018)

    Article  Google Scholar 

  41. S. Liu, L. Wang, X. V. Wang, Sensorless haptic control for physical human–robot interaction. Ibid. Chapter 13

    Google Scholar 

  42. B. Yao, Z. Zhou, L. Wang, W. Xu, Q. Liu, Sensor-less external force detection for industrial manipulators to facilitate physical human–robot interaction. J. Mech. Sci. Technol. 32(10), 4909–4923 (2018)

    Article  Google Scholar 

  43. D. Hoffman, The Case Against Reality: why Evolution Hid the Truth from Our Eyes (W.W. Norton & Company, 2019). ISBN 9780393254709.

    Google Scholar 

  44. D.D. Hoffman, M. Singh, J. Mark, Does evolution favor true perceptions? in Proceedings vol. 8651, Human Vision and Electronic Imaging XVIII, IS&T/SPIE Electronic Imaging (Burlingame CA, USA, 2013), pp. 10–21

    Google Scholar 

  45. G. Horváth, G. Erdős, Zs. Kemény, Human–robot collaborative workcell calibration and control. Ibid. Chapter 10

    Google Scholar 

  46. B. Schleich, N. Anwer, L. Mathieu, S. Wartzack, Shaping the digital twin for design and production engineering. CIRP Ann. 66(1), 141–144 (2017)

    Google Scholar 

  47. R. Stark, S. Kind, S. Neumeyer, Innovations in digital modelling for next generation manufacturing system design. CIRP Ann. 66(1), 169–172 (2017)

    Article  Google Scholar 

  48. G. Erdős, I. Paniti, B. Tipary, Transformation of robotic workcells to digital twins. CIRP Ann. 69(1) (in print), (2020) https://doi.org/10.1016/j.cirp.2020.03.003

  49. Y. Lu, C. Liu, C., K.I.-K. Wang, H. Huang, X. Xu, Digital twin-driven smart manufacturing: connotation, reference model, applications and research issues. Robot. Comput.-Integr. Manuf. 61, 101837 (2020)

    Google Scholar 

  50. G. Lindemann-von Trzebiatowski, I. Münch, The role concept for agents in multiagent systems. Sozionik-Aktuell 2001(3), 22–38 (2001)

    Google Scholar 

  51. S. Qi, S-C. Zhu, Intent-aware multi-agent reinforcement learning, in Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA) (Brisbane, Australia, 2018), pp. 7533–7540

    Google Scholar 

  52. A. Mohammed, L. Wang, Intelligent human–robot assembly enabled by brain EEG. Ibid. Chapter 14

    Google Scholar 

  53. X.V. Wang, L. Wang, Augmented reality enabled human–robot collaboration. Ibid. Chapter 16

    Google Scholar 

  54. M. Holm, R. Senington, W. Wang, J. Lindblom, Real-world industrial demonstrators on human–robot collaborative assembly. Ibid. Chapter 17

    Google Scholar 

  55. ISO, ISO 10075-1:2017: ergonomic principles related to mental workload—Part 1: general issues and concepts, terms and definitions. https://www.iso.org/standard/66900.html, Accessed 22 Mar 2020

  56. M. Gombolay, A. Bair, C. Huang, J. Shah, Computational design of mixed-initiative human–robot teaming that considers human factors: situational awareness, workload, and workflow preferences. Int. J. Robot. Res. 36(5–7), 597–617 (2017)

    Google Scholar 

  57. A. Dini, C. Murko, S. Yahyanejad, U. Augsdörfer, M. Hofbaur, L. Paletta, Measurement and prediction of situation awareness in human–robot interaction based on a framework of probabilistic attention, in Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Vancouver, Canada, 2017), pp. 4354–4361

    Google Scholar 

  58. D. Tsutsumi, D. Gyulai, E. Takács, J. Bergmann, Y. Nonaka, K. Fujita, Personalized work instruction system for revitalizing human–machine interaction, in 53rd CIRP Conference on Manufacturing Systems (2020, in print)

    Google Scholar 

  59. M.G. Haselton, D. Nettle, D.R. Murray, The evolution of cognitive bias, in The Handbook of Evolutionary Psychology (Wiley, 2016), pp. 968–987. ISBN 9781118755884

    Google Scholar 

  60. L. Sun, C. Bunchapattanasakda, Employee engagement: a literature review. Int J Hum Resour Stud 9(1), 63–80 (2019)

    Article  Google Scholar 

  61. M.B. Shuck, K.K. Wollard, A historical perspective of employee engagement: an emerging definition, in Proceedings of the Eighth Annual College of Education & GSN Research Conference (Miami FL, USA, 2009), pp. 133–139

    Google Scholar 

  62. M.S. Cole, F. Walter, A.G. Bedeian, E.H. O’Boyle, Job burnout and employee engagement: a meta-analytic examination of construct proliferation. J Manage 38(5), 1550–1581 (2012)

    Google Scholar 

  63. T. Keller, C. Bayer, P. Bausch, J. Metternich, Benefit evaluation of digital assistance systems for assembly workstations. Procedia CIRP 81, 441–446 (2019)

    Article  Google Scholar 

  64. A. Mohammed, L. Wang, Brainwaves driven human–robot collaborative assembly. CIRP Ann 67(1), 13–16 (2018)

    Article  Google Scholar 

  65. I. Aaltonen, T. Salmi, I. Marstio, Refining levels of collaboration to support the design and evaluation of human–robot interaction in the manufacturing industry. Procedia CIRP 72, 93–98 (2018)

    Article  Google Scholar 

  66. J. Schmidtler, V. Knott, C. Hölzel, K. Bengler, Human centered assistance applications for the working environment of the future. Occup Ergon 12(3), 83–95 (2015)

    Article  Google Scholar 

  67. J.S. Santos, J.O. Zahn, E.A. Silvestre, V.T. Silva, W.W. Vasconcelos, Detection and resolution of normative conflicts in multi-agent systems: a literature survey. Auton. Agent. Multi-Agent Syst. 31, 1236–1282 (2017)

    Article  Google Scholar 

  68. J. Tumova, D.V. Dimarogonas, Multi-agent planning under local LTL specifications and event-based synchronization. Automatica 70, 239–248 (2016)

    Article  MathSciNet  Google Scholar 

  69. P. Schillinger, M. Bürger, D.V. Dimarogonas, Simultaneous task allocation and planning for temporal logic goals in heterogeneous multi-robot systems. Int. J. Robot. Res. 37(7), 818–838 (2018)

    Article  Google Scholar 

  70. H. Liu, L. Wang, Remote human–robot collaboration: a cyber–physical system application for hazard manufacturing environment. J Manuf Syst 54, 24–34 (2020)

    Article  Google Scholar 

  71. M. Johnson, J.M. Bradshaw, P.J. Feltovich, C.M. Jonker, M.B. van Riemsdijk, M. Sierhuis, Coactive design: designing support for interdependence in joint activity. J Hum-Robot Interact 3(1), 43–69 (2014)

    Article  Google Scholar 

  72. X.V. Wang, Zs. Kemény, J. Váncza, L. Wang, Human–robot collaborative assembly in cyber-physical production: classification framework and implementation. CIRP Ann. 66(1), 5–8 (2017)

    Google Scholar 

  73. P.D. O’Brien, R.C. Nicol, FIPA—towards a standard for software agents. BT Technol. J. 16(3), 51–59 (1998)

    Article  Google Scholar 

  74. J. Ferber, O. Gutknecht, F. Michel, Agent/group/roles: simulating with organizations, in Proceedings of the Fourth International Workshop on Agent-Based Simulation (Montpellier, France, 2003), pp. 12–18

    Google Scholar 

  75. R. Lowe, Y.I. Wu, A. Tamar, J. Harb, P. Abbeel, I. Mordatch, Multi-agent actor-critic for mixed cooperative-competitive environments, in Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017) (Long Beach, CA, USA, 2017), pp. 6379–6390

    Google Scholar 

  76. Cs. Kardos, A. Kovács, J. Váncza, A constraint model for assembly planning. J. Manuf. Syst. 54, 196–203 (2020)

    Google Scholar 

  77. G. Erdős, A. Kovács, J. Váncza, Optimized joint motion planning for redundant industrial robots. CIRP Ann. 65(1), 451–454 (2016)

    Article  Google Scholar 

  78. Y. Wang, L. Wang, Real-time collision-free multi-objective robot motion generation. Ibid. Chapter 5

    Google Scholar 

  79. A. Zacharaki, I. Kostavelis, A. Gasteratos, I. Dokas, Safety bounds in human robot interaction: a survey. Saf. Sci. 127, 104667 (2020)

    Article  Google Scholar 

  80. L. Wang, B. Schmidt, A.Y.C. Nee, Vision-guided active collision avoidance for human–robot collaborations. Manuf Lett 1(1), 5–8 (2013)

    Article  Google Scholar 

  81. H. Liu, L. Wang, Collision-free human–robot collaboration based on context awareness. Robot Comput-Integr Manuf 67, 101997 (2021)

    Article  Google Scholar 

  82. H. Liu, Y. Wang, W. Ji, L. Wang, A context-aware safety system for human–robot collaboration. Procedia Manuf. 17, 238–245 (2018)

    Article  Google Scholar 

  83. I. Marstio, J. Montonen, T. Malm, Dynamic safety system for safe collaboration with industrial robots. Ibid. Chapter 6

    Google Scholar 

  84. B. Schmidt, Real-time collision detection and collision avoidance. Ibid. Chapter 4

    Google Scholar 

  85. ISO, Ergonomics of human–system interaction—Part 210: human-centred design for interactive systems, in ISO 9241-210:2019 (2019)

    Google Scholar 

  86. H. Kagermann, Y. Nonaka, Revitalizing human–machine interaction for the advancement of society, acatech Discussion (2019)

    Google Scholar 

  87. ISO ISO 13855: safety of machinerypositioning of safeguards with respect to the approach speeds of parts of the human body (2010)

    Google Scholar 

  88. ISO, ISO 10218-1: Robots and robotic devices—safety requirements for industrial robots—Part 1: Robots (2011)

    Google Scholar 

  89. ISO, ISO 10218-2: Robots and robotic devicessafety requirements for industrial robots, Part 2: Robot systems and integration (2011)

    Google Scholar 

  90. ISO, ISO/TS 15066 Robotics and robotic devices—collaborative robots (2016)

    Google Scholar 

  91. ISO, ISO 26000: guidance on social responsibility (2010)

    Google Scholar 

Download references

Acknowledgements

Research for work presented in this chapter has been in part funded by the Hungarian National Research, Development and Innovation Office (NKFIH) under Grant GINOP-2.3.2-15-2016-00002 “Industry 4.0 Research and Innovation Centre of Excellence”, and the ED_18-22018-0006 grant titled “Research on prime exploitation of the potential provided by the industrial digitalisation” supported by the National Research, Development and Innovation Fund of Hungary, financed under the public funding scheme according to Chapter 13. §(2) of the Scientific Research, Development and Innovation Act.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zsolt Kemény .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kemény, Z., Váncza, J., Wang, L., Wang, X.V. (2021). Human–Robot Collaboration in Manufacturing: A Multi-agent View. In: Wang, L., Wang, X.V., Váncza, J., Kemény, Z. (eds) Advanced Human-Robot Collaboration in Manufacturing. Springer, Cham. https://doi.org/10.1007/978-3-030-69178-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69178-3_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69177-6

  • Online ISBN: 978-3-030-69178-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics