Abstract
The Industry 4.0 paradigm promises shorter development times, increased ergonomy, higher flexibility and resource efficiency in manufacturing environments. Collaborative robots are an important tangible technology for implementing such a paradigm. A major bottleneck to effectively deploy collaborative robots to manufacturing industries is developing task planning algorithms that enable them to recognize and naturally adapt to varying and even unpredictable human actions while simultaneously ensuring an overall efficiency in terms of production cycle time. In this context, an architecture encompassing task representation, task planning, sensing and robot control has been designed, developed and evaluated in a real industrial environment. A pick-and-place palletization task, which requires the collaboration between humans and robots, is investigated. The architecture uses AND/OR graphs for representing and reasoning upon human–robot collaboration models online. Furthermore, objective measures of the overall computational performance and subjective measures of naturalness in human–robot collaboration have been evaluated by performing experiments with production-line operators. The results of this user study demonstrate how human–robot collaboration models like the one we propose can leverage the flexibility and the comfort of operators in the workplace. In this regard, an extensive comparison study among recent models has been carried out.
Similar content being viewed by others
Notes
References
International Federation of Robotics. Executive summary world robotics 2018 industrial robots. https://ifr.org/downloads/press2018/Executive_Summary_WR_2018_Industrial_Robots.pdf. 2018. [Accessed 2019-10-10]
Colgate JE, Edward J, Peshkin MA, Wannasuphoprasit W (1996 November) Cobots: Robots for collaboration with human operators. In: Proceedings of the 1996 ASME international mechanical engineering congress and exhibition (imece). atlanta, usa
Akella P, Peshkin M, Colgate E, Wannasuphoprasit W, Nagesh N, Wells J, Holland S, Pearson T, Peacock B (1999) Cobots for the automobile assembly line. In: Proceedings of the 1999 IEEE international conference on robotics and automation (ICRA). Detroit, USA
Bicchi A, Peshkin MA, Colgate JE (2008) Safety for physical human–robot interaction. Springer handbook of robotics, Springer, Berlin, pp 1335–1348
Cherubini A, Passama R, Crosnier A, Lasnier A, Fraisse P (2016) Collaborative manufacturing with physical human–robot interaction. Robot Comput Integr Manuf 40(C):1–13
Johannsmeier L, Haddadin S (2017) A hierarchical human–robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robot Autom Lett 2(1):41–48
Munzer T, Mollard Y, Lopes M (2017) Impact of robot initiative on human–robot collaboration. In: Proceedings of the 2017 ACM/IEEE international conference on human–robot interaction (HRI). Vienna, Austria
Arai T, Kato R, Fujita M (2010) Assessment of operator stress induced by robot collaboration in assembly. CIRP Ann 59(1):5–8
Capitanelli A, Maratea M, Mastrogiovanni F, Vallati M (2018) On the manipulation of articulated objects in human–robot cooperation scenarios. Robot Auton Syst 109:139–155
Cannata G, Denei S, Mastrogiovanni F (2010) Tactile sensing: steps to artificial somatosensory maps. In: Proceedings of the 2010 IEEE symposium on robot and human interactive communication (RO-MAN). Viareggio, Italy
Dolan P, Adams M (1998) Repetitive lifting tasks fatigue the back muscles and increase the bending moment acting on the lumbar spine. J Biomech 31(8):713–721
Ranz F, Hummel V, Sihn W (2017) Capability-based task allocation in human–robot collaboration. Procedia Manuf 9:182–189
Pinto C, Amorim P, Veiga G, Moreira A (2017) A review on task planning in human-robot teams. In: Proceedings of the 2017 robotics science and systems conference: workshop on mathematical models, algorithms, and human–robot interaction (RSS). Cambridge, USA
Gombolay MC, Huang C, Shah JA (2015) Coordination of human–robot teaming with human task preferences. In: Proceedings of the 2015 AAAI artificial intelligence for human–robot interaction (AI-HRI). Arlington, USA
Chen F, Sekiyama K, Sasaki H, Huang J, Sun B, Fukuda T (2011) Assembly strategy modeling and selection for human and robot coordinated cell assembly. In: Proceedings of the 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). San Francisco, USA
Tsarouchi P, Michalos G, Makris S, Athanasatos T, Dimoulas K, Chryssolouris G (2017) On a human–robot workplace design and task allocation system. Int J Comput Integr Manuf 30(12):1272–1279
Takata S, Hirano T (2011) Human and robot allocation method for hybrid assembly systems. CIRP Ann-ManufTechnol 60(1):9–12
Darvish K, Bruno B, Simetti E, Mastrogiovanni F, Casalino G (2018) Interleaved online task planning, simulation, task allocation and motion control for flexible human–robot cooperation. In: Proceedings of the 2018 IEEE international symposium on robot and human interactive communication (RO-MAN). Nanjing, China
Gerkey BP, Matarić MJ (2004) A formal analysis and taxonomy of task allocation in multi-robot systems. Int J Robot Res 23(9):939–954
Gombolay M, Jensen R, Stigile J, Son SH, Shah J (2016) Apprenticeship scheduling: learning to schedule from human experts. In: Proceedings of the 2016 AAAI international joint conferences on artificial intelligence (IJCAI). New York, USA
Darvish K, Wanderlingh F, Bruno B, Simetti E, Mastrogiovanni F, Casalino G (2018) Flexible human–robot cooperation models for assisted shop-floor tasks. Mechatronics 51:97–114
Chen F, Sekiyama K, Cannella F, Fukuda T (2014) Optimal subtask allocation for human and robot collaboration within hybrid assembly system. IEEE Trans Autom Sci Eng 11(4):1065–1075
Bortot D, Born M, Bengler K (2013) Directly or on detours: how should industrial robots approximate humans. In: Proceedings of the 2013 ACM/IEEE international conference on human–robot interaction (HRI). Tokyo, Japan
Levine SJ, Williams BC (2014) Concurrent plan recognition and execution for human–robot teams. In: Proceedings of the 2014 international conference on automated planning and scheduling (ICAPS). Portsmouth, USA
Crandall JW, Oudah M, Ishowo-Oloko F, Abdallah S, Bonnefon JF, Cebrian M, Shariff A, Goodrich MA, Rahwan I (2018) Cooperating with machines. Nat Commun 9(1):233
Toussaint M, Munzer T, Mollard Y, Wu LY, Vien NA, Lopes M (2016) Relational activity processes for modeling concurrent cooperation. In: Proceedings of the 2016 IEEE international conference on robotics and automation (ICRA). Stockholm, Sweden
Lamon E, De Franco A, Peternel L, Ajoudani A (2019) A capability-aware role allocation approach to industrial assembly tasks. IEEE Robot Autom Lett 4(4):3378–3385
Colledanchise M, Ögren P (2018) Behavior trees in robotics and Al: an introduction. CRC Press, Boca Ratan
Coronado LE, Mastrogiovanni F, Venture G (2018) Design of a human-centered robot framework for end-user programming and applications. In: Proceedings of the 2018 CISM IFToMM symposium on robot design, dynamics and control (ROMANSY). Rennes, France
Coronado LE, Mastrogiovanni F, Venture G (2018) Development of intelligent behaviours for social robots via user-friendly and modular programming tools. In: Proceedings of the 2018 IEEE workshop on advanced robotics and its social impact (ARSO). Genoa, Italy
Mastrogiovanni F, Sgorbissa A (2013) A behaviour sequencing and composition architecture based on ontologies for entertainment humanoid robot. Robot AutonSyst 2(61):170–183
Paxton C, Hundt A, Jonathan F, Guerin K, Hager GD (2017) CoSTAR: instructing collaborative robots with behavior trees and vision. In: Proceedings of the 2017 IEEE international conference on robotics and automation (ICRA). Singapore, Singapore
Colledanchise M (2017) Behavior trees in robotics. Ph.D. thesis, KTH Royal Institute of Technology
Korsah GA, Stentz A, Dias MB (2013) A comprehensive taxonomy for multi-robot task allocation. Int J Robot Res 32(12):1495–1512
Hawkins KP, Bansal S, Vo NN, Bobick AF (2014) Anticipating human actions for collaboration in the presence of task and sensor uncertainty. In: Proceedings of the 2014 IEEE international conference on robotics and automation (ICRA). Hong Kong, China
Rozo L, Silvério J, Calinon S, Caldwell DG (2016) Exploiting interaction dynamics for learning collaborative robot behaviors. In: Proceedings of the 2016 AAAI international joint conference on artificial intelligence: Interactive machine learning workshop (IJCAI). New York, USA
Koppula HS, Saxena A (2016) Anticipating human activities using object affordances for reactive robotic response. IEEE Trans Pattern Anal Mach Intell 38(1):14–29
Caccavale R, Finzi A (2017) Flexible task execution and attentional regulations in human–robot interaction. IEEE Trans Cogn Dev Syst 9(1):68–79
Haigh KZ, Veloso MM (1998) Planning, execution and learning in a robotic agent. In: Proceedings of the 1998 AAAI international conference on artificial intelligence planning systems (ICAPS). Pittsburgh, USA
Wilcox R, Nikolaidis S, Shah J (2012) Optimization of temporal dynamics for adaptive human-robot interaction in assembly manufacturing. In: Proceedings of the 2012 robotics: science and systems (RSS). Sydney, Australia
Agostini A, Torras C, Wörgötter F (2011) Integrating task planning and interactive learning for robots to work in human environments. In: Proceedings of the 2011 AAAI international joint conference on artificial intelligence (IJCAI). Barcelona, Spain
Sanderson A, Peshkin M, de Mello LH (1988) Task planning for robotic manipulation in space applications. IEEE Trans Aerosp Electron Syst 24(5):619–629
Fikes RE, Nilsson NJ (1971) Strips: a new approach to the application of theorem proving to problem solving. Artif Intell 2(3–4):189–208
Bruno B, Mastrogiovanni F, Sgorbissa A (2014) A public domain dataset for adl recognition using wrist-placed accelerometers. In: Proceedings of the 2014 IEEE symposium on robot and human interactive communication (RO-MAN). Edinburgh, Scotland
Carfì A, Motolese C, Bruno B, Mastrogiovanni F (2018) Online human gesture recognition using recurrent neural networks and wearable sensors. In: Proceedings of the 2018 IEEE symposium on robot and human interactive communication (RO-MAN). Nanjing, China
Krause KW, DeMotte DD, Dinsmoor CA, Evans JA, Nowak GF, Ross GA, Rutledge GJ, Slabe CF (2003) Robotic system with teach pendant. May 6. US Patent 6,560,513
Billard A, Siegwart R (2004) Robot learning from demonstration. Robot Auton Syst 2(47):65–67
Hoener S, Mellert FT (1985) Offline programming of industrial robots. In: Toward the factory of the future. Springer, pp 597–602
Akgun B, Cakmak M, Yoo JW, Thomaz AL (2012) Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective. In: Proceedings of the 2012 ACM/IEEE international conference on human–robot interaction (HRI). Boston, USA
De Mello LH, Sanderson AC (1990) AND/OR graph representation of assembly plans. IEEE Trans Robot Autom 6(2):188–199
Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: Proceedings of the 2009 international conference on robotics and automation (ICRA): workshop on open source software. Kobe, Japan
Latombe JC (2012) Robot motion planning, vol 124. Springer, Berlin
Andersen TT (2015) Optimizing the universal robots ROS driver. Technical University of Denmark, Department of Electrical Engineering. Tech Rep. http://orbit.dtu.dk/en/publications/optimizing-the-universal-robots-ros-driver.html
User Manual: UR10/CB3 version 3.4.5. Universal Robots A/S. 2017. https://s3-eu-west-1.amazonaws.com/ur-support-site/27484/UR10_User_Manual_en_E67ON_Global-3.4.5.pdf
Goodrich MA, Schultz AC (2008) Human–robot interaction: a survey. Found Trends® Human-Comput Interact 1(3):203–275
Tsai RY, Lenz RK (1989) A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans Robot Autom 5(3):345–358
Silvio Traversaro AS (2016) Multibody dynamics notation. Technische Universiteit Eindhoven, Tech Rep. 4;https://pure.tue.nl/ws/files/25753352/Traversaro_en_Saccon_DC_2016.064.pdf
Kareem SY, Buoncompagni L, Mastrogiovanni F (2018) Arianna+: scalable human activity recognition by reasoning with a network of ontologies. In: Proceedings of the 17th International conference of the Italian association for artificial intelligence (AIxIA). Trento, Italy
Luger GF (2005) Artificial intelligence: structures and strategies for complex problem solving. Pearson education, London
Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, Malaysia
Carfì A, Foglino F, Bruno B, Mastrogiovanni F (2019) A multi-sensor dataset of human–human handover. Data Brief 22:109–117
Sebastia L, Onaindia E, Marzal E (2001) Stella: an optimal sequential and parallel planner. In: Proceedings of the 2002 Portuguese conference on artificial intelligence (EPIA). Porto, Portugal
Koenig N, Howard A (2004) Design and use paradigms for gazebo, an open-source multi-robot simulator. In: Proceedings of the 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS). Sendai, Japan
Shah JA (2011) Fluid coordination of human–robot teams. Ph.D. thesis, Massachusetts Institute of Technology
Robla-Gómez S, Becerra VM, Llata JR, Gonzalez-Sarabia E, Torre-Ferrero C, Perez-Oria J (2017) Working together: a review on safe human–robot collaboration in industrial environments. IEEE Access 5:26754–26773
Mastrogiovanni F, Paikan A, Sgorbissa A (2013) Semantic-aware real-time scheduling in robotics. IEEE Trans Rob 1(29):118–135
Espiau B, Chaumette F, Rives P (1992) A new approach to visual servoing in robotics. IEEE Trans Robot Autom 8(3):313–326
Michalos G, Makris S, Tsarouchi P, Guasch T, Kontovrakis D, Chryssolouris G (2015) Design considerations for safe human–robot collaborative workplaces. Procedia CIrP. 37:248–253
Acknowledgements
The authors would like to thank the Schaeffler Group, Slovakia, for the opportunity to perform research in their manufacturing plant and solve real-world problems in human–robot collaboration. The work has been supported by the European Master on Advanced Robotics Plus (EMARO+) programme.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Murali, P.K., Darvish, K. & Mastrogiovanni, F. Deployment and evaluation of a flexible human–robot collaboration model based on AND/OR graphs in a manufacturing environment. Intel Serv Robotics 13, 439–457 (2020). https://doi.org/10.1007/s11370-020-00332-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11370-020-00332-9