Skip to main content
SpringerLink
Log in

Programming-Free Approaches for Human–Robot Collaboration in Assembly Tasks

  • Chapter
  • First Online:
Advanced Human-Robot Collaboration in Manufacturing

Abstract

Collaborative robots are paving the way for new applications, and are becoming the new frontiers, especially in manufacturing industries. The concept of combining the cognitive flexibility of humans and the physical strength of the robots is rapidly gaining interest and is leading to novel application areas. In this chapter, we introduce various challenges faced by collaborative robotic systems. The focus is on the recent development of programming interfaces to efficiently install collaborative robotic systems in manufacturing applications and the requirement for such interfaces. An extensive review on the state-of-the-art programming approaches is given. Detailed description of novel interfaces that build upon and extend the current state of the art is presented. A qualitative comparison against different criteria required to consider while developing programming interfaces is carried out in detail. The main objective of the organisation and content of the chapter is to introduce different programming interfaces and paradigms that can be exploited for easy programming of robots by non-experts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. V. Villani, F. Pini, F. Leali, C. Secchi, Survey on human–robot collaboration in industrial settings: safety, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018)

    Article  Google Scholar 

  2. B.D. Argall, S. Chernova, M. Veloso, B. Browning, A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)

    Article  Google Scholar 

  3. A. Bauer, D. Wollherr, M. Buss, Human–robot collaboration: a survey. Int. J. Humanoid Robot. 5, 47–66 (2008)

    Article  Google Scholar 

  4. K. Dautenhahn, Methodology and themes of human–robot interaction: a growing research field. Int. J. Adv. Robot. Syst. 4, 103–108 (2007)

    Google Scholar 

  5. M.A. Goodrich, A.C. Schultz, Human–robot interaction: a survey. Found. Trends Hum. Comput. Interact. 1(3), 203–275 (2007)

    Google Scholar 

  6. M.R. Pedersen, L. Nalpantidis, R.S. Andersen, C. Schou, S. Bøgh, V. Krüger, O. Madsen, Robot skills for manufacturing: from concept to industrial deployment. Robot. Comput.-Integr. Manuf. 37, 282–291 (2016)

    Article  Google Scholar 

  7. R. Maclin, J.W. Shavlik, Creating advice-taking reinforcement learners. Mach. Learn. 22, 251–281 (1996)

    MATH  Google Scholar 

  8. G. Kuhlmann, P. Stone, R. Mooney, J. Shavlik, Guiding a reinforcement learner with natural language advice: initial results in RoboCup soccer, in The AAAI-2004 Workshop on Supervisory Control of Learning and Adaptive Systems (2004)

    Google Scholar 

  9. D.L. Moreno, C.V. Regueiro, R. Iglesias, S. Barro, Using prior knowledge to improve reinforcement learning in mobile robotics, in Proceedings of the Towards Autonomous Robotics Systems (University of Essex, UK, 2004)

    Google Scholar 

  10. M.N. Nicolescu, M.J. Mataric, Natural methods for robot task learning: Instructive demonstrations, generalization and practice, in Proceedings of The Second International Joint Conference on Autonomous Agents and Multiagent Systems (2003), pp. 241–248

    Google Scholar 

  11. B. Finkemeyer, T. Kröger, F.M. Wahl, Executing assembly tasks specified by manipulation primitive nets. Adv. Robot. 19, 591–611 (2005)

    Article  Google Scholar 

  12. H. Bruyninckx, J. De Schutter, Specification of force-controlled actions in the ‘task frame formalism’—a synthesis. IEEE Trans. Robot. Autom. 12, 581–589 (1996)

    Article  Google Scholar 

  13. H. Mosemann, F.M. Wahl, Automatic decomposition of planned assembly sequences into skill primitives. IEEE Trans. Robot. Autom. 17, 709–718 (2001)

    Article  Google Scholar 

  14. F. Steinmetz, R. Weitschat, Skill parametrization approaches and skill architecture for human–robot interaction, in IEEE International Conference on Automation Science and Engineering (CASE) (2016), pp. 280–285

    Google Scholar 

  15. M.R. Pedersen, V. Krüger, Automated planning of industrial logistics on a skill-equipped robot, in IROS 2015 workshop Task Planning for Intelligent Robots in Service and Manufacturing (Hamburg, Germany, 2015)

    Google Scholar 

  16. E. Dean-Leon, K. Ramirez-Amaro, F. Bergner, I. Dianov, P. Lanillos, G. Cheng, Robotic technologies for fast deployment of industrial robot systems, in IECON 2016—42nd Annual Conference of the IEEE Industrial Electronics Society, vol. 42 (2016), pp. 6900–6907

    Google Scholar 

  17. D. Holz, A. Topalidou-Kyniazopoulou, F. Rovida, M. Pedersen, V. Krüger, S. Behnke, A skill-based system for object perception and manipulation for automating kitting tasks, in IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA), vol. 20 (2015), pp. 1–9

    Google Scholar 

  18. A. Billard, S. Calinon, R. Dillmann, S. Schaal, Robot programming by demonstration. Springer Handbook of Robotics 59, 1371–1394 (2008)

    Article  Google Scholar 

  19. W.K.H. Ko, Y. Wu, K.P. Tee, J. Buchli, Towards industrial robot learning from demonstration, in Proceedings of the 3rd International Conference on Human-Agent Interaction, vol. 3 (2015), pp. 235–238

    Google Scholar 

  20. M. Stenmark, E.A. Topp, From demonstrations to skills for high-level programming of industrial robots, in AAAI Fall Symposium (2016)

    Google Scholar 

  21. R.S. Sutton, A.G. Barto, Reinforcement Learning: an Introduction, vol. 1, no. 1 (MIT press, Cambridge, 1998)

    Google Scholar 

  22. U. Kartoun, H. Stern, Y. Edan, A human–robot collaborative reinforcement learning algorithm. J. Intell. Rob. Syst. 60(2), 217–239 (2010)

    Article  Google Scholar 

  23. S. Griffith, K. Subramanian, J. Scholz, C.L. Isbell, A.L. Thomaz, Policy shaping: integrating human feedback with reinforcement learning, in Advances in Neural Information Processing Systems (2013), pp. 2625–2633

    Google Scholar 

  24. W.B. Knox, P. Stone, C. Breazeal, Training a robot via human feedback: a case study, in International Conference on Social Robotics (Springer, 2013), pp. 460–470

    Google Scholar 

  25. H.B. Suay, S. Chernova, Effect of human guidance and state space size on interactive reinforcement learning, in International Conference on Robot & Human Interactive Communication (RO-MAN), vol. 25 (2011), pp. 1–6

    Google Scholar 

  26. A.L. Thomaz, G. Hoffman, C. Breazeal, Reinforcement learning with human teachers: understanding how people want to teach robots, in International Conference on Robot & Human Interactive Communication (RO-MAN), vol. 25 (2006), pp. 352–357

    Google Scholar 

  27. B. Peng, J. MacGlashan, R. Loftin, M.L. Littman, D.L. Roberts, M.E. Taylor, A need for speed: adapting agent action speed to improve task learning from non-expert humans, in Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (2016), pp. 957–965

    Google Scholar 

  28. W.B. Knox, P. Stone, Interactively shaping agents via human reinforcement: the TAMER framework. Proceedings of the Fifth International Conference on Knowledge Capture 5, 9–16 (2009)

    Article  Google Scholar 

  29. L. Rozo Castañeda, S. Calinon, D. Caldwell, P. Jimenez Schlegl, C. Torras, Learning collaborative impedance-based robot behaviors, in Twenty-Seventh AAAI Conference on Artificial Intelligence, vol. 27 (2013), pp. 1422–1428

    Google Scholar 

  30. L. Rozo Castañeda, S. Calinon, D. Caldwell, Learning force and position constraints in human–robot cooperative transportation, in International Symposium on Robot and Human Interactive Communication, vol. 23 (2014), pp. 619–624

    Google Scholar 

  31. S. C. Akkaladevi, M. Plasch, A. Pichler, B. Rinner, Human robot collaboration to reach a common goal in an assembly process, in Proceedings of the European Starting AI Researcher Symposium (STAIRS) (2016), pp. 3–14

    Google Scholar 

  32. A. Pichler, S.C. Akkaladevi, M. Ikeda, M. Hofmann, M. Plasch, C. Wögerer, G. Fritz, Towards shared autonomy for robotic tasks in manufacturing. Procedia Manuf. 11, 72–82 (2017)

    Article  Google Scholar 

  33. S. Akkaladevi, M. Plasch, A. Pichler, Skill-based learning of an assembly process, e & i Elektrotechnik und Informationstechnik 134(6), 312–315 (2017)

    Google Scholar 

  34. V. Vyatkin, IEC 61499 as enabler of distributed and intelligent automation: state-of-the-art review. IEEE Trans. Industr. Inf. 7(4), 768–781 (2011)

    Article  Google Scholar 

  35. SYMBIO-TIC, Symbiotic human–robot collaborative assembly: technologies, innovations and competitiveness, H2020-EU.2.1.5.1 (2019). https://www.symbio-tic.eu/, Last Accessed 08 Jan 2020

  36. L. Wang, R.X. Gao, J. Váncza, J. Krüger, X.V. Wang, S. Makris, G. Chryssolouris, Symbiotic human–robot collaborative assembly. CIRP Ann. Manuf. Technol. 68(2), 701–726 (2019)

    Google Scholar 

  37. ABB Robotics, Robot studio—the world's most used offline programming tool for robotics (2020). https://new.abb.com/products/robotics/robotstudio

  38. Drag&Bot GmbH, Drag&Bot—easy robot operation instead of complicated programming (2020). https://www.dragandbot.com/product/

  39. 4DIAC, Framework for distributed industrial automation and control (2020). https://www.eclipse.org/4diac/

  40. SYMBIO-TIC Consortium, Demonstrator 3 completion report—automotive component assembly (2019). https://cordis.europa.eu/project/id/637107/results, Last Accessed 15 Jan 2020

  41. V. Valeria, P. Fabio, L. Francesco, S. Cristian, Survey on human–robot collaboration in industrial settings: saefty, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018)

    Article  Google Scholar 

  42. N.C. Chitturi, P. Kulha, M. Plasch, M. Ganglbauer, A. Weinbacher, M. Zillich, A. Pichler, Intuitive human–robot interaction with inkjet printed capacitive sensors, in Fraunhofer Direct Digital Manufacturing Conference 2020 (2020)

    Google Scholar 

  43. TexasInstruments, FDC 1004 product description, in Texas Instruments (2020). https://www.ti.com/product/FDC1004

  44. STMicroelectronics, STM32 32-bit Arm Cortex MCUs (2020). https://www.st.com/en/microcontrollers-microprocessors/stm32-32-bit-arm-cortex-mcus.html

  45. UniversalRobots, UNIVERSAL ROBOTS (2020). https://www.universal-robots.com/products/ur10-robot/

  46. UniversalRobots, About Universal Robots+ (2020). https://www.universal-robots.com/plus/

  47. L. Hiesmair, M. Plasch, H. Nöhmayer, A. Pichler, Intuitive Human Machine Interaction based on multimodal command fusion and interpretation, in Proceedings of the ARW & OAGM Workshop 2019 (2019). https://doi.org/10.3217/978-3-85125-663-5-02

  48. F.C. Pereira, D.H. Warren, Definite clause grammars for language analysis—a survey of the formalism and a comparison with augmented transition networks. Artif. Intell. 13(3), 231–278 (1980)

    Article  MathSciNet  Google Scholar 

  49. Profactor GmbH, SYMBIO-TIC multimodal robot programming (2019). https://youtu.be/AbJ8VaxxwzI, Last Accessed 20 Aug 2020

  50. A. Billard, S. Calinon, R. Dillmann, S. Schaal, Robot programming by demonstration, in Springer Handbook of Robotics (2008), 1371–1394

    Google Scholar 

  51. M. Ikeda, S. Maddukuri, M. Hofmann, A. Pichler, X. Zhang, A. Polydoros, J. Piater, K. Winkler, K. Brenner, I. Harton, U. Neugebauer, FlexRoP—flexible, assistive robots for customized production. Austrian Robotics Workshop 2018, 53–58 (2018)

    Google Scholar 

  52. X. Zhang, A. S. Polydoros, J. Piater, Learning movement assessment primitives for force interaction skills (2018). arXiv preprint arXiv:1805.04354

  53. S.C. Akkaladevi, A. Pichler, M. Plasch, M. Ikeda, M. Hofmann, Skill-based programming of complex robotic assembly tasks for industrial application. e & i Elektrotechnik und Informationstechnik 136(7), 326–333 (2019)

    Google Scholar 

  54. Vive.com, VIVE tracker (2020). https://www.vive.com/eu/vive-tracker. Last Accessed 13 Jan 2020

  55. Ati-ia.com, ATI industrial automation: F/T sensor delta (2020). https://www.ati-ia.com. Last Accessed: 13 Jan 2020

  56. S.C. Akkaladevi, M. Plasch, A. Pichler, M. Ikeda, Towards reinforcement based learning of an assembly process for human robot collaboration. Procedia Manuf 38, 1491–1498 (2019)

    Article  Google Scholar 

  57. S. Akkaladevi, M. Ankerl, C. Heindl, A. Pichler, Tracking multiple rigid symmetric and non-symmetric objects in real-time using depth data, in International Conference on Robotics and Automation (ICRA) (2016), pp. 5644–5649

    Google Scholar 

  58. S.C. Akkaladevi, M. Plasch, C. Eitzinger, B. Rinner, Context enhanced multi-object tracker for human robot collaboration, in Proceedings of the ACM/IEEE International Conference on Human–Robot Interaction (late breaking reports) (2017), pp. 61–62

    Google Scholar 

  59. S.C. Akkaladevi, C. Heindl, Action recognition for human robot interaction in industrial applications, in International Conference on Computer Graphics, Vision and Information Security (CGVIS) (2015), pp. 94–99

    Google Scholar 

  60. S.C. Akkaladevi, M. Plasch, S. Maddukuri, C. Eitzinger, A. Pichler, B. Rinner, Toward an interactive reinforcement based learning framework for human robot collaborative assembly processes. Front. Robot. AI 5(126) (2018). https://doi.org/10.3389/frobt.2018.00126

  61. Profactor GmbH, Towards an interactive reinforcement based learning framework for human robot collaboration—video demonstration (2018). https://www.youtube.com/watch?v=hIsWTUllu3g. Last Accessed 20 Aug 2020

  62. S.C. Akkaladevi, M. Plasch, C. Eitzinger, S.C. Maddukuri, B. Rinner, Towards learning to handle deviations using user preferences in a human robot collaboration scenario, in 8th International Conference on Intelligent Human Computer Interaction, vol. 8, pp. 3–14 (2017)

    Google Scholar 

  63. S.C. Akkaladevi, M. Plasch, C. Eitzinger, A. Pichler, B. Rinner, Towards a context enhanced framework for multi object tracking in human robot collaboration, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Madrid, Spain, 2018) pp. 168–173

    Google Scholar 

  64. K. Ramirez-Amaro, M. Beetz, G. Cheng, Transferring skills to humanoid robots by extracting semantic representations from observations of human activities. Artif. Intell. 247, 95–118 (2017)

    Article  MathSciNet  Google Scholar 

  65. Y. Mollard, T. Munzer, A. Baisero, M. Toussaint, M. Lopes, Robot programming from demonstration, feedback and transfer, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Hamburg, 2015), pp. 1825–1831

    Google Scholar 

  66. L. Ehrlinger, W. Wöß, Towards a definition of knowledge graphs. SEMANTiCS (Posters, Demos, SuCCESS) 48, 1–4 (2016)

    Google Scholar 

  67. K. Munir, M.S. Anjum, The use of ontologies for effective knowledge modelling and information retrieval. Appl Comput Inform 14(2), 116–126 (2018)

    Article  Google Scholar 

  68. M. Tenorth, M. Beetz, Representations for robot knowledge in the know rob framework. Artif. Intell. 247, 151–169 (2017)

    Article  Google Scholar 

  69. B. Goertzel, Patterns, hypergraphs and embodied general intelligence, in IEEE International Joint Conference on Neural Network Proceedings (2006), pp. 451–458

    Google Scholar 

  70. D. Hart, B. Goertzel, Opencog: a software framework for integrative artificial general intelligence, in AGI (2008), pp. 468–472

    Google Scholar 

  71. S. Klarman, Modelling data with hypergraphs—a closer look at GRAKN.AI’s hypergraph data model (2017). https://blog.grakn.ai/modelling-data-withhypergraphs-edff1e12edf0. Last Accessed 20 Aug 2020

  72. Grakn Labs, GraknAI—A knowledge graph (2020). https://grakn.ai/about. Last Accessed 20 Aug 2020

  73. ArtiMinds Robotics GmbH, Artiminds robot programming suite (2013–2020). https://www.artiminds.com/. Last Accessed 20 Aug 2020

  74. OCTOPUZ Inc., OCTOPUZ offline robot programming software (2020). https://octopuz.com/. Last Accessed 20 Aug 2020

  75. Hypertherm, Inc., Robotmaster—CAD/CAM for robots (2019). https://www.robotmaster.com/en/. Last Accessed 20 Aug 2020

Download references

Acknowledgements

The work presented in this chapter is a cumulative effort from various projects and is jointly funded by projects SYMBIO TIC (EC 637107), Bridge Human–Robot Interaction “TeachBots” (Industrial), HoliSafeMRK (FFG 864872), FELICE (EU GrantID 101017151), DigiManu and Smart Factory Lab (Funded by the State of Upper Austria).

Author information

Authors and Affiliations

  1. Department of Robotics and Automation Systems, Profactor GmbH, Steyr-Gleink, Austria

    Sharath Chandra Akkaladevi, Matthias Propst, Michael Hofmann, Leopold Hiesmair, Markus Ikeda, Naresh Chowdary Chitturi & Andreas Pichler

  2. Institute of Networked and Embedded Systems, Alpen-Adria Universität Klagenfurt, Klagenfurt, Austria

    Sharath Chandra Akkaladevi

Authors
  1. Sharath Chandra Akkaladevi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Propst
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Hofmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leopold Hiesmair
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Ikeda
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Naresh Chowdary Chitturi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andreas Pichler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sharath Chandra Akkaladevi .

Editor information

Editors and Affiliations

  1. Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden

    Lihui Wang

  2. Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden

    Xi Vincent Wang

  3. Research Laboratory on Engineering and Management Intelligence, SZTAKI Institute for Computer Science and Control, Eötvös Loránd Research Network, Budapest, Hungary

    József Váncza

  4. Research Laboratory on Engineering and Management Intelligence, SZTAKI Institute for Computer Science and Control, Eötvös Loránd Research Network, Budapest, Hungary

    Zsolt Kemény

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Akkaladevi, S.C. et al. (2021). Programming-Free Approaches for Human–Robot Collaboration in Assembly Tasks. In: Wang, L., Wang, X.V., Váncza, J., Kemény, Z. (eds) Advanced Human-Robot Collaboration in Manufacturing. Springer, Cham. https://doi.org/10.1007/978-3-030-69178-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69178-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69177-6

  • Online ISBN: 978-3-030-69178-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation