Enabling Domain Experts to Model and Execute Tasks in Flexible Human-Robot Teams

  • Dominik Riedelbauch
  • Tobias Werner
  • and Dominik Henrich
Conference paper


Recent advances in safe human-robot coexistence make collaboration of humans and robots in achieving common goals feasible. We propose a concept that treats human and robot agents as equal partners in executing a task specified by a shared task model. Equality between agents offers high flexibility, as e.g. the team composition may change arbitrarily without interrupting the working progress. The main challenge in achieving flexible teaming is coordinating the robot with operations executed by human partners. We contribute an approach to this problem that is based on observing pre- and postconditions of operations using a robot-mounted camera system. The coordination mechanism is embedded into a framework that allows domain experts to create, test, and dispatch new task models to collaborative execution. The approach is validated by experimental tasks composed of pick-and-place operations.


human-robot collaboration equal-partners teamwork humanrobot coordination task modelling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1. M. Foster and C. Matheson, Following Assembly Plans in Cooperative, Task-Based Human-Robot Dialogue, Proc. of the 12th Workshop on the Semantics and Pragmatics of Dialogue, 2008Google Scholar
  2. 2. M. Fritzsche, N. Elkmann and E. Schulenburg, Tactile Sensing: A Key Technology for Safe Physical Human Robot Interaction, 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Lausanne, 2011Google Scholar
  3. 3. P. Ferreira, S. Doltsinis and N. Lohse, Symbiotic Assembly Systems – A New Paradigm, Procedia CIRP, vol. 17, pp. 26–31, 2014Google Scholar
  4. 4. T. Fong et al., The Human-Robot Interaction Operating System, Proc. of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, 2006Google Scholar
  5. 5. T. Gecks and D. Henrich, Sensor-Based Online Planning of Time-Optimized Paths in Dynamic Environments, GWR09 German Workshop on Robotics, Germany, 2009Google Scholar
  6. 6. T. Hamabe, H. Goto and J. Miura, A Programming by Demonstration System for Human-Robot Collaborative Assembly Tasks, IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, 2015Google Scholar
  7. 7. G. Hoffman and C. Breazeal, Collaboration in Human-Robot Teams, Proc. of the AIAA 1st Intelligent Systems Technical Conference, Chicago, 2004Google Scholar
  8. 8. L.S. Homem de Mello and A.C. Sanderson, Representations of Mechanical Assembly Sequences, IEEE Transactions on Robotics and Automation, vol. 7, no.2, 1991Google Scholar
  9. 9. N. Jennings, On Agent-Based Software Engineering, Artififial Intelligence, Volume 117, Issue 2, 2000Google Scholar
  10. 10. L. Johannsmeier and S. Haddadin, A Hierarchical Human-Robot Interaction-Planning Framework for Task Allocation in Collaborative Industrial Assembly Processes, IEEE Robotics and Automation Letters 2.1: 41-48, 2017Google Scholar
  11. 11. C. M. Jonker, M. B. van Riemsdijk and B. Vermeulen, Shared Mental Models: A Conceptual Analysis, COIN 2010 International Workshop, Lecture Notes in Artificial Intelligence Volume 6541, 2010Google Scholar
  12. 12. H. Kimura, T. Horiuchi and K. Ikeuchi, Task-Model Based Human Robot Cooperation Using Vision, Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyongju, 1999Google Scholar
  13. 13. S. Lalle et al., Towards a Platform-Independent Cooperative Human Robot Interaction System: III An Architecture for Learning and Executing Actions and Shared Plans, IEEE Transactions on Autonomous Mental Development, vol. 4, no. 3, 2012Google Scholar
  14. 14. M. E. Latoschik and H. Tramberend: A Scala-Based Actor-Entity Architecture for Intelligent Interactive Simulations, 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), 2012.Google Scholar
  15. 15. J. Laursen, U.Schultz and L. Ellekilde, Automatic Error Recovery in Robot Assembly Operations using Reverse Execution, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, 2015Google Scholar
  16. 16. C. Lenz et al., Joint-Action for Humans and Industrial Robots for Assembly Tasks, 17th IEEE International Symposium on Robot and Human Interactive Communication, Munich, 2008Google Scholar
  17. 17. S. J. Levine and B. C. Williams, Concurrent Plan Recognition and Execution for Human-Robot Teams, 24th International Conference on Automated Planning and Scheduling (ICAPS), 2014Google Scholar
  18. 18. B. Lim and K. Klein, Team Mental Models and Team Performance: A Field Study of the Effects of Team Mental Model Similarity and Accuracy, Journal of Organizational Behavior, 27(4):403, 2006Google Scholar
  19. 19. E. Mathieu et al., The Influence of Shared Mental Models on Team Process and Performance, The Journal of Applied Psychology, 2000Google Scholar
  20. 20. G. Milliez et al., Using Human Knowledge Awareness to adapt Collaborative Plan Generation, Explanation and Monitoring, 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, 2016Google Scholar
  21. 21. S. Nikolaidis and J. Shah, Human-Robot Cross-Training: Computational Formulation, Modeling and Evaluation of a Human Team Training Strategy, 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Tokyo, 2013Google Scholar
  22. 22. A. Schneider, J. Sturm et al., Object Identification with Tactile Sensors using Bagof-Features, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), St. Louis, 2009Google Scholar
  23. 23. D. Schroeter et al., Methodology to Identify Applications for Collaborative Robots in Powertrain Assembly, Procedia CIRP, vol. 55, pp. 12–17, 2016Google Scholar
  24. 24. J. Shah et al., Improved Human-Robot Team Performance using Chaski, a Human-Inspired Plan Execution System, Proc. of the 6th International Conference on Human-Robot Interaction (HRI), Lausanne, 2011Google Scholar
  25. 25. U. Thomas, B. Finkemeyer, T. Kroger and F. M. Wahl, Error-Tolerant Execution of Complex Robot Tasks based on Skill Primitives, IEEE International Conference on Robotics and Automation (ICRA), Taipei, 2003Google Scholar
  26. 26. T. Werner and D. Henrich, Efficient and Precise Multi-Camera Reconstruction, 8th ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), Venice, 2014Google Scholar
  27. 27. T. Werner et al., ENACT: An Efficient and Extensible Entity-Actor Framework for Modular Robotics Software Components, 47th International Symposium on Robotics (ISR), 2016Google Scholar
  28. 28. H. Yussof et al., Low Force Control Scheme for Object Hardness Distinction in Robot Manipulation based on Tactile Sensing, IEEE International Conference on Robotics and Automation (ICRA), Pasadena, 2008Google Scholar

Copyright information

© Springer-Verlag GmbH Deutschland 2017

Authors and Affiliations

  • Dominik Riedelbauch
    • 1
  • Tobias Werner
    • 1
  • and Dominik Henrich
    • 1
  1. 1.Chair for Robotics and Embedded SystemsUniversit¨at BayreuthBayreuthDeutschland

Personalised recommendations