Interaction Design Patterns for Adaptive Human-Agent-Robot Teamwork in High-Risk Domains

  • Mark A. NeerincxEmail author
  • Jurriaan van Diggelen
  • Leo van Breda
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9736)


Integrating cognitive agents and robots into teams that operate in high-demand situations involves mutual and context-dependent behaviors of the human and agent/robot team-members. We propose a cognitive engineering method that includes the development of Interaction Design patterns for such systems as re-usable, theoretically and empirically founded, design solutions. This paper presents an overview of the background, the method and three example patterns.


Design patterns Human-Agent teamwork Cognitive robots Cognitive engineering High-Risk domain 



This research is supported by the EU FP7 project 609763 (TRADR), and the TNO Defense research program V1340 on Unmanned Systems.


  1. 1.
    Alexander, C., Ishikawa, S., Silverstein, M.A.: Pattern Language: Towns, Buildings, Construction. Center for Environmental Structure series. Oxford University Press, Berkeley (1977)Google Scholar
  2. 2.
    Arciszewski, H.F., De Greef, T.E., Van Delft, J.H.: Adaptive automation in a naval combat management system. IEEE Trans. Syst. Cybern. A Syst. Hum. 39(6), 1188–1199 (2009)CrossRefGoogle Scholar
  3. 3.
    Bagosi, T., de Greeff, J., Hindriks, K.V., Neerincx, M.A.: Designing a Knowledge Representation Interface for Cognitive Agents. In: Baldoni, M., Baresi, L., Dastani, M. (eds.) EMAS 2015. LNCS, vol. 9318, pp. 33–50. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-26184-3_3 CrossRefGoogle Scholar
  4. 4.
    Banbury, S., Gauthier, M., Scipione, A., Hou, M.: Intelligent adaptive systems: literature research of design guidance for intelligent adaptive automation and interfaces. Defence Research and Development Canada - Toronto, CR 2007-075 (2007)Google Scholar
  5. 5.
    Borchers, J.O.: A pattern approach to interaction design. AI SOC. 15(4), 359–376 (2001)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bradshaw, J.M., Feltovich, P., Johnson, M., Breedy, M., Bunch, L., Eskridge, T., Jung, H., Lott, J., Uszok, A., van Diggelen, J.: From Tools to Teammates: Joint Activity in Human-Agent-Robot Teams. In: Kurosu, M. (ed.) HCD 2009. LNCS, vol. 5619, pp. 935–944. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Dearden, A., Finlay, J.: Pattern languages in HCI: a critical review. Hum. Comput. Inter. 21(1), 49–102 (2006)CrossRefGoogle Scholar
  8. 8.
    De Greef, T.E., Arciszewski, H.F.R., Neerincx, M.A.: Adaptive automation based on an object-oriented task model: implementation and evaluation in a realistic C2 environment. J. Cognitive Eng. Decis. Mak. 4, 152–173 (2010)CrossRefGoogle Scholar
  9. 9.
    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addinson-Wesley, Reading (1995)zbMATHGoogle Scholar
  10. 10.
    Gevers, J.M.P., Uitdewilligen, S., Margarida Passos, A.: Dynamics of team cognition and team adaptation: introduction to the special issue. Eu. J. Work Organ. Psychol. 24(5), 645–651 (2015)CrossRefGoogle Scholar
  11. 11.
    Harrison, J.A., Forster, M.J.: Human systems integration requirements in systems acquisition. In: Booher, H.R. (ed.) Handbook of Human Systems Integration (2003). Wiley, Hoboken (2003)Google Scholar
  12. 12.
    Hindriks, K.V.: Programming rational agents in goal. In: El Fallah Seghrouchni, A., Dix, J., Dastani, M., Bordini, H.R. (eds.) Multi-agent Programming, pp. 119–157. Springer, New York (2009)CrossRefGoogle Scholar
  13. 13.
    Jennings, N.R.: An agent-based approach for building complex software systems. Commun. ACM 44(4), 35–41 (2001)CrossRefGoogle Scholar
  14. 14.
    Kahn, P.H., Freier, N.G., Kanda, T., Ishiguro, H., Ruckert, J.H., Severson, R.L., Kane, S.K.: Design patterns for sociality in human-robot interaction. In: Proceedings of the 3rd ACM/IEEE International Conference on Human-Robot Interaction, pp. 97–104. ACM (2008)Google Scholar
  15. 15.
    Kayal, A., Brinkman, W.P., Neerincx, M.A. van Riemsdijk, M.B.: A social commitment model for location sharing applications in the family domain. Int. J. Hum. Comput. Stud. (to appear)Google Scholar
  16. 16.
    Kohls, C.: The theories of design patterns and their practical implications exemplified for e-learning patterns. Dissertation, Katolischen Universität Eichstätt-Ingolstadt. Eichstätt (2013)Google Scholar
  17. 17.
    Mioch, T., Ledegang, W., Paulissen, R., Neerincx, M.A., Van Diggelen, J.: Interaction design patterns for coherent and re-usable shape specifications of human-robot collaboration. In: EICS 2014 (Rome, Italy, June 17–20), pp. 75–83. ACM (2014)Google Scholar
  18. 18.
    Murphy, R.R.: Disaster Robotics. MIT Press, Cambridge (2014)Google Scholar
  19. 19.
    NATO. The NATO Unmanned Aircraft Systems Human Systems Integration Guidebook (2012)Google Scholar
  20. 20.
    NATO. The NATO Human View Handbook (2007)Google Scholar
  21. 21.
    Neerincx, M.A.: Situated cognitive engineering for crew support in space. Pers. Ubiquitous Comput. 15(5), 445–456 (2011)CrossRefGoogle Scholar
  22. 22.
    Reising, J. (ed.): Uninhabited Military Vehicles (UMVs): Human factors Issues in Augmenting the Force. NATO RTO technical report RTO-TR-HFM-078. NATO, Brussel (2009)Google Scholar
  23. 23.
    Schulte, A., Donath, D., Lange, D.S.: Design patterns for human-cognitive agent teaming. In: 13th Conference on Engineering Psychology & Cognitive Ergonomics. LNCS. SpringerGoogle Scholar
  24. 24.
    Schümmer, T., Lukosch, S.: Patterns for Computer-Mediated Interaction. Wiley, Chichester (2007)Google Scholar
  25. 25.
    Tidwell, J.: Designing Interfaces. O’Reilly Media, Inc., Sebastopol (2010)Google Scholar
  26. 26.
    Van Breda, L. (ed.): Supervisory Control of Multiple Uninhabited Systems – Methodologies and Enabling Human-Robot Interface Technologies. NATO RTO technical report AC/323(HFM-170)TP/451. NATO, Neuilly-sur-Seine Cedex, France (2012). ISBN:978-92-837-0167-5Google Scholar
  27. 27.
    Van der Aalst, W., Hofstede, A.H.M., Kiepuszewski, B., Barros, A.P.: Workflow patterns. Distributed Parallel Databases 14(1), 5–51 (2003)CrossRefGoogle Scholar
  28. 28.
    Van Welie, M., Van der Veer, G.C.: Pattern languages in interaction design: structure and organization. In: Proceedings of Interact 2003, Zürich, Switzerland, pp. 527–534. IOS Press, Amsterdam, 1–5 September 2003Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Mark A. Neerincx
    • 1
    Email author
  • Jurriaan van Diggelen
    • 1
  • Leo van Breda
    • 1
  1. 1.TNOSoesterbergNetherlands

Personalised recommendations