Self-scaling Human-Agent Cooperation Concept for Joint Fighter-UCAV Operations

  • Florian ReichEmail author
  • Felix Heilemann
  • Dennis Mund
  • Axel Schulte
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 499)


In this article, we describe human automation integration concepts that allow the guidance and the mission management of multiple UCAVs (Unmanned Combat Aerial Vehicles) from aboard a manned single-seat fighter aircraft. The conceptual basis of our approach is dual-mode cognitive automation. This concept uses two distinct modes of human-agent cooperation, a hierarchical relationship with agents working in delegation mode, and a heterarchical relationship with an agent working in assistance mode. For the hierarchical relationship we suggest three delegation modes (team-, intent-, and task-based). The agent in heterarchical relationship, i.e. the assistant system, adapts the operator-assistant system cooperation and the guidance of UCAVs according to the named delegation modes. The adaptation is shaped by the assessment of the operator’s mental state and external situation features. Thereby, we aim at balancing the operator’s activity and work demands. Future research at our institute will concentrate on developing a software prototype for human-in-the-loop experiments.


Self-scaling automation Human-agent cooperation Dual-mode cognitive automation Assistant system Multiple UCAV guidance Delegation modes Operator-centered automation adaption 


  1. 1.
    Chappelle, W.L., McDonald, K., McMillan, K.: Important and Critical Psychological Attributes of USAF MQ-1 Predator and MQ-9 Reaper Pilots According to Subject Matter Experts, pp. 1–35 (2011)Google Scholar
  2. 2.
    de Freitas, M.M., Cunha, F.S., Ribeiro, A.M.R., Azinheira, J.R., Carvalho, R.J.S., Cabrita Freitas, J., Avalle, M.: UCAV : A Technology Assessment Project as a Complex Problem Case Study (2009)Google Scholar
  3. 3.
    Meitinger, C., Schulte, A.: Cognitive machine co-operation as basis for guidance of multiple UAVs. In: NATO RTO HFM Symposium on Human Factors of Uninhabited Military Vehicles as Force Multipliers. Biarritz, France (2006)Google Scholar
  4. 4.
    Meitinger, C., Schulte, A.: Human-UAV co-operation based on artificial cognition. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 91–100 (2009)Google Scholar
  5. 5.
    Gangl, S., Lettl, B., Schulte, A.: Management of multiple unmanned combat aerial vehicles from a single-seat fighter cockpit in manned-unmanned fighter missions. In: AIAA Infotech@Aerospace (I@A) Conference, pp. 1–18. American Institute of Aeronautics and Astronautics, Reston, Virginia (2013)Google Scholar
  6. 6.
    Gangl, S.: Kooperative Führung mehrerer unbemannter Luftfahrzeuge aus einem einsitzigen Kampfflugzeug (2015)Google Scholar
  7. 7.
    Lettl, B., Schulte, A.: Self-explanation capability for cognitive agents on-board of UCAVs to improve cooperation in a manned-unmanned fighter team. In: AIAA Infotech@Aerospace (I@A) Conference, pp. 1–11. American Institute of Aeronautics and Astronautics, Reston, Virginia (2013)Google Scholar
  8. 8.
    Wiener, E.L.: Cockpit automation. In: Wiener, E.L., Nagel, D.C. (eds.) Human Factors in Aviation, pp. 433–461. Academic Press, London (1988)Google Scholar
  9. 9.
    Wickens, C.D., Hollands, J.G.: Engineering Psychology and Human Performance. Prentice Hall, Upper Saddle River (2000)Google Scholar
  10. 10.
    Billings, C.E.: Aviation Automation: The Search for a Human-Centered Approach. Lawrence Erlbaum Associates, Mahwah (1997)Google Scholar
  11. 11.
    Kaber, D.B., Endsley, M.R.: Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety. Process Saf. Prog. 16, 126–131 (1997)CrossRefGoogle Scholar
  12. 12.
    Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse, abuse. Hum. Factors J. Hum. Factors Ergon. Soc. 39, 230–253 (1997)Google Scholar
  13. 13.
    Endsley, M.R., Kiris, E.O.: The out-of-the-loop performance problem and level of control in automation. Hum. Factors J. Hum. Factors Ergon. Soc. 37, 381–394 (1995)Google Scholar
  14. 14.
    Sarter, N.B., Woods, D.D.: Team play with a powerful and independent agent: operational experiences and automation surprises on the Airbus A-320. Hum. Factors J. Hum. Factors Ergon. Soc. 39, 553–569 (1997)Google Scholar
  15. 15.
    Wiener, E.L., Curry, R.E.: Flight-deck automation: promises and problems. Ergonomics 23, 995–1011 (1980)CrossRefGoogle Scholar
  16. 16.
    Kidwell, B., Calhoun, G.L., Ruff, H.A., Parasuraman, R.: Adaptable and adaptive automation for supervisory control of multiple autonomous vehicles. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 56, 428–432 (2012)Google Scholar
  17. 17.
    Onken, R., Schulte, A.: System-Ergonomic Design of Cognitive Automation: Dual-Mode Cognitive Design of Vehicle Guidance and Control Work Systems. Springer, Berlin (2010)Google Scholar
  18. 18.
    Theißing, N.: Designing a support system to mitigate pilot error while minimizing out-of-the-loop-effects. In: 13th Conference on Engineering Psychology and Cognitive Ergonomics. Lecture Notes Computer Science (2016)Google Scholar
  19. 19.
    Uhrmann, J., Schulte, A.: Task-based guidance of multiple uav using cognitive automation. In: COGNITIVE 2011, The Third International Conference on Advanced Cognitive Technologies and Applications, pp. 47–52 (2011)Google Scholar
  20. 20.
    Rauschert, A., Schulte, A.: Cognitive and cooperative assistant system for aerial manned-unmanned teaming missions. NATO Res. Technol. Agency, Hum. Factors Med. Panel, Task Gr. HFM-170 Superv. Control Mult. Uninhabited Syst. Methodol. Enabling Oper. Interface Technol. RTO-TR-HFM 170, 1–16 (2012)Google Scholar
  21. 21.
    Schulte, A., Donath, D., Lange, D.S.: Design patterns for human-cognitive agent teaming. In: 13th Conference on Engineering Psychology and Cognitive Ergonomics. Lecture Notes Computer Science (2016)Google Scholar
  22. 22.
    Bradshaw, J.M., Feltovich, P.J., Jung, H., Kulkarni, S., Taysom, W., Uszok, A.: Dimensions of adjustable autonomy and mixed-initiative interaction. In: Agents and Computational Autonomy, pp. 17–39. Springer, Berlin (2003)Google Scholar
  23. 23.
    Schulte, A., Donath, D., Honecker, F.: Human-system interaction analysis for military pilot activity and mental workload determination. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1375–1380. IEEE (2015)Google Scholar
  24. 24.
    Klein, G., Woods, D.D., Bradshaw, J.M., Hoffman, R.R., Feltovich, P.J.: Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intell. Syst. 19, 91–95 (2004)CrossRefGoogle Scholar
  25. 25.
    Bevacqua, G., Cacace, J., Finzi, A., Lippiello, V.: Mixed-initiative planning and execution for multiple drones in search and rescue missions. In: ICAPS, pp. 315–323 (2015)Google Scholar
  26. 26.
    Baxter, J.W., Horn, G.S., Leivers, D.P.: Fly-by-agent: controlling a pool of UAVs via a multi-agent system. Knowledge-Based Syst. 21, 232–237 (2008)CrossRefGoogle Scholar
  27. 27.
    Parasuraman, R., Barnes, M., Cosenzo, K., Mulgund, S.: Adaptive Automation for Human-Robot Teaming in Future Command and Control Systems. DTIC Document (2007)Google Scholar
  28. 28.
    Greef, T., Lafeber, H., Oostendorp, H., Lindenberg, J.: Eye movement as indicators of mental workload to trigger adaptive automation. In: Schmorrow, D.D., Estabrooke, I.V, Grootjen, M. (eds.) Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience: 5th International Conference, pp. 219–228. Springer, Berlin (2009)Google Scholar
  29. 29.
    Wilson, G.F., Russell, C.A.: Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Hum. Factors J. Hum. Factors Ergon. Soc. 49, 1005–1018 (2007)Google Scholar
  30. 30.
    Wang, Z., Hope, R.M., Wang, Z., Ji, Q., Gray, W.D.: Cross-subject workload classification with a hierarchical Bayes model. Neuroimage 59, 64–69 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Florian Reich
    • 1
    Email author
  • Felix Heilemann
    • 1
  • Dennis Mund
    • 1
  • Axel Schulte
    • 1
  1. 1.Institute of Flight Systems (IFS)Universität der Bundeswehr Munich (UBM)NeubibergGermany

Personalised recommendations