Abstract
Human-autonomy teaming is essential for future military operations, but guidance on how to design controls and displays to support joint decision-making and task completion is scarce. Transportation planning experts, drawing from their design experience, recently compiled questions that they postulated would be useful to guide human-autonomy interaction designs for other domains. To explore their generalizability, these questions were employed as a framework for describing the human-autonomy teaming interfaces developed for an implemented prototype military multi-unmanned vehicle (UV) control station. This process demonstrated that the proposed questions indeed address pertinent UV control station requirements. Their “situation driver” questions are aligned with the station’s expanded play-based adaptable automation approach, whereby a wide spectrum of control from manual to fully autonomous can be flexibly applied across UVs/mission tasks to enable shared human-autonomy workload. Their questions regarding “visualizations and control mechanisms” would have likely led to the UV station’s design that features pictorial icons to present actionable concise information in an integrated manner and intuitively support calling UV management plays. Finally, the questions addressing autonomy’s “solution generation and presentation” are correlated to the prototype’s interfaces for presenting multiple courses of action and ongoing play status. Thus, the questions inspired by military airlift planning should also be useful for human-autonomy interface design for the UV command and control domain. Suggested enhancements to the question set are provided as well as detailed recommendations on research needed to extend the human-autonomy teaming interfaces to better support bi-directional communication and directability in more complex, collaborative UV missions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atahary T, Taha T, Douglass SA, Webber F (2015) Knowledge mining for cognitive agents through path based forward checking. Proc of the 16th IEEE/ACIS Intern Conf on Software Eng, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD 2015), Takamatsu, Japan
Bartik J, Rowe A, Draper M, Frost E et al (2020) Autonomy Strategic Challenge (ASC) Allied IMPACT final report. TTCP technical report (ASC-01-2020)
Bartik J, Ruff H, Behymer K, Frost E, Calhoun G, Spriggs S, Hammack T (2017) Mission-coded map icon decision aids for play-based multi-unmanned vehicle autonomy delegation. Proc Hum Factors Ergon Soc Annu Meet 61(1):237–241
Behymer K (2017) Interface design for the supervisory control of multiple heterogeneous unmanned vehicles. Dissertation, Wright State University. Available at: https://etd.ohiolink.edu/!etd.send_file?accession=wright1495016420685509&disposition=inline. Accessed Jun 2020
Behymer KJ, Mersch EM, Ruff HA, Calhoun GL, Spriggs SE (2015) Unmanned vehicle plan comparison visualizations for effective human-autonomy teaming. Procedia Manufacturing 3:1022–1029
Behymer K, Ruff H, Calhoun G, Bartik J, Frost E (2018) Presentation of autonomy-generated plans: determining ideal number and extent differ. Intern Conf on Applied Hum Factors and Ergon. Springer, Cham, pp. 90–101
Behymer KJ, Ruff HA, Mersch EM, Calhoun GL, Spriggs SE (2015) Visualization methods for communicating unmanned vehicle plan status. Proc of the Intern Symp on Aviation Psychology
Bradshaw JM, Acquisti A, Allen J, Breedy MR, Bunch L, Chambers N et al (2004) Teamwork-centered autonomy for extended human-agent interaction in space applications. Proc of the AAAI 2004 Spring Symp, pp 22–24
Brooke J (1996) SUS: a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester BA (eds) Usability evaluation in industry. Taylor and Francis, London, pp 189–194
Calhoun G, Draper M, Miller C, Ruff H, Breeden C, Hamell J (2013) Adaptable automation interface for multi-unmanned aerial systems control: preliminary usability evaluation. Proc Hum Factors Ergon Soc Annu Meet 57(1):26–30
Calhoun GL, Goodrich MA, Dougherty JR, Adams JA (2016) Human-autonomy collaboration and coordination toward multi-RPA missions. In: Cooke N, Rowe L, Bennett W (eds) Remotely piloted aircraft: a human systems integration perspective. Wiley, New York, chapter 5, pp 101–136
Calhoun GL, Ruff HA, Behymer KJ, Frost EM (2017a) Human-autonomy teaming interface design considerations for multi-unmanned vehicle control. Theor Issues in Ergonomics Sci 19(3):321–352
Calhoun GL, Ruff HA, Behymer KJ, Mersch EM (2017b) Operator-autonomy teaming interfaces to support multi-unmanned vehicle missions. In: Savage-Knepshield P, Chen J (eds) Advances in human factors in robots and unmanned systems: advances in intelligent systems and computing, vol 499. Springer International Publishing, Switzerland, pp 113–126
Defense Science Board (2016) Defense science board summer study on autonomy. Under Secretary of Defense, Washington, D.C. https://www.hsdl.org/?abstract&did=794641. Accessed 22 June 2020
Draper M, Calhoun G, Spriggs S, Evans D, Behymer K (2017) Intelligent multi-UxV planner with adaptive collaborative/control technologies (IMPACT). Proc of the Intern Symp on Aviation Psychology
Draper M, Rowe A, Douglass S, Calhoun G, Spriggs S, Kingston D et al (2018) Realizing autonomy via intelligent hybrid control: adaptable autonomy for achieving UxV RSTA team decision superiority (also known as intelligent multi-UxV planner with adaptive collaborative/control technologies (IMPACT)) (AFRL-RH-WP-TR-2018-0005). Air Force Research Laboratory, Wright-Patterson Air Force Base, Dayton, OH
Frost EM, Bartik J, Calhoun G, Spriggs S, Ruff H, Behymer K (2018) Creating a well-situation human-autonomy team: the effects of team structure. Proc of the NATO hum factors and medicine panel, Symp on hum autonomy teaming (NATO: STO-MP-HFM-300)
Frost E, Calhoun G, Ruff H, Bartik J, Behymer K, Spriggs S, Buchanan A (2019) Collaboration interface supporting human-autonomy teaming for unmanned vehicle management. Inter Symp of Aviation Psych. https://www.sto.nato.int/publications/pages/results.aspx?k=Creating%20a%20well-situated&s=Search%20All%20STO%20Reports
Hansen M (2020) Voice2json: command-line tools for speech and intent recognition on Linux. https://voice2json.org/whitepaper. Accessed 22 June 2020
Hansen M, Calhoun G, Douglass S, Evans D (2016) Courses of action display for multi-unmanned vehicle control: a multi-disciplinary approach. AAAI fall symposium series: cross-disciplinary challenges for autonomous systems, technical report FS-16-03. https://www.aaai.org/ocs/index.php/FSS/FSS16/paper/viewFile/14104/13687
Hutchins AR, Cummings ML, Draper M, Hughes T (2015) Representing autonomous systems’ self-confidence through competency boundaries. Proc Hum Factors Ergon Soc Annu Meet 59(1):279–283
Joe JC, O’Hara J, Medema HD, Oxstrand JH (2014) Identifying requirements for effective human-automation teamwork. Proceedings of the 12th International Conference on Probabilistic Safety Assessment and Management, PSAM-370-1, Honolulu, HI
Johnson M, Bradshaw JM, Hoffman RR, Feltovich PJ, Woods DD (2014) Seven cardinal virtues of human-machine teamwork: examples from the DARPA robotic challenge. IEEE Intell Syst 29(6):74–80
Kaber DB (2018) Issues in human-automation interaction modeling: presumptive aspects of frameworks of types and levels of automation. J Cogn Eng Decis Mak 12(1):7–24
Klein G, Woods DD, Bradshaw JM, Hoffman RR, Feltovich PJ (2004) Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intell Syst 19(6):91–95
Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46(1):50–80
Lewis JR, Sauro J (2009) The factor structure of the system usability scale. Inter Conf on Human Centered Design. Springer, Berlin, pp 94–103
Mersch EM, Behymer KJ, Calhoun GL, Ruff HA, Dewey JS (2016) Game-based delegation interface design for unmanned vehicles color coding and icon row assignment. Proc Hum Factors Ergon Soc Annu Meet 60(1):122–126
Miller CA, Funk HB, Dorneich M, Whitlow SD (2002) A playbook interface for mixed initiative control of multiple unmanned vehicle teams. Proc of the IEEE Digital Avionics Systems Conf, Vol. 2, pp 7E4-7E4
Miller CA, Parasuraman R (2007) Designing for flexible interaction between humans and automation: delegation interfaces for supervisory control. Hum Factors 49(1):57–75
Parasuraman R, Riley V (1997) Humans and automation: use, misuse, disuse, abuse. Hum Factors 39(2):1293–1302
Rasmussen S, Kalyanam K, Kingston D (2016) Field experiment of a fully autonomous multiple UAV/UGS intruder detection and monitoring system. IEEE Inter Conf on Unmanned Aircraft Systems (ICUAS), pp 1293–1302
Roth E, Depass B, Harter J, Scott R, Wampler J (2018) Beyond levels of automation: developing more detailed guidance for human automation interaction design. Proc Hum Factors Ergon Soc Annu Meet 62(1):150–154
Ruff H Calhoun G (2013) Human supervision of multiple autonomous vehicles (AFRL-RH-WP-2013-0143). Air Force Research Laboratory, Wright-Patterson Air Force Base, Dayton, OH
Sarter N (2018) Design-induced error and error-informed design: a two-way street. In: Smith PJ, Hoffman RR (eds) Cognitive systems engineering: the future for a changing world. CRC Press, Chapter 11, pp 209–221
Sarter NB, Woods DD, Billings CE (1997) Automation surprises. In: Salvendy G (ed) Handbook of human factors and ergonomics. Wiley, New York, pp 1926–1943
Schulte A, Donath D, Lange DS (2016) Design patterns for human-cognitive agent teaming. Inter Conf on Eng Psychol and Cogn Ergon. Springer, Cham, pp. 231–243
Shneiderman B, Plaisant C, Cohen M, Jacobs S, Elmqvist N, Diakopoulos N (2016) Designing the user interface: strategies for effective human-computer interaction. Pearson
Smith PJ (2018) Conceptual frameworks to guide design. J Cogn Eng Decis Mak 12(1):50–52
USAF Science and Technology Strategy (2019) Strengthening USAF Science and technology for 2030 and beyond. https://www.airforcemag.com/PDF/DocumentFile/Documents/2019/USAF%20Science%20and%20Technology%20Strategy.pdf19979. Accessed 22 June 2020
Verbancsics P, Lange D. (2013) Using autonomics to exercise command and control networks in degraded environments. 18th Intern Command and Control Research and Technology Symp (DTIC ADA 587015). Alexandria, VA
Vicente KJ, Rasmussen J (1990) The ecology of human-machine systems. II: mediating direct perception in complex work domains. Ecol Psychol 2(3):207–249
Woods DD, Hollnagel E (2006) Joint cognitive systems: foundations of cognitive systems engineering. CRC Press, Boca Raton
Funding
A significant portion of the effort involved in designing and evaluating these human-autonomy interfaces was supported by the United States Office of the Assistant Secretary of Defense for Research and Engineering (ASD) R&E)) through an Autonomy Research Pilot Initiative (ARPI). The United States Air Force Research Laboratory has also supported enhancements of the interfaces instantiated in the Intelligent Multi-UxV Planner with Adaptive Collaborative/Control Technologies (IMPACT) control station.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Calhoun, G., Bartik, J., Ruff, H. et al. Enabling human-autonomy teaming with multi-unmanned vehicle control interfaces. Hum.-Intell. Syst. Integr. 3, 155–174 (2021). https://doi.org/10.1007/s42454-020-00020-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42454-020-00020-0