Skip to main content
SpringerLink
Log in

A Review of NASA Human-Robot Interaction in Space

  • Space Robotics (Y Gao, Section Editor)
  • Published:
Current Robotics Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

This review provides an overview of the motivation, challenges, state-of-the-art, and recent research for human-robot interaction (HRI) in space. For context, we focus on NASA space missions, use cases, and systems (both flight and research). However, the discussion is broadly applicable to all activities in space that require or make use of human-robot teams.

Recent Findings

To date, HRI in space has largely been limited to remote interaction between humans on Earth and robots in space. This interaction is associated with telerobotic operations—from direct (manual) control to intermittent, supervisory control. Recent work, however, has begun to address a wide range of human-robot arrangements (co-located, remote, 1:1, groups, etc.). In addition, researchers have been studying human-robot teaming theory and system design, efficient interaction methods, and human-robot communication.

Summary

We begin by describing NASA’s use of robots in space for both deep space science and human exploration. We then describe several aspects of HRI that are important for space missions, with emphasis on factors that are critical or unique for the space environment. Next, we provide a brief overview of HRI associated with space systems, including technology demonstrations. Finally, we conclude with a short survey of recent research, which will affect human-robot interaction for both Artemis missions and future missions to Mars.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

Papers of particular interest, published recently, have been highlighted as: • Of importance

  1. Ticker RL, Cepollina F, Reed BB. AIAA Space 2015 Conference and Exposition. In: NASA’s in-space robotic servicing; 2015. p. 4644. https://doi.org/10.2514/6.2015-4644.

  2. Callen P. Robotic transfer and interfaces for external ISS payloads. 3rd Annual ISS Research and Development Conference; 2014. https://ntrs.nasa.gov/api/citations/20140008717/downloads/20140008717.pdf.

  3. Rankin A, Maimone M, Biesiadecki J, Patel N, Levine D, Toupet O. Driving curiosity: Mars rover mobility trends during the first seven years. In: 2020 IEEE Aerospace Conference: IEEE; 2020. p. 1–19.

  4. Smith M, Craig D, Herrmann N, Mahoney E, Krezel J, McIntyre N, et al. The Artemis Program: an overview of NASA’s activities to return humans to the moon. In: 2020 IEEE Aerospace Conference: IEEE; 2020. p. 1–10.

  5. Ferketic J, Goldblatt L, Hodgson E, Murray S, Wichowski R, Bradley A, Erkorkmaz C. Toward human–robot interface standards II: A closer examination of common elements in human-robot interactions across the space enterprise. In: Proceedings of the 2006 AIAA Space Conference. https://doi.org/10.2514/6.2006-7388.

  6. Diftler MA, Mehling JS, Abdallah ME, Radford NA, Bridgwater LB, Sanders AM, et al. Robonaut 2-the first humanoid robot in space. In: 2011 IEEE international conference on robotics and automation: IEEE; 2011. p. 2178–83.

  7. Diftler MA, Ahlstrom TD, Ambrose RO, Radford NA, Joyce CA, De La Pena N, et al. Robonaut 2—initial activities on-board the ISS. In: 2012 IEEE Aerospace Conference: IEEE; 2012. p. 1–12.

  8. Fong T, Micire M, Morse T, Park E, Provencher C, To V, Wheeler DW, Mittman D, Torres RJ, Smith E. Smart SPHERES: a telerobotic free-flyer for intravehicular activities in space. In Proceedings of AIAA Space. AIAA-2013-5338 2013; 2013. https://doi.org/10.2514/6.2013-5338.

  9. Smith T, Barlow J, Bualat M, Fong T, Provencher C, Sanchez H, Smith E. Astrobee: a new platform for free-flying robotics on the International Space Station. In: Proceedings of 13th international symposium on artificial intelligence, robotics, and automation in space; 2016. https://ntrs.nasa.gov/citations/20160007769.

  10. Fredrickson SE, Duran S, Howard N, Wagenknecht JD. Application of the mini AERCam free flyer for orbital inspection. In: Spacecraft Platforms and Infrastructure, vol. 5419: International Society for Optics and Photonics; 2004. p. 26–35.

  11. Bualat M, Schreckenghost D, Pacis E, Fong T, Kalar D, Beutter B. (2014) Results from testing crewcontrolled surface telerobotics on the International Space Station. iSAIRAS - International Symposium on Artificial Intelligence, Robotics and Automation in Space, June 17–19, 2014, Montreal, Quebec.

  12. Crusan JC, Smith RM, Craig DA, Caram JM, Guidi J, Gates M, et al. Deep space gateway concept: extending human presence into cislunar space. In: 2018 IEEE Aerospace Conference: IEEE; 2018. p. 1–10.

  13. Mueller RP, Connolly JC, Whitley RJ. NASA human spaceflight ArchitectureTeam: lunar surface exploration strategies. GLEX-2012.02.P.17.x12620; 2012. https://ntrs.nasa.gov/api/citations/20120008182/downloads/20120008182.pdf.

  14. Harrison DA, Ambrose R, Bluethmann B, Junkin L. Next generation rover for lunar exploration. In: 2008 IEEE aerospace conference: IEEE; 2008. p. 1–14.

  15. Fong T, Rochlis Zumbado J, Currie N, Mishkin A, Akin DL. Space telerobotics: unique challenges to human–robot collaboration in space. Rev Hum Factors Ergon. 2013;9(1):6–56.

    Article  Google Scholar 

  16. • Marquez J. Unique considerations for human-robotic interaction in human spaceflight. In: Kanki B, Clervoy J-F, Sandal G, editors. Space Safety and Human Performance: Elsevier; 2017. This book chapter provides information unique to NASA for integrating humans and robotics into human spaceflight missions.

  17. IJtsma M, Ma L, Pritchett A, Feigh K. Computational methodology for the allocation of work and interaction in human-robot teams. J Cogn Eng Decis Mak. 2019;13(4):221–41.

    Article  Google Scholar 

  18. Marquez JJ, Feary M, Rochlis JZ, Billman D. Evidence report: risk of inadequate design of human and automation/robotic integration. Houston: NASA Johnson Space Center; 2013.

    Google Scholar 

  19. Ferketic J, Goldblatt L, Hodgson E, Murray S, Wichowski R, Bradley A, et al. Toward human-robot interface standards: Use of standardization and intelligent subsystems for advancing human-robotic competency in space exploration: SAE Technical Paper; 2006.

  20. Fong T, Nourbakhsh I. Interaction challenges in human-robot space exploration. ACM Interact. 2005;12(2):42–5.

    Article  Google Scholar 

  21. Ferketic J, Goldblatt L, Hodgson E, Murray S, Wichowski R, Bradley A, et al. Toward human–robot interface standards II: a closer examination of common elements in human-robot interactions across the space enterprise. In: Proceedings of the 2006 AIAA Space Conference; 2006.

    Google Scholar 

  22. Fong T, Berka R, Bualat M, Diftler M, Micire M, Mittman D, SunSpiral V, Provencher C. The Human Exploration Telerobotics Project. GLEX-2012.01.2.4x12180. In: Proceedings of IAF/AIAA Global Space Exploration Conference. Washington, DC; 2012.

  23. Chang M, Marquez JJ. Human-automation allocations for current robotic space operations: space station remote manipulator system. NASA TM -2018-220042. Moffett Field: NASA Ames Research Center; 2018.

    Google Scholar 

  24. Grotzinger JP, Crisp J, Vasavada AR, Anderson RC, Baker CJ, Barry R, et al. Mars Science Laboratory mission and science investigation. Space Sci Rev. 2012;170(1-4):5–6.

    Article  Google Scholar 

  25. Carsten J, Rankin A, Ferguson D, Stentz A. Global planning on the Mars Exploration Rovers: software integration and surface testing. J Field Rob. 2009;26(4):337–57.

    Article  Google Scholar 

  26. McCurdy M. Planning tools for Mars surface operations: human-computer interaction lessons learned. In: 2009 IEEE Aerospace conference: IEEE; 2009. p. 1–12.

  27. Norris JS, Powell MW, Vona MA, Backes PG, Wick JV. Mars exploration rover operations with the science activity planner. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation: IEEE; 2005. p. 4618–23.

  28. Fong T, Bualat M, Burns J, Hopkins J, Pratt W. Testing astronaut-controlled telerobotic operation of rovers from the International Space Station as a precursor to lunar missions. IAC-14-A3-2A-7. In: Proceedings of the 65th International Astronautical Congress. Toronto, Canada; 2014. https://ntrs.nasa.gov/api/citations/20190028898/downloads/20190028898.pdf.

  29. Bualat M, Schreckenghost D, Pacis E, Fong T, Kalar D, Beutter B. Results from testing crew-controlled surface telerobotics on the International Space Station. In International Symposium on Artificial Intelligence, Robotics, and Automation in Space, June 17, 2014' 2014. https://ntrs.nasa.gov/citations/20150007985.

  30. Ahlstrom T, Diftler M, Berka R, Badger J, Yayathi S, Curtis A, Joyce C. Robonaut 2 on the International Space Station: Status update and preparations for IVA mobility. In: Proceedings of AIAA Space; 2013. https://doi.org/10.2514/6.2013-5340.

  31. Farrell L, Strawser P, Hambuchen K, Baker W, Badger J. Supervisory control of a humanoid robot in microgravity for manipulation tasks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS): IEEE; 2017. p. 3797–802.

  32. Ma LM, Fong T, Micire MJ, Kim YK, Feigh K. Human-robot teaming: concepts and components for design. In: Field and service robotics. Cham: Springer; 2018. p. 649–63.

    Google Scholar 

  33. Fong T, Scholtz J, Shah JA, Fluckiger L, Kunz C, Lees D, et al. A preliminary study of peer-to-peer human-robot interaction. In: 2006 IEEE International Conference on Systems, Man and Cybernetics, vol. 4: IEEE; 2006. p. 3198–203.

  34. Fong T, Nourbakhsh I. Peer-to-peer human-robot interaction for space exploration. In: Proceedings of the AAAI Fall Symposium: The Intersection of Cognitive Science and Robotics: From Interfaces to Intelligence; 2004. https://www.aaai.org/Papers/Symposia/Fall/2004/FS-04-05/FS04-05-014.pdf.

  35. Banerjee S, Gombolay M, Chernova S. A tale of two suggestions: action and diagnosis recommendations for responding to robot failure. In: 2020 29th IEEE international conference on robot and human interactive communication (RO-MAN): IEEE. p. 398–405.

  36. Bales G, Kong Z. Neurophysiological and behavioral correlates of human-Multiagent task performance. Manuscript; 2020.

  37. Kintz JR, Clark TK. NASA human research program investigators’ workshop: Galveston; 2020. p. 27–30.

  38. Maghareh A, Lenjani A, Dyke SJ, Marais K, Whitaker D, Bobet A, Ramirez J, Modiriasari A, Theinat A. Resilience-oriented design of extraterrestrial habitat systems. In: AIAA propulsion and energy 2019 forum; 2019. p. 3972. https://doi.org/10.2514/6.2019-3972.

  39. Gervits F, Fong TW, Scheutz M. Shared mental models to support distributed human-robot teaming in space. In: 2018 AIAA SPACE and astronautics forum and exposition; 2018. p. 5340. https://doi.org/10.2514/6.2018-5340.

  40. Gervits F, Thurston D, Thielstrom R, Fong T, Pham Q, Scheutz M. Toward genuine robot teammates: improving human-robot team performance using robot shared mental models. In: Proceedings of the 19th international conference on autonomous agents and MultiAgent systems; 2020. p. 429–37. https://dl.acm.org/doi/pdf/10.5555/3398761.3398815.

  41. Reig S, Carter E, Fong T, Forlizzi J, Steinfeld A. Flailing, hailing, prevailing: perceptions of multi-robot failure recovery strategies. To appear in proceedings of the ACM conference on human-robot interaction; 2021. https://dl.acm.org/doi/10.1145/3434073.3444659.

  42. McGuire S, Furlong PM, Fong T, Heckman C, Szafir D, Julier SJ, et al. Everybody needs somebody sometimes: validation of adaptive recovery in robotic space operations. IEEE Robot Autom Lett. 2019;4(2):1216–23.

    Article  Google Scholar 

  43. Kent D, Saldanha C, Chernova S. A comparison of remote robot teleoperation interfaces for general object manipulation. In: Proceedings of the 2017 ACM/IEEE international conference on human-robot interaction; 2017. p. 371–9. https://dl.acm.org/doi/abs/10.1145/2909824.3020249.

  44. Kent D, Saldanha C, Chernova S. Leveraging depth data in remote robot teleoperation interfaces for general object manipulation. Int J Robot Res. 2020;39(1):39–53.

    Article  Google Scholar 

  45. Kent D, Chernova S. Human-centric active perception for autonomous observation. In: Proceedings of the IEEE international conference on robotics and automation; 2020. https://doi.org/10.1109/ICRA40945.2020.9197201.

  46. Schreckenghost DL, Milam T, Fong T. Techniques and tools for summarizing performance of robots operating remotely. In: 14th international conference on space operations; 2016. p. 2310. https://doi.org/10.2514/6.2016-2310.

  47. Schreckenghost D, Fong T, Utz H, Milam T. Measuring robot performance in real-time for NASA robotic reconnaissance operations. In: Proceedings of the 9th workshop on performance metrics for intelligent systems; 2009. p. 194–202. https://dl.acm.org/doi/10.1145/1865909.1865950.

  48. Schreckenghost D, Milam T, Fong T. Measuring performance in real time during remote human-robot operations with adjustable autonomy. IEEE Intell Syst. 2010;23(5):36–45.

    Article  Google Scholar 

  49. • Lasota P, Fong T, Shah J. A survey of methods for safe human-robot interaction. Foundation and Trends in Robotics. 2017;5(4). https://doi.org/10.1561/2300000052. This book discusses safety implications of human robot interaction and methods to increase safe HRI for both the physical and mental well-being of the humans.

  50. Kim H, Park YW, Baker E, Adams J, Fong T. Design issues for real-time remote robotic science operations support tools: observations from the field. In: 14th international conference on space operations; 2016. p. 2476.

  51. Hedayati H, Walker M, Szafir D. Improving collocated robot teleoperation with augmented reality. In: Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction; 2018. p. 78–86. https://dl.acm.org/doi/10.1145/3171221.3171251.

  52. Walker M, Hooman H, Lee J, Szafir D. Communicating robot motion intent with augmented reality. In: Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction; 2018. p. 316–24. https://dl.acm.org/doi/abs/10.1145/3171221.3171253.

  53. Arumugam D, Karamcheti S, Gopalan N, Williams EC, Rhee M, Wong LL, et al. Grounding natural language instructions to semantic goal representations for abstraction and generalization. Auton Robot. 2019;43(2):449–68.

    Article  Google Scholar 

  54. Arkin J, Park D, Roy S, Walter MR, Roy N, Howard TM, et al. Multimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions. Int J Robot Res. 2020;5:0278364920917755.

    Google Scholar 

  55. Szafir D, Mutlu B, Fong T. Communicating directionality in flying robots. In: 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI): IEEE; 2015. p. 19–26.

  56. Szafir D, Mutlu B, Fong T. Designing planning and control interfaces to support user collaboration with flying robots. Int J Robot Res. 2017;36(5-7):514–42.

    Article  Google Scholar 

  57. Cha E, Fitter NT, Kim Y, Fong T, Matarić M. Generating expressive light signals for appearance-constrained robots. In: International Symposium on Experimental Robotics. Cham: Springer; 2018. p. 595–607.

    Google Scholar 

  58. Cha E, Fitter NT, Kim Y, Fong T, Matarić MJ. Effects of robot sound on auditory localization in human-robot collaboration. In: Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction; 2018. p. 434–42. https://dl.acm.org/doi/10.1145/3171221.3171285.

  59. Cha E, Kim Y, Fong T, Mataric M. 2018. A survey of nonverbal signaling methods for non-humanoid robots. Found Trends Robot. 2018;6(4):211–323.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NASA Johnson Space Center, Houston, TX, USA

    Kimberly Hambuchen

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Jessica Marquez & Terrence Fong

Authors
  1. Kimberly Hambuchen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jessica Marquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terrence Fong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimberly Hambuchen.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Space Robotics

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hambuchen, K., Marquez, J. & Fong, T. A Review of NASA Human-Robot Interaction in Space. Curr Robot Rep 2, 265–272 (2021). https://doi.org/10.1007/s43154-021-00062-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43154-021-00062-5

Keywords

Search

Navigation