Abstract
It is hypothesized that telerobotic operations, under altered gravity, can be improved by offering enhanced sources of sensory information to the astronaut. This paper summarizes preliminary results from Earth-ground testing carried out in preparation for an International Space Station (ISS) experiment with astronaut subjects (targeted for mid-to-late 2019). Cognitive and physical performance was compared in 10 operators remotely controlling a prototype rover on a lunar analogue, under different sensory conditions provided by a telerobotics user-interface - visual/visual plus auditory stimuli for a navigation task; visual plus auditory, visual plus somatosensory (force feedback), and visual plus auditory and somatosensory stimuli for a sample collection task. Three time-delay conditions were simulated: 0 s, 0.5 s and 1 s; corresponding to telerobotic operations on the lunar surface (via pressurized rovers or space habitats), from cis-lunar space and from the ISS, respectively. The audio feedback was delayed in all cases with 150 ms. Results indicate that auditory stimuli optimized operators’ performance in navigation and sample collection tasks when proceeding visual and force feedback stimuli at 350 ms and 850 ms. Combined visual and auditory stimuli improved operators’ performance in navigation tasks and 0 s time-delay conditions. At the same time combined visual and auditory feedback are lowering task completion times, mental and physical load when compared to visual stimuli only. Thus, this combination could improve telerobotic operations from pressurized rovers or space habitats. In contrast, not adapted force feedback was associated with great instability in operators’ performance during a sample collection task with increased time-delays – also observed when auditory stimuli preceded visual and force feedback stimuli at 350 ms and 850 ms. Increased time-delays: 0.5 s and 1 s - were linked to increases in task completion times in operators.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Human-Enhanced Robotic Architecture and Capability for Lunar Exploration and Science (HERACLES) project - a project to improve deep space exploration capabilities and generate groundbreaking opportunities for lunar exploration through human-robotic partnership, via cooperation of multiple space agencies.
- 2.
This research project is integrated in the METERON project: an European initiative to help prepare for future human-robot exploration missions to the moon, Mars and other celestial bodies.
- 3.
Similar to the laptop computer to be used in the ISS experiment.
- 4.
Magnitude of acceleration represented via the square root of the sum of the squares of each axis of data (kCalories/min).
References
Fong, T., Zumbado, J.R., Currie, N., Mishkin, A., Akin, D.L.: Space telerobotics: unique challenges to human-robot collaboration in space. Rev. Hum. Factors Ergon. 9, 6–56 (2013)
Bualat, M., Fong, T., Allan, M.B., Bouyssounouse, X., Cohen, T., Fluckiger, L., Gogna, R., Kobayashi, L., Lee, G., Lee, S., Provencher, C.: Surface telerobotics: development and testing of a crew controlled planetary rover system. In: AIAA SPACE 2013 Conference and Exposition, AIAA SPACE Forum, 1–10 (2013)
Grenouilleau, J., Carey, W.: Lunar exploration precursor (LEAP) mission description document. ESTEC - European Space Agency (2017)
Podnar, G., Dolan, J.M., Elfes, A.: Telesupervised robotic systems and the human exploration of Mars. J. Cosmol. 12, 4058–4067 (2010)
NASA Spaceflight. NASA examines options and flight paths for SLS EM-2 mission. https://www.nasaspaceflight.com/2016/03/nasa-examines-options-flight-paths-sls-em-2/. Last accessed 7 Nov 2017
Ennis, M., Fagan, A., Pogue, J., Porter, S., Snape, J.: Feasibility assessment of all science concepts within south pole-Aitken basin. In: Kring, D. (eds.) A Global Lunar Landing Site Study to Provide the Scientific Context for Exploration of the Moon, pp. 477–562 (2012)
Lester, D., Thronson, H.: Human space exploration and human spaceflight: latency and the cognitive scale of the universe. Space Policy 27(2), 89–93 (2011)
Giang, W., Santhakumaran, S., Masnavi, E., Glussich, D., Kline, J., Chui, F., Burns, C.M., Histon, J.M., Zelek, J.: Multimodal interfaces. literature review of ecological interface design, multimodal perception and attention, and intelligent adaptive multimodal interfaces. In: Defence R&D Canada Contract Report, 1-269 (2010)
Gutfreund, Y., King, A.: What is the role of vision in the development of the auditory space map? In: The Handbook of Multisensory Processing, pp. 574–587. The MIT Press, Cambridge (2012)
Lester, D., Hodges, K.V., Anderson, R.C.: Exploration telepresence: a strategy for optimizing scientific research at remote space destinations. Sci. Robot. 2(7), 1–2 (2017)
Minsky, M.: Telepresence. In: Omni magazine, pp. 45–51 (1980)
Sheridan, T.B.: Teleoperation, telerobotics and telepresence: a progress report. Control. Eng. Pract. 3(2), 205–214 (1995)
Aracil, R., Buss, M., Cobos, S., Ferre, M., Hirche, S., Kuschel, M., Peer, A.: The human role in telerobotics. Advances in telerobotics. In: STAR 31, pp. 11–24 (2007)
Durlach, N.I., Mavor, A.S.: Virtual Reality: Scientific and Technological Challenges. National Academy Press, Washington, DC (1995)
Schiele, A., Aiple, M., Kruger, T., van der Hulst, F., Kimmer, S., Sisek, J., Emiel, D.: Haptics-1: preliminary results from the first stiffness JND identification experiment in space. In: Bello, F., et al. (eds.) EuroHaptics 2016, I, pp. 13–22 (2016)
Nitsch, V., Farber, B.: A meta-analysis of the effects of haptic interfaces on task performance with teleoperation systems. IEEE Trans. Haptics 6(4), 387–398 (2013)
Wildenbeest, J.G.: The impact of haptic feedback quality on the performance of teleoperated assembly tasks. IEEE Trans. Haptics 6(2), 242–252 (2013)
Weber, B., Schätzle, S., Riecke, C., Brunner, B., Tarassenko, S., Artigas, J., Balachandran, R., Albu-Schäffer, A.: Weight and weightlessness effects on sensorimotor performance during manual tracking. In: Bello, F., et al. (eds.) EuroHpatics 2016, pp. 1–21 (2016)
Oosterhout, J.V., Wildenbeest, J.G., Boessenkool, H., Heemskerk, C.J., de Baar, M.R., van der Helm, F.C., Abbink, D.A.: Haptic shared control in tele-manipulation: effects of inaccuracies in guidance on task execution. IEEE Trans. Haptics 8(2), 164–175 (2015)
Clément, G.: Fundamentals of Space Medicine. Springer Science and Business Media, New York (2011)
Manzey, D., Lorenz, T.B., Heuers, H., Sangals, J.: Impairments of manual tracking performance during spaceflight: more converging evidence from a 20-day space mission. Ergonomics 43(5), 589–609 (2000)
Paloski, W.H., Oman, C.M., Bloomberg, J.J., Reschke, M.F., Wood, S.J., Harm, D.L., Peters, B.T., Mulavara, A.P., Locke, J.P., Stone, L.S.: Risk of sensory-motor performance failures affecting vehicle control during space missions: a review of the evidence. J. Gravit. Physiol. 15, 1–29 (2008)
Schacter, D., Gilbert, D., Wegner, D.: Psychology. European Edition. Palgrave Macmillan, New York (2011)
Stein, B.: The New Handbook of Multisensory Processing. The MIT Press, Cambridge (2012)
Sweller, J., Ayres, P., Kalyuga, S.: Cognitive Load Theory. Springer Science, New York (2011)
Gutfreund, Y., King, A.: What is the role of vision in the development of the auditory space map? In: The Handbook of Multisensory Processing, pp. 574–587. The MIT Press, Cambridge (2012)
Hagmann, C.E., Russo, N.: Multisensory integration of redundant trisensory stimulation. Atten. Percept. Psychophys. 78(8), 1–21 (2016)
Todd, J.W.: Reaction to multiple stimuli. In: Archives of Psychology, 25. Science Press, New York (1912)
Massimino, M.J., Sheridan, T.B.: Sensory substitution for force feedback in teleoperation. IFAC Proc. Vol. 25(9), 109–114 (1992)
McMahan, W., Gewirtz, J., Standish, D., Martin, P., Kunkel, J.A., Lilavois, M., Wedmid, A., Lee, D.I., Kuchenbecker, K.: Tool contact acceleration feedback for telerobotic surgery. IEEE Trans. Haptics 4(3), 210–220 (2011)
Wozny, D.R., Beierholm, U.R., Shams, L.: Human trimodal perception follows optimal statistical inference. J. Vis. 8(3), 1–11 (2008)
Bresciani, J.P., Ernst, M.O., Drewing, K., Bouyer, G., Maury, V., Kheddar, A.: Feeling what you hear: auditory signals can modulate tactile tap perception. Exp. Brain Res. 162, 172–180 (2005)
Smith, T.J., Smith, K.U.: The human factors of workstation telepresence. In: Nasa, Third Annual Workshop on Space Operations Automation and Robotics (SOAR 1989), pp. 235–250 (1990)
Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D., Craven, P.L.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat., Space, Environ. Med. 78(5), B231–B244 (2007)
Galvan, R.: Effects of fatigue on simulated space telerobotics performance: a preliminary study analysis. Masters dissertation, Department of Aeronautics and Astronautics. MIT (2012)
NASA Evidence Report. Risk of performance decrements and adverse health outcomes resulting from sleep loss, circadian desynchronization, and work overload. In: Human Research Program. Behavioral Health and Performance Element, pp. 1–83 (2016)
Fowler, B., Meehan, S., Singhal, A.: Perceptual-motor performance and associated kinematics in space. Hum. Factors 50(6), 879–892 (2008)
Monk, T.H., Buysse, D.J., Billy, B.D., Kennedy, K.S., Willrich, L.M.: Sleep and circadian rhythms in four orbiting astronauts. J. Biol. Rhythm. 13(3), 188–201 (1998)
Williamson, P.: MSS Lessons Learned, DX23/ISS Mechanical & Robotic Systems Training Lesson handout. Houston, TX, NASA JSC DX23 ISS Mechanical and Robotic Systems Training, 7 (2007)
Berka, C., Levendowski, D.J., Cvetinovic, M., Petrovic, M., Davis, G., Lumicao, M., Zivkovic, V.T., Popovic, M.V., Olmstead, R.: Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. Int. J. Hum.-Comput. Interact. 17, 151–170 (2004)
Posner, M.I.: Measuring Alertness. NY Acad. Sci. 1129, 193–199 (2008)
Sasaki, J.E., John, D., Freedson, P.S.: Validation and comparison of ActiGraph activity monitors. J. Sci. Med. Sport 14(5), 411–416 (2011)
Meredith, M.A., Nemitz, J.W., Stein, B.E.: Determinants of multisensory integration insuperior colliculus neurons. I. Temporal factors. J. Neurosci. 7(10), 3215–3229 (1987)
Thomson, J.M., Ottensmeyer, M.P., Sheridan, T.B.: Human factors in telesurgery: effects of time-delay and asynchrony in video and control feedback with local manipulative assistance. Telemed. J. 5(2), 129–130 (1999)
Onda, K., Osa, T., Sugita, N., Hashizume, M., Mitsuishi, M.: Asynchronous force and visual feedback in teleoperative laparoscopic surgical system. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 844–849 (2010)
Acknowledgments
This study was supported by METERON and Fundacao para a Ciencia e Tecnologia, Portugal. We thank to HRE-S and Dr. Julia Teles.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ferraz, M. et al. (2019). Multisensory Real-Time Space Telerobotics. In: Arai, K., Bhatia, R., Kapoor, S. (eds) Intelligent Computing. CompCom 2019. Advances in Intelligent Systems and Computing, vol 997. Springer, Cham. https://doi.org/10.1007/978-3-030-22871-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-22871-2_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22870-5
Online ISBN: 978-3-030-22871-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)