Skip to main content
Log in

A New Method to Evaluate Human-Robot System Performance

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

One of the key issues in space exploration is that of deciding what space tasks are best done with humans, with robots, or a suitable combination of each. In general, human and robot skills are complementary. Humans provide as yet unmatched capabilities to perceive, think, and act when faced with anomalies and unforeseen events, but there can be huge potential risks to human safety in getting these benefits. Robots provide complementary skills in being able to work in extremely risky environments, but their ability to perceive, think, and act by themselves is currently not error-free, although these capabilities are continually improving with the emergence of new technologies. Substantial past experience validates these generally qualitative notions. However, there is a need for more rigorously systematic evaluation of human and robot roles, in order to optimize the design and performance of human-robot system architectures using well-defined performance evaluation metrics. This article summarizes a new analytical method to conduct such quantitative evaluations. While the article focuses on evaluating human-robot systems, the method is generally applicable to a much broader class of systems whose performance needs to be evaluated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akin, D. Space Systems Laboratory, University of Maryland (http://www.ssl. umd.edu/)

  • Cabrol, N.A. and Grin, E.A. 1999. Distribution, classification, and ages of martian impact crater lakes. Icarus, 142:160.

    Google Scholar 

  • Cooper, B.L. 2001. High-payback technology development for robot associates. In American Institute of Aeronautics and Astronautics, Houston Section, 2000 Annual Technical Symposium.

  • Cooper, B.L. and Donnell, J.O. 2000. Robot associate study. In Oceaneering, Space Systems.

  • Deardorff, A.V. 2001. Website of Deardorff's Glossary of International Economics, University of Michigan. http://wwwpersonal. umich.edu/~alandear/glossary/

  • Fitts, P.M. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6):381–391.

    Google Scholar 

  • Fitts, P.M. et al. 1964. Information capacity of discrete motor responses. Journal of Experimental Psychology, 67(2):103–112.

    Google Scholar 

  • Gell-Mann, M. 1995. What is complexity? Complexity, Wiley Interscience Journal, 1(1).

  • Gell-Mann, M. 1996. The simple and the complex. In Proceedings of the Conference on Complexity, Global Politics, and National Security, D.S. Alberts and T.J. Czerwinski (Ed.), National Defense University: Washington, D.C., pp. 13–14. http://www.ndu.edu/ ndu/inss/books/complexity/index.html

    Google Scholar 

  • Gittleman, M.M. 1988. FTS/EVA task compatibility study. Ocean Systems Engineering.

  • Gittleman, M.M. 1990. Task complexity assessment study. Ocean Systems Engineering.

  • Hoffman, S.J. and Kaplan, D.L. (Eds.). 1997. Human exploration of Mars: The reference mission of the NASA Mars exploration study team. NASA Publication 6107.

  • Malin, M.C. and Edgett, K.S. 2001. Sedimentary rocks of early mars. Science.

  • Matijevic, J. 1998. Autonomous navigation and the sojourner microrover. Science, 276.

  • Nicolis, G. and Prigogine, I. 1989. Exploring Complexity: An Introduction. W.H. Freeman: New York.

    Google Scholar 

  • Papoulis, A. 1965. Probability, Random Variables, and Stochastic Processes, McGraw-Hill: New York.

    Google Scholar 

  • Pollack, J.B. et al. 1987. Case for a wet, warm climate on early Mars. Icarus, 71:203.

    Google Scholar 

  • Shannon, C.E. 1948. A mathematical theory of communication. Bell System Technical Journal, 27:379–423, 623û656.

    Google Scholar 

  • Smith, D.E. et al. 1999. The global topography of Mars and implications for surface evolution. Science, 284:1495–1503.

    Google Scholar 

  • Smith, D.E. et al. 2000. Mars orbiter laser altimeter (MOLA): Experiment summary after the first year of global mapping of Mars. Journal of Geophysical Research, submitted. http://ltpwww.gsfc.nasa.gov/tharsis/mola.html

  • Squyres, S.W. 1989. Ancient lakes on Mars. Icarus, 79:229.

    Google Scholar 

  • Squyres, S.W. and Kasting, J.F. 1994. Early Mars: How warm and how wet? Science, 265:744.

    Google Scholar 

  • Theobold, D. and Roberts, B. 1999. Design of an astronaut assistant for martian exploration. American Institute of Aeronautics and Astronautics.

  • Waldrop, M.M. 1992. Complexity: The Emerging Science at the Edge of Order and Chaos. Simon & Schuster: New York.

    Google Scholar 

  • Wallechinsky, D. 2000. The Complete Book of the Summer Olympics. The Overlook Press: Woodstock and New York.

    Google Scholar 

  • Yagil, G. 1985. On the structural complexity of simple biosystems. Journal of Theoretical Biology, 112:1–23.

    Google Scholar 

  • Yagil, G. 1993. On the structural complexity of templated systems. In 1992 Lectures in Complex Systems, D. Stein and L. Nadel (Eds.), Addison-Wesley and Santa Fe Institute, pp. 519–530.

  • Zarnowski, F. 1996. A Basic Guide to Decathlon, An Official United States Olympic Committee Sports Series. Griffin Publishing: Glendale, CA.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rodriguez, G., Weisbin, C. A New Method to Evaluate Human-Robot System Performance. Autonomous Robots 14, 165–178 (2003). https://doi.org/10.1023/A:1022279618991

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1022279618991

Navigation