Autonomous Robots

, Volume 14, Issue 2–3, pp 165–178 | Cite as

A New Method to Evaluate Human-Robot System Performance

  • G. Rodriguez
  • C.R. Weisbin


One of the key issues in space exploration is that of deciding what space tasks are best done with humans, with robots, or a suitable combination of each. In general, human and robot skills are complementary. Humans provide as yet unmatched capabilities to perceive, think, and act when faced with anomalies and unforeseen events, but there can be huge potential risks to human safety in getting these benefits. Robots provide complementary skills in being able to work in extremely risky environments, but their ability to perceive, think, and act by themselves is currently not error-free, although these capabilities are continually improving with the emergence of new technologies. Substantial past experience validates these generally qualitative notions. However, there is a need for more rigorously systematic evaluation of human and robot roles, in order to optimize the design and performance of human-robot system architectures using well-defined performance evaluation metrics. This article summarizes a new analytical method to conduct such quantitative evaluations. While the article focuses on evaluating human-robot systems, the method is generally applicable to a much broader class of systems whose performance needs to be evaluated.

robotics human-robot systems performance analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akin, D. Space Systems Laboratory, University of Maryland (http://www.ssl. Scholar
  2. Cabrol, N.A. and Grin, E.A. 1999. Distribution, classification, and ages of martian impact crater lakes. Icarus, 142:160.Google Scholar
  3. Cooper, B.L. 2001. High-payback technology development for robot associates. In American Institute of Aeronautics and Astronautics, Houston Section, 2000 Annual Technical Symposium.Google Scholar
  4. Cooper, B.L. and Donnell, J.O. 2000. Robot associate study. In Oceaneering, Space Systems.Google Scholar
  5. Deardorff, A.V. 2001. Website of Deardorff's Glossary of International Economics, University of Michigan. http://wwwpersonal. Scholar
  6. Fitts, P.M. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6):381–391.Google Scholar
  7. Fitts, P.M. et al. 1964. Information capacity of discrete motor responses. Journal of Experimental Psychology, 67(2):103–112.Google Scholar
  8. Gell-Mann, M. 1995. What is complexity? Complexity, Wiley Interscience Journal, 1(1).Google Scholar
  9. Gell-Mann, M. 1996. The simple and the complex. In Proceedings of the Conference on Complexity, Global Politics, and National Security, D.S. Alberts and T.J. Czerwinski (Ed.), National Defense University: Washington, D.C., pp. 13–14. ndu/inss/books/complexity/index.htmlGoogle Scholar
  10. Gittleman, M.M. 1988. FTS/EVA task compatibility study. Ocean Systems Engineering.Google Scholar
  11. Gittleman, M.M. 1990. Task complexity assessment study. Ocean Systems Engineering.Google Scholar
  12. Hoffman, S.J. and Kaplan, D.L. (Eds.). 1997. Human exploration of Mars: The reference mission of the NASA Mars exploration study team. NASA Publication 6107.Google Scholar
  13. Malin, M.C. and Edgett, K.S. 2001. Sedimentary rocks of early mars. Science.Google Scholar
  14. Matijevic, J. 1998. Autonomous navigation and the sojourner microrover. Science, 276.Google Scholar
  15. Nicolis, G. and Prigogine, I. 1989. Exploring Complexity: An Introduction. W.H. Freeman: New York.Google Scholar
  16. Papoulis, A. 1965. Probability, Random Variables, and Stochastic Processes, McGraw-Hill: New York.Google Scholar
  17. Pollack, J.B. et al. 1987. Case for a wet, warm climate on early Mars. Icarus, 71:203.Google Scholar
  18. Shannon, C.E. 1948. A mathematical theory of communication. Bell System Technical Journal, 27:379–423, 623û656.Google Scholar
  19. Smith, D.E. et al. 1999. The global topography of Mars and implications for surface evolution. Science, 284:1495–1503.Google Scholar
  20. Smith, D.E. et al. 2000. Mars orbiter laser altimeter (MOLA): Experiment summary after the first year of global mapping of Mars. Journal of Geophysical Research, submitted. Scholar
  21. Squyres, S.W. 1989. Ancient lakes on Mars. Icarus, 79:229.Google Scholar
  22. Squyres, S.W. and Kasting, J.F. 1994. Early Mars: How warm and how wet? Science, 265:744.Google Scholar
  23. Theobold, D. and Roberts, B. 1999. Design of an astronaut assistant for martian exploration. American Institute of Aeronautics and Astronautics.Google Scholar
  24. Waldrop, M.M. 1992. Complexity: The Emerging Science at the Edge of Order and Chaos. Simon & Schuster: New York.Google Scholar
  25. Wallechinsky, D. 2000. The Complete Book of the Summer Olympics. The Overlook Press: Woodstock and New York.Google Scholar
  26. Yagil, G. 1985. On the structural complexity of simple biosystems. Journal of Theoretical Biology, 112:1–23.Google Scholar
  27. Yagil, G. 1993. On the structural complexity of templated systems. In 1992 Lectures in Complex Systems, D. Stein and L. Nadel (Eds.), Addison-Wesley and Santa Fe Institute, pp. 519–530.Google Scholar
  28. Zarnowski, F. 1996. A Basic Guide to Decathlon, An Official United States Olympic Committee Sports Series. Griffin Publishing: Glendale, CA.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • G. Rodriguez
    • 1
  • C.R. Weisbin
    • 1
  1. 1.Jet Propulsion LaboratoryCalifornia Institute of TechnologyPasadenaUSA

Personalised recommendations