© 2009

Performance Evaluation and Benchmarking of Intelligent Systems

  • Raj Madhavan
  • Edward Tunstel
  • Elena Messina

Table of contents

  1. Front Matter
    Pages i-xix
  2. Robert N. Lass, Evan A. Sultanik, William C. Regli
    Pages 1-19
  3. Birsen Donmez, Patricia E. Pina, M.L. Cummings
    Pages 21-40
  4. Katherine M. Tsui, David J. Feil-Seifer, Maja J. Matarić, Holly A. Yanco
    Pages 41-66
  5. S. Balakirsky, S. Carpin, G. Dimitoglou, B. Balaguer
    Pages 113-137
  6. Patrick Courtney, Olivier Michel, Angelo Cangelosi, Vadim Tikhanoff, Giorgio Metta, Lorenzo Natale et al.
    Pages 139-168
  7. Michael Lewis, Jijun Wang
    Pages 169-186
  8. C. Scrapper, R. Madhavan, R. Lakaemper, A. Censi, A. Godil, A. Wagan et al.
    Pages 221-248
  9. Edward Tunstel, John M. Dolan, Terrence Fong, Debra Schreckenghost
    Pages 249-268
  10. Roger Eastman, Tsai Hong, Jane Shi, Tobias Hanning, Bala Muralikrishnan, S. Susan Young et al.
    Pages 269-310
  11. Jane Shi, Rick F. Rourke, Dave Groll, Peter W. Tavora
    Pages 311-338

About this book


Performance Evaluation and Benchmarking of Intelligent Systems presents research dedicated to the subject of performance evaluation and benchmarking of intelligent systems by drawing from the experiences and insights of leading experts gained both through theoretical development and practical implementation of intelligent systems in a variety of diverse application domains. This contributed volume offers a detailed and coherent picture of state-of-the-art, recent developments, and further research areas in intelligent systems.

The chapters cover a broad range of applications, such as assistive robotics, planetary surveying, urban search and rescue, and line tracking for automotive assembly. Subsystems or components described in this book include human-robot interaction, multi-robot coordination, communications, perception, and mapping. Chapters are also devoted to simulation support and open source software for cognitive platforms, providing examples of the type of enabling underlying technologies that can help intelligent systems to propagate and increase in capabilities.

Performance Evaluation and Benchmarking of Intelligent Systems serves as a professional reference for researchers and practitioners in the field. This book is also applicable to advanced courses for graduate level students and robotics professionals in a wide range of engineering and related disciplines including computer science, automotive, healthcare, manufacturing, and service robotics.


Tracking automation communication currentjm intelligent systems linear optimization measurement perception performance performance evaluation performance metrics quantitative performance evaluation robot robotics simulation

Editors and affiliations

  • Raj Madhavan
    • 1
  • Edward Tunstel
    • 2
  • Elena Messina
    • 3
  1. 1.Computer Science & Mathematics Div.Oak Ridge National LaboratoryOak RidgeU.S.A.
  2. 2.Applied Physics Lab.Johns Hopkins UniversityLaurelU.S.A.
  3. 3.Technology (NIST)National Institute of Standards &GaithersburgU.S.A.

Bibliographic information


From the reviews: “This book is the result of the 2008 performance Metrics for Intelligent Systems Workshop at the National Institute of Standards and Technology (NIST). It investigates intelligent systems, starting with multiagent systems … . presents an impressive quantity of data, diagrams, proposals, experimental methods, and open topics, and also includes current solutions and relevant literature. … Intelligent systems developers and users will be able to build on this book’s material.” (G. Gini, ACM Computing Reviews, December, 2009)