Perspectives on Human-Robot Team Performance from an Evaluation of the DARPA Robotics Challenge

Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 121)


The DARPA Robotics Challenge (DRC) was a competition designed to advance the capabilities of remotely teleoperated semi-autonomous humanoid robots performing in a disaster response scenario with degraded communications. Throughout the DRC, our evaluation team conducted two studies of human-robot interaction (HRI) for the Trials and Finals competitions. From these studies, we have generated recommendations and design guidelines for HRI with remote, semi-autonomous humanoids, but our findings also have implications outside of the competition’s domain. In this article, we discuss our perspectives on effective and ineffective human-robot teams based upon our experiences at the DRC. We consider the impact of various interfacing and control techniques, the effect of versatile robot design on task performance, and the operational context under which these factors work together to function in a human-centric environment. We use these underlying components of HRI to review how the advancements made at the DRC can be applied to present day robot applications and key capabilities for effective human-robot teams in the future.



This research has been supported in part by DARPA under W31P4Q-13-C-0136. The views, opinions, and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. Approved for Public Release, Distribution Unlimited. The authors would like to thank DARPA, the staff members of the DRC, and all affiliated personnel that enabled this research. Thank you to Josh Peri for his assistance with data analysis. In particular, thank you to all of the teams that consented to participate in our studies over the years.


  1. Atkeson, C. G., Babu, B. P. W., Banerjee, N., Berenson, D., Bove, C. P., Cui, X. et al. (2015). No falls, no resets: Reliable humanoid behavior in the DARPA robotics challenge. In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) (pp. 623–630).Google Scholar
  2. Blitch, J. G. (2003). Adaptive mobility for rescue robots. In Proceedings of SPIE Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Defense and Law Enforcement II (Vol. 5071, pp. 315–321).Google Scholar
  3. Chen, J. Y., Haas, E. C., & Barnes, M. J. (2007). Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 37(6), 1231–1245.CrossRefGoogle Scholar
  4. Cisneros, R., Nakaoka, S. I., Morisawa, M., Kaneko, K., Kajita, S., Sakaguchi, T., et al. (2016). Effective teleoperated manipulation for humanoid robots in partially unknown real environments: Team AIST-NEDO’s approach for performing the plug task during the DRC finals. Advanced Robotics, 30(24), 1544–1558.CrossRefGoogle Scholar
  5. Endsley, M. R. (1995). Towards a theory of situation awareness in dynamic systems. Human factors, 32–64.CrossRefGoogle Scholar
  6. Goodrich, M. A., & Olsen, D. R. (2003). Seven principles of efficient human robot interaction. In IEEE International Conference on Systems, Man and Cybernetics (Vol. 4, pp. 3942–3948).Google Scholar
  7. Jameson, S. M. (2001). Architectures for distributed information fusion to support situation awareness on the digital battlefield. In 4th International Conference on Data Fusion (pp. 7–10).Google Scholar
  8. Keyes, B., Casey, R., Yanco, H. A., Maxwell, B. A., Georgiev, Y. (2006). Camera placement and multi-camera fusion for remote robot operation. In Proceedings of the IEEE International Workshop on Safety, Security and Rescue Robotics (pp. 22–24).Google Scholar
  9. Keyes, B., Micire, M., Drury, J. L., & Yanco, H. A. (2010). Improving human-robot interaction through interface evolution. In D. Chugo (Ed.), Human-robot interaction (pp. 183–202).Google Scholar
  10. Lim, J., Lee, I., Shim, I., Jung, H., Joe, H. M., Bae, H., et al. (2017). Robot system of DRC-HUBO+ and control strategy of team KAIST in DARPA robotics challenge finals. Journal of Field Robotics, 34(4), 802–829.CrossRefGoogle Scholar
  11. McGill, S. G., Yi, S. J., Yi, H., Ahn, M. S., Cho, S., Liu, K., et al. (2017). Team THOR’s entry in the DARPA robotics challenge finals 2015. Journal of Field Robotics, 34(4), 775–801.CrossRefGoogle Scholar
  12. Murphy, R. R. (2014). International cooperation in deploying robots for disasters: Lessons for the future from the great east Japan earthquake. Journal of the Robotics Society of Japan, 32(2), 104–109.CrossRefGoogle Scholar
  13. Nagatani, K., Kiribayashi, S., Okada, Y., Otake, K., Yoshida, K., Tadokoro, S., et al. (2013). Emergency response to the nuclear accident at the Fukushima Daiichi nuclear power plants using mobile rescue robots. Journal of Field Robotics, 30(1), 44–63.CrossRefGoogle Scholar
  14. Nielsen, C. W., Goodrich, M. A., & Ricks, R. W. (2007). Ecological interfaces for improving mobile robot teleoperation. IEEE Transactions on Robotics, 23(5), 927–941.CrossRefGoogle Scholar
  15. Norton, A., Ober, W., Baraniecki, L., McCann, E., Scholtz, J., Shane, D., et al. (2017). Analysis of human-robot interaction at the DARPA robotics challenge finals. The International Journal of Robotics Research, 36(5–7), 483–513.CrossRefGoogle Scholar
  16. Oh, P., Sohn, K., Jang, G., Jun, Y., & Cho, B. K. (2017). Technical overview of team DRC-Hubo@ UNLV’s approach to the 2015 DARPA robotics challenge finals. Journal of Field Robotics, 34(5), 874–896.CrossRefGoogle Scholar
  17. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 30(3), 286–297.CrossRefGoogle Scholar
  18. Stepanova, E. R., von der Heyde, M., Kitson, A., Schiphorst, T., & Riecke, B. E. (2017). Gathering and applying guidelines for mobile robot design for urban search and rescue application. In International Conference on Human-Computer Interaction 2017 (pp. 562–581).Google Scholar
  19. Yanco, H. A., Norton, A., Ober, W., Shane, D., Skinner, A., & Vice, J. (2015). Analysis of human-robot interaction at the DARPA robotics challenge trials. Journal of Field Robotics, 32(3), 420–444.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.New England Robotics Validation and Experimentation (NERVE) CenterUniversity of Massachusetts LowellLowellUSA
  2. 2.Boston Engineering CorporationWalthamUSA
  3. 3.AnthroTronixSilver SpringUSA
  4. 4.Black Moon, LLCWashington, D.C.USA
  5. 5.Computer Science DepartmentUniversity of Massachusetts LowellLowellUSA

Personalised recommendations