An Architecture for Human-Guided Autonomy: Team TROOPER at the DARPA Robotics Challenge Finals

  • Steven Gray
  • Robert Chevalier
  • David Kotfis
  • Benjamin Caimano
  • Kenneth Chaney
  • Aron Rubin
  • Kingsley Fregene
  • Todd Danko
Chapter
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 121)

Abstract

Recent robotics efforts have automated simple, repetitive tasks to increase execution speed and lessen an operator’s cognitive load, allowing them to focus on higher-level objectives. However, an autonomous system will eventually encounter something unexpected, and if this exceeds the tolerance of automated solutions there must be a way to fall back to teleoperation. Our solution is a largely autonomous system with the ability to determine when it is necessary to ask a human operator for guidance. We call this approach human-guided autonomy. Our design emphasizes human-on-the-loop control where an operator expresses a desired high-level goal for which the reasoning component assembles an appropriate chain of subtasks. We introduce our work in the context of the DARPA Robotics Challenge (DRC) Finals. We describe the software architecture Team TROOPER developed and used to control an Atlas humanoid robot. We employ perception, planning, and control automation for execution of subtasks. If subtasks fail, or if changing environmental conditions invalidate the planned subtasks, the system automatically generates a new task chain. The operator is able to intervene at any stage of execution, to provide input and adjustment to any control layer, enabling operator involvement to increase as confidence in automation decreases. We present our performance at the DRC Finals and a discussion about lessons learned.

Notes

Acknowledgements

The authors thank the Lockheed Martin Corporation for supporting this work. This material is based on research sponsored by DARPA under agreement number FA8750-12-2-0311. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. We are also grateful to Boston Dynamics and the Institute for Human Machine Cognition for their support during the DRC Finals. Special thanks to Ken White bread for thoughtful conversations and leadership on the development of human-guided autonomy.

References

  1. Atkeson, C., Babu, B., Banerjee, N., Berenson, D., Bove, C., Cui, X. et al. (2015). No falls, no resets: Reliable humanoid behavior in the darpa robotics challenge. In IEEE-RAS International Conference on Humanoid Robots (Humanoids 2015) (pp. 623–630).Google Scholar
  2. Berenson, D., Srinivasa, S., & Kuffner, J. (2011). Task space regions: A framework for pose-constrained manipulation planning. International Journal of Robotics Research (IJRR), 30(12), 1435–1460.CrossRefGoogle Scholar
  3. Brooks, R. (1986). A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 2(1), 14–23.CrossRefGoogle Scholar
  4. de Brun, M. L., Moffitt, V. Z., Franke, J., Yiantsios, D., Housten, T., Hughes, A., et al. (2008). Mixed-initiative adjustable autonomy for human/unmanned system teaming. In AUVSI Unmanned Systems North America Conference Google Scholar
  5. Cacace, J., Finzi, A., & Lippiello, V. (2014). A mixed-initiative control system for an aerial service vehicle supported by force feedback. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014) (pp. 1230–1235).Google Scholar
  6. Carey, M. W., Kurz, E. M., Matte, J. D., Perrault, T. D., & Padir, T. (2012). Novel EOD robot design with dexterous gripper and intuitive teleoperation. In IEEE World Automation Congress (WAC 2012) (pp. 1–6).Google Scholar
  7. Chaomin, L., Yang, S. X., Krishnan, M., & Paulik, M. (2014). An effective vector-driven biologically-motivated neural network algorithm to real-time autonomous robot navigation. In IEEE International Conference on Robotics and Automation (ICRA 2014) (pp. 4094–4099).Google Scholar
  8. Conner, D., Kohlbrecher, S., Romay, A., Stumpf, A., Maniatopoulos, S., Schappler, M., et al. (2015). Team ViGIR. Tech. Rep. DTIC ADA623035, TORC Robotics LLC, Blacksburg, Virginia.Google Scholar
  9. Cote, N., Canu, A., Bouzid, M., & Mouaddib, A. I. (2012). Humans-robots sliding collaboration control in complex environments with adjustable autonomy. In IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, (Vol. 2, pp. 146–153).Google Scholar
  10. Crandall, J., & Goodrich, M. (2002) Characterizing efficiency of human robot interaction: A case study of shared-control teleoperation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2002) (Vol. 2, pp. 1290–1295).Google Scholar
  11. Desai, J. P., Ostrowski, J., & Kumar, V. (1998). Controlling formations of multiple mobile robots. In IEEE International Conference on Robotics and Automation (ICRA 1998) (pp. 2864–2869).Google Scholar
  12. DRC Teams. (2015). Comparison of atlas robot controllers. http://www.cs.cmu.edu/cga/drc/atlas-control/. Retrieved 08 April 2016.
  13. Fallon, M., Kuindersma, S., Karumanchi, S., Antone, M., Schneider, T., Dai, H., et al. (2015). An architecture for online affordance-based perception and whole-body planning. Journal of Field Robotics, 32(2), 229–254.CrossRefGoogle Scholar
  14. Feng, S., Whitman, E., Xinjilefu, X., & Atkeson, C.G.: Optimization-based full body control for the DARPA robotics challenge. Journal of Field Robotics, 32(2), 293–312 (2015).  https://doi.org/10.1002/rob.21559.CrossRefGoogle Scholar
  15. Goodrich, M. A., & Schultz, A. C. (2007). Human-robot interaction: A survey. Foundations and Trends in Human-Computer Interaction, 1(3), 203–275.CrossRefGoogle Scholar
  16. Hebert, P., Bajracharya, M., Ma, J., Hudson, N., Aydemir, A., Reid, J., et al. (2015). Mobile manipulation and mobility as manipulationdesign and algorithms of RoboSimian. Journal of Field Robotics, 32(2), 255–274.CrossRefGoogle Scholar
  17. Henry, C. J. (2009). The meta state machine (MSM) library. http://www.boost.org/doc/libs/1_61_0/libs/msm/doc/HTML/index.html.
  18. Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., & Burgard, W. (2013). OctoMap: An efficient probabilistic 3D mapping framework based on octrees. Autonomous Robots.  https://doi.org/10.1007/s10514-012-9321-0. http://octomap.github.com.
  19. Huebner, K., Ruthotto, S., & Kragic, D. (2008). Minimum volume bounding box decomposition for shape approximation in robot grasping. In IEEE International Conference on Robotics and Automation (ICRA 2008) (pp. 1628–1633).Google Scholar
  20. Johnson, M., Shrewsbury, B., Bertrand, S., Wu, T., Duran, D., Floyd, M., et al. (2015). Team IHMC’s lessons learned from the DARPA robotics challenge trials. Journal of Field Robotics, 32(2), 192–208.CrossRefGoogle Scholar
  21. Katyal, K., Brown, C., Hechtman, S., Para, M., McGee, T., Wolfe, K., et al. (2014). Approaches to robotic teleoperation in a disaster scenario: From supervised autonomy to direct control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014) (pp. 1874–1881).Google Scholar
  22. Kuindersma, S., Deits, R., Fallon, M., Valenzuela, A., Dai, H., Permenter, F., et al. (2015). Optimization-based locomotion planning, estimation, and control design for the Atlas humanoid robot. Autonomous Robots, (1–27).Google Scholar
  23. Miller, A. T., Knoop, S., Christensen, H., Allen, P. K., et al. (2003). Automatic grasp planning using shape primitives. In IEEE International Conference on Robotics and Automation (ICRA 2003) (Vol. 2, pp. 1824–1829).Google Scholar
  24. Murphy, R., Kravitz, J., Peligren, K., Milward, J., & Stanway, J. (2008). Preliminary report: Rescue robot at Crandall Canyon, Utah, mine disaster. In IEEE International Conference on Robotics and Automation (ICRA 2008) (pp. 2205–2206).Google Scholar
  25. Muszynski, S., Stuckler, J., & Behnke, S. (2012). Adjustable autonomy for mobile teleoperation of personal service robots. In IEEE International Symposium on Robot and Human Interactive Communication (pp. 933–940).Google Scholar
  26. Nagatani, K., Kiribayashi, S., Okada, Y., Otake, K., Yoshida, K., Tadokoro, S., et al. (2013). Emergency response to the nuclear accident at the Fukushima Daiichi nuclear power plants using mobile rescue robots. Journal of Field Robotics, 30(1), 44–63.CrossRefGoogle Scholar
  27. Naseer, M., Bokhari, M., & Ahmad, A. (2005). A multi-layered behavioral architecture for semi-autonomous agents. In IEEE International Pakistan Section Multitopic Conference (INMIC 2005) (pp. 1–7).Google Scholar
  28. Pomerleau, F., Magnenat, S., Colas, F., Liu, M., & Siegwart, R. (2011) Tracking a depth camera: Parameter exploration for fast ICP. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2011) (pp. 3824–3829).Google Scholar
  29. Quigley, M., Conley, K., Gerkey, B. P., Faust, J., Foote, T., Leibs, J., et al. (2009). ROS: An open-source robot operating system. In IEEE International Conference on Robotics and Automation (ICRA 2009), Workshop on Open Source Software.Google Scholar
  30. Radford, N. A., Strawser, P., Hambuchen, K., Mehling, J. S., Verdeyen, W. K., Donnan, A. S. et al. (2015). Valkyrie: NASA’s first bipedal humanoid robot. Journal of Field Robotics, 32(3), 397–419.  https://doi.org/10.1002/rob.21560.
  31. Russell, S., & Norvig, P. (1995). Artificial intelligence: A modern approach. Prentice HallGoogle Scholar
  32. Rusu, R., & Cousins, S. (2011). 3D is here: Point Cloud Library (PCL). In IEEE International Conference on Robotics and Automation (ICRA 2011) (pp. 1–4).Google Scholar
  33. Şucan, I. A., Moll, M., & Kavraki, L. E. (2012). The open motion planning library. IEEE Robotics and Automation Magazine, 19(4), 72–82.  https://doi.org/10.1109/MRA.2012.2205651.CrossRefGoogle Scholar
  34. Tang, H., Cao, X., Song, A., Guo, Y., & Bao, J. (2009). Human-robot collaborative teleoperation system for semi-autonomous reconnaissance robot. In International Conference on Mechatronics and Automation (pp. 1934–1939).Google Scholar
  35. Tedrake, R. (2014). Drake: A planning, control, and analysis toolbox for nonlinear dynamical systems. http://drake.mit.edu.
  36. Zhang, L., Lee, S. L., Yang, G. Z., & Mylonas, G. (2014). Semi-autonomous navigation for robot assisted tele-echography using generalized shape models and co-registered RGB-D cameras. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014) (pp. 3496–3502).Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Steven Gray
    • 1
  • Robert Chevalier
    • 1
  • David Kotfis
    • 1
  • Benjamin Caimano
    • 1
  • Kenneth Chaney
    • 1
  • Aron Rubin
    • 1
  • Kingsley Fregene
    • 1
  • Todd Danko
    • 1
  1. 1.Lockheed Martin Advanced Technology LaboratoriesCherry HillUSA

Personalised recommendations