Bandit-Based Online Candidate Selection for Adjustable Autonomy

  • Boris Sofman
  • J. Andrew Bagnell
  • Anthony Stentz
Conference paper
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 62)


In many robot navigation scenarios, the robot is able to choose between some number of operating modes. One such scenario is when a robot must decide how to trade-off online between human and tele-operation control. When little prior knowledge about the performance of each operator is known, the robot must learn online to model their abilities and be able to take advantage of the strengths of each. We present a bandit-based online candidate selection algorithm that operates in this adjustable autonomy setting and makes choices to optimize overall navigational performance. We justify this technique through such a scenario on logged data and demonstrate how the same technique can be used to optimize the use of high-resolution overhead data when its availability is limited.


Mobile Robot Online Algorithm Candidate Selection Bandit Problem Traversal Time 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Stentz, A., Bares, J., Pilarski, T., Stager, D.: The crusher system for autonomous navigation. In: AUVSIs Unmanned Systems North America (August 2007)Google Scholar
  2. 2.
    Silver, D., Bagnell, J.A., Stentz, A.: High performance outdoor navigation from overhead data using imitation learning. In: Robotics Science and Systems (June 2008)Google Scholar
  3. 3.
    Dias, M.B., Kannan, B., Browning, B., Jones, E., Argall, B., Dias, M.F., Zinck, M.B., Veloso, M.M., Stentz, A.: Sliding autonomy for peer-to-peer human-robot teams. In: 10th International Conference on Intelligent Autonomous Systems 2008 (July 2008)Google Scholar
  4. 4.
    Scerri, P., Pynadath, D.V., Tambe, M.: Towards adjustable autonomy for the real world. Journal of Artificial Intelligence Research 17, 2002 (2002)MathSciNetGoogle Scholar
  5. 5.
    Grace, R., Byrne, V., Bierman, D., Legrand, J.-M., Gricourt, D., Davis, B., Staszewski, J., Carnahan, B.: A drowsy driver detection system for heavy vehicles. In: Proceedings of the 17th Digital Avionics Systems Conference, vol. 2, pp. I36/1 – I36/8 (2001)Google Scholar
  6. 6.
    Vahidi, A., Eskandarian, A.: Research advances in intelligent collision avoidance and adaptive cruise control. IEEE Transactions on Intelligent Transportation Systems 4(3), 143–153 (2003)CrossRefGoogle Scholar
  7. 7.
    Bishop, R.: Intelligent vehicle applications worldwide. IEEE Intelligent Systems 15(1), 78–81 (2000)Google Scholar
  8. 8.
    Krotkov, E., Simmons, R., Cozman, F., Koenig, S.: Safeguarded teleoperation for lunar rovers: From human factors to field trials. In: Proc. IEEE Planetary Rover Technology and Systems Workshop (1996)Google Scholar
  9. 9.
    Krupa, A., de Mathelin, M., Doignon, C., Gangloff, J., Morel, G., Soler, L., Marescaux, J.: Development of semi-autonomous control modes in laparoscopic surgery using automatic visual servoing. In: Niessen, W.J., Viergever, M.A. (eds.) MICCAI 2001. LNCS, vol. 2208, pp. 1306–1307. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  10. 10.
    Heger, F.W., Singh, S.: Sliding autonomy for complex coordinated multi-robot tasks: Analysis & experiments. In: Sukhatme, G.S., Schaal, S., Burgard, W., Fox, D. (eds.) Robotics: Science and Systems. The MIT Press, Cambridge (2006)Google Scholar
  11. 11.
    Fong, T.W., Thorpe, C., Baur, C.: Multi-robot remote driving with collaborative control. IEEE Transactions on Industrial Electronics (2003)Google Scholar
  12. 12.
    Stentz, A., Dima, C., Wellington, C., Herman, H., Stager, D.: A system for semi-autonomous tractor operations. Auton. Robots 13(1), 87–104 (2002)zbMATHCrossRefGoogle Scholar
  13. 13.
    Horvitz, E., Jacobs, A., Hovel, D.: Attention-sensitive alerting. In: Laskey, K.B., Prade, H., Cal, S.F. (eds.) Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI 1999), July 30-August 1, pp. 305–313. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  14. 14.
    Hexmoor, H.: A cognitive model of situated autonomy. In: Kowalczyk, R., Loke, S.W., Reed, N.E., Graham, G. (eds.) PRICAI-WS 2000. LNCS (LNAI), vol. 2112, pp. 325–334. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  15. 15.
    Gunderson, J.P., Martin, W.N.: Effects of uncertainty on variable autonomy in maintenance robots. In: Workshop on Autonomy Control Software, pp. 26–34 (1999)Google Scholar
  16. 16.
    Goodrich, M.A., Schultz, A.C.: Human-robot interaction: A survey. Foundations and Trends in Human-Computer Interaction 1(3), 203–275 (2007)zbMATHCrossRefGoogle Scholar
  17. 17.
    Robbins, H.: Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. 58(5), 527–535 (1952)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Lai, T., Robbins, H.: Asymptotically efficient adaptive allocation rules. Advances in applied mathematics (Print) 6(1), 4–22 (1985)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine Learning 47 2(3), 235–256 (2002)CrossRefGoogle Scholar
  20. 20.
    Wang, C., Kulkarni, S., Poor, H.: Bandit problems with side observations. IEEE Transactions on Automatic Control 50(3), 338–355 (2005)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. The Journal of Machine Learning Research 3, 397–422 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Sofman, B., Ratliff, E.L., Bagnell, J.A., Cole, J., Vandapel, N., Stentz, A.: Improving robot navigation through self-supervised online learning. Journal of Field Robotics 23(1) (December 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Boris Sofman
    • 1
  • J. Andrew Bagnell
    • 1
  • Anthony Stentz
    • 1
  1. 1.Robotics InstituteCarnegie Mellon UniversityPittsburgh

Personalised recommendations