Skip to main content
SpringerLink
Log in

Online adaptive teleoperation via motion primitives for mobile robots

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Assistive teleoperation aims to help operators control robotic systems with ease. In this work, we present a novel adaptive teleoperation approach that is amenable to mobile systems using motion primitives for long-duration teleoperation, such as exploration using mobile vehicles or walking for humanoid systems. We first describe teleoperation using motion primitives, which are dynamically feasible and safe local trajectories based on a kinematic or dynamic model. We take a predict-and-adapt approach to assistive teleoperation, whereby adaptation is based on the predicted user intent. By representing the operator as an optimizing controller, a probabilistic distribution can be constructed for the available future actions based on some reward function. Adaptation is provided in the form of subsampling, which tailors the set of available actions based on the likelihood of action selection. We describe the framework for general systems and delineate the extrapolation to ground, air, and legged mobile robots, and demonstrate generalizability of this framework on two systems via simulation and experimentation; namely, a quadrotor micro air vehicle, and a simulated 3D humanoid system. Both systems show provably better performance in teleoperation by measures of behavioral entropy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. An example of goal-based task definition is a command with a fixed goal location in the configuration space, such as go to the door. A description based task is a command that may describe an objective that is nebulous in the execution required to achieve it; for example, inspect the building for damages.

References

  • Admoni, H., & Srinivasa, S. (2016). Predicting user intent through eye gaze for shared autonomy. In The 2016 AIII fall symposium series: Shared autonomy in research and practice. Technical Report FS-16-05.

  • Aigner, P., & McCarragher, B. (1997). Human integration into robot control utilising potential fields. In Robotics and automation, 1997. Proceedings., 1997 IEEE international conference on (Vol. 1, pp. 291–296). IEEE.

  • Anderson, S. J., Walker, J. M., & Iagnemma, K. (2014). Experimental performance analysis of a homotopy-based shared autonomy framework. IEEE Transactions on Human-Machine Systems, 44, 190–199.

    Article  Google Scholar 

  • Bachrach, A. G. (2013). Trajectory bundle estimation for perception-driven planning. Ph.D. Thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge.

  • Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., & Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neuroscience and Biobehavioral Reviews, 44, 58–75.

    Article  Google Scholar 

  • Carlson, T., & Demiris, Y. (2008). Human-wheelchair collaboration through prediction of intention and adaptive assistance. In Proceedings of the IEEE international conference on robot and automation, Pasadena, CA, pp. 3926–3931.

  • Chevallereau, C., Grizzle, J. W., & Shih, C. L. (2010). Steering of a 3D bipedal robot with an underactuated ankle. In Intelligent robots and systems (IROS), 2010 IEEE/RSJ international conference on (pp. 1242–1247). IEEE.

  • Cohen, B. J., Subramania, G., Chitta, S., & Likhachev, M. (2011). Planning for manipulation with adaptive motion primitives. In Proceedings of the IEEE international conference on robot and automation (pp. 5478–5485). IEEE.

  • Crandall, J. W., & Goodrich, M. A. (2002). Characterizing efficiency of human robot interaction: A case study of shared-control teleoperation. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Lausanne, Switzerland, pp. 1–6.

  • Delson, N., & West H. (1994). Robot programming by human demonstration: The use of human inconsistency in improving 3D robot trajectories. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. (pp. 1248–1255).

  • Demeester, E., Hüntemann, A., Vanhooydonck, D., Vanacker, G., Brussel, H. V., & Nuttin, M. (2008). User-adapted plan recognition and user-adapted shared control: A bayesian approach to semi-autonomous wheelchair driving. In Autonomous robots (pp. 193–211).

  • Demeester, E., Hüntemann, A., Poorten, E. V., & Schutter, J. D. (2012a). ML, MAP and greedy POMDP shared control: Comparison of wheelchair navigation assistance for switch interfaces. In Proceedings of the international symposium on robotics, Taipei, Taiwan, pp. 1106–1111.

  • Demeester, E., Poorten, E. V., Hüntemann, A., De Schutter, J., Lau, B., Kuderer, M., et al. (2012b). Robotic adaptation to humans adapting to robots. In 1st International Conference on Systems and Computer Science (ICSCS 2012).

  • Derry, M., & Argall, B. (2014). A probabilistic representation of user intent for assistive robots. In International conference on intelligent robots and systems workshop on rehabilitation and assistive robotics, Chicago, IL, USA.

  • Dragan, A. D., & Srinivasa, S. S. (2013). A policy-blending formalism for shared control. The International Journal of Robotics Research, 32(7), 790–805.

    Article  Google Scholar 

  • Gao, M., Oberl, J., Schamm, T., & Marius, J. Z. (2014). Contextual task-aware shared autonomy for assistive mobile robot teleoperation. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Chicago, IL, USA, pp. 3311–3318.

  • Glinton, R., Owens, S., Giampapa, J., Sycara, K., Lewis, M., & Grindle, C. (2005). Intent inference using a potential field model of environmental influences intent inference data flow. In 2005 7th international conference on information fusion. IEEE.

  • Gnatzig, S., Schuller, F., Lienkamp, M. (2012). Human–machine interaction as key technology for driverless driving: A trajectory-based shared autonomy control approach. In Proceedings of the IEEE international symposium on robot and human interactive communication (pp. 913–918).

  • Goil, A., Derry, M., & Argall, B. D. (2013). Using machine learning to blend human and robot controls for assisted wheelchair navigation. In: IEEE international conference on rehabilitation robotics, Seattle, WA, USA.

  • Goodrich, M. A., Boer, E. R., Crandall, J. W., Ricks, R. W., & Quigley, M. L. (2004). Behavioral entropy in human–robot interaction. Technical reports on Brigham Young University.

  • Grizzle, J. W., Abba, G., & Plestan, F. (2001). Asymptotically stable walking for biped robots: Analysis via systems with impulse effects. IEEE Transactions on Automatic Control, 46(1), 51–64.

    Article  MathSciNet  MATH  Google Scholar 

  • Grizzle, J. W., Chevallereau, C., Ames, A. D., & Sinnet, R. W. (2010). 3D Bipedal robotic walking: Models, feedback control, and open problems. IFAC Proceedings Volumes, 43(14), 505–532.

    Article  MATH  Google Scholar 

  • Hauser, K. (2013). Recognition, prediction, and planning for assisted teleoperation of freeform tasks. Autonomous Robots, 35(4), 241–254.

    Article  Google Scholar 

  • Hauser, K., Bretl, T., Harada, K., & Latombe, J. C. (2008). Using motion primitives in probabilistic sample-based planning for humanoid robots. In Algorithmic foundation of robotics VII (pp. 507–522).

  • Hereid, A., Cousineau, E. A., Hubicki, C. M., & Ames, A. D. (2016). 3D Dynamic walking with underactuated humanoid robots: A direct collocation framework for optimizing hybrid zero dynamics. In IEEE international conference on intelligent robotics and automation.

  • Huntemann, A., Demeester, E., Poorten, E. V., & Brussel, H. V. (2013). Probabilistic approach to recognize local navigation plans by fusing past driving information with a personalized user model. In Proceedings of the IEEE International Conference on Intelligent Robotics and Automation, Karlsruhe, Germany, pp. 4376–4383.

  • Jain, S., & Argall, B. (2016). An approach for online user customization of shared autonomy for intelligent assistive devices. In Proceedings of the IEEE international conference on intelligent robotics and automation.

  • Javdani, S., Srinivasa, S. S., & Bagnell, J. A. (2015). Shared autonomy via hindsight optimization. In Proceedings of robotics: Science and system, Rome, Italy.

  • Javdani, S., Bagnell, J. A., & Srinivasa, S. S. (2016). Minimizing user cost for shared autonomy. In 2016 11th ACM/IEEE international conference on human–robot interaction (HRI) (pp. 621–622).

  • Jimenez-Fabian, R., & Verlinden, O. (2012). Review of control algorithms for robotic ankle systems in lower-limb orthoses, prostheses, and exoskeletons. Medical Engineering & Physics, 34(4), 397–408.

    Article  Google Scholar 

  • Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2), 99–134.

    Article  MathSciNet  MATH  Google Scholar 

  • Kawamoto, H., Kanbe, S., & Sankai, Y. (2003a). Power assist method for HAL-3 estimating operator’s intention based on motion information. In Robot and human interactive communication, 2003. Proceedings. ROMAN 2003. The 12th IEEE international workshop on (pp. 67–72). IEEE.

  • Kawamoto, H., Lee, S., Kanbe, S., & Sankai, Y. (2003b). Power assist method for HAL-3 using EMG-based feedback controller. In Systems, man and cybernetics, 2003. IEEE international conference on (Vol. 2, pp. 1648–1653). IEEE.

  • Kofman, J., Wu, X., Luu, T. J., & Verma, S. (2005). Teleoperation of a robot manipulator using a vision-based human–robot interface. IEEE Transactions on Industrial Electronics, 52(5), 1206–1219.

    Article  Google Scholar 

  • Kretzschmar, H., Kuderer, M., & Burgard, W. (2014). Learning to predict trajectories of cooperatively navigating agents. In Proceedings of the IEEE international conference on robotics and automation.

  • Kulic, D., & Croft, E. A. (2003). Estimating intent for human–robot interaction. In IEEE international conference on advanced robotics.

  • Loeb, G. E. (2012). Optimal isn’t good enough. Biological Cybernetics, 106(11), 757–765.

    Article  Google Scholar 

  • MacAdam, C. C. (2003). Understanding and modeling the human driver. Vehicle System Dynamics, 40(1–3), 101–134.

    Article  Google Scholar 

  • McLachlan, S., Arblaster, J., Liu, D., Miro, J. V., & Chenoweth, L. (2005). A multi-stage shared control method for an intelligent mobility assistant. In IEEE international conference on rehabilitation robotics, Chicago, IL, USA.

  • Medina, J. R., Lorenz, T., & Hirche, S. (2015). Synthesizing anticipatory haptic assistance considering human behavior uncertainty. IEEE Transactions on Robotics, 31(1), 180–190.

    Article  Google Scholar 

  • Mellinger, D., & Kumar, V. (2011). Minimum snap trajectory generation and control for quadrotors. In Proceedings of the IEEE international conference on robotics and automation (pp. 2520–2525). IEEE.

  • Milliken, L., & Hollinger, G. A. (2016). Modeling user expertise for choosing levels of shared autonomy. In Proceedings of robotics: Science and system workshop on planning for human–robot interaction, Ann Arbor, MI, USA.

  • Mombaur, K., Jp, Laumond, & Yoshida, E. (2010). An optimal control-based formulation to determine natural locomotor paths for humanoid robots. Advanced Robotics, 24, 515–535.

    Article  Google Scholar 

  • Motahar, M. S., Veer, S., & Poulakakis, I. (2016). Composing limit cycles for motion planning of 3D bipedal walkers. In IEEE conference on decision and control.

  • Muelling , K., Venkatraman, A., Valois , JS., Downey , JE., Weiss , J., Javdani , S., Hebert , M., Schwartz , AB., Collinger, JL., & Bagnell , JA . (2015) . Autonomy infused teleoperation with application to BCI manipulation. In Proceedings of robotics: Science and systems, Ann Arbor, MI, USA.

  • Nakayama, O., Futami, T., & Nakamura, T. (1999). SAE Technical development of a steering entropy method for evaluating driver workload. Technical Report (p. 724).

  • Nelson, E. A., & Michael, N. (2015). Environment model adaptation for autonomous exploration. Master’s Thesis, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.

  • Pivtoraiko, M., & Kelly, A. (2011). Kinodynamic motion planning with state lattice motion primitives. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 2172–2179). IEEE.

  • Ralston, H. J. (1958). Energy-speed relation and optimal speed during level walking. European Journal of Applied Physiology and Occupational Physiology, 17(4), 277–283.

    Article  Google Scholar 

  • Sa, I., & Corke, P. (2014). Vertical infrastructure inspection using a quadcopter and shared autonomy control. In Field and service robotics (pp. 219–232). Springer.

  • Sheridan, T. B., & Parasuraman, R. (2005). Human-automation interaction. Reviews of Human Factors and Ergonomics, 1(1), 89–129.

    Article  Google Scholar 

  • Shih, C. L., Grizzle, J., & Chevallereau, C. (2012). From stable walking to steering of a 3d bipedal robot with passive point feet. Robotica, 30(07), 1119–1130.

    Article  Google Scholar 

  • Vanhooydonck, D., Demeester, E., Hüntemann, A., Philips, J., Vanacker, G., Brussel, H. V., et al. (2010). Adaptable navigational assistance for intelligent wheelchairs by means of an implicit personalized user model. Robotics and Autonomous Systems, 58(8), 963–977.

    Article  Google Scholar 

  • Varol, H. A., & Goldfarb, M. (2007). Real-time intent recognition for a powered knee and ankle transfemoral prosthesis. In Rehabilitation robotics, 2007. ICORR 2007. IEEE 10th international conference on (pp. 16–23). IEEE.

  • Varol, H. A., Sup, F., & Goldfarb, M. (2010). Multiclass real-time intent recognition of a powered lower limb prosthesis. IEEE Transactions on Biomedical Engineering, 57(3), 542–551.

    Article  Google Scholar 

  • Vijayakumar, S., D’Souza, A., & Schaal, S. (2005). Incremental online learning in high dimensions. Neural Computation, 17(12), 2602–2634.

    Article  MathSciNet  Google Scholar 

  • Wang, Z., Mülling, K., Deisenroth, M. P., Amor, H. B., Vogt, D., Schölkopf, B., et al. (2013). Probabilistic movement modeling for intention inference in human–robot interaction. The International Journal of Robotics Research, 32(7), 841–858.

    Article  Google Scholar 

  • Wasson, G., Sheth, P., Huang, C., & Ledoux, A. (2004). A physics-based model for predicting user intent in shared-control pedestrian mobility aids. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Sendai, Japan, pp. 1914–1919.

  • Westervelt, E. R., Grizzle, J. W., Chevallereau, C., Choi, J. H., & Morris, B. (2007). Feedback control of dynamic bipedal robot locomotion (Vol. 28). Boca Raton: CRC Press.

    Google Scholar 

  • Yang, X., Sreenath, K., Michael, N. (2017). A framework for efficient teleoperation via online adaptation. In International conference on robotics and automation (ICRA), Singapore.

Download references

Author information

Authors and Affiliations

  1. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Xuning Yang & Nathan Michael

  2. Department of Mechanical Engineering, University of California, Berkeley, CA, 94720, USA

    Ayush Agrawal & Koushil Sreenath

Authors
  1. Xuning Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayush Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Koushil Sreenath
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nathan Michael
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuning Yang.

Additional information

The work of A. Agrawal and K. Sreenath is supported in part by NSF Grant IIS-1526515.

This is one of the several papers published in Autonomous Robots comprising the Special Issue on Learning for Human-Robot Collaboration.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Agrawal, A., Sreenath, K. et al. Online adaptive teleoperation via motion primitives for mobile robots. Auton Robot 43, 1357–1373 (2019). https://doi.org/10.1007/s10514-018-9753-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-018-9753-2

Keywords

Search

Navigation