Skip to main content
SpringerLink
Log in

Robot Learning for Persistent Autonomy

  • Chapter
  • First Online:
Handling Uncertainty and Networked Structure in Robot Control

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 42))

  • 1117 Accesses

Abstract

Autonomous robots are not very good at being autonomous. They work well in structured environments, but fail quickly in the real world facing uncertainty and dynamically changing conditions. In this chapter, we describe robot learning approaches that help to elevate robot autonomy to the next level, the so-called ‘persistent autonomy’. For a robot to be ‘persistently autonomous’ means to be able to perform missions over extended time periods (e.g. days or months) in dynamic, uncertain environments without need for human assistance. In particular, persistent autonomy is extremely important for robots in difficult-to-reach environments such as underwater, rescue, and space robotics. There are many facets of persistent autonomy, such as: coping with uncertainty, reacting to changing conditions, disturbance rejection, fault tolerance, energy efficiency and so on. This chapter presents a collection of robot learning approaches that address many of these facets. Experiments with robot manipulators and autonomous underwater vehicles demonstrate the usefulness of these learning approaches in real world scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abidi MA, Eason RO, Gonzalez RC (1991) Autonomous robotic inspection and manipulation using multisensor feedback. Computer 24(4):17–31

    Article  Google Scholar 

  • Ahmadzadeh SR, Kormushev P, Caldwell DG (2013a) Autonomous robotic valve turning: a hierarchical learning approach. In: 2013 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4614–4619

    Google Scholar 

  • Ahmadzadeh SR, Leonetti M, Kormushev P (2013b) Online direct policy search for thruster failure recovery in autonomous underwater vehicles. In: 6th international workshop on evolutionary and reinforcement learning for autonomous robot system (ERLARS 2013), Taormina, Italy

    Google Scholar 

  • Ahmadzadeh SR, Jamisola RS, Kormushev P, Caldwell DG (2014a) Learning reactive robot behavior for autonomous valve turning. In: Proceedings of the IEEE international conference on humanoid robots (Humanoids 2014), Madrid, Spain

    Google Scholar 

  • Ahmadzadeh SR, Leonetti M, Carrera A, Carreras M, Kormushev P, Caldwell DG (2014b) Online discovery of AUV control policies to overcome thruster failure. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 6522–6528

    Google Scholar 

  • Ajoudani A, Lee J, Rocchi A, Ferrati M, Mingo E, Settimi A, Caldwell DG, Bicchi A, Tsagarakis N (2014) A manipulation framework for compliant humanoid COMAN: application to a valve turning task. In: 2014 IEEE-RAS international conference on humanoid robots (Humanoids 2014). IEEE, pp 664–670

    Google Scholar 

  • Alessandri A, Caccia M, Veruggio G (1998) A model-based approach to fault diagnosis in unmanned underwater vehicles. In: OCEANS’98 conference proceedings, vol 2. IEEE, pp 825–829

    Google Scholar 

  • Alunni N, Phillips-Grafftin C, Suay HB, Lofaro D, Berenson D, Chernova S, Lindeman RW, Oh P (2013) Toward a user-guided manipulation framework for high-dof robots with limited communication. In: 2013 IEEE international conference on technologies for practical robot applications (TePRA). IEEE, pp 1–6

    Google Scholar 

  • Anisi DA, Persson E, Heyer C (2011) Real-world demonstration of sensor-based robotic automation in oil & gas facilities. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 235–240

    Google Scholar 

  • Anisi DA, Skourup C, Petrochemicals A (2012) A step-wise approach to oil and gas robotics. In: IFAC workshop on automatic control in offshore oil and gas production, Trondheim, Norway, vol 31

    Google Scholar 

  • Antonelli G (2003) A survey of fault detection/tolerance strategies for AUVs and ROVs. In: Caccavale F, Villani L (eds) Fault diagnosis and fault tolerance for mechatronic systems: recent advances. Springer, Berlin, pp 109–127

    Chapter  Google Scholar 

  • Antonelli G (2006) Underwater robots: motion and force control of vehicle-manipulator systems. Springer tracts in advanced robotics. Springer, New York

    Google Scholar 

  • Bristow D, Tharayil M, Alleyne AG et al (2006) A survey of iterative learning control. IEEE Control Syst 26(3):96–114

    Article  Google Scholar 

  • Caccia M, Bono R, Bruzzone G, Bruzzone G, Spirandelli E, Veruggio G (2001) Experiences on actuator fault detection, diagnosis and accomodation for ROVs. In: International symposiyum of unmanned untethered sub-mersible technology

    Google Scholar 

  • Carrera A, Ahmadzadeh S, Ajoudani A, Kormushev P, Carreras M, Caldwell D (2012) Towards autonomous robotic valve turning. Cybern Inf Technol 12(3):17–26

    Google Scholar 

  • Cheng ASF, Leonard NE (1999) Fin failure compensation for an unmanned underwater vehicle. In: Proceedings of the 11th international symposium on unmanned untethered submersible technology

    Google Scholar 

  • Das SN, Das SK (2004) Determination of coupled sway, roll, and yaw motions of a floating body in regular waves. Int J Math Math Sci 41:2181–2197

    Article  Google Scholar 

  • Hamilton K, Lane D, Taylor N, Brown K (2001) Fault diagnosis on autonomous robotic vehicles with recovery: an integrated heterogeneous-knowledge approach. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, 2001, vol 4. IEEE, pp 3232–3237

    Google Scholar 

  • Ijspeert AJ, Nakanishi J, Hoffmann H, Pastor P, Schaal S (2013) Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput 25(2):328–373

    Article  MathSciNet  MATH  Google Scholar 

  • Jamisola RS, Oetomo DN, Ang MH, Khatib O, Lim TM, Lim SY (2005) Compliant motion using a mobile manipulator: an operational space formulation approach to aircraft canopy polishing. Adv Robot 19(5):613–634

    Article  Google Scholar 

  • Karras GC, Bechlioulis CP, Leonetti M, Palomeras N, Kormushev P, Kyriakopoulos KJ, Caldwell DG (2013) On-line identification of autonomous underwater vehicles through global derivative-free optimization. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS)

    Google Scholar 

  • Khatib O (1987) A unified approach for motion and force control of robot manipulators: the operational space formulation. IEEE J Robot Autom 3(1):43–53

    Article  Google Scholar 

  • Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  • Konidaris G, Osentoski S, Thomas PS (2011) Value function approximation in reinforcement learning using the fourier basis. In: AAAI

    Google Scholar 

  • Kormushev P, Caldwell DG (2013a) Improving the energy efficiency of autonomous underwater vehicles by learning to model disturbances. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), Tokyo, Japan

    Google Scholar 

  • Kormushev P, Caldwell DG (2013b) Towards improved AUV control through learning of periodic signals. In: Proceedings of the MTS/IEEE international conference on OCEANS 2013, San Diego

    Google Scholar 

  • Kormushev P, Calinon S, Caldwell DG (2011) Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Adv Robot 25(5):581–603

    Article  Google Scholar 

  • Lane DM, Maurelli F, Kormushev P, Carreras M, Fox M, Kyriakopoulos K (2012) Persistent autonomy: the challenges of the PANDORA project. In: Proceedings of the IFAC MCMC

    Google Scholar 

  • Leonetti M, Kormushev P, Sagratella S (2012) Combining local and global direct derivative-free optimization for reinforcement learning. Cybern Inf Technol 12(3):53–65

    Google Scholar 

  • Leonetti M, Ahmadzadeh SR, Kormushev P (2013) On-line learning to recover from thruster failures on autonomous underwater vehicles. In: OCEANS 2013. IEEE

    Google Scholar 

  • Moore KL (2012) Iterative learning control for deterministic systems. Springer Science & Business Media, London

    Google Scholar 

  • Orsag M, Korpela C, Bogdan S, Oh P (2014) Valve turning using a dual-arm aerial manipulator. In: 2014 international conference on unmanned aircraft systems (ICUAS). IEEE, pp 836–841

    Google Scholar 

  • PANDORA (2012) Persistent autonomy through learning, adaptation, observation and re-planning. http://persistentautonomy.com/, PANDORA European Project

  • Perrault D, Nahon M (1998) Fault-tolerant control of an autonomous underwater vehicle. In: OCEANS’98 conference proceedings, vol 2. IEEE, pp 820–824

    Google Scholar 

  • Podder T, Antonelli G, Sarkar N (2000) Fault tolerant control of an autonomous underwater vehicle under thruster redundancy: simulations and experiments. In: Proceedings of the IEEE international conference on robotics and automation, ICRA’00, 2000, vol 2. IEEE, pp 1251–1256

    Google Scholar 

  • Podder TK, Sarkar N (2001) Fault-tolerant control of an autonomous underwater vehicle under thruster redundancy. Robot Auton Syst 34(1):39–52

    Article  Google Scholar 

  • Raibert MH, Craig JJ (1981) Hybrid position/force control of manipulators. J Dyn Syst, Measur, Control 103(2):126–133

    Article  Google Scholar 

  • Ribas D, Palomeras N, Ridao P, Carreras M, Mallios A (2012) Girona 500 AUV: from survey to intervention. IEEE/ASME Trans Mechatron 17(1):46–53

    Article  Google Scholar 

  • Schaal S, Ijspeert A, Billard A (2003) Computational approaches to motor learning by imitation. Philoso Trans R Soc Lond Ser B: Biol Sci 358(1431):537–547

    Article  Google Scholar 

  • Seto ML (2011) An agent to optimally re-distribute control in an underactuated AUV. Int J Intell Def Support Syst 4(1):3–19

    MathSciNet  Google Scholar 

  • Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    Article  MathSciNet  MATH  Google Scholar 

  • Wang L (1999) A course on fuzzy systems. Prentice-Hall press, Upper Saddle River

    Google Scholar 

  • Yoshikawa T, Zheng XZ (1993) Coordinated dynamic hybrid position/force control for multiple robot manipulators handling one constrained object. Int J Robot Res 12(3):219–230

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Professor David Lane from the Ocean Systems Laboratory, Heriot-Watt University, UK, for introducing us to the topic of persistent autonomy.

We are grateful to Arnau Carrera, Narcís Palomeras, and Marc Carreras from the Computer Vision and Robotics Group (VICOROB), University of Girona, Spain, for making it possible to conduct real-world experiments with the Girona 500 AUV.

This work was supported by the European project PANDORA: Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning, contract FP7-ICT-288273 (PANDORA 2012).

Author information

Authors and Affiliations

  1. Dyson School of Design Engineering, Imperial College London, London, SW7 2AZ, UK

    Petar Kormushev

  2. iCub Facility, Istituto Italiano di Tecnologia, Via Morego 30, 16163, Genoa, Italy

    Seyed Reza Ahmadzadeh

Authors
  1. Petar Kormushev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seyed Reza Ahmadzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petar Kormushev .

Editor information

Editors and Affiliations

  1. Automation Department, Technical University of Cluj-Napoca, Cluj-Napoca, Romania

    Lucian Busoniu

  2. Automation Department, Technical University of Cluj-Napoca, Cluj-Napoca, Romania

    Levente Tamás

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kormushev, P., Ahmadzadeh, S.R. (2015). Robot Learning for Persistent Autonomy. In: Busoniu, L., Tamás, L. (eds) Handling Uncertainty and Networked Structure in Robot Control. Studies in Systems, Decision and Control, vol 42. Springer, Cham. https://doi.org/10.1007/978-3-319-26327-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26327-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26325-0

  • Online ISBN: 978-3-319-26327-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation