Skip to main content
SpringerLink
Log in

Kalman Smoothing for Distributed Optimal Feedback Control of Unicycle Formations

  • Chapter
Redundancy in Robot Manipulators and Multi-Robot Systems

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 57))

Abstract

In many multi-agent control problems, the ability to compute an optimal feedback control is severely limited by the dimension of the state space. In this work, deterministic, nonholonomic agents are tasked with creating and maintaining a formation based on observations of their neighbors, and each agent in the formation independently computes its feedback control from a Hamilton-Jacobi-Bellman (HJB) equation. Since an agent does not have knowledge of its neighbors’ future motion, we assume that the unknown control to be applied by neighbors can be modeled as Brownian motion. The resulting probability distribution of its neighbors’ future trajectory allows the HJB equation to be written as a path integral over the distribution of optimal trajectories. We describe how the path integral approach to stochastic optimal control allows the distributed control problems to be written as independent Kalman smoothing problems over the probability distribution of the connected agents’ future trajectories. Simulations show five unicycles achieving the formation of a regular pentagon.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, B., Fidan, B., Yu, C., Walle, D.: UAV formation control: theory and application. In: Blondel, V., Boyd, S., Kimura, H. (eds.) Recent Advances in Learning and Control, pp. 15–34. Springer (2008)

    Google Scholar 

  2. Bell, B.M., Burke, J.V., Pillonetto, G.: An inequality constrained nonlinear Kalman-Bucy smoother by interior point likelihood maximization. Automatica 45(1), 25–33 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bellingham, J.G., Rajan, K.: Robotics in remote and hostile environments. Science 318(5853), 1098–1102 (2007)

    Article  Google Scholar 

  4. Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-dynamic programming. Athena Scientific, Belmont (1996)

    MATH  Google Scholar 

  5. van den Broek, B., Wiegerinck, W., Kappen, B.: Graphical model inference in optimal control of stochastic multi-agent systems. Journal of Artificial Intelligence Research 32(1), 95–122 (2008)

    MATH  Google Scholar 

  6. van den Broek, B., Wiegerinck, W., Kappen, B.: Optimal Control in Large Stochastic Multi-agent Systems. In: Tuyls, K., Nowe, A., Guessoum, Z., Kudenko, D. (eds.) Adaptive Agents and MAS III. LNCS (LNAI), vol. 4865, pp. 15–26. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  7. Bullo, F., Cortes, J., Martinez, S.: Distributed control of robotic networks: A mathematical approach to motion coordination algorithms. Princeton University Press, Princeton (2009)

    MATH  Google Scholar 

  8. Dimarogonas, D.: On the rendezvous problem for multiple nonholonomic agents. IEEE Transactions on Automatic Control 52(5), 916–922 (2007)

    Article  MathSciNet  Google Scholar 

  9. Elkaim, G., Kelbley, R.: A Lightweight Formation Control Methodology for a Swarm of Non-Holonomic Vehicles. In: IEEE Aerospace Conference. IEEE, Big Sky (2006)

    Google Scholar 

  10. Fleming, W., Soner, H.: Logarithmic Transformations and Risk Sensitivity. In: Controlled Markov Processes and Viscosity Solutions, ch.6. Springer, Berlin (1993)

    Google Scholar 

  11. Freidlin, M., Wentzell, A.: Random Perturbations of Dynamical Systems. Springer (1984)

    Google Scholar 

  12. Gamerman, D.: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman and Hall (1997)

    Google Scholar 

  13. Gelb, A.: Applied Optimal Estimation. The M.I.T. Press, Cambridge (1974)

    Google Scholar 

  14. Goldstein, H.: Classical Mechanics, 2nd edn. Addison-Wesley (1980)

    Google Scholar 

  15. van Kampen, N.G.: Stochastic Processes in Physics and Chemistry, 3rd edn. North-Holland (2007)

    Google Scholar 

  16. Kappen, H.: Linear Theory for Control of Nonlinear Stochastic Systems. Physical Review Letters 95(20), 1–4 (2005)

    Article  MathSciNet  Google Scholar 

  17. Kappen, H.: Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics: Theory and Experiment 2005, P11,011 (2005)

    Google Scholar 

  18. Kappen, H., Wiegerinck, W., van den Broek, B.: A path integral approach to agent planning. In: Autonomous Agents and Multi-Agent Systems, Citeseer (2007)

    Google Scholar 

  19. Kappen, H.J.: Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics, Theory and Experiment 2005, 21 (2005)

    Article  MathSciNet  Google Scholar 

  20. Kappen, H.J., Gómez, V., Opper, M.: Optimal control as a graphical model inference problem. Machine Learning 87(2), 159–182 (2012)

    Article  MATH  Google Scholar 

  21. Khas’minskii, R.: On the principle of averaging the Itô’s stochastic differential equations. Kybernetika 4(3), 260–279 (1968)

    MathSciNet  Google Scholar 

  22. Kumar, V., Rus, D., Sukhatme, G.S.: Networked Robotics. In: Sciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics. ch. 41, pp. 943–958 (2008)

    Google Scholar 

  23. Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd edn. Springer (2001)

    Google Scholar 

  24. Milutinović, D.: Utilizing Stochastic Processes for Computing Distributions of Large-Size Robot Population Optimal Centralized Control. In: Martinoli, A., Mondada, F., Correll, N., Mermoud, G., Egerstedt, M., Hsieh, M.A., Parker, L.E., Støy, K. (eds.) Distributed Autonomous Robotic Systems. STAR, vol. 83, pp. 359–372. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  25. Milutinović, D., Garg, D.P.: Stochastic model-based control of multi-robot systems. Tech. Rep. 0704, Duke University, Durham, NC (2009)

    Google Scholar 

  26. Milutinović, D., Garg, D.P.: A sampling approach to modeling and control of a large-size robot population. In: Proceedings of the 2010 ASME Dynamic Systems and Control Conference. ASME, Boston (2010)

    Google Scholar 

  27. Oksendal, B.: Stochastic Differential Equations: An Introduction with Applications, 6th edn. Springer, Berlin (2003)

    Google Scholar 

  28. Palmer, A., Milutinović, D.: A Hamiltonian Approach Using Partial Differential Equations for Open-Loop Stochastic Optimal Control. In: Proceedings of the 2011 American Control Conference, San Francisco, CA (2011)

    Google Scholar 

  29. Parker, L.E.: Multiple Mobile Robot Systems. In: Sciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, ch.40, pp. 921–941. Springer (2008)

    Google Scholar 

  30. Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley Interscience, Hoboken (2007)

    Book  MATH  Google Scholar 

  31. Ren, W., Beard, R.: Distributed consensus in multi-vehicle cooperative control: Theory and applications. Springer, New York (2007)

    Google Scholar 

  32. Ren, W., Beard, R., Atkins, E.: A survey of consensus problems in multi-agent coordination. In: Proceedings of the 2005, American Control Conference, pp. 1859–1864 (2005)

    Google Scholar 

  33. Ryan, A., Zennaro, M., Howell, A., Sengupta, R., Hedrick, J.: An overview of emerging results in cooperative UAV control. In: 2004 43rd IEEE Conference on Decision and Control, vol. 1, pp. 602–607 (2004)

    Google Scholar 

  34. Särkkä, S.: Continuous-time and continuous-discrete-time unscented Rauch-Tung-Striebel smoothers. Signal Processing 90(1), 225–235 (2010)

    Article  MATH  Google Scholar 

  35. Singh, A., Batalin, M., Stealey, M., Chen, V., Hansen, M., Harmon, T., Sukhatme, G., Kaiser, W.: Mobile robot sensing for environmental applications. In: Field and Service Robotics, pp. 125–135. Springer (2008)

    Google Scholar 

  36. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  37. Theodorou, E., Buchli, J., Schaal, S.: A Generalized Path Integral Control Approach to Reinforcement Learning. The Journal of Machine Learning Research 11, 3137–3181 (2010)

    MathSciNet  MATH  Google Scholar 

  38. Theodorou, E., Buchli, J., Schaal, S.: Reinforcement learning of motor skills in high dimensions: A path integral approach. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), vol. 4, pp. 2397–2403. IEEE (2010)

    Google Scholar 

  39. Todorov, E.: General duality between optimal control and estimation. In: 47th IEEE Conference on Decision and Control, vol. 5, pp. 4286–4292. IEEE, Cancun (2008)

    Chapter  Google Scholar 

  40. Wang, M.C., Uhlenbeck, G.: On the theory of Brownian Motion II. Reviews of Modern Physics 17(2-3), 323–342 (1945)

    Article  MathSciNet  MATH  Google Scholar 

  41. Wiegerinck, W., van den Broek, B., Kappen, B.: Stochastic optimal control in continuous space-time multi-agent systems. In: Proceedings UAI. Citeseer (2006)

    Google Scholar 

  42. Wiegerinck, W., van den Broek, B., Kappen, B.: Optimal on-line scheduling in stochastic multiagent systems in continuous space-time. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, p. 1 (2007)

    Google Scholar 

  43. Yin, G.G., Zhu, C.: Hybrid Switching Diffusions. Springer, New York (2010)

    Book  MATH  Google Scholar 

  44. Yong, J.: Relations among ODEs, PDEs, FSDEs, BDSEs, and FBSDEs. In: Proceedings of the 36th IEEE Conference on Decision and Control, pp. 2779–2784. IEEE, San Diego (1997)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California, Santa Cruz, 1156 High Street, Santa Cruz, CA, 95060, USA

    Ross P. Anderson & Dejan Milutinović

Authors
  1. Ross P. Anderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dejan Milutinović
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ross P. Anderson .

Editor information

Editors and Affiliations

  1. , Applied Mathematics and Statistics Dept, University of California, High St, M/S SOE2 1156, Santa Cruz, 95064, USA

    Dejan Milutinović

  2. , Computer Engineering Dept, University of California, High St 1156, Santa Cruz, 95064, California, USA

    Jacob Rosen

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Anderson, R.P., Milutinović, D. (2013). Kalman Smoothing for Distributed Optimal Feedback Control of Unicycle Formations. In: Milutinović, D., Rosen, J. (eds) Redundancy in Robot Manipulators and Multi-Robot Systems. Lecture Notes in Electrical Engineering, vol 57. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33971-4_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33971-4_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33970-7

  • Online ISBN: 978-3-642-33971-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation