Skip to main content
SpringerLink
Log in

Topology-preserving flocking of nonlinear agents using optimistic planning

  • Published:
Control Theory and Technology Aims and scope Submit manuscript

Abstract

We consider the generalized flocking problem in multiagent systems, where the agents must drive a subset of their state variables to common values, while communication is constrained by a proximity relationship in terms of another subset of variables. We build a flocking method for general nonlinear agent dynamics, by using at each agent a near-optimal control technique from artificial intelligence called optimistic planning. By defining the rewards to be optimized in a well-chosen way, the preservation of the interconnection topology is guaranteed, under a controllability assumption. We also give a practical variant of the algorithm that does not require to know the details of this assumption, and show that it works well in experiments on nonlinear agents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. Olfati-Saber, J. A. Fax, R. M. Murray. Consensus and cooperation in networked multiagent systems. Proceedings of the IEEE, 2007, 95(1): 215–233.

    Article  Google Scholar 

  2. W. Ren, R. W. Beard. Distributed Consensus in Multi-vehicle Cooperative Control: Theory and Applications. Communications and Control Engineering. Berlin: Springer, 2008.

    Google Scholar 

  3. R. Olfati-Saber. Flocking for multi-agent dynamic systems: Algorithms and theory. IEEE Transactions on Automatic Control, 2006, 51(3): 401–420.

    Article  MathSciNet  Google Scholar 

  4. H. G. Tanner, A. Jadbabaie, G. J. Pappas. Flocking in fixed and switching networks. IEEE Transactions on Automatic Control, 2007, 52(5): 863–868.

    Article  MathSciNet  Google Scholar 

  5. W. Dong. Flocking of multiple mobile robots based on backstepping. IEEE Transactions on Systems, Man, and Cybernetics — Part B: Cybernetics, 2011, 41(2): 414–424.

    Article  Google Scholar 

  6. J.-F. Hren, R. Munos. Optimistic planning of deterministic systems. Proceedings 8th European Workshop on Reinforcement Learning, Villeneuve d’Ascq, France: Springer, 2008: 151–164.

    Google Scholar 

  7. C. De Persis, P. Frasca. Robust self-triggered coordination with ternary controllers. IEEE Transactions on Automatic Control, 2013, 58(12): 3024–3038.

    Article  Google Scholar 

  8. J. Mei, W. Ren, G. Ma. Distributed coordinated tracking with a dynamic leader for multiple Euler-Lagrange systems. IEEE Transactions on Automatic Control, 2011, 56(6): 1415–1421.

    Article  MathSciNet  Google Scholar 

  9. M. M. Zavlanos, G. J. Pappas. Distributed connectivity control of mobile networks. IEEE Transactions on Robotics, 2008, 24(6): 1416–1428.

    Article  Google Scholar 

  10. M. Fiacchini, I. C. Morărescu. Convex conditions on decentralized control for graph topology preservation. IEEE Transactions on Automatic Control, 2014, 59(6): 1640–1645.

    Article  Google Scholar 

  11. F. Bullo, J. Cortés, S. Martinez. Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms. Princeton: Princeton University Press, 2009.

    Book  Google Scholar 

  12. J. Zhu, J. Lu, X. Yu. Flocking of multi-agent non-holonomic systems with proximity graphs. IEEE Transactions on Circuits and Systems, 2013, 60(1): 199–210.

    Article  MathSciNet  Google Scholar 

  13. H. Su, G. Chen, X. Wang, et al. Adaptive second-order consensus of networked mobile agents with nonlinear dynamics. Automatica, 2011, 47(2): 368–375.

    Article  MATH  MathSciNet  Google Scholar 

  14. J. Zhou, X. Wu, W. Yu, et al. Flocking of multi-agent dynamical systems based on pseudo-leader mechanism. Systems & Control Letters, 2012, 61(1): 195–202.

    Article  MATH  MathSciNet  Google Scholar 

  15. H. Tanner, A. Jadbabaie, G. Pappas. Flocking in teams of nonholonomic agents. Cooperative Control. V. Kumar, N. Leonard, A. Morse (eds.), Berlin: Springer, 2005: 458–460.

    Google Scholar 

  16. L. Buşoniu, C. Morărescu. Optimistic planning for consensus. Proceedings of the American Control Conference, Washington D.C.: IEEE, 2013: 6735–6740.

    Google Scholar 

  17. Lucian Buşoniu, C. Morărescu. Consensus for blackbox nonlinear agents using optimistic optimization. Automatica, 2014, 50(4): 1201–1208.

    Article  MATH  MathSciNet  Google Scholar 

  18. B. Jakubczyk, E. D. Sontag. Controllability of nonlinear discretetime systems: A Lie-algebraic approach. SIAM Journal of Control and Optimization, 1990, 28(1): 1–33.

    Article  MATH  MathSciNet  Google Scholar 

  19. R. S. Sutton, A. G. Barto. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.

    Google Scholar 

  20. F. Lewis, D. Liu, eds. Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Hoboken: John Wiley & Sons, 2012.

    Google Scholar 

  21. R. Munos. The optimistic principle applied to games, optimization and planning: Towards foundations of Monte-Carlo tree search. Foundations and Trends in Machine Learning, 2014, 7(1): 1–130.

    Article  MATH  Google Scholar 

  22. R. R. Negenborn, B. De Schutter, H. Hellendoorn. Multi-agent model predictive control for transportation networks: Serial versus parallel schemes. Engineering Applications of Artificial Intelligence, 2008, 21(3): 353–366.

    Article  Google Scholar 

  23. J. Liu, X. Chen, D. M. de la Peña, et al. Sequential and iterative architectures for distributed model predictive control of nonlinear process systems. American Institute of Chemical Engineers (AIChE) Journal, 2010, 56(8): 2137–2149.

    Google Scholar 

  24. L. Buşoniu, R. Munos, Robert Babuška. A review of optimistic planning in Markov decision processes. Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control. F. Lewis, D. Liu (eds.), Hoboken: John Wiley & Sons, 2012: DOI 10.1002/9781118453988.ch22.

    Google Scholar 

  25. L. Buşoniu, D. Ernst, B. De Schutter, et al. Approximate dynamic programming with a fuzzy parameterization. Automatica, 2010, 46(5): 804–814.

    Article  MATH  MathSciNet  Google Scholar 

  26. B. Kiumarsi, F. Lewis, H. Modares, et al. Reinforcement Qlearning for optimal tracking control of linear discrete-time systems with unknown dynamics. Automatica, 2014, 50(4): 1167–1175.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Automation, Technical University of Cluj-Napoca, Memorandumului 28, 400114, Cluj-Napoca, Romania

    Lucian Buşoniu

  2. CRAN, UMR 7039 and CNRS, CRAN, UMR 7039, Université de Lorraine, 2 Avenue de la Forêt de Haye, Vandoeuvre-lès-Nancy, France

    Irinel-Constantin Morărescu

Authors
  1. Lucian Buşoniu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Irinel-Constantin Morărescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lucian Buşoniu.

Additional information

This work was supported by a Programme Hubert Curien-Brancusi cooperation grant (CNCS-UEFISCDI contract no. 781/2014 and Campus France grant no. 32610SE). Additionally, the work of L. Buşoniu was supported by the Romanian National Authority for Scientific Research, CNCS-UEFISCDI (No. PNII-RU-TE-2012-3-0040). The work of I.-C. Morărescu was partially funded by the National Research Agency (ANR) project “Computation Aware Control Systems” (No. ANR-13-BS03-004-02).

Lucian BUŞONIU received the M.Sc. degree (valedictorian) from the Technical University of Cluj-Napoca, Cluj-Napoca, Romania, in 2003, and the Ph.D. degree (cum laude) from the Delft University of Technology, Delft, the Netherlands, in 2009. He is an associate professor with the Department of Automation, Technical University of Cluj-Napoca, Romania. His research interests include planning-based methods for nonlinear optimal control, reinforcement learning and dynamic programming with function approximation, multi-agent systems, and, more generally, intelligent and learning techniques for control. He has authored a book as well as a number of journals, conferences, and chapter publications on these topics. Dr. Buşoniu was the recipient of the 2009 Andrew P. Sage Award for the best paper in the IEEE Transactions on Systems, Man, and Cybernetics.

Irinel-Constantin MORĂRESCU is currently an associate professor at Université de Lorraine and a researcher at the Research Centre of Automatic Control (CRAN UMR 7039 CNRS) in Nancy, France. He received the B.Sc. and the M.Sc. degrees in Mathematics from University of Bucharest, Romania, in 1997 and 1999, respectively. In 2006, he received the Ph.D. degree in Mathematics and Technology of Information and Systems from University of Bucharest and University of Technology of Compiègne, respectively. His works concern stability and control of time-delay systems, tracking control for nonsmooth mechanical systems, consensus and synchronization problems.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Buşoniu, L., Morărescu, IC. Topology-preserving flocking of nonlinear agents using optimistic planning. Control Theory Technol. 13, 70–81 (2015). https://doi.org/10.1007/s11768-015-4107-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11768-015-4107-5

Keywords

Search

Navigation