Skip to main content
SpringerLink
Log in

A statistical learning strategy for closed-loop control of fluid flows

  • Original Article
  • Published:
Theoretical and Computational Fluid Dynamics Aims and scope Submit manuscript

Abstract

This work discusses a closed-loop control strategy for complex systems utilizing scarce and streaming data. A discrete embedding space is first built using hash functions applied to the sensor measurements from which a Markov process model is derived, approximating the complex system’s dynamics. A control strategy is then learned using reinforcement learning once rewards relevant with respect to the control objective are identified. This method is designed for experimental configurations, requiring no computations nor prior knowledge of the system, and enjoys intrinsic robustness. It is illustrated on two systems: the control of the transitions of a Lorenz’63 dynamical system, and the control of the drag of a cylinder flow. The method is shown to perform well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gerhard, J., Pastoor, M., King, R., Noack, B.R., Dillmann, A., Morzynski, M., Tadmor, G.: Model-based control of vortex shedding using low-dimensional Galerkin models. AIAA J. 4262(2003), 115–173 (2003)

    Google Scholar 

  2. Bergmann, M., Cordier, L.: Optimal control of the cylinder wake in the laminar regime by trust-region methods and pod reduced-order models. J. Comput. Phys. 227(16), 7813–7840 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  3. Ma, Z., Ahuja, S., Rowley, C.W.: Reduced-order models for control of fluids using the eigensystem realization algorithm. Theor. Comput. Fluid Dyn. 25(1–4), 233–247 (2011)

    Article  MATH  Google Scholar 

  4. Joe, W.T., Colonius, T., MacMynowski, D.G.: Feedback control of vortex shedding from an inclined flat plate. Theor. Comput. Fluid Dyn. 25(1–4), 221–232 (2011)

    Article  MATH  Google Scholar 

  5. Mathelin, L., Pastur, L., Le Maître, O.: A compressed-sensing approach for closed-loop optimal control of nonlinear systems. Theor. Comput. Fluid Dyn. 26(1–4), 319–337 (2012)

    Article  MATH  Google Scholar 

  6. Cordier, L., Noack, B.R., Tissot, G., Lehnasch, G., Delville, J., Balajewicz, M., Daviller, G., Niven, R.K.: Identification strategies for model-based control. Exp. Fluids 54(8), 1–21 (2013)

    Article  Google Scholar 

  7. Lee, C., Kim, J., Babcock, D., Goodman, R.: Application of neural networks to turbulence control for drag reduction. Phys. Fluids 9(6), 1740–1747 (1997)

    Article  Google Scholar 

  8. Kegerise, M.A., Cambell, R.H., Cattafesta, L.N.: Real time feedback control of flow-induced cavity tones—part 2: adaptive control. J. Sound Vib. 307, 924–940 (2007)

    Article  Google Scholar 

  9. Huang, S.-C., Kim, J.: Control and system identification of a separated flow. Phys. Fluids 20(10), 101509 (2008)

    Article  MATH  Google Scholar 

  10. Hervé, A., Sipp, D., Schmid, P.J., Samuelides, M.: A physics-based approach to flow control using system identification. J. Fluid Mech. 702, 26–58 (2012)

    Article  MATH  Google Scholar 

  11. Gautier, N., Aider, J.-L., Duriez, T., Noack, B.R., Segond, M., Abel, M.W.: Closed-loop separation control using machine learning. J. Fluid Mech. 770, 442–457 (2015)

    Article  Google Scholar 

  12. Brunton, S., Noack, B.: Closed-loop turbulence control: progress and challenges. Appl. Mech. Rev. 67(5), 050801 (2015)

    Article  Google Scholar 

  13. Slaney, M., Casey, M.: Locality-sensitive hashing for finding nearest neighbors [lecture notes]. IEEE Signal Process. Mag. 25(2), 128–131 (2008)

    Article  Google Scholar 

  14. Kaiser, E., Noack, B.R., Cordier, L., Spohn, A., Segond, M., Abel, M., Daviller, G., Östh, J., Krajnović, S., Niven, R.K.: Cluster-based reduced-order modelling of a mixing layer. J. Fluid Mech. 754, 365–414 (2014)

    Article  MATH  Google Scholar 

  15. Mandl, P.: Estimation and control in markov chains. Adv. Appl. Probab. 6(1), 40–60 (1974)

  16. Watkins, C., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)

    MATH  Google Scholar 

  17. Gosavi, A.: Target-sensitive control of Markov and semi-Markov processes. Int. J. Control Autom. 9(5), 941–951 (2011)

    Article  MathSciNet  Google Scholar 

  18. Lin, C.T., Jou, C.P.: Controlling chaos by GA-based reinforcement learning neural network. IEEE Trans. Neural Netw. 10(4), 846–859 (1999)

    Article  Google Scholar 

  19. Gadaleta, S., Dangelmayr, G.: Optimal chaos control through reinforcement learning. Chaos 9(3), 775–788 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  20. Grassberger, P., Procaccia, I.: Measuring the strangeness of strange attractors. Phys. D 9, 189–208 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  21. Takens, F., Rand, D.A., Young, L.S.: Dynamical systems and turbulence. Lect. Notes Math. 898(9), 366 (1981)

    Article  MathSciNet  Google Scholar 

  22. Carter, J.L., Wegman, M.N.: Universal classes of hash functions. In: Proceedings of 9th Annual ACM Theory of Computing, pp. 106–112. ACM (1977)

  23. Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, pp. 459–468. IEEE (2006)

  24. Johnson, W., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26, 189–206 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  25. Novikov, E.A.: Two-particle description of turbulence, Markov property, and intermittency. Phys. Fluids 1(2), 326–330 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  26. Renner, C., Peinke, J., Friedrich, R.: Experimental indications for Markov properties of small-scale turbulence. J. Fluid Mech. 433, 383–409 (2001)

    Article  MATH  Google Scholar 

  27. Bellman, R.: On the theory of dynamic programming. Proc. Natl. Acad. Sci. USA 38(8), 716 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  28. Powell, W.: Approximate Dynamic Programming: Solving the Curses of Dimensionality, vol. 703. Wiley, London (2007)

    Book  MATH  Google Scholar 

  29. Lewis, F., Vrabie, D.: Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag. 9(3), 32–50 (2009)

    Article  MathSciNet  Google Scholar 

  30. Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20(2), 130–141 (1963)

    Article  Google Scholar 

  31. Le Maître, O.P., Scanlan, R.H., Knio, O.M.: Estimation of the flutter derivatives of an NACA airfoil by means of Navier–Stokes simulation. J. Fluids Struct. 17(1), 1–28 (2003)

    Article  Google Scholar 

  32. Lusseyran, F., Pastur, L.R., Letellier, C.: Dynamical analysis of an intermittency in an open cavity flow. Phys. Fluids 20(11), 114101 (2008)

    Article  MATH  Google Scholar 

  33. Gorodetsky, A.A., Karaman, S., Marzouk, Y.M.: Efficient high-dimensional stochastic optimal motion control using tensor-train decomposition. In: Robotics: Science and Systems XI, Sapienza University of Rome, Italy, July 13–17 (2015)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Florida State University, Tallahassee, FL, 32306-4510, USA

    Florimond Guéniat & M. Yousuff Hussaini

  2. LIMSI-CNRS, rue J. von Neumann, Campus Universitaire d’Orsay, 91405, Orsay Cedex, France

    Lionel Mathelin

Authors
  1. Florimond Guéniat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lionel Mathelin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Yousuff Hussaini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lionel Mathelin.

Additional information

Communicated by Omar M. Knio.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guéniat, F., Mathelin, L. & Hussaini, M.Y. A statistical learning strategy for closed-loop control of fluid flows. Theor. Comput. Fluid Dyn. 30, 497–510 (2016). https://doi.org/10.1007/s00162-016-0392-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00162-016-0392-y

Keywords

Search

Navigation