Skip to main content

Advertisement

SpringerLink
Log in

Generalization techniques of neural networks for fluid flow estimation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

We demonstrate several techniques to encourage practical uses of neural networks for fluid flow estimation. In the present paper, three perspectives which are remaining challenges for applications of machine learning to fluid dynamics are considered: 1. interpretability of machine-learned results, 2. bulking out of training data, and 3. generalizability of neural networks. For the interpretability, we first demonstrate two methods to observe the internal procedure of neural networks, i.e., visualization of hidden layers and application of gradient-weighted class activation mapping (Grad-CAM), applied to canonical fluid flow estimation problems—(1) drag coefficient estimation of a cylinder wake and (2) velocity estimation from particle images. It is exemplified that both approaches can successfully tell us evidences of the great capability of machine learning-based estimations. We then utilize some techniques to bulk out training data for super-resolution analysis and temporal prediction for cylinder wake and NOAA sea surface temperature data to demonstrate that sufficient training of neural networks with limited amount of training data can be achieved for fluid flow problems. The generalizability of machine learning model is also discussed by accounting for the perspectives of inter/extrapolation of training data, considering super-resolution of wakes behind two parallel cylinders. We find that various flow patterns generated by complex interaction between two cylinders can be reconstructed well, even for the test configurations regarding the distance factor. The present paper can be a significant step toward practical uses of neural networks for both laminar and turbulent flow problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Kreinovich VY (1991) Arbitrary nonlinearity is sufficient to represent all functions by neural networks: a theorem. Neural Netw. 4:381–383

    Google Scholar 

  2. Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw. 4:251–257

    Google Scholar 

  3. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2:303–314

    MathSciNet  MATH  Google Scholar 

  4. Baral C, Fuentes O, Kreinovich V (2018) Why deep neural networks: a possible theoretical explanation. In: Ceberio M, Kreinovich V (eds) Constraint programming and decision making: theory and applications. Studies in systems, decision and control, vol 100. Springer, Cham. https://doi.org/10.1007/978-3-319-61753-4_1

    Chapter  Google Scholar 

  5. Brunton SL, Noack BR, Koumoutsakos P (2020) Machine learning for fluid mechanics. Annu. Rev. Fluid Mech. 52:477–508

    MATH  Google Scholar 

  6. Duraisamy K, Iaccarino G, Xiao H (2019) Turbulence modeling in the age of data. Annu. Rev. Fluid. Mech. 51:357–377

    MathSciNet  MATH  Google Scholar 

  7. Gamahara M, Hattori Y (2017) Searching for turbulence models by artificial neural network. Phys. Rev. Fluids 2(5):054604

    Google Scholar 

  8. Maulik R, San O (2017) A neural network approach for the blind deconvolution of turbulent flows. J. Fluid Mech. 831:151–181

    MathSciNet  MATH  Google Scholar 

  9. Maulik R, San O, Jacob JD, Crick C (2019) Sub-grid scale model classification and blending through deep learning. J. Fluid Mech. 870:784–812

    MathSciNet  MATH  Google Scholar 

  10. Maulik R, San O, Rasheed A, Vedula P (2019) Subgrid modelling for two-dimensional turbulence using neural networks. J. Fluid Mech. 858:122–144

    MathSciNet  MATH  Google Scholar 

  11. Yang XIA, Zafar S, Wang J-X, Xiao H (2019) Predictive large-eddy-simulation wall modeling via physics-informed neural networks. Phys. Rev. Fluids 4(3):034602

    Google Scholar 

  12. Pawar S, San O, Rasheed A, Vedula P (2020) A priori analysis on deep learning of subgrid-scale parameterizations for kraichnan turbulence. Theor. Comput. Fluid Dyn. 34:429–455

    MathSciNet  Google Scholar 

  13. Ling J, Kurzawski A, Templeton J (2016) Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807:155–166

    MathSciNet  MATH  Google Scholar 

  14. Milani PM, Ling J, Eaton JK (2020) Turbulent scalar flux in inclined jets in crossflow: counter gradient transport and deep learning modelling. J Fluid Mech 906:A27

    MathSciNet  MATH  Google Scholar 

  15. Geneva N, Zabaras N (2019) Quantifying model form uncertainty in Reynolds-averaged turbulence models with bayesian deep neural networks. J. Comput. Phys. 383:125–147

    MathSciNet  MATH  Google Scholar 

  16. Novati G, de Laroussilhe HL, Koumoutsakos P (2021) Automating turbulence modeling by multi-agent reinforcement learning. Nat. Mach. Intell. 3:87–96

    Google Scholar 

  17. Taira K, Hemati MS, Brunton SL, Sun Y, Duraisamy K, Bagheri S, Dawson S, Yeh CA (2020) Modal analysis of fluid flows: Applications and outlook. AIAA J. 58(3):998–1022

    Google Scholar 

  18. Wang Z, Xiao D, Fang F, Govindan R, Pain CC, Guo Y (2018) Model identification of reduced order fluid dynamics systems using deep learning. Int. J. Numer. Methods Fluids 86(4):255–268

    MathSciNet  Google Scholar 

  19. Srinivasan PA, Guastoni L, Azizpour H, Schlatter P, Vinuesa R (2019) Predictions of turbulent shear flows using deep neural networks. Phys. Rev. Fluids 4:054603

    Google Scholar 

  20. Milano M, Koumoutsakos P (2002) Neural network modeling for near wall turbulent flow. J. Comput. Phys. 182:1–26

    MATH  Google Scholar 

  21. Fukami K, Hasegawa K, Nakamura T, Morimoto M, Fukagata K (2020) Model order reduction with neural networks: application to laminar and turbulent flows. SN Comput Sci 2:467

    Google Scholar 

  22. Murata T, Fukami K, Fukagata K (2020) Nonlinear mode decomposition with convolutional neural networks for fluid dynamics. J. Fluid Mech. 882:A13

    MathSciNet  MATH  Google Scholar 

  23. Fukami K, Nakamura T, Fukagata K (2020) Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data. Phys. Fluids 32:095110

    Google Scholar 

  24. Fukami K, Fukagata K, Taira K (2020) Assessment of supervised machine learning for fluid flows. Theor. Comput. Fluid Dyn. 34(4):497–519

    MathSciNet  MATH  Google Scholar 

  25. Fukami K, Nabae Y, Kawai K, Fukagata K (2019) Synthetic turbulent inflow generator using machine learning. Phys. Rev. Fluids 4:064603

    Google Scholar 

  26. Salehipour H, Peltier WR (2019) Deep learning of mixing by two ‘atoms’ of stratified turbulence. J. Fluid Mech. 861:R4

    MathSciNet  MATH  Google Scholar 

  27. Fukami K, Fukagata K, Taira K (2019) Super-resolution reconstruction of turbulent flows with machine learning. J. Fluid Mech. 870:106–120

    MathSciNet  MATH  Google Scholar 

  28. Fukami K, Fukagata K, Taira K (2019) Super-resolution analysis with machine learning for low-resolution flow data. In: 11th International Symposium on Turbulence and Shear Flow Phenomena (TSFP11), Southampton, UK, number 208,

  29. Liu B, Tang J, Huang H, Lu X-Y (2020) Deep learning methods for super-resolution reconstruction of turbulent flows. Phys. Fluids 32:025105

    Google Scholar 

  30. Deng Z, He C, Liu Y, Kim KC (2019) Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework. Phys. Fluids 31:125111

    Google Scholar 

  31. Fukami K, Fukagata K, Taira K (2021) Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows. J. Fluid Mech., 909(A9),

  32. Cai S, Zhou S, Xu C, Gao Q (2019) Dense motion estimation of particle images via a convolutional neural network. Exp. Fluids 60:60–73

    Google Scholar 

  33. Morimoto M, Fukami K, Fukagata K (2021) Experimental velocity data estimation for imperfect particle images using machine learning. Phys Fluids 33:087121

    Google Scholar 

  34. Brunton SL, Hemanti MS, Taira K (2020) Special issue on machine learning and data-driven methods in fluid dynamics. Theor. Comput. Fluid Dyn. 34(4):333–337

    MathSciNet  Google Scholar 

  35. Lee C, Kim J, Babcock D, Goodman R (1997) Application of neural networks to turbulence control for drag reduction. Phys. Fluids 9(6):1740–1747

    Google Scholar 

  36. Choi H, Moin P, Kim J (1994) Active turbulence control for drag reduction in wall-bounded flows. J. Fluid Mech. 262(3):75–110

    MATH  Google Scholar 

  37. Garnier P, Viquerat J, Rabault J, Larcher A, Kuhnle A, Hachem E (2021) A review on deep reinforcement learning for fluid mechanics. Comput Fluids 225:104973

    MathSciNet  MATH  Google Scholar 

  38. Rabault J, Kuchta M, Jensen A, Réglade U, Cerardi N (2019) Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. J. Fluid Mech. 865:281–302

    MathSciNet  MATH  Google Scholar 

  39. Tang H, Rabault J, Kuhnle A, Wang Y, Wang T (2020) Robust active flow control over a range of reynolds numbers using an artificial neural network trained through deep reinforcement learning. Phys. Fluids 32(5):053605

    Google Scholar 

  40. Maulik R, Fukami K, Ramachandra N, Fukagata K, Taira K (2020) Probabilistic neural networks for fluid flow surrogate modeling and data recovery. Phys. Rev. Fluids 5:104401

    Google Scholar 

  41. Jagodinski E, Zhu X, Verma S (2020) Uncovering dynamically critical regions in near-wall turbulence 3D convolutional neural networks. arXiv:2004.06187

  42. Selvaraju R. R., Das A, Vedantam R, Cogswell M, Parikh D, Batra D (2016) Grad-CAM: Why did you say that? arXiv:1611.07450

  43. Selvaraju R. R., Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: Proc. IEEE Int. Conf. Comput. Vis., pages 618–626

  44. Kim J, Lee C (2020) Prediction of turbulent heat transfer using convolutional neural networks. J. Fluid Mech. 882:A18

    MathSciNet  MATH  Google Scholar 

  45. Kutz JN (2017) Deep learning in fluid dynamics. J. Fluid Mech. 814:1–4

    MATH  Google Scholar 

  46. Hasegawa K, Fukami K, Murata T, Fukagata K (2020) CNN-LSTM based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers. Fluid Dyn. Res. 52(6):065501

    MathSciNet  Google Scholar 

  47. Hasegawa K, Fukami K, Murata T, Fukagata K (2020) Machine-learning-based reduced-order modeling for unsteady flows around bluff bodies of various shapes. Theor. Comput. Fluid Dyn. 34(4):367–388

    MathSciNet  Google Scholar 

  48. Erichson NB, Mathelin L, Yao Z, Brunton SL, Mahoney MW, Kutz JN (2020) Shallow learning for fluid flow reconstruction with limited sensors. Proc. Royal Soc. A 476(2238):20200097

    MATH  Google Scholar 

  49. Kor H, Ghomizad M. Badri, Fukagata K (2017) A unified interpolation stencil for ghost-cell immersed boundary method for flow around complex geometries. J. Fluid Sci. Technol., 12(1):JFST0011

  50. Franke R, Rodi W, Schonung B (1990) Numerical calculation of laminar vortex shedding flow past cylinders. J. Wind Eng. Ind. Aerodyn. 35:237–257

    Google Scholar 

  51. Robichaux J, Balachandar S, Vanka SP (1999) Three-dimensional floquet instability of the wake of square cylinder. Phys. Fluids 11:560

    MathSciNet  MATH  Google Scholar 

  52. Caltagirone JP (1994) Sur l’interaction fluide-milieu poreux: application au calcul des efforts excerses sur un obstacle par un fluide visqueux. C. R. Acad. Sci. Paris 318:571–577

    MATH  Google Scholar 

  53. Bai H, Alam MdM (2018) Dependence of square cylinder wake on Reynolds number. Phys. Fluids 30:015102

    Google Scholar 

  54. Available on https://www.esrl.noaa.gov/psd/

  55. Weller HG, Tabor G, Jasak H, Fureby C (1998) A tensorial approach to computational continuum mechanics using object-oriented techniques. Comput. Phys. 12(6):620–631

    Google Scholar 

  56. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagation errors. Nature 322:533–536

    MATH  Google Scholar 

  57. Domingos P (2012) A few useful things to know about machine learning. Communications of the ACM 55(10):78–87

    Google Scholar 

  58. Lui HFS, Wolf WR (2019) Construction of reduced-order models for fluid flows using deep feedforward neural networks. J. Fluid Mech. 872:963–994

    MathSciNet  MATH  Google Scholar 

  59. Yu J, Hesthaven JS (2019) Flowfield reconstruction method using artificial neural network. AIAA J. 57(2):482–498

    Google Scholar 

  60. Kingma D. P., Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  61. Nair V, Hinton G. E. (2010) Rectified linear units improve restricted boltzmann machines. Proc. Int. Conf. Mach. Learn., pages 807–814

  62. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc. IEEE 86(11):2278–2324

    Google Scholar 

  63. Matsuo M, Nakamura T, Morimoto M, Fukami K, Fukagata K (2021) Supervised convolutional network for three-dimensional fluid data reconstruction from sectional flow fields with adaptive super-resolution assistance. arXiv:2103.09020

  64. Moriya N, Fukami K, Nabae Y, Morimoto M, Nakamura T, Fukagata K (2021) Inserting machine-learned virtual wall velocity for large-eddy simulation of turbulent channel flows. arXiv:2106.09271

  65. Nakamura T, Fukami K, Fukagata K (2021) Comparison of linear regressions and neural networks for fluid flow problems assisted with error-curve analysis. arXiv:2105.00913

  66. Du X, Qu X, He Y, Guo D (2018) Single image super-resolution based on multi-scale competitive convolutional neural network. Sensors 18(789):1–17

    Google Scholar 

  67. Zhang Y, Sung W, Marvis D (2018) Application of convolutional neural network to predict airfoil lift coefficient. AIAA paper, 2018–1903

  68. Miyanawala T.P., Jaiman R.K. (2018) A novel deep learning method for the predictions of current forces on bluff bodies. In: Proceedings of the ASME 2018 37th International Conference on Ocean, Offshore and Arctic Engineering OMAE2018, pages 1–10

  69. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations

  70. Guastoni L, Güemes A, Ianiro A, Discetti S, Schlatter P, Azizpour H, Vinuesa R (2020) Convolutional-network models to predict wall-bounded turbulence from wall quantities. J Fluid Mech 928:A27

    MathSciNet  MATH  Google Scholar 

  71. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J. Big Data 6(1):60

    Google Scholar 

  72. Huang J, Liu H, Wang Q, Cai W (2020) Limited-projection volumetric tomography for time-resolved turbulent combustion diagnostics via deep learning. Aerosp. Sci. Technol., 106(106123)

  73. Mikołajczyk A, Grochowski M (2018) Data augmentation for improving deep learning in image classification problem. In: 2018 international interdisciplinary PhD workshop (IIPhDW), pages 117–122. IEEE

  74. Maulik R, San O (2017) Resolution and energy dissipation characteristics of implicit LES and explicit filtering models for compressible turbulence. Fluids 2(2):14

    Google Scholar 

  75. Duraisamy K (2021) Perspectives on machine learning-augmented reynolds-averaged and large eddy simulation models of turbulence. Phys. Rev. Fluids 6:050504

    Google Scholar 

  76. Fukami K, Maulik R, Ramachandra N, Fukagata K, Taira K (2021) Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning. Nat Mach Intell. https://doi.org/10.1038/s42256-021-00402-2

    Article  Google Scholar 

  77. Stengel K, Glaws A, Hettinger D, King RN (2020) Adversarial super-resolution of climatological wind and solar data. Proc. Natl. Acad. Sci. USA 117(29):16805–16815

    Google Scholar 

  78. Nakamura T, Fukami K, Hasegawa K, Nabae Y, Fukagata K (2021) Convolutional neural network and long short-term memory based reduced order surrogate for minimal turbulent channel flow. Phys. Fluids 33:025116

    Google Scholar 

  79. Brunton SL, Proctor JL, Kutz JN (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. USA 113(15):3932–3937

    MathSciNet  MATH  Google Scholar 

  80. Fukami K, Murata T, Fukagata K (2021) Sparse identification of nonlinear dynamics with low-dimensionalized flow representations. J Fluid Mech 926:A10

    MathSciNet  MATH  Google Scholar 

  81. Saku Y, Aizawa M, Ooi T, Ishigami G (2021) Spatio-temporal prediction of soil deformation in bucket excavation using machine learning. Adv Robot. https://doi.org/10.1080/01691864.2021.1943521

    Article  Google Scholar 

  82. Morimoto M, Fukami K, Zhang K, Nair AG, Fukagata K (2021) Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low-dimensionalization. Theor Comput Fluid Dyn 35:633–658

    MathSciNet  Google Scholar 

  83. Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science 367(6481):1026–1030

    MathSciNet  MATH  Google Scholar 

  84. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2021) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24

    MathSciNet  Google Scholar 

  85. Inubushi M, Goto S (2019) Transferring reservoir computing: Formulation and application to fluid physics. In: International Conference on Artificial Neural Networks, pages 193–199. Springer

  86. Manohar K, Brunton BW, Kutz JN, Brunton SL (2018) Data-driven sparse sensor placement for reconstruction: Demonstrating the benefits of exploiting known patterns. IEEE Control Syst. 38(3):63–86

    MathSciNet  MATH  Google Scholar 

  87. Nakai K, Yamada K, Nagata T, Saito Y, Nonomura T (2020) Effect of objective function on data-driven sparse sensor optimization. IEEE Access 9:46731–46743

    Google Scholar 

  88. Saito Y, Nonomura T, Nankai K, Yamada K, Asai K, Tsubakino Y, Tsubakino D () Data-driven vector-measurement-sensor selection based on greedy algorithm. IEEE Sensors Letters, 2020

Download references

Acknowledgments

This work was supported from Japan Society for the Promotion of Science (KAKENHI grant number: 18H03758, 21H05007).

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Keio University, Yokohama, 223-8522, Japan

    Masaki Morimoto, Kai Fukami & Koji Fukagata

  2. Department of Mechanical and Aerospace Engineering, University of California, Los Angeles, CA, 90095, USA

    Kai Fukami

  3. Department of Mechanical and Aerospace Engineering, Rutgers University, Piscataway, NJ, 08854, USA

    Kai Zhang

Authors
  1. Masaki Morimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Fukami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Koji Fukagata
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masaki Morimoto.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morimoto, M., Fukami, K., Zhang, K. et al. Generalization techniques of neural networks for fluid flow estimation. Neural Comput & Applic 34, 3647–3669 (2022). https://doi.org/10.1007/s00521-021-06633-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06633-z

Keywords

Search

Navigation