Advertisement

The European Physical Journal Special Topics

, Volume 223, Issue 13, pp 2665–2684 | Cite as

Exploiting sparsity and equation-free architectures in complex systems

  • J. L. Proctor
  • S. L. Brunton
  • B. W. Brunton
  • J. N. Kutz
Review
Part of the following topical collections:
  1. Advanced Computational and Experimental Techniques in Nolinear Dynamics. Guest Editors: Elbert E.N. Macau and Carlos L. Pando Lambruschini (Eds.)

Abstract

Complex systems exhibit dynamics that typically evolve on low-dimensional attractors and may have sparse representation in some optimal basis. Recently developed compressive sensing techniques exploit this sparsity for state reconstruction and/or categorical identification from limited measurements. We argue that data-driven dimensionality reduction methods integrate naturally with sparse sensing in the context of complex systems. This framework works equally well with a physical model or in an equation-free context, where data is available but the governing equations may be unknown. We demonstrate the advantages of combining these methods on three prototypical examples: classification of dynamical regimes, optimal sensor placement, and equation-free dynamic model reduction. These examples motivate the potentially transformative role that state-of-the-art data methods and machine learning can play in the analysis of complex systems.

Keywords

Singular Value Decomposition European Physical Journal Special Topic Proper Orthogonal Decomposition Sparse Representation Proper Orthogonal Decomposition Mode 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    P. Holmes, J. Guckenheimer, Nonlinear oscillations, dynamical systems, and bifurcations of vector fields, Vol. 42 of Applied Mathematical Sciences (Springer-Verlag, Berlin, Heidelberg, 1983)Google Scholar
  2. 2.
    C.M. Bender, S.A. Orszag, Advanced Mathematical Methods for Scientists and Engineers: Asymptotic Methods and Perturbation Theory (Springer, 1999)Google Scholar
  3. 3.
    M. Pastoor, L. Henning, B.R. Noack, R. King, G. Tadmor, J. Fluid Mech. 608, 161 (2008)ADSCrossRefzbMATHGoogle Scholar
  4. 4.
    B.R. Noack, et al., Bull. Am. Phys. Soc. 58, 418 (2013)Google Scholar
  5. 5.
    C. Bishop, Pattern Recognition and Machine Learning, Information Science and Statistics (Springer, 2006)Google Scholar
  6. 6.
    R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification (Wiley-Interscience, 2000)Google Scholar
  7. 7.
    J.N. Kutz, Data-Driven Modeling & Scientific Computation: Methods for Complex Systems & Big Data (Oxford University Press, 2013)Google Scholar
  8. 8.
    U. Grenander, M. Miller, Pattern Theory: From Representation to Inference (Oxford, 2007)Google Scholar
  9. 9.
    D. Mumford, A. Desolneux, Pattern Theory: The Stochastic Analysis of Real-World Signals (A. K. Peters Ltd., 2010)Google Scholar
  10. 10.
    B. Becker, E. Ortiz, IEEE IC on Automatic Face and Gesture Recognition (2008)Google Scholar
  11. 11.
    M. Koste, NPR (2013)Google Scholar
  12. 12.
    R. Tibshirani, T. Hastie, B. Narasimhan, G. Chu, Proc. Natl. Acad. Sci. 99, 6567 (2002)ADSCrossRefGoogle Scholar
  13. 13.
    J. Zhou, J. Liu, V. Narayan, J. Ye, Proc. 18th ACM SIGKDD, 1095 (2012)Google Scholar
  14. 14.
    D.L. Donoho, IEEE Trans. Inform. Theory 52, 1289 (2006)CrossRefMathSciNetzbMATHGoogle Scholar
  15. 15.
    E.J. Candès, J. Romberg, T. Tao, Commun. Pure Appl. Math. 8 (59)Google Scholar
  16. 16.
    R.G. Baraniuk, V. Cevher, M.F. Duarte, C. Hegde, IEEE Trans. Inform. Theory 56, 1982 (2010)CrossRefMathSciNetGoogle Scholar
  17. 17.
    J.A. Tropp, J.N. Laska, M.F. Duarte, J.K. Romberg, R.G. Baraniuk, IEEE Trans. Inform. Theory 56, 520 (2010)CrossRefMathSciNetGoogle Scholar
  18. 18.
    V. Parezanovic, et al., J. Flow Turbul. Combus (2014) (submitted)Google Scholar
  19. 19.
    E. Kaiser, et al., J. Fluid Mech. [arXiv:1309:0524] [physics.flu–dyn] (2013) (submitted)
  20. 20.
    T. Duriez, et al., Under revision for Physical Review Letters (2014)Google Scholar
  21. 21.
    L.N. Trefethen, D. Bau, Numerical Linear Algebra (SIAM, 1997)Google Scholar
  22. 22.
    G.H. Golub, C. Reinsch, Numer. Math. 14, 403 (1970)CrossRefMathSciNetzbMATHGoogle Scholar
  23. 23.
    A. Antoulas, D. Sorensen, Int. J. Appl. Math. Comput. Sci. 11, 1093 (2001)MathSciNetzbMATHGoogle Scholar
  24. 24.
    C.W. Rowley, T. Colonius, R.M. Murray, Physica D 189, 115 (2004)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  25. 25.
    C. Eckart, G. Young, Psychometrika 1, 211 (1936)CrossRefzbMATHGoogle Scholar
  26. 26.
    L. Mirsky, Quart. J. Math. Oxford, 50 (1960)Google Scholar
  27. 27.
    D. Jackson, Ecology 8, 2204 (1993)CrossRefGoogle Scholar
  28. 28.
    D. Donoho, M. Gavish, arXiv (1304:2085v3) (2013)
  29. 29.
    D. Donoho, M. Gavish, arXiv (1305:5870v2) (2013)
  30. 30.
    K. Pearson, Phil. Mag. 2, 559 (1901)CrossRefGoogle Scholar
  31. 31.
    H. Hotelling, J. Edu. Psycho. 24 (417–441, 198–520)Google Scholar
  32. 32.
    E.N. Lorenz, Report 1: Statistical Forecasting Project, MIT (1956)Google Scholar
  33. 33.
    G.R. North, Report 1: Statistical Forecasting Project, MIT (1956)Google Scholar
  34. 34.
    M. Loeve, Probability Theory (Van Nostrand, New York, 1955)Google Scholar
  35. 35.
    G. Berkooz, P. Holmes, J.L. Lumley, Ann. Rev. Fluid Mech. 23, 539 (1993)ADSCrossRefMathSciNetGoogle Scholar
  36. 36.
    P.J. Holmes, J.L. Lumley, G. Berkooz, C.W. Rowley, Turbulence, coherent structures, dynamical systems and symmetry, Cambridge Monographs in Mechanics, 2nd edn. (Cambridge University Press, Cambridge, England, 2012)Google Scholar
  37. 37.
    S.S. Ravindran, Int. J. Numer. Methods Fluids 34, 425 (2000)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  38. 38.
    X. Ma, G.E. Karniadakis, J. Fluid Mech. 458, 181 (2002)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  39. 39.
    B.R. Noack, K. Afanasiev, M. Morzynski, G. Tadmor, F. Thiele, J. Fluid Mech. 497, 335 (2003)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  40. 40.
    S. Sirisup, G.E. Karniadakis, D. Xiu, I.G. Kevrekidis, J. Comput. Phys. 207, 568 (2005)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  41. 41.
    G. Tadmor, O. Lehmann, B.R. Noack, M. Morzynski, Phys. Fluids 22, 034102 (2010)ADSCrossRefGoogle Scholar
  42. 42.
    D. Tang, D. Kholodar, J.N. Juang, E.H. Dowell, TM 2001-211243, NASA (2001)Google Scholar
  43. 43.
    E. Shlizerman, E. Ding, M. Williams, J.N. Kutz, Int. J. Opt. 2012, 831604 (2012)CrossRefGoogle Scholar
  44. 44.
    M. Williams, E. Shlizerman, J.N. Kutz, J. Opt. Soc. Am. B 27, 2471 (2010)ADSCrossRefGoogle Scholar
  45. 45.
    E. Ding, E. Shlizerman, J.N. Kutz, Phys. Rev. A 82, 023823 (2010)ADSCrossRefGoogle Scholar
  46. 46.
    D. Riskin, et al., J. Theoretical Biology, 2008 (2008)Google Scholar
  47. 47.
    J. Cusuman, B. Bai, Chaos, Solitons, Fractals, 515 (1993)Google Scholar
  48. 48.
    J. Cusuman, B. Bai, Phil. Trans. Roy. Soc. Lond., 421 (1994)Google Scholar
  49. 49.
    B. Epureanu, L. Tang, M. Paidoussis, Int. J. of Non-Linear Mech., 977 (2004)Google Scholar
  50. 50.
    D. Boe, J. Golinval, Struc. Health Monit., 137 (2003)Google Scholar
  51. 51.
    U. Feldmann, E. Kreuzer, F. Pinto, Nonlinear Dyn., 183 (2000)Google Scholar
  52. 52.
    M. Barrault, Y. Maday, N. Nguyen, A. Patera, C. R. Math. 9, 667 (2004)CrossRefMathSciNetGoogle Scholar
  53. 53.
    S. Chaturantabut, D. Sorensen, Decision and Control Conference 2009 (2009), p. 4316Google Scholar
  54. 54.
    S. Chaturantabut, D. Sorensen, SIAM Journal on Numerical Analysis (2012), p. 46Google Scholar
  55. 55.
    S. Sirisup, G. Karniadakis, Comp. Phys. 194, 92 (2004)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  56. 56.
    T. Colonius, J. Freund, AIAA Paper (2002), p. 2002Google Scholar
  57. 57.
    M. Rathinam, L. Petzold, SIAM J. Numer. Anal 5, 1893 (2003)CrossRefMathSciNetGoogle Scholar
  58. 58.
    T. Smith, Ph.D. thesis Princeton University, 2003Google Scholar
  59. 59.
    M. Ilak, C.W. Rowley, Phys. Fluids 20, 034103 (2008)ADSCrossRefGoogle Scholar
  60. 60.
    B.C. Moore, IEEE Trans. Automatic Control AC-26(1), 17 (1981)CrossRefGoogle Scholar
  61. 61.
    V. Adamjan, D. Arov, M. Krein, Math. USSR Sbornik 15, 31 (1971)CrossRefGoogle Scholar
  62. 62.
    K. Glover, Int. J. Contr. 39, 1115 (1984)CrossRefMathSciNetzbMATHGoogle Scholar
  63. 63.
    V. Adamjan, D. Arov, M. Krein, Amer. Math. Soc. Trans. 111, 133 (1978)Google Scholar
  64. 64.
    S. Lall, J. Marsden, S. Glavaški, Proc. IFAC World Congr. F, 473 (1999)Google Scholar
  65. 65.
    S. Lall, J. Marsden, S. Glavaški, Int. J. Robust Nonlinear Control 12, 519 (2002)CrossRefzbMATHGoogle Scholar
  66. 66.
    K. Willcox, J. Peraire, AIAA J. 40, 2323 (2002)ADSCrossRefGoogle Scholar
  67. 67.
    C.W. Rowley, Int. J. Bifurc. Chaos 15, 997 (2005)CrossRefMathSciNetzbMATHGoogle Scholar
  68. 68.
    S. Bagheri, L. Brandt, D.S. Henningson, J. Fluid Mech. 620, 263 (2009)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  69. 69.
    S.J. Illingworth, A.S. Morgans, C.W. Rowley, J. Sound Vib. 330, 1567 (2010)ADSCrossRefGoogle Scholar
  70. 70.
    S. Ahuja, C.W. Rowley, J. Fluid Mech. 645, 447 (2010)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  71. 71.
    A.C. Or, J.L. Speyer, J. Fluid Mech. 662, 36 (2010)CrossRefMathSciNetzbMATHGoogle Scholar
  72. 72.
    J.H. Tu, C.W. Rowley, J. Comput. Phys. 231, 5317 (2012)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  73. 73.
    L. Sirovich, M. Kirby, J. Opt. Soc. Am. A 4, 519 (1987)ADSCrossRefGoogle Scholar
  74. 74.
    C.W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, D.S. Henningson, J. Fluid Mech. 641, 115 (2009)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  75. 75.
    P.J. Schmid, J. Fluid Mech. 656, 5 (2010)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  76. 76.
    I.G. Kevrekidis, et al., Comm. Math. Sci. 1, 715 (2003)CrossRefMathSciNetzbMATHGoogle Scholar
  77. 77.
    P.J. Schmid, Exper. Fluids 50, 1123 (2011)ADSCrossRefGoogle Scholar
  78. 78.
    P.J. Schmid, D. Violato, F. Scarano, Exper. Fluids 52, 1567 (2012)ADSCrossRefGoogle Scholar
  79. 79.
    J.H. Tu, D.M. Luchtenburg, C.W. Rowley, S.L. Brunton, J.N. Kutz (2013) (submitted)Google Scholar
  80. 80.
    J.H. Tu, C.W. Rowley, J.N. Kutz (2013) (submitted)Google Scholar
  81. 81.
    K.K. Chen, J.H. Tu, C.W. Rowley, J. Nonlinear Sci. 22, 887 (2012)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  82. 82.
    I. Mezić, Ann. Rev. Fluid Mech. 45, 357 (2013)ADSCrossRefGoogle Scholar
  83. 83.
    M. Grilli, P. Schmid, S. Hickel, N. Adams, J. Fluid Mech., 16 (2012)Google Scholar
  84. 84.
    A. Seena, H. Sung, Int. J. Heat Fluid Flo 6, 1098 (2011)CrossRefGoogle Scholar
  85. 85.
    L. Massa, R. Kumar, P. Ravindran, Phys. Fluids 6 (2012)Google Scholar
  86. 86.
    O. Semeraro, G. Bellani, F. Lundell, Exp. Fluids 5, 1203 (2012)CrossRefGoogle Scholar
  87. 87.
    S.L. Brunton, J.L. Proctor, J.N. Kutz (2013) (submitted)Google Scholar
  88. 88.
    Z. Bai, et al., Physics based compressive sensing approach applied to airfoil data collection, analysis, AIAA Paper 2013-0772, 51st Aerospace Sciences Meeting (2013)Google Scholar
  89. 89.
    M.R. Jovanović, P.J. Schmid, J.W. Nichols, Center for Turbulence Research (2012)Google Scholar
  90. 90.
    D. Duke, J. Soria, D. Honnery, Exper. Fluids 52, 529 (2012)ADSCrossRefGoogle Scholar
  91. 91.
    A. Wynn, D.S. Pearson, B. Ganapathisubramani, P.J. Goulart, J. Fluid Mech. 733, 473 (2013)ADSCrossRefMathSciNetzbMATHGoogle Scholar
  92. 92.
    V.M. Patel, R. Chellappa, Sparse representations and compressive sensing for imaging and vision, Briefs in Electrical and Computer Engineering (Springer, 2013)Google Scholar
  93. 93.
    M. Lustig, D. Donoho, J. Pauly, Magn. Reson. Med. 58, 1182 (2007)CrossRefGoogle Scholar
  94. 94.
    M. Lustig, D. Donoho, J. Santo, J. Pauly, IEEE Signal Process. Mag. 72, 1182 (2008)Google Scholar
  95. 95.
    Y. Mishchenko, L. Paninski, J. Comput. Neurosci. 33, 371 (2012)CrossRefMathSciNetGoogle Scholar
  96. 96.
    J. Wright, A. Yang, A. Ganesh, S. Sastry, Y. Ma, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 31, 210 (2009)CrossRefGoogle Scholar
  97. 97.
    E.J. Candès, Proceedings of the International Congress of Mathematics (2006)Google Scholar
  98. 98.
    E.J. Candès, T. Tao, IEEE Trans. Info. Theory 52, 5406 (2006)CrossRefGoogle Scholar
  99. 99.
    R.G. Baraniuk, IEEE Signal Process. Mag. 24, 118 (2007)ADSCrossRefGoogle Scholar
  100. 100.
    H. Nyquist, Trans. A. I. E. E. (1928), p. 617Google Scholar
  101. 101.
    H. Yoo, et al., Radio Frequency Integrated Circuits Symposium (2012), p. 31Google Scholar
  102. 102.
    E.J. Candès, J. Romberg, T. Tao, IEEE Trans. Info. Theory 52, 489 (2006)CrossRefzbMATHGoogle Scholar
  103. 103.
    A.C. Gilbert, P. Indyk, Proc. IEEE 98, 937 (2010)CrossRefGoogle Scholar
  104. 104.
    I. Bright, G. Lin, J.N. Kutz, Phys. Fluids 25, 127102 (2013)ADSCrossRefGoogle Scholar
  105. 105.
    J.V. Shi, W. Yin, A.C. Sankaranarayanan, R.G. Baraniuk (2013) (submitted)Google Scholar
  106. 106.
    S.L. Brunton, J.H. Tu, I. Bright, J.N. Kutz (2013) (submitted)Google Scholar
  107. 107.
    R. Tibshirani, J. of Roy. Statistical Soc. 58, 267 (1996)MathSciNetzbMATHGoogle Scholar
  108. 108.
    D. Donoho, X. Huo, IEEE Trans. Info. Theory 47, 2845 (2001)CrossRefMathSciNetzbMATHGoogle Scholar
  109. 109.
    D. Donoho, M. Elad, Proc. Natl. Acad. Sci. 5, 2197 (2003)ADSCrossRefMathSciNetGoogle Scholar
  110. 110.
    A. Gilbert, S. Muthukrishnan, M. Strauss, Proceedings of 2003 SIAM Symposium on Discrete Algorithms SODA (2003), p. 243Google Scholar
  111. 111.
    A. Tropp, A. Gilbert, S. Muthukrishnan, M. Strauss, IEEE International conference on image processing ICIP (2003), p. 37Google Scholar
  112. 112.
    H. Zou, T. Hastie, R. Tibshirani, J. Comp. Graph. Stat. 15, 262 (2006)CrossRefMathSciNetGoogle Scholar
  113. 113.
    C. Line, T. Hastie, D. Witten, B.E. ll, Technometrics 53, 406 (2011)CrossRefMathSciNetGoogle Scholar
  114. 114.
    E.J. Candès, X. Li, Y. Ma, J. Wright, J. ACM 58, 11 (2011)CrossRefMathSciNetGoogle Scholar
  115. 115.
    D. Needell, J.A. Tropp, Comm. ACM 53, 93 (2010)CrossRefGoogle Scholar
  116. 116.
    J.A. Tropp, IEEE Trans. Inform. Theory 52, 1030 (2006)CrossRefMathSciNetzbMATHGoogle Scholar
  117. 117.
    M.C. Cross, P.C. Hohenberg, Rev. Mod. Phys. 65, 851 (1993)ADSCrossRefGoogle Scholar
  118. 118.
    X. Fu, S.L. Brunton, J.N. Kutz (2013) (submitted)Google Scholar
  119. 119.
    S.L. Brunton, X. Fu, J.N. Kutz (2014) (submitted)Google Scholar
  120. 120.
    B.W. Brunton, S.L. Brunton, J.L. Proctor, J.N. Kutz (2013) (submitted)Google Scholar
  121. 121.
    A. Siegman, Lasers (University Science Books, 1986)Google Scholar
  122. 122.
    G. Bell, T. Hey, A. Szalay, Science 323, 1297 (2009)CrossRefGoogle Scholar

Copyright information

© EDP Sciences and Springer 2014

Authors and Affiliations

  • J. L. Proctor
    • 1
  • S. L. Brunton
    • 2
    • 3
  • B. W. Brunton
    • 2
    • 4
  • J. N. Kutz
    • 2
  1. 1.Institute for Disease ModelingBellevueUSA
  2. 2.Department of Applied MathematicsUniversity of WashingtonSeattleUSA
  3. 3.Department of Mechanical EngineeringUniversity of WashingtonSeattleUSA
  4. 4.Department of BiologyUniversity of WashingtonSeattleUSA

Personalised recommendations