Leveraging Sparsity and Compressive Sensing for Reduced Order Modeling

  • J. Nathan Kutz
  • Syuzanna Sargsyan
  • Steven L. Brunton
Part of the MS&A book series (MS&A, volume 17)


Sparsity can be leveraged with dimensionality-reduction techniques to characterize and model parametrized nonlinear dynamical systems. Sparsity is used for both sparse representation, via proper orthogonal decomposition (POD) modes in different dynamical regimes, and by compressive sensing, which provides the mathematical architecture for robust classification of POD subspaces. The method relies on constructing POD libraries in order to characterize the dominant, low-rank coherent structures. Using a greedy sampling algorithm, such as gappy POD and one of its many variants, an accurate Galerkin-POD projection approximating the nonlinear terms from a sparse number of grid points can be constructed. The selected grid points for sampling, if chosen well, can be shown to be effective sensing/measurement locations for classifying the underlying dynamics and reconstruction of the nonlinear dynamical system. The use of sparse sampling for interpolating nonlinearities and classification of appropriate POD modes facilitates a family of local reduced-order models for each physical regime, rather than a higher-order global model. We demonstrate the sparse sampling and classification method on the canonical problem of flow around a cylinder. The method allows for a robust mathematical framework for robustly selecting POD modes from a library, accurately constructing the full state space, and generating a Galerkin-POD projection for simulating the nonlinear dynamical system.



J.N. Kutz would like to acknowledge support from the Air Force Office of Scientific Research (FA9550-15-1-0385).


  1. 1.
    Amsallem, D., Cortial, J., Farhat, C.: On demand CFD-based aeroelastic predictions using a database of reduced-order bases and models. In: AIAA Conference (2009)CrossRefGoogle Scholar
  2. 2.
    Amsallem, D., Tezaur, R., Farhat, C.: Real-time solution of computational problems using databases of parametric linear reduced-order models with arbitrary underlying meshes. J. Comput. Phys. 326, 373–397 (2016)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Amsallem, D., Zahr, M.J., Washabaugh, K.: Fast local reduced basis updates for the efficient reduction of nonlinear systems with hyper-reduction. Adv. Comput. Math. (2015). doi:10.1007/s10444-015-9409-0MathSciNetzbMATHGoogle Scholar
  4. 4.
    Astrid, P.: Fast reduced order modeling technique for large scale LTV systems. In: Proceedings of 2004 American Control Conference, vol. 1, pp. 762–767 (2004)Google Scholar
  5. 5.
    Baraniuk, R.G.: Compressive sensing. IEEE Signal Process. Mag. 24(4), 118–120 (2007)CrossRefGoogle Scholar
  6. 6.
    Barrault, M., Maday, Y., Nguyen, N.C., Patera, A.T.: An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations. C. R. Math. Acad. Sci. Paris 339, 667–672 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57, 483–531 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bishop, C.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)zbMATHGoogle Scholar
  9. 9.
    Bright, I., Lin, G., Kutz, J.N.: Compressive sensing based machine learning strategy for characterizing the flow around a cylinder with limited pressure measurements. Phys. Fluids 25, 127102 (2013)CrossRefzbMATHGoogle Scholar
  10. 10.
    Brunton, S.L., Tu, J.H., Bright, I., Kutz, J.N.: Compressive sensing and low-rank libraries for classification of bifurcation regimes in nonlinear dynamical systems. SIAM J. Appl. Dyn. Syst. 13, 1716–1732 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Brunton, B.W., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Sparse sensor placement optimization for classification. SIAM J. App. Math. 76, 2099–2122 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Candès, E.J.: In: Sanz-Solé, M., Soria, J., Varona, J.L., Verdera, J. (eds.) Compressive sensing. In: Proceeding of the International Congress of Mathematicians, vol. 2, pp. 1433–1452 (2006)Google Scholar
  13. 13.
    Candès, E.J., Tao, T.: Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Candès, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Carlberg, K., Farhat, C., Cortial, J., Amsallem, D.: The GNAT method for nonlinear model reduction: effective implementation and application to computational fluid dynamics and turbulent flows. J. Comput. Phys. 242, 623–647 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Carlberg, K., Barone, M., Antil, H.: Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction. J. Comput. Phys. 330, 693–734 (2017)Google Scholar
  18. 18.
    Chaturantabut, S., Sorensen, D.: Nonlinear model reduction via discrete empirical interpolation. SIAM J. Sci. Comput. 32, 2737–2764 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Choi, Y., Amsallem, D., Farhat, C.: Gradient-based constrained optimization using a database of linear reduced-order models. arXiv:1506.07849 (2015)Google Scholar
  20. 20.
    Deane, A.E., Kevrekidis, I.G., Karniadakis, G.E., Orszag, S.A.: Low-dimensional models for complex geometry flows: application to grooved channels and circular cylinders. Phys. Fluids 3, 2337 (1991)CrossRefzbMATHGoogle Scholar
  21. 21.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Donoho, D.L.: For most large underdetermined systems of linear equations the minimal 1-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59(6), 797–829 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Drmać, Z., Gugercin, S.: A new selection operator for the discrete empirical interpolation method—improved a priori error bound and extensions. SIAM J. Sci. Comput. 38(2), A631–A648 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Du, Q., Gunzburger, M.: Model reduction by proper orthogonal decomposition coupled with centroidal Voronoi tessellation. In: Proceedings of the Fluids Engineering Division Summer Meeting, FEDSM2002–31051. The American Society of Mechanical Engineers (2002)Google Scholar
  25. 25.
    Eftang, J.L., Patera, A.T., Rønquist, E.M.: An HP certified reduced-basis method for parameterized elliptic PDEs. In: SIAM SISC (2010)zbMATHGoogle Scholar
  26. 26.
    Elad, M.: Sparse and Redundant Representations. Springer, Berlin (2010)CrossRefzbMATHGoogle Scholar
  27. 27.
    Eldar, Y., Kutyniok, G. (eds.): Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)Google Scholar
  28. 28.
    Everson, R., Sirovich, L.: Karhunen-Loéve procedure for gappy data. J. Opt. Soc. Am. A 12, 1657–1664 (1995)CrossRefGoogle Scholar
  29. 29.
    Galletti, B., Bruneau, C.H., Zannetti, L.: Low-order modelling of laminar flow regimes past a confined square cylinder. J. Fluid Mech. 503, 161–170 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Holmes, P.J., Lumley, J.L., Berkooz, G., Rowley, C.W.: Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge Monographs in Mechanics, 2nd edn. Cambridge University Press, Cambridge (2012)Google Scholar
  31. 31.
    Kaiser, E., Noack, B.R., Cordier, L., Spohn, A., Segond, M., Abel, M., Daviller, G., Osth, J., Krajnovic, S., Niven, R. K.: Cluster-based reduced-order modelling of a mixing layer. J. Fluid Mech. 754, 365–414 (2014)CrossRefzbMATHGoogle Scholar
  32. 32.
    Kaspers, K., Mathelin, L., Abou-Kandil, H.: A machine learning approach for constrained sensor placement. American Control Conference, Chicago, IL, July 1–3, 2015 (2015)Google Scholar
  33. 33.
    Liberge, E., Hamdouni, A.: Reduced order modelling method via proper orthogonal decomposition (POD) for flow around an oscillating cylinder. J. Fluids Struct. 26, 292—311 (2010)CrossRefGoogle Scholar
  34. 34.
    Ma, X., Karniadakis, G.E.: A low-dimensional model for simulating three-dimensional cylinder flow. J. Fluid Mech. 458, 181–190 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Maday, Y., Mula, O.: A generalized empirical interpolation method: application of reduced basis techniques to data assimilation. Anal. Numer. Partial Differ. Eqn. 4, 221–235 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Murphy, K.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)zbMATHGoogle Scholar
  37. 37.
    Nguyen, N.C., Patera, A.T., Peraire, J.: A “best points” interpolation method for efficient approximation of parametrized functions. Int. J. Num. Methods Eng. 73, 521–543 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Peherstorfer, B., Willcox, K.: Online adaptive model reduction for nonlinear systems via low-rank updates. SIAM J. Sci. Comput. 37(4), A2123–A2150 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Peherstorfer, B., Willcox, K.: Dynamic data-driven reduced-order models. Comput. Methods Appl. Mech. Eng. 291, 21–41 (2015)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Peherstorfer, B., Willcox, K.: Detecting and adapting to parameter changes for reduced models of dynamic data-driven application systems. Proc. Comput. Sci. 51, 2553–2562 (2015)CrossRefGoogle Scholar
  41. 41.
    Quarteroni, A., Rozza, G. (eds.): Reduced Order Methods for Modeling and Computational Reduction. Springer, Berlin (2014)zbMATHGoogle Scholar
  42. 42.
    Sargsyan, S., Brunton, S.L., Kutz, J.N.: Nonlinear model reduction for dynamical systems using sparse optimal sensor locations from learned nonlinear libraries. Phys. Rev. E 92, 033304 (2015)CrossRefGoogle Scholar
  43. 43.
    Sargasyan, S., Brunton, S.L., Kutz, J.N.: Online interpolation point refinement for reduced order models using a genetic algorithm. arxiv:1607.07702Google Scholar
  44. 44.
    Venturi, D., Karniadakis, G.E.: Gappy data and reconstruction procedures for flow past cylinder. J. Fluid Mech. 519, 315–336 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Willcox, K.: Unsteady flow sensing and estimation via the gappy proper orthogonal decomposition. Comput. Fluids 35, 208–226 (2006)CrossRefzbMATHGoogle Scholar
  46. 46.
    Yildirim, B., Chryssostomidis, C., Karniadakis, G.E.: Efficient sensor placement for ocean measurements using low-dimensional concepts. Ocean Model. 273(3–4), 160–173 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • J. Nathan Kutz
    • 1
  • Syuzanna Sargsyan
    • 1
  • Steven L. Brunton
    • 2
  1. 1.Department of Applied MathematicsUniversity of WashingtonSeattleUSA
  2. 2.Department of Mechanical EngineeringUniversity of WashingtonSeattleUSA

Personalised recommendations