Skip to main content
Log in

Parallel data-driven decomposition algorithm for large-scale datasets: with application to transitional boundary layers

Theoretical and Computational Fluid Dynamics Aims and scope Submit manuscript


Many fluid flows of engineering interest, though very complex in appearance, can be approximated by low-order models governed by a few modes, able to capture the dominant behavior (dynamics) of the system. This feature has fueled the development of various methodologies aimed at extracting dominant coherent structures from the flow. Some of the more general techniques are based on data-driven decompositions, most of which rely on performing a singular value decomposition (SVD) on a formulated snapshot (data) matrix. The amount of experimentally or numerically generated data expands as more detailed experimental measurements and increased computational resources become readily available. Consequently, the data matrix to be processed will consist of far more rows than columns, resulting in a so-called tall-and-skinny (TS) matrix. Ultimately, the SVD of such a TS data matrix can no longer be performed on a single processor, and parallel algorithms are necessary. The present study employs the parallel TSQR algorithm of (Demmel et al. in SIAM J Sci Comput 34(1):206–239, 2012), which is further used as a basis of the underlying parallel SVD. This algorithm is shown to scale well on machines with a large number of processors and, therefore, allows the decomposition of very large datasets. In addition, the simplicity of its implementation and the minimum required communication makes it suitable for integration in existing numerical solvers and data decomposition techniques. Examples that demonstrate the capabilities of highly parallel data decomposition algorithms include transitional processes in compressible boundary layers without and with induced flow separation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions


  1. Benson, A.R., Gleich, D.F., Demmel, J.: Direct QR factorization for tall-and-skinny matrices in MapReduce architectures. In: IEEE International Conference in Big Data (2013)

  2. Cadieux F., Domaradzki J.A., Sayadi T., Bose S.: Direct numerical simulation and large Eddy simulation of laminar separation bubbles at moderate Reynolds numbers. J. Fluids Eng. 136(6), 060902(1–5) (2014)

    Article  Google Scholar 

  3. Demmel J., Grigori L., Hoemmen M., Langou J.: Communication-optimal parallel and sequential QR and LU factorizations. SIAM J. Sci. Comput. 34(1), 206–239 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  4. Grilli M., Schmid P.J., Hickel S., Adams N.A.: Analysis of unsteady behaviour in shockwave turbulent boundary layer interaction. J. Fluid Mech. 700, 16–28 (2012)

    Article  MATH  Google Scholar 

  5. Heroux, M., Bartlett, R., Hoekstra, V.H.R., Hu, J., Kolda, T., Lehoucq, R., Long, K., Pawlowski, R., Phipps, E., Salinger, A., Thornquist, H., Tuminaro, R., Willenbring, J., Williams, A.: An overview of trilinos. Technical Report SAND2003-2927, Sandia National Laboratories (2003)

  6. Jovanović M.R., Schmid P.J., Nichols J.W.: Sparsity-promoting dynamic mode decomposition. Phys. Fluids 26(2), 024103 (2014)

    Article  Google Scholar 

  7. Luethi, P., Studer, C., Duetsch, S., Zgraggen, E., Kaeslin, H., Felber, N., Fichtner, W.: Gram-Schmidt-based QR decomposition for MIMO detection: VLSI implementation and comparison. In: IEEE Asia Pacific Conference on Circuits and systems, pp 830–833 (2008)

  8. Mezic I.: Analysis of fluid flows via spectral properties of the Koopman operator. Annu. Rev. Fluid Mech. 45(1), 357–378 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Muld T.W., Efraimsson G., Henningson D.: Mode decomposition on surface-mounted cube. Flow Turbul. Combust. 88(3), 279–310 (2012)

    Article  MATH  Google Scholar 

  10. Perugini S., Gonçalves M., Fox E.A.: Recommender systems research: a connection-centric survey. J. Intel. Inf. Syst. 23(2), 107–143 (2004)

    Article  MATH  Google Scholar 

  11. Rowley C.W.: Model reduction for fluids using balanced proper orthogonal decomposition. Int. J. Bifurc. Chaos 15(3), 997–1013 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  12. Rowley C.W., Mezic I., Bagheri S., Schlatter P., Henningson D.S.: Spectral analysis of nonlinear flows. J. Fluid Mech. 641, 115–127 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  13. Sayadi T., Hamman C.W., Moin P.: Direct simulation of complete H-type and K-type transitions with implications for the structure of turbulent boundary layers. J. Fluid Mech. 724, 480–509 (2013)

    Article  MATH  Google Scholar 

  14. Sayadi T., Schmid P.J., Nichols J.W., Moin P.: Reduced-order representation of near-wall structures in the late transitional boundary layer. J. Fluid Mech. 748, 278–301 (2014)

    Article  Google Scholar 

  15. Sayadi T., Schmid P.J., Richecoeur F., Durox D.: Parametrized data-driven decomposition for bifurcation analysis, with application to thermo-acoustically unstable systems. Phys. Fluids 27(3), 037,102 (2015)

    Article  Google Scholar 

  16. Schmid P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  17. Schmid P.J., Violato D., Scarano F.: Decomposition of time-resolved tomographic PIV. Exp. Fluids 52(6), 1567–1579 (2012)

    Article  Google Scholar 

  18. Spalart P., Strelets M.: Mechanisms of transition and heat transfer in a separation bubble. J. Fluid Mech. 403, 329–349 (2000)

    Article  MATH  Google Scholar 

  19. Statnikov, V., Sayadi, T., Meinke, M., Schmid, P., Schröder, W.: Analysis of pressure perturbation sources on a generic space launcher after-body in supersonic flow using zonal turbulence modeling and dynamic mode decomposition. Phys. Fluids 27(1), 016103(1–21) (2015)

  20. Willcox K., Peraire J.: Balanced model reduction via the proper orthogonal decomposition. AIAA J. 40(11), 2323–2330 (2002)

    Article  Google Scholar 

  21. Ye, J., Li, Q., Xiong, H., Park, H., Janardan, R., Kumar, V.: IDR/QR: an incremental dimension reduction algorithm via QR decomposition. IEEE Trans Knowl Data Eng 17(9):1208–1222 (2005)

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Taraneh Sayadi.

Additional information

Communicated by Tim Colonius.

Electronic supplementary material

The Below is the Electronic Supplementary Material.

ESM (MOV 812 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sayadi, T., Schmid, P.J. Parallel data-driven decomposition algorithm for large-scale datasets: with application to transitional boundary layers. Theor. Comput. Fluid Dyn. 30, 415–428 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: