Skip to main content
SpringerLink
Log in

Applications of time parallelization

Computing and Visualization in Science volume 23, Article number: 11 (2020) Cite this article

Abstract

This review article serves to summarize the many advances in time-parallel computations since the excellent review article by Gander, “50 years of Time Parallel Integration” (Gander, in: 50 years of time parallel time integration. Multiple shooting and time domain decomposition, Springer, Berlin, 2015). We focus, when possible, on applications of time parallelism and the observed speedup and efficiency, highlighting the challenges and benefits of parallel time computations. The applications covered range from numerous PDE-based simulations (both hyperbolic and parabolic), to PDE-constrained optimization, powergrid simulations, and machine learning. The time-parallel methods covered range from various iterative schemes (multigrid, waveform, multiple shooting, domain decomposition) to direct time-parallel methods.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

39,95 €

Price includes VAT (Finland)

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. We will do the same regarding the convergence of the other iterative methods (multigrid, waveform, and domain decomposition).

  2. FAS [12] is the full approximation scheme multigrid cycling strategy commonly used for nonlinear problems.

  3. Ideal restriction (R) and interpolation (P) are defined to yield a Schur Complement coarse-grid operator with RAP.

  4. http://parallel-in-time.org.

References

  1. Agboh, W.C., Ruprecht, D., Dogar, M.R.: Combining coarse and fine physics for manipulation using parallel-in-time integration (2019). arxiv:1903.08470

  2. Bal, G., Maday, Y.: A “Parareal” time discretization for non-linear PDE’s with application to the pricing of an American Put. In: Pavarino, L., Toselli, A. (eds.) Recent Developments in Domain Decomposition Methods. Lecture Notes in Computational Science and Engineering, vol. 23, pp. 189–202. Springer, Berlin (2002). https://doi.org/10.1007/978-3-642-56118-4_12

    Chapter  Google Scholar 

  3. Bast, D., Kulchytska-Ruchka, I., Schoeps, S., Rain, O.: Accelerated steady-state torque computation for induction machines using parallel-in-time algorithms. IEEE Trans. Magn. 56, 1–9 (2020). https://doi.org/10.1109/tmag.2019.2945510

    Article  Google Scholar 

  4. Baudron, A.M., Lautard, J.J., Maday, Y., Mula, O.: The parareal in time algorithm applied to the kinetic neutron diffusion equation. Domain Decomposition Methods in Science and Engineering XXI. Lecture Notes in Computational Science and Engineering, pp. 437–445. Springer International Publishing, Berlin (2014). https://doi.org/10.1007/978-3-319-05789-7_41

    Chapter  MATH  Google Scholar 

  5. Baudron, A.M., Lautard, J.J., Maday, Y., Riahi, M.K., Salomon, J.: Parareal in time 3D numerical solver for the LWR Benchmark neutron diffusion transient model. J. Comput. Phys. 279, 67–79 (2014). https://doi.org/10.1016/j.jcp.2014.08.037

    Article  MathSciNet  MATH  Google Scholar 

  6. Benedusi, P., Garoni, C., Krause, R., Li, X., Serra-Capizzano, S.: Space-time FE-DG discretization of the anisotropic diffusion equation in any dimension: the spectral symbol. SIAM J. Matrix Anal. Appl. 39(3), 1383–1420 (2018). https://doi.org/10.1137/17M113527X

    Article  MathSciNet  MATH  Google Scholar 

  7. Benedusi, P., Hupp, D., Arbenz, P., Krause, R.: A parallel multigrid solver for time-periodic incompressible Navier–Stokes equations in 3D. Numerical Mathematics and Advanced Applications ENUMATH 2015, pp. 265–273. Springer, Berlin (2016). https://doi.org/10.1007/978-3-319-39929-4_26

    Chapter  Google Scholar 

  8. Bennequin, D., Gander, M.J., Gouarin, L., Halpern, L.: Optimized Schwarz waveform relaxation for advection reaction diffusion equations in two dimensions. Numer. Math. 134(3), 513–567 (2016). https://doi.org/10.1007/s00211-015-0784-8

    Article  MathSciNet  MATH  Google Scholar 

  9. Blayo, E., Rousseau, A., Tayachi, M.: Boundary conditions and Schwarz waveform relaxation method for linear viscous shallow water equations in hydrodynamics. SMAI J. Comput. Math. 3, 117–137 (2017). https://doi.org/10.5802/smai-jcm.22

    Article  MathSciNet  MATH  Google Scholar 

  10. Blumers, A.L., Li, Z., Karniadakis, G.E.: Supervised parallel-in-time algorithm for long-time lagrangian simulations of stochastic dynamics: Application to hydrodynamics. J. Comput. Phys. 393, 214–228 (2019). https://doi.org/10.1016/j.jcp.2019.05.016

    Article  MathSciNet  Google Scholar 

  11. Bolten, M., Moser, D., Speck, R.: A multigrid perspective on the parallel full approximation scheme in space and time. Numer. Linear Algebra Appl. 24(6), e2110 (2017). https://doi.org/10.1002/nla.2110.E2110nla.2110

    Article  MathSciNet  MATH  Google Scholar 

  12. Brandt, A.: Multi-level adaptive solutions to boundary-value problems. Math. Comput. 31(138), 333–390 (1977)

    Article  MathSciNet  Google Scholar 

  13. Brandt, A.: General highly accurate algebraic coarsening. Electron. Trans. Numer. Anal. 10, 1–20 (2000)

    MathSciNet  MATH  Google Scholar 

  14. Burrage, K.: Parallel and Sequential Methods for Ordinary Differential Equations. Numerical Mathematics and Scientific Computation. The Clarendon Press, Oxford University Press, New York (1995)

    MATH  Google Scholar 

  15. Cadeau, T., Magoules, F.: Coupling the Parareal algorithm with the waveform relaxation method for the solution of differential algebraic equations. In: Distributed Computing and Applications to Business, Engineering and Science (DCABES), 2011 Tenth International Symposium on, pp. 15–19 (2011). https://doi.org/10.1109/DCABES.2011.34

  16. Chartier, P., Philippe, B.: A parallel shooting technique for solving dissipative ODE’s. Computing 51(3–4), 209–236 (1993). https://doi.org/10.1007/BF02238534

    Article  MathSciNet  MATH  Google Scholar 

  17. Christlieb, A., Guo, W., Morton, M., Qiu, J.M.: A high order time splitting method based on integral deferred correction for semi-Lagrangian Vlasov simulations. J. Comput. Phys. 267, 7–27 (2014). https://doi.org/10.1016/j.jcp.2014.02.012

    Article  MathSciNet  MATH  Google Scholar 

  18. Christlieb, A.J., Haynes, R.D., Ong, B.W.: A parallel space-time algorithm. SIAM J. Sci. Comput. 34(5), C233–C248 (2012). https://doi.org/10.1137/110843484

    Article  MathSciNet  MATH  Google Scholar 

  19. Christlieb, A.J., Macdonald, C.B., Ong, B.W.: Parallel high-order integrators. SIAM J. Sci. Comput. 32(2), 818–835 (2010). https://doi.org/10.1137/09075740X

    Article  MathSciNet  MATH  Google Scholar 

  20. Christlieb, A.J., MacDonald, C.B., Ong, B.W., Spiteri, R.J.: Revisionist integral deferred correction with adaptive step-size control. Commun. Appl. Math. Comput. Sci. 10, 1–25 (2015). https://doi.org/10.2140/camcos.2015.10.1

    Article  MathSciNet  MATH  Google Scholar 

  21. Christlieb, A.J., Ong, B.W.: Implicit parallel time integrators. J. Sci. Comput. 49(2), 167–179 (2011). https://doi.org/10.1007/s10915-010-9452-4

    Article  MathSciNet  MATH  Google Scholar 

  22. Clarke, A.T., Davies, C.J., Ruprecht, D., Tobias, S.M.: Parallel-in-time integration of kinematic dynamos. J. Comput. Phys. X 7, 100057 (2020). https://doi.org/10.1016/j.jcpx.2020.100057

    Article  MathSciNet  Google Scholar 

  23. Courvoisier, Y., Gander, M.J.: Time domain Maxwell equations solved with Schwarz waveform relaxation methods. Domain Decomposition Methods in Science and Engineering XX. Lecture Notes in Computational Science and Engineering, vol. 91, pp. 263–270. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35275-1_30

    Chapter  Google Scholar 

  24. Descombes, S., Dolean, V., Gander, M.J.: Schwarz waveform relaxation methods for systems of semi-linear reaction-diffusion equations. Domain Decomposition Methods in Science and Engineering XIX. Lecture Notes in Computational Science and Engineering, vol. 78, pp. 423–430. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-11304-8_49

    Chapter  MATH  Google Scholar 

  25. Dobrev, V.A., Kolev, T., Petersson, N.A., Schroder, J.B.: Two-level convergence theory for multigrid reduction in time (mgrit). SIAM J. Sci. Comput. 39(5), S501–S527 (2017). https://doi.org/10.1137/16M1074096

    Article  MathSciNet  MATH  Google Scholar 

  26. Douglas, C., Kim, I., Lee, H., Sheen, D.: Higher-order schemes for the Laplace transformation method for parabolic problems. Comput. Vis. Sci. 14(1), 39–47 (2011). https://doi.org/10.1007/s00791-011-0156-6

    Article  MathSciNet  MATH  Google Scholar 

  27. Dutt, A., Greengard, L., Rokhlin, V.: Spectral deferred correction methods for ordinary differential equations. BIT Numer. Math. 40(2), 241–266 (2000). https://doi.org/10.1023/A:1022338906936

    Article  MathSciNet  MATH  Google Scholar 

  28. Emmett, M., Minion, M.L.: Toward an efficient parallel in time method for partial differential equations. Commun. Appl. Math. Comput. Sci. 7(1), 105–132 (2012). https://doi.org/10.2140/camcos.2012.7.105

    Article  MathSciNet  MATH  Google Scholar 

  29. Emmett, M., Minion, M.L.: Efficient implementation of a multi-level parallel in time algorithm. Domain Decomposition Methods in Science and Engineering XXI. Lecture Notes in Computational Science and Engineering, vol. 98, pp. 359–366. Springer International Publishing, Berlin (2014). https://doi.org/10.1007/978-3-319-05789-7_33

    Chapter  MATH  Google Scholar 

  30. Falgout, R.D., Friedhoff, S., Kolev, T.V., MacLachlan, S.P., Schroder, J.B.: Parallel time integration with multigrid. SIAM J. Sci. Comput. 36(6), C635–C661 (2014). LLNL-JRNL-645325

    Article  MathSciNet  Google Scholar 

  31. Falgout, R.D., Friedhoff, S., Kolev, T.V., MacLachlan, S.P., Schroder, J.B., Vandewalle, S.: Multigrid methods with space-time concurrency. Comput. Vis. Sci. 18(4), 123–143 (2017). https://doi.org/10.1007/s00791-017-0283-9

    Article  MathSciNet  MATH  Google Scholar 

  32. Falgout, R.D., Katz, A., Kolev, T.V., Schroder, J.B., Wissink, A.M., Yang, U.M.: Parallel time integration with multigrid reduction for a compressible fluid dynamics application. Tech. rep., Lawrence Livermore National Laboratory (2014). https://computation.llnl.gov/project/parallel-time-integration/pubs/strand2d-pit.pdf

  33. Falgout, R.D., Manteuffel, T.A., O’Neill, B., Schroder, J.B.: Multigrid reduction in time for nonlinear parabolic problems: a case study. SIAM J. Sci. Comput. 39(5), S298–S322 (2017). https://doi.org/10.1137/16M1082330

    Article  MathSciNet  MATH  Google Scholar 

  34. Falgout, R.D., Manteuffel, T.A., Southworth, B., Schroder, J.B.: Parallel-in-time for moving meshes. Tech. rep., Lawrence Livermore National Laboratory (2016). https://doi.org/10.2172/1239230

  35. Franco, S.R., Gaspar, F.J., Pinto, M.A.V., Rodrigo, C.: Multigrid method based on a space-time approach with standard coarsening for parabolic problems. Appl. Math. Comput. 317(C), 25–34 (2018). https://doi.org/10.1016/j.amc.2017.08.043

    Article  MathSciNet  MATH  Google Scholar 

  36. Friedhoff, S., Falgout, R.D., Kolev, T.V., MacLachlan, S.P., Schroder, J.B.: A multigrid-in-time algorithm for solving evolution equations in parallel. In: Presented at: Sixteenth Copper Mountain Conference on Multigrid Methods, Copper Mountain, CO, United States, Mar 17 - Mar 22, 2013 (2013). http://www.osti.gov/scitech/servlets/purl/1073108

  37. Friedhoff, S., Hahne, J., Kulchytska-Ruchka, I., Schöps, S.: Exploring parallel-in-time approaches for eddy current problems. Progress in Industrial Mathematics at ECMI 2018, pp. 373–379. Springer, Berlin (2019). https://doi.org/10.1007/978-3-030-27550-1_47

    Chapter  Google Scholar 

  38. Friedhoff, S., MacLachlan, S.: A generalized predictive analysis tool for multigrid methods. Numer. Linear Algebra Appl. 22(4), 618–647 (2015). https://doi.org/10.1002/nla.1977

    Article  MathSciNet  MATH  Google Scholar 

  39. Gahvari, H., Dobrev, V.A., Falgout, R.D., Kolev, T.V., Schroder, J.B., Schulz, M., Meier Yang, U.: A performance model for allocating the parallelism in a multigrid-in-time solver. In: 7th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (2016). https://doi.org/10.1109/PMBS.2016.8

  40. Gander, M., Halpern, L., Rannou, J., Ryan, J.: A direct time parallel solver by diagonalization for the wave equation. SIAM J. Sci. Comput. 41(1), A220–A245 (2019). https://doi.org/10.1137/17M1148347

    Article  MathSciNet  MATH  Google Scholar 

  41. Gander, M.J.: 50 years of time parallel time integration. Multiple Shooting and Time Domain Decomposition. Springer, Berlin (2015). https://doi.org/10.1007/978-3-319-23321-5_3

    Chapter  Google Scholar 

  42. Gander, M.J., Güttel, S.: PARAEXP: a parallel integrator for linear initial-value problems. SIAM J. Sci. Comput. 35(2), C123–C142 (2013). https://doi.org/10.1137/110856137

    Article  MathSciNet  MATH  Google Scholar 

  43. Gander, M.J., Güttel, S., Petcu, M.: A nonlinear paraexp algorithm. In: Bjørstad, P.E., Brenner, S.C., Halpern, L., Kim, H.H., Kornhuber, R., Rahman, T., Widlund, O.B. (eds.) Domain Decomposition Methods in Science and Engineering XXIV, pp. 261–270. Springer International Publishing, Cham (2018). https://doi.org/10.1007/978-3-319-93873-8_24

    Chapter  Google Scholar 

  44. Gander, M.J., Halpern, L., Labbé, S., Santugini-Repiquet, K.: An optimized Schwarz waveform relaxation algorithm for micro-magnetics. Domain Decomposition Methods in Science and Engineering XVII. Lecture Notes in Computational Science and Engineering, vol. 60, pp. 203–210. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-75199-1_22

    Chapter  Google Scholar 

  45. Gander, M.J., Halpern, L., Ryan, J., Tran, T.T.B.: A direct solver for time parallelization. In: Dickopf, T., Gander, M.J., Halpern, L., Krause, R., Pavarino, L.F. (eds.) Domain Decomposition Methods in Science and Engineering XXII, pp. 491–499. Springer International Publishing, Berlin (2016). https://doi.org/10.1007/978-3-319-18827-0_50

    Chapter  Google Scholar 

  46. Gander, M.J., Jiang, Y.L., Li, R.J.: Parareal Schwarz waveform relaxation methods. Domain Decomposition Methods in Science and Engineering XX. Lecture Notes in Computational Science and Engineering, vol. 91, pp. 451–458. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35275-1_53

    Chapter  Google Scholar 

  47. Gander, M.J., Jiang, Y.L., Li, R.J.: Parareal Schwarz Waveform Relaxation Methods. In: Bank, R., Holst, M., Widlund, O., Xu, J. (eds.) Domain Decomposition Methods in Science and Engineering XX. Lecture Notes in Computational Science and Engineering, vol. 91, pp. 451–458. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35275-1_53

    Chapter  Google Scholar 

  48. Gander, M.J., Jiang, Y.L., Song, B.: A superlinear convergence estimate for theparareal schwarz waveform relaxation algorithm. SIAM J. Sci. Comput. 41(2), A1148–A1169 (2019). https://doi.org/10.1137/18M1177226

    Article  MATH  Google Scholar 

  49. Gander, M.J., Kwok, F., Zhang, H.: Multigrid interpretations of the parareal algorithm leading to an overlapping variant and mgrit. Comput. Vis. Sci. (2018). https://doi.org/10.1007/s00791-018-0297-y

    Article  MathSciNet  Google Scholar 

  50. Gander, M.J., Neumüller, M.: Analysis of a new space-time parallel multigrid algorithm for parabolic problems. SIAM J. Sci. Comput. 38(4), A2173–A2208 (2016)

    Article  MathSciNet  Google Scholar 

  51. Gander, M.J., Stuart, A.M.: Space-Time Continuous Analysis of Waveform Relaxation for the Heat Equation. SIAM Journal on Scientific Computing 19(6), 2014–2031 (1998). https://doi.org/10.1137/S1064827596305337

    Article  MathSciNet  MATH  Google Scholar 

  52. Gander, M.J., Vandewalle, S.: Analysis of the Parareal time-parallel time-integration method. SIAM J. Sci. Comput. 29(2), 556–578 (2007). https://doi.org/10.1137/05064607X

    Article  MathSciNet  MATH  Google Scholar 

  53. Gaspar, F.J., Rodrigo, C.: Multigrid waveform relaxation for the time-fractional heat equation. SIAM J. Sci. Comput. 39(4), A1201–A1224 (2017). https://doi.org/10.1137/16M1090193.10.1137/16M1090193

  54. Gear, C.W.: Parallel methods for ordinary differential equations. CALCOLO 25(1–2), 1–20 (1988). https://doi.org/10.1007/BF02575744

    Article  MathSciNet  MATH  Google Scholar 

  55. Gear, C.W.: Waveform methods for space and time parallelism. In: Proceedings of the International Symposium on Computational Mathematics (Matsuyama, 1990), vol. 38, pp. 137–147 (1991)

  56. Giladi, E., Keller, H.B.: Space-time domain decomposition for parabolic problems. Numer. Math. 93(2), 279–313 (2002). https://doi.org/10.1007/s002110100345

    Article  MathSciNet  MATH  Google Scholar 

  57. Götschel, S., Minion, M.L.: Parallel-in-time for parabolic optimal control problems using PFASST. In: Bjørstad, P.E., Brenner, S.C., Halpern, L., Kim, H.H., Kornhuber, R., Rahman, T., Widlund, O.B. (eds.) Domain Decomposition Methods in Science and Engineering XXIV, pp. 363–371. Springer International Publishing, Berlin (2018). https://doi.org/10.1007/978-3-319-93873-8_34

    Chapter  MATH  Google Scholar 

  58. Gotschel, S., Minion, M.L.: An efficient parallel-in-time method for optimization with parabolic pdes. SIAM J. Sci. Comput. 41(6), C603–C626 (2019). https://doi.org/10.1137/19M1239313

    Article  MathSciNet  MATH  Google Scholar 

  59. Günther, S., Gauger, N.R., Schroder, J.B.: A non-intrusive parallel-in-time adjoint solver with the XBraid library. Comput. Vis. Sci. 19(3–4), 85–95 (2018). https://doi.org/10.1007/s00791-018-0300-7

    Article  MathSciNet  Google Scholar 

  60. Günther, S., Gauger, N.R., Schroder, J.B.: A non-intrusive parallel-in-time approach for simultaneous optimization with unsteady PDEs. Optim. Methods Softw. 1–16 (2018). https://doi.org/10.1080/10556788.2018.1504050

  61. Günther, S., Ruthotto, L., Schroder, J.B., Cyr, E.C., Gauger, N.R.: Layer-parallel training of deep residual neural networks. SIAM J. Math. Data Sci. 2(1), 1–23 (2020). https://doi.org/10.1137/19M1247620

    Article  MathSciNet  Google Scholar 

  62. Haber, E., Ruthotto, L.: Stable architectures for deep neural networks. Inverse Probl. 34(1), 014004 (2017)

    Article  MathSciNet  Google Scholar 

  63. Hackbusch, W.: Parabolic Multigrid Methods. Computing Methods in Applied Sciences and Engineering. VI (Versailles, 1983), pp. 189–197. North-Holland, Amsterdam (1984)

  64. Hamon, F.P., Schreiber, M., Minion, M.L.: Multi-level spectral deferred corrections scheme for the shallow water equations on the rotating sphere. J. Comput. Phys. 376, 435–454 (2019). https://doi.org/10.1016/j.jcp.2018.09.042

    Article  MathSciNet  MATH  Google Scholar 

  65. Hamon, F.P., Schreiber, M., Minion, M.L.: Parallel-in-Time Multi-Level Integration of the Shallow-Water equations on the rotating sphere (2020). https://doi.org/10.1016/j.jcp.2019.109210

    Article  Google Scholar 

  66. Haut, T., Wingate, B.: An asymptotic parallel-in-time method for highly oscillatory PDEs. SIAM J. Sci. Comput. 36(2), A693–A713 (2014). https://doi.org/10.1137/130914577

    Article  MathSciNet  MATH  Google Scholar 

  67. Haut, T.S., Babb, T., Martinsson, P.G., Wingate, B.A.: A high-order time-parallel scheme for solving wave propagation problems via the direct construction of an approximate time-evolution operator. IMA J. Numer. Anal. 36(2), 688–716 (2015). https://doi.org/10.1093/imanum/drv021

    Article  MathSciNet  MATH  Google Scholar 

  68. Haynes, R.D., Ong, B.W.: MPI-OpenMP algorithms for the parallel space-time solution of time dependent PDEs. Domain Decomposition Methods in Science and Engineering XXI. Lecture Notes in Computational Science and Engineering, vol. 98, pp. 179–187. Springer International Publishing, Berlin (2014). https://doi.org/10.1007/978-3-319-05789-7_14

    Chapter  Google Scholar 

  69. Hessenthaler, A., Nordsletten, D., Röhrle, O., Schroder, J.B., Falgout, R.D.: Convergence of the multigrid reduction in time algorithm for the linear elasticity equations. Numer. Linear Algebra Appl. 25(3), e2155 (2018). https://doi.org/10.1002/nla.2155

    Article  MathSciNet  MATH  Google Scholar 

  70. Hessenthaler, A., Southworth, B.S., Nordsletten, D., Röhrle, O., Falgout, R.D., Schroder, J.B.: Multilevel convergence analysis of multigrid-reduction-in-time. SIAM J. Sci. Comput. 42(2), A771–A796 (2020). https://doi.org/10.1137/19M1238812

    Article  MathSciNet  MATH  Google Scholar 

  71. Hofer, C., Langer, U., Neumüller, M., Schneckenleitner, R.: Parallel and robust preconditioning for space-time isogeometric analysis of parabolic evolution problems. SIAM J. Sci. Comput. 41(3), A1793–A1821 (2019). https://doi.org/10.1137/18M1208794

    Article  MathSciNet  MATH  Google Scholar 

  72. Hofer, C., Langer, U., Neumüller, M., Toulopoulos, I.: Time-multipatch discontinuous galerkin space-time isogeometric analysis of parabolic evolution problems. Electron. Trans. Numerica. Anal. 49, 126–150 (2018). https://doi.org/10.1553/etna_vol49s126

    Article  MathSciNet  MATH  Google Scholar 

  73. Horton, G.: The time-parallel multigrid method. Comm. Appl. Numer. Methods 8(9), 585–595 (1992). https://doi.org/10.1002/cnm.1630080906

    Article  MathSciNet  MATH  Google Scholar 

  74. Horton, G., Vandewalle, S.: A space-time multigrid method for parabolic partial differential equations. SIAM J. Sci. Comput. 16(4), 848–864 (1995). https://doi.org/10.1137/0916050

    Article  MathSciNet  MATH  Google Scholar 

  75. Horton, G., Vandewalle, S., Worley, P.: An algorithm with polylog parallel complexity for solving parabolic partial differential equations. SIAM J. Sci. Comput. 16(3), 531–541 (1995). https://doi.org/10.1137/0916034

    Article  MathSciNet  MATH  Google Scholar 

  76. van der Houwen, P., Sommeijer, B.: Analysis of parallel diagonally implicit iteration of Runge–Kutta methods. Appl. Numer. Math. 11(1), 169–188 (1993). https://doi.org/10.1016/0168-9274(93)90047-U

    Article  MathSciNet  MATH  Google Scholar 

  77. van der Houwen, P.J., Sommeijer, B.P.: Iterated Runge–Kutta methods on parallel computers. SIAM J. Sci. Stat. Comput. 12(5), 1000–1028 (1991). https://doi.org/10.1137/0912054

    Article  MathSciNet  MATH  Google Scholar 

  78. Howse, A., Sterck, H., Falgout, R., MacLachlan, S., Schroder, J.: Parallel-in-time multigrid with adaptive spatial coarsening for the linear advection and inviscid burgers equations. SIAM J. Sci. Comput. 41(1), A538–A565 (2019). https://doi.org/10.1137/17M1144982

    Article  MathSciNet  MATH  Google Scholar 

  79. Hu, X., Rodrigo, C., Gaspar, F.J.: Using hierarchical matrices in the solution of the time-fractional heat equation by multigrid waveform relaxation. J. Comput. Phys. 416, 109540 (2020). https://doi.org/10.1016/j.jcp.2020.109540

    Article  MathSciNet  MATH  Google Scholar 

  80. Hulbert, G.M., Hughes, T.J.: Space-time finite element methods for second-order hyperbolic equations. Comput. Methods Appl. Mech. Eng. 84(3), 327–348 (1990). https://doi.org/10.1016/0045-7825(90)90082-W

    Article  MathSciNet  MATH  Google Scholar 

  81. hypre: High performance preconditioners. http://www.llnl.gov/CASC/hypre/

  82. Janssen, J., Vandewalle, S.: Multigrid waveform relaxation on spatial finite element meshes. Contributions to multigrid (Amsterdam. 1993), CWI Tract, vol. 103, pp. 75–86. Math. Centrum Centrum Wisk. Inform, Amsterdam (1994)

  83. Kaber, S.M., Maday, Y.: Parareal in time approximation of the Korteveg–deVries–Burgers’ equations. PAMM 7, 1026403–1026404 (2007). https://doi.org/10.1002/pamm.200700574

    Article  Google Scholar 

  84. Kwok, F., Ong, B.: Schwarz waveform relaxation with adaptive pipelining. SIAM J. Sci. Comput. 41(1), A339–A364 (2019). https://doi.org/10.1137/17M115311X

    Article  MathSciNet  MATH  Google Scholar 

  85. Lai, C.H.: On Transformation Methods and the Induced Parallel Properties for the Temporal Domain. In: Substructing Techniques and Domain Decomposition Methods, Computational Science, Engineering & Technology Series, pp. 45–70. Saxe-Coburg, UK (2010). https://doi.org/10.4203/csets.24.3

  86. Langer, U., Matculevich, S., Repin, S.: Guaranteed error bounds and local indicators for adaptive solvers using stabilised space-time IgA approximations to parabolic problems. Computers & Mathematics with Applications 78, 2641–2671 (2019). https://doi.org/10.1016/j.camwa.2019.04.009

    Article  MathSciNet  MATH  Google Scholar 

  87. Lecouvez, M., Falgout, R.D., Woodward, C.S., Top, P.: A parallel multigrid reduction in time method for power systems. In: 2016 IEEE Power and Energy Society General Meeting (PESGM), pp. 1–5 (2016). https://doi.org/10.1109/PESGM.2016.7741520

  88. Legoll, F., Leliévre, T., Samaey, G.: A micro–macro Parareal algorithm: application to singularly perturbed ordinary differential equations. SIAM J. Sci. Comput. 35(4), A1951–A1986 (2013). https://doi.org/10.1137/120872681

    Article  MathSciNet  MATH  Google Scholar 

  89. Lelarasmee, E., Ruehli, A.E., Sangiovanni-Vincentelli, A.L.: The waveform relaxation method for time-domain analysis of large scale integrated circuits. IEEE CAD 1(3), 131–145 (1982). https://doi.org/10.1109/TCAD.1982.1270004

    Article  Google Scholar 

  90. Lindelöf, E.: Sur l’application des méthodes d’approximations succesives à l’étude des intégrales réelles des équations différentielles ordinaires. Journal de matheématiques pures et appliquées 10(4), 117–128 (1894)

    MATH  Google Scholar 

  91. Lions, J.L., Maday, Y., Turinici, G.: A “parareal” in time discretization of PDE’s. Comptes Rendus de l’Académie des Sciences - Series I - Mathematics 332, 661–668 (2001). https://doi.org/10.1016/S0764-4442(00)01793-6

    Article  MATH  Google Scholar 

  92. Liu, J., Jiang, Y.L.: A parareal algorithm based on waveform relaxation. Math. Comput. Simul. 82(11), 2167–2181 (2012). https://doi.org/10.1016/j.matcom.2012.05.017

    Article  MathSciNet  MATH  Google Scholar 

  93. Liu, Y., Hu, J.: Modified propagators of parareal in time algorithm and application to Princeton Ocean model. Int. J. Numer. Methods Fluids 57(12), 1793–1804 (2008). https://doi.org/10.1002/fld.1703

    Article  MathSciNet  MATH  Google Scholar 

  94. Livne, O.E.: Coarsening by compatible relaxation. Numer. Linear Algebra Appl. 11(2–3), 205–227 (2004). https://doi.org/10.1002/nla.378

    Article  MathSciNet  MATH  Google Scholar 

  95. Lorin, E.: Domain decomposition method for the n-body time-independent and time-dependent Schrödinger equations. Numer. Algorithms 81(2), 655–694 (2019). https://doi.org/10.1007/s11075-018-0566-3

    Article  MathSciNet  MATH  Google Scholar 

  96. Maday, Y., Rønquist, E.M.: Parallelization in time through tensor-product space-time solvers. Comptes Rendus Mathematique 346(1–2), 113–118 (2008). https://doi.org/10.1016/j.crma.2007.09.012

    Article  MathSciNet  MATH  Google Scholar 

  97. Manteuffel, T., Ruge, J., Southworth, B.: Nonsymmetric algebraic multigrid based on local approximate ideal restriction. SIAM J. Sci. Comput. 40(6), A4105–A4130 (2018). https://doi.org/10.1137/17M1144350

    Article  MathSciNet  MATH  Google Scholar 

  98. Mercerat, D., Guillot, L., Vilotte, J.P.: Application of the Parareal algorithm for acoustic wave propagation. In: AIP Conference Proceedings, vol. 1168, pp. 1521–1524 (2009). https://doi.org/10.1063/1.3241388

  99. Minion, M.L.: A hybrid parareal spectral deferred corrections method. Commun. Appl. Math. Comput. Sci. 5(2), 265–301 (2010). https://doi.org/10.2140/camcos.2010.5.265

    Article  MathSciNet  MATH  Google Scholar 

  100. Minion, M.L., Speck, R., Bolten, M., Emmett, M., Ruprecht, D.: Interweaving PFASST and parallel multigrid. SIAM J. Sci. Comput. 37, S244–S263 (2015). https://doi.org/10.1137/14097536X

    Article  MathSciNet  MATH  Google Scholar 

  101. Miranker, W.L., Liniger, W.: Parallel methods for the numerical integration of ordinary differential equations. Math. Comput. 21(99), 303–320 (1967). https://doi.org/10.1090/S0025-5718-1967-0223106-8

    Article  MathSciNet  MATH  Google Scholar 

  102. Nievergelt, J.: Parallel methods for integrating ordinary differential equations. Commun. ACM 7(12), 731–733 (1964). https://doi.org/10.1145/355588.365137

    Article  MathSciNet  MATH  Google Scholar 

  103. Ong, B., Kwok, F., High, S.: Pipeline Schwarz waveform relaxation. In: Methods in Science and Engineering XXII, Lecture Notes in Computational Science and Engineering. Spring–Verlag (2015). https://doi.org/10.1007/978-3-319-18827-0_36

  104. Ong, B.W., Haynes, R.D., Ladd, K.: Algorithm 965: RIDC methods: a family of parallel time integrators. ACM Trans. Math. Softw. 43(1), 8:1–8:13 (2016). https://doi.org/10.1145/2964377

    Article  MathSciNet  MATH  Google Scholar 

  105. Ong, B.W., Mandal, B.C.: Pipeline implementations of Neumann–Neumann and Dirichlet–Neumann waveform relaxation methods. Numer. Algorithms 78(1), 1–20 (2018). https://doi.org/10.1007/s11075-017-0364-3

    Article  MathSciNet  MATH  Google Scholar 

  106. Ong, B.W., Melfi, A., Christlieb, A.J.: Parallel semi-implicit time integrators (2012). arxiv:1209.4297. ArXiv:1209.4297 [cs.DC]

  107. Peddle, A.G., Haut, T., Wingate, B.: Parareal convergence for oscillatory PDEłowercases with finite time-scale separation (2019). https://doi.org/10.1137/17m1131611

  108. Picard, E.: Remarques sur les équations différentielles. Acta Math. 17(1), 297–300 (1893). https://doi.org/10.1007/BF02391996. Extrait d’une lettre adress’ee à M. Mittag-Leffler

  109. Quaife, B., Biros, G.: Adaptive time stepping for vesicle suspensions. J. Comput. Phys. 306, 478–499 (2016). https://doi.org/10.1016/j.jcp.2015.11.050

    Article  MathSciNet  MATH  Google Scholar 

  110. Quarteroni, A.: Mathematical models in science and engineering. Not. Am. Math. Soc. 56(1), 10–19 (2009)

    MathSciNet  MATH  Google Scholar 

  111. Reynolds-Barredo, J.M., Newman, D.E., Sánchez, R.S., Berry, L.A.: Modelling parareal convergence in 2D drift wave plasma turbulence. In: High Performance Computing and Simulation (HPCS), 2012 International Conference on, pp. 726–727 (2012). https://doi.org/10.1109/HPCSim.2012.6267004

  112. Reynolds-Barredo, J.M., Newman, D.E., Sánchez, R.S., Samaddar, D., Berry, L.A., Elwasif, W.R.: Mechanisms for the convergence of time-parallelized, parareal turbulent plasma simulations. J. Comput. Phys. 231(23), 7851–7867 (2012). https://doi.org/10.1016/j.jcp.2012.07.028

    Article  Google Scholar 

  113. Samaddar, D., Newman, D.E., Sánchez, R.S.: Parallelization in time of numerical simulations of fully-developed plasma turbulence using the parareal algorithm. J. Comput. Phys. 229, 6558–6573 (2010). https://doi.org/10.1016/j.jcp.2010.05.012

    Article  MATH  Google Scholar 

  114. Samaey, G., Slawig, T.: A micro/macro parallel-in-time (parareal) algorithm applied to a climate model with discontinuous non-monotone coefficients and oscillatory forcing. arXiv:1806.04442 [math.NA] (2018). arxiv:1806.04442

  115. Schmitt, A., Schreiber, M., Peixoto, P., Schäfer, M.: A numerical study of a semi-Lagrangian parareal method applied to the viscous burgers equation. Comput. Vis. Sci. 19(1), 45–57 (2018). https://doi.org/10.1007/s00791-018-0294-1

    Article  MathSciNet  MATH  Google Scholar 

  116. Schreiber, M., Loft, R.: A parallel time integrator for solving the linearized shallow water equations on the rotating sphere. Numer. Linear Algebra Appl. 26(2), e2220 (2019). https://doi.org/10.1002/nla.2220

    Article  MathSciNet  MATH  Google Scholar 

  117. Schreiber, M., Peixoto, P.S., Haut, T., Wingate, B.: Beyond spatial scalability limitations with a massively parallel method for linear oscillatory problems. Int. J. High Perform. Comput. Appl. 32(6), 913–933 (2018). https://doi.org/10.1177/1094342016687625

    Article  Google Scholar 

  118. Schreiber, M., Schaeffer, N., Loft, R.: Exponential integrators with parallel-in-time rational approximations for shallow-water equations on the rotating sphere. Parallel Comput. (2019). https://doi.org/10.1016/j.parco.2019.01.005

    Article  MathSciNet  Google Scholar 

  119. Schroder, J.B., Falgout, R.D., Woodward, C.S., Top, P., Lecouvez, M.: Parallel-in-time solution of power systems with scheduled events. In: 2018 IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5. IEEE (2018). https://doi.org/10.1109/PESGM.2018.8586435

  120. Sheen, D., Sloan, I.H., Thomée, V.: A parallel method for time-discretization of parabolic problems based on contour integral representation and quadrature. Math. Comput. 69(229), 177–195 (2000). https://doi.org/10.1090/S0025-5718-99-01098-4

    Article  MathSciNet  MATH  Google Scholar 

  121. Sheen, D., Sloan, I.H., Thomée, V.: A parallel method for time discretization of parabolic equations based on Laplace transformation and quadrature. IMA J. Numer. Anal. 23(2), 269–299 (2003). https://doi.org/10.1093/imanum/23.2.269

    Article  MathSciNet  MATH  Google Scholar 

  122. Sommeijer, B.: Parallel-iterated Runge–Kutta methods for stiff ordinary differential equations. J. Comput. Appl. Math. 45(1), 151–168 (1993). https://doi.org/10.1016/0377-0427(93)90271-C

    Article  MathSciNet  MATH  Google Scholar 

  123. Southworth, B.S.: Necessary conditions and tight two-level convergence bounds for parareal and multigrid reduction in time. SIAM J. Matrix Anal. Appl. 40(2), 564–608 (2019). https://doi.org/10.1137/18M1226208

    Article  MathSciNet  MATH  Google Scholar 

  124. Speck, R.: Parallelizing spectral deferred corrections across the method. Comput. Vis. Sci. 19, 75–83 (2018). https://doi.org/10.1007/s00791-018-0298-x

    Article  MathSciNet  Google Scholar 

  125. Speck, R., Ruprecht, D.: Toward fault-tolerant parallel-in-time integration with PFASST. Parallel Comput. 62, 20–37 (2017). https://doi.org/10.1016/j.parco.2016.12.001

    Article  MathSciNet  Google Scholar 

  126. Speck, R., Ruprecht, D., Emmett, M., Minion, M.L., Bolten, M., Krause, R.: A multi-level spectral deferred correction method. BIT Numer. Math. 55, 843–867 (2015). https://doi.org/10.1007/s10543-014-0517-x

    Article  MathSciNet  MATH  Google Scholar 

  127. Speck, R., Ruprecht, D., Krause, R., Emmett, M., Minion, M.L., Winkel, M., Gibbon, P.: A massively space-time parallel N-body solver. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC ’12, pp. 92:1–92:11. IEEE Computer Society Press, Los Alamitos, CA, USA (2012). https://doi.org/10.1109/SC.2012.6

  128. Speck, R., Ruprecht, D., Krause, R., Emmett, M., Minion, M.L., Winkel, M., Gibbon, P.: Integrating an N-body problem with SDC and PFASST. Domain Decomposition Methods in Science and Engineering XXI. Lecture Notes in Computational Science and Engineering, vol. 98, pp. 637–645. Springer International Publishing, Berlin (2014). https://doi.org/10.1007/978-3-319-05789-7_61

    Chapter  MATH  Google Scholar 

  129. Steinbach, O., Yang, H.: Comparison of algebraic multigrid methods for an adaptive space-time finite-element discretization of the heat equation in 3d and 4d. Numer. Linear Algebra Appl. 25(3), e2143 (2018). https://doi.org/10.1002/nla.2143

    Article  MathSciNet  MATH  Google Scholar 

  130. Sterck, H.D., Falgout, R.D., Friedhoff, S., Krzysik, O.A., MacLachlan, S.P.: Optimizing MGRIT and parareal coarse-grid operators for linear advection (2019). arXiv:1910.03726

  131. Thomée, V.: A high order parallel method for time discretization of parabolic type equations based on Laplace transformation and quadrature. Int. J. Numer. Anal. Model. 2(1), 85–96 (2005)

    MathSciNet  MATH  Google Scholar 

  132. Trottenberg, U., Oosterlee, C., Sch\(\ddot{\text{u}}\)ller, A.: Multigrid. Academic Press, London, UK (2001)

  133. Uddin, B.: Revisionist integral deferred correction methods with application to the moving method of lines. Master’s thesis, Memorial University (2018)

  134. Van Der Houwen, P.J., Sommeijer, B.P.: Parallel iteration of high-order Runge–Kutta methods with stepsize control. J. Computat. Appl. Math. 29(1), 111–127 (1990). https://doi.org/10.1016/0377-0427(90)90200-J

    Article  MathSciNet  MATH  Google Scholar 

  135. Vandewalle, S., Horton, G.: Fourier mode analysis of the multigrid waveform relaxation and time-parallel multigrid methods. Computing 54(4), 317–330 (1995). https://doi.org/10.1007/BF02238230

    Article  MathSciNet  MATH  Google Scholar 

  136. Vandewalle, S., Piessens, R.: Efficient parallel algorithms for solving initial-boundary value and time-periodic parabolic partial differential equations. SIAM J. Sci. Stat. Comput. 13(6), 1330–1346 (1992). https://doi.org/10.1137/0913075

    Article  MathSciNet  MATH  Google Scholar 

  137. Vandewalle, S., Van de Velde, E.: Space-time concurrent multigrid waveform relaxation. Ann. Numer. Math. 1(1–4), 347–360 (1994). https://doi.org/10.13140/2.1.1146.1761

    Article  MathSciNet  MATH  Google Scholar 

  138. Vaněk, P., Mandel, J., Brezina, M.: Algebraic multigrid based on smoothed aggregation for second and fourth order problems. Computing 56, 179–196 (1996)

    Article  MathSciNet  Google Scholar 

  139. Vanka, S.P.: Block-implicit multigrid solution of Navier–Stokes equations in primitive variables. J. Comput. Phys. 65(1), 138–158 (1986). https://doi.org/10.1016/0021-9991(86)90008-2

    Article  MathSciNet  MATH  Google Scholar 

  140. Weinzierl, T., Köppl, T.: A geometric space-time multigrid algorithm for the heat equation. Numer. Math. Theory Methods Appl. 5(1), 110–130 (2012). https://doi.org/10.1017/S1004897900000258

    Article  MathSciNet  MATH  Google Scholar 

  141. Winkel, M., Speck, R., Hübner, H., Arnold, L., Krause, R., Gibbon, P.: A massively parallel, multi-disciplinary barnes-hut tree code for extreme-scale n-body simulations. Comput. Phys. Commun. 183(4), 880–889 (2012). https://doi.org/10.1016/j.cpc.2011.12.013

    Article  MathSciNet  Google Scholar 

  142. XBraid: Parallel multigrid in time. https://github.com/XBraid/xbraid/

  143. Yue, X., Shu, S., Xu, X., Bu, W., Pan, K.: Parallel-in-time with fully finite element multigrid for 2-d space-fractional diffusion equations. arXiv preprint arXiv:1805.06688 (2018)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Michigan Technological University, Houghton, MI, USA

    Benjamin W. Ong

  2. Department of Mathematics and Statistics, University of New Mexico, Albuquerque, NM, USA

    Jacob B. Schroder

Authors
  1. Benjamin W. Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacob B. Schroder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacob B. Schroder.

Additional information

Communicated by Daniel Ruprecht.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ong, B.W., Schroder, J.B. Applications of time parallelization. Comput. Visual Sci. 23, 11 (2020). https://doi.org/10.1007/s00791-020-00331-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00791-020-00331-4

Keywords

  • Time-parallelization
  • Multigrid-in-time
  • Waveform relaxation
  • Parallel-in-time

Mathematics Subject Classification

  • 34-02
  • 65-02
  • 65-Y05

Search

Navigation