Skip to main content
SpringerLink
Log in

50 Years of Time Parallel Time Integration

  • Conference paper
Multiple Shooting and Time Domain Decomposition Methods

Part of the book series: Contributions in Mathematical and Computational Sciences ((CMCS,volume 9))

Abstract

Time parallel time integration methods have received renewed interest over the last decade because of the advent of massively parallel computers, which is mainly due to the clock speed limit reached on today’s processors. When solving time dependent partial differential equations, the time direction is usually not used for parallelization. But when parallelization in space saturates, the time direction offers itself as a further direction for parallelization. The time direction is however special, and for evolution problems there is a causality principle: the solution later in time is affected (it is even determined) by the solution earlier in time, but not the other way round. Algorithms trying to use the time direction for parallelization must therefore be special, and take this very different property of the time dimension into account.We show in this chapter how time domain decomposition methods were invented, and give an overview of the existing techniques. Time parallel methods can be classified into four different groups: methods based on multiple shooting, methods based on domain decomposition and waveform relaxation, space-time multigrid methods and direct time parallel methods. We show for each of these techniques the main inventions over time by choosing specific publications and explaining the core ideas of the authors. This chapter is for people who want to quickly gain an overview of the exciting and rapidly developing area of research of time parallel methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    “In diesem Artikel studieren wir verschiedene Versionen einer Klasse paralleler Algorithmen, die ursprünglich von A. Bellen und M. Zennaro für Differenzengleichungen konzipiert und von ihnen ‘across the steps’ Methode genannt worden ist.”

  2. 2.

    “Pour commencer, on expose l’idée sur l’exemple simple.”

  3. 3.

    “C’est alors un exercice que de montrer la:”

  4. 4.

    “Actually this method of continuing the computation is highly inefficient and is not recommended”, see [59].

  5. 5.

    “The spectacular growth in the scale of integrated circuits being designed in the VLSI era has generated the need for new methods of circuit simulation. “Standard” circuit simulators, such as SPICE and ASTAP, simply take too much CPU time and too much storage to analyze a VLSI circuit”, see [52].

  6. 6.

    “Note that since the oscillator is highly non unidirectional due to the feedback from v 3 to the NOR gate, the convergence of the iterated solutions is achieved with the number of iterations being proportional to the number of oscillating cycles of interest”, see [52].

  7. 7.

    This example had already been proposed by Fox in 1954.

References

  1. Al-Khaleel, M., Ruehli, A.E., Gander, M.J.: Optimized waveform relaxation methods for longitudinal partitioning of transmission lines. IEEE Trans. Circuits Syst. Regul. Pap. 56, 1732–1743 (2009)

    Article  MathSciNet  Google Scholar 

  2. Al-Khaleel, M.D., Gander, M.J., Ruehli, A.E.: Optimization of transmission conditions in waveform relaxation techniques for RC circuits. SIAM J. Numer. Anal. 52, 1076–1101 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Amodio, P., Brugnano, L.: Parallel solution in time of ODEs: some achievements and perspectives. Appl. Numer. Math. 59, 424–435 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  4. Axelsson, A., Verwer, J.: Boundary value techniques for initial value problems in ordinary differential equations. Math. Comput. 45, 153–171 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bastian, P., Burmeister, J., Horton, G.: Implementation of a parallel multigrid method for parabolic differential equations. In: Parallel Algorithms for Partial Differential Equations. Proceedings of the 6th GAMM seminar Kiel, pp. 18–27 (1990)

    Google Scholar 

  6. Bellen, A., Zennaro, M.: Parallel algorithms for initial-value problems for difference and differential equations. J. Comput. Appl. Math. 25, 341–350 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bennequin, D., Gander, M.J., Halpern, L.: A homographic best approximation problem with application to optimized Schwarz waveform relaxation. Math. Comput. 78, 185–223 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bjørhus, M.: On domain decomposition, subdomain iteration and waveform relaxation. Ph.D. thesis, University of Trondheim, Norway (1995)

    Google Scholar 

  9. Brugnano, L., Trigiante, D.: A parallel preconditioning technique for boundary value methods. Appl. Numer. Math. 13, 277–290 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  10. Brugnano, L., Trigiante, D.: Convergence and stability of boundary value methods for ordinary differential equations. J. Comput. Appl. Math. 66, 97–109 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  11. Brugnano, L., Trigiante, D.: Solving Differential Equations by Multistep Initial and Boundary Value Methods. CRC Press, Boca Raton (1998)

    MATH  Google Scholar 

  12. Burrage, K.: Parallel and Sequential Methods for Ordinary Differential Equations. Clarendon Press, New York (1995)

    MATH  Google Scholar 

  13. Chartier, P., Philippe, B.: A parallel shooting technique for solving dissipative ODEs. Computing 51, 209–236 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  14. Christlieb, A.J., Macdonald, C.B., Ong, B.W.: Parallel high-order integrators. SIAM J. Sci. Comput. 32, 818–835 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Crann, D., Davies, A.J., Lai, C.-H., Leong, S.H.: Time domain decomposition for European options in financial modelling. Contemp. Math. 218, 486–491 (1998)

    Article  MathSciNet  Google Scholar 

  16. Deshpande, A., Malhotra, S., Schultz, M., Douglas, C.: A rigorous analysis of time domain parallelism. Parallel Algorithms Appl. 6, 53–62 (1995)

    Article  MATH  Google Scholar 

  17. Douglas, J.Jr., Santos, J.E., Sheen, D., Bennethum, L.S.: Frequency domain treatment of one-dimensional scalar waves. Math. Models Methods Appl. Sci. 3, 171–194 (1993)

    Google Scholar 

  18. Douglas, C., Kim, I., Lee, H., Sheen, D.: Higher-order schemes for the Laplace transformation method for parabolic problems. Comput. Vis. Sci. 14, 39–47 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Emmett, M., Minion, M.L.: Toward an efficient parallel in time method for partial differential equations. Commun. Appl. Math. Comput. Sci. 7, 105–132 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Engquist, B., Ying, L.: Sweeping preconditioner for the Helmholtz equation: moving perfectly matched layers. Multiscale Model. Simul. 9, 686–710 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  21. Falgout, R., Friedhoff, S., Kolev, T.V., MacLachlan, S., Schroder, J.B.: Parallel time integration with multigrid. SIAM J. Sci. Comput. 36, C635–C661 (2014)

    Article  MATH  Google Scholar 

  22. Farhat, C., Cortial, J., Dastillung, C., Bavestrello, H.: Time-parallel implicit integrators for the near-real-time prediction of linear structural dynamic responses. Int. J. Numer. Methods Eng. 67, 697–724 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  23. Franklin, M.A.: Parallel solution of ordinary differential equations. IEEE Trans. Comput. 100, 413–420 (1978)

    Article  Google Scholar 

  24. Gander, M.J.: Overlapping Schwarz for linear and nonlinear parabolic problems. In: Proceedings of the 9th International Conference on Domain Decomposition, pp. 97–104 (1996). ddm.org

    Google Scholar 

  25. Gander, M.J.: A waveform relaxation algorithm with overlapping splitting for reaction diffusion equations. Numer. Linear Algebra Appl. 6, 125–145 (1998)

    Article  MathSciNet  Google Scholar 

  26. Gander, M.J.: Optimized Schwarz methods. SIAM J. Numer. Anal. 44, 699–731 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  27. Gander, M.J., Güttel, S.: ParaExp: a parallel integrator for linear initial-value problems. SIAM J. Sci. Comput. 35, C123–C142 (2013)

    Article  MATH  Google Scholar 

  28. Gander, M.J., Hairer, E.: Nonlinear convergence analysis for the parareal algorithm. In: Widlund, O.B., Keyes, D.E. (eds.) Domain Decomposition Methods in Science and Engineering XVII. Lecture Notes in Computational Science and Engineering, vol. 60, pp. 45–56. Springer, Berlin (2008)

    Chapter  Google Scholar 

  29. Gander, M.J., Halpern, L.: Absorbing boundary conditions for the wave equation and parallel computing. Math. Comput. 74, 153–176 (2004)

    Article  MathSciNet  Google Scholar 

  30. Gander, M.J., Halpern, L.: Optimized Schwarz waveform relaxation methods for advection reaction diffusion problems. SIAM J. Numer. Anal. 45, 666–697 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  31. Gander, M.J., Halpern, L.: A direct solver for time parallelization. In: 22nd International Conference of Domain Decomposition Methods. Springer, Berlin (2014)

    Google Scholar 

  32. Gander, M.J., Neumüller, M.: Analysis of a new space-time parallel multigrid algorithm for parabolic problems (2015, submitted)

    Google Scholar 

  33. Gander, M.J., Petcu, M.: Analysis of a Krylov subspace enhanced parareal algorithm for linear problems. In: ESAIM: Proceedings. EDP Sciences, vol. 25, pp. 114–129 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. Gander, M.J., Stuart, A.M.: Space-time continuous analysis of waveform relaxation for the heat equation. SIAM J. Sci. Comput. 19, 2014–2031 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  35. Gander, M.J., Vandewalle, S.: Analysis of the parareal time-parallel time-integration method. SIAM J. Sci. Comput. 29, 556–578 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  36. Gander, M.J., Halpern, L., Nataf, F.: Optimal convergence for overlapping and non-overlapping Schwarz waveform relaxation. In: Lai, C.-H., Bjørstad, P., Cross, M., Widlund, O. (eds.) Eleventh International Conference of Domain Decomposition Methods (1999). ddm.org

    Google Scholar 

  37. Gander, M.J., Halpern, L., Nataf, F.: Optimal Schwarz waveform relaxation for the one dimensional wave equation. SIAM J. Numer. Anal. 41, 1643–1681 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  38. Gander, M.J., Jiang, Y.-L., Li, R.-J.: Parareal Schwarz waveform relaxation methods. In: Widlund, O.B., Keyes, D.E. (eds.) Domain Decomposition Methods in Science and Engineering XX. Lecture Notes in Computational Science and Engineering, vol. 60, pp. 45–56. Springer, Berlin (2013)

    Google Scholar 

  39. Gander, M.J., Kwok, F., Mandal, B.: Dirichlet-Neumann and Neumann-Neumann waveform relaxation algorithms for parabolic problems (2015, submitted)

    Google Scholar 

  40. Gear, C.W.: Parallel methods for ordinary differential equations. Calcolo 25, 1–20 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  41. Giladi, E., Keller, H.B.: Space time domain decomposition for parabolic problems. Numer. Math. 93, 279–313 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  42. Güttel, S.: A parallel overlapping time-domain decomposition method for ODEs. In: Domain Decomposition Methods in Science and Engineering XX, pp. 459–466. Springer, Berlin (2013)

    Google Scholar 

  43. Hackbusch, W.: Parabolic multi-grid methods. In: Glowinski, R. Lions, J.-L. (eds.) Computing Methods in Applied Sciences and Engineering, VI. pp. 189–197. North-Holland, Amsterdam (1984)

    Google Scholar 

  44. Hoang, T.-P., Jaffré, J., Japhet, C., Kern, M., Roberts, J.E.: Space-time domain decomposition methods for diffusion problems in mixed formulations. SIAM J. Numer. Anal. 51, 3532–3559 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  45. Horton, G., Vandewalle, S.: A space-time multigrid method for parabolic partial differential equations. SIAM J. Sci. Comput. 16, 848–864 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  46. Horton, G., Vandewalle, S., Worley, P.: An algorithm with polylog parallel complexity for solving parabolic partial differential equations, SIAM J. Sci. Comput. 16, 531–541 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  47. Keller, H.B.: Numerical Solution for Two-Point Boundary-Value Problems. Dover Publications Inc, New York (1992)

    Google Scholar 

  48. Kiehl, M.: Parallel multiple shooting for the solution of initial value problems. Parallel Comput. 20, 275–295 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  49. Kogge, P.M., Stone, H.S.: A parallel algorithm for the efficient solution of a general class of recurrence equations. IEEE Trans. Comput. 100, 786–793 (1973)

    Article  MathSciNet  Google Scholar 

  50. Kwok, F.: Neumann-Neumann waveform relaxation for the time-dependent heat equation. In: Domain Decomposition Methods in Science and Engineering, DD21. Springer, Berlin (2014)

    Google Scholar 

  51. Lai, C.-H.: On transformation methods and the induced parallel properties for the temporal domain. In: Magoulès, F. (ed.) Substructuring Techniques and Domain Decomposition Methods, pp. 45–70. Saxe-Coburg Publications, Scotland (2010)

    Chapter  Google Scholar 

  52. Lelarasmee, E., Ruehli, A.E., Sangiovanni-Vincentelli, A. L.: The waveform relaxation method for time-domain analysis of large scale integrated circuits. IEEE Trans. CAD IC Syst. 1, 131–145 (1982)

    Article  Google Scholar 

  53. Lindelöf, E.: Sur l’application des méthodes d’approximations successives à l’étude des intégrales réelles des équations différentielles ordinaires. J. de Math. Pures et Appl. 10, 117–128 (1894)

    MATH  Google Scholar 

  54. Lions, J.-L., Maday, Y., Turinici, G.: A parareal in time discretization of PDEs. C.R. Acad. Sci. Paris, Serie I 332, 661–668 (2001)

    MathSciNet  MATH  Google Scholar 

  55. Lubich, C., Ostermann, A.: Multi-grid dynamic iteration for parabolic equations. BIT 27, 216–234 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  56. Maday, Y., Rønquist, E.M.: Parallelization in time through tensor-product space–time solvers. C.R. Math. 346, 113–118 (2008)

    Google Scholar 

  57. Maday, Y., Turinici, G.: The parareal in time iterative solver: a further direction to parallel implementation. In: Domain Decomposition Methods in Science and Engineering, pp. 441–448. Springer, Berlin (2005)

    Google Scholar 

  58. Mandal, B.: A time-dependent Dirichlet-Neumann method for the heat equation. In: Domain Decomposition Methods in Science and Engineering, DD21. Springer, Berlin (2014)

    Google Scholar 

  59. Milne, W.E., Wiley, J.: Numerical Solution of Differential Equations. vol. 19(3). Wiley, New York (1953)

    Google Scholar 

  60. Minion, M.L.: A hybrid parareal spectral deferred corrections method. Commun. Appl. Math. Comput. Sci. 5, 265–301 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  61. Miranker, W.L., Liniger, W.: Parallel methods for the numerical integration of ordinary differential equations. Math. Comput. 91, 303–320 (1967)

    Article  MathSciNet  Google Scholar 

  62. Nataf, F., Rogier. F.: Factorization of the convection-diffusion operator and the Schwarz algorithm. M 3 AS 5, 67–93 (1995)

    Google Scholar 

  63. Neumüller, M.: Space-time methods: fast solvers and applications, Ph.D. thesis, University of Graz (2013)

    Google Scholar 

  64. Nievergelt, J.: Parallel methods for integrating ordinary differential equations. Commun. ACM 7, 731–733 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  65. Picard, E.: Sur l’application des méthodes d’approximations successives à l’étude de certaines équations différentielles ordinaires. J. de Math. Pures et Appl. 9, 217–271 (1893)

    MATH  Google Scholar 

  66. Ruprecht, D., Krause, R.: Explicit parallel-in-time integration of a linear acoustic-advection system. Comput. Fluids 59, 72–83 (2012)

    Article  MathSciNet  Google Scholar 

  67. Saha, P., Stadel, J., Tremaine, S.: A parallel integration method for solar system dynamics. Astron. J. 114, 409–415 (1997)

    Article  Google Scholar 

  68. Saltz, J.H., Naik, V.K.: Towards developing robust algorithms for solving partial differential equations on MIMD machines. Parallel Comput. 6, 19–44 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  69. Sameh, A.H., Brent, R.P.: Solving triangular systems on a parallel computer. SIAM J. Numer. Anal. 14, 1101–1113 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  70. Schwarz, H.A.: Über einen Grenzübergang durch alternierendes Verfahren. Vierteljahrsschrift der Naturforschenden Gesellschaft in Zürich 15, 272–286 (1870)

    Google Scholar 

  71. Shampine, L., Watts, H.: Block implicit one-step methods. Math. Comput. 23, 731–740 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  72. Sheen, D., Sloan, I., Thomée, V.: A parallel method for time-discretization of parabolic problems based on contour integral representation and quadrature. Math. Comput. Am. Math. Soc. 69, 177–195 (1999)

    Article  Google Scholar 

  73. Sheen, D., Sloan, I.H., Thomée, V.: A parallel method for time discretization of parabolic equations based on Laplace transformation and quadrature. IMA J. Numer. Anal. 23, 269–299 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  74. Simoens, J., Vandewalle, S.: Waveform relaxation with fast direct methods as preconditioner. SIAM J. Sci. Comput. 21, 1755–1773 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  75. Speck, R., Ruprecht, D., Krause, R., Emmett, M., Minion, M., Winkel, M., Gibbon, P.: A massively space-time parallel n-body solver. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, Salt Lake City, p. 92. IEEE Computer Society Press, Los Alamitos (2012)

    Google Scholar 

  76. Speck, R., Ruprecht, D., Emmett, M., Minion, M., Bolten, M., Krause, R.: A multi-level spectral deferred correction method. arXiv:1307.1312 (2013, arXiv preprint)

    Google Scholar 

  77. Thomée, V.: A high order parallel method for time discretization of parabolic type equations based on Laplace transformation and quadrature. Int. J. Numer. Anal. Model. 2, 121–139 (2005)

    Google Scholar 

  78. Vandewalle, S., Van de Velde, E.: Space-time concurrent multigrid waveform relaxation. Ann. Numer. Math. 1, 347–363 (1994)

    MathSciNet  MATH  Google Scholar 

  79. Womble, D.E.: A time-stepping algorithm for parallel computers. SIAM J. Sci. Stat. Comput. 11, 824–837 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  80. Worley, P.: Parallelizing across time when solving time-dependent partial differential equations. In: Sorensen, D. (ed.) Proceedings of 5th SIAM Conference on Parallel Processing for Scientific Computing. SIAM, Houston (1991)

    Google Scholar 

Download references

Acknowledgements

The author is very thankful for the comments of Stefan Vandewalle, which greatly improved this manuscript and also made the content more complete. We thank the Bibliotheque de Geneve for granting permission to reproduce pictures from the original sources.

Author information

Authors and Affiliations

  1. Section of Mathematics, University of Geneva, Rue du Lievre 2-4, CP 64, 1211, Geneva 4, Switzerland

    Martin J. Gander

Authors
  1. Martin J. Gander
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin J. Gander .

Editor information

Editors and Affiliations

  1. Heidelberg University, Institute for Applied Mathematics, Heidelberg, Germany

    Thomas Carraro

  2. Heidelberg University, Institute for Applied Mathematics, Heidelberg, Germany

    Michael Geiger

  3. Heidelberg University, Interdisciplinary Center for Scientific Computing, Heidelberg, Germany

    Stefan Körkel

  4. Heidelberg University, Institute for Applied Mathematics, Heidelberg, Germany

    Rolf Rannacher

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Gander, M.J. (2015). 50 Years of Time Parallel Time Integration. In: Carraro, T., Geiger, M., Körkel, S., Rannacher, R. (eds) Multiple Shooting and Time Domain Decomposition Methods. Contributions in Mathematical and Computational Sciences, vol 9. Springer, Cham. https://doi.org/10.1007/978-3-319-23321-5_3

Download citation

Publish with us

Policies and ethics

Search

Navigation