Lossy data compression reduces communication time in hybrid time-parallel integrators

  • Lisa Fischer
  • Sebastian Götschel
  • Martin WeiserEmail author
Special Issue Parallel-in-Time Methods


Parallel-in-time methods for solving initial value problems are a means to increase the parallelism of numerical simulations. Hybrid parareal schemes interleaving the parallel-in-time iteration with an iterative solution of the individual time steps are among the most efficient methods for general nonlinear problems. Despite the hiding of communication time behind computation, communication has in certain situations a significant impact on the total runtime. Here we present strict, yet not sharp, error bounds for hybrid parareal methods with inexact communication due to lossy data compression, and derive theoretical estimates of the impact of compression on parallel efficiency of the algorithms. These and some computational experiments suggest that compression is a viable method to make hybrid parareal schemes robust with respect to low bandwidth setups.



Partial funding by BMBF Project SOAK and DFG Project WE 2937/6-1 is gratefully acknowledged. The authors would also like to thank Th. Steinke, F. Wende, A. Kammeyer and the North-German Supercomputing Alliance (HLRN) for supporting the numerical experiments.


  1. 1.
    Alhubail, M., Wang, Q., Williams, J.: The swept rule for breaking the latency barrier in time advancing two-dimensional PDEs. Preprint (2016). arXiv:1602.07558
  2. 2.
    Alhubail, M., Wang, Q.: The swept rule for breaking the latency barrier in time advancing PDEs. J. Comput. Phys. 307, 110–121 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Aubanel, E.: Scheduling of tasks in the parareal algorithm. Parallel Comput. 37, 172–182 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bal, G.: On the convergence and the stability of the parareal algorithm to solve partial differential equations. In: Kornhuber, R., Hoppe, R., Périaux, J., Pironneau, O., Widlund, O., Xu, J. (eds.) Proceedings of DD15, volume 40 of Lecture Notes in Computational Science and Engineering, pp. 425–432. Springer (2004)Google Scholar
  5. 5.
    Barker, A.T.: A minimal communication approach to parallel time integration. Int. J. Comput. Math. 91(3), 601–615 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bolten, M., Moser, D., Speck, R.: Asymptotic convergence of the parallel full approximation scheme in space and time for linear problems. Preprint (2017). arXiv:1703.07120
  7. 7.
    Christlieb, A.J., Macdonald, C.B., Ong, B.W.: Parallel high-order integrators. SIAM J. Sci. Comput. 32(2), 818–835 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Duarte, M., Massot, M., Descombes, S.: Parareal operator splitting techniques for multi-scale reaction waves: numerical analysis and strategies. M2AN 45(5), 825–852 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Dutt, A., Greengard, L., Rokhlin, V.: Spectral deferred correction methods for ordinary differential equations. BIT 40(2), 241–266 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Emmett, M., Minion, M.L.: Toward an efficient parallel in time method for partial differential equations. Commun. Appl. Math. Comput. Sci. 7(1), 105–132 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Filgueira, R., Singh, D.E., Carretero, J., Calderón, A., García, F.: Adaptive-compi: enhancing MPI-based applications’ performance and scalability by using adaptive compression. Int. J. High Perform. Comput. Appl. 25(1), 93–114 (2011)CrossRefGoogle Scholar
  12. 12.
    Gander, M.: 50 years of time parallel time integration. In: Carraro, T., Geiger, M., Körkel, S., Rannacher, R. (eds.) Multiple Shooting and Time Domain Decomposition Methods, volume 9 of Contributions in Mathematical and Computational Sciences, pp. 69–113. Springer (2015)Google Scholar
  13. 13.
    Gander, M.J., Hairer, E.: Nonlinear convergence analysis for the parareal algorithm. In: Langer, U., Discacciati, M., Keyes, D.E., Widlund, O.B., Zulehner, W. (eds.) Domain Decomposition Methods in Science and Engineering XVII, pp. 45–56. Springer, Berlin (2008)CrossRefGoogle Scholar
  14. 14.
    Gander, M.J., Neumüller, M.: Analysis of a new space-time parallel multigrid algorithm for parabolic problems. SIAM J. Sci. Comput. 38(4), A2173–A2208 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Gander, M.J., Vandewalle, S.: Analysis of the parareal time-parallel time-integration method. SIAM J. Sci. Comput. 29(2), 556–578 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Götschel, S., Weiser, M.: Lossy compression for PDE-constrained optimization: adaptive error control. Comput. Optim. Appl. 62, 131–155 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Götschel, S., Weiser, M., Schiela, A.: Solving optimal control problems with the Kaskade 7 finite element toolbox. In: Dedner, A., Flemisch, B., Klöfkorn, R. (eds.) Advances in DUNE, pp. 101–112. Springer, Berlin (2012)CrossRefGoogle Scholar
  18. 18.
    Guibert, D., Tromeur-Dervout, D.: Parallel deferred correction method for CFD problems. In: Kwon, J.-H., Periaux, J., Fox, P., Satofuka, N., Ecer, A. (eds.) Parallel Computational Fluid Dynamics 2006: Parallel Computing and Its Applications, pp. 131–136 (2007)Google Scholar
  19. 19.
    Ke, J., Burtscher, M., Speight, E.: Runtime compression of MPI messages to improve the performance and scalability of parallel applications. In: Supercomputing, 2004. Proceedings of the ACM/IEEE SC2004 Conference, p. 59 (2004)Google Scholar
  20. 20.
    Keckler, S.W., Dally, W.J., Khailany, B., Garland, M., Glasco, D.: Gpus and the future of parallel computing. IEEE Micro 31(5), 7–17 (2011)CrossRefGoogle Scholar
  21. 21.
    Klatt, T., Emmett, M., Ruprecht, D., Speck, R., Terzi, S.: PFASST++. (2015). Retrieved: 04 May 2017
  22. 22.
    Leyffer, S., Wild, S.M., Fagan, M., Snir, M., Palem, K., Yoshii, K., Finkel, H.: Doing Moore with less—Leapfrogging Moore’s law with inexactness for supercomputing. CoRR (2016). arXiv:1610.02606
  23. 23.
    Lions, J.-L., Maday, Y., Turinici, G.: A parareal in time discretization of pdes. C. R. Acad. Sci. Paris Ser. I 332, 661–668 (2001)CrossRefzbMATHGoogle Scholar
  24. 24.
    Liu, J., Wang, Y., Li, R.: A hybrid algorithm based on optimal quadratic spline collocation and parareal deferred correction for parabolic PDEs. Math. Probl. Eng. Article ID 6943079 (2016)Google Scholar
  25. 25.
    McDonald, E., Wathen, A.J.: A simple proposal for parallel computation over time of an evolutionary process with implicit time stepping. In: Proceedings of the ENUMATH 2015, Lecture Notes in Computational Science and Engineering. Springer (2016)Google Scholar
  26. 26.
    Minion, M.L.: A hybrid parareal spectral deferred corrections method. Commun. Appl. Math. Comput. Sci. 5(2), 265–301 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Nielsen, A.S., Brunner, G., Hesthaven, J.S.: Communication-aware adaptive parareal with application to a nonlinear hyperbolic system of partial differential equations. EPFL-ARTICLE 228189, EPFL (2017)Google Scholar
  28. 28.
    Núñez, A., Filgueira, R., Merayo, M.G.: SANComSim: a scalable, adaptive and non-intrusive framework to optimize performance in computational science applications. Proc. Comput. Sci. 18, 230–239 (2013)CrossRefGoogle Scholar
  29. 29.
    Palmer, T.: Build imprecise supercomputers. Nature 526, 32–33 (2015)CrossRefGoogle Scholar
  30. 30.
    Ruprecht, D.: Wave propagation characteristics of Parareal. Comput. Vis. Sci. (2017)Google Scholar
  31. 31.
    Saravanan, K.P., Carpenter, P.M., Ramirez, A.: A performance perspective on energy efficient HPC links. In: ICS ’14 Proceedings of the 28th ACM International Conference on Supercomputing, pp. 313–322 (2014)Google Scholar
  32. 32.
    Srinivasan, A., Chandra, N.: Latency tolerance through parallelization of time in scientific applications. Parallel Comput. 31, 777–796 (2005)CrossRefGoogle Scholar
  33. 33.
    Toselli, A., Widlund, O.B.: Domain Decomposition Methods-Algorithms and Theory, volume 34 of Computational Mathematics. Springer, Berlin (2005)CrossRefzbMATHGoogle Scholar
  34. 34.
    Weiser, M.: Faster SDC convergence on non-equidistant grids by DIRK sweeps. BIT Numer. Math. 55(4), 1219–1241 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Weiser, M., Götschel, S.: State trajectory compression for optimal control with parabolic PDEs. SIAM J. Sci. Comput. 34(1), A161–A184 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Wu, S.-L., Zhou, T.: Convergence analysis for three parareal solvers. SIAM J. Sci. Comput. 37(2), A970–A992 (2015)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Lisa Fischer
    • 1
  • Sebastian Götschel
    • 1
  • Martin Weiser
    • 1
    Email author
  1. 1.Zuse Institute BerlinBerlinGermany

Personalised recommendations