Skip to main content
SpringerLink
Log in

Accelerating Algebraic Multigrid Methods via Artificial Neural Networks

  • Original Article
  • Published:
Vietnam Journal of Mathematics Aims and scope Submit manuscript

Abstract

We present a novel deep learning-based algorithm to accelerate—through the use of Artificial Neural Networks (ANNs)—the convergence of Algebraic Multigrid (AMG) methods for the iterative solution of the linear systems of equations stemming from finite element discretizations of Partial Differential Equations (PDE). We show that ANNs can be successfully used to predict the strong connection parameter that enters in the construction of the sequence of increasingly smaller matrix problems standing at the basis of the AMG algorithm, so as to maximize the corresponding convergence factor of the AMG scheme. To demonstrate the practical capabilities of the proposed algorithm, which we call AMG-ANN, we consider the iterative solution of the algebraic system of equations stemming from finite element discretizations of two-dimensional model problems. First, we consider an elliptic equation with a highly heterogeneous diffusion coefficient and then a stationary Stokes problem. We train (off-line) our ANN with a rich dataset and present an in-depth analysis of the effects of tuning the strong threshold parameter on the convergence factor of the resulting AMG iterative scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 3
Algorithm 4
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Algorithm 5
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Abhyankar, S., Brown, J., Constantinescu, E.M., Ghosh, D., Smith, B.F., Zhang, H.: PETSc/TS: a modern scalable ODE/DAE solver library. arXiv:1806.01437 (2018)

  2. Antonietti, P.F., Dassi, F., Manuzzi, E.: Machine learning based refinement strategies for polyhedral grids with applications to Virtual Element and polyhedral discontinuous Galerkin methods. J. Comput. Phys. In press (2022)

  3. Antonietti, P.F., Manuzzi, E.: Refinement of polygonal grids using convolutional neural networks with applications to polygonal discontinuous Galerkin and virtual element methods. J. Comput. Phys. 452, 110900 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  4. Antonietti, P.F., Melas, L.: Algebraic multigrid schemes for high-order nodal discontinuous Galerkin methods. SIAM J. Sci. Comput. 42, A1147–A1173 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  5. Arndt, D., Bangerth, W., Clevenger, T.C., Davydov, D., Fehling, M., Garcia-Sanchez, D., Harper, G., Heister, T., Heltai, L., Kronbichler, M., Kynch, R.M., Maier, M., Pelteret, J.-P., Turcksin, B., Wells, D.: The deal.II library, version 9.1. J. Numer. Math. 27, 203–213 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  6. Ashby, S.F., Falgout, R.D.: A parallel multigrid preconditioned conjugate gradient algorithm for groundwater flow simulations. Nuclear Sci. Eng. 124, 145–159 (1996)

    Article  Google Scholar 

  7. Baker, A.H., Falgout, R.D., Kolev, T.V., Yang, U.M.: Scaling Hypre’s multigrid solvers to 100,000 cores. In: Berry, M. W., et al. (eds.) High-Performance Scientific Computing, pp. 261–279. Springer, London (2012)

  8. Baker, A.H., Kolev, Tz.V., Yang, U.M.: Improving algebraic multigrid interpolation operators for linear elasticity problems. Numer. Linear Algebra Appl. 17, 495–517 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bank, R., Falgout, R., Jones, T., Manteuffel, T.A., McCormick, S.F., Ruge, J.W.: Algebraic multigrid domain and range decomposition (AMG-DD/AMG-RD). SIAM J. Sci. Comput. 37, S113–S136 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bastian, P., Blatt, M., Scheichl, R.: Algebraic multigrid for discontinuous Galerkin discretizations of heterogeneous elliptic problems. Numer. Linear Algebra Appl. 19, 367–388 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Boffi, D., Brezzi, F., Fortin, M.: Mixed Finite Element Methods and Applications. Springer Series in Computational Mathematics, Vol. 44. Springer, Berlin (2013)

    Book  MATH  Google Scholar 

  12. Brandt, A., Livne, O.E.: Multigrid Techniques. SIAM, Philadelphia (2011)

  13. Brannick, J., Brezina, M., MacLachlan, S., Manteuffel, T., McCormick, S., Ruge, J.: An energy-based AMG coarsening strategy. Numer. Linear Algebra Appl. 13, 133–148 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  14. Brezina, M., Cleary, A.J., Falgout, R.D., Henson, V.E., Jones, J.E., Manteuffel, T.A., McCormick, S.F., Ruge, J.W.: Algebraic multigrid based on element interpolation (AMGe). SIAM J. Sci. Comput. 22, 1570–1592 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  15. Brezis, H.: Functional Analysis, Sobolev Spaces and Partial Differential Equations. Springer, New York (2010)

    Google Scholar 

  16. Bui, Q.M., Wang, L., Osei-Kuffuor, D.: Algebraic multigrid preconditioners for two-phase flow in porous media with phase transitions. Adv. Water Resour. 114, 19–28 (2018)

    Article  Google Scholar 

  17. Cleary, A.J., Falgout, R.D., Henson, V.E., Jones, J.E., Manteuffel, T.A., McCormick, S.F., Miranda, G.N., Ruge, J.W.: Robustness and scalability of algebraic multigrid. SIAM J. Sci. Comput. 21, 1886–1908 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  18. Discacciati, N., Hesthaven, J.S., Ray, D.: Controlling oscillations in high-order discontinuous Galerkin schemes using artificial viscosity tuned by neural networks. J. Comput. Phys. 409, 109304 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  19. Falgout, R., Vassilevski, S.: On generalizing the algebraic multigrid framework. SIAM J. Sci. Comput. 42, 1669–1693 (2004)

    MathSciNet  MATH  Google Scholar 

  20. Falgout, R., Yang, U.: Hypre: a library of high performance preconditioners. In: Sloot, P.M.A., et al. (eds.) Computational science—ICCS 2002. Lecture Notes in Computer Science, vol. 2331, pp. 632–641. Springer, Berlin (2002)

  21. Fresca, S., Dede’, L., Manzoni, A.: A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs. J. Sci. Comput. 87, 61 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  22. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. The MIT Press, Massachusetts (2016)

    MATH  Google Scholar 

  23. Gottschalk, H., Kahl, K.: Coarsening in algebraic multigrid using gaussian processes. Electron. Trans. Numer. Anal. 54, 514–533 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  24. Greenfeld, D., Galun, M., Basri, R., Yavneh, I., Kimmel, R.: Learning to optimize multigrid PDE solvers. In: International Conference on Machine Learning, pp. 2415–2423. PMLR (2019)

  25. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)

  26. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  27. Heinlein, A., Klawonn, A., Lanser, M., Weber, J.: Combining machine learning and adaptive coarse spaces—a hybrid approach for robust FETI-DP methods in three dimensions. SIAM J. Sci. Comput. 43, S816–S838 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  28. Henson, V.E., Vassilevski, P.S.: Algebraic multigrid preconditioners for two-phase flow in porous media with phase transitions. SIAM J. Sci. Comput. 23, 629–650 (2001)

    Article  MathSciNet  Google Scholar 

  29. Hesthaven, J.S., Ubbiali, S.: Non-intrusive reduced order modeling of nonlinear problems using neural networks. J. Comput. Phys. 363, 55–78 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  30. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  31. Hughes, T.J.R.: The Finite Element Method: Linear Static and Dynamic Finite Element Analysis. Courier Corporation (2012)

  32. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 (2015)

  33. Janssens, M., Hulshoff, S.: Advancing artificial neural network parameterization for atmospheric turbulence using a variational multiscale model. J. Adv. Model. Earth Syst. 14, e2021MS002490 (2021)

    Google Scholar 

  34. Jones, J., Lee, B.: A multigrid method for variable coefficient Maxwell’s equations. SIAM J. Sci. Comput. 27, 1689–1708 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  35. Katrutsa, A., Daulbaev, T., Oseledets, I.: Black-box learning of multigrid parameters. J. Comput. Appl. Math. 368, 112524 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  36. Kingma, D.P., Ba Adam, J.: A method for stochastic optimization. arXiv:1412.6980 (2014)

  37. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., Hadsell, R.: Overcoming catastrophic forgetting in neural networks. Proc. National Acad. Sci. 114, 3521–3526 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  38. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  39. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)

    Article  Google Scholar 

  40. Li, R., Zhang, C.: Efficient parallel implementations of sparse triangular solves for GPU architectures. In: Proceedings of the 2020 SIAM Conference on Parallel Processing for Scientific Computing, pp. 106–117. SIAM (2020)

  41. Lions, J.L., Magenes, E.: Non-Homogeneous Boundary Value Problems and Applications. Springer, Berlin (1972)

    Book  MATH  Google Scholar 

  42. Mishra, S.: A machine learning framework for data driven acceleration of computations of differential equations. Math. Eng. 1, 118–146 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  43. Neittaanmäki, P., Repin, S.: Artificial intelligence and computational science. In: Tuovinen, T., Periaux, J., Neittaanmäki, P. (eds.) Computational Sciences and Artificial Intelligence in Industry. Intelligent Systems, Control and Automation: Science and Engineering, vol. 76, pp. 27–35. Springer, Cham (2022)

  44. Quarteroni, A.: Numerical Models for Differential Problems. MS&A, vol. 16. Springer, Cham (2017)

  45. Quarteroni, A., Valli, A.: Numerical Approximation of Partial Differential Equations. Springer, Berlin (1994)

    Book  MATH  Google Scholar 

  46. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Machine learning of linear differential equations using Gaussian processes. J. Comput. Phys. 348, 683–693 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  47. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  48. Regazzoni, F., Dede’, L., Quarteroni, A.: Machine learning for fast and reliable solution of time-dependent differential equations. J. Comput. Phys. 397, 108852 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  49. Ruge, J.W., Stüben, K.: 4. Algebraic multigrid. In: Mccormick, S.F (ed.) Multigrid Methods, pp. 73–130. SIAM (1987)

  50. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  51. Seabold, S., Perktold, J.: Statsmodels: Econometric and Statistical Modeling with Python. In: 9th Python in science conference (2010)

  52. Siefert, C., Tuminaro, R., Gerstenberger, A., Scovazzi, G., Collis, S.S.: Algebraic multigrid techniques for discontinuous Galerkin methods with varying polynomial order. Comput. Geosci. 18, 597–612 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  53. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  54. Stüben, K.: An introduction to algebraic multigrid. In: Multigrid, Chapter A pp. 413– 532. Elsevier Science (2001)

  55. Stüben, K.: A review of algebraic multigrid. J. Comput. Appl. Math. 128, 281–309 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  56. Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on Machine Learning, pp. 1139–1147 (2013)

  57. Tassi, T., Zingaro, A., Dede’, L.: A machine learning approach to enhance the SUPG stabilization method for advection-dominated differential problems. Math. Eng. 5(2), 1–26 (2023)

    Article  MathSciNet  Google Scholar 

  58. Tieleman, T., Hinton, G.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4, 26–31 (2012)

    Google Scholar 

  59. Waskom, M.L.: Seaborn: statistical data visualization. J. Open Source Softw. 6, 3021 (2021)

    Article  Google Scholar 

  60. Webster, R.: An algebraic multigrid solver for navier-stokes problems. Int. J. Numer. Methods Fluids 18, 761–780 (1994)

    Article  MATH  Google Scholar 

  61. Xu, J., Zikatanov, L.: Algebraic multigrid methods. Acta Numer. 26, 591–721 (2017)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

P.F.A and L.D. are members of the INdAM Research group GNCS. P.F.A has been partially funded by the research projects PRIN17 (n. 201744KLJL) and PRIN 2020 (n. 20204LN5N5), funded by Italian Ministry of University and Research (MUR). L.D. has been partially funded by the research project PRIN 2020 (n. 20204LN5N5) funded by MUR.

Author information

Authors and Affiliations

  1. MOX, Dipartimento di Matematica, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133, Milano, Italy

    Paola F. Antonietti, Matteo Caldana & Luca Dede’

Authors
  1. Paola F. Antonietti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matteo Caldana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luca Dede’
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Dede’.

Additional information

Dedicated to Professor Alfio Quarteroni on the occasion of his 70th birthday.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Antonietti, P.F., Caldana, M. & Dede’, L. Accelerating Algebraic Multigrid Methods via Artificial Neural Networks. Vietnam J. Math. 51, 1–36 (2023). https://doi.org/10.1007/s10013-022-00597-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10013-022-00597-w

Keywords

Mathematics Subject Classification (2010)

Search

Navigation