Skip to main content
SpringerLink
Log in

Towards HPC simulations of billion-cell reservoirs by multiscale mixed methods

  • Original Paper
  • Published:
Computational Geosciences Aims and scope Submit manuscript

Abstract

A three dimensional parallel implementation of Multiscale Mixed Methods based on non-overlapping domain decomposition techniques is proposed for multi-core computers and its computational performance is assessed by means of numerical experiments. As a prototypical method, from which many others can be derived, the Multiscale Robin Coupled Method is chosen and its implementation explained in detail. Numerical results for problems ranging from millions up to more than 2 billion computational cells in highly heterogeneous anisotropic rock formations based on the SPE10 benchmark are shown. The proposed implementation relies on direct solvers for both local problems and the interface coupling system. We find good weak and strong scalalability as compared against a state-of-the-art global fine grid solver based on Algebraic Multigrid preconditioning in single and two-phase flow problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

Data will be made available on reasonable request.

References

  1. Fung, L.S.K., Dogru, A.H.: Parallel unstructured-solver methods for simulation of complex giant reservoirs. SPE J. 13(04), 440–446 (2008)

    Article  Google Scholar 

  2. Odeh, A.S.: Comparison of solutions to a three-dimensional black-oil reservoir simulation problem. J. Petrol. Tech. 33(01), 13–25 (1981)

    Article  Google Scholar 

  3. Christie, M., Blunt, M.J.: Tenth SPE comparative solution project: a comparison of upscaling techniques. 2001 SPE Reservoir Simulation Symposium (2001)

  4. Dogru, A.H., Fung, L.S.K., Middya, U., Al-Shaalan, T., Pita, J.A.: A next-generation parallel reservoir simulator for giant reservoirs. SPE reservoir simulation conference (2009)

  5. Manea, A.M., Sewall, J., Tchelepi, H.A.: Parallel multiscale linear solver for highly detailed reservoir models. SPE J. 21(06), 2062–2078 (2016)

    Article  Google Scholar 

  6. Durlofsky, L.J., Efendiev, Y., Ginting, V.: An adaptive local–global multiscale finite volume element method for two-phase flow simulations. Adv. Water Resour. 30(3), 576–588 (2007)

    Article  Google Scholar 

  7. Lie, K.-A., Møyner, O., Natvig, J.R., Kozlova, A., Bratvedt, K., Watanabe, S., Li, Z.: Successful application of multiscale methods in a real reservoir simulator environment. Comput. Geosci. 21(5-6), 981–998 (2017)

    Article  Google Scholar 

  8. Francisco, A., Ginting, V., Pereira, F., Rigelo, J.: Design and implementation of a multiscale mixed method based on a nonoverlapping domain decomposition procedure. Math. Comput. Simul. 99, 125–138 (2014)

    Article  Google Scholar 

  9. Guiraldello, R.T., Ausas, R.F., Sousa, F.S., Pereira, F., Buscaglia, G.C.: The multiscale Robin coupled method for flows in porous media. J. Comput. Phys. 355, 1–21 (2018)

    Article  Google Scholar 

  10. Arbogast, T., Pencheva, G., Wheeler, M.F., Yotov, I.: A multiscale mortar mixed finite element method. Multiscale Model. Simul. 6(1), 319–346 (2007)

    Article  Google Scholar 

  11. Jenny, P., Lee, S.H., Tchelepi, H.A.: Multi-scale finite-volume method for elliptic problems in subsurface flow simulation. J. Comput. Phys. 187(1), 47–67 (2003)

    Article  Google Scholar 

  12. Efendiev, Y., Galvis, J., Hou, T.Y.: Generalized multiscale finite element methods (gmsfem). J. Comput. Phys. 251, 116–135 (2013)

    Article  Google Scholar 

  13. Hou, T.Y., Wu, X-H: A multiscale finite element method for elliptic problems in composite materials and porous media. J. Comput. Phys. 134(1), 169–189 (1997)

    Article  Google Scholar 

  14. Harder, C., Paredes, D., Valentin, F.: A family of multiscale hybrid-mixed finite element methods for the darcy equation with rough coefficients. J. Comput. Phys. 245, 107–130 (2013)

    Article  Google Scholar 

  15. Wang, Y., Hajibeygi, H., Tchelepi, H.A.: Algebraic multiscale solver for flow in heterogeneous porous media. J. Comput. Phys. 259, 284–303 (2014)

    Article  Google Scholar 

  16. Møyner, O., Lie, K.-A.: A multiscale restriction-smoothed basis method for high contrast porous media represented on unstructured grids. J. Comput. Phys. 304, 46–71 (2016)

    Article  Google Scholar 

  17. Aarnes, J.E.: On the use of a mixed multiscale finite element method for greater flexibility and increased speed or improved accuracy in reservoir simulation. Multiscale Model. Simul. 2(3), 421–439 (2004)

    Article  Google Scholar 

  18. Kippe, V., Aarnes, J.E., Lie, K.-A.: A comparison of multiscale methods for elliptic problems in porous media flow. Comput. Geosci. 12(3), 377–398 (2008)

    Article  Google Scholar 

  19. Rocha, F.F., Sousa, F.S., Ausas, R.F., Buscaglia, G.C., Pereira, F.: Multiscale mixed methods for two-phase flows in high-contrast porous media. J. Comput. Phys. 409, 109316 (2020)

    Article  Google Scholar 

  20. Manea, A.M., Sewall, J., Tchelepi, H.A.: Parallel multiscale linear solver for highly detailed reservoir models. SPE J. 21(6), 2062–2078 (2016)

    Article  Google Scholar 

  21. Manea, A.M., Hajibeygi, H., Vassilevski, P., Tchelepi, H.A.: Parallel enriched algebraic multiscale solver. https://doi.org/10.2118/182694-MS (2017)

  22. Puscas, M., Enchery, G., Desroziers, S.: Application of the mixed multiscale finite element method to parallel simulations of two-phase flows in porous media. Oil Gas Sci. Technol. 73 (2018)

  23. Kozlova, A., Li, Z., Natvig, J.R., Watanabe, S., Zhou, Y., Bratvedt, K., Lee, S.H.: A real-field multiscale black-oil reservoir simulator. SPE J. 21(06), 2049–2061 (2016)

    Article  Google Scholar 

  24. Abreu, E., Ferraz, P., Espírito Santo, A.M., Pereira, F., Santos, L.G.C., Sousa, F.S.: Recursive formulation and parallel implementation of multiscale mixed methods (2020)

  25. Guiraldello, R.T., Ausas, R.F., Sousa, F.S., Pereira, F., Buscaglia, G.C.: Interface spaces for the multiscale robin coupled method in reservoir simulation. Math. Comput. Simul. 164, 103–119 (2019)

    Article  Google Scholar 

  26. Rocha, F.F.: Enhanced multiscale mixed methods for two-phase flows in high-contrast porous media. PhD thesis, Instituto de Ciências Matemáticas e de Computação USP (2020)

  27. Rocha, F., Sousa, F.S., Ausas, P.F., Buscaglia, G.C.: Interface spaces based on physics for multiscale mixed methods applied to flows in fractured-like porous media. Comput. Methods Appl. Mech. Eng. 385 (2021)

  28. Guiraldello, R.T., Ausas, R.F., Sousa, F.S., Pereira, F., Buscaglia, G.C.: Velocity postprocessing schemes for multiscale mixed methods applied to contaminant transport in subsurface flows. Comput. Geosci. 24(3), 1141–1161 (2020)

    Article  Google Scholar 

  29. Sheldon, J.W., Cardwell, W.T.: One-dimensional, incompressible, noncapillary, two-phase fluid flow in a porous medium. Trans. AIME 216(01), 290–296 (1959)

    Article  Google Scholar 

  30. Ascher, U.M., Ruuth, S.J., Wetton, B.T.R.: Implicit-explicit methods for time-dependent partial differential equations. SIAM J. Numer. Anal. 32(3), 797–823 (1995)

    Article  Google Scholar 

  31. Chen, Z., Huan, G., Li, B.: An improved IMPES method for two-phase flow in porous media. Transp. Porous Media 54(3), 361–376 (2004)

    Article  Google Scholar 

  32. Paz, S., Jaramillo, A., Guiraldello, R.T., Ausas, R.F., Sousa, F.S., Pereira, F., Buscaglia, G.C.: An adaptive time stepping algorithm for IMPES with high order polynomial extrapolation. Appl. Math. Model. 91, 1100–1116 (2021)

    Article  Google Scholar 

  33. Lie, K.-A.: An Introduction to Reservoir Simulation Using MATLAB/GNU Octave: User Guide for the MATLAB Reservoir Simulation Toolbox (MRST). Cambridge University Press, Cambridge (2019)

    Book  Google Scholar 

  34. Chen, Z., Huan, G., Li, B.: An improved impes method for two-phase flow in porous media. Transp. Porous Media 54(3), 361–376 (2004)

    Article  Google Scholar 

  35. Falgout, R.: Hypre user’s manual. http://www.llnl.gov/CASC/hypre. Accessed 30 Nov 2020 (2017)

  36. Ruge, J.W., Stüben, K.: Algebraic multigrid. In: Multigrid Methods. Society for Industrial and Applied Mathematics, pp 73–130 (1987)

  37. Amestoy, P.R., Duff, I.S., Koster, J., L’Excellent, J.-Y.: A fully asynchronous multifrontal solver using distributed dynamic scheduling. SIAM J. Matrix Anal. Appl. 23(1), 15–41 (2001)

    Article  Google Scholar 

  38. Amestoy, P.R., Guermouche, A., L’Excellent, J.-Y., Pralet, S.: Hybrid scheduling for the parallel solution of linear systems. Parallel Comput. 32(2), 136–156 (2006)

    Article  Google Scholar 

  39. Al-Shaalan, T.M., Dogru, A.H., Al-Youbi, A.H., Ismail, H.: Parallel computation for complex wells modeling in full-field reservoir simulation. SPE Reservoir Simulation Conference (2017)

  40. Courant, R., Friedrichs, K., Lewy, H.: Über die partiellen differenzengleichungen der mathematischen physik. Math. Ann. 100(1), 32–74 (1928)

    Article  Google Scholar 

  41. Keith, H., et al.: Coats A note on IMPES and some impes-based simulation models. SPE J. 5 (03), 245–251 (2000)

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the financial support received from the Brazilian oil company Petrobras grant 2015/00400-4, from the São Paulo Research Foundation FAPESP/CEPID/CeMEAI grant 2013/07375-0, and from Brazilian National Council for Scientific and Technological Development CNPq grants 305599/2017-8 and 310990/2019-0. This research was carried out using computational resources from the Cluster Euler, Centre for Mathematical Sciences Applied to Industry (CeMEAI). This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Author information

Authors and Affiliations

  1. Institute of Mathematics and Computer Science, University of São Paulo, Av. Trab. São Carlense, 400 - Centro, São Carlos, 13566-590, SP, Brazil

    Alfredo Jaramillo, Rafael T. Guiraldello, Roberto F. Ausas, Fabricio S. Sousa & Gustavo C. Buscaglia

  2. Departamento de Matemáticas, Universidad del Valle, Calle 13 No. 100-00, Cali, Colombia

    Stevens Paz

  3. Department of Mathematical Sciences, The University of Texas, Dallas, TX, USA

    Felipe Pereira

Authors
  1. Alfredo Jaramillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafael T. Guiraldello
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stevens Paz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roberto F. Ausas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fabricio S. Sousa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Felipe Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gustavo C. Buscaglia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alfredo Jaramillo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jaramillo, A., Guiraldello, R.T., Paz, S. et al. Towards HPC simulations of billion-cell reservoirs by multiscale mixed methods. Comput Geosci 26, 481–501 (2022). https://doi.org/10.1007/s10596-022-10131-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10596-022-10131-z

Keywords

Search

Navigation