Skip to main content

Advertisement

SpringerLink
Log in

Multilevel Graph Partitioning for Three-Dimensional Discrete Fracture Network Flow Simulations

  • Published:
Mathematical Geosciences Aims and scope Submit manuscript

A Correction to this article was published on 29 June 2021

This article has been updated

Abstract

We present a topology-based method for mesh-partitioning in three-dimensional discrete fracture network (DFN) simulations that takes advantage of the intrinsic multi-level nature of a DFN. DFN models are used to simulate flow and transport through low-permeability fractured media in the subsurface by explicitly representing fractures as discrete entities. The governing equations for flow and transport are numerically integrated on computational meshes generated on the interconnected fracture networks. Modern high-fidelity DFN simulations require high-performance computing on multiple processors where performance and scalability depends partially on obtaining a high-quality partition of the mesh to balance work-loads and minimize communication across all processors. The discrete structure of a DFN naturally lends itself to various graph representations, which can be thought of as coarse-scale representations of the computational mesh. Using this concept, we develop two applications of the multilevel graph partitioning algorithm to partition the mesh of a DFN. In the first, we project a partition of the graph based on the DFN topology onto the mesh of the DFN and in the second, this DFN-based projection is used as the initial condition for further partitioning refinement of the mesh. We compare the performance of these methods with standard multi-level graph partitioning using graph-based metrics (cut, imbalance, partitioning time), computational-based metrics (FLOPS, iterations, solver time), and total run time. The DFN-based and the mesh-based partitioning methods are comparable in terms of the graph-based metrics, but the time required to obtain the partition is several orders of magnitude faster using the DFN-based partitions. The computation-based metrics show comparable performance between both methods so, in combination, the DFN-based partitions are several orders of magnitude faster than the mesh-based partition. Moreover, the method which uses the DFN-partition solution as the initial condition of the mesh partition provided cut and imbalance values that were close to the mesh-based partition but in a fraction of the time. In turn, this hybrid method outperformed both of the other methods in terms of the total run time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Change history

References

  • Aldrich G, Hyman JD, Karra S, Gable CW, Makedonska N, Viswanathan H, Woodring J, Hamann B (2017) Analysis and visualization of discrete fracture networks using a flow topology graph. IEEE Trans Vis Comput Graph 23(8):1896–1909

    Google Scholar 

  • Andresen CA, Hansen A, Le Goc R, Davy P, Hope SM (2013) Topology of fracture networks. Front Phys 1:Art-7

  • Balay S, Abhyankar S, Adams M, Brown J, Brune P, Buschelman K, Dalcin L, Eijkhout V, Gropp W, Kaushik D et al (2017) PETSC users manual revision 3.8. Tech. rep., Argonne National Lab. (ANL), Argonne

  • Berre I, Doster F, Keilegavlen E (2018) Flow in fractured porous media: a review of conceptual models and discretization approaches. Transp Porous Media. https://doi.org/10.1007/s11242-018-1171-6

    Article  Google Scholar 

  • Berrone S, Pieraccini S, Scialo S (2013) A PDE-constrained optimization formulation for discrete fracture network flows. SIAM J Sci Comput 35(2):B487–B510

    Google Scholar 

  • Berrone S, Pieraccini S, Scialò S, Vicini F (2015) A parallel solver for large scale DFN flow simulations. SIAM J Sci Comput 37(3):C285–C306

    Google Scholar 

  • Bonnet E, Bour O, Odling NE, Davy P, Main I, Cowie P, Berkowitz B (2001) Scaling of fracture systems in geological media. Rev Geophys 39(3):347–383

    Google Scholar 

  • Brandt A, Ron D (2003) Multigrid solvers and multilevel optimization strategies. In: Cong J, Shinnerl JR (eds) Multilevel optimization in VLSICAD. Springer, Berlin, pp 1–69

  • Bui TN, Jones C (1993) A heuristic for reducing fill-in in sparse matrix factorization. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Tech. rep

  • Buluç A, Meyerhenke H, Safro I, Sanders P, Schulz C (2016) Recent advances in graph partitioning. In: Kliemann L, Sanders P (eds) Algorithm engineering. Springer, Berlin, pp 117–158

  • Cacas MC, Ledoux E, De Marsily G, Barbreau A, Calmels P, Gaillard B, Margritta R (1990) Modeling fracture flow with a stochastic discrete fracture network: calibration and validation: 2. The transport model. Water Resour Res 26(3):491–500

    Google Scholar 

  • Cheng CK, Wei YC (1991) An improved two-way partitioning algorithm with stable performance (VLSI). IEEE Trans Comput Aided Des Integr Circuits Syst 10(12):1502–1511

    Google Scholar 

  • Chevalier C, Safro I (2009) Comparison of coarsening schemes for multilevel graph partitioning. In: Stützle T (ed) International conference on learning and intelligent optimization. Springer, Berlin, pp 191–205

  • Davy P, Le Goc R, Darcel C (2013) A model of fracture nucleation, growth and arrest, and consequences for fracture density and scaling. J Geophys Res Solid Earth 118(4):1393–1407

    Google Scholar 

  • Dershowitz W, Fidelibus C (1999) Derivation of equivalent pipe network analogues for three-dimensional discrete fracture networks by the boundary element method. Water Resour Res 35(9):2685–2691

    Google Scholar 

  • de Dreuzy JR, Darcel C, Davy P, Bour O (2004) Influence of spatial correlation of fracture centers on the permeability of two-dimensional fracture networks following a power law length distribution. Water Resour Res 40(1):1–11

  • de Dreuzy JR, Méheust Y, Pichot G (2012) Influence of fracture scale heterogeneity on the flow properties of three-dimensional discrete fracture networks. J Geophys Res Solid Earth 117(B11):1–21

  • Erhel J, de Dreuzy JR, Poirriez B (2009) Flow simulation in three-dimensional discrete fracture networks. SIAM J Sci Comput 31(4):2688–2705

    Google Scholar 

  • Fiduccia CM, Mattheyses RM (1988) A linear-time heuristic for improving network partitions. In: Newton AR (ed) Papers on twenty-five years of electronic design automation. ACM, New York, pp 241–247

  • Frampton A, Cvetkovic V (2010) Inference of field-scale fracture transmissivities in crystalline rock using flow log measurements. Water Resour Res 46(11):1–17

  • Garbers J, Promel HJ, Steger A (1990) Finding clusters in VLSI circuits. In: 1990 IEEE International conference on computer-aided design. IEEE, pp 520–523

  • Garey MR, Johnson DS, Stockmeyer L (1974) Some simplified NP-complete problems. In: Proceedings of the sixth annual ACM symposium on theory of computing. ACM, pp 47–63

  • Gerke H, Van Genuchten MT (1993) A dual-porosity model for simulating the preferential movement of water and solutes in structured porous media. Water Resour Res 29(2):305–319

    Google Scholar 

  • Hadgu T, Karra S, Kalinina E, Makedonska N, Hyman JD, Klise K, Viswanathan HS, Wang Y (2017) A comparative study of discrete fracture network and equivalent continuum models for simulating flow and transport in the far field of a hypothetical nuclear waste repository in crystalline host rock. J Hydrol 553:59–70

    Google Scholar 

  • Hagen L, Kahng AB (1992) A new approach to effective circuit clustering. In: Proceedings of the 1992 IEEE/ACM international conference on computer-aided design. IEEE Computer Society Press, pp 422–427

  • Hager WW, Hungerford JT, Safro I (2018) A multilevel bilinear programming algorithm for the vertex separator problem. Comput Optim Appl 69(1):189–223

    Google Scholar 

  • Hendrickson B, Leland RW (1995) A multi-level algorithm for partitioning graphs. SC 95(28):1–14

    Google Scholar 

  • Hope SM, Davy P, Maillot J, Le Goc R, Hansen A (2015) Topological impact of constrained fracture growth. Front Phys 3:75

    Google Scholar 

  • Hu Y, Scott JA (2001) A multilevel algorithm for wavefront reduction. SIAM J Sci Comput 23(4):1352–1375

    Google Scholar 

  • Huseby O, Thovert J, Adler P (1997) Geometry and topology of fracture systems. J Phys A Math Gen 30(5):1415

    Google Scholar 

  • Hyafil L, Rivest RL (1973) Graph partitioning and constructing optimal decision trees are polynomial complete problems. IRIA, Laboratoire de Recherche en Informatique et Automatique, Paris

    Google Scholar 

  • Hyman JD (2020) Flow channeling in fracture networks: characterizing the effect of density on preferential flow path formation. Water Resour Res 56(9):e2020WR027986

    Google Scholar 

  • Hyman JD, Jiménez-Martínez J (2018) Dispersion and mixing in three-dimensional discrete fracture networks: nonlinear interplay between structural and hydraulic heterogeneity. Water Resour Res 54(5):3243–3258. https://doi.org/10.1029/2018WR022585

    Article  Google Scholar 

  • Hyman JD, Gable CW, Painter SL, Makedonska N (2014) Conforming Delaunay triangulation of stochastically generated three dimensional discrete fracture networks: a feature rejection algorithm for meshing strategy. SIAM J Sci Comput 36(4):A1871–A1894

    Google Scholar 

  • Hyman JD, Karra S, Makedonska N, Gable CW, Painter SL, Viswanathan HS (2015) dfnWorks: a discrete fracture network framework for modeling subsurface flow and transport. Comput Geosci 84:10–19

    Google Scholar 

  • Hyman J, Jiménez-Martínez J, Viswanathan H, Carey J, Porter M, Rougier E, Karra S, Kang Q, Frash L, Chen L et al (2016) Understanding hydraulic fracturing: a multi-scale problem. Philos Trans R Soc A 374(2078):20150426

    Google Scholar 

  • Hyman JD, Hagberg A, Srinivasan G, Mohd-Yusof J, Viswanathan H (2017) Predictions of first passage times in sparse discrete fracture networks using graph-based reductions. Phys Rev E 96:013304. https://doi.org/10.1103/PhysRevE.96.013304

    Article  Google Scholar 

  • Hyman JD, Hagberg A, Osthus D, Srinivasan S, Viswanathan H, Srinivasan G (2018) Identifying backbones in three-dimensional discrete fracture networks: a bipartite graph-based approach. Multiscale Model Simul 16(4):1948–1968

    Google Scholar 

  • Hyman JD, Dentz M, Hagberg A, Kang P (2019a) Emergence of stable laws for first passage times in three-dimensional random fracture networks. Phys Rev Lett 123(24):248501

    Google Scholar 

  • Hyman JD, Dentz M, Hagberg A, Kang P (2019b) Linking structural and transport properties in three-dimensional fracture networks. J Geophys Res Solid Earth 124:1185–1204

    Google Scholar 

  • Hyman JD, Jimenez-Martinez J, Gable CW, Stauffer PH, Pawar RJ (2020) Characterizing the impact of fractured caprock heterogeneity on supercritical CO\(_2\) injection. Transp Porous Media 131(3):935–955

    Google Scholar 

  • Jenkins C, Chadwick A, Hovorka SD (2015) The state of the art in monitoring and verification—10 years on. Int J Greenh Gas Control 40:312–349

    Google Scholar 

  • Joyce S, Hartley L, Applegate D, Hoek J, Jackson P (2014) Multi-scale groundwater flow modeling during temperate climate conditions for the safety assessment of the proposed high-level nuclear waste repository site at Forsmark, Sweden. Hydrogeol J 22(6):1233–1249

    Google Scholar 

  • Karra S, Makedonska N, Viswanathan H, Painter S, Hyman J (2015) Effect of advective flow in fractures and matrix diffusion on natural gas production. Water Resour Res 51(10):8646–8657

    Google Scholar 

  • Karypis G, Kumar V (1998) A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J Sci Comput 20(1):359–392

    Google Scholar 

  • Kernighan BW, Lin S (1970) An efficient heuristic procedure for partitioning graphs. Bell Syst Tech J 49(2):291–307

    Google Scholar 

  • Kueper BH, McWhorter DB (1991) The behavior of dense, nonaqueous phase liquids in fractured clay and rock. Ground Water 29(5):716–728

    Google Scholar 

  • Kumar V, Grama A, Gupta A, Karypis G (1994) Introduction to parallel computing: design and analysis of algorithms, vol 400. Benjamin/Cummings, Redwood City

    Google Scholar 

  • Lichtner P, Karra S (2014) Modeling multiscale-multiphase-multicomponent reactive flows in porous media: application to CO\(_2\) sequestration and enhanced geothermal energy using PFLOTRAN. In: Al-Khoury R, Bundschuh J (eds) Computational models for CO\(_2\) geo-sequestration and compressed air energy storage. CRC Press, Boca Raton, pp 81–136

    Google Scholar 

  • Lichtner P, Hammond G, Lu C, Karra S, Bisht G, Andre B, Mills R, Kumar J (2015) PFLOTRAN user manual: a massively parallel reactive flow and transport model for describing surface and subsurface processes. Tech. rep., (Report No.: LA-UR-15-20403) Los Alamos National Laboratory

  • Long J, Remer J, Wilson C, Witherspoon P (1982) Porous media equivalents for networks of discontinuous fractures. Water Resour Res 18(3):645–658

    Google Scholar 

  • Maillot J, Davy P, Le Goc R, Darcel C, De Dreuzy JR (2016) Connectivity, permeability, and channeling in randomly distributed and kinematically defined discrete fracture network models. Water Resour Res 52(11):8526–8545

    Google Scholar 

  • Makedonska N, Hyman JD, Karra S, Painter SL, Gable CWW, Viswanathan HS (2016) Evaluating the effect of internal aperture variability on transport in kilometer scale discrete fracture networks. Adv Water Resour 94:486–497

    Google Scholar 

  • Makedonska N, Hyman JD, Kwicklis E, Birdsell K (2018) Discrete fracture network modeling and simulation of subsurface transport for the Topopah Spring aquifer at Pahute Mesa. In: 52nd US rock mechanics/geomechanics symposium. American Rock Mechanics Association

  • Mansour N, Ponnusamy R, Choudhary A, Fox GC (1993) Graph contraction for physical optimization methods: a quality-cost tradeoff for mapping data on parallel computers. In: Proceedings of the 7th international conference on supercomputing. ACM, pp 1–10

  • Meyerhenke H, Monien B, Sauerwald T (2009) A new diffusion-based multilevel algorithm for computing graph partitions. J Parallel Distrib Comput 69(9):750–761

    Google Scholar 

  • Middleton R, Carey J, Currier R, Hyman J, Kang Q, Karra S, Jiménez-Martínez J, Porter M, Viswanathan H (2015) Shale gas and non-aqueous fracturing fluids: opportunities and challenges for supercritical CO\(_{2}\). Appl Energy 147:500–509

    Google Scholar 

  • Murphy M, Mount DM, Gable CW (2001) A point-placement strategy for conforming Delaunay tetrahedralization. Int J Comput Geom Appl 11(06):669–682

    Google Scholar 

  • Mustapha H, Mustapha K (2007) A new approach to simulating flow in discrete fracture networks with an optimized mesh. SIAM J Sci Comput 29:1439

    Google Scholar 

  • Mustapha H, Dimitrakopoulos R, Graf T, Firoozabadi A (2011) An efficient method for discretizing 3D fractured media for subsurface flow and transport simulations. Int J Numer Methods Fluids 67(5):651–670

    Google Scholar 

  • National Research Council (1996) Rock fractures and fluid flow: contemporary understanding and applications. National Academy Press

  • Neuman S (2005) Trends, prospects and challenges in quantifying flow and transport through fractured rocks. Hydrogeol J 13(1):124–147

    Google Scholar 

  • Neuman SP, Depner JS (1988) Use of variable-scale pressure test data to estimate the log hydraulic conductivity covariance and dispersivity of fractured granites near oracle, arizona. J Hydrol 102(1–4):475–501

    Google Scholar 

  • Nordqvist AW, Tsang YW, Tsang CF, Dverstorp B, Andersson J (1992) A variable aperture fracture network model for flow and transport in fractured rocks. Water Resour Res 28(6):1703–1713. https://doi.org/10.1029/92WR00216

    Article  Google Scholar 

  • Osthus D, Hyman JD, Karra S, Panda N, Srinivasan G (2020) A probabilistic clustering approach for identifying primary subnetworks of discrete fracture networks with quantified uncertainty. SIAM/ASA J Uncertain Quantif 8(2):573–600

    Google Scholar 

  • Pichot G, Erhel J, de Dreuzy JR (2010) A mixed hybrid mortar method for solving flow in discrete fracture networks. Appl Anal 89(10):1629–1643

    Google Scholar 

  • Pichot G, Erhel J, de Dreuzy JR (2012) A generalized mixed hybrid mortar method for solving flow in stochastic discrete fracture networks. SIAM J Sci Comput 34(1):B86–B105

    Google Scholar 

  • Ron D, Safro I, Brandt A (2011) Relaxation-based coarsening and multiscale graph organization. Multiscale Model Simul 9(1):407–423

    Google Scholar 

  • Safro I, Ron D, Brandt A (2006) Graph minimum linear arrangement by multilevel weighted edge contractions. J Algorithm 60(1):24–41

    Google Scholar 

  • Safro I, Ron D, Brandt A (2009) Multilevel algorithms for linear ordering problems. J Exp Algorithmics (JEA) 13:4

    Google Scholar 

  • Safro I, Sanders P, Schulz C (2015) Advanced coarsening schemes for graph partitioning. J Exp Algorithmics (JEA) 19:2–2

    Google Scholar 

  • Sanders P, Schulz C (2011) Engineering multilevel graph partitioning algorithms. In: European symposium on algorithms. Springer, Berlin, pp 469–480

  • Sanders P, Schulz C (2013) Think locally, act globally: highly balanced graph partitioning. In: International symposium on experimental algorithms. Springer, Berlin, pp 164–175

  • Srinivasan G, Hyman JD, Osthus DA, Moore BA, O’Malley D, Karra S, Rougier E, Hagberg AA, Hunter A, Viswanathan HS (2018) Quantifying topological uncertainty in fractured systems using graph theory and machine learning. Sci Rep 8(1):11665

    Google Scholar 

  • Sweeney MR, Gable CW, Karra S, Stauffer PH, Pawar RJ, Hyman JD (2019) Upscaled discrete fracture matrix model (UDFM): an octree-refined continuum representation of fractured porous media. Comput Geosci 24:1–18

    Google Scholar 

  • Tsang Y, Tsang C, Hale F, Dverstorp B (1996) Tracer transport in a stochastic continuum model of fractured media. Water Resour Res 32(10):3077–3092

    Google Scholar 

  • Valera M, Guo Z, Kelly P, Matz S, Cantu VA, Percus AG, Hyman JD, Srinivasan G, Viswanathan HS (2018) Machine learning for graph-based representations of three-dimensional discrete fracture networks. Comput Geosci. https://doi.org/10.1007/s10596-018-9720-1

    Article  Google Scholar 

  • VanderKwaak J, Sudicky E (1996) Dissolution of non-aqueous-phase liquids and aqueous-phase contaminant transport in discretely-fractured porous media. J Contam Hydrol 23(1–2):45–68

    Google Scholar 

  • Viswanathan HS, Hyman J, Karra S, O’Malley D, Srinivasan S, Hagberg A, Srinivasan G (2018) Advancing graph-based algorithms for predicting flow and transport in fractured rock. Water Resour Res 54:6085–6099

    Google Scholar 

  • Walshaw C, Cross M (2000) Mesh partitioning: a multilevel balancing and refinement algorithm. SIAM J Sci Comput 22(1):63–80

    Google Scholar 

  • Walshaw C, Cross M, McManus K (2000) Multiphase mesh partitioning. Appl Math Model 25(2):123–140

    Google Scholar 

  • Zimmerman RW, Chen G, Hadgu T, Bodvarsson GS (1993) A numerical dual-porosity model with semianalytical treatment of fracture/matrix flow. Water Resour Res 29(7):2127–2137

    Google Scholar 

Download references

Acknowledgements

This work was funded by the Department of Energy at Los Alamos National Laboratory through the Laboratory-Directed Research and Development Program LANL LDRD Grant #20170103DR. J.D.H. and M.R.S. acknowledges support from the LANL LDRD program office Grant Number #20180621ECR. Los Alamos National Laboratory is operated by Triad National Security, L.L.C., for the National Nuclear Security Administration of the U.S. Department of Energy (Contract No. 89233218CNA000001).

Author information

Authors and Affiliations

  1. Fujitsu Laboratories of America, Inc., Sunnyvale, CA, 94085, USA

    Hayato Ushijima-Mwesigwa

  2. Computational Earth Science (EES-16), Earth and Environmental Sciences Division, Los Alamos National Laboratory, Los Alamos, NM, 87505, USA

    Jeffrey D. Hyman, Satish Karra, Carl W. Gable & Matthew R. Sweeney

  3. Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos, NM, USA

    Aric Hagberg

  4. Computer and Information Sciences, University of Delaware, Newark, DE, USA

    Ilya Safro

  5. Verification and Analysis (XCP-8), X Computational Physics, Los Alamos National Laboratory, Los Alamos, NM, USA

    Gowri Srinivasan

Authors
  1. Hayato Ushijima-Mwesigwa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey D. Hyman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aric Hagberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ilya Safro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Satish Karra
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carl W. Gable
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Matthew R. Sweeney
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Gowri Srinivasan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeffrey D. Hyman.

Additional information

The original online version of this article was revised: The affiliation of author Satish Karra was wrong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ushijima-Mwesigwa, H., Hyman, J.D., Hagberg, A. et al. Multilevel Graph Partitioning for Three-Dimensional Discrete Fracture Network Flow Simulations. Math Geosci 53, 1699–1724 (2021). https://doi.org/10.1007/s11004-021-09944-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11004-021-09944-y

Keywords

Search

Navigation