Efficient use of sparsity by direct solvers applied to 3D controlled-source EM problems
- 44 Downloads
Controlled-source electromagnetic (CSEM) surveying becomes a widespread method for oil and gas exploration, which requires fast and efficient software for inverting large-scale EM datasets. In this context, one often needs to solve sparse systems of linear equations with a large number of sparse right-hand sides, each corresponding to a given transmitter position. Sparse direct solvers are very attractive for these problems, especially when combined with low-rank approximations which significantly reduce the complexity and the cost of the factorization. In the case of thousands of right-hand sides, the time spent in the sparse triangular solve tends to dominate the total simulation time, and here we propose several approaches to reduce it. A significant reduction is demonstrated for marine CSEM application by utilizing the sparsity of the right-hand sides (RHS) and of the solutions that results from the geometry of the problem. Large gains are achieved by restricting computations at the forward substitution stage to exploit the fact that the RHS matrix might have empty rows (vertical sparsity) and/or empty blocks of columns within a non-empty row (horizontal sparsity). We also adapt the parallel algorithms that were designed for the factorization to solve-oriented algorithms and describe performance optimizations particularly relevant for the very large numbers of right-hand sides of the CSEM application. We show that both the operation count and the elapsed time for the solution phase can be significantly reduced. The total time of CSEM simulation can be divided by approximately a factor of 3 on all the matrices from our set (from 3 to 30 million unknowns, and from 4 to 12 thousands RHSs).
KeywordsControlled-source electromagnetics (CSEM) Marine electromagnetics Numerical modeling Direct solver Multiple sparse right-hand sides
Mathematics Subject Classification (2010)15A06 15A23 65F05 65F50 65Y05 65Z05 68U20 68W10 78A25 86-04 86-08 86A20 86A22
Unable to display preview. Download preview PDF.
This work was partially supported by the MUMPS consortium and by LABEX MILYON (ANR-10-LABX-0070) of Université de Lyon, within the program “Investissements d’Avenir” (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR).
- 7.Amestoy, P.R., L’Excellent, J.Y., Rouet, F.H., Sid-Lakhdar, W.M.: Modeling 1D distributed-memory dense kernels for an asynchronous multifrontal sparse solver. In: High Performance Computing for Computational Science, VECPAR 2014 - 11th International Conference, Eugene, Oregon, USA, June 30 - July 3, 2014, Revised Selected Papers, pp. 156–169 (2014)Google Scholar
- 9.Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK users’ guide. SIAM Press (1997)Google Scholar
- 19.Hanssen, P., Nguyen, A.K., Fogelin, L.T.T., Jensen, H.R., Skaro, M., Mittet, R., Rosenquist, M., Suilleabhain, L.O., van der Sman, P.: The next generation offshore CSEM acquisition system, pp. 1194–1198. Society of Exploration Geophysicists. https://library.seg.org/doi/abs/10.1190/segam2017-17725809.1 (2017)
- 20.Hiner, M., Martinez, Y., Sun, S.: Delineating salt bodies with 3D CSEM technology. In: Salt Challenges in Hydrocarbon Exploration, SEG Annual Meeting Post-convention Workshop. New Orleans (2015)Google Scholar
- 22.Mary, T.: Block low-rank multifrontal solvers: complexity, performance, and scalability. PhD thesis, Université de Toulouse (2017)Google Scholar
- 23.Nguyen, A.K., Nordskag, J.I., Wiik, T., Bjorke, A.K., Boman, L., Pedersen, O.M., Ribaudo, J., Mittet, R.: Comparing large-scale 3D Gauss-Newton and BFGS CSEM inversions, pp. 872–877. Society of Exploration Geophysicists (2016). https://doi.org/10.1190/segam2016-13858633.1
- 26.Rouet, F.H.: Memory and performance issues in parallel multifrontal factorizations and triangular solutions with sparse right-hand sides. PhD thesis, Institut National Polytechnique de Toulouse (2012)Google Scholar
- 28.Slavova, Tz.: Parallel triangular solution in the out-of-core multifrontal approach for solving large sparse linear systems. Ph.D. dissertation, Institut National Polytechnique de Toulouse (2009). Available as CERFACS Report TH/PA/09/59Google Scholar
- 31.Zach, J., Bjorke, A., Storen, T., Maao, F.: 3D inversion of marine CSEM data using a fast finite-difference time-domain forward code and approximate Hessian-based optimization. In: SEG Technical Program Expanded Abstracts 2008, pp. 614–618 (2008). https://library.seg.org/doi/abs/10.1190/1.3063726