Applied Mathematics in EM Studies with Special Emphasis on an Uncertainty Quantification and 3-D Integral Equation Modelling

Abstract

Despite impressive progress in the development and application of electromagnetic (EM) deterministic inverse schemes to map the 3-D distribution of electrical conductivity within the Earth, there is one question which remains poorly addressed—uncertainty quantification of the recovered conductivity models. Apparently, only an inversion based on a statistical approach provides a systematic framework to quantify such uncertainties. The Metropolis–Hastings (M–H) algorithm is the most popular technique for sampling the posterior probability distribution that describes the solution of the statistical inverse problem. However, all statistical inverse schemes require an enormous amount of forward simulations and thus appear to be extremely demanding computationally, if not prohibitive, if a 3-D set up is invoked. This urges development of fast and scalable 3-D modelling codes which can run large-scale 3-D models of practical interest for fractions of a second on high-performance multi-core platforms. But, even with these codes, the challenge for M–H methods is to construct proposal functions that simultaneously provide a good approximation of the target density function while being inexpensive to be sampled. In this paper we address both of these issues. First we introduce a variant of the M–H method which uses information about the local gradient and Hessian of the penalty function. This, in particular, allows us to exploit adjoint-based machinery that has been instrumental for the fast solution of deterministic inverse problems. We explain why this modification of M–H significantly accelerates sampling of the posterior probability distribution. In addition we show how Hessian handling (inverse, square root) can be made practicable by a low-rank approximation using the Lanczos algorithm. Ultimately we discuss uncertainty analysis based on stochastic inversion results. In addition, we demonstrate how this analysis can be performed within a deterministic approach. In the second part, we summarize modern trends in the development of efficient 3-D EM forward modelling schemes with special emphasis on recent advances in the integral equation approach.

Appendices

Appendix 1: The Misfit Renormalization

Quadratic approximation [${ - {\textstyle \frac{1}{2}} \left( { 2{\mathbf{g}}_{k}^T{\mathbf{s}}+ {\mathbf{s}}^T{\mathtt{H}}_{{{k}}}{\mathbf{s}}}\right) }$, cf. Eq. (26)] of the misfit ${\beta _d}({\mathbf{m}})$ might not work well in the desired vicinity of the current conductivity model ${\mathbf{m}}^{(k)}$. In this case we suggest to use the following renormalization of the misfit

$$\begin{aligned} {\beta _d}\,\mapsto \,\frac{{\beta _d}}{\kappa } \end{aligned}$$

(72)

which tells us that the larger local uncertainty ellipsoid (explained later in this section) has more chances to catch the true minimum point as illustrated in Fig. 8.

Renormalization (72) leads to a modification in Eqs. (28), (32), (33), (58) and (37) as follows

$$\frac{{\mathbf{g}}_{k}^T}{\kappa }{\mathbf{s}}+ {\textstyle\frac{1}{2}}{\mathbf{s}}^T \frac{{\mathtt{H}}_{{{k}}}}{\kappa}{\mathbf{s}} = {\textstyle \frac{1}{2}} \left| {{\mathbf{F}}_{k}\frac{\mathbf{s}}{\sqrt{\kappa }} +{\mathbf{F}}_{k}^{-T}\frac{{\mathbf{g}}_{k}}{\sqrt{\kappa }}}\right| ^2 - {\textstyle \frac{1}{2}} \left| {{\mathbf{F}}_{k}^{-T}\frac{{\mathbf{g}}_{k}}{\sqrt{\kappa }}}\right| ^2 ,$$

(73)

$$\begin{aligned} {\mathbf{g}}_{k}'& = -{\mathbf{F}}_{k}^{-T}\frac{\mathbf{g}_{k}}{\sqrt{\kappa }} ,\end{aligned}$$

(74)

$${\mathbf{y}} = {\mathbf{F}}_{k}\frac{\mathbf{s}}{\sqrt{\kappa }} ,$$

(75)

$$\begin{aligned} {\mathbf{s}}& = \sqrt{\kappa } \, {\mathbf{F}}_{k}^{-1}{\mathbf{y}}\end{aligned}$$

(76)

$$\begin{aligned} {\mathbf{s}}& = - {\mathtt{H}}_{{{k}}}^{-1} {\mathbf{g}}_{k}+ \sqrt{\kappa } \, {\mathbf{F}}_{k}^{-1} {\mathcal {N}}_{N_{\mathcal {M}}}({\mathbf{0}},1) , \end{aligned}$$

(77)

and to corresponding equations for $\tilde{\mathbf{F}}$ (recall that $\tilde{\mathbf{F}}$ is a low-rank approximation for $\mathbf{F}$, see Sect. 5).

We see that the renormalization leads to the similarity transformation of the uncertainty ellipsoid, but it does not change the Newtonian step $\left( {-{\mathtt{H}}_{{{k}}}^{-1}{\mathbf{g}}_{k}}\right)$. We call here as uncertainty ellipsoid the following set

$$\begin{aligned} {\mathcal {U}} = - {\mathtt{H}}_{{{k}}}^{-1} {\mathbf{g}}_{k}+ \sqrt{\kappa } {\mathbf{F}}_{k}^{-1}B({\mathbf{0}},1) , \end{aligned}$$

(78)

where $B(\mathbf{0},1)$ is a unit sphere in ${\mathbb{R}}^{N_{\mathcal {M}}}$ centered at the origin.

Finally, the renormalization factor $\kappa$ might be one of the parameters that are needed to keep the acceptance rate (percentage of the accepted models in the sample) to be in the desired interval; e.g., in the M–H method it is generally accepted that the rate should be between 20 and 70 %.

Fig. 8

Illustration of necessity to introduce renormalization factor $\kappa$, see details in the text

Appendix 2: Summary of Formulae to Calculate Data Misfit Gradient

This and follow-up appendices summarize the results presented in Pankratov and Kuvshinov (2015). But before we proceed with final formulae we introduce definitions to be used.

1.1 Green’s Operators

Let us define an operator ${\mathbf{G}}^{\cdot \cdot }{}$ in 3-D space ${\mathbb{R}}^3_{}$

$$\begin{aligned} \left( \begin{array}{c}{\mathbf{E}}\\ {\mathbf{H}}\end{array}\right) = {\mathbf{G}}^{\cdot \cdot }{\left( \begin{array}{c}{\mathbf{j}^{{\rm imp}}}\\ {\mathbf{h}^{{\rm imp}}}\end{array}\right) } \Leftrightarrow \left\{ { \begin{array}{l} \nabla \times \mathbf{H}= \sigma \mathbf{E}+ \mathbf{j}^{{\rm imp}}, \\ \\ \nabla \times \mathbf{E}= i \omega \mu \mathbf{H}+ \mathbf{h}^{\rm imp}, \\ \\ \mathbf{E}(\mathbf{r}),\mathbf{H}(\mathbf{r})\longrightarrow 0\ \; \text{ as } \; |\mathbf{r}|\longrightarrow \infty , \end{array} }\right. \end{aligned}$$

(79)

where $\mathbf{E}$ and $\mathbf{H}$ are electric and magnetic fields, $\mathbf{j}^{\rm imp}$ and $\mathbf{h}^{\rm imp}$ are impressed (extraneous) electric and magnetic sources, respectively, $\mathbf{r}\in {\mathbb{R}}^3_{}$ is a position vector, $i=\sqrt{-1}, {\omega }=2\pi /{\text {Period}}$ is an angular frequency, $\sigma \left( {\mathbf{r}}\right)$ and $\mu \left( {\mathbf{r}}\right)$ are electric conductivity and magnetic permeability distributions in an Earth’s model, respectively. In this appendix we assume that $\sigma \left( {\mathbf{r}}\right)$ is a real-valued function. One can readily generalize the concept for complex-valued conductivity. The corresponding formulae are provided in the last appendix of Pankratov and Kuvshinov (2015). All fields, $\mathbf{E}, \mathbf{H}, \mathbf{j}^{\rm imp}, \mathbf{h}^{\rm imp}$, are complex-valued functions of ${\omega }$ and $\mathbf{r}$. In addition the fields $\mathbf{E}$ and $\mathbf{H}$ depend on $\sigma$ and $\mu$. We study the derivatives with respect to $\sigma$ only. Green’s operator $\mathbf{G}^{\cdot \cdot }$ depends on functional arguments $\mathbf{j}^{\rm imp}$ and $\mathbf{h}^{\rm imp}$. Hereinafter the dependence of Green’s operator on $\sigma , \mathbf{r}$, and ${\omega }$ is omitted but implied. Time dependence of fields is accounted for by $e^{-i{\omega }t}$, which reads, for example, for the electric field as $\breve{\mathbf{E}}(\mathbf{r},t)=\int \mathbf{E}(\mathbf{r},{\omega })e^{-i{\omega }t}\hbox {d}\omega$. At this stage we do not specify the coordinate system in ${\mathbb{R}}^3_{}$; this means that $\mathbf{r}$ can be, for example, a triplet of Cartesian coordinates, (x, y, z), or a triplet of spherical coordinates, $(r, \theta , \phi )$. As far as the column in the left-hand side (LHS) of Eq. (79) contains two fields, $\mathbf{E}$ and $\mathbf{H}$, operator $\mathbf{G}^{\cdot \cdot }$ can be represented via operators $\mathbf{G}^{e\cdot }, \mathbf{G}^{h\cdot } \, \mathbf{G}^{ee}, \mathbf{G}^{eh}, \mathbf{G}^{he}, \mathbf{G}^{hh}$ as follows

$$\begin{aligned} \mathbf{G}^{\cdot \cdot }= \left( \begin{array}{c}{\mathbf{G}^{e\cdot }}\\ {\mathbf{G}^{h\cdot }}\end{array}\right) = \left( \begin{array}{cc}{\mathbf{G}^{ee}}&{}{\mathbf{G}^{eh}}\\ {\mathbf{G}^{he}}&{}{\mathbf{G}^{hh}}\end{array}\right) , \quad \mathbf{G}^{e\cdot }= \left( {\mathbf{G}^{ee}},\,{\mathbf{G}^{eh}}\right) , \quad {\mathbf{G}^{h\cdot }} = \left( {\mathbf{G}^{he}},\,{\mathbf{G}^{hh}}\right) , \end{aligned}$$

(80)

where operators $\mathbf{G}^{e\cdot }$ and $\mathbf{G}^{h\cdot }$ are electric and magnetic components of $\mathbf{G}^{\cdot \cdot }$, operator $\mathbf{G}^{ee}$ is a restriction of $\mathbf{G}^{e\cdot }$ to electric sources etc.

Let us introduce an electromagnetic field, $\mathbf{u}$, as

$$\begin{aligned} \mathbf{u}(\mathbf{r},{\omega })=\left( \begin{array}{c}{\mathbf{E}(\mathbf{r},{\omega })}\\ {\mathbf{H}(\mathbf{r},{\omega })}\end{array}\right) , \end{aligned}$$

(81)

which is a complex-valued six-dimensional (6-D) vector. Let us denote the space of such vectors as ${\mathcal {U}}\cong {\mathbb{C}}^6$. Note that once we have chosen coordinates in 3-D space ${\mathbb{R}}^3_{}$ with the following basis

$$\begin{aligned} {\mathbf{e}}_{1}, {\mathbf{e}}_{2}, {\mathbf{e}}_{3}, \end{aligned}$$

(82)

then we naturally and unambiguously have a coordinate system and basis ${\mathbf{e}}'_{1}, \cdots , {\mathbf{e}}'_{6}$ in 6-D complex space ${\mathcal {U}}$

$$\begin{aligned} \mathbf{u}(\mathbf{r},{\omega }) = \sum \limits _{\alpha =1}^{6} u_\alpha {\mathbf{e}}'_{\alpha } , \end{aligned}$$

(83)

saying that ${\mathbf{e}}'_{1}, {\mathbf{e}}'_{2}, {\mathbf{e}}'_{3}$ are ${\mathbf{e}}_{1}, {\mathbf{e}}_{2}, {\mathbf{e}}_{3}$ for electric fields, whereas ${\mathbf{e}}'_{4}, {\mathbf{e}}'_{5}, {\mathbf{e}}'_{6}$ are ${\mathbf{e}}_{1}, {\mathbf{e}}_{2}, {\mathbf{e}}_{3}$ for magnetic fields, respectively.

1.2 Polarizations/Sources

Let

$$\begin{aligned} \left\{ { \mathbf{f}^{\rm imp}_p }\right\} _{{ p\in {\mathcal {P}}}} , \quad {\mathcal {P}}= \left\{ {1,2,\ldots ,N_{\mathcal {P}}}\right\} , \end{aligned}$$

(84)

be a set of linearly independent distributions (in space and frequency) of the impressed sources, $\mathbf{f}^{\rm imp}_p$. For example, in magnetotelluric (MT) studies, $N_{\mathcal {P}}=2$, and $\mathbf{f}^{\rm imp}_1$ and $\mathbf{f}^{\rm imp}_2$ correspond to the plane waves of different orientations. Each $\mathbf{f}^{\rm imp}_p$ produces electric, $\mathbf{E}_p$, and magnetic, $\mathbf{H}_p$, fields that constitute an EM field $\mathbf{u}_p$ that can be written via $\mathbf{G}^{\cdot \cdot }$ operator (104) as

$$\begin{aligned} \mathbf{u}_p= \mathbf{G}^{\cdot \cdot }\left( {\mathbf{f}^{\rm imp}_p}\right) . \end{aligned}$$

(85)

1.3 Inversion Domain and Parameterization

As far as the inversion is usually done numerically, let the inversion domain, $V^{\rm inv}$, be represented as

$$\begin{aligned} V^{\rm inv} = \bigcup \limits _{l=1}^{N_{\mathcal {M}}}V_l , \end{aligned}$$

(86)

where $\left\{ { V_l }\right\} _{{ l\in {\mathcal {M}}}}, \ \ {\mathcal {M}}= \left\{ { 1,\ldots ,N_{\mathcal {M}}}\right\}$, represent a set of elementary volumes $V_l$, and within each volume $V_l$ let the conductivity be a constant $\sigma \left( {\mathbf{r}}\right) =\sigma _l$. We assemble this conductivity distribution in the following vector

$$\begin{aligned} {\varvec{\sigma }}= \left( {\sigma _1},\,{\ldots },\,{\sigma _{N_{\mathcal {M}}}}\right) ^T , \end{aligned}$$

(87)

and introduce model parameterization as

$$\begin{aligned} {\mathbf{m}}= \left( {m_1},\,{\ldots },\,{m_{N_{\mathcal {M}}}}\right) ^T, \ \ m_l= \nu ^{-1}(\sigma _l), \ \ l\in {\mathcal {M}}, \end{aligned}$$

(88)

where function ${\mathbf{m}}= \mathbf{\nu }^{-1}({\varvec{\sigma }})$ can be implemented, for example, to preserve conductivity to be positive. Note that a popular choice is ${\mathbf{m}}=\ln {\varvec{\sigma }}$. We also remark that some volumes $V_l$ might be cells (or combinations of cells) of the 3-D part of the model.

1.4 Observation Sites, Frequencies, Response Functions and Misfit

Let

$$\begin{aligned} \varPhi _g,~ \Delta \varPhi _g, \qquad g\in {\mathcal {G}}= \left\{ { 1,2,\ldots ,N_{{\mathcal {G}}}}\right\} , \end{aligned}$$

(89)

be the experimental responses and their uncertainties, respectively, and let $N_{{\mathcal {G}}}$ be the number of all responses. Let $\mathbf{r}_g$, and ${\omega }_g$ be the spatial location and the frequency, respectively, at which the response $\varPhi _g$ has been obtained.

Let ${\mathcal {S}}$ be a set of observation sites

$$\begin{aligned} {\mathcal {S}}= {\left\{ {\mathbf{r}_g}\,\left| \,{g\in {\mathcal {G}}}\right. \right\} } =\left\{ {{\mathbf{s}}_1,\ldots ,{\mathbf{s}}_{N_{\mathcal {S}}}}\right\} , \end{aligned}$$

(90)

where ${\mathbf{s}}_1,\ldots ,{\mathbf{s}}_{N_{\mathcal {S}}}$ are different observation sites, and ${N_{\mathcal {S}}}$ is the number of sites.

Let ${\varOmega }$ be a set of observation frequencies

$$\begin{aligned} {\varOmega }= {\left\{ {{\omega }_g}\,\left| \,{g\in {\mathcal {G}}}\right. \right\} } =\left\{ {f_1,\ldots ,f_{N_{{\varOmega }}}}\right\} , \end{aligned}$$

(91)

where $f_1,\ldots ,f_{N_{{\varOmega }}}$ are different observation frequencies, and ${N_{{\varOmega }}}$ is the number of frequencies. The definitions (89)–(91) are introduced in this specific way intentionally in order to stress the fact that in practice an actual set of experimental responses to be used for inversion varies with frequency and site.

For each $g\in {\mathcal {G}}$, the predicted response, $\theta _g$, can be written in the following form

$$\begin{aligned} \theta _g({\mathbf{m}})=\varPsi _g\left( {\mathbf{u}_1({\mathbf{m}},\mathbf{r}_g,{\omega }_g),\mathbf{u}_2({\mathbf{m}},\mathbf{r}_g,{\omega }_g),\ldots ,\mathbf{u}_{N_{\mathcal {P}}}({\mathbf{m}},\mathbf{r}_g,{\omega }_g)}\right) . \end{aligned}$$

(92)

Finally the misfit is introduced as

$$\begin{aligned} {\beta _d}({\mathbf{m}}) = \sum \limits _{ { g\in {\mathcal {G}}}} \ \ \Bigg | \frac{ {\theta _g}({\mathbf{m}}) - {\varPhi _g} }{\Delta \varPhi _g} \Bigg |^2 . \end{aligned}$$

(93)

Following Pankratov and Kuvshinov (2015) we write the elements of data misfit gradient as

$$\begin{aligned} \frac{\partial {{{\beta _d}}}}{\partial {{m_l}}} = 2 \nu _l' \mathop {\text {Re}}\nolimits \sum _{ { {\omega }\in {\varOmega }\;p\in \mathcal {P}}} \int \limits _{V_l} \mathbf{E}_p({\omega }) \cdot \mathbf{G}^{e\cdot }\left( { \mathbf{J}^M_{p}({\omega }) }\right) \ \hbox {d}v , \quad l=1,2,\ldots ,N_{\mathcal {M}}, \end{aligned}$$

(94)

where an adjoint source $\mathbf{J}^M_{p}$ is given by

$$\begin{aligned} \mathbf{J}^M_{p}({\omega }) = \sum _{ g:{\omega }_g={\omega }} \frac{(\theta _g-\varPhi _g)^*}{|{\Delta \varPhi _g}|^2} \frac{\partial {{\varPsi _g}}}{\partial {{\mathbf{u}_p}}} \delta _{\mathbf{r}_g} \Biggr |_{ { {\omega }}} , \end{aligned}$$

(95)

where $^*$ stands for complex conjugation. Table 2 summarizes the steps needed to calculate the misfit gradient. From the Eq. (94) it is seen that we need $2N_{\mathcal {P}}{N_{{\varOmega }}}$ forward modellings in total to calculate the data misfit gradient.

Table 2 The steps needed to calculate the gradient of the data misfit ${\beta _d}$

Appendix 3: Summary of Formulae to Calculate the Hessian-Vector Products

We are interested to calculate $\mathop {\text {Hess}}\nolimits _{{\beta _d}}\mathbf{a}_k, k=1,\ldots ,K$, where $\mathbf{a}_k$ we represent as

$$\begin{aligned} \mathbf{a}_k = \sum \limits _{m=1}^{N_{\mathcal {M}}} a_{km}\mathbf{1}_{V_m}(\mathbf r) , \end{aligned}$$

(96)

where ${\displaystyle \mathbf{1}}_{\scriptscriptstyle V_l}$($\mathbf{r}$) is an indicator function given by

$$\begin{aligned} \mathbf{1}_{V_m}(\mathbf{r}) = \left\{ { \begin{array}{c} 1, \; \mathbf{r}\in V_m,\\ 0, \; \mathbf{r}\notin V_m . \end{array} }\right. \end{aligned}$$

(97)

Following Pankratov and Kuvshinov (2015), the l-th element of the Hessian-vector product $\mathop {\text {Hess}}\nolimits _{{\beta _d}}\mathbf{a}_k$ (which is a vector) has a form

$$\begin{aligned} \Big [\mathop {\text {Hess}}\nolimits _{{\beta _d}}\mathbf{a}_k \Big ]_{l} = \mathop {\text {Re}}\nolimits ({\mathcal {B}}^{A}_{kl} + {\mathcal {B}}^{L}_{kl}) ,\quad l=1,2,\ldots {N_{\mathcal {M}}}, \end{aligned}$$

(98)

where ${\mathcal {B}}^{A}_{kl}$ and ${\mathcal {B}}^{L}_{kl}$ are

$$\begin{aligned} {\mathcal {B}}^{A}_{kl}& = 2 \sum \limits _{{\omega }\in {\varOmega }}\sum \limits _{p\in {\mathcal {P}}} \int \limits _{V_l}\left. { \nu _l' {{\mathbf{E}_p}} \cdot \mathbf{G}^{e\cdot }\left( {\mathbf{J}^B_{p}( \mathbf{\nu }' \mathbf{a}_k )}\right) \hbox {d}v }\right| _{{{\omega }}},\end{aligned}$$

(99)

$$\begin{aligned} {\mathcal {B}}^{L}_{kl}& = 2 \sum \limits _{{\omega }\in {\varOmega }} \sum \limits _{p\in {\mathcal {P}}} \int \limits _{V_l} \left. { \nu _l' {{\mathbf{E}_p}} \cdot \mathbf{G}^{e\cdot }\left( {{\mathbf{J}^\varPsi _{p}( \mathbf{\nu }' \mathbf{a}_k )}}\right) \hbox {d}v }\right| _{{{\omega }}}\nonumber \\&\quad + 2 \sum \limits _{{\omega }\in {\varOmega }} \sum \limits _{p\in {\mathcal {P}}} \int \limits _{V_l} \big \{ \big ( \nu _l' \mathbf{G}^{ee}\left( { \mathbf{\nu }' \mathbf{a}_k\, \mathbf{E}_p }\right) + \mathbf{\nu }'' \mathbf{a}_k\, {{\mathbf{E}_p}} \big ) \cdot \mathbf{G}^{e\cdot }\left( {\mathbf{J}^M_{p}}\right) \nonumber \\&\quad + \left. { \nu _l' \mathbf{E}_p \mathbf{G}^{ee}\left( { \mathbf{\nu }' \mathbf{a}_k\, \mathbf{G}^{e\cdot }\left( {\mathbf{J}^M_{p}}\right) }\right) \big \} \hbox {d}v }\right| _{{{\omega }}}, \end{aligned}$$

(100)

where $\mathbf{J}^M_{p}({\omega })$ is defined in Eq. (95) and $\mathbf{G}^{e\cdot }\left( {\mathbf{J}^B_{p}(\mathbf{a}_k,{\omega })}\right)$ and $\mathbf{G}^{e\cdot }\left( {\mathbf{J}^\varPsi _{p}(\mathbf{a}_k,{\omega })}\right)$ are as follows

$$\begin{aligned}&\mathbf{G}^{e\cdot }\left( { \mathbf{J}^B_{p}(\mathbf{a}_k,{\omega }) }\right) =\nonumber \\&\quad \sum \limits _{g:\; {\omega }_g={\omega }} \sum \limits _{q\in {\mathcal {P}}} \left. { \frac{1}{|{\Delta \varPhi _g}|^2} \Bigg \{ \sum \limits _{m=1}^{N_{\mathcal {M}}} \int \limits _{V_m} a_{km}\,{{\mathbf{E}_q}} \cdot \mathbf{G}^{e\cdot }\left( { \frac{\partial {{\varPsi _g}}}{\partial {{\mathbf{u}_q}}} \, \delta _{\mathbf{r}_g} }\right) \Bigg \} ^* \, \mathbf{G}^{e\cdot }\left( { \frac{\partial {{\varPsi _g}}}{\partial {{\mathbf{u}_p}}} \, \delta _{\mathbf{r}_g} }\right) }\right| _{{ { {\omega }}}} ,\end{aligned}$$

(101)

$$\begin{aligned}&\mathbf{G}^{e\cdot }\left( { \mathbf{J}^\varPsi _{p}(\mathbf{a}_k,{\omega }) }\right) = \sum \limits _{g:{\omega }_g={\omega }} \sum \limits _{q\in {\mathcal {P}}} \left. { \frac{(\theta _g-\varPhi _g)^*}{|{\Delta \varPhi _g}|^2} \mathbf{G}^{e\cdot }\left( { \frac{\partial ^2{{\varPsi _g}}}{\partial {{{\mathbf{u}_p}}}\partial {{{\mathbf{u}_q}}}} \, \mathbf{G}^{\cdot e}\left( {\mathbf{a}_k\mathbf{E}_q}\right) \, \delta _{\mathbf{r}_g} }\right) }\right| _{{ { {\omega }}}} . \end{aligned}$$

(102)

We make here three notes.

• Term $\int \limits _{V_l}{{{\mathbf{E}_p}}}\cdot \mathbf{G}^{e\cdot }\left( {{\mathbf{J}^\varPsi _{p}({\displaystyle \mathbf{1}}_{\scriptscriptstyle V_k})}}\right) \hbox {d}v$ vanishes if the response $\varPsi$ is a linear function of EM field $\mathbf{u}$ (e.g., for most of the CSEM methods).

• Term $\int \limits _{V_l}\delta _{lk}\nu _l''{{\mathbf{E}_p}({\omega })}\cdot \mathbf{G}^{e\cdot }\left( {\mathbf{J}^M_{p}({\omega })}\right) \, \hbox {d}v$ vanishes if ${\varvec{\sigma }}={\mathbf{m}}$.

• One can readily generalize the concept for complex-valued conductivity $\sigma$. The corresponding formulae are provided in the last appendix of Pankratov and Kuvshinov (2015).

Table 3 provides a number of forward modellings needed to calculate the Hessian-vector product K times. As seen from the table, a single Hessian-vector product can be calculated for a price of $O({N_{{\varOmega }}}{N_{\mathcal {S}}})$ forward problem runs. Moreover, if such a product is calculated multiple times the price drops down to $2{N_{P} N_{\varOmega }}$ per product. Note also that computation of the Hessian itself is merely calculation of the Hessian-vector product for $K=N_{\mathcal {M}}$ times with respective vectors $\mathbf{a}_k = {\displaystyle \mathbf{1}}_{\scriptscriptstyle V_k}, ~ k=1,\ldots ,N_{\mathcal {M}}$.

Table 3 The steps needed to calculate the Hessian-vector products $\mathop {\text {Hess}}\nolimits _{{\beta _d}}\mathbf{a}_k, k=1,\ldots ,K$

Appendix 4: Contracting Integral Equation in a Nutshell

Let $\sigma (\mathbf{r})$ be a desired 3-D model of complex-valued conductivity, including the real part term $\gamma (\mathbf{r})=\mathop {\text {Re}}\nolimits \sigma (\mathbf{r})>0$ as well as the imaginary part term $\eta (\mathbf{r}) = \mathop {\text {Im}}\nolimits \sigma (\mathbf{r})$ that describes displacement currents and/or induced polarization effects. Let the model be excited by electric source $\mathbf{j}^{\rm imp}$. Let us search for the electric field excited by $\mathbf{j}^{\rm imp}$ in the model $\sigma$. This electric field is the electric field solution of Maxwell’s equations

$$\begin{aligned} \mathbf{E}= \mathbf{G}_{{}}^{ee}(\mathbf{j}^{\rm imp}) \Leftrightarrow \left\{ { \begin{array}{l} \nabla \times \mathbf{H}= \sigma \mathbf{E}+ \mathbf{j}^{\rm imp}, \\ \\ \nabla \times \mathbf{E}= i \omega \mu \mathbf{H}, \\ \\ \mathbf{E}(\mathbf{r}),\mathbf{H}(\mathbf{r})\longrightarrow 0\ \; \text{ as } \; |\mathbf{r}|\longrightarrow \infty . \end{array} }\right. \end{aligned}$$

(103)

Let $\sigma _b(\mathbf{r})$ be any model where we can evaluate Green’s operator $\mathbf{G}_{{\sigma _b}}^{ee}$ that is electric field solution of the following Maxwell’s equations

$$\begin{aligned} \mathbf{E}_b = \mathbf{G}_{{\sigma _b}}^{ee}(\mathbf{j}^{\rm imp}) \Leftrightarrow \left\{ { \begin{array}{l} \nabla \times \mathbf{H}_b = \sigma _b \mathbf{E}_b + \mathbf{j}^{\rm imp}, \\ \\ \nabla \times \mathbf{E}_b = i \omega \mu \mathbf{H}_b, \\ \\ \mathbf{E}_b(\mathbf{r}),\mathbf{H}_b(\mathbf{r})\longrightarrow 0\ \; \text{ as } \; |\mathbf{r}|\longrightarrow \infty . \end{array} }\right. \end{aligned}$$

(104)

We refer to $\sigma _b$ as a reference model, e.g., it could be a background (host) model. In further discussion we assume that $\sigma _b$ describes one-dimensional (1-D) conductivity section, i.e. $\sigma _b \equiv \sigma _b(z)$. With such introduced reference model, an action of operator $\mathbf{G}_{{\sigma _b}}^{ee}$ on the field $\mathbf{j}^{\rm imp}$ is represented by the following convolution integral

$$\begin{aligned} \mathbf{E}_b=\mathbf{G}_{{\sigma _b}}^{ee}(\mathbf{j}^{\rm imp})=\int \limits _{\mathbb {V}} \mathbf{G}_{{\sigma _b}}^{ee}(x-x',y-y',z,z')\, \mathbf{j}^{\rm imp}(x',y',z') \hbox {d}v', \end{aligned}$$

(105)

where ${\mathbb {V}}$ is the volume, occupied by $\mathbf{j}^{\rm imp}$. Note, that analogously we can obtain “reference” magnetic field, $\mathbf{H}_b$, via corresponding Green’s operator $\mathbf{G}^{he}_{\sigma _b}$, namely, $\mathbf{H}_b=\mathbf{G}^{he}_{\sigma _b}(\mathbf{j}^{\rm imp})$.

Let $\sigma _a$ be an anomalous conductivity distribution

$$\begin{aligned} \sigma _a(\mathbf{r}) =\sigma (\mathbf{r}) - \sigma _b(\mathbf{r}). \end{aligned}$$

(106)

By trivial manipulations with Eqs. (103) and (104) one can arrive at the scattering (integral) equation with respect to $\mathbf{E}$

$$\begin{aligned} \mathbf{E}= A \mathbf{E}+ \mathbf{E}_b , \end{aligned}$$

(107)

where

$$\begin{aligned} A = \mathbf{G}_{{\sigma _b}}^{ee}\circ \sigma _a . \end{aligned}$$

(108)

Note that the composition operator $A=\mathbf{G}_{{\sigma _b}}^{ee}\circ \sigma _a$ acts on $\mathbf{E}$ as $A\mathbf{E}=\mathbf{G}_{{\sigma _b}}^{ee}(\sigma _a\mathbf{E})$. The solution of Eq. (107) can be written in the following form

$$\begin{aligned} \mathbf{E}= \left( { {\mathbf {1}}- A }\right) ^{-1} \mathbf{E}_b , \end{aligned}$$

(109)

where ${\mathbf {1}}$ is a unit operator. If operator A is contracting (which means that $\Vert A\Vert <1)$ then $\left( {{\mathbf {1}}- A}\right) ^{-1}$ can be represented as

$$\begin{aligned} \left( { {\mathbf {1}}- A }\right) ^{-1} = {\mathbf {1}}+ A + A^2 + \cdots , \end{aligned}$$

(110)

and thus the solution of Eq. (109) reads as the following Neumann series

$$\begin{aligned} \mathbf{E}= \left( {{\mathbf {1}}- A }\right) ^{-1} \mathbf{E}_b = \mathbf{E}_b + A\mathbf{E}_b + A(A(\mathbf{E}_b)) + \cdots . \end{aligned}$$

(111)

Generally, contracting properties of operator $\mathbf{G}_{{\sigma _b}}^{ee}\circ \sigma _a$ are not known to us.

In Pankratov et al. (1995) it is shown that the energy inequality for Maxwell’s equations can be expressed as

$$\begin{aligned} \Vert \mathbf{K}_{{\sigma _b}}\Vert \le 1 , \end{aligned}$$

(112)

or in alternative form

$$\begin{aligned} \Vert \mathbf{K}_{{\sigma _b}}(\chi )\Vert \le \Vert \chi \Vert , \quad \text{ for } \text{ any } \text{ vector } \text{ field } \chi , \end{aligned}$$

(113)

where

$$\begin{aligned} \mathbf{K}_{{\sigma _b}}={\mathbf {1}}+2\sqrt{\gamma _b}\circ \mathbf{G}_{{\sigma _b}}^{ee}\circ \sqrt{\gamma _b} . \end{aligned}$$

(114)

Here $\gamma _b=\mathop {\text {Re}}\nolimits \sigma _b$ is the real part of the reference conductivity. Energy inequality (113) is sharp in the following sense: it turns into equality for a reference model with real-valued conductivity ($\mathop {\text {Im}}\nolimits \sigma _b=0$). In the presence of the imaginary part of conductivity, some part of the energy can radiate into the space, which makes (113) an inequality.

Using inequality (112) it is possible to obtain a new scattering equation with contracting operator. Let us first rewrite scattering equation (107) in the form

$$\begin{aligned} x=Ax+b , \end{aligned}$$

(115)

where $x=\mathbf{E}$ and $b=\mathbf{E}_b$. Let us renormalize it as

$$\begin{aligned} x=P\chi +Q \end{aligned}$$

(116)

and

$$\begin{aligned} x+\lambda x=Ax+\lambda x+b , \end{aligned}$$

(117)

with some unknown multipliers $P(\mathbf{r}), Q(\mathbf{r}), \lambda (\mathbf{r})$. Such renormalization modifies scattering equation (115) to a new scattering equation

$$\begin{aligned} \chi =B\chi +\beta \end{aligned}$$

(118)

with new linear operator

$$\begin{aligned} B = \frac{1}{P} \frac{1}{1+\lambda } (\lambda {\mathbf {1}}+A)\circ P \end{aligned}$$

(119)

and new source term

$$\begin{aligned} \beta = \frac{1}{P} \frac{1}{1+\lambda } ( AQ -Q + b ) . \end{aligned}$$

(120)

Let us then require that new scattering operator B be expressed as a composition of $\mathbf{K}_{{\sigma _b}}$ with some multiplication operator R

$$\begin{aligned} B=\mathbf{K}_{{\sigma _b}}\circ R , \end{aligned}$$

(121)

and hence we obtain

$$\begin{aligned} P(\mathbf{r}) = \frac{2g\sqrt{\gamma _b}}{\sigma +\sigma _b^*} , \quad Q(\mathbf{r}) = g \frac{\mathbf{j}^{\rm imp}}{\sigma +\sigma _b^*} , \quad \lambda (\mathbf{r}) = \frac{\sigma -\sigma _b}{2\gamma _b} , \quad R(\mathbf{r})= \frac{\sigma -\sigma _b}{\sigma +\sigma _b^*} . \end{aligned}$$

(122)

Here $g\ne 0$ is an undefined constant, thus we can assign

$$\begin{aligned} g=1 , \end{aligned}$$

(123)

as it is done in all works on CIE.

Deducing that $\frac{1}{P}\frac{1}{1+\lambda } = \sqrt{\gamma _b}$ and substituting expressions (122)–(123) into Eqs. (116) and (120), we get the expressions for $\mathbf{E}$ and $\beta$ as follows

$$\begin{aligned} \mathbf{E}& = \frac{2\sqrt{\gamma _b}\chi +\mathbf{j}^{\rm imp}}{\sigma +\sigma _b^*} ,\end{aligned}$$

(124)

$$\begin{aligned} \beta& = \sqrt{\gamma _b}\left( { \mathbf{E}_b - \frac{\mathbf{j}^{\rm imp}}{\sigma +\sigma _b^*} +\mathbf{G}_{{\sigma _b}}^{ee}\left( {\frac{\sigma -\sigma _b}{\sigma +\sigma _b^*}\mathbf{j}^{\rm imp}}\right) }\right) . \end{aligned}$$

(125)

It is proven in Pankratov et al. (1995) that

$$\begin{aligned} \Vert R\Vert \equiv \max |R(\mathbf{r})| = q < 1 , \end{aligned}$$

(126)

for feasible conductivity distributions ($\mathop {\text {Re}}\nolimits \sigma >0, \mathop {\text {Re}}\nolimits \sigma _b>0$), thus together with Eq. (113) it implies that new scattering operator (121) is contracting

$$\begin{aligned} \Vert B\Vert \le q <1 , \end{aligned}$$

(127)

and the Neumann series for Eq. (118)

$$\begin{aligned} \chi = ({\mathbf {1}}- B)^{-1}\beta = \beta +B\beta +B^2\beta +\cdots , \end{aligned}$$

(128)

is always convergent. We call new scattering equation (118) the contracting integral equation (CIE). An optimum choice of the reference conductivity, as well as an estimate of condition number of CIE system operator, ${\mathbf {1}}- B$, is discussed in the next appendix.

Appendix 5: Choice of Optimal Reference Model and Estimate of Condition Number for CIE System Operator

In the previous appendix we considered always contracting series (128) to solve Maxwell’s equations (103). Now we are interested in a choice of “optimum” reference conductivity, i.e. the conductivity that delivers the fastest convergence of the series (128). Let us imagine that we want to obtain the solution with desired accuracy $\varepsilon$, i.e.

$$\begin{aligned} \varepsilon = \frac{\Vert \chi -\chi _n\Vert }{\Vert \chi \Vert } , \end{aligned}$$

(129)

where

$$\begin{aligned} \chi _n = \beta +B\beta +\cdots +B^{n-1}\beta . \end{aligned}$$

(130)

Using the relation

$$\begin{aligned} \Vert \chi -\chi _n\Vert =\Vert B^n\chi \Vert \sim q^n \Vert \chi \Vert , \end{aligned}$$

(131)

we obtain that the number of iterations is governed by the following approximate equality

$$\begin{aligned} n \sim \ln \varepsilon /\ln q . \end{aligned}$$

(132)

From this relation and Eq. (126) it follows that the minimum number of iterations is achieved for the reference conductivity $\sigma _b(z)=\sigma _{\rm opt}(z)$ such that

$$\begin{aligned} \max \limits _{\sigma \in M(z)} |R(\sigma ,\sigma _{\rm opt}(z))| = \min \limits _{\sigma _b\in \varPi }\max \limits _{\sigma \in M(z)} |R(\sigma ,\sigma _b)| , \quad \text{ for } \text{ each } z , \end{aligned}$$

(133)

where

$$\begin{aligned} R(\sigma ,\sigma _b) := \frac{\sigma -\sigma _b}{\sigma +\sigma _b^*} . \end{aligned}$$

(134)

Here $\varPi$ stands for the right half plane of complex variable $\sigma$

$$\begin{aligned} \varPi = \left\{ {\sigma \in {\mathbb{C}}}\,\left| \,{\mathop {\text {Re}}\nolimits \sigma >0}\right. \right\} , \end{aligned}$$

(135)

and $M(z)\subset \varPi$ is the range of values of $\sigma (x,y,z)$ at horizontal plane $z=Const$

$$\begin{aligned} M(z) = \left\{ {\sigma (x,y,z)\in {\mathbb{C}}}\,\left| \,{x,y\in {\mathbb{R}}}\right. \right\} . \end{aligned}$$

(136)

Under these assumptions, it turns out that

$$\begin{aligned} q_{\rm opt}= \max \limits _{z} \min \limits _{\sigma _b\in \varPi } \max \limits _{\sigma \in M(z)} |R(\sigma ,\sigma _b)| . \end{aligned}$$

(137)

Now we notice that

$$\begin{aligned} |R(\sigma ,\sigma _b)| = \left| \frac{\sigma -\sigma _b}{\sigma +\sigma _b^*}\right| =\mathop {\text {tanh}}\nolimits {\frac{1}{2}{\mathcal {S}}(\sigma ,\sigma _b)} , \end{aligned}$$

(138)

where

$$\begin{aligned} {\mathcal {S}}(\sigma ,\sigma _b) = 2\mathop {\text {artanh}}\nolimits \left| \frac{\sigma -\sigma _b}{\sigma +\sigma _b^*}\right| = 2\mathop {\text {arsinh}}\nolimits \frac{|\sigma -\sigma _b|}{2\sqrt{\gamma }\sqrt{\gamma _b}} , \end{aligned}$$

(139)

is the Lobachevsky–Bolyai (Prasolov 2004) distance in the right half plane $\varPi$. Using Lobachevsky–Bolyai geometry formalism, with some efforts, it can be shown that

$$\begin{aligned} q_{\rm opt}(z) = \mathop {\text {tanh}}\nolimits \left( { \frac{1}{4} \mathop {\text {diam}}\nolimits {\mathcal {O}}(M(z)) }\right) , \end{aligned}$$

(140)

where ${\mathcal {O}}(M(z))$ is the minimum-size Lobachevsky–Bolyai circle that contains the set M(z). Moreover it can be shown that

$$\begin{aligned} \mathop {\text {diam}}\nolimits (M) \le \mathop {\text {diam}}\nolimits ({\mathcal {O}}(M)) \le 2 \mathop {\text {arsinh}}\nolimits \left( {\frac{\mathop {\text {sinh}}\nolimits \left( \frac{1}{2}\mathop {\text {diam}}\nolimits (M)\right) }{\sqrt{3}/{2}}}\right) . \end{aligned}$$

(141)

Here $\mathop {\text {diam}}\nolimits (M)$ is the diameter of set M in the Lobachevsky–Bolyai half plane $\varPi$

$$\begin{aligned} \mathop {\text {diam}}\nolimits M = \max _{\sigma ,\sigma _b\in M} {\mathcal {S}}(\sigma ,\sigma _b) . \end{aligned}$$

(142)

In addition it can be proven that $2\mathop {\text {arsinh}}\nolimits \left( {\frac{\mathop {\text {sinh}}\nolimits x}{\sqrt{3}/{2}}}\right)$ is monotonically increasing and convex upward (for $x>0$) function, from which the following inequality is valid

$$\begin{aligned} 2 \mathop {\text {arsinh}}\nolimits \left( {\frac{\mathop {\text {sinh}}\nolimits \left( \frac{1}{2}\mathop {\text {diam}}\nolimits (M)\right) }{\sqrt{3}/{2}}}\right) \le \min \left( { \frac{ \mathop {\text {diam}}\nolimits (M)}{\sqrt{3}/{2}} ,\, \mathop {\text {diam}}\nolimits (M) - 2 \ln \frac{\sqrt{3}}{2} }\right) . \end{aligned}$$

(143)

From Eqs. (140) to (143) it follows that the optimum number of iterations for the desired accuracy $\varepsilon$ is specified by the following inequality

$$\begin{aligned} E \le n_{\rm opt} \le 2E/\sqrt{3} , \quad E = \max \limits _z \tilde{E}(z) \cdot \ln \frac{1}{\varepsilon }, \end{aligned}$$

(144)

where

$$\begin{aligned} \tilde{E}(z) \sim \frac{1}{2} \sqrt{ \left( {\sqrt{\frac{\gamma _1}{\gamma _2}}-\sqrt{\frac{\gamma _2}{\gamma _1}}}\right) ^2 + \frac{\left( {\xi _1-\xi _2}\right) ^2}{\gamma _1 \gamma _2} } = \frac{|\sigma _1-\sigma _2|}{2\sqrt{\gamma _1}\sqrt{\gamma _2}} = {\text {sinh}} \frac{1}{2} {\mathcal {S}}(\sigma _1,\sigma _2) . \end{aligned}$$

(145)

Here conductivity pair $\sigma _1=\gamma _1+i\xi _1, \sigma _2=\gamma _2+i\xi _2$ from M(z) is the most mutually distant (in terms of Lobachevsky–Bolyai geometry) pair, i.e. a pair that delivers a maximum to expression in (145).

Using the developed formalism, it is also possible to estimate condition number $\kappa$ of CIE system operator ${\mathbf {1}}-B$ which obeys the following inequality

$$\begin{aligned}\kappa \le \frac{4}{\sqrt{3}} \max \limits _z\tilde{E}(z) . \end{aligned}$$

(146)

Let us illustrate an application of formulae (144)–(145) for the Cole-Cole induced polarization (IP) conductivity model

$$\begin{aligned} \sigma (x,y,z,\omega ) = \sigma _\infty (x,y,z) \left( { 1 - \frac{\eta (x,y,z)}{1+\left( {i\omega \tau (x,y,z)}\right) ^c} }\right) , \end{aligned}$$

(147)

with the following typically adopted parameters

$$\begin{aligned} 0 \le \eta (x,y,z) \le \eta _{\max } \approx 0.5 , \qquad c = 1/2 . \end{aligned}$$

(148)

Let us decompose the conductivity into real and imaginary parts as follows

$$\begin{aligned} \sigma =\gamma +i\xi , \end{aligned}$$

(149)

and let us vary location (x, y, z) in a thin layer at depth z, thus denoting in this layer

$$\begin{aligned} \gamma _{\min }(z,\omega ) = \min \limits _{x,y}\gamma (x,y,z,\omega ), \quad \gamma _{\max }(z,\omega ) = \max \limits _{x,y}\gamma (x,y,z,\omega ) , \end{aligned}$$

(150)

and in the same manner for the imaginary part, thus getting that for all locations in the layer z, complex-valued conductivity values belong to the following (Euclidean) rectangle

$$\begin{aligned} \left\{ \begin{array}{l}{ \gamma _{\min }(z,\omega ) \le \gamma (x,y,z,\omega ) \le \gamma _{\max }(z,\omega ),}\\ { \xi _{\min }(z,\omega ) \le \xi (x,y,z,\omega ) \le \xi _{\max }(z,\omega ).}\end{array} \right. \end{aligned}$$

(151)

Let us evaluate $\tilde{E}(z)$ for the Euclidean rectangle (151) using formulae (144)–(145). Value of $\tilde{E}(z)$ is then a maximum value of (145) for any pair of complex numbers $\gamma _1+i\xi _1, \gamma _2+i\xi _2$ that belong to Euclidean rectangle (151) as follows

$$\tilde{E}(z) = \max \left( { \tilde{E}_1(z) , \tilde{E}_2(z) }\right),$$

(152)

where

$$\begin{aligned} \tilde{E}_1(z) \sim \frac{\xi _{\max } - \xi _{\min }}{\gamma _{\min }} , \qquad \tilde{E}_2(z) \sim \frac{1}{2} \sqrt{ \frac{\gamma _{\max }}{\gamma _{\min }} + \frac{\left( {\xi _{\max }-\xi _{\min }}\right) ^2}{\gamma _{\max }\gamma _{\min }} } . \end{aligned}$$

(153)

Next, taking into account Cole–Cole Eqs. (147)–(148) we get $|\xi _{\max }| \sim 0.1|\gamma _{\max }|$ and thus

$$\begin{aligned} \tilde{E}(z) \sim \frac{1}{2} \max \left( { 0.1 \frac{\gamma _{\max }}{\gamma _{\min }} , \sqrt{1.01} \sqrt{\frac{\gamma _{\max }}{\gamma _{\min }}} }\right) . \end{aligned}$$

(154)

We see that for a high-contrasting inductive polarized z-layer the value of $\tilde{E}(z)$ grows as the contrast of the real part of conductivity, $K:=\frac{\gamma _{\max }}{\gamma _{\min }}$. The other consequence is that the IP contrast in Cole–Cole model plays significant role in finding the optimum model, if the contrast of the real part of conductivity is greater than 100. The latter value is a threshold: for $K\ge 100$ we get $\tilde{E}(z) \sim \frac{1}{20}K$, whereas for $K\le 100$ we have $\tilde{E}(z) \sim \frac{1}{2} \sqrt{K}$.

Final remark of this section is that if conductivity is a real-valued function (which is the most common case in EM studies), then for $\tilde{E}(z)$ the following equality holds

$$\begin{aligned} \tilde{E}(z) = \frac{1}{2}\sqrt{\frac{\gamma _{\max }(z)}{\gamma _{\min }(z)}} , \end{aligned}$$

(155)

where

$$\begin{aligned} \gamma _{\min }(z) = \min \limits _{x,y}\gamma (x,y,z), \quad \gamma _{\max }(z) = \max \limits _{x,y}\gamma (x,y,z) . \end{aligned}$$

(156)

Equation (155) means that the number of iterations and condition number for the optimal model are proportional to the square root of the maximum lateral contrast in the model. As for the optimum conductivity, $\sigma _{\rm opt}(z)$, it is equal in this case to the conductivity of the host section outside the depths occupied by the inhomogeneities, but at depths with laterally inhomogeneous distribution of conductivity it has the form

$$\begin{aligned} \sigma _{\rm opt}(z) \equiv \gamma _{\rm opt}(z) = \sqrt{\gamma _{\min }(z)\gamma _{\max }(z)} . \end{aligned}$$

(157)