Solving Elliptic Equations with Brownian Motion: Bias Reduction and Temporal Difference Learning

Abstract

The Feynman-Kac formula provides a way to understand solutions to elliptic partial differential equations in terms of expectations of continuous time Markov processes. This connection allows for the creation of numerical schemes for solutions based on samples of these Markov processes which have advantages over traditional numerical methods in some cases. However, naïve numerical implementations suffer from issues related to statistical bias and sampling efficiency. We present methods to discretize the stochastic process appearing in the Feynman-Kac formula that reduce the bias of the numerical scheme. We also propose using temporal difference learning to assemble information from random samples in a way that is more efficient than the traditional Monte Carlo method.

Appendix: Expected Exit Time of a Brownian Bridge: Proof of Eq. 7

We establish the following inequality about the exit time of a Brownian bridge: Let 0 < a < x. Consider a Brownian bridge with initial position B(0) = 0 and final position B(Δτ) = x. Let $T_{a} = \inf \{t>0: B(t) > a\}$ be the first time the Brownian bridge crosses a barrier at a. Then, we have that $\mathbb {E}[T_{a}]$ obeys the inequality:

$$ \frac{1}{1 +x^{-2}{\varDelta} \tau } \leq \frac{\mathbb{E}[T_{a}]}{\frac{a}{x}{\varDelta} \tau} \leq 1. $$

(13)

In our setting, Eq. 7 follows immediately from this fact by taking the barrier $a = |\rho _{\partial {\varOmega }}(\vec B_{\text {old}})|$ and the final position $x = {\varDelta }\rho = \rho _{\partial {\varOmega }}(\vec B_{\text {new}}) - \rho _{\partial {\varOmega }}(\vec B_{\text {old}})$.

To prove Eq. 13, we use the probability density of T_a from Eq. 5, to find that $\mathbb {E}{\left [T_{a}\right ]}$ is given by

$$ \begin{array}{@{}rcl@{}} \mathbb{E}\left[T_{a}\right] &=& a{\int}_{0}^{{\varDelta}\tau}\frac{\sqrt{{\varDelta} \tau}}{\sqrt{2\pi}t^{1/2}({\varDelta}\tau - t)^{1/2}}\exp\left( \frac{x^{2}}{2{\varDelta} \tau}-\frac{(a-x)^{2}}{2({\varDelta}\tau - t)} - \frac{a^{2}}{2t}\right)~\ d t \\ & =& a {\int}_{0}^{{\varDelta}\tau}\rho\left( t,a \right)~\ d t = a\mathbb{E}\left[L_{a}\right], \end{array} $$

where $\rho \left (t,a \right )$ denotes the probability density of the Brownian bridge to be at B(t) = a at time t, and L_a is the local time at a of this Brownian bridge. The probability density for this local time has an explicit formula from Equation (3) in Pitman (1999), namely,

$$\mathbb{P}(L_{a} > y) = \exp\left( -\frac{1}{2{\varDelta}\tau}\left( (|a| + |x-a| + y)^{2} - x^{2}\right)\right).$$

For 0 < a < x, we have |a| + |x − a| = x, which yields

$$ \mathbb{E}\left[T_{a}\right] = a{\int}_{0}^{\infty} \exp\left( -\frac{1}{2{\varDelta}\tau}\left( 2xy + y^{2}\right)\right)~\ d y. $$

Finally, we can compute by a change of variable that

$$ \begin{array}{@{}rcl@{}} \frac{\mathbb{E}\left[T_{a}\right]}{\frac{a}{x}{\varDelta}\tau} &=& \frac{x}{{\varDelta}\tau}{\int}_{0}^{\infty} \exp\left( -\frac{1}{2{\varDelta}\tau}\left( 2xy + y^{2}\right)\right)~\ d y \\ & =& \sqrt{2\pi}x\exp\left( \frac{x^{2}}{2{\varDelta}\tau}\right){\int}_{0}^{\infty} \frac{1}{\sqrt{2\pi}{\varDelta}\tau}\exp\left( -\frac{1}{2}\left( \frac{y + x}{{\varDelta}\tau}\right)^{2}\right)~\ d y \\ & =& \sqrt{2\pi}\frac{x}{\sqrt{{\varDelta} \tau}}\exp\left( \frac{x^{2}}{2{\varDelta}\tau}\right)\mathbb{P}\left( X > \frac{x}{\sqrt{{\varDelta}\tau}}\right), \textrm{ where } X\sim \mathcal{N}(0,1). \end{array} $$

The Mill’s ratio inequality from Lemma 12.9 in Mörters and Peres (2012), which holds for all c > 0, gives

$$ \frac{1}{\sqrt{2\pi}}\frac{1}{c+c^{-1}} e^{-c^{2} / 2} \leq \mathbb{P}(X > c) \leq \frac{1}{\sqrt{2\pi}} \frac{1}{c} e^{-c^{2} / 2} . $$

This gives the desired result of Eq. 13 by setting $c = x/{\sqrt {{\varDelta } \tau }}$.