The Role of Differential Equations in Applied Statistics

Abstract

The target of this paper is to discuss, investigate and present how the differential equations are applied in Statistics. The stochastic orientation of Statistics creates problems to adopt the differential equations as an individual tool, but Applied Statistics is using the differential equations either through applications from other fields, like Chemistry or as a tool to explain “variation” in stochastic processes.

Appendices

Appendix 1

Let us consider the main types of fundamental electrical circuits, so essential to build a differential equations approach to electrical circuits. To avoid any confusion the imaginary unit is denoted by j = (0, 1) and the current is denoted by i = i(t).

• 1. An RL—circuit.

  The voltage source, where it is assumed that it is AC, and thus $E = E_0 \sin \omega t$. Then based on the 2nd law of Kirchoff:

  $$\displaystyle \begin{aligned} L\frac{di}{dt} + R_i = E_0\sin\omega t\qquad t\geq 0 \end{aligned}$$

  The (general) solution is:

  $$\displaystyle \begin{aligned} \begin{array}{rcl} & i(t) = i_0 e^{-\frac{R}{L}t} + E_0\left(R^2 + L^2\omega^2\right)^{-1/2} \sin\left(\omega t - Arctan\frac{\omega L}{R} \right)\\ & \qquad + E_0\omega\left(R^2 + L^2\omega^2\right)^{-1} e^{-\frac{R}{L}t} \end{array} \end{aligned} $$

• 2. An RC—Circuit.

  Then it holds:

  $$\displaystyle \begin{aligned} R_i + \frac{1}{C}\int_{0}^{t} idt_1 = E(t) \end{aligned}$$

  And if E(t) =  constant = E ₀ then:

  $$\displaystyle \begin{aligned} R_i + \frac{1}{C}\int_{0}^{t} idt_1 = E_0 \end{aligned}$$

  And the solution is:

  $$\displaystyle \begin{aligned} i = i(t) = \frac{E_0}{R}e^{-t/RC}\text{,} \end{aligned}$$

• 3. RC—circuit with $E = E_0\cos {}(\omega t)$

  Then it holds:

  $$\displaystyle \begin{aligned} R_i + \frac{1}{C}\int_{0}^{t} idt_1 = E_0\cos{}(\omega t) \end{aligned}$$

  And if we let:

  $$\displaystyle \begin{aligned} u = \int_{0}^{t} idt_1 \end{aligned}$$

  We obtain that:

  $$\displaystyle \begin{aligned} R\frac{du}{dt} + \frac{u}{C} = E_0\cos{}(\omega t)\text{,}\qquad u(0) = 0 \end{aligned}$$

  Thus:

  $$\displaystyle \begin{aligned} u & = t e^{-t/RC}\int_{0}^{t}\frac{E_0}{R}e^{-t_1/RC}\cos{}(t)du \\ & = \frac{E_0}{R}\left[\frac{\left(1/RC\right)\left(\cos{}(\omega t) - e^{-t/RC}\right) + \omega\sin{}(\omega t)}{(1/RC)^2 + \omega^2}\right] = u_1 + u_2 \end{aligned} $$

  With:

  $$\displaystyle \begin{aligned} u_1 = \frac{-E_0/R\left(e^{-t/RC}/RC\right)}{(1/RC)^2 + \omega^2} \text{ ,}\quad u_2 = \frac{E_0}{R} \frac{\left(1/RC\right)\cos{}(\omega t) + \omega\sin{}(\omega t)}{(1/RC)^2 + \omega^2} \end{aligned}$$

  Eventually we can evaluate that:

  $$\displaystyle \begin{aligned} i_1 &= \frac{du_1}{dt} = \frac{-E_0/R\left(e^{-t/RC}/RC\right)^2}{(1/RC)^2 + \omega^2} = \frac{E_0}{R}e^{-t/RC} / (1 + \omega^2 (RC)^2) \\ i_2 &= \frac{du_1}{dt} = \frac{E_0}{R}\left[\frac{-\omega RC \sin{}(\omega t) + \omega^2(RC)^2\cos{}(\omega t)}{1 + (RC)^2 \omega^2}\right] \\ &\cong\frac{E_0}{R}\left[\frac{-\omega RC \sin{}(\omega t)}{1}\right] = -\omega CE_0\sin{}(\omega t) \end{aligned} $$

  Notice that with t ≪ i ₁, t = RC say then: $i_1 \cong \frac {E_0}{R} \frac {1}{e}\text{.}$

• 4. RL—circuit

  Assuming that E = E ₀ the (de) concerning this circuit is:

  $$\displaystyle \begin{aligned} L \frac{di}{dt} + Ri = E_0\text{,}\qquad L(0) = 0 \end{aligned} $$

  With solution: $i = \frac {E_0}{R} - \frac {E_0}{R} e^{-\frac {R}{L}t}$

• 5. LC—circuit with E = E ₀.

  The corresponding (de) is:

  $$\displaystyle \begin{aligned} L\frac{di}{dt} + \frac{1}{C}\int_{0}^{t} i dt_1 = E(t) \text{ and the solution is:} \end{aligned} $$
  $$\displaystyle \begin{aligned} i = E_0\sqrt{C/L}\sin\left(t / \sqrt{LC}\right) \text{ with } i(0) = 0 \text{ and } L_i^{\prime}(0) = E_0 \end{aligned} $$

• 6. LC—circuit with $E = E_0\cos {}(\omega t)$.

  The corresponding equation is: $L\frac {di}{dt} + \frac {1}{C}\int _{0}^{t} i dt = E_0\cos {}(\omega t)\text{.}$

  With solution eventually: $i = \frac {e^{j\omega t}}{\left (-L\omega ^2 + j\omega R + 1 / C\right )}$

  As it is known that it holds:

  $$\displaystyle \begin{aligned} \sin{}(\omega t) = \frac{e^{j \omega t} - e^{-j \omega t}}{2 j} \end{aligned} $$

  with the appropriate calculations the general solution eventually is:

  $$\displaystyle \begin{aligned} i = -\frac{E_0\omega \left(1 / C - L\omega^2\right)\sin{}(\omega t + E_0 R \omega^2\cos{}(\omega t)}{\left(-L\omega^2 + 1 / C\right)^2 + \omega^2 R^2} \end{aligned}$$

  There is an extensive approach which is beyond the target of this appendix.

Appendix 2: Introduction to Heat Equation

Consider a random walk, where we assume that the probability to move to the closest up, down, backwards, forwards points are equal to 1∕4. Let P(x, y;t) be the probability that a particle at time t is at (x, y) ∈ R ² point. Then we can see that at time t + 1 holds:

$$\displaystyle \begin{aligned} P(x,y; t + 1) {=} \frac{1}{4}\left\{P(x {-} 1, y; t) {+} P(x, y {-} 1; t) {+} P(x + 1, y, t) {+} P(x, y + 1; t)\right\} \end{aligned}$$

Thus for the difference:

$$\displaystyle \begin{aligned} P(x,y; t + 1) - P(x,y; t) &= \frac{1}{4} \Big\{ P(x + 1, y; t) - 2 P(x, y; t) + P(x - 1, y, t)\\ &\quad + P(x, y + 1; t) - 2P(x, y; t) + P(x, y - 1; t)\Big\} \end{aligned} $$

This difference equation approximated by the two-dimensional heat equation:

$$\displaystyle \begin{aligned} \frac{\partial P}{\partial t} = C\left(\frac{\partial^2 P}{\partial x^2} + \frac{\partial^2 P}{\partial y^2}\right) \end{aligned}$$

If we assume that the random—walk takes place in a limited domain D, usually of the form D = (l, u) × [L, U] ⊆ R ². More over it is assumed that the particle is absorbed when it reaches the boundary. Let (x ₀, y ₀) be the boundary points and W = W(x _b, y _b) the associated “profit that is paid out”.

If we denote by P(x, y;x ₀, y ₀) the probability that the particle start from the (interior) point (x, y) to be absorbed at the boundary (x _b, y _b) the expected “profit” is:

$$\displaystyle \begin{aligned} u(x,y) = \sum_{b} P(x, y; x_b, y_b) W(x_b, y_b) \end{aligned}$$

and satisfies the difference equation

$$\displaystyle \begin{aligned} u(x, y) = \frac{1}{4}\left\{u(x + 1, y) + u(x - 1, y) + u(x, y + 1) + u(x, y - 1)\right\} \end{aligned}$$

with u(x _b, y _b) = W(x _b, y _b). This equation is a well-known approximation of the Laplace equation:

$$\displaystyle \begin{aligned} \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = 0 \end{aligned}$$

Recall that

$$\displaystyle \begin{aligned} \nabla^2 u = \left\{ \begin{array}{ll} 0 & \text{Laplace equation} \\ {} C(x,y) & \text{Poisson equation} \end{array} \right. \end{aligned}$$

Through the Laplace equation three different well-known problems, associated with the boundary are defined: Dirichlet’s problem, Neuman’s problem, Robbin–Churchill’s problem for elliptic equations.