The distribution of the maximum likelihood estimates of the change point and their relation to random walks

Abstract

The problem of estimating the change point in a sequence of independent observations is considered. Hinkley (1970) demonstrated that the maximum likelihood estimate of the change point is associated with a two-sided random walk in which the ascending and descending epochs and heights are the key elements for its evaluation. The aim here is to expand the information generated from the random walks and from fluctuation theory and applied to the change point formulation. This permits us to obtain computable expressions for the asymptotic distribution of the change point with respect to convolutions and Laplace transforms of the likelihood ratios. Further, if moment expressions of the likelihood ratios are known, explicit representations of the asymptotic distribution of the change point become accessible up to the second order with respect to the moments. In addition, the rate of convergence between the finite and infinite distribution of the change point distribution is established and it is shown to be of polynomial order.

Appendices

Appendix A

1.1 Wiener–Hopf formulae and their consequences

Some of the results below are emanated from the work Spitzer (1974), Feller (1971, XII, and XVIII), Chung (1974), Inglehart (1974), Prabhu (1981), Asmussen (2003) among many others. They represent the behavior of a random walk on the line and conditioned that the random walk stays on the positive.

To gain more insight into the significance of Theorem 1.1 and Theorem 1.2, we first concentrate on the asymptotic expressions that appear in the theorems. The following result reveals some expressions between the time the maximum occurs and the time the random walk stays positive. The following results are a modification of Borovkov’s (1976) claim.

Theorem A.1

(Borovkov 1976). The sequence $\left\{ {Y_{k}^{\left( 1 \right)} \in \mathcal{X}:k \in {\mathbb{N}}} \right\}$ satisfies:

$$ P\left( {\overline{S}_{\infty }^{\left( 1 \right)} < \infty } \right) = 1,\;P\left( {\overline{S}_{\infty }^{\left( 1 \right)} = 0} \right) > 0\;{\text{and}}\;\sum\nolimits_{1 \le k} {k^{ - 1} P\left( {S_{k}^{\left( 1 \right)} > 0} \right)} < \infty . $$

Based on Corollary 6 in Borovkov (1976, p. 93), we have:

Theorem A.2

If $Y_{k}^{\left( 1 \right)}$, $k \in {\mathbb{N}}$ are non-lattice, then,

$$ J_{\infty }^{\left( 1 \right)} = \sup \left\{ {k \in {\mathbb{N}}:S_{k}^{\left( 1 \right)} = \overline{S}_{\infty }^{\left( 1 \right)} } \right\} = \inf \left\{ {k \in {\mathbb{N}}:S_{k}^{\left( 1 \right)} = \overline{S}_{\infty }^{\left( 1 \right)} } \right\}. $$

Proof

When $\left| z \right| < 1$, ${\text{Im}} \left( \lambda \right) = 0$, then $\left| {z \, \hat{f}_{{Y^{\left( 1 \right)} }} \left( \lambda \right)} \right| = \left| {z\int_{\mathbb{R}} {e^{i\lambda x} dP\left( {Y^{\left( 1 \right)} \le x} \right)} } \right| < 1$. Set $\wp_{z}^{\left( 1 \right)} \left( \lambda \right) = 1 - z \, \hat{f}_{{Y^{\left( 1 \right)} }}^{\left( 1 \right)} \left( \lambda \right)$, then the function $\wp_{z}^{\left( 1 \right)} \left( \lambda \right)$ (see e.g., Spitzer 1974) represents the Wiener–Hopf equation, that is written as,

$$ \wp_{z}^{\left( 1 \right)} \left( \lambda \right) = \wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)\wp_{{z^{ - } }}^{\left( 1 \right)} \left( \lambda \right)\exp \left( { - \sum\nolimits_{1 \le k} {z^{k} k^{ - 1} P\left( {S_{k}^{\left( 1 \right)} = 0} \right)} } \right), $$

(A.1)

where,

$$ \wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right) = \exp \left\{ { - \sum\nolimits_{1 \le k} {z^{k} k^{ - 1} \int_{{\left( {0,\infty } \right)}} {e^{i\lambda x} } dP\left( {S_{k}^{\left( 1 \right)} \le x} \right)} } \right\},\;{\text{and}} $$

(A.2)

$$ \wp_{{z^{ - } }}^{\left( 1 \right)} \left( \lambda \right) = \exp \left\{ { - \sum\nolimits_{1 \le k} {z^{k} k^{ - 1} \int_{{\left( { - \infty ,0} \right)}} {e^{i\lambda x} } dP\left( {S_{k}^{\left( 1 \right)} \le x} \right)} } \right\}. $$

(A.3)

Since $Y_{k}^{\left( 1 \right)}$ is non-lattice $P\left( {S_{k}^{\left( 1 \right)} = 0} \right) = 0$, $k \in {\mathbb{N}}$. Thus, (A.1) is simplified into,

$$ \wp_{z}^{\left( 1 \right)} \left( \lambda \right) = \wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)\wp_{{z^{ - } }}^{\left( 1 \right)} \left( \lambda \right). $$

(A.4)

Set $_{u} J_{\infty }^{\left( 1 \right)} = \sup \left\{ {k \in {\mathbb{N}}:S_{k}^{\left( 1 \right)} = \overline{S}_{\infty }^{\left( 1 \right)} } \right\}$ and $_{l} J_{\infty }^{\left( 1 \right)} = \inf \left\{ {k \in {\mathbb{N}}:S_{k}^{\left( 1 \right)} = \overline{S}_{\infty }^{\left( 1 \right)} } \right\}$. Applying Corollary 6 (Borovkov 1976, p. 93) we have

$$ \begin{aligned} E\left[ {\rho^{{_{u} J_{\infty }^{\left( 1 \right)} }} \exp \left( {i\lambda \overline{S}_{\infty }^{\left( 1 \right)} } \right)} \right] = & \frac{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}{{\wp_{{\rho^{ + } }}^{\left( 1 \right)} \left( \lambda \right)}}\; {\text{and}}\; \\ E\left[ {\rho^{{_{l} J_{\infty }^{\left( 1 \right)} }} \exp \left( {i\lambda \overline{S}_{\infty }^{\left( 1 \right)} } \right)} \right] = & \frac{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}{{\wp_{{\rho^{ + } }}^{\left( 1 \right)} \left( \lambda \right)}}\exp \left\{ { - \sum\nolimits_{1 \le k} {k^{ - 1} \left( {1 - \rho^{k} } \right)P\left( {S_{k}^{\left( 1 \right)} = 0} \right)} } \right\}. \\ \end{aligned} $$

Since $P\left( {S_{k}^{\left( 1 \right)} = 0} \right) = 0$, $k \in {\mathbb{N}}$, it follows that $_{u} J_{\infty }^{\left( 1 \right)} =_{l} J_{\infty }^{\left( 1 \right)} = J_{\infty }^{\left( 1 \right)}$.$\hfill\square$

This completes the proof of Theorem A.2.

To see the relationship between $J_{\infty }^{\left( 1 \right)}$ and the ascending and descending epochs, we have:

Corollary A.1

We have $P\left( {J_{\infty }^{\left( 1 \right)} = n} \right) = P\left( {T_{ \le }^{\left( 1 \right)} > n} \right)P\left( {T_{ > }^{\left( 1 \right)} = \infty } \right)$.

Proof

Since $S_{j + 1:n} =_{D} S_{n - j}$, $j = 0, \cdots ,n$, and since $\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)$, $\left( {Y_{n + 1}^{\left( 1 \right)} ,Y_{n + 2}^{\left( 1 \right)} , \cdots } \right)$, $n \in {\mathbb{N}}\backslash \left\{ 0 \right\}$ are independent, we have,

$$ \begin{aligned} P\left( {J_{\infty }^{\left( 1 \right)} = n} \right) = & P\left( {\left\{ {S_{n}^{\left( 1 \right)} > S_{j}^{\left( 1 \right)} ,j = 0,1, \cdots ,n - 1} \right\} \cap \left\{ {S_{n}^{\left( 1 \right)} \ge S_{j}^{{\left( {} \right)}} ,j = n + 1, \cdots } \right\}} \right) \\ = & P\left( {\left\{ {S_{j + 1:n}^{\left( n \right)} ,j = 0,1, \cdots ,n - 1} \right\} \cap \left\{ {S_{n:j}^{\left( 1 \right)} \le 0,j = n + 1, \cdots } \right\}} \right) \\ = & P\left( {S_{j + 1:n}^{\left( n \right)} > 0,j = 0,1, \cdots ,n - 1} \right)P\left( {S_{n:j}^{\left( 1 \right)} \le 0,j = n + 1, \cdots } \right) \\ = & P\left( {T_{ \le }^{\left( 1 \right)} > n} \right)P\left( {T_{ > }^{\left( 1 \right)} = \infty } \right). \\ \end{aligned} $$

This completes the proof of Corollary A.1.

The following two results are of importance to express probabilities or mixed characteristics and moment generating functions with respect to walk stays positive.$\hfill\square$

Theorem A.3

If $z \in \left( {0,1} \right)$, ${\text{Im}} \left( \lambda \right) = 0$, and $Y_{k}^{\left( 1 \right)}$, $k \in {\mathbb{N}}$, are non-lattice, then,

$$ Q^{\left( 1 \right)} \left( {s,\lambda } \right) = \frac{{1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} e^{{i\lambda S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} }} } \right]}}{{1 - z\int_{\mathbb{R}} {e^{{i\lambda Y^{\left( n \right)} }} } }} = {1 \mathord{\left/ {\vphantom {1 {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)}}} \right. \kern-0pt} {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)}}, $$

where $Q^{\left( 1 \right)} \left( {z,\lambda } \right) = \sum\nolimits_{0 \le n} {z^{n} E\left[ {e^{{i\lambda S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} \right]}$.

Proof

From the Baxter’s equations that determines the joint distributions of $\left( {T_{ > }^{\left( 1 \right)} ,S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$ and $\left( {T_{ \le }^{\left( 1 \right)} ,S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$, and for $z \in \left( {0,1} \right)$ and $\lambda \in {\mathbb{R}}$, the following identities hold,

$$ \hat{f}_{ > }^{\left( 1 \right)} \left( {z,\lambda } \right): = 1 - E\left[ {z^{{T_{ > }^{\left( 1 \right)} }} \exp \left( {i\lambda S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)} \right] = \wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right),\;\left| z \right| < 1\;{\text{and}}\; $$

(A.5)

A similar identification of $1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} \exp \left( {i\lambda S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)} \right]$ is obtained when $\left| z \right| \ge 1$ by simply replacing the event $\left\{ {S_{n}^{\left( 1 \right)} > 0} \right\}$ with its complementary event $\left\{ {S_{n}^{\left( 1 \right)} \le 0} \right\}$. Note that, in contrast to the series (A.5), the series with the complementary events fails to converge at $z = 1$. The verification that the series is convergent at $z = 1$ relies on the fact that the formula contains an exponential of Laurent series which converges throughout $\left| z \right| \ge 1$ and emphasizes the fact that $1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} \exp \left( {i\lambda S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)} \right]$ has a zero at $z = 1$ when $\lambda = 0$ (see, e.g., Bayer 1996),

$$ \hat{f}_{ \le }^{\left( 1 \right)} \left( {z,\lambda } \right): = 1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} \exp \left( {i\lambda S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)} \right] = \wp_{{z^{ - } }}^{\left( 1 \right)} \left( \lambda \right),\;\left| z \right| \ge 1 $$

(A.6)

Call $\hat{f}_{ > }^{\left( 1 \right)} \left( {1,\lambda } \right): = \hat{f}_{ > }^{\left( 1 \right)} \left( \lambda \right)$, ${\text{Im}} \left( \lambda \right) \ge 0$ and $\hat{f}_{ \le }^{\left( 1 \right)} \left( {1,\lambda } \right): = \hat{f}_{ \le }^{\left( 1 \right)} \left( \lambda \right)$, ${\text{Im}} \left( \lambda \right) \le 0$. From the Wiener–Hopf factorization, we also have

$$ 1 - \hat{f}^{\left( 1 \right)} \left( \lambda \right) = \left( {1 - \hat{f}_{ > }^{\left( 1 \right)} \left( \lambda \right)} \right)\left( {1 - \hat{f}_{ \le }^{\left( 1 \right)} \left( \lambda \right)} \right), $$

(A.7)

where $\hat{f}_{ > }^{\left( 1 \right)}$ and $\hat{f}_{ \le }^{\left( 1 \right)}$ are the Fourier-Stieltjes transforms of the random variables $S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)}$ and $S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)}$, respectively. Writing $B^{\left( 1 \right)} < \infty$, $F_{ > }^{\left( 1 \right)}$ and $F_{ \le }^{\left( 1 \right)}$ for the distribution function of $S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)}$ and $S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)}$, the following identity is well known

$$ F^{\left( 1 \right)} = F_{ > }^{\left( 1 \right)} + F_{ \le }^{\left( 1 \right)} - F_{ > }^{\left( 1 \right)} * F_{ \le }^{\left( 1 \right)} , $$

where $F^{\left( 1 \right)}$ is expressed with respect to the distributions $F_{ > }^{\left( 1 \right)}$ and $F_{ \le }^{\left( 1 \right)}$ defined on $\left( { - \infty ,0} \right]$ and $\left( {0,\infty } \right)$, respectively. Setting $z = 1$ and $\lambda = 0$ in (A.5) and (A.6), we have that,

$$ A^{\left( 1 \right)} = \sum\nolimits_{1 \le k} {k^{ - 1} P\left( {S_{k}^{\left( 1 \right)} \le 0} \right)} ,\;{\text{and}}\;B^{\left( 1 \right)} = \sum\nolimits_{1 \le k} {k^{ - 1} P\left( {S_{k}^{\left( 1 \right)} > 0} \right)} . $$

Since $A^{\left( 1 \right)} + B^{\left( 1 \right)} = \infty$ and $E\left[ {Y^{\left( 1 \right)} } \right] < 0$, it yields that,

$$ B^{\left( 1 \right)} < \infty \;{\text{and}}\;A = \infty ,{\text{ i}}.{\text{e}}.,F_{ > }^{\left( 1 \right)} \left( \infty \right) = 1 - e^{{ - B^{\left( 1 \right)} }} \;{\text{and}}\;F_{ \le }^{\left( 1 \right)} \left( {0_{ + } } \right) = 1. $$

(A.8)

Incorporating (A.5) and (A.6), the following holds,

$$ \frac{{1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} e^{{i\lambda S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} }} } \right]}}{{1 - z\int_{\mathbb{R}} {e^{{i\lambda Y^{\left( n \right)} }} } }} = \frac{{\wp_{{z^{ - } }}^{\left( 1 \right)} \left( \lambda \right)}}{{\wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)\wp_{{z^{ - } }}^{\left( 1 \right)} \left( \lambda \right)}} = {1 \mathord{\left/ {\vphantom {1 {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)}}} \right. \kern-0pt} {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( \lambda \right)}}. $$

(A.9)

Let the progressive $\sigma -$ algebra be defined as $F_{n - 1}^{\left( 1 \right)} = \sigma \left( {S_{0}^{\left( 1 \right)} ,S_{1}^{\left( 1 \right)} , \cdots ,S_{n - 1}^{\left( 1 \right)} } \right)$. The numerator in (A.9) can be then expressed as,

$$ \begin{aligned} & 1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} e^{{i\lambda S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} }} } \right] = 1 - \sum\nolimits_{0 \le n} {z^{n} E\left[ {e^{{i\lambda S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} = n} \right)} \right]} \\ = & 1 - \sum\nolimits_{0 \le n} {z^{n} E\left[ {e^{{i\lambda S_{n}^{\left( 1 \right)} }} \left\{ {I\left( {T_{ \le }^{\left( 1 \right)} > n - 1,Y_{n}^{\left( 1 \right)} + S_{n - 1}^{\left( 1 \right)} \le 0} \right) - I\left( {T_{ \le }^{\left( 1 \right)} > n,Y_{n}^{\left( 1 \right)} + S_{n - 1}^{\left( 1 \right)} \le 0} \right)} \right\}} \right]} \\ = & 1 - \sum\nolimits_{0 \le n} {z^{n} E\left[ {\left. {I\left( {T_{ \le }^{\left( 1 \right)} > n - 1} \right)E\left[ {e^{{i\lambda \left( {X_{n} + S_{n - 1}^{\left( 1 \right)} } \right)}} I\left( {Y_{n}^{\left( 1 \right)} + S_{n - 1}^{\left( 1 \right)} \le 0} \right)} \right]} \right|F_{n - 1}^{\left( 1 \right)} } \right]} \\ = & 1 - \sum\nolimits_{0 \le n} {z^{n} E\left[ {\left. {I\left( {T_{ \le }^{\left( 1 \right)} > n - 1} \right)e^{{i\lambda S_{n - 1}^{\left( 1 \right)} }} \left\{ {E\left[ {e^{{i\lambda Y_{1}^{\left( 1 \right)} }} } \right] - E\left[ {e^{{i\lambda Y_{1}^{\left( 1 \right)} }} I\left( {Y_{n}^{\left( 1 \right)} + S_{n - 1}^{\left( 1 \right)} > 0} \right)} \right]} \right\}} \right|F_{n - 1}^{\left( 1 \right)} } \right]} \\ = & \sum\nolimits_{0 \le n} {z^{n} E\left[ {e^{{i\lambda S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} \right]\left\{ {1 - z\hat{f}^{\left( 1 \right)} \left( \lambda \right)} \right\}} \\ = & - z\hat{f}^{\left( 1 \right)} \left( \lambda \right)\sum\nolimits_{0 \le n} {z^{n} E\left[ {e^{{i\lambda S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} \right] + 1 + \sum\nolimits_{1 \le n} {z^{n} E\left[ {e^{{i\lambda S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} \right]} } \\ \end{aligned} $$

(A.10)

In conjunction with (A.9) and (A.10), the proof of Theorem A.3 is completed.

The Baxter equation and the Spitzer’s formula are the highest intellectual achievements of the classical random walk theory. These equations show that the join distribution of $\left( {T_{ > }^{\left( 1 \right)} ,S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$ is determined by the mixed transform of $E\left[ {z^{{T_{ > }^{\left( 1 \right)} }} \exp \left( {i\lambda S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)} \right]$. In other words, this mixed transform can be inverted to obtain the joint probabilities associated with $\left( {T_{ > }^{\left( 1 \right)} ,S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$. Similar remarks apply to $\left( {T_{ \le }^{\left( 1 \right)} ,S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$. These formulae are rather complex and depend on the knowledge of $F_{{Y^{\left( 1 \right)} }}^{n * }$, for all $n \ge 0$ as indicated in Theorem A.3 and the Corollary A.1, below. However, before we even continue to apply the Baxter’s equation, we require to show that,

$$ \log \left( {{z \mathord{\left/ {\vphantom {z {\left( {1 - z\hat{f}_{{Y^{\left( 1 \right)} }}^{\left( 1 \right)} \left( \lambda \right)} \right)}}} \right. \kern-0pt} {\left( {1 - z\hat{f}_{{Y^{\left( 1 \right)} }}^{\left( 1 \right)} \left( \lambda \right)} \right)}}} \right) = \int_{\mathbb{R}} {\left( {e^{i\lambda x} - 1} \right)} \left( {\sum\nolimits_{n \ge 1} {\frac{{z^{n} }}{n}F_{{Y^{\left( 1 \right)} }}^{n * } } \left( {dx} \right)} \right) \equiv \int_{\mathbb{R}} {\left( {e^{i\lambda x} - 1} \right)\upsilon \left( {dx} \right)} $$

(A.11)

is bounded above. Note that the right-hand side of (A.11) is just a version of the Lévy-Khintchine representation. Therefore, to show the boundness of (A.11) the following lemma is of value.$\hfill\square$

Lemma A.1

Suppose that the measure $\upsilon$ on ${\mathbb{R}}$ is given by,

$$ \psi \left( s \right): = \int_{\mathbb{R}} {\left( {e^{isx} - 1} \right)\upsilon \left( {dx} \right)} ,\;s \in {\mathbb{R}}, $$

that satisfies, $\upsilon \left( {\left\{ 0 \right\}} \right) = 0$ and $\int_{\mathbb{R}} {\left( {\left| x \right| \wedge 1} \right)} \upsilon \left( {dx} \right) < \infty$. Then, there exists a random variable $Y$ with probability measure $\mu$ that has characteristic function $\hat{\mu }\left( s \right)$ given by

$$ \hat{\mu }\left( s \right) = \exp \left\{ {\psi \left( s \right)} \right\}, $$

and this representation is unique.

Upon Lemma A.1, it is then necessary to demonstrate that $\upsilon \left( {\left\{ 0 \right\}} \right) = 0$ and $\int_{\mathbb{R}} {\left( {\left| x \right| \wedge 1} \right)} \upsilon \left( {dx} \right) < \infty$ hold for the random walk $S^{\left( 1 \right)}$ (see, also Resnick 1992).

Lemma A.2

We have,

$$ \int_{\mathbb{R}} {\left( {e^{i\lambda x} - 1} \right)} \left( {\sum\nolimits_{n \ge 1} {\frac{{z^{n} }}{n}F_{{Y^{\left( 1 \right)} }}^{n * } } \left( {dx} \right)} \right) < \infty . $$

Proof

Note that,

$$ \int_{\mathbb{R}} {\left( {\left| x \right| \wedge 1} \right)\upsilon \left( {dx} \right)} = \int_{\left| x \right| \le 1} {\left( {\left| x \right| \wedge 1} \right)\upsilon \left( {dx} \right)} + \int_{\left| x \right| > 1} {\left( {\left| x \right| \wedge 1} \right)\upsilon \left( {dx} \right)} . $$

(A.12)

To show that each component on the right-hand side of (A.12) is bounded, we introduce the random variable $N \sim Geom\left( z \right)$, $z \in \left( {0,1} \right)$, that is independent of the random walk $S^{\left( 1 \right)}$. Thus,

$$ \begin{aligned} \int_{\left| x \right| \le 1} {\left( {\left| x \right| \wedge 1} \right)\upsilon \left( {dx} \right)} = & \int_{\left| x \right| \le 1} {\sum\nolimits_{n \ge 1} {\frac{{z^{n} }}{n}F_{{Y^{\left( 1 \right)} }}^{n * } } \left( {dx} \right)} \\ = & \left( {1 - z} \right)^{ - 1} \sum\nolimits_{n \ge 1} {\frac{{\left( {1 - z} \right)z^{n} }}{n}F_{{Y^{\left( 1 \right)} }}^{n * } } \left( {dx} \right) \\ = & \left( {1 - z} \right)^{ - 1} \sum\nolimits_{n \ge 1} {n^{ - 1} P\left( {S_{n}^{\left( 1 \right)} \in \left[ { - 1,1} \right],N = n} \right)} \\ \le & \left( {1 - z} \right)^{ - 1} \sum\nolimits_{n \ge 1} {P\left( {S_{n}^{\left( 1 \right)} \in \left[ { - 1,1} \right],N = n} \right)} \\ = & \left( {1 - z} \right)^{ - 1} P\left( {S_{N}^{\left( 1 \right)} \in \left[ { - 1,1} \right]} \right) < \infty . \\ \end{aligned} $$

Applying similar arguments as above, it can be also shown that,

$$ \int_{\left| x \right| > 1} {\left( {\left| x \right| \wedge 1} \right)} \upsilon \left( {dx} \right) \le \left( {1 - z} \right)^{ - 1} P\left( {S_{N} \notin \left[ { - 1,1} \right]} \right) < \infty . $$

This completes the proof of Lemma A.2.

The following corollary is a simple consequence of Theorem A.3.$\hfill\square$

Corollary A.2

If $z \in \left( {0,1} \right)$, and $Y_{k}^{\left( 1 \right)}$, $k \in {\mathbb{N}}$, are non-lattice, then,

$$ P\left( {T_{ > }^{\left( 1 \right)} = \infty } \right) = \wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right) $$

$$ Q^{\left( 1 \right)} \left( z \right) = \frac{{1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} } \right]}}{1 - z} = {1 \mathord{\left/ {\vphantom {1 {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}} \right. \kern-0pt} {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}, $$

where $Q^{\left( 1 \right)} \left( z \right) = \sum\nolimits_{0 \le n} {z^{n} P\left( {T_{ \le }^{\left( 1 \right)} > n} \right)}$.

Proof

From the Wiener–Hopf Eq. (2.3) and setting $\lambda = 0$ yields,which in turn yields,

$$ 1 - z = \left( {1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} } \right]} \right)\left( {1 - E\left[ {z^{{T_{ > }^{\left( 1 \right)} }} } \right]} \right). $$

$$ 1 - E\left[ {z^{{T_{ > }^{\left( 1 \right)} }} } \right] = \frac{{1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} } \right]}}{1 - z}. $$

(A.13)

Letting $z \uparrow 1$ in (A.12), we have,

$$ E\left[ {T_{ \le }^{\left( 1 \right)} } \right] = {1 \mathord{\left/ {\vphantom {1 {P\left( {T_{ > }^{\left( 1 \right)} = \infty } \right)}}} \right. \kern-0pt} {P\left( {T_{ > }^{\left( 1 \right)} = \infty } \right)}}. $$

(A.14)

Considering (A.13) and (A.14), it yields from Theorem A.3 when $\lambda = 0$ that,

$$ P\left( {T_{ > }^{\left( 1 \right)} = \infty } \right)^{ - 1} = \lim_{z \uparrow 1} \frac{{1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} } \right]}}{1 - z} = {1 \mathord{\left/ {\vphantom {1 {\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}} \right. \kern-0pt} {\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)}} $$

Also, from Theorem A.3 setting $\lambda = 0$, it yields,

$$ \frac{{1 - E\left[ {z^{{T_{ \le }^{\left( 1 \right)} }} } \right]}}{1 - z} = \sum\nolimits_{0 \le n} {z^{n} P\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} = Q\left( z \right) = {1 \mathord{\left/ {\vphantom {1 {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}} \right. \kern-0pt} {\wp_{{z^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}. $$

(A.16)

This completes the proof of Corollary 2.1.$\hfill\square$

Appendix B

2.1 Overshoot of the random walk

Let $K\left( s \right)$ denote the cumulant generating function (CGF) of the random variable $Y$, defined by

$$ K\left( s \right) = \ln \left\{ {\int_{\mathbb{C}} {e^{sx} F\left( {dx} \right)} } \right\},\;s \in {\mathbb{C}}. $$

For each $s \in {\mathbf{S}} = \left\{ {s \in {\mathbf{S}}:K\left( s \right) < \infty } \right\}$, we denote by $P_{s}$ the probability measure with density $e^{sx - K\left( s \right)}$ with respect to probability measure $P$. In statistical terminology, $\left\{ {F_{s} :s \in {\mathbf{S}}} \right\}$, the distribution function under the probability measure $P_{s}$, represents the exponential family generated by $F$. Specifically, the random variable $Y$ and the probability measure $P$ generate an exponential family of equivalent probability measure $P_{s}$, $s \in {\mathbf{S}}$ by,

$$ \frac{{dP_{s} }}{{dP_{{}} }} = \exp \left\{ {sY - K\left( s \right)} \right\}. $$

(B.1)

Let $\mu \left( s \right) = E_{s} \left[ Y \right]$ and $\sigma^{2} \left( s \right) = {\text{Var}}_{s} \left( Y \right)$ denote the mean and variance, respectively under the probability measure $P_{s}$. Let $K_{s} \left( u \right)$ be the CGF of $F_{s}$. Then, it follows that,

$$ K_{s} \left( u \right) = K\left( {u + s} \right) - K\left( s \right),\;E_{s} \left[ Y \right] = K^{\prime } \left( s \right)\;{\text{and}}\;{\text{Var}}_{s} \left[ Y \right] = K^{\prime \prime } \left( s \right),\;s \in {\mathbf{S}}. $$

Since $K\left( s \right)$ is strictly convex and the supp $\left( F \right) \cap \left( {0,\infty } \right) \ne \emptyset$, $K\left( s \right) \to \infty$ as $s \to \infty$. Quite often $K\left( \cdot \right)$ has finite radius of convergence (exponential family) and in many cases (heavy tails) $K\left( s \right) = \infty$, $\forall s > 0$. More often, the precise characterization of the heavy or light-tailed distribution are expressed in terms of the survival distribution, $\overline{F} = 1 - F$. Specifically, when $\overline{F}\left( {\ln x} \right) = l\left( x \right)$, where $l\left( \cdot \right)$ is slowly varying function, $F$ is heavy tailed distribution. In this case, $K\left( s \right) = \infty$, $\forall s > 0$ and the expectation may be finite or infinite. When $\overline{F}\left( {\ln x} \right) = x^{\delta } l\left( x \right)$, $\delta < 0$, $F$ is medium tailed distribution. In this case, $K\left( s \right) < \infty$, $\forall s > 0$ and the expectation is finite. When $\overline{F}\left( {\ln x} \right) \in R_{ - \infty }$, where $R_{ - \infty }$ is the class of functions whose Karamata indices are both $- \infty$ (see, e.g., Bingham et al. 1987), $F$ is of short tailed distribution. In this case, $K\left( s \right) < \infty$, $\forall s > 0$ and the expectation is finite. Various other variations of the above categories also exist (see, e.g., Rojo 1996). Here, we assume that the moment generating function is finite and $- \infty < E\left[ Y \right] < 0$ are satisfied, which, in turn, implies that we are dealing with short-tailed distributions. Under the short tail distribution and when the expectation of $Y$ is strictly negative, there exists $s_{0} > 0:$$K^{\prime}\left( {s_{0} } \right) = 0$ and $s_{1} > 0$: $K\left( {s_{1} } \right) = 0$. Specifically, it is seen that $\mu_{{s_{0} }} : = E_{{s_{0} }} \left[ Y \right] = K^{\prime}\left( {s_{0} } \right) = 0$ and if further the convexity property is applied, we also have $\mu_{{s_{1} }} : = E_{{s_{1} }} \left[ Y \right] =$$K^{\prime}\left( {s_{1} } \right) > 0$.

Translating the above remarks in terms of the change point scenario, the CGF of the random variable $Y^{\left( 1 \right)}$ is formed as $K_{{Y^{\left( 1 \right)} }} \left( \lambda \right) = \ln \left\{ {\int_{X} {f_{2}^{\lambda } \left( x \right)f_{1}^{1 - \lambda } \left( x \right)dv\left( x \right)} } \right\}$, $\lambda \in \left( {0,1} \right)$, in which case, $K_{{Y^{\left( 1 \right)} }} \left( 0 \right) = K_{{Y^{\left( 1 \right)} }} \left( 1 \right) = 0$ and $\exists \, \lambda_{0} \in \left( {0,1} \right)$: $K_{{Y^{\left( 1 \right)} }} \left( {\lambda_{0} } \right) < 1$ and $K^{\prime}_{{Y^{\left( 1 \right)} }} \left( {\lambda_{0} } \right) = 0$, $K^{\prime\prime}_{{Y^{\left( 1 \right)} }} \left( {\lambda_{0} } \right) > 0$, and in the support of $\lambda \in \left( {0,1} \right)$, $K_{{Y^{\left( 1 \right)} }}$ is may be infinitely differentiable. Note that $K^{\prime}_{{Y^{\left( 1 \right)} }} \left( 0 \right)$ and $K^{\prime}_{{Y^{\left( 1 \right)} }} \left( 1 \right)$ have opposite signs. To further obtain a knowledge of the above properties and see some implications significant to the change point, we next introduce the exponential family with parameter restricted on $\lambda \in \left( {0,1} \right)$ and a density given by $\frac{{dP_{{Y^{\left( 1 \right)} ,\lambda }} }}{{dP_{{Y^{\left( 1 \right)} }} }} = e^{{\lambda x - K_{{Y^{\left( 1 \right)} }} \left( \lambda \right)}}$ with $P_{{Y^{\left( 1 \right)} ,0}} : = P_{{Y^{\left( 1 \right)} }}$. Note that the expected value and variance under $P_{{Y^{\left( 1 \right)} ,\lambda }}$ are expressed as, $\mu_{\lambda }^{\left( 1 \right)} : = K^{\prime}_{{Y^{\left( 1 \right)} }} \left( \lambda \right) = E_{\lambda } \left[ {Y^{\left( 1 \right)} } \right]$ and $\sigma_{\lambda }^{\left( 1 \right)2} : = {\text{Var}}\left( {Y^{\left( 1 \right)} } \right) = K^{\prime\prime}_{{Y^{\left( 1 \right)} }} \left( \lambda \right)$. Hence, letting $\, x_{0} \in {\text{int}} X$, the maximum likelihood estimates $\hat{\lambda }\left( {x_{0} } \right): = \hat{\lambda }$ under the exponential family are defined as a solution of the equation,

$$ \hat{l}^{\left( 1 \right)} \left( {x_{0} } \right): = K_{{Y^{\left( 1 \right)} }} \left( {\hat{\lambda }} \right) - \hat{\lambda }x_{0} = \inf_{{\lambda \in X \cap \left( {0,1} \right)}} \left\{ {K_{{Y^{\left( 1 \right)} }} \left( \lambda \right) - \lambda x_{0} } \right\}. $$

Applying the Kullback–Leibler divergence for the exponential family, the information divergence turns out to be the Legendre-Fenchel transform given by $\hat{l}\left( {x_{0} } \right)$, i.e.,

$$ D_{KL} \left( {P_{{Y^{\left( 1 \right)} ,\hat{\lambda }}} ||P_{{Y^{\left( 1 \right)} }} } \right) = \int_{X} {\ln \left( {\frac{{dP_{{Y^{\left( 1 \right)} ,\hat{\lambda }}} }}{{dP_{{Y^{\left( 1 \right)} }} }}\left( x \right)} \right)} \, dP_{{Y^{\left( 1 \right)} }} \left( x \right) = - \left\{ {K_{{Y^{\left( 1 \right)} }} \left( {\hat{\lambda }} \right) - \hat{\lambda }\mu^{\left( 1 \right)} } \right\}. $$

(B.2)

Note that the entropy $\lim_{n \to \infty } n^{ - 1} \log P\left( {n^{ - 1} S_{n}^{\left( 1 \right)} \in A} \right) = s^{\left( 1 \right)} \left( A \right)$, for any Borel set $A$, is closely related to Legendre-Fenchel transform via $s\left( x \right) = - \hat{l}\left( x \right)$, which opens a new direction of how alternately the change point can be handled.

If $E_{\lambda }$ is the expectation operator corresponding to $P_{{Y^{\left( 1 \right)} ,\lambda }}$, then for any fixed $n \in {\mathbb{N}}$

$$ E_{\lambda } \left[ {f\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)} \right] = E\left[ {e^{{\lambda S_{n}^{\left( 1 \right)} - nK_{{Y^{\left( 1 \right)} }} \left( \lambda \right)}} f\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)} \right], $$

for all measurable functions $f:{\mathbb{R}}^{n} \to {\mathbb{R}}$, which are bounded or nonnegative (see, Asmussen 2003, XIII). Replacing first $f\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)$ by $e^{{ - \lambda S_{n} + nK_{{Y^{\left( 1 \right)} }} \left( \lambda \right)}} f\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)$ and specializing next to an indicator function of a Borel set $A \subseteq {\mathbb{R}}^{n}$, it yields,

$$ E\left[ {f\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)} \right] = E_{\lambda } \left[ {e^{{ - \lambda S_{n}^{\left( 1 \right)} + nK_{{Y^{\left( 1 \right)} }} \left( \lambda \right)}} f\left( {Y_{1}^{\left( 1 \right)} , \cdots ,Y_{n}^{\left( 1 \right)} } \right)} \right],\;{\text{and}} $$

(B.3)

$$ P\left( A \right) = E_{\lambda } \left[ {e^{{ - \lambda S_{n}^{\left( 1 \right)} + nK_{{Y^{\left( 1 \right)} }} \left( \lambda \right)}} ,A} \right] $$

(B.4)

The point of formulations as (B.3) and (B.4) is that in several cases the $F_{\lambda }$-distribution has more accessible properties than $F$, as it is demonstrated below.

When $K^{\prime}_{{Y^{\left( 1 \right)} }} \left( 0 \right) = E\left[ {Y^{\left( 1 \right)} } \right] < 0$, the Lundberg equation expressed as $K_{{Y^{\left( 1 \right)} }} \left( \lambda \right) = 0$ suggest that $\lambda = 1$ and $K^{\prime}_{{Y^{\left( 1 \right)} }} \left( 1 \right) > 0$. In what follows we exploit the exponential change of measure corresponding to $\lambda = 1$ and write $P_{{Y^{\left( 1 \right)} ,1}} : = P_{1}$. Specifically, the Cramér-Lundberg theory is now applicable to provide asymptotic expressions of the distribution of the $\overline{S}_{\infty }^{\left( 1 \right)}$ that can be consequently of use to develop expressions for the distribution of $\xi_{\infty }$. Applying (2.4), we have $P\left( G \right) = E_{1} \left[ {e^{{ - \lambda S_{{T_{ > }^{\left( 1 \right)} \left( x \right)}}^{\left( 1 \right)} }} ,G} \right]$, for $G \in F_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)}$, $G \subseteq \left\{ {T_{ > }^{\left( 1 \right)} \left( x \right) < \infty } \right\}$ and $x \in {\mathbb{R}}_{ > }$. Define $B^{\left( 1 \right)} \left( x \right): = S_{{T_{ > }^{\left( 1 \right)} \left( x \right)}}^{\left( 1 \right)} - x$ be the overshoot variable. Then, the Lundberg’s inequality (see Asmussen 2003, XIII) is written as,

$$ P\left( {\overline{S}_{\infty }^{\left( 1 \right)} > x} \right) \le e^{ - x} ,\;{\text{for}}\;{\text{all}}\;x \in {\mathbb{R}}_{ > } $$

(B.5)

The argument in (B.5) is used just to neglect the excess over the boundary. A small refinement produces a version of the Cramér-Lundberg approximation, which is given without any proof in the following theorem (see, Veraverbeke 1977 or Asmussen 2003, XIII).

Theorem B.1

If $B^{\left( 1 \right)} \left( x \right)$ converges in $P_{1} -$ distribution as $x \to \infty$, $\lim_{x \to \infty } B^{\left( 1 \right)} \left( x \right) = B^{\left( 1 \right)} \left( \infty \right)$, then,

$$ P\left( {\overline{S}_{\infty }^{\left( 1 \right)} > x} \right) \sim C^{\left( 1 \right)} e^{ - x} , $$

where $C^{\left( 1 \right)} = E_{1} \left[ {e^{{ - B^{\left( 1 \right)} \left( \infty \right)}} } \right]$.

The next theorem provides expressions of the ultimum overshoot time $B^{\left( 1 \right)} \left( \infty \right)$.

Theorem B.2

For the random walk $S^{\left( 1 \right)}$, $B^{\left( 1 \right)} \left( \infty \right)$ exists with respect to $P_{1}$. In this case the constant $C$ is given by

$$ C^{\left( 1 \right)} = \frac{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}{{\int_{0}^{\infty } {xe^{x} F_{ > } \left( {dx} \right)} }} = \frac{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)\left\{ {1 - \int_{ - \infty }^{0} {e^{x} F_{ \le } \left( {dx} \right)} } \right\}}}{{E_{1} \left[ {Y^{\left( 1 \right)} } \right]}}. $$

From (A.10) and since $E\left[ {\exp \left( {Y^{\left( 1 \right)} } \right)} \right] = 1$, we also obtain using the Wald’s identity that,

$$ 1 - \int_{ - \infty }^{0} {e^{x} F_{ \le } \left( {dx} \right) = 1 - E\left[ {\exp \left( {S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)} \right]} = \frac{{E\left[ {\exp \left( {Y^{\left( 1 \right)} } \right)} \right]}}{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( { - i} \right)}} = \frac{1}{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( { - i} \right)}},\;i^{2} = - 1 $$

which yields that,

$$ C^{\left( 1 \right)} = \frac{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( 0 \right)}}{{\wp_{{1^{ + } }}^{\left( 1 \right)} \left( { - i} \right)}}. $$

Note, again, that the constant $C^{\left( 1 \right)}$ is now explicitly determined by the n-times convolutions of the random walk $S^{\left( 1 \right)}$.

From Corollary, A.2, we have that $Q^{\left( 1 \right)} \left( z \right)\wp_{{z^{ + } }}^{\left( 1 \right)} \left( 0 \right) = 1$, for $z \in \left( {0,1} \right)$. Note that when $f\left( x \right)$ and $g\left( x \right)$ are n-times differentiable functions, then the Leibnitz rule states that the product $h\left( x \right) = f\left( x \right)g\left( x \right)$ is also n-times differentiable and its nth derivative is given by $h^{\left( n \right)} \left( x \right) = \sum\nolimits_{0 \le k \le n} {\left( \begin{gathered} n \hfill \\ k \hfill \\ \end{gathered} \right)f^{{\left( {n - k} \right)}} \left( x \right)g^{\left( k \right)} \left( x \right)}$. Setting $b_{n}^{\left( 1 \right)} = P\left( {S_{n}^{\left( 1 \right)} > 0} \right)$, $q_{n}^{\left( 1 \right)} = P\left( {T_{ \le }^{\left( 1 \right)} > n} \right)$ and then applying the Leibnitz rule for $Q^{\left( 1 \right)} \left( z \right)\wp_{{z^{ + } }}^{\left( 1 \right)} \left( 0 \right) = 1$, we obtain that,

$$ nq_{n}^{\left( 1 \right)} = \sum\nolimits_{0 \le k \le n - 1} {b_{n - k}^{\left( 1 \right)} q_{k}^{\left( 1 \right)} } ,\;n \in {\mathbb{N}}\;{\text{and}}\;q_{0}^{\left( 1 \right)} = 1. $$

(B.6)

Applying Theorem A.3, we also have that $Q^{\left( 1 \right)} \left( {1, - i\lambda } \right)\wp_{{1^{ + } }}^{\left( 1 \right)} \left( { - i\lambda } \right) = 1$. Setting $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{b}_{n}^{\left( 1 \right)} = E\left[ {e^{{ - S_{n}^{\left( 1 \right)} }} I\left( {S_{n}^{\left( 1 \right)} > 0} \right)} \right]$, $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{q}_{n}^{\left( 1 \right)} = E\left[ {e^{{ - S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} \right]$ and then employing the Leibnitz rule, as in (B.6), we have that,

$$ n\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{q}_{n}^{\left( 1 \right)} = \sum\nolimits_{0 \le k \le n - 1} {\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{b}_{n - k}^{\left( 1 \right)} \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{q}_{k}^{\left( 1 \right)} } ,\;n \in {\mathbb{N}}\;{\text{and}}\;\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{q}_{0}^{\left( 1 \right)} = 1. $$

(B.7)

Building on the above expressions, we have provided a complete solution of the random walk problems. In a way, the results in Appendix A and Appendix B show that the probability distributions of $\left( {T_{ \le }^{\left( 1 \right)} ,S_{{T_{ \le }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$, $\left( {T_{ > }^{\left( 1 \right)} ,S_{{T_{ > }^{\left( 1 \right)} }}^{\left( 1 \right)} } \right)$, $\overline{S}_{\infty }^{\left( 1 \right)}$ and $q_{n}^{\left( 1 \right)} = P\left( {T_{ \le }^{\left( 1 \right)} > n} \right)$ are explicitly determined by the knowledge of the distribution of $F_{{Y^{\left( 1 \right)} }}$ and consequently being able to evaluate the n-time convolutions. Thereby expressions $E\left[ {e^{{ - S_{n}^{\left( 1 \right)} }} I\left( {T_{ \le }^{\left( 1 \right)} > n} \right)} \right]$ are plainly determined via the statements $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{b}_{n}^{\left( 1 \right)} = E\left[ {e^{{ - S_{n}^{\left( 1 \right)} }} I\left( {S_{n}^{\left( 1 \right)} > 0} \right)} \right]$, i.e., functionals of the n-time convolutions of $F_{{Y^{\left( 1 \right)} }}$.

Appendix C

3.1 Edgeworth expansion

For any given $X,X_{1} , \cdots ,X_{n}$ of i.i.d. random variables with mean $\mu = E\left[ X \right]$, let $\overline{X}_{n} = n^{ - 1} \sum\nolimits_{k = 1}^{n} {X_{k} }$. Throughout the all the appendices assume that $X$ is not arithmetic with cumulative distribution function $F$. The classical Edgeworth expansion approximates the density $U = \sqrt n \left( {\overline{X}_{n} - \mu } \right)$ via the characteristic as (see, Jensen 1995)

$$ \begin{aligned} E\left[ {e^{itU} } \right] = & E\left[ {e^{{it\sqrt n \overline{X}_{n} }} } \right]e^{ - i\sqrt n t\mu } \\ & = \exp \left\{ { - \frac{{t^{2} }}{2}\kappa_{2} + \frac{{\left( {it} \right)^{3} \kappa_{3} }}{{3!n^{{{1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} }} + \cdots + \frac{{\left( {it} \right)^{m} \kappa_{m} }}{{m!n^{{{{\left( {m - 2} \right)} \mathord{\left/ {\vphantom {{\left( {m - 2} \right)} 2}} \right. \kern-0pt} 2}}} }} + \omega \frac{{c_{2} E\left[ {\left| X \right|^{m + 1} } \right]}}{{n^{{{{\left( {m - 1} \right)} \mathord{\left/ {\vphantom {{\left( {m - 1} \right)} 2}} \right. \kern-0pt} 2}}} }}\left| t \right|^{m + 1} } \right\}\\ & = e^{{ - {{t^{2} \kappa_{2} } \mathord{\left/ {\vphantom {{t^{2} \kappa_{2} } 2}} \right. \kern-0pt} 2}}} \exp \left( {\beta + \eta } \right), \\ \end{aligned} $$

(C.1)

where $\beta = \sum\nolimits_{j = 3}^{m} {\frac{{\left( {it} \right)^{j} \kappa_{j} }}{{j!n^{{{{\left( {j - 2} \right)} \mathord{\left/ {\vphantom {{\left( {j - 2} \right)} 2}} \right. \kern-0pt} 2}}} }}}$ and $\eta = \omega \frac{{c_{2} E\left[ {\left| X \right|^{m + 1} } \right]}}{{n^{{{{\left( {m - 1} \right)} \mathord{\left/ {\vphantom {{\left( {m - 1} \right)} 2}} \right. \kern-0pt} 2}}} }}\left| t \right|^{m + 1}$, for $\left| t \right| \le \sqrt n c_{1} E\left[ {\left| X \right|^{m + 1} } \right]$. Here $\omega \in {\mathbb{C}}$ and it is such that $\left\| \omega \right\| \le 1$. Before implemented (C.1), the exponential $\exp \left( {\beta + \eta } \right)$ is expanded in a power series to have,

$$ \left| {E\left[ {e^{itU} } \right] - e^{{ - {{\kappa_{2} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2} t^{2} } 2}} \right. \kern-0pt} 2}}} \left\{ {1 + \sum\nolimits_{j = 3}^{m} {n^{{ - {{\left( {j - 2} \right)} \mathord{\left/ {\vphantom {{\left( {j - 2} \right)} 2}} \right. \kern-0pt} 2}}} P_{j} \left( {it} \right)} } \right\}} \right| \le \frac{{c_{4} \left( {\kappa_{2} ,E\left[ {\left| X \right|^{m + 1} } \right]} \right)}}{{n^{{{{\left( {m - 1} \right)} \mathord{\left/ {\vphantom {{\left( {m - 1} \right)} 2}} \right. \kern-0pt} 2}}} }}\left\{ {\left| t \right|^{m + 1} + \left| t \right|^{{3\left( {m - 1} \right)}} } \right\}, $$

(C.2)

for $\left| t \right| \le \sqrt n c_{3} \left( {\kappa_{2} ,E\left[ {\left| X \right|^{m + 1} } \right]} \right)$, and $c_{3}$,$c_{4}$ are constants depending on $\kappa_{2}$ and the absolute moment $E\left[ {\left| X \right|^{m + 1} } \right]$. The $P_{s}$’s are orthogonal polynomials with coefficients depending on the cumulants. For $m = 4$, we have

$$ P_{3} \left( {it} \right) = \frac{{\kappa_{3} \left( {it} \right)^{3} }}{3!},P_{4} \left( {it} \right) = \frac{{\kappa_{4} \left( {it} \right)^{4} }}{4!} + \frac{{\kappa_{3}^{2} \left( {it} \right)^{6} }}{{\left( {3!} \right)^{2} 2!}}. $$

The theorem below represents the basis of how a density can be recovered from the characteristic function, known as the inversion theorem.

Theorem C1

(Inversion formula). If $\gamma \in {\mathbf{L}}_{1} \left( {\mathbb{R}} \right)$ then $X$ has a bounded continuous density $f$ with respect to Lebesgue measure given by

$$ f\left( x \right) = \frac{1}{2\pi }\int_{\mathbb{R}} {e^{ - itx} \gamma \left( t \right)dt} . $$

Since $E\left[ {e^{itU} } \right]{, }e^{{ - {{\kappa_{2} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2} t^{2} } 2}} \right. \kern-0pt} 2}}} \left\{ {1 + \sum\nolimits_{j = 3}^{m} {n^{{ - {{\left( {j - 2} \right)} \mathord{\left/ {\vphantom {{\left( {j - 2} \right)} 2}} \right. \kern-0pt} 2}}} P_{j} \left( {it} \right)} } \right\} \in {\mathbf{L}}_{1} \left( {\mathbb{R}} \right)$, it follows from Theorem A1 that $U$ and the series expansion have a bounded density $g_{n}$ and a real-valued inversion function $h_{n}$ given by

$$ g_{n} \left( x \right) = \frac{1}{2\pi }\int_{\mathbb{R}} {e^{ - itx} E\left[ {e^{itU} } \right]dt} $$

$$ \begin{aligned} h_{n} \left( x \right) = & \frac{1}{2\pi }\int_{\mathbb{R}} {e^{ - itx} e^{{ - {{\kappa_{2} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2} t^{2} } 2}} \right. \kern-0pt} 2}}} \left\{ {1 + \sum\nolimits_{j = 3}^{m} {n^{{ - {{\left( {j - 2} \right)} \mathord{\left/ {\vphantom {{\left( {j - 2} \right)} 2}} \right. \kern-0pt} 2}}} P_{j} \left( {it} \right)} } \right\}dt} \\ = & \frac{{e^{{ - {{x^{2} } \mathord{\left/ {\vphantom {{x^{2} } {2\kappa_{2} }}} \right. \kern-0pt} {2\kappa_{2} }}}} }}{{\sqrt {2\pi \kappa_{2} } }} + \sum\nolimits_{j = 3}^{m} {n^{{ - {{\left( {j - 2} \right)} \mathord{\left/ {\vphantom {{\left( {j - 2} \right)} 2}} \right. \kern-0pt} 2}}} } \frac{1}{2\pi }\int_{\mathbb{R}} {e^{ - itx} P_{j} \left( {it} \right)e^{{ - {{\kappa_{2} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2} t^{2} } 2}} \right. \kern-0pt} 2}}} dt} . \\ \end{aligned} $$

satisfying $\left| {g_{n} \left( x \right) - h_{n} \left( x \right)} \right| = o\left( {n^{{{{\left( {m - 2} \right)} \mathord{\left/ {\vphantom {{\left( {m - 2} \right)} 2}} \right. \kern-0pt} 2}}} } \right)$. In addition, since $\int_{\mathbb{R}} {\left( {it} \right)^{j} e^{ - itx} e^{{ - {{\kappa_{2} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2} t^{2} } 2}} \right. \kern-0pt} 2}}} dt} = \kappa_{2}^{{ - {j \mathord{\left/ {\vphantom {j 2}} \right. \kern-0pt} 2}}} H_{j} \left( {{x \mathord{\left/ {\vphantom {x {\sqrt {\kappa_{2} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2} } }}} \right)e^{{ - {{x^{2} } \mathord{\left/ {\vphantom {{x^{2} } {2\kappa_{2} }}} \right. \kern-0pt} {2\kappa_{2} }}}}$, $j \ge 3$, where $H_{j}$ denotes the Hermite polynomials, the $m = 4$ approximation for $g_{n}$, the probability density of $U$ can be approximated as

$$ g_{n} \left( x \right) \approx e^{{ - {{x^{2} } \mathord{\left/ {\vphantom {{x^{2} } {2\kappa_{2} }}} \right. \kern-0pt} {2\kappa_{2} }}}} \left\{ {1 + \frac{{\lambda_{3} }}{{3!n^{{{1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} }}H_{3} \left( {{x \mathord{\left/ {\vphantom {x {\sqrt {\kappa_{2} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2} } }}} \right) + \frac{1}{n}\left[ {\frac{{\lambda_{4} }}{4!}H_{4} \left( {{x \mathord{\left/ {\vphantom {x {\sqrt {\kappa_{2} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2} } }}} \right) + \frac{{\lambda_{3}^{2} }}{{\left( {3!} \right)^{2} 2!}}H_{6} \left( {{x \mathord{\left/ {\vphantom {x {\sqrt {\kappa_{2} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2} } }}} \right)} \right]} \right\}, $$

(C.3)

where $\lambda_{j} = {{\kappa_{j} } \mathord{\left/ {\vphantom {{\kappa_{j} } {\kappa_{2}^{{{j \mathord{\left/ {\vphantom {j 2}} \right. \kern-0pt} 2}}} }}} \right. \kern-0pt} {\kappa_{2}^{{{j \mathord{\left/ {\vphantom {j 2}} \right. \kern-0pt} 2}}} }}$, $j \ge 3$, the standardized cumulants. This is known as the Edgeworth expansion for 1-dimension.

Appendix D

4.1 Esscher tilting

The basic idea behind the approximations below stemmed from the work of Esscher (1932). To understand how the derivations will be developed, the abstract formulation below will guide us to the solution. Let $P$ and $Q$ be equivalent measures. Then, for any Borel set $A$, we have that,

$$ \begin{aligned} P\left( {Y \in A} \right) = & \int_{A} {P\left( {dx} \right)} = \int_{A} {\frac{dP}{{dQ}}\left( x \right)Q\left( {dx} \right)} \\ = & \int_{A} {\left\{ {\frac{dQ}{{dP}}\left( x \right)} \right\}^{ - 1} Q\left( {dx} \right)} \\ = & \left\{ {\frac{dQ}{{dP}}\left( {x_{Q} } \right)} \right\}^{ - 1} \int_{A} {\left[ {\frac{dQ}{{dP}}\left( {x_{Q} } \right)\left\{ {\frac{dQ}{{dP}}\left( x \right)} \right\}^{ - 1} } \right]Q\left( {dx} \right)} , \\ \end{aligned} $$

(D.1)

where $x_{Q}$ can be any given arbitrary point choice.

To have a sense of how $x_{Q}$ is chosen, we again refer to Jensen (1995, p. 12). A consequence of (D.1) is to determine the density of a statistic $T$ with respect to some measure $\nu$. To see this, let $P_{T}$ be the measure of the statistic and let $Q_{T}$ be an equivalent measure. Then, applying (D.1) to a single point, we have

$$ \frac{{dP_{T} }}{d\nu }\left( x \right) = \frac{{dP_{T} }}{{dQ_{T} }}\left( x \right)\frac{{dQ_{T} }}{d\nu }\left( x \right) = \left\{ {\frac{{dQ_{T} }}{{dP_{T} }}\left( x \right)} \right\}^{ - 1} \frac{{dQ_{T} }}{d\nu }\left( x \right). $$

(D.2)

The measure $Q$ is taken such that $x$ is a central point of the distribution, i.e., $Q$ is the tilted measure.

In applying (B.2), we let the statistic of interest, $T = \overline{X}_{n}$, be the sample mean. Let ${\mathbf{S}} = \left\{ {s \in \infty :K\left( s \right) < \infty } \right\}$ and $f_{n}$ be the density function of $\overline{X}_{n}$, where $K$ denotes the cumulative generating function of $X$. The following lemma is of importance.

Lemma D1

For $s \in {\mathbf{S}}$, we have

$$ f_{n} \left( x \right) = n^{{{1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} e^{{n\left\{ {K\left( s \right) - sx} \right\}}} f_{n,s} \left( 0 \right),\;x \in {\mathbb{R}}_{ + } , $$

where $f_{n,s}$ is the density of $\sqrt n \left( {\overline{X}_{n} - x} \right)$ under $P_{s}$.

Proof

Set $\frac{{dQ_{{\overline{X}_{n} }} }}{{dP_{{\overline{X}_{n} }} }} = e^{{n\left\{ {sx - K\left( s \right)} \right\}}}$ to be the density of the exponential family. Set $U = \sqrt n \left( {\overline{X}_{n} - x} \right)$. Applying (B.2) for the exponential measure, we have

$$ f_{n} \left( x \right) = nf_{{S_{n} }} \left( x \right) = ne^{{n\left\{ {K\left( s \right) - sx} \right\}}} f_{{S_{n} ,s}} \left( x \right) = n^{{{1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} e^{{n\left\{ {K\left( s \right) - sx} \right\}}} f_{n,s} \left( 0 \right), $$

where $S_{n} = \sum\nolimits_{k = 1}^{n} {X_{k} }$.

To see how the density $f_{n,s}$ works for any value $y \in {\mathbb{R}}$, $y > x$, or how (D.1) is implemented for Borel sets to the right of $x \in {\mathbb{R}}_{ + }$, it is convenient to consider the cumulative distribution function of the statistic $T = \overline{X}_{n}$, or even some moment expressions of $T$. We here confine the estimates to the tail distribution $P\left( {\overline{X}_{n} > x} \right)$ and to the moment expression given by $E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X} > x} \right)} \right]$, $x,u \in {\mathbb{R}}_{ + }$. To expedient the computations, assume that $F_{n,s}$ is absolute continuous (with respect to Lebesgue measure) with density $f_{n,s}$ for all $s \in {\mathbf{S}}$.$\hfill\square$

Lemma D2

For $s \in {\mathbf{S}}$, we have

$$ P\left( {\overline{X}_{n} > x} \right) = e^{{n\left\{ {K\left( s \right) - nx} \right\}}} E_{s} \left[ {e^{{ - \sqrt {nk_{2,s} } sU}} } \right],\;x \in {\mathbb{R}}_{ + } . $$

(D.3)

If $s \in {\mathbf{S}} \cap \left( {0,\infty } \right)$, then

$$ P\left( {\overline{X}_{n} > x} \right) = e^{{n\left\{ {K\left( s \right) - nx} \right\}}} \int_{{\left[ {0,\infty } \right)}} {\sqrt {nk_{2,s} } se^{{ - \sqrt {nk_{2,s} } sy}} \left\{ {F_{n,s} \left( y \right) - F_{n,s} \left( {0_{ - } } \right)} \right\}dy} ,\;x \in {\mathbb{R}}_{ + } , $$

(D.4)

where $U = \sqrt n {{\left( {\overline{X}_{n} - x} \right)} \mathord{\left/ {\vphantom {{\left( {\overline{X}_{n} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}$ and $F_{n,s}$ is the cumulative distribution of $U$ under $P_{s}$.

If $F_{n,s}$ is absolute continuous (with respect to Lebesgue measure) with density $f_{n,s}$ for all $s \in {\mathbf{S}} \cap \left( {0,\infty } \right)$, then,

$$ P\left( {\overline{X}_{n} > x} \right) = e^{{n\left\{ {K\left( s \right) - nx} \right\}}} \int_{{\left[ {0,\infty } \right)}} {e^{{ - \sqrt {nk_{2,s} } sy}} f_{n,s} \left( y \right)dy} . $$

(D.5)

Proof

We have

$$ P\left( {\overline{X}_{n} > x} \right) = \int_{{\left( {nx,\infty } \right)}} {f_{{S_{n} }} \left( y \right)dy = e^{{n\left( {K\left( s \right) - ns} \right)}} \int_{{\left( {nx,\infty } \right)}} {e^{{ - ns\left( {y - x} \right)}} F_{{S_{n} ,s}} \left( {dy} \right)} } . $$

Then, replacing $S_{n} = nx + \sqrt {n\kappa_{2,s} } U$, (D.3) follows immediately. By integration by parts the integral (D.3) results (D.4). The last statement is trivial.

To relate the above lemma with the Edgeworth expansion in Appendix C, the random variable $G = X - \mu_{s}$ is defined under the probability measure $P_{s}$. Note that its characteristic function is defines as

$$ E_{s} \left[ {e^{{it\left( {X - \mu_{s} } \right)}} } \right] = \int_{\mathbb{R}} {e^{{it\left( {x - \mu_{s} } \right)}} e^{sx - K\left( s \right)} f_{X} } \left( x \right)dx = e^{{ - K\left( s \right) - it\mu_{s} + K\left( {s + it} \right)}} = \phi_{s} \left( t \right)e^{{ - it\mu_{s} }} . $$

Then the following lemma is of importance.$\hfill\square$

Lemma D3

For $s \in {\mathbf{S}}$ and $E_{s} \left[ {e^{itU} } \right] \in {\mathbf{L}}_{1} \left( {\mathbb{R}} \right)$,

$$ P\left( {\overline{X}_{n} > x} \right) = \frac{{e^{{n\left\{ {K\left( s \right) - sx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\frac{1}{2\pi }\int_{\mathbb{R}} {\phi_{s}^{n} \left( {{t \mathord{\left/ {\vphantom {t {\sqrt {n\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {n\kappa_{2,s} } }}} \right)\frac{{e^{{i{{t\sqrt n \left( {\mu_{s} - x} \right)} \mathord{\left/ {\vphantom {{t\sqrt n \left( {\mu_{s} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}}} }}{{1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}}}dt} , $$

where $\mu_{s}$ and $\kappa_{2,s}$ are the first and the second cumulants under the measure $P_{s}$.

Proof

Using (D.3) and the inversion formula, Theorem C1, we have

$$ \begin{aligned} P\left( {\overline{X}_{n} > x} \right) = & \frac{{e^{{n\left\{ {K\left( s \right) - nx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\int_{{\left[ {0,x} \right)}} {\sqrt {n\kappa_{2,s} } se^{{ - \sqrt {n\kappa_{2,s} } sy}} f_{n,s} \left( y \right)} dy \\ = & \frac{{e^{{n\left\{ {K\left( s \right) - nx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\int_{{\left[ {0,\infty } \right)}} \sqrt {n\kappa_{2,s} } se^{{ - \sqrt {n\kappa_{2,s} } sy}} \frac{1}{2\pi } \\ &\quad \int_{\mathbb{R}} {e^{ - ity} E_{s} \left[ {\exp \left( {it\frac{{S_{n} - n\mu_{s} }}{{\sqrt {n\kappa_{2,s} } }}} \right)} \right]e^{{it\sqrt n {{\left( {\mu_{s} - x} \right)} \mathord{\left/ {\vphantom {{\left( {\mu_{s} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}}} dtdy} . \\ \end{aligned} $$

In the expression above, Fubini’s theorem justifies the interchanging the order of integration since all points of the integration contour have a positive real part which bounded away from zero. Thus,

$$ \begin{aligned} P\left( {\overline{X}_{n} > x} \right) & = \frac{{e^{{n\left\{ {K\left( s \right) - nx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\frac{1}{2\pi }\int_{\mathbb{R}} {E_{s} \left[ {\exp \left( {i\frac{t}{{\sqrt {n\kappa_{2,s} } }}\left( {S_{n} - n\mu_{s} } \right)} \right)} \right]e^{{it\sqrt n {{\left( {\mu_{s} - x} \right)} \mathord{\left/ {\vphantom {{\left( {\mu_{s} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}}} } \\ &\quad \int_{{\left[ {0,\infty } \right)}} {\sqrt {n\kappa_{2,s} } e^{{ - it\sqrt {n\kappa_{2,s} } s\left( {1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}} \right)y}} dy} dt,\end{aligned} $$

which leads to the desired result after integrating with respect to $y$.

Lastly, the next result of interest is $E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X} > x} \right)} \right]$. Again, the idea here is to express the integral with respect to the characteristic to connect it with Appendix C.$\hfill\square$

Lemma D4

For $s \in {\mathbf{S}} \cap \left( {0,\infty } \right)$ and $E_{s} \left[ {e^{itU} } \right] \in {\mathbf{L}}_{1} \left( {\mathbb{R}} \right)$,

$$ E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X} > x} \right)} \right] = \frac{s}{s + u}\frac{{e^{{n\left\{ {K\left( s \right) - sx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\frac{1}{2\pi }\int_{\mathbb{R}} {\phi_{s}^{n} \left( {{t \mathord{\left/ {\vphantom {t {\sqrt {n\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {n\kappa_{2,s} } }}} \right)\frac{{e^{{i{{t\sqrt n \left( {\mu_{s} - x} \right)} \mathord{\left/ {\vphantom {{t\sqrt n \left( {\mu_{s} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}}} }}{{1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}}}dt} ,\;x,u \in {\mathbb{R}}_{ + } . $$

Proof

We have

$$ \begin{aligned} E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X} > x} \right)} \right] & = \int_{{\left( {x,\infty } \right)}} {e^{{ - nu\left( {y - x} \right)}} f_{{s_{n} }} \left( y \right)dy = P\left( {\overline{X}_{n} > x} \right)} \\ &\quad - nu\int_{{\left( {x,\infty } \right)}} e^{{ - nu\left( {y - x} \right)}} P\left( {S_{n} > ny} \right)dy .\end{aligned} $$

Replacing from Lemma B2 the $P\left( {\overline{X}_{n} > y} \right)$ and using the Fubini’s theorem, we have

$$ \begin{aligned} E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X} > x} \right)} \right] = & \frac{{e^{{n\left\{ {K\left( s \right) - sx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\left\{ {1 - nu\int_{{\left( {x,\infty } \right)}} {e^{{ - n\left( {u + s} \right)\left( {y - x} \right)}} dy} } \right\}\frac{1}{2\pi } \\ &\quad \int_{\mathbb{R}} {\phi_{s}^{n} \left( {{t \mathord{\left/ {\vphantom {t {\sqrt {n\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {n\kappa_{2,s} } }}} \right)\frac{{e^{{i{{t\sqrt n \left( {\mu_{s} - x} \right)} \mathord{\left/ {\vphantom {{t\sqrt n \left( {\mu_{s} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}}} }}{{1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}}}dt} \\ = & \frac{s}{s + u}\frac{{e^{{n\left\{ {K\left( s \right) - sx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\frac{1}{2\pi }\int_{\mathbb{R}} {\phi_{s}^{n} \left( {{t \mathord{\left/ {\vphantom {t {\sqrt {n\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {n\kappa_{2,s} } }}} \right)\frac{{e^{{i{{t\sqrt n \left( {\mu_{s} - x} \right)} \mathord{\left/ {\vphantom {{t\sqrt n \left( {\mu_{s} - x} \right)} {\sqrt {\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {\kappa_{2,s} } }}}} }}{{1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}}}dt} . \\ \end{aligned} $$

This proves the lemma.$\hfill\square$

Appendix E

5.1 Saddle approximation

For $T = \overline{X}_{n}$. Here, the parameter $s$ is chosen via the saddlepoint, which in return simplifies the number of computations seen in Appendix D. Thus, let $\hat{s}: = \hat{s}\left( x \right)$, that is determined by the equation $K^{\prime}\left( s \right) = x = \mu_{s}$. When the exponential family is regular, that is, ${\mathbf{S}}$ is open, which implies that the exponential family is steep. When the family is neither regular nor steep, the equation $K^{\prime}\left( s \right) = x$, cab still solved for values of $x \in {\text{int}} {\mathbf{S}}$ (see, e.g., Barndorff-Nielsen (1978, p.153). To obtain the firs order saddlepoint approximations, we simply replace $\phi_{s}^{n} \left( {{t \mathord{\left/ {\vphantom {t {\sqrt {n\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {n\kappa_{2,s} } }}} \right)$ by the approximation as shown in (C.2). Consider the

$$ \frac{1}{2\pi }\int_{\mathbb{R}} {\phi_{s}^{n} \left( {{t \mathord{\left/ {\vphantom {t {\sqrt {n\kappa_{2,s} } }}} \right. \kern-0pt} {\sqrt {n\kappa_{2,s} } }}} \right)\frac{1}{{1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}}}dt} \approx \frac{1}{2\pi }\int_{\mathbb{R}} {e^{{ - {{\kappa_{2,s} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2,s} t^{2} } 2}} \right. \kern-0pt} 2}}} \left\{ {1 + \sum\nolimits_{j = 3}^{m} {n^{{ - {{\left( {s - 2} \right)} \mathord{\left/ {\vphantom {{\left( {s - 2} \right)} 2}} \right. \kern-0pt} 2}}} P_{j,s} \left( {it} \right)} } \right\}} \frac{dt}{{1 + \tfrac{it}{{\sqrt {n\kappa_{2,s} } s}}}}, $$

(E.1)

where $P_{j,s}$ are again orthogonal polynomial of the same form as in (C.2), but now are indexed by the parameter s. In any event, $P_{j,s}$ still include expressions of a constant term multiplied by $\left( {it} \right)^{k}$, $k \in {\mathbb{N}}$. Thus, the goal is to first evaluate the integrals of the form (see, e.g., Jensen 1995)

$$ \frac{{B\left( {\kappa_{2,s} \lambda } \right)}}{{\kappa_{2,s}^{k + 1} }} = \frac{1}{2\pi }\int_{\mathbb{R}} {e^{{ - {{\kappa_{2,s} t^{2} } \mathord{\left/ {\vphantom {{\kappa_{2,s} t^{2} } 2}} \right. \kern-0pt} 2}}} } \frac{{\left( {it} \right)^{k} dt}}{{1 + \tfrac{it}{\lambda }}}, $$

before (C.1) is estimated.

Lemma E1

(Jensen 1995, p.24).

$$ B_{k} \left( \lambda \right) = \frac{1}{2\pi }\int_{\mathbb{R}} {e^{{ - {{t^{2} } \mathord{\left/ {\vphantom {{t^{2} } 2}} \right. \kern-0pt} 2}}} \frac{{\left( {it} \right)^{k} dt}}{{1 + \tfrac{it}{\lambda }}}} = \left. {\left( { - 1} \right)^{k} \frac{{d^{k} }}{{dx^{k} }}\left\{ {\lambda e^{{ - \tfrac{{\lambda^{2} }}{2} + \lambda x}} \overline{\Phi }\left( {\lambda + x} \right)} \right\}} \right|_{x = 0} ,\;k \in {\mathbb{N}}_{0} , $$

where $\overline{\Phi }\left( x \right) = 1 - \Phi \left( x \right) = 1 - \frac{1}{{\sqrt {2\pi } }}\int_{{\left( { - \infty ,x} \right]}} {e^{{ - {{x^{2} } \mathord{\left/ {\vphantom {{x^{2} } 2}} \right. \kern-0pt} 2}}} dx}$.

To have a clear sense of the approximations derived below, we will primarily deal with the following terms,$B_{0} ,B_{3} ,B_{4}$ and $B_{6}$. Thus, some additional details of $B_{j}$’s expression with respect to $\lambda = \sqrt {n\kappa_{2,s} } s$ are necessary to identify their magnitude in relation to size $n$.

$$ B_{0} \left( \lambda \right) = \lambda \exp \left( {{{\lambda^{2} } \mathord{\left/ {\vphantom {{\lambda^{2} } 2}} \right. \kern-0pt} 2}} \right)\overline{\Phi }\left( \lambda \right), $$

$$ B_{3} \left( \lambda \right) = - \left\{ {\lambda^{3} B_{0} \left( \lambda \right) - \left( {\lambda^{3} - \lambda } \right)\left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} } \right\}, $$

$$ B_{4} \left( \lambda \right) = \lambda^{4} B_{0} \left( \lambda \right) - \left( {\lambda^{4} - \lambda^{2} } \right)\left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} ,\;{\text{and}} $$

$$ B_{6} \left( \lambda \right) = \lambda^{6} B_{0} \left( \lambda \right) - \left( {\lambda^{6} - \lambda^{4} + 3\lambda^{2} } \right)\left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} . $$

In addition, an expansion of the normal tail distribution will reveal how the $B_{j}$’s act with respect to $n \to \infty$.

Lemma E2

(Gradshteyn and Ryzhik 2000, 8.254). For $n \to \infty$, we have that, for $\lambda = \sqrt {n\kappa_{2,s} } s$,

$$ \overline{\Phi }\left( \lambda \right) = \frac{\varphi \left( \lambda \right)}{\lambda }\left\{ {\sum\nolimits_{k = 0}^{m} {\frac{{\left( { - 1} \right)^{k} \left( {2k - 1} \right)!!}}{{\lambda^{2k} }} + O\left( {\lambda^{ - 2m} } \right)} } \right\}. $$

Further, as $\lambda = \sqrt {n\kappa_{2,s} } s \to \infty$

$$ B_{0} \left( \lambda \right) \to \left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} ,B_{4} \left( \lambda \right) \to 3\left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \;and\;B_{6} \left( \lambda \right) \to - 15\left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} . $$

Also, ${{B_{3} \left( \lambda \right)} \mathord{\left/ {\vphantom {{B_{3} \left( \lambda \right)} \lambda }} \right. \kern-0pt} \lambda } \to - 3\left( {2\pi } \right)^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}}$.

To estimate (E.1) or more specific the tail distribution of the sample means in Lemma B3 and the restricted moments $E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X} > x} \right)} \right]$ in Lemma D4, the use of Lemma E1 is adopted to provide the following.

Lemma E3

For $s \in {\mathbf{S}} \cap \left( {0,\infty } \right)$ and $x,u \in {\mathbb{R}}_{ + }$,

$$ \begin{gathered} P\left( {\overline{X}_{n} > x} \right) = \frac{{e^{{n\left\{ {K\left( s \right) - nx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\left\{ {B_{0} \left( \lambda \right) + \frac{{\lambda_{3,s} }}{\sqrt n 3!}B_{3} \left( \lambda \right) + \frac{1}{n}\left[ {\frac{{\lambda_{4,s} }}{4!}B_{4} \left( \lambda \right) + \frac{{\lambda_{3,s}^{2} }}{{\left( {3!} \right)^{2} 2!}}B_{6} \left( \lambda \right)} \right] + O\left( {B_{0} \left( \lambda \right)n^{{ - {3 \mathord{\left/ {\vphantom {3 2}} \right. \kern-0pt} 2}}} } \right)} \right\} \hfill \\ E\left[ {e^{{ - u\left( {\overline{X}_{n} - x} \right)}} I\left( {\overline{X}_{n} > x} \right)} \right] = \frac{{e^{{n\left\{ {K\left( s \right) - nx} \right\}}} }}{{\sqrt {n\kappa_{2,s} } s}}\frac{s}{s + u} \hfill \\ \left\{ {B_{0} \left( \lambda \right) + \frac{{\lambda_{3,s} }}{\sqrt n 3!}B_{3} \left( \lambda \right) + \frac{1}{n}\left[ {\frac{{\lambda_{4,s} }}{4!}B_{4} \left( \lambda \right) + \frac{{\lambda_{3,s}^{2} }}{{\left( {3!} \right)^{2} 2!}}B_{6} \left( \lambda \right)} \right] + O\left( {B_{0} \left( \lambda \right)n^{{ - {3 \mathord{\left/ {\vphantom {3 2}} \right. \kern-0pt} 2}}} } \right)} \right\}, \hfill \\ \end{gathered} $$

where $\lambda = \sqrt {n\kappa_{2,s} } s$.

Remark

The error term can be sharpened when the parameter $s$ is bounded away from zero. Further, from Lemma C1 it can be shown that $\frac{{\lambda_{3,s} }}{\sqrt n 3!}B_{3} \left( {\sqrt {n\kappa_{2,s} } s} \right) = O\left( {n^{ - 1} } \right)$ for $s \in {\mathbf{S}} \cap \left( {0,\infty } \right)$ and $s > \varepsilon$ for positive fixed $\varepsilon$, such that the saddle point approximation, corresponding to $B_{0}$ term in Lemma C2 has relative error $O\left( {n^{ - 1} } \right)$ for $s > \varepsilon$.