Preliminaries

Abstract

In this chapter, we address and review key concepts that will be used in later chapters. In Sect. 1.1, we review briefly the key notions of probability functions and moments of random vectors. Then, we introduce step and impulse functions, which will prove useful in describing and analyzing non-linear statistics, in Sect. 1.2. Section 1.3 is devoted to the discussion on signs and magnitudes of random variables, which will play an important role in describing and understanding order and rank statistics. In the discussion, we will find interesting applications of the step and impulse functions. In Sect. 1.4, we introduce concisely the notions of order statistics, magnitude order statistics, ranks, and magnitude ranks of random vectors, which are the themes of the following chapters.

Appendices

Appendix 1: Inverse Cumulative Distribution Functions

The inverse cdf $F^{-1}$, the inverse function of a cdf F, can be defined as [11, 21]

$$\displaystyle \begin{aligned} \begin{array}{rcl} F^{-1} (u) \ = \ \inf \{ x: \, F(x) \ge u \} {} \end{array} \end{aligned} $$

(1.A1.1)

for $0 < u < 1$. Because a cdf is a non-decreasing function, we have $\big \{ x: \, F(x) \ge y_2 \big \} \subseteq \left \{ x: \, F(x) \ge y_1 \right \}$ when $y_1 < y_2$ and, consequently, $\inf \big \{ x: F(x) \ge y_1 \big \} \le \inf \big \{ x: F(x) \ge y_2 \big \}$. Thus,

$$\displaystyle \begin{aligned} \begin{array}{rcl} F^{-1} \left ( y_1 \right ) \ \le \ F^{-1} \left ( y_2 \right ) {} \end{array} \end{aligned} $$

(1.A1.2)

for $y_1 < y_2$. In other words, like a cdf, an inverse cdf is a non-decreasing function.

Theorem 1.A1.1

Let F and$F^{-1}$be a cdf and its inverse, respectively. Then,

$$\displaystyle \begin{aligned} \begin{array}{rcl} F \left ( F^{-1} (u) \right ) \ \ge \ u {} \end{array} \end{aligned} $$

(1.A1.3)

for$0 < u < 1$. In addition, if F is continuous,

$$\displaystyle \begin{aligned} \begin{array}{rcl} F \left ( F^{-1} (u) \right ) \ = \ u. {} \end{array} \end{aligned} $$

(1.A1.4)

Proof

Let $S_u$ be the set of all x such that $F(x) \ge u$, and $x_L$ be the smallest number in $S_u$. Then, because a cdf is continuous from the right-hand side, $S_u$ is in the form $S_u = \left [ x_L, \infty \right )$ or $S_u = \left \{ x_L, x_L +1, \, \cdots \right \}$ when the cdf is continuous or of a step type, respectively. Thus,

$$\displaystyle \begin{aligned} \begin{array}{rcl} F^{-1} (u) \ = \ x_L {} \end{array} \end{aligned} $$

(1.A1.5)

and

$$\displaystyle \begin{aligned} \begin{array}{rcl} F \left ( x_L \right ) \ \ge \ u {} \end{array} \end{aligned} $$

(1.A1.6)

because $F(x) \ge u$ for any point $x \in S_u$. In addition, because $S_u$ is the set of all x such that $F(x) \ge u$, we have $F(x) < u$ for any point $x \in S_u^c$. Now, $x_L$ is the smallest number in $S_u$ and thus $x_L - \varepsilon \in S_u^c$ when $\varepsilon > 0$. Consequently, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} F \left ( x_L - \varepsilon \right ) < u {} \end{array} \end{aligned} $$

(1.A1.7)

when $\varepsilon > 0$.

The inequality (1.A1.3) is easily proved from (1.A1.5) in (1.A1.6). Next, using (1.A1.5) into the inequalities (1.A1.6) and (1.A1.7), we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} F \left ( F^{-1} (u) - \varepsilon \right ) \ < \ u \ \le \ F \left ( F^{-1} (u) \right ) {} \end{array} \end{aligned} $$

(1.A1.8)

when $\varepsilon >0$: that is, u is a number between $F \left ( F^{-1} (u) - \varepsilon \right )$ and $F \left ( F^{-1} (u) \right )$. Thus, if F is a continuous function, $u=F \left ( F^{-1} (u) \right )$ because $\lim \limits _{\varepsilon \to 0} F \big ( F^{-1} (u) - \varepsilon \big ) = F \left ( F^{-1} (u) \right )$. $\spadesuit$

Example 1.A1.1

Consider the cdf

$$\displaystyle \begin{aligned} \begin{array}{rcl} F_1(x) \ = \ \left\{\begin{array}{llll} 0, & x \le 0; & \quad \frac{x}{3}, & 0 \le x < 1; \\ \\ \frac{1}{2} , & 1 \le x \le 2; & \quad \frac{1}{2} (x-1), & 2 \le x \le 3; \\ \\ 1, & x \ge 3. \end{array} \right. \end{array} \end{aligned} $$

(1.A1.9)

Then, the inverse cdf is

$$\displaystyle \begin{aligned} \begin{array}{rcl} F_1^{-1}(u) \ = \ \left\{\begin{array}{llll} 3u, & 0 < u \le \frac{1}{3}; & \quad 1, & \frac{1}{3} \le u \le \frac{1}{2} ; \\ \\ 2u+1, & \frac{1}{2} < u < 1 . \end{array} \right. \end{array} \end{aligned} $$

(1.A1.10)

For example, we have $F_1^{-1} \left ( \frac {1}{3} \right ) = \inf \left \{ x: \, F_1(x) \ge \frac {1}{3} \right \} = \inf \left \{ x: \, x \ge 1 \right \} = 1$ and $F_1^{-1} \left ( \frac {1}{2} \right ) = \inf \left \{ x: \, F_1(x) \ge \frac {1}{2} \right \} = \inf \left \{ x: \, x \ge 1 \right \} = 1$. Note that, while a cdf is continuous from the right-hand side, an inverse cdf is continuous from the left-hand side. Figure 1.17 shows the cdf $F_1$ and its inverse $F_1^{-1}$. $\diamondsuit$

Fig. 1.17

The cdf $F_1$ and the inverse cdf $F_1^{-1}$

It is noteworthy that we do not have $F^{-1} (F (x))= x$ but

$$\displaystyle \begin{aligned} \begin{array}{rcl} F^{-1} (F (x)) \ \le \ x {} \end{array} \end{aligned} $$

(1.A1.11)

even if F is a continuous function. The strict inequality in (1.A1.11) holds true when the cdf $F(x)$ is constant over an interval: for example, in Fig. 1.17, we have $F^{-1} (F (x)) < x$ over the interval $(1, 2]$ of x. In essence, for the inequality (1.A1.11), it should be noted that

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{P} \left ( F^{-1} (F (X)) \ne X \right ) \ = \ 0 {} \end{array} \end{aligned} $$

(1.A1.12)

when $X \sim F$. In addition, we have $F^{-1}(u) \le F^{-1}(F(x)) \le x$ if $u \le F(x)$ from the inequality (1.A1.11), and $u \le F\left (F^{-1}(u)\right ) \le F(x)$ if $F^{-1}(u) \le x$ from the inequality (1.A1.3): in short,

$$\displaystyle \begin{aligned} \begin{array}{rcl} u \le F(x) \ \ \rightleftarrows \ \ F^{-1}(u) \le x . {} \end{array} \end{aligned} $$

(1.A1.13)

Theorem 1.A1.2

For a cdf F with a continuous pdf$f(x) = \frac {d}{dx} F(x)$, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac {d}{du} F^{-1} (u) \ = \ \frac {1}{f \left ( F^{-1} (u) \right )} {} \end{array} \end{aligned} $$

(1.A1.14)

if$f \left ( F^{-1} (u) \right ) \ne 0$.

Proof

When the pdf f is continuous, the cdf F is also continuous. Letting $F^{-1} (u) =v$, we get $\frac {d}{du} F^{-1} (u) = \frac {1}{f(v)} \frac {dv}{dv}$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac {d}{du} F^{-1} (u) & =&\displaystyle \frac {1}{f \left ( F^{-1} (u) \right )} {} \end{array} \end{aligned} $$

(1.A1.15)

because $F(v)=F\left ( F^{-1} (u) \right ) = u$ and $du = f(v) dv$ from (1.A1.4). $\spadesuit$

Example 1.A1.2

Find the function which transforms a random variable X with cdf $F_X$ into another random variable Y  with cdf $F_Y$.

Solution

Assume for brevity that both the cdf $F_X$ and the inverse $F_Y^{-1}$ of the cdf $F_Y$ are continuous and increasing. Then, for the random variable

$$\displaystyle \begin{aligned} \begin{array}{rcl} Z \ = \ F_X(X), \end{array} \end{aligned} $$

(1.A1.16)

we have $X = F_X^{-1}(Z)$ and $\left \{ F_X (X) \le z \right \} = \left \{ X \le F_X^{-1} (z) \right \}$ because $F_X$ is continuous and increasing. Therefore, the cdf of Z is $F_Z(z) = \mathsf {P} ( Z\le z ) = \mathsf {P} \left ( F_X (X) \le z \right ) = \mathsf {P} \left ( X \le F_X^{-1} (z) \right ) = F_X \left (F_X^{-1} (z) \right ) = z$for $0 < z < 1$ using (1.A1.4) because $F_X$ is a continuous function. In other words, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} Z \ \sim \ U(0,1) . {} \end{array} \end{aligned} $$

(1.A1.17)

Next, consider the random variable

$$\displaystyle \begin{aligned} \begin{array}{rcl} V \ = \ F_{Y}^{-1}(Z) . {} \end{array} \end{aligned} $$

(1.A1.18)

Then, we get the cdf $\mathsf {P} ( V\le y ) = \mathsf {P} \left ( F_{Y}^{-1}(Z) \le y \right ) = \mathsf {P} \left ( Z \le F_Y(y) \right ) = F_Z\left ( F_Y(y) \right ) = F_Y(y)$ of V  from $F_Z (x) = x$ for $x \in (0,1)$. In short, when $X \sim F_X$, we have $F_{Y}^{-1} \left ( F_X(X) \right ) \sim F_Y$. $\diamondsuit$

The essence of Example 1.A1.2 is summarized as a theorem below.

Theorem 1.A1.3

The function$g = F_{Y}^{-1} \circ F_X$transforms a random variable X with cdf$F_X$into a random variable with cdf$F_Y$.

Figure 1.18 illustrates some of the interesting results such as

$$\displaystyle \begin{aligned} \begin{array}{rcl} X \sim F_X & \rightarrow &\displaystyle F_X(X) \sim U(0, 1), {} \end{array} \end{aligned} $$

(1.A1.19)

$$\displaystyle \begin{aligned} \begin{array}{rcl} Z \sim U(0,1) & \rightarrow &\displaystyle F_Y^{-1}(Z) \sim F_Y, {} \end{array} \end{aligned} $$

(1.A1.20)

and

$$\displaystyle \begin{aligned} \begin{array}{rcl} X \sim F_X & \rightarrow &\displaystyle F_Y^{-1} \left ( F_X (X) \right ) \sim F_Y . {} \end{array} \end{aligned} $$

(1.A1.21)

Theorem 1.A1.3 can be used in the generation of random numbers, for instance.

Fig. 1.18

Transformation of a random variable $X \sim F_X$ into $Z \sim U[0,1)$ and then into $V \sim F_Y$

Example 1.A1.3

From $X \sim U(0, 1)$, obtain the Rayleigh random variable $Y \sim f_Y ( y) = \frac {y}{\alpha ^2} \exp \left ( - \frac {y^2}{2 \alpha ^2} \right ) u(y)$.

Solution

From $F_Y (y) = \left \{ 1 - \exp \left ( -\frac {y^2}{2 \alpha ^2} \right ) \right \} u(y)$, we have $g(x) = F_Y^{-1} (x) = \sqrt { -2 \alpha ^2 \ln (1-x) }$. In other words, if $X \sim U(0, 1)$, then $Y = \sqrt { -2 \alpha ^2 \ln (1-X) }$ has the cdf $F_Y (y) = \left \{ 1 - \exp \left ( -\frac {y^2}{2 \alpha ^2} \right ) \right \} u(y)$. Note that we conversely have $V = 1 - \exp \left ( -\frac {Y^2}{2 \alpha ^2} \right ) \sim U(0,1)$. $\diamondsuit$

Example 1.A1.4

Assume$X \sim U(0,1)$. Then, the integer Y  satisfying

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sum_{k=0}^{Y-1} p_k \ < \ X \ \le \ \sum_{k=0}^{Y} p_k \end{array} \end{aligned} $$

(1.A1.22)

with $p_0=0$ is the random variable having the pmf $p_Y \left ( y_n \right ) = \mathsf {P} \left ( Y = y_n \right ) = p_n$ for $n \in \mathbb {J}_{1,\infty }$ and 0 otherwise. $\diamondsuit$

Appendix 2: Quadratic Formulas and Standard Normal Distribution

For the quadratic function

$$\displaystyle \begin{aligned} \begin{array}{rcl} Q( \boldsymbol{x} )\ =\ \sum_{j=1}^{n} \sum_{i=1}^{n} a_{ij} x_i x_j {} \end{array} \end{aligned} $$

(1.A2.1)

of $\boldsymbol {x} = \left ( x_1 , x_2 , \, \cdots , x_n \right )$, consider [9, 21]

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\bar Q}_n \ = \ \int_{0}^{\infty} \int_{0}^{\infty} \cdots \int_{0}^{\infty} \exp\{-Q( \boldsymbol{x} )\} \, d\boldsymbol{x} , {} \end{array} \end{aligned} $$

(1.A2.2)

where $d\boldsymbol {x} = dx_1 dx_2 \cdots dx_n$. When $n=1$ with $Q( \boldsymbol {x} )= a_{11} x_1^2$, we easily get

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\bar Q}_1 \ =\ \frac{\sqrt{\pi}}{2\sqrt{a_{11}}} {} \end{array} \end{aligned} $$

(1.A2.3)

for $a_{11} >0$. When $n=2$, assume $Q( \boldsymbol {x} )= a_{11} x_1^2 + a_{22} x_2^2+ 2a_{12} x_1 x_2$ with $a_{11} >0$, $a_{22} >0$, and $\varDelta _2 = a_{11}a_{22}-a_{12}^2 >0$. We then get

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\bar Q}_2\ =\ \frac{ 1 } { \sqrt{\varDelta_2} } \left ( \frac{\pi}{2} - \tan^{-1} \frac{a_{12}} { \sqrt{\varDelta_2} } \right ). {} \end{array} \end{aligned} $$

(1.A2.4)

In addition, when $n=3$, assume $Q( \boldsymbol {x} )= a_{11} x_1^2 + a_{22} x_2^2 + a_{33} x_3^2 + 2a_{12} x_1 x_2 + 2a_{23} x_2 x_3 + 2a_{31} x_3 x_1$ with $\left \{ a_{ii} >0 \right \}_{i=1}^{3}$ and $\varDelta _3 = a_{11}a_{22}a_{33} -a_{11}a_{23}^2-a_{22}a_{31}^2-a_{33}a_{12}^2+2a_{12}a_{23}a_{31} >0$. Then, we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\bar Q}_3 & =&\displaystyle \frac{ \sqrt{\pi} } { 4 \sqrt{\varDelta_3} } \left ( \frac{\pi}{2} + \sum^{c}\tan^{-1} \frac{ a_{ij}a_{ki}-a_{ii}a_{jk} } { \sqrt{a_{ii}\varDelta_3 } } \right ) {} \end{array} \end{aligned} $$

(1.A2.5)

after some manipulations, where $\sum \limits ^c$ denotes the cyclic sum: for example, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sum^c \tan^{-1} \rho_{ij} \ = \ \tan^{-1} \rho_{12} + \tan^{-1} \rho_{23}+ \tan^{-1}\rho_{31} . {} \end{array} \end{aligned} $$

(1.A2.6)

Recall the gamma function defined as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varGamma(\alpha) \ = \ \int_0^{\infty} x^{\alpha-1} e^{-x} dx {} \end{array} \end{aligned} $$

(1.A2.7)

for¹²$\alpha >0$. The gamma function is a generalization of the factorial and satisfies

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varGamma(n) \ = \ (n-1)! {} \end{array} \end{aligned} $$

(1.A2.8)

for $n \in \mathbb {J}_{1,\infty }$ and

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varGamma(\alpha) \ = \ (\alpha -1)\varGamma(\alpha-1) . {} \end{array} \end{aligned} $$

(1.A2.9)

With (1.A2.9), the value $\varGamma (\alpha )$ can be expressed and evaluated as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varGamma (\alpha) \ = \ \frac{\varGamma (\alpha+n)}{\alpha (\alpha+1)\, \cdots (\alpha+n-1) } \end{array} \end{aligned} $$

(1.A2.10)

for $\alpha \in (-n , -n+1)$, where $n \in \mathbb {J}_{1,\infty }$. Note that we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varGamma\left ( \frac{1}{2} \right ) \ = \ \sqrt{\pi} {} \end{array} \end{aligned} $$

(1.A2.11)

by letting $x= \frac {1}{2}$ in the Euler reflection formula [21]

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varGamma ( 1-x )\varGamma (x) \ = \ \frac{\pi}{\sin\pi x} {} \end{array} \end{aligned} $$

(1.A2.12)

for $0 < x < 1$. Figure 1.19 shows the gamma function $\varGamma (\alpha )$ for $\alpha \in \mathbb {R}$.

Fig. 1.19

The gamma function: the minimum for $\alpha >0$ is $\varGamma _0 \approx 0.8856$ at $\alpha =\alpha _0 \approx 1.4616$

Now, consider the standard normal pdf

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varphi (x) \ = \ \frac{1}{\sqrt{2\pi}} \, \exp \left(- \frac{x^2}{2}\right) {} \end{array} \end{aligned} $$

(1.A2.13)

and the standard normal cdf

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varPhi(x) \ = \ \int_{-\infty}^x \varphi (t) \, dt {} \end{array} \end{aligned} $$

(1.A2.14)

introduced in (1.1.68) and (1.1.69), respectively. Using $\varGamma \left ( \frac {1}{2} \right ) = \sqrt {\pi }$ shown in (1.A2.11), we have $\int _{-\infty }^{\infty } \exp \left (-\alpha x^2 \right ) dx = 2 \int _{0}^{\infty } \frac { e^{-t} } {2 \sqrt {\alpha t }} dt = \frac {1}{ \sqrt {\alpha } } \varGamma \left ( \frac {1}{2} \right )$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \int_{-\infty}^{\infty} \exp \left (-\alpha x^2 \right ) dx & =&\displaystyle \sqrt{\frac{\pi}{\alpha}} , {} \end{array} \end{aligned} $$

(1.A2.15)

which can also be obtained from $\int _{-\infty }^{\infty } \frac {\sqrt {\alpha }} {\sqrt {\pi }} \exp \left ( - \alpha x^2 \right ) dx = 1$. Based on (1.A2.3) or on (1.A2.15), we get $\int _{-\infty }^{\infty } \varphi ^m (x) dx = \frac {1}{ ( 2\pi )^{\frac {m}{2}} } \int _{-\infty }^{\infty } \exp \left (- \frac {m x^2}{2}\right ) dx$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \int_{-\infty}^{\infty} \varphi^m (x) \, dx & =&\displaystyle \frac{1}{\sqrt{m (2\pi)^{m-1}}} . {} \end{array} \end{aligned} $$

(1.A2.16)

Differentiating (1.A2.15) k times with respect to $\alpha$, we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} \int_{-\infty}^{\infty} x^{2k} \exp \left (-\alpha x^2 \right ) dx \ = \ \frac{ (2k-1) !!}{2^k \alpha^k} \sqrt{\frac{\pi}{\alpha}} {} \end{array} \end{aligned} $$

(1.A2.17)

for $k \in \mathbb {J}_{1,\infty }$, where

$$\displaystyle \begin{aligned} \begin{array}{rcl} (2k-1) !! \ = \ (2k-1) (2k-3) \times \cdots \times 3 \times 1 {} \end{array} \end{aligned} $$

(1.A2.18)

for $k \in \mathbb {J}_{1,\infty }$. In obtaining (1.A2.17), the following theorem [19] is used.

Theorem 1.A2.1

Assume that$a(x)$and$b(x)$are integrable functions and that both$g(t,x)$and$\frac {\partial }{\partial x} g(t,x)$are continuous in x and t. Then, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{d}{d x} \int_{a(x)}^{b(x)} g(t,x) dt & =&\displaystyle g( b(x),x) \frac{d b(x)}{dx} - g( a(x),x) \frac{d a(x)}{dx} \\ & &\displaystyle \quad + \int_{a(x)}^{b(x)} \frac{\partial g(t,x)}{\partial x} dt , {} \end{array} \end{aligned} $$

(1.A2.19)

which is called Leibnitz’s rule.

The formula (1.A2.17) can also be obtained as $\int _{-\infty }^{\infty } x^{2k} \exp (-\alpha x^2) dx = 2 \int _{0}^{\infty } \left (\frac {t}{\alpha }\right )^{k} e^{-t} \frac { dt}{2 \sqrt {\alpha t }} = \frac {1}{ \alpha ^k \sqrt {\alpha }}\varGamma \left (k+ \frac {1}{2} \right ) = \frac {(2k-1)!!}{2^k\alpha ^k \sqrt {\alpha }} \varGamma \left ( \frac {1}{2} \right )$from $\varGamma \left (k+ \frac {1}{2} \right ) = \left (k- \frac {1}{2} \right ) \varGamma \left (k- \frac {1}{2} \right ) =\left (k- \frac {1}{2} \right )\left (k-\frac {3}{2} \right ) \cdots \frac {1}{2} \varGamma \left ( \frac {1}{2} \right ) = \frac {(2k-1)!!}{2^k} \varGamma \left ( \frac {1}{2} \right )$. Based on the even symmetry of the pdf $f(x)$ of $\mathcal {N} \left ( 0, \sigma ^2 \right )$ and (1.A2.17), we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{E} \left \{ X^n \right \} \ = \ \left\{\begin{array}{ll} 0, & ~ n\mbox{ is odd},\\ (n-1)!! \, \sigma^n, & ~ n\mbox{ is even} \end{array} \right. {} \end{array} \end{aligned} $$

(1.A2.20)

for a random variable $X \sim \mathcal {N} \left ( 0, \sigma ^2 \right )$: for instance, $\mathsf {E}\left \{X^4\right \} = 3 \sigma ^4$ and $\mathsf {E}\left \{X^6\right \} = 15 \sigma ^6$.

Now, we easily get $\int _{0}^{\infty } x^n \exp \left ( -a^2 x^2 \right ) dx = \int _{0}^{\infty } \frac {t^{\frac {n}{2}}} {a^n} \frac {e^{-t} }{2 a \sqrt { t }}dt = \frac {1}{ 2 a^{n+1} } \int _{0}^{\infty } t^{\frac {n-1}{2}}$$e^{-t}dt$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \int_{0}^{\infty} x^n \exp\left ( -a^2 x^2 \right ) dx & =&\displaystyle \frac{1}{ 2 a^{n+1} } \varGamma\left(\frac{n+1}{2} \right) . {} \end{array} \end{aligned} $$

(1.A2.21)

Thus, when $n=2k+1$ and $k\in \mathbb {J}_{0,\infty }$, we get $\mathsf {E} \left \{|X|{ }^n \right \} = \int _{-\infty }^{\infty } |x|{ }^{n} f (x) dx = \frac {\sqrt {2}}{\sqrt {\pi \sigma ^2}} \int _{0}^{\infty } x^{2k+1} \exp \left ( - \frac {x^2}{2 \sigma ^2}\right ) dx$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{E} \left \{|X|{}^{2k+1} \right \} \ = \ 2^k k! \sqrt{\frac{2}{\pi}} \sigma^{2k+1} , {} \end{array} \end{aligned} $$

(1.A2.22)

which can be expressed also as $\mathsf {E} \left \{|X|{ }^n \right \} = 2^{\frac {n-1}{2}} \varGamma \left (\frac {n+1}{2} \right ) \sqrt {\frac {2}{\pi }} \sigma ^{n}$ for n an odd number.

Next, consider

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_n (a) & =&\displaystyle 2 \pi \ \int_{-\infty}^{\infty} \varPhi^n(ax) \ \varphi^2(x) \, dx \\ & =&\displaystyle \int_{-\infty}^{\infty} \varPhi^n(ax) \ \exp \left (-x^2 \right ) \, dx {} \end{array} \end{aligned} $$

(1.A2.23)

for $n\in \mathbb {J}_{0,\infty }$. Letting $n=0$ in (1.A2.23), we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_0 (a) \ = \ \sqrt{\pi} {} \end{array} \end{aligned} $$

(1.A2.24)

using (1.A2.16), and letting $a=0$ in (1.A2.23), we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_n (0) \ = \ \frac { \sqrt{\pi} }{ 2^{n} } . {} \end{array} \end{aligned} $$

(1.A2.25)

For $m \in \mathbb {J}_{0,\infty }$, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \int_{-\infty}^{\infty} \left \{ \varPhi(ax) - \frac{1}{2} \right \}^{2m+1} \ \exp \left (-x^2 \right ) \, dx \ = \ 0 {} \end{array} \end{aligned} $$

(1.A2.26)

because $(2m+1)$ is an odd number, $\varPhi (ax) - \frac {1}{2}$ is an odd symmetric function of x, and $\exp \left (-x^2 \right )$ is an even symmetric function of x. From (1.A2.23) and $\left \{ \varPhi (ax) - \frac {1}{2} \right \}^{2m+1}= \sum \limits _{i=0}^{2m+1} \left ( - \frac {1}{2} \right )^i \, _{2m+1}\mbox{C}_{i} \varPhi ^{2m+1-i} (ax)$, the result (1.A2.26) can subsequently be expressed as $\sum \limits _{i=0}^{2m+1} \left ( - \frac {1}{2} \right )^i \, _{2m+1}\mbox{C}_{i}\tilde {I}_{2m+1-i} (a) = 0$ after some steps. This result in turn can be rewritten as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_{2m+1} (a) \ = \ \sum_{i=1}^{2m+1} 2^{-i}(-1)^{i+1}\, _{2m+1}\mbox{C}_{i} \ \tilde {I}_{2m+1-i} (a) {} \end{array} \end{aligned} $$

(1.A2.27)

for $m \in \mathbb {J}_{0,\infty }$.

Then, when $m=0$, we get $\tilde {I}_1 (a) = \frac {1}{2} \tilde {I}_0 (a)$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_1 (a) & =&\displaystyle \frac{\sqrt{\pi}}{2} {} \end{array} \end{aligned} $$

(1.A2.28)

from (1.A2.24) and (1.A2.27). Similarly, when $m=1$, from (1.A2.24), (1.A2.27), and (1.A2.28), we get $\tilde {I}_3 (a) = \frac {3}{2}\tilde {I}_2 (a) - \frac {3}{4}\tilde {I}_1 (a) + \frac {1}{8}\tilde {I}_0 (a)$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_3 (a) & =&\displaystyle \frac{3}{2}\tilde {I}_2 (a) - \frac{1}{4}\tilde {I}_0 (a). {} \end{array} \end{aligned} $$

(1.A2.29)

Next, recollecting that $\frac {d}{da} \varPhi (ax) = x \varphi (ax)$ and $\frac {d}{da} \varPhi ^2 (ax) = 2x \varPhi (ax) \varphi (ax)$, if we differentiate $\tilde {I}_2(a)$ with respect to a using Leibnitz’s rule (1.A2.19), integrate by parts, and then use (1.A2.17), we get $\frac {d}{da} \tilde {I}_2 (a) = 2 \pi \int _{-\infty }^{\infty } 2x \varPhi (ax) \varphi (ax) \varphi ^2(x) = \frac {2} {\sqrt {2 \pi } } \int _{-\infty }^{\infty } \varPhi (ax) x \exp \left (- \frac { 2+a^2 }{2}x^2\right ) dx = \sqrt {\frac {2}{\pi }} \left [- \frac {\varPhi (ax)}{2+a^2} \exp \left (- \frac {2+a^2} {2}x^2\right ) \right ]_{x=-\infty }^{\infty } + \sqrt { \frac {2}{\pi } } \frac {a} { 2+a^2 } \int _{-\infty }^{\infty } \varphi (ax) \exp \left \{- \frac { \left ( 2+a^2 \right ) x^2 } {2}\right \} dx = \frac {a}{\pi \left ( 2+a^2 \right )} \int _{-\infty }^{\infty } \exp \left \{- \left ( 1+a^2 \right ) x^2 \right \} dx$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{d}{da} \tilde {I}_2 (a)& =&\displaystyle \frac{a}{\pi\left( 2+a^2 \right)} \sqrt { \frac{\pi}{1+a^2} } . \end{array} \end{aligned} $$

(1.A2.30)

Consequently, noting (1.A2.25) and $\frac {d}{da} \tan ^{-1} \sqrt {1+a^2} = \frac {a}{(2+a^2)\sqrt {1+a^2}}$ from $\frac {d}{dx} \tan ^{-1} x = \frac {1}{1+x^2}$, we finally obtain

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_2 (a) \ = \ \frac{1}{\sqrt{\pi}} \tan^{-1} \sqrt{1+a^2}, {} \end{array} \end{aligned} $$

(1.A2.31)

and then

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde {I}_3 (a) \ = \ \frac{3}{2\sqrt{\pi}} \tan^{-1} \sqrt{1+a^2} - \frac{\sqrt{\pi}}{4} {} \end{array} \end{aligned} $$

(1.A2.32)

from (1.A2.29) and (1.A2.31). The formulas $\left \{ {\bar Q}_k \right \}_{k=1}^3$ and $\left \{ \tilde {I}_k(a) \right \}_{k=0}^3$ we have derived so far, together with $\varphi ^{\prime } (x) = -x \varphi (x)$, are the bases in obtaining the moments of the order statistics of the standard normal distribution for small values of n later in Sect. 2.3.5.3.

Appendix 3: Miscellaneous Topics

1.1.1 Equalities from the Binomial Theorem and Beta Function

In this appendix, we review equalities from the binomial theorem together with those between beta and incomplete beta functions.

Consider the beta function

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde{B}(\alpha, \beta) \ = \ \int_{0}^{1} x^{\alpha-1}(1-x)^{\beta-1} dx {} \end{array} \end{aligned} $$

(1.A3.1)

introduced in (1.1.50) and the incomplete beta function [1]

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde{B}_y (\alpha, \beta) \ = \ \int_0^y t^{\alpha-1} (1-t)^{\beta-1} dt {} \end{array} \end{aligned} $$

(1.A3.2)

for complex numbers $\alpha$ and $\beta$ such that $\mbox{Re} (\alpha ) > 0$ and $\mbox{Re} (\beta ) > 0$, where $0 \le y \le 1$. The right-hand side of (1.A3.1) is called the Eulerian integral of the first kind. In some cases, $I_y (\alpha ,\beta ) =\frac {\tilde {B}_y (\alpha , \beta )}{\tilde {B}(\alpha ,\beta )}$ is called the incomplete beta function instead of $\tilde {B}_y (\alpha , \beta )$. The beta function $\tilde {B}(\alpha , \beta )$ is related as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde{B}(\alpha, \beta) \ = \ \frac{\varGamma(\alpha)\varGamma(\beta)}{\varGamma(\alpha+\beta)} {} \end{array} \end{aligned} $$

(1.A3.3)

with the gamma function introduced in (1.A2.7).

As shown in Exercise 1.36, the incomplete beta function can be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \tilde{B}_p (r, n-r+1) \ = \ \tilde{B}(r, n-r+1) \sum_{j=r}^n \, _{n}\mbox{C}_{j} p^j (1-p)^{n-j} {} \end{array} \end{aligned} $$

(1.A3.4)

in terms of the beta function and binomial coefficients. Next, from the binomial theorem

$$\displaystyle \begin{aligned} \begin{array}{rcl} (1+x)^{n} \ = \ \sum_{i=0}^{n} \, _{n} \mbox{C}_i \ x^i , {} \end{array} \end{aligned} $$

(1.A3.5)

we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} n(1+x)^{n-1} \ = \ \sum_{i=1}^{n} i \, _{n}\mbox{C}_i \ x^{i-1} {} \end{array} \end{aligned} $$

(1.A3.6)

by differentiation. Differentiating again, we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} n(n-1)(1+x)^{n-2} \ = \ \sum_{i=2}^{n} i(i-1) \, _{n}\mbox{C}_i \ x^{i-2}. {} \end{array} \end{aligned} $$

(1.A3.7)

Then, we easily get $\sum \limits _{k=1}^n \, _{n-1}\mbox{C}_{k-1} w^{k-1} (1-w)^{n-k} = \sum \limits _{i=0}^{n-1} \, _{n-1}\mbox{C}_i \, w^i (1-w)^{n-1-i} = (1-w)^{n-1} \sum \limits _{i=0}^{n-1} \, _{n-1}\mbox{C}_i \left ( \frac {w}{1-w}\right )^i =(1-w)^{n-1} \left ( 1+ \frac {w}{1-w}\right )^{n-1}$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sum_{k=1}^n \, _{n-1}\mbox{C}_{k-1} w^{k-1} (1-w)^{n-k} & =&\displaystyle 1 {} \end{array} \end{aligned} $$

(1.A3.8)

using (1.A3.5). We also get $\sum \limits _{k=1}^{n} k \, _{n-1}\mbox{C}_{k-1} w^{k-1} ( 1- w)^{n-k} = \sum \limits _{t=0}^{n-1} (t+1) \, _{n-1}\mbox{C}_{t} \left ( \frac {w}{1-w} \right )^{t} ( 1- w)^{n-1} = (1- w)^{n-1} \left \{ \sum \limits _{t=1}^{n-1} t \ _{n-1}\mbox{C}_t \left ( \frac {w}{1-w} \right )^{t-1} \frac {w}{1-w} \right .$$\left . + \sum \limits _{t=0}^{n-1} \ _{n-1}\mbox{C}_t\left ( \frac {w}{1-w}\right )^t \right \} = (1-w)^{n-1} \left \{ (n-1) \left ( 1+ \frac {w}{1-w}\right )^{n-2} \frac {w}{1-w} + \left ( \frac {1}{1-w} \right )^{n-1} \right \}$, i.e.,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sum_{k=1}^{n} k \, _{n-1}\mbox{C}_{k-1} w^{k-1} ( 1- w)^{n-k} & =&\displaystyle (n-1)w +1 . {} \end{array} \end{aligned} $$

(1.A3.9)

Next, we easily get

(1.A3.10)

and

(1.A3.11)

from integrations by parts, where F is a cdf, $f(x) = \frac {d}{dx}F(x)$, $G_F(x) = 2F^{\mathrm {o}}(x)$, and $g_F(x) = \frac {d}{dx}G_F(x) = 2f^{\mathrm {e}}(x)$.

In (1.A3.10), replacing $G_F$ and $g_F$ with F and f, respectively, and then changing the lower limit of the integral from 0 into $-\infty$, we will still have the equality. Similarly, an equality similar to (1.A3.11) can be obtained by replacing F and f with $G_F$ and $g_F$, respectively, in (1.A3.11). In addition, we can show (1.A3.10) also as $\int _0^y G_F^{r-1}(x) \left \{ G_F(y)-G_F(x) \right \}^{s-r-1} g_F(x) dx = G_F^{s-r-1} (y) G_F^{r}(y) \int _{0}^{1} v^{r-1} (1-v)^{s-r-1} dv = G_F^{s-1}(y) \tilde {B}(r, s-r)$by letting $\frac {G_F(x)}{G_F(y)} =v$ and noting $g_F(x) dx = G_F(y) dv$, $v=0$ when $x=0$, and $v=1$ when $x=y$. These results are useful in some evaluations for order statistics and in Exercise 1.7, for example.

1.1.2 Selecting Elements with No Element Excluded

The number of ways to select r elements from a set $\varOmega = \left \{\omega _1, \omega _2, \, \cdots , \omega _n \right \}$ of n distinct elements without repetition is ${ }_n\mbox{C}_r$: each of the n elements will be selected ${ }_n\mbox{C}_r \times \frac {r}{n} = { }_{n-1}\mbox{C}_{r-1}$ times over the ${ }_n\mbox{C}_r$ selections because every element will be selected as many times as any other element.

First, each of the n elements of $\varOmega$ will be included at least once if we choose

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{k}_L \ = \ \left\lceil \frac {n}{r} \right \rceil \end{array} \end{aligned} $$

(1.A3.12)

selections appropriately among the ${ }_n\mbox{C}_r$ possible selections. For example, when $\varOmega = \{ 1, 2, 3, 4, 5 \}$ and $r=2$, we have $\bar {k}_1 = \left \lceil \frac {5}{2} \right \rceil =3$. Then, each of the five elements of $\varOmega$ will be included at least once in three selections chosen appropriately, for example, in $\{1, 2\}$, $\{3, 4\}$, and $\{4, 5\}$.

Next, it is possible that one or more elements will not be included if we consider ${ }_{n-1}\mbox{C}_r$ selections or less among the ${ }_n\mbox{C}_r$ possible selections. For example, when $\varOmega =\{ 1, 2, 3, 4, 5 \}$ and $r=2$, the element 5 is not included in the choice $\{1, 2\}$, $\{1, 3\}$, $\{1, 4\}$, $\{2, 3\}$, $\{2, 4\}$, and $\{3, 4\}$ of ${ }_{4}\mbox{C}_2 = 6$ selections. On the other hand, each of the n elements will be included at least once in any choice of

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{k}_{A} \ = \ 1 \, + \, {}_{n-1}\mbox{C}_{r} {} \end{array} \end{aligned} $$

(1.A3.13)

selections among the ${ }_n\mbox{C}_r$ options. Here, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} {}_{n-1}\mbox{C}_{r} & =&\displaystyle \left ( 1- \frac{r}{n}\right ) \,{}_{n}\mbox{C}_{r} \\ & =&\displaystyle {}_{n}\mbox{C}_{r} - \,{}_{n-1}\mbox{C}_{r-1}. {} \end{array} \end{aligned} $$

(1.A3.14)

The identity (1.A3.14) implies that, when selecting r elements from a set of n distinct elements, the number of ways for a specific element not to be included is the same as the following two numbers.

• (1) The number ${ }_{n-1}\mbox{C}_{r}$ of ways to select r elements from a set of $(n-1)$ distinct elements.

• (2) The difference between the number ${ }_{n}\mbox{C}_{r}$ of ways to select r elements from a set of n distinct elements and the number ${ }_{n-1}\mbox{C}_{r-1}$ of ways for a specific element to be included when selecting r elements from a set of n distinct elements.

1.1.3 Further Properties of the Impulse Function

Let us first note that $g^{\prime } \left ( f(x) \right ) = \left [ \frac {d g(y) }{dy} \right ]_{y=f(x)} \ne \frac {d}{dx} g(f(x))$. In other words, $g^{\prime } \left ( f(x) \right )$ denotes $\left [ \frac {d g(y) }{dy} \right ]_{y=f(x)}$ or $\frac {d g(f) }{df}$, but not $\frac {d}{dx} g(f(x))$. For the function $\delta ^{\prime } \left ( f(x) \right ) = \left . \frac {d\delta (v)}{dv} \right |{ }_{v=f(x)}$ or

$$\displaystyle \begin{aligned} \begin{array}{rcl} \delta^{\prime} \left ( f(x) \right ) & =&\displaystyle \frac{d\delta(f(x))}{df(x)} , \end{array} \end{aligned} $$

(1.A3.15)

the following theorem can be obtained.

Theorem 1.A3.1

We have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \delta^{\prime} \left ( f(x) \right ) \ = \ \sum_{m=1}^n \frac{ f^{\prime} \left ( x_m \right ) \delta^{\prime} \left ( x-x_m \right ) + f^{\prime\prime} \left ( x_m \right ) \delta \left ( x-x_m \right ) } { \left | f^{\prime} \left ( x_m \right ) \right |{}^3} , {} \end{array} \end{aligned} $$

(1.A3.16)

where$\left \{ x_m \right \}_{m=1}^n$denote the real simple zeroes of$f(x)$.

Proof

Note that

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{ \delta^{\prime} \left ( x-x_m \right ) } {f^{\prime}(x)} \ = \ \frac{f^{\prime\prime} \left ( x_m \right ) } {\left\{ f^{\prime} \left ( x_m \right ) \right \}^2} \delta \left ( x-x_m \right ) + \frac{\delta^{\prime} \left ( x-x_m \right ) } {f^{\prime} \left ( x_m \right ) } {} \end{array} \end{aligned} $$

(1.A3.17)

from the formula (1.2.37) because $\left . \left \{ \frac {1}{f^{\prime }(x)} \right \}^{\prime } \right |{ }_{x=x_m} = -\frac {f^{\prime \prime }\left ( x_m \right )}{\left \{f^{\prime }\left ( x_m \right ) \right \}^2}$. Also

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{ d } {dx} \delta \left ( f(x) \right ) \ = \ \sum_{m=1}^n \frac{\delta^{\prime} \left ( x-x_m \right ) }{\left|f^{\prime} \left ( x_m \right ) \right|} {} \end{array} \end{aligned} $$

(1.A3.18)

from the formula (1.2.41). Then, because $\delta ^{\prime } (f) = \frac {d \delta (f)}{df} = \frac {dx}{df} \frac {d \delta (f)}{dx}$, we have

$$\displaystyle \begin{aligned} \begin{array}{rcl} \delta^{\prime} (f(x)) & =&\displaystyle \frac{1}{f^{\prime}(x)} \sum_{m=1}^n \frac{\delta^{\prime} \left ( x-x_m \right ) }{\left|f^{\prime} \left ( x_m \right ) \right|} {} \end{array} \end{aligned} $$

(1.A3.19)

from (1.A3.18). Now, employing (1.A3.17) into (1.A3.19) completes the proof. $\spadesuit$

Similarly to Theorem 1.2.2, Theorem 1.A3.1 indicates that $\delta ^{\prime } \left ( f(x) \right )$ can be expressed as a linear combination of $\left \{ \delta \left (x-x_m \right ) , \delta ^\prime \left (x-x_m \right ) \right \}_{m=1}^{n}$.

Example 1.A3.1

The function $f(x) = (x-1)(x-2)$ has two simple zeroes $x=1$ and 2. Thus, $\delta ^{\prime } ((x-1)(x-2)) = 2 \delta (x-1) + 2 \delta (x-2) - \delta ^{\prime } (x-1) + \delta ^{\prime } (x-2)$ because $f^{\prime } (1)= -1$, $f^{\prime \prime } (1) = 2$, $f^{\prime } (2)= 1$, and $f^{\prime \prime } (2) = 2$. $\diamondsuit$

Example 1.A3.2

The function $f(x) = \sinh 2x$ has one simple zero $x=0$. Then, $\delta ^{\prime } (\sinh 2x) = \frac {1}{2} \left \{ \frac {1}{2} \delta ^{\prime } (x) +0 \right \} = \frac {1}{4}\delta ^{\prime } (x)$ from $f^{\prime } (0)= 2 \cosh 0 = 2$ and $f^{\prime \prime } (0) = 4 \sinh 0 = 0$. $\diamondsuit$

1.1.4 Inverse Tangent and Inverse Sine Functions

Let us show the relation

$$\displaystyle \begin{aligned} \begin{array}{rcl} 2 \tan^{-1} \sqrt{h} + \sin^{-1} \frac{1-h}{1+h} \ = \ \frac {\pi} {2} , {} \end{array} \end{aligned} $$

(1.A3.20)

a special case of which for $h=2$, that is, $2 \tan ^{-1} \sqrt {2} - \sin ^{-1} \frac {1}{3} = \frac {\pi } {2}$, will be used in Table 2.1 later in Sect. 2.3.5.3.

We first show the equality (1.A3.20) geometrically when $h \ge 1$. As shown in Fig. 1.20, consider a right-angled triangle $\mbox{ABC}$ with $\overline {\mbox{AB}}=1$, $\overline {\mbox{BC}}=\sqrt {h}$, and $\overline {\mbox{CA}}=\sqrt {h+1}$. Let $\angle \mbox{BAC} = \alpha$: then, $\tan \alpha = \sqrt {h}$ and $\sin \alpha = \sqrt {\frac {h}{h+1}}$. Next, draw the right-angled triangle $\mbox{ACD}$ with $\overline {\mbox{CD}}=\sqrt {h(h+1)}$ and $\angle \mbox{ACD}=\frac {\pi }{2}$. Then, we have $\overline {\mbox{AD}}= \sqrt { \overline {\mbox{AC}}^2 + \overline {\mbox{CD}}^2 } = h+1$ and $\angle \mbox{CAD} = \tan ^{-1} \sqrt {h} = \alpha$. Denote by l, $\mbox{E}$, and $\mbox{F}$ the straight line parallel to the line segment $\mbox{BC}$ and passing through point $\mbox{A}$, the cross point of l and the line segment $\mbox{CD}$, and the foot of the perpendicular from point $\mbox{D}$ on l, respectively. Then, from $\angle \mbox{CAE} + \angle \mbox{AEC} = \frac {\pi }{2}$ and $\angle \mbox{CAE} + \angle \mbox{BAC} = \frac {\pi }{2}$, we have $\angle \mbox{AEC} = \angle \mbox{BAC} = \alpha$. Subsequently, we have $\angle \mbox{DEF} = \alpha$ from $\angle \mbox{DEF} = \angle \mbox{AEC}$.

Fig. 1.20

A figure showing geometrically the equality $2 \alpha - \beta = \frac {\pi }{2}$ for $h \ge 1$, where $\alpha$$= \tan ^{-1} \sqrt {h}$ and $\beta = \sin ^{-1} \frac {h-1}{h+1}$

Now, $\overline {\mbox{CE}} = \frac { \overline { \mbox{AC}}} {\tan \alpha } = \sqrt {\frac {h+1}{h}}$, and therefore, $\overline {\mbox{DE}} = \overline {\mbox{CD}} - \overline {\mbox{CE}} = (h-1)\sqrt { \frac {h+1}{h} }$. We thus get $\overline {\mbox{DF}} = \overline {\mbox{DE}} \sin \alpha = h-1$, implying that $\angle \mbox{DAF} = \sin ^{-1} \frac { \overline {\mbox{DF}} } { \overline {\mbox{AD}} } = \sin ^{-1} \frac {h-1}{h+1}$. Finally, from $\angle \mbox{BAD} = \angle \mbox{BAC} + \angle \mbox{CAD}= 2 \tan ^{-1}\sqrt {h}$ and $\angle \mbox{BAD} = \angle \mbox{BAF} + \angle \mbox{DAF}= \frac {\pi }{2} + \sin ^{-1}\frac {h-1}{h+1}$, we have $2 \tan ^{-1}\sqrt {h}= \frac {\pi }{2} + \sin ^{-1}\frac {h-1}{h+1}$, which is the same as (1.A3.20) from $\sin ^{-1} ( -\theta ) = - \sin ^{-1} \theta$. Similar steps will lead us to (1.A3.20) when $0 \le h \le 1$.

Next, let us show (1.A3.20) analytically. First, showing the equality (1.A3.20) is equivalent to showing

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sin \left( 2 \tan^{-1} \sqrt{h} + \sin^{-1} \frac{h-1}{h+1} \right ) \ = \ 1. {} \end{array} \end{aligned} $$

(1.A3.21)

Note that $\sin \left ( \tan ^{-1} \sqrt {h} \right ) = \frac {\sqrt {h}}{\sqrt {h+1}}$ and $\cos \left ( \tan ^{-1} \sqrt {h} \right ) = \frac {1}{\sqrt {h+1}}$. We then get $\sin \left ( 2 \tan ^{-1} \sqrt {h} + \sin ^{-1} \frac {h-1}{h+1} \right )= \sin \left ( 2 \tan ^{-1} \sqrt {h} \right ) \cos \left ( \sin ^{-1} \frac {h-1}{h+1} \right ) + \cos \big ( 2 \tan ^{-1} \sqrt {h} \big ) \sin \left ( \sin ^{-1} \frac {h-1}{h+1} \right )= 2\sin \left ( \tan ^{-1} \sqrt {h} \right ) \cos \left ( \tan ^{-1} \sqrt {h} \right ) \frac {2\sqrt {h}}{h+1} + \Big \{ 2 \cos ^2 \left ( \tan ^{-1} \sqrt {h} \Big ) -1 \right \} \frac {h-1}{h+1} =2 \sqrt { \frac {h}{h+1}} \frac {1}{\sqrt {h+1}}\frac {2\sqrt {h}}{h+1}+ \left ( \frac {2}{h+1} -1 \right ) \frac {h-1}{h+1}=1$.

1.1.5 Change of a Quadruple Sum

Let us show

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sum_{s=2}^{n} \sum_{r=1}^{s-1} \sum_{j=s}^{n} \sum_{i=r}^{j} \varDelta_{j,i} \ = \ \sum_{k=2}^{n} \sum_{l=1}^{k} \left \{ (k-1)l - \frac{l(l-1)}{2} \right\} \varDelta_{k,l} . {} \end{array} \end{aligned} $$

(1.A3.22)

First, expanding the sum over s on the left-hand side of (1.A3.22), we get

(1.A3.23)

Next, combining terms of the same value of r on the right-hand side of (1.A3.23), we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sum_{s=2}^{n} \sum_{r=1}^{s-1} \sum_{j=s}^{n} \sum_{i=r}^{j} \varDelta_{j,i} & =&\displaystyle \left ( \sum_{j=2}^{n} + \sum_{j=3}^{n} + \sum_{j=4}^{n} + \cdots + \sum_{j=n}^{n} \right) \sum_{i=1}^{j} \varDelta_{j,i} \\ & &\displaystyle \quad \ + \left ( \sum_{j=3}^{n} + \sum_{j=4}^{n} + \cdots + \sum_{j=n}^{n} \right) \sum_{i=2}^{j} \varDelta_{j,i} \\ & &\displaystyle \quad \ \quad \ \quad \ \quad \ \quad \ + \cdots + \left ( \sum_{j=n}^{n} \right) \sum_{i=n-1}^{j} \varDelta_{j,i} . \qquad \quad {} \end{array} \end{aligned} $$

(1.A3.24)

Let us now count how many times $\varDelta _{k,l}$ is added on the right-hand side of (1.A3.24), where $k \in \mathbb {J}_{2,n}$ and $l \in \mathbb {J}_{1,k}$. First, the case of $(j, i) =(k, l)$ occurs $(k-1)$ times in the first line, $(k-2)$ times in the second line, $\cdots$, $(k-l)$ times in the l-th line. In short, $\varDelta _{k,l}$ is added $(k-1)+(k-2)+ \cdots + (k-l) = kl - \frac {l(l+1)}{2} = (k-1)l - \frac {l(l-1)}{2}$ times. Based on this, the right-hand side of (1.A3.24) can be written as the right-hand side of (1.A3.22).

The formula (1.A3.22) will be used in (2.A1.3) for proving Theorem 2.2.4 later in section “Proofs and Calculations” in Appendix 1, Chap. 2. In the meantime, if we calculate both sides of (1.A3.22) directly, we get $\sum \limits _{s=2}^{n} \sum \limits _{r=1}^{s-1} \sum \limits _{j=s}^{n} \sum \limits _{i=r}^{j} 1 = \frac {1}{12}(n-1)n(n+1)(n+2)$ and $\sum \limits _{k=2}^{n} \sum \limits _{l=1}^{k} \left \{ (k-1)l - \frac {l(l-1)}{2} \right \}=\frac {1}{12}(n-1)n(n+1)(n+2)$.

Exercises

Exercise 1.1

When A and B are independent of each other, show that $A^c$ and B are independent of each other and that $A^c$ and $B^c$ are independent of each other.

Exercise 1.2

Assume the sample space $\mathcal {S} = \mathbb {J}_{1,n}$ defined in (1.4.5) and event space $\mathcal {F}=2^{\mathcal {S}}$. When $n \ge 4$, two events can be independent of each other with some probability measure: for example, the events $\{1, 2\}$ and $\{2, 3\}$ are independent of each other when $\mathsf {P} (1)= \mathsf {P} (4)= \frac {3}{10}$ and $\mathsf {P} (2)= \mathsf {P} (3)= \frac {2}{10}$ for $n=4$. Show that no two events, except $\mathcal {S}$ and $\emptyset$, are independent of each other for any probability measure such that $\mathsf {P} (1)>0$, $\mathsf {P} (2)>0$, and $\mathsf {P} (3)>0$ when $n=3$.

Exercise 1.3

For two events A and B, show the following.

• (1) If $\mathsf {P} (A)=0$, then $\mathsf {P} (AB)=0$.

• (2) If $\mathsf {P} (A)=1$, then $\mathsf {P} (AB)=\mathsf {P} (B)$.

Exercise 1.4

Let the cdf of X be

$$\displaystyle \begin{aligned} \begin{array}{rcl} F_X (x) \ = \ \left\{\begin{array}{llll} 0, & x <0; & \quad \frac{1}{4}(x^2+1), & 0\leq x <1;\\ \frac{1}{4}(x+2), & 1\leq x <2; & \quad 1, & x \geq 2 . \end{array} \right. \end{array} \end{aligned} $$

(1.E.1)

• (1) Obtain $\mathsf {P} (0<X<1)$ and $\mathsf {P} (1 \leq X <1.5)$.

• (2) Obtain the mean $\mathsf {E}\{X\}$ and variance $\mathsf {Var} \{X\}$ of X.

Exercise 1.5

The joint pdf of $(X, Y)$ is $f_{X,Y} (x,y) = a \left \{ 1+\left ( x^2 - y^2 \right )xy \right \} u ( 1- |x| )u ( 1- |y| )$. Determine the constant a. Are X and Y  independent of each other? If not, obtain the correlation coefficient $\rho _{XY}$ between X and Y .

Exercise 1.6

Show that

$$\displaystyle \begin{aligned} \begin{array}{rcl} \lim\limits_{x \to \infty} xF(-x) \ = \ 0 {} \end{array} \end{aligned} $$

(1.E.2)

for an absolutely continuous cdf F. Using this result, show that

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{E}\{X\} \ = \ \int_{0}^{\infty} \left \{1 - F_X (x) \right \} dx - \int_{-\infty}^{0} F_X (x) dx {} \end{array} \end{aligned} $$

(1.E.3)

for a random variable X with a continuous and absolutely integrable pdf.

Exercise 1.7

For a random vector $(X, Y)$, assume the joint pdf

$$\displaystyle \begin{aligned} \begin{array}{rcl} f_{X, Y} ( x, y) & =&\displaystyle \frac{n!}{(i-1)! (k-i-1)! (n-k)! } F^{i-1} (x) \{ F(y) - F(x) \}^{k-i-1} \\ & &\displaystyle \times \{1- F(y) \}^{n-k} f(x) f(y) u(y-x), \end{array} \end{aligned} $$

(1.E.4)

where i, k, and n are natural numbers such that $1 \leq i < k \leq n$, F is the cdf of a random variable, and $f(t) = \frac {d}{dt} F(t)$. Obtain the pdf $f_X$ of X and pdf $f_Y$ of Y .

Exercise 1.8

For a random variable X with the beta pdf $f(r)= \frac {r^{\alpha -1} (1-r)^{\beta -1}} {\tilde {B}(\alpha , \beta )} u(r) u(1-r)$ introduced in (1.1.49), show that the k-th moment is

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{E}\left \{X^k\right \} & =&\displaystyle \frac{\varGamma(\alpha+k)\varGamma(\alpha+\beta)}{\varGamma(\alpha+\beta+k)\varGamma(\alpha)} . {} \end{array} \end{aligned} $$

(1.E.5)

Exercise 1.9

For a random variable X with the logistic pdf $f_L (x) = \frac {be^{-bx}}{\left (1+e^{-bx} \right )^2}$, show that $\mathsf {E} \left \{ X^2 \right \} = \frac {\pi ^2}{3b^2}$, $\mathsf {E} \left \{X^4 \right \} = \frac {7\pi ^4}{15b^4}$, and $m_L^+ = \frac {\ln 2}{b} = - m_L^-$, where the half means $m_L^+$ and $m_L^-$ are defined in (1.3.33).

Exercise 1.10

For a continuous random variable X with cdf $F_X$ and pdf $f_X$, show that the conditional cdf’s $F_{X|X \le b} (x)$ and $F_{X|X \ge a} (x)$ can be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl} F_{X|X \le b} (x ) & =&\displaystyle u(x-b) + \frac{F_X(x)}{F_X(b)} u(b-x) {} \end{array} \end{aligned} $$

(1.E.6)

with $u(0) = \frac {1}{2}$ and

$$\displaystyle \begin{aligned} \begin{array}{rcl} F_{X|X \ge a} (x ) & =&\displaystyle \frac{F_X(x) - F_X(a)}{1-F_X(a)} u(x-a). {} \end{array} \end{aligned} $$

(1.E.7)

Express the conditional pdf’s $f_{X|X \le b} (x)$ and $f_{X|X \ge a} (x)$ in terms of $f_X$ and $F_X$. An example of $f_{X|X \le b} (x )$ is shown in Fig. 1.21.

Fig. 1.21

The pdf $f_X (x)$ and the conditional pdf $f_{X|X\le b}(x)$

Exercise 1.11

Confirm

$$\displaystyle \begin{aligned} \begin{array}{rcl} \lim\limits_{m \to \infty} \int_{-\infty}^{\infty} f(x) \frac{\sin mx}{\pi x} dx \ = \ f(0) {} \end{array} \end{aligned} $$

(1.E.8)

shown in (1.2.21) based on the inverse Fourier transform of product.

Exercise 1.12

Recollect the definition of the unit step function.

• (1) Express $u(ax+b)$ with $a \ne 0$, $u(\sin x)$, and $u\left (e^x - \pi \right )$ in other formulas.

• (2) Obtain $\int _{-\infty }^x u(t-y) dt$.

Exercise 1.13

Let us address the Fourier transform of the unit step function $u(x)$ by following the steps shown below.

• (1) Show that the Fourier transform of the impulse function $\delta (x)$ is $\mathfrak {F} \{ \delta (x) \} =1$. Then, noting that [20] $\mathfrak {F} \{ A(x) \}= 2\pi a(-\omega )$ when $A (\omega ) =\mathfrak {F} \{ a(x) \}$ , we get

  $$\displaystyle \begin{aligned} \begin{array}{rcl} \mathfrak{F} \{ 1 \} \ =\ 2 \pi \, \delta ( \omega ) . {} \end{array} \end{aligned} $$

  (1.E.9)

  It is noteworthy that (1.E.9) implies an interesting result

  $$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{1}{2\pi} \int_{-\infty}^{\infty} e^{-j xy} dx & =&\displaystyle \delta(y) \end{array} \end{aligned} $$

  (1.E.10)

  of integration.

• (2) Let the Fourier transform of $s_{\alpha } (x) = \mathrm {sgn}(x) e^{-\alpha |x|}$ be $S_{\alpha } (\omega ) = \mathfrak {F} \left \{ s_{\alpha } (x)\right \}$, where $\alpha > 0$. Show $\lim \limits _{\alpha \to 0} S_{\alpha } (\omega ) = \frac {2}{j \omega }$, which can be expressed equivalently as

  $$\displaystyle \begin{aligned} \begin{array}{rcl} \mathfrak{F} \{ \mathrm{sgn} (x) \} \ = \ \frac{2}{j \omega} . {} \end{array} \end{aligned} $$

  (1.E.11)

• (3) Based on $u(x) = \frac {1}{2} \{1+ \mathrm {sgn}(x)\}$, (1.E.9), and (1.E.11), confirm the Fourier transform

  $$\displaystyle \begin{aligned} \begin{array}{rcl} \mathfrak{F} \{ u(x) \} & =&\displaystyle \pi \delta (\omega) + \frac{1}{j \omega} {} \end{array} \end{aligned} $$

  (1.E.12)

  of the unit step function $u(x)$.

Note

In communications and signal processing, Fourier transform and Hilbert transform, the convolution of a function with $\frac {1}{\pi x}$, are important tools for the discussion of signals via complex representations. For example, let the Fourier and Hilbert transforms of a real function $g(x)$ be $G (\omega ) = \mathfrak {F} \{ g(x) \}$ and ${\hat g}(x) = \mathfrak {H} \{ g(x) \} = g(x) * \frac {1}{\pi x} = \frac {1}{\pi } \int _{-\infty }^{\infty } \frac {g(y)}{x-y} dy$, respectively. Then, using (1.E.11) and noting that $\mathfrak {F} \{ A(x) \}= 2\pi a(-\omega )$ when $A (\omega ) =\mathfrak {F} \{ a(x) \}$, the Fourier transform $\mathfrak {F} \left \{ {\hat g}(x) \right \}$ of the Hilbert transform ${\hat g}(x)$ can be expressed as $\mathfrak {F} \left \{ {\hat g}(x) \right \} = - j G (\omega ) \mathrm {sgn} (\omega )$, a key equation in the discussion of narrow-band and single-sideband signals.

Exercise 1.14

Answer the problems below.

• (1) Calculate $\int _{-2\pi }^{2\pi } e^{\pi x} \delta \left (x^2 - \pi ^2 \right ) dx$. When $0 \le x < 2 \pi$, express $\delta (\sin x)$ in another formula.

• (2) Express $\delta ^{\prime } (x) \cos x$ in another formula.

• (3) Calculate $\int _{-\infty }^{\infty } \delta ^{\prime } \left (x^2 -3x+2 \right ) dx$ and $\int _{-\infty }^{\infty } (\cos x + \sin x )\delta ^{\prime } \left (x^3+ x^2+x \right ) dx$.

• (4) When $a \ne b \ne c \ne a$ and $a, b, c \in \mathbb {R}$, express $\delta \big ( (x-a) (x-b)(x-c)\big )$ as a linear combination of $\delta (x-a)$, $\delta (x-b)$, and $\delta (x-c)$.

Exercise 1.15

Recollect that $g_m(x) = \frac {x^m}{m!}u(x)$ is the primitive function of $\tilde {h}_m(x) = h_m(x) + \frac {x^m}{m!}\delta (x)$, where $h_m(x) = \frac {x^{m-1}}{(m-1)!}u(x)$. Show that $h(x) = \lim \limits _{m \to 0} h_m(x)$ satisfies the conditions (1.2.18) and (1.2.19) and is thus an impulse function.

Exercise 1.16

Show that $h(x) = \lim \limits _{k \to 1} (k-1) |x|{ }^{k-2}$ is an impulse function. One way is, for example, to confirm $h(x) = 2\delta (x)$ by showing that $\frac {1}{2} h(x)$ satisfies the conditions (1.2.18) and (1.2.19) of the impulse function.

Exercise 1.17

A multi-dimensional impulse function can be defined as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \delta \left ( x_1 , x_2 , \, \cdots , x_n \right ) \ =\ \prod_{i=1}^{n} \delta \left( x_i \right). \end{array} \end{aligned} $$

(1.E.13)

Show that

$$\displaystyle \begin{aligned} \begin{array}{rcl} \delta ( x , y ) \ = \ \frac{ \delta \left( r_2 \right) } {\pi r_2} \end{array} \end{aligned} $$

(1.E.14)

and

$$\displaystyle \begin{aligned} \begin{array}{rcl} \delta ( x , y, z ) \ = \ \frac{ \delta \left( r_3 \right) } {2 \pi r_3^2} , \end{array} \end{aligned} $$

(1.E.15)

where $r_2 = \sqrt {x^2 + y^2}$ and $r_3 = \sqrt {x^2 + y^2 + z^2}$.

Exercise 1.18

For a random variable X with pdf $f_X (x) = u(x) - u(x-1)$, obtain the joint pdf $f_{X,Y}$ of X and $Y=2X+1$.

Exercise 1.19

For X and $Y=cX+a$, show that the joint cdf is

$$\displaystyle \begin{aligned} \begin{array}{rcl} F_{X, cX+a} (x,y) = \left\{\begin{array}{ll} F_X(x) - D_{F_X} \left (x , \frac{y-a}{c} \right) u\left (x-\frac{y-a}{c} \right), & \quad c > 0, \\ F_X(x) u (y-a) , & \quad c=0, \\ D_{F_X} \left (x , \frac{y-a}{c} \right)u\left (x- \frac{y-a}{c} \right) , & \quad c < 0 \end{array} \right. \quad {} \end{array} \end{aligned} $$

(1.E.16)

with $u(0)=1$ and the joint pdf is

$$\displaystyle \begin{aligned} \begin{array}{rcl} f_{X, cX+a} (x,y) &=& \left\{\begin{array}{ll} \frac{1}{|c|}f_X(x) \delta \left (\frac{y-a}{c}- x \right) , & \quad c \ne 0, \\ f_X(x) \delta (y-a) , & \quad c=0 \end{array} \right. \\ &=& f_X(x) \delta (y- cx -a) . {} \end{array} \end{aligned} $$

(1.E.17)

Confirm that the limits of the first and third lines on the right-hand side of (1.E.16) as $c \to 0^+$ and $c \to 0^-$, respectively, are both equal to the second line.

Exercise 1.20

Let f and F be the pdf and cdf, respectively, of a continuous random variable X, and let $Y=X^2$.

• (1) Obtain the joint cdf $F_{X,Y}$.

• (2) Obtain the joint pdf $f_{X,Y}$, and then confirm $\int _{-\infty }^{\infty } f_{X,Y} (x,y) dy = f(x)$ and

  $$\displaystyle \begin{aligned} \begin{array}{rcl} \int_{-\infty}^{\infty} f_{X,Y} (x,y) dx \ = \ \frac{1}{2 \sqrt{y}} \left \{ f \left ( \sqrt{y} \right ) + f \left ( -\sqrt{y} \right ) \right \} u(y) \end{array} \end{aligned} $$

  (1.E.18)

  by integration.

• (3) Obtain the conditional pdf $f_{X|Y}$.

Exercise 1.21

Show that the pdf $f_{X, cX}$ given in (1.2.57) satisfies $\int _{-\infty }^{\infty } \int _{-\infty }^{\infty }$$f_{X, cX} (x,y) dy dx = 1$ and $\int _{-\infty }^{\infty } \int _{-\infty }^{\infty } f_{X, cX} (x,y) dx dy = 1$.

Exercise 1.22

Express the joint cdf $F_{X,Y}$ and pdf $f_{X,Y}$ of the input X and output $Y=Xu(X)$ of a half-wave rectifier in terms of the pdf $f_X$ and cdf $F_X$ of X.

Exercise 1.23

Obtain the joint pdf (1.2.50) of X and $X+a$ from $F_{X, X+a} (x,y) = F_X ( \min \{x,y-a\} )$ shown in (1.2.48).

Exercise 1.24

Based on the joint cdf (1.3.42), show that the joint pdf of the magnitude $Y=|X|$ and sign $Z=\mathrm {sgn} (X)$ of a random variable X with pdf $f_X$ can be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl} f_{Y,Z} (y, z) & =&\displaystyle \left\{ \delta (z+1) f_X(-y) + \delta(z-1) f_X(y) \right\} u(y). {} \end{array} \end{aligned} $$

(1.E.19)

Using this result, obtain the pdf $f_Y$ of Y , pdf $f_Z$ of Z, and joint moment $\mathsf {E} \{|X|Z\}$.

Exercise 1.25

Consider the absolute value $Y = |X|$ for a continuous random variable X with pdf $f_X$. If we consider the half means$m_X^{\pm }$ defined in (1.3.33), the mean $\mathsf {E} \{ X \} = m_X$ of X can be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{E} \{ X \} \ = \ m_X^+ + m_X^-. \end{array} \end{aligned} $$

(1.E.20)

Show that the mean $\mathsf {E} \{ |X| \}$ and variance $\mathsf {Var} \{ |X| \} =\sigma _{|X|}^2$ of $Y = |X|$ can be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{E} \{ |X| \} \ = \ m_X^+ - m_X^- {} \end{array} \end{aligned} $$

(1.E.21)

and

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sigma_{|X|}^2 \ = \ \sigma_X^2 +4 m_X^+ m_X^- , \end{array} \end{aligned} $$

(1.E.22)

respectively, and that

$$\displaystyle \begin{aligned} \begin{array}{rcl} \sigma_X^2 +4 m_X^+ m_X^- \ \ge \ 0 \end{array} \end{aligned} $$

(1.E.23)

for the variance $\sigma _{|X|}^2$.

Exercise 1.26

Show that the correlation coefficient between X and $Y=|X|$ is

$$\displaystyle \begin{aligned} \begin{array}{rcl} \rho_{X Y} & =&\displaystyle \frac { 1 } { \sigma_X \sigma_{|X|} } \int_{-\infty}^{\infty} |x| \left ( x-m_X \right ) f_X (x) dx . {} \end{array} \end{aligned} $$

(1.E.24)

Here, $f_X$, $m_X$, and $\sigma _X^2$ are the pdf, mean, and variance, respectively, of X; and $\sigma _{|X|}^2 = \sigma _X^2 + 4 m_X^+ m_X^-$ is the variance of $| X|$ with $m_X^{\pm }$ the half means defined in (1.3.33). Obtain the value of $\rho _{X Y}$ and compare it with what can be obtained intuitively in each of the following cases:

• (1) $f_X(x)$ is an even symmetric function of x.

• (2) $f_X(x) >0$ only for $x \ge 0$.

• (3) $f_X(x) >0$ only for $x \le 0$.

Exercise 1.27

Consider a continuous random variable X and its magnitude $Y=|X|$. Show that the conditional pdf $f_{X | Y}$ can be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl} f_{X | Y} (x | y) = \frac{f_X(x)\delta(x+y)}{f_X(x) + f_X(-x) }u(-x) + \frac{f_X(x)\delta(x-y)}{f_X(x) + f_X(-x) }u(x) {} \end{array} \end{aligned} $$

(1.E.25)

for $y \in \big \{ t\,| \, \left \{ f_X(t) + f_X(-t) \right \} u(t) > 0 \big \}$, where $f_X$ is the pdf of X. Obtain the conditional pdf $f_{Y|X}$.

Exercise 1.28

Based on the joint cdf (1.3.31), show that the joint pdf of a random variable X and its sign $Z=\mathrm {sgn} (X)$ is

$$\displaystyle \begin{aligned} \begin{array}{rcl} f_{X,Z} (x,z) & =&\displaystyle \{ u(x) \delta (z-1) + u(-x) \delta(z+1) \} f_X(x), {} \end{array} \end{aligned} $$

(1.E.26)

where $f_X$ is the pdf of X. Using the joint pdf (1.E.26), obtain the pdf of X, the pdf of Z, and the joint moment $\mathsf {E} \{ZX\}$.

Exercise 1.29

Assume that a continuous random variable X has variance $\sigma _X^2$, pdf $f_X$, and cdf $F_X$ with $F_X(0) \neq 0,1$. Denote the half means of X by $m_X^+$ and $m_X^-$ as in (1.3.33). Let $Y=|X|$ and $Z = \mathrm {sgn} (X)$. Recollecting the joint pdf $f_{Y,Z} (y, z) = \{ \delta (z+1) f_X(-y) + \delta (z-1) f_X(y) \} u(y)$ shown in (1.E.19) of Y  and Z, show the correlation coefficient

$$\displaystyle \begin{aligned} \begin{array}{rcl} \rho_{ Y Z} & =&\displaystyle \frac {2 } { \sigma_{|X|}\sigma_{Z} } \left [ F_X(0)m_X^+ + \left \{ 1-F_X(0) \right \}m_X^- \right ] {} \end{array} \end{aligned} $$

(1.E.27)

between Y  and Z.

Exercise 1.30

When the random vector is $\boldsymbol {X} = (3, 4, 5)$, obtain the order statistics $X_{[1]}$, $X_{[2]}$, and $X_{[3]}$. Also obtain the order statistic vector $\boldsymbol {X_{[\cdot ]}}$.

Exercise 1.31

Obtain the probability $\mathsf {P} \left ( X_2 < X_5 < X_9, \ X_3 > X_7 \right )$ for continuous i.i.d. random variables $X_1 , X_2 , \, \cdots , X_n$.

Exercise 1.32

Consider i.i.d. random variables $\left \{ X_i \right \}_{i=1}^{\infty }$ with the marginal distribution $U(0,1)$. Let $X_0=x$ and $N=\min \left \{ n: \, X_n <X_{n-1} \right \}$.

• (1) Show

  $$\displaystyle \begin{aligned} \begin{array}{rcl} \mathsf{P} (N\ge k) \ = \ \frac{(1-x)^{k-1}}{(k-1)!}. {} \end{array} \end{aligned} $$

  (1.E.28)

• (2) Based on (1) above, obtain $\mathsf {E} \{N\}$.

Exercise 1.33

When the random vector is $\boldsymbol {X} = (4, -6, 1, 9)$, obtain $\boldsymbol {Z}$, $\boldsymbol {X_{[\cdot ]}}$, $\boldsymbol {R}$, $\boldsymbol {|X|}$, $\boldsymbol {|X|{ }_{[\cdot ]}}$, and $\boldsymbol {Q}$.

Exercise 1.34

When the sign, magnitude rank, and magnitude order statistic vectors of $\boldsymbol {X}$ are $(-1, 1, -1, 1)$, $(3, 2, 4, 1)$, and $(1, 2, 3, 4)$, respectively, obtain $\boldsymbol {X}$.

Exercise 1.35

Consider the set

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varOmega \ = \ \left \{ \left (a_{k_1}, a_{k_2}, \, \cdots, a_{k_n} \right ) \right \}_{k=1}^{n!} \end{array} \end{aligned} $$

(1.E.29)

of the permutations made from a set of n distinct numbers $a_1 <a_2 < \cdots <a_n$. In a permutation, when $a_i<a_j$ and $a_i$ is located on the right-hand side of $a_j$, we refer to $a_i$ and $a_j$ having been switched. Let $N_i$ be the number of switches with respect to $a_i$ in a permutation: then, $N_1=0$ and the number of switches in the permutation is $N=\sum \limits _{i=1}^{n} N_i$. For instance, for the permutation $(4, 2, 1, 5, 3)$, we have $\left \{ a_j =j \right \}_{j=1}^{5}$, $N_1 = 0$, $N_2 = 1$, $N_3 = 0$, $N_4 = 3$, $N_5 = 1$, and $N=1+3+1=5$: this implies that in the permutation $(4, 2, 1, 5, 3)$, we have switches at $(4, 2)$, $(4, 1)$, $(4, 3)$, $(2, 1)$, and $(5, 3)$. Now, assume we choose a permutation from $\varOmega$ at random.

• (1) Show that the random variables $\left \{ N_j \right \}_{j=1}^{n}$ are independent.

• (2) Obtain the distribution of $N_i$.

• (3) Obtain the mean $\mathsf {E} \{N\}$ and variance $\mathsf {Var} \{N\}$.

Exercise 1.36

Prove $\tilde {B}_p (r, n-r+1) = \tilde {B} (r, n-r+1) \sum \limits _{j=r}^n \, _{n}\mbox{C}_{j}p^j (1-p)^{n-j}$ shown in (1.A3.4).