Further Properties of Time Series

Abstract

As part of the study of time series, it is important to distinguish between the terms stochastic and deterministic when considering time series. Stochastic comes from the Greek word στoχoς pronounced “stokhos” and which means “a target”. If you throw a dart at the bulls-eye on a target many times, you will probably hit the bulls-eye only a few times. At other times, the dart will miss the target and be spread randomly about that point. Stochastic processes contain such random errors.

Appendices

Appendix 5.1: The Binomial Theorem

It may be verified that (x + a)¹ = (x + a), (x + a)² = x² + 2ax + a², (x + a)³ = x³ + 3ax² + 3a²x + a³, (x + a)⁴ = x⁴ + 4ax³ + 6a²x² + 4a³x + a⁴ etc. The powers of x descend by one and the powers of a ascend by one as one moves from term to term. The numerical coefficients are given by Pascal’s triangle:

Simply sum two consecutive digits in any row and write the total in between them a row lower. Hence the next row would be 1, 5, 10, 10, 5, 1 whereby:

$$ {\left(\mathrm{x}+\mathrm{a}\right)}^5={\mathrm{x}}^5+5{\mathrm{a}\mathrm{x}}^4+10\;{\mathrm{a}}^2{\mathrm{x}}^3+10{\mathrm{a}}^3{\mathrm{x}}^2+5{\mathrm{a}}^4\mathrm{x}+{\mathrm{a}}^5. $$

Pascal’s triangle only works for (x + a)ⁿ where n is an integer. For negative and fractional values of the power n we turn to the Binomial Theorem which states that for all n:

$$ \boxed{{\left(\mathrm{x}+\mathrm{a}\right)}^{\mathrm{n}}={\mathrm{x}}^{\mathrm{n}}+{\mathrm{n}\mathrm{ax}}^{\mathrm{n}-1}+\frac{n\left(n-1\right)}{2!}{\mathrm{a}}^2{\mathrm{x}}^{\mathrm{n}-2}+\frac{n\left(n-1\right)\left(n-2\right)}{3!}{\mathrm{a}}^3{\mathrm{x}}^{\mathrm{n}-3}+\frac{n\left(n-1\right)\left(n-2\right)\left(n-3\right)}{4!}{\mathrm{a}}^4{\mathrm{x}}^{\mathrm{n}-4}+\dots } $$

where ! is the factorial notation such that, for example, 4! = 4 × 3 × 2 × 1 = 24. You may wish to verify the result for (x + a)⁵ via the Binomial Theorem. If the power n is an integer, then the number of terms involved in the expansion is (n + 1), so an expansion of the quadratic (x + a)² involves three terms, expanding a cubic involves four terms etc. However, if n is a fraction or negative then the expansion has an infinite number of terms.

As an example, consider the expansion of (1 + x)⁻¹, by replacing x by 1, a by x and n by −1 in the binomial expansion on the previous page:

$$ {\displaystyle \begin{array}{l}{\left(1+\mathrm{x}\right)}^{-1}={1}^{-1}+\left(-1\right)\mathrm{x}{.1}^{-2}+\frac{\left(-1\right)\left(-2\right)}{2!}{\mathrm{x}}^2{.1}^{-3}\\ {}+\frac{\left(-1\right)\left(-2\right)\left(-3\right)}{3!}{\mathrm{x}}^3{.1}^{-4}+\frac{\left(-1\right)\left(-2\right)\left(-3\right)\left(-4\right)}{4!}{\mathrm{x}}^4{.1}^{-5}+\dots {\left(1+x\right)}^{-1}=1\hbox{--} \mathrm{x}\\ {}+{\mathrm{x}}^2\hbox{--} {\mathrm{x}}^3+{\mathrm{x}}^4+\dots \end{array}} $$

This series expansion is convergent if and only if |x| < 1. The same rule of convergence applies to the series expansion for (1 − x)ⁿ. By convergent, I mean that the sum of the series approaches a finite limit. For example, the geometric series $1+\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\dots$ converges towards a limit of 2 as you add more and more terms. Conversely the geometric series 1 + 4 + 16 + 64 + … is divergent because it becomes larger and larger as you add more terms. The latter series does not approach a finite limit; rather the sum becomes infinite.

Given the above rule for convergence , we can insert x = 0.01 into the above series expansion to establish that:

$$ {(1.01)}^{-1}=1-0.01+{(0.01)}^2\hbox{--} {(0.01)}^3+{(0.01)}^4+\dots $$

and summing the five terms on the right hand side, we obtain that (1.01)⁻¹ is approximately equal to 0.99009901. In fact, (1.01)⁻¹ = 0.990099009. Note that it would be invalid to insert x = 3 in the above series expansion in order to approximate (4)⁻¹, because x = 3 is outside the range of convergence. The series would diverge away from the correct answer that (4)⁻¹ = 0.25.

Appendix 5.2: The Quadratic Equation

The equation y = ax² + bx + c in which a, b and c are constants and a ≠ 0 is called a quadratic. When we solve a quadratic equals zero we are said to be establishing the roots of that quadratic (i.e. where it cross the x-axis). For example, to solve x² − x − 12 = 0 we use the quadratic’s factors i.e. (x − 4)(x + 3) = 0 and the roots are x = 4 and x = −3. The quadratic x² − 10x + 25 = 0 has repeated roots, since (x − 5)(x − 5) = 0 and x = 5.

Sometimes the roots of a quadratic may be a complex number. A complex number is of the form c + id where c and d are positive or negative constants and is defined to equal $\sqrt{-1}$. For example, $4+7\mathrm{i}=4+7\sqrt{-1}$ is a complex number in which 4 is called the real (Re) part and 7i is called the imaginary (Im) part. The real part of a complex number can be zero, for example $\sqrt{-16}=4\mathrm{i}$ since 4i × 4i = 16 × i² = −16. Similarly, $\sqrt{-289}=17\mathrm{i}$. A complex number may be represented on what is called an Argand diagram in which the horizontal axis is the Re part and the vertical axis is the Im part. Figure 5.2 graphs the complex number 2 + 3i. The line joining the origin to this point is called a vector. The length (or modulus) of this vector is found by Pythagoras to be $\sqrt{2^2+{3}^2}=3.606$. Similarly, the length of the vector representing 2 − 3i would also be 3.606.

Fig. 5.2

An Argand plot of the complex number 2 + 3i

Consider solving the quadratic ax² + bx + c = 0. Divide throughout by a to derive ${\mathrm{x}}^2+\frac{b}{a}\mathrm{x}+\frac{c}{a}=0$. Add $\frac{b^2}{4{a}^2}$ to both sides:

$$ {\displaystyle \begin{array}{c}{\mathrm{X}}^2+\frac{b}{a}\mathrm{x}+\frac{b^2}{4{a}^2}=\frac{b^2}{4{a}^2}-\frac{c}{a}\\ {}{\left(\mathrm{x}-\frac{b}{2a}\right)}^2=\frac{b^2-4 ac}{4{a}^2}\\ {}\mathrm{x}-\frac{b}{2a}=\frac{\pm \sqrt{b^2-4 ac}}{2a}\end{array}} $$

(since a square root can be + or −, e.g. $\sqrt{144} = \pm 12$) and we establish that:

$$ \boxed{\mathrm{x}=\frac{-b\pm \sqrt{b^2-4 ac}}{2a}.} $$

This formula gives the roots of a quadratic. For example, if we are to solve x² − 13x + 42 = 0, we have that a = 1, b = −13 and c = 42, whereby:

$$ \mathrm{X}=\frac{13\pm \sqrt{169-168}}{2}=\frac{13\pm 1}{2} $$

and the roots are x = 7 and x = 6. These are real roots as opposed to complex roots. Hence when factorised, the quadratic must be (x − 6)(x − 7) = 0. Note that a quadratic can only have either two real roots, two complex roots or a repeated root; it cannot have one real root and one complex one. For a general polynomial of order n, complex roots always appear in pairs i.e. there is an even number of them.

The quantity b² − 4ac in the above formula for the roots of a quadratic is called the discriminant . If the discriminate is negative, the quadratic must have complex roots (since we are taking the square root of a negative number). If the discriminant equals zero, then the quadratic has repeated roots. Consider x² − 4x + 28 = 0. The discriminant, 16 − 4(28), is clearly negative and

$$ \mathrm{X}=\frac{4\pm \sqrt{16-112}}{2}=\frac{4\pm \sqrt{-96}}{2}=\frac{4\pm \sqrt{(16)\left(-6\right)}}{2}=2\pm 2\sqrt{-6}=2\pm 4.898\mathrm{i}\kern1em \mathrm{since}\kern0.5em \sqrt{6}=2.449. $$

This quadratic has a pair of complex roots x = 2 + 4.898i and x = 2 − 4.898i.

There is a relationship between the coefficients of a quadratic and the roots of the quadratic. Suppose a quadratic has roots δ and γ. Then solving the quadratic equals zero:

$$ {\displaystyle \begin{array}{c}{\mathrm{ax}}^2+\mathrm{bx}+\mathrm{c}=0\ \mathrm{implies}\ \mathrm{that}\kern0.5em \left(\mathrm{divide}\ \mathrm{throughout}\ \mathrm{by}\ \mathrm{a}\right)\\ {}{\mathrm{x}}^2+\frac{b}{a}\mathrm{x}+\frac{c}{a}=0\end{array}} $$

but we know the roots are δ and γ, so (x − δ)(x − γ) = x² − (δ + γ)x + δγ = 0.

The only way that the last two lines can be equal to zero is if:

$$ \boxed{\begin{array}{l}\mathrm{Sum}\ \mathrm{of}\ \mathrm{the}\ \mathrm{roots},\delta +\gamma =-\frac{b}{a}\\ {}\mathrm{and}\kern0.4em \mathrm{Product}\ \mathrm{of}\ \mathrm{the}\ \mathrm{roots},\delta \gamma =\frac{c}{a}\end{array}} $$

Reconsider the roots of x² − 4x + 28 = 0, which we have seen are x = 2 + 4.898i and x = 2 − 4.898i. Now for this quadratic, a = 1, b = −4 and c = 28. Therefore the sum of the roots should be equal to $-\frac{b}{a}=4$. The sum of the roots is (2 + 4.898i) + (2 − 4.898i) which is indeed 4. The product of the roots should be equal to $\frac{c}{a}=28$. As a check, the product of the roots is (2 + 4.898i) (2 − 4.898i) = 4 + 0i + (4.898i)(−4.898i) = 4 − 24i² = 4 + 24 = 28, as required, since i² = −1.