Theoretical Foundation for Statistical Analysis

Abstract

This chapter introduces the reader to the theoretical foundations of statistical analysis by presenting a rigorous, multivariate, set-based approach to probability and statistical theory. The foundation is laid with consideration of the key statistical axioms and definitions, which formalise many of the concepts introduced in Chapter 1. Probability density functions, sample space, moments, the expectation operator, and various marginal functions are examined. Next, the most common statistical distributions, including the normal, Student’s t-, χ²-, F-, binomial, and Poisson distributions, are described by providing their key mathematical properties and computational implementation. Using these ideas, the subject of parameter estimation, that is, determining unknown values given a data set and an assumed model, is considered. Key topics include method of moments estimation, likelihood estimation, and regression estimation. Finally, the ability to compare two statistical variables using hypothesis testing and confidence intervals is introduced for many different commonly encountered cases, including means, variances, ratios, and paired values. Detailed examples are provided for all of the key concepts using simple, but relevant, examples. By the end of the chapter, the reader should have a strong understanding of the mathematical framework of statistics. As well, the ability to estimate parameters for a given situation and conduct appropriate hypothesis testing should be understood.

Appendix A2: A Brief Review of Set Theory and Notation

In mathematics, sets are defined as a collection of objects that share some type of property. A set is delimited using curly brackets “{}” and often denoted using double struck letters ($\mathbb{A}$, $\mathbb{B}$, ℂ,…). Common sets include:

• 1. $\mathbb{R}$ (U + 211D),⁵ which is the set of all real numbers;

• 2. ℕ (U + 2215), which is the set of all natural numbers and defined as ℕ = {0, 1, 2,…};

• 3. ℤ (U + 2124), which is the set of all integers;

• 4. ℂ (U + 2102), the set of complex numbers; and

• 5. {} or ∅ (U + 2205), the null or empty set. This is used to represent a set that contains no members or elements.

The element operator ∈ (U + 2208) states that a given variable is a member or element of the set, for example, 1 ∈ ℕ, states that 1 belongs to (or is an element of) the set of natural numbers. The exclusion operator \ states that some given set is to be excluded, for example, ℕ\{0} is the set of natural numbers excluding zero.

There are two common set operations: union and intersection. The union of two sets, denoted as ∪ (U + 222A), is the set that contains all elements found in both sets, while the intersection of two sets, denoted as ∩ (U + 2229), is the set that contains only those elements that are common (found) in both sets. For example, if $\mathbb{A}$ = {1, 2, 3, 4} and $\mathbb{B}$ = {4, 5, 6, 7}, then $\mathbb{A}$ ∪ $\mathbb{B}$ = {1, 2, 3, 4, 5, 6, 7}, while $\mathbb{A}$ ∩ $\mathbb{B}$ = {4}.