Theory of Probability

Abstract

The theory of probability is the mathematical framework for the study of the probability of occurrence of events. The first step is to establish a method to assign the probability of an event, for example, the probability that a coin lands heads up after a toss. The frequentist—or empirical—approach and the subjective—or Bayesian—approach are two methods that can be used to calculate probabilities. The fact that there is more than one method available for this purpose should not be viewed as a limitation of the theory, but rather as the fact that for certain parts of the theory of probability, and even more so for statistics, there is an element of subjectivity that enters the analysis and the interpretation of the results. It is therefore the task of the statistician to keep track of any assumptions made in the analysis, and to account for them in the interpretation of the results. Once a method for assigning probabilities is established, the Kolmogoroff axioms are introduced as the ‘rules’ required to manipulate probabilities. Fundamental results known as Bayes’ theorem and the theorem of total probability are used to define and interpret the concepts of statistical independence and of conditional probability, which play a central role in much of the material presented in this book.