Defining the Observed Significance Level of a Test: A Simple Example Using the Binomial Distribution
- 74k Downloads
When we make inferences to a population, we rely on a statistic in our sample to make a decision about a population parameter. At the heart of our decision is a concern with Type I error. Before we reject our null hypothesis, we want to be fairly confident that it is in fact false for the population we are studying. For this reason, we want the observed risk of a Type I error in a test of statistical significance to be as small as possible. But how do statisticians calculate that risk? How do they define the observed significance level associated with the outcome of a test?