Type I and II errors
C. Supakorn and V. Cave
Hypothesis testing is the art of making decisions using data. It involves evaluating two mutually exclusive statements: the null hypothesis (Ho) and the alternative hypothesis (Ha). The strength of evidence against the null hypothesis (as provided by the observed data) is often measured using a p-value. The smaller the p-value, against the null hypothesis, the greater the proof.
A p-value is a probability of observing a test statistic at least as extreme as the one observed from the data assuming
the null hypothesis is true.
When we make a decision using a hypothesis test, there are four possible outcomes: two representing correct decisions and two representing incorrect decisions. The incorrect decisions are due to either Type I or Type II errors.
Type I error (or a “false positive”) occurs when we reject a null hypothesis when in fact it is true. In other words, this is the error of accepting the alternative hypothesis when the results can be attributed to chance. In hypothesis testing, the probability of making a Type I error is often referred to as the “level of significance” and denoted by alpha (α). Typically, the Type I error rate is set to 0.05 giving a 1 in 20 chance (i.e. 5%) that a true null hypothesis will be rejected.
Type II errors are also referred to as “false negatives.” These occur when we fail to reject the null hypothesis when in fact it is false. The probability of making a Type II error is usually denoted by β and depends on the power of the hypothesis test (β = 1 – power). You can reduce the chance of making a Type II error by ensuring your hypothesis test has enough power.
The strength of a hypothesis test is the probability of denying
an incorrect null hypothesis.