The null hypothesis (\(H_0\)) is the statement of "no effect / no difference" that a test tries to reject; a Type I error is rejecting a true \(H_0\), and a Type II error is failing to reject a false \(H_0\).
Null hypothesis. A definite statement about a population parameter, assumed true until the sample gives strong evidence against it. Example: "The new teaching method does not change the mean score", \(H_0: \mu = 60\). The alternative hypothesis \(H_1\) is what we accept if \(H_0\) is rejected, e.g. \(H_1: \mu \ne 60\) (two-tailed) or \(\mu > 60\) (one-tailed).
The four possible outcomes
| Decision | \(H_0\) true | \(H_0\) false |
|---|---|---|
| Reject \(H_0\) | Type I error (\(\alpha\)) | Correct (power \(1-\beta\)) |
| Do not reject \(H_0\) | Correct (\(1-\alpha\)) | Type II error (\(\beta\)) |
Type I error (false positive). Rejecting \(H_0\) when it is actually true. Its probability is the level of significance \(\alpha\), usually 5% or 1%. Example: concluding that a medicine works when it actually does not.
Type II error (false negative). Accepting (not rejecting) \(H_0\) when it is actually false. Its probability is \(\beta\). Example: concluding that a medicine has no effect when it actually works.
Relationship. For a fixed sample size, reducing \(\alpha\) increases \(\beta\) and vice versa. Both can be reduced together only by increasing the sample size. The power of a test, \(1-\beta\), is the probability of correctly detecting a real effect.
Court analogy: \(H_0\) = "accused is innocent". Punishing an innocent person is a Type I error; letting a guilty person go free is a Type II error.