Probability is a number between 0 and 1 that measures how likely an event is; in statistical inference it is the tool that lets us draw conclusions about a population from a sample and state how confident we are.
Definition (classical). If an experiment has \(n\) equally likely outcomes and \(m\) of them favour event \(A\),
\[P(A) = \frac{m}{n},\qquad 0 \le P(A) \le 1.\]
Example: probability of getting an even number on a fair die = 3/6 = 0.5. Other views: relative frequency (proportion of times the event occurs in many trials) and axiomatic (Kolmogorov: \(P(A) \ge 0\), \(P(S) = 1\), and probabilities of mutually exclusive events add).
Significance in statistical inference
- Handling sampling uncertainty: a sample mean differs from the population mean by chance. Probability distributions (normal, t, chi-square, F) describe how large that chance variation can be.
- Estimation: confidence intervals are built from probability. Example: if 400 students have mean height 160 cm with standard deviation 8 cm, a 95% interval for the population mean is \(160 \pm 1.96\times 8/\sqrt{400} = 160 \pm 0.78\) cm.
- Hypothesis testing: the p-value is the probability of getting a result at least as extreme as observed if \(H_0\) is true. Example: a coin gives 9 heads in 10 tosses; \(P(\ge 9 \text{ heads}) = 11/1024 \approx 0.011\) for a fair coin, which is small, so we doubt the coin is fair.
- Measuring errors: the significance level \(\alpha\) is the probability of a Type I error; power \(1-\beta\) is also a probability.
- Prediction and models: regression, Bayesian updating and machine-learning classifiers (e.g. Naive Bayes) express outputs as probabilities.
Without probability, a sample could only describe itself; with it, we can generalise to the population with a known risk of being wrong.