Bayes' theorem gives the probability of a cause given an observed effect: \(P(A_i\mid B) = \dfrac{P(A_i)\,P(B\mid A_i)}{\sum_j P(A_j)\,P(B\mid A_j)}\).
Statement. If \(A_1, A_2, \dots, A_n\) are mutually exclusive and exhaustive events and \(B\) is any event with \(P(B) > 0\), then
\[P(A_i\mid B) = \frac{P(A_i)P(B\mid A_i)}{P(B)}\]
where, by the total probability theorem, \(P(B) = \sum_j P(A_j)P(B\mid A_j)\).
- \(P(A_i)\): prior probability (before seeing evidence).
- \(P(B\mid A_i)\): likelihood.
- \(P(A_i\mid B)\): posterior probability (after seeing evidence).
Example. Machines A, B and C produce 50%, 30% and 20% of a factory's bolts. Their defect rates are 3%, 4% and 5%. A bolt picked at random is defective. Find the probability it came from machine A.
Total probability of a defective bolt:
\[P(D) = 0.5(0.03) + 0.3(0.04) + 0.2(0.05)\]
\[P(D) = 0.015 + 0.012 + 0.010 = 0.037\]
By Bayes' theorem:
\[P(A\mid D) = \frac{0.015}{0.037} \approx 0.405\]
Similarly \(P(B\mid D) = 0.012/0.037 \approx 0.324\) and \(P(C\mid D) = 0.010/0.037 \approx 0.270\) (they add to 1).
Uses: medical test interpretation, spam filtering, Naive Bayes classification in machine learning, and updating beliefs as new data arrives.