Question

[6 points] Prove Bayes’ Theorem. Briefly explain why it is useful for machine learning problems, i.e., by converting posterior probability to likelihood and prior probability.

Answer 1

Answer: Mathematically Bayes’ theorem is defined as

P(A\B)=P(B\A) ×P(A)

P(B)

Bayes theorem is defined as where A and B are events, P(A|B) is the conditional probability that event A occurs given that event B has already occurred (P(B|A) has the same meaning but with the roles of A and B reversed) and P(A) and P(B) are the marginal probabilities of event A and event B occurring respectively.

Step-by-step explanation: for example, picking a card from a pack of traditional playing cards. There are 52 cards in the pack, 26 of them are red and 26 are black. What is the probability of the card being a 4 given that we know the card is red?

To convert this into the math symbols that we see above we can say that event A is the event that the card picked is a 4 and event B is the card being red. Hence, P(A|B) in the equation above is P(4|red) in our example, and this is what we want to calculate. We previously worked out that this probability is equal to 1/13 (there 26 red cards and 2 of those are 4's) but let’s calculate this using Bayes’ theorem.

We need to find the probabilities for the terms on the right-hand side. They are:

P(B|A) = P(red|4) = 1/2

P(A) = P(4) = 4/52 = 1/13

P(B) = P(red) = 1/2

When we substitute these numbers into the equation for Bayes’ theorem above we get 1/13, which is the answer that we were expecting.