Question

What is bayes theorem's concept in simple language?

Answer 1

Bayes Theorem may be stated

$P(A|B) = \dfrac{P(B|A) P(A)}{P(B|A) P(A) + P(B|\overline{A}) P(\overline{A})}$

We have a world where an event A can occur with a certain probability P(A).  That's called the prior probability of A, what we know before anything happens.  Then we get some new information. namely that B has occurred.  Bayes Theorem tells us how to adjust our prior P(A) to get a new estimate of the probability of A given that B has occurred, written P(A|B).  This conditional probability is called the posterior probability, what we know after something has happened, after B has occurred.

We can see from our equation, the posterior probability P(A|B) depends not only on the prior probability P(A), but also on two other conditional probabilities, P(B|A) and P(B|not(A)).  In other words, the probability of A given B has occurred depends on the probability of B given A has occurred and the probability of B given A hasn't occurred.  

The typical example is medical testing.   The question is essentially how much to worry when you get a positive result, a finding of rare disease, on a test that has the possibility of false positives.  

We'll make A the event the person being tested has a certain disease, B the event the test for the disease comes back positive.   We'll make the disease rare, one in ten thousand, prior P(A)=0.0001.  We'll make the probability of a false negative (the test misses the disease)  small, i.e. P(B|A)=.999.   That says the probability of a positive test given the disease is present is very high, i.e. the probability of a false negative is low.

Let's say the false positive rate is pretty high too, P(B | not A) = .10, ten percent.  That's high, but not atypical of some tests.  The question is what is the probability that someone who tests positive has the disease.  We have to weigh the positive test against the rarity of the disease and the accuracy of the test.  That's what Bayes Theorem does.

The denominator of Bayes Theorem has two parts, one of which is in the numerator too.  Let's calculate them

P(B|A) P(A) = .999 (.0001) = .000999

P(B|not A) P(not A) = .1 (.9999) = .09999

P(A|B) = .000999 / ( .000999 +  .09999) = 0.00989... ≈ .01

Bayes Theorem tells us even though the test showed a positive result, our posterior probability of disease is only 1 percent.   That's a hundred times more than it was before the test, but we can still be legitimately hopeful the disease is absent despite the positive test.