Gentle Introduction Into Bayesian Inference

Following on from a previous post, lets look into Bayesian inference.

Recap

Recall that under Bayesian probability which have this formula.

$$ P(A \cap B) = P(A)\times P(B | A) $$

We can rearrange this:

$$ P(A | B) = P(A)\times \frac{P(B|A)}{P(B)} $$

Now we have the equation for Bayesian inference.

The central idea for Bayesian inference is this:

$$ posterior \propto prior \times likelihood $$

Right now all these things don’t really make sense, so lets revisit the equation we have above. Instead of just having events (A) and (B), we can instead use something more meaningful.

$$ P(Truth | Evidence) = P(Truth)\times \frac{P(Evidence | Truth)}{P(Evidence)} $$

Here we can say that (P(Truth) is the prior, (\frac{P(Evidence | Truth)}{P(Evidence)} ) is the likelihood, and (P(Truth | Evidence)) is the posterior.

The idea in Bayesian inference is that the posterior is proportional to combining prior and likelihood. When making comparisons we don’t really care about absolute amounts, but instead only the relative difference when making inference.

A Simple Example

Lets suppose we have a medical test.

• The probability of having a disease is 10% (this is our prior)
• Having a positive test and having disease is 95% (this is our likelihood)
• Having a positive test and not having disease is 20% (this is our likelihood)

So then for our posterior (which is the probability of having the disease given evidence of a positive test)

Looking at these numbers we can see that it is still more likely that you don’t have a disease after getting a positive result (since (0.18 > 0.095)).