What is the difference between the «likelihood function» and «probability density function» terms?

(Dmitry Fedyuk) #1

If I have a fair coin (parameter value) then the probability that it will come up heads is 0.5.
If I flip a coin 100 times and it comes up heads 52 times then it has a high likelihood of being fair (the numeric value of likelihood potentially taking a number of forms).

The difference between probability and likelihood is closely related to the difference between probability and statistics.
In a sense probability and statistics concern themselves with problems that are opposite or inverse to one another.

The distinction between probability and likelihood is fundamentally important:

  • probability attaches to possible results;
  • likelihood attaches to hypotheses.

C. Randy Gallistel - «Bayes for Beginners: Probability and Likelihood» (2015)

Probability quantifies anticipation (of outcome), likelihood quantifies trust (in model).

Although a likelihood function might look just like a probability density function, it’s fundamentally different:

Unlike probability density functions, likelihoods aren’t normalized.
The area under their curves does not have to add up to 1.

What is a «probability density function» («PDF»)?
What is the «likelihood function»?
(Dmitry Fedyuk) #2

See also: