If I have a fair coin (parameter value) then the probability that it will come up heads is 0.5.
If I flip a coin 100 times and it comes up heads 52 times then it has a high likelihood of being fair (the numeric value of likelihood potentially taking a number of forms).
stats.stackexchange.com/a/2642
The difference between probability and likelihood is closely related to the difference between probability and statistics.
In a sense probability and statistics concern themselves with problems that are opposite or inverse to one another.
stats.stackexchange.com/a/2649
The distinction between probability and likelihood is fundamentally important:
- probability attaches to possible results;
- likelihood attaches to hypotheses.
C. Randy Gallistel - «Bayes for Beginners: Probability and Likelihood» (2015)
Probability quantifies anticipation (of outcome), likelihood quantifies trust (in model).
stats.stackexchange.com/a/56035
Although a likelihood function might look just like a probability density function, it’s fundamentally different:
- A probability density function is a function of x, your data point, and it will tell you how likely it is that certain data points appear.
- A likelihood function, on the other hand, takes the data set as a given, and represents the likeliness of different parameters for your distribution.
Unlike probability density functions, likelihoods aren’t normalized.
The area under their curves does not have to add up to 1.
statisticshowto.datasciencecentral.com/likelihood-function#post-47406