## Distinguishing Likelihood From Probability

The distinction between probability and likelihood is fundamentally important:

probabilityattaches topossible results;likelihoodattaches tohypotheses.

Possibleresults aremutually exclusiveandexhaustive.

Suppose we ask a subject to predict the outcome of each of10tosses of a coin.

There are only11possible results (0to10correct predictions).

The actual result will always be one and only one of the possible results.

Thus, theprobabilities that attach to the possible results must sum to 1.

Hypotheses, unlike results, areneither mutually exclusive nor exhaustive.

Suppose that the first subject we test predicts7of the10outcomes correctly.

I might hypothesize that the subject justguessed, and you might hypothesize that the subject may be somewhatclairvoyant, by which you mean that the subject may be expected to correctly predict the results at slightly greater than chance rates over the long run.

These aredifferent hypotheses, but they arenot mutually exclusive, because you hedged when you said “may be.”

You therebyallowed your hypothesis to include mine.

In technical terminology,my hypothesis is nested within yours.

Someone else might hypothesize that the subject isstrongly clairvoyantand that the observed result underestimates the probability that her next prediction will be correct.

Another person could hypothesize something else altogether.

There is no limit to the hypotheses one might entertain.

The set of hypotheses to which we attach likelihoods is

limited by our capacity to dream themup.

In practice, we canrarely be confident that we have imagined all the possible hypotheses.Our concern is to estimate the extent to which the experimental results affect the relative likelihood of the hypotheses we and others currently entertain.

Because we generally

do not entertain the full set of alternative hypothesesand becausesome are nested within others, the likelihoods that we attach to our hypotheses do not have any meaning in and of themselves;only the relative likelihoods — that is, the ratios of two likelihoods — have meaning.

## Using the Same Function ‘Forwards’ and ‘Backwards’

The difference between probability and likelihood becomes clear when one uses the probability distribution function in general-purpose programming languages.

In the present case, the function we want is the binomial distribution function.

It is called

`BINOM.DIST`

in the most common spreadsheet software and`binopdf`

in the language I use.

It has three input arguments:

- the
number of successes,- the
number of tries,- the
probability of a success.

When one uses it to compute probabilities, one assumes that the latter two arguments (number of tries and the probability of success) are

given.

They are the parameters of the distribution.

One varies the first argument (the different possible numbers of successes) in order to find the probabilities that attach to those different possible results.

Regardless of the given parameter values, theprobabilities always sum to 1.

The binomial probability distribution function, given

10tries atp = .5.

The binomial likelihood function, given

7successes in10tries.

Both panels were computed using the

`binopdf`

function.

- In the first panel, I varied the possible results.
- In the second panel, I varied the values of the
pparameter.

- The probability distribution function is
discretebecause there are only11possible experimental results (hence, a bar plot).- By contrast, the likelihood function is
continuousbecause the probability parameter p can take on any of the infinite values between0and1.

- The
probabilitiesin the first plot sum to1.- The integral of the continuous
likelihoodfunction in the bottom panel is muchless than 1; that is, the likelihoods do not sum to 1.

psychologicalscience.org/observer/bayes-for-beginners-probability-and-likelihood