When designing features or algorithms for learning features, our goal is usually to separate the factors of variation that explain the observed data.
In this context, we use the word “factors” simply to refer to separate sources of influence; the factors are usually not combined by multiplication.
Such factors are often not quantities that are directly observed.
Instead, they may exist either as unobserved objects or unobserved forces in the physical world that affect observable quantities.
They may also exist as constructs in the human mind that provide useful simplifying explanations or inferred causes of the observed data.
They can be thought of as concepts or abstractions that help us make sense of the rich variability in the data.
When analyzing a speech recording, the factors of variation include:
- the speaker’s age,
- their sex,
- their accent
- the words that they are speaking.
When analyzing an image of a car, the factors of variation include:
- the position of the car,
- its color,
- the angle and brightness of the sun.