Limitations of linear models

Theory
, ,
1

Linear models have many limitations.
Most famously, they cannot learn the XOR function, where:

  • f([0, 1], w) = 1
  • f([1, 0], w) = 1
  • f([1, 1], w) = 0
  • f([0, 0], w) = 0.

Critics who observed these flaws in linear models caused a backlash against biologically inspired learning in general (Minsky and Papert, 1969).
This was the first major dip in the popularity of neural networks.

Goodfellow, Bengio, Courville - «Deep Learning» (2016)