The dependence on representations is a general phenomenon that appears throughout computer science and even daily life.
In computer science, operations such as searching a collection of data can proceed exponentially faster if the collection is structured and indexed intelligently.
People can easily perform arithmetic on Arabic numerals, but find arithmetic on Roman numerals much more time-consuming.
It is not surprising that the choice of representation has an enormous effect on the performance of machine learning algorithms.Many artificial intelligence tasks can be solved by designing the right set of features to extract for that task, then providing these features to a simple machine learning algorithm.
However, for many tasks, it is difficult to know what features should be extracted.
Suppose we want to separate two categories of data by drawing a line between them in a scatterplot.
- In the plot on the left, we represent some data using Cartesian coordinates, and the task is impossible.
- In the plot on the right, we represent the data with polar coordinates and the task becomes simple to solve with a vertical line.