Feature Engineering

Before deep learning, the dominant approach to computer vision followed a two-step recipe: (1) engineer features that capture relevant properties of the image, and (2) feed those features into a classical machine learning model.

The first step — feature engineering — required both expertise and creativity. You had to look at an image and think: "What properties would help a model distinguish a cat from a dog? Color distributions? Edge patterns? Texture statistics?" Then you had to figure out how to compute those properties mathematically and express them as numbers.

This was not a bad approach. It produced okay results, and it gave practitioners deep intuition about what makes images different from one another. But it had a fundamental ceiling, and that ceiling became apparent as the problems we wanted to solve grew more complex.