The Perceptron and Its Limits
In 1958, a young assistant professor named Frank Rosenblatt made the next leap. He called it the Perceptron, and he described it as "the first machine having human qualities." That quote, made in 1958, lands differently when you read it today.
The Perceptron kept the same weighted-sum structure as McCulloch and Pitts' neuron, but added one critical ingredient: a threshold. The output is 1 if the weighted sum exceeds the threshold, and 0 otherwise. This is the simplest possible activation function.
More importantly, Rosenblatt showed that the weights in the Perceptron could be learned from data. This was the moment that turned a mathematical curiosity into a trainable machine.
The Non-Linearity Problem
But the Perceptron has a fundamental limitation: it can only represent linear decision boundaries.
Imagine you are trying to classify data points — like houses to buy or not to buy — and the boundary between them is curved or complex. A single neuron can only draw a straight line. With a two-input network, the decision boundary is always:
w₁x₁ + w₂x₂ = 0
That is a line. Always a line. If your data is not linearly separable — and most real-world data is not — a single Perceptron cannot solve the problem.
This limitation was famously documented by Minsky and Papert in 1969, and it contributed significantly to the first AI Winter. The field had over-promised. The solution, when it arrived, was beautifully simple.
Real-World Application
Linear separability is a useful lens for thinking about when simple models will and won't work. If you are building a binary classifier for fraud detection, for example, the boundary between fraud and non-fraud is almost certainly non-linear — fraud patterns are complex, context-dependent, and adversarial. A logistic regression might give you 80% accuracy on straightforward cases, but the edge cases that matter most are usually exactly where linear models break down.
A single Perceptron is trained to classify emails as spam or not spam. The training data is not linearly separable. What is the fundamental consequence of this?