The Artificial Neuron
Before we build a skyscraper, we need to understand a single brick. In neural networks, that brick is the artificial neuron.
The biological neuron has three main components that matter to us: dendrites, which receive incoming signals; an axon, which carries the processed signal forward; and terminals, which pass the signal to the next neuron. These signals are electrochemical — a mix of electrical and chemical processes that scientists have been trying to replicate mathematically for decades.
McCulloch and Pitts' thought: what if we could model this with math?
Their answer was elegant and, in hindsight, obvious. An artificial neuron computes a weighted sum of its inputs:
f(x, w) = x₁w₁ + x₂w₂ + … + xₙwₙ
Each input xi has a corresponding weight wi. The neuron multiplies each input by its weight and adds everything up. In the original 1943 model, those weights were set entirely by hand — there was no learning yet, just configuration.
This Is Just Linear Regression
If this equation looks familiar, trust that instinct. It is familiar — this is linear regression. And this is not a coincidence.
Linear regression is the foundational operation of virtually every algorithm in machine learning. The neuron is a linear regression. Stacked neurons become neural networks. The sophistication comes not from abandoning linearity, but from combining many linear operations in clever ways — and adding just enough non-linearity to unlock remarkable capabilities.
In what sense is a single artificial neuron equivalent to a linear regression model? What does each weight represent?
Real-World Application
In practice, every prediction your neural network makes is ultimately a series of weighted sums passed through transformations. When you are debugging a model that is producing nonsense outputs, one of the first questions to ask is: "Are my inputs scaled properly?" Because a weighted sum is sensitive to the magnitude of its inputs, preprocessing your data — normalizing or standardizing it — is not optional housekeeping. It is foundational to getting the model to work at all.