Pooling Layers
Pooling Layers: Strategic Forgetting
After a convolutional layer produces feature maps, we typically follow it with a pooling layer. The role of a pooling layer is to reduce the spatial dimensions of those feature maps, trading some spatial resolution for computational efficiency and translation robustness.
The most common approach is max pooling with a 2 × 2 window and stride 2: slide a 2 × 2 window across the feature map, and at each position, keep only the maximum value within that window. The spatial dimensions are halved. The most prominent features are preserved. The least prominent are discarded.
Pooling layers serve three purposes:
- Reduce computational complexity by shrinking feature map dimensions.
- Provide spatial invariance such that small shifts in where exactly a feature appears do not change the pooled output significantly.
- Act as a form of regularization, discarding some detail and helping prevent overfitting.
Max Pooling vs. Average Pooling
Max pooling selects the highest activation in each window. Intuitively, it answers the question "did this feature appear anywhere in this region?" — presence matters more than exact location or average intensity. This makes it well-suited to detecting whether a feature exists, rather than how prominent it is on average.
Average pooling takes the mean of all activations in the window. It retains a sense of overall activation strength across the region rather than just the peak. In practice, max pooling tends to work better for image classification tasks because CNNs care about feature presence, not average activation level.
A third variant, global average pooling (GAP), reduces each entire feature map down to a single number — its mean activation. GAP is common at the end of modern architectures (like GoogLeNet and ResNet variants) as a parameter-free alternative to a large fully connected layer, dramatically reducing the risk of overfitting.
Max vs. Average Pooling Explorer
Color intensity = activation strength
Hover an output cell to see its pooling window and the contributing input value(s). Click any input cell to change it.
Hover any output cell to highlight its pooling window and the exact value(s) that determined the result. Click any input cell to change it and watch both outputs update instantly.
Pooling Has No Learnable Parameters
Unlike convolutional layers, pooling layers have no weights to learn. They are fixed, deterministic operations: max or average over a window. This means pooling adds no parameters to your model — it only changes the spatial dimensions of the feature maps. This is one reason pooling is so computationally attractive.
Spatial Invariance: The Real Benefit
Pooling provides a degree of translation invariance within the pooling window. If a feature detector fires at position (4, 7) rather than (4, 8), max pooling over a 2 × 2 region will produce the same output either way — both activations fall in the same pool window. This small invariance accumulates through multiple pooling layers, so by the time we reach the classification head, the network is somewhat indifferent to exactly where a feature appears.
This is a desirable property for recognition tasks: a cat is still a cat regardless of whether its ear is one pixel to the left. Pooling is part of how CNNs build this robustness into their architecture.
Modern Trend: Strided Convolutions Instead of Pooling
In some modern architectures (including many in the ResNet family and most diffusion model backbones), explicit pooling layers are replaced by convolutional layers with stride 2. A strided convolution also halves spatial dimensions, but unlike pooling, it is a learned operation — the network can discover the best way to downsample for the task. Some researchers argue this gives the model more flexibility. For now, pooling layers remain a common part of the standard CNN architecture.
A feature map of shape [14, 14, 32] passes through a max pooling layer with a 2×2 window and stride 2. What is the output shape? (Hint: use the same formula as in a convolutional layer!)