Source Summary: Rosenblatt 1958 — The Perceptron
Full title: The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain
Authors: F. Rosenblatt
Affiliation: Cornell Aeronautical Laboratory
Venue: Psychological Review, Vol. 65, No. 6, pp. 386–408 (1958)
PDF: papers/rosenblatt-1958-the-perceptron-a-probabilistic-model.pdf
Contributions
- The perceptron learning algorithm — first algorithm that provably learns a linear classifier from labeled examples, given the target is linearly separable.
- Perceptron convergence theorem — if a set of patterns is linearly separable, the perceptron learning procedure will find a correct classifier in a finite number of steps.
- Probabilistic approach to brain modeling — moved from McCulloch-Pitts Boolean determinism to a probabilistic, adaptive framework that can handle noisy/imprecise neural connectivity.
- Three-layer architecture — sensory units (S-units) → association units (A-units) → response units (R-units); an early neural network architecture.
Motivation
Rosenblatt framed the perceptron as a response to two competing theories of memory:
- Coded representation: memory stores an image/code of the stimulus (like a photograph) — requires a specific mapping
- Connectionist/empiricist: memory is stored as strengthened connections between neurons — no explicit code, information is in the pattern of connectivity
The perceptron takes the connectionist position: it does not store a representation of the stimulus but learns connection weights that enable correct responses to future stimuli.
He explicitly critiques models like McCulloch-Pitts as “logical contrivances” — perfectly specified Boolean networks that cannot deal with imprecise neural connectivity or generalization.
The Perceptron Model
Architecture
[S-units] → [A-units] → [R-units]
(sensory (association (response
input) cells) output)
- S-units: photoreceptors or sensory inputs; binary, no learning
- A-units: receive excitatory/inhibitory signals from S-units; fire if weighted sum ≥ threshold θ
- R-units: output units; one fires, all others inhibit
The learning rule
For a correctly classified pattern: no weight change.
For a misclassified pattern where correct class is j but unit k fired instead:
Δwᵢⱼ = +α (for each Aᵢ that responded) ← reinforce correct
Δwᵢₖ = −α (for each Aᵢ that responded) ← penalize incorrect
This is the perceptron learning rule — a supervised update that strengthens correct associations and weakens incorrect ones.
The convergence theorem
Theorem: If a set of patterns is linearly separable (a hyperplane exists that correctly classifies all patterns), then the perceptron learning procedure will converge to a correct classifier in a finite number of error corrections.
The number of corrections before convergence is bounded by 1/δ², where δ is the margin (how far the separating hyperplane is from the nearest training points).
Key Results
- Mark I Perceptron (hardware implementation at Cornell, 1960): 20×20 pixel camera, 512 A-units, learned to classify letters
- Learned to distinguish triangles from circles, letters from each other, etc.
- Showed generalization to previously unseen instances of learned categories
What Rosenblatt Claimed (and Didn’t)
Claimed: The perceptron can learn any linearly separable function; it generalizes beyond the training set.
Claimed: The architecture is biologically plausible — random initial connectivity, local learning rule, no central controller.
Did not claim: That a single-layer perceptron can learn all functions (XOR was known to be a problem).
Did not fully address: What happens when the target function is not linearly separable.
The exuberant press coverage (New York Times: “Navy Reveals Brain-Like Computer”) overclaimed beyond the paper — but the paper itself is carefully hedged.
Limitations
- Learning is only guaranteed for linearly separable targets
- No mechanism for multi-layer training (this was the missing piece Rumelhart et al. 1986 solved)
- Binary units; no continuous activations
Key References
- McCulloch & Pitts 1943 — the logical neuron being probabilistically extended
- Hebb 1949 — learning rule inspiration (“cell assembly,” synaptic strengthening)
- Minsky & Papert 1969 — formal proof of the perceptron’s limitations (XOR, parity, connectedness)