Neural Networks and Physical Systems with Emergent Collective Computational Abilities

Authors: John J. Hopfield
Year: 1982
Source: Proceedings of the National Academy of Sciences, 79(8)

Contributions

Introduced the Hopfield network: a recurrent neural network with symmetric weights governed by an energy function
Demonstrated content-addressable (associative) memory: the network can retrieve a complete stored pattern from a partial or noisy input
Drew an explicit analogy with spin glass physics from statistical mechanics, enabling analysis of memory capacity and error correction
Laid the conceptual groundwork for Boltzmann machines and energy-based models in deep learning

Defined a network energy function analogous to the Ising spin glass Hamiltonian
Proved that synchronous and asynchronous update rules both descend the energy landscape (no cycles in asynchronous case)
Calculated memory storage capacity analytically: approximately 0.138N patterns for N binary neurons
Used statistical physics techniques (replica method) to analyze the retrieval phase transition

Showed that a network of N neurons can reliably store and retrieve approximately 0.138N random patterns
Demonstrated that partial inputs activate attractor dynamics that converge to the nearest stored pattern
Established that spurious attractors (mixture states) are an intrinsic property of associative memory networks
Inspired the development of Boltzmann machines by Hinton and Sejnowski (1983) and the broader field of energy-based learning

Storage capacity is limited to ~0.138N patterns; beyond this, retrieval error rates increase rapidly
Spurious attractors (mixtures of stored patterns) can cause incorrect retrievals
Binary neuron model is biologically simplified; continuous-valued extensions exist but complicate analysis
Convergence in synchronous update mode is not guaranteed; asynchronous updates required for energy monotonicity