A Path Towards Autonomous Machine Intelligence
Authors: Yann LeCun
Year: 2022 | OpenReview preprint
Source: lecun-2022-a-path-towards-autonomous-machine
What it is
LeCun’s 2022 position paper is the most thorough critique of autoregressive language models as a path to intelligence, and the most complete proposal for what should replace them. The central claim: machines that predict the next token cannot build a model of the world sufficient for reasoning, planning, or genuine understanding. The alternative is JEPA (Joint Embedding Predictive Architecture) — a world model trained in representation space, combined with a hierarchical planning architecture driven by intrinsic motivation.
The core critique of LLMs
LeCun identifies three fundamental limitations of autoregressive LLMs:
1. Token prediction ≠ world modeling
A language model predicts the next token from the previous context. This requires memorizing statistical regularities in text but not a model of the world that generated the text. The same limitation applied to GPT-3 applies to GPT-100: you can improve the statistical model indefinitely without producing causal understanding.
2. No grounding in physical reality
Language models are trained on text produced by humans who have physical, embodied experience of the world. The model inherits statistical patterns from that text but not the underlying physical knowledge. This is why LLMs confuse spatial reasoning, fail basic physics questions, and hallucinate facts.
3. Sample inefficiency
Human children learn object permanence, gravity, and basic cause-and-effect from months of physical interaction — far less data than LLMs require. The difference: humans build causal world models from sparse sensory experience; LLMs compress statistical patterns from massive text corpora.
JEPA: Joint Embedding Predictive Architecture
The key architectural proposal:
Standard generative prediction (bad):
observation(t) → Encoder → representation(t)
→ Decoder → predicted observation(t+1)
[must predict every pixel — irrelevant detail, computational waste]
JEPA (good):
observation(t) → Encoder → s(t) ─────────────────────────┐
observation(t+1) → Encoder → s(t+1) │ compare
│
s(t) + action(t) → Predictor → ŝ(t+1) ────────────────────┘
[predicts abstract representation — only structure, not surface]
The encoder compresses observations into abstract representations that capture structure relevant for prediction. The predictor models dynamics in this abstract space. Training: minimize distance between predicted and actual representations.
Why prediction in representation space is better:
- No need to predict irrelevant details (exact pixel colors)
- Representations can be learned to make the prediction problem tractable
- The model learns what changes and what persists — naturally
H-JEPA: Hierarchical World Models
JEPA stacked at multiple levels of temporal and spatial abstraction:
H-JEPA Level 3: [day-scale representations] ← long-horizon plans
H-JEPA Level 2: [minute-scale representations] ← mid-term plans
H-JEPA Level 1: [second-scale representations] ← immediate actions
H-JEPA Level 0: [raw sensory input]
Higher levels operate at coarser granularity — predicting abstract goals rather than specific motor commands. This mirrors the hierarchical control systems of the brain (prefrontal cortex → motor cortex → spinal cord).
The proposed cognitive architecture
LeCun proposes six modules for an autonomous intelligent agent:
| Module | Function |
|---|---|
| Perception | Encode current observation into abstract representation |
| World model | Predict future states given current state + candidate action (H-JEPA) |
| Cost module | Evaluate states against intrinsic and task objectives |
| Actor | Select actions via optimization through the world model |
| Short-term memory | Current state representation |
| Long-term memory | Episodic memory; persists across interactions |
The agent plans by imagining: run the world model forward for candidate action sequences, evaluate outcomes with the cost module, select the best action. This is model-predictive control implemented with a learned JEPA world model.
Intrinsic motivation
LeCun argues that external rewards (reinforcement learning) are insufficient as the primary training signal — too sparse, too slow. Instead, agents should be driven by intrinsic objectives:
- Prediction accuracy: minimize world model error (curiosity)
- Representation quality: maximize the usefulness of learned representations
- Efficiency: minimize the number of steps to achieve goals
These intrinsic objectives operate continuously, without requiring an external reward signal for each behavior.
Connection to symbolism and causality
LeCun’s world model architecture is a connectionist implementation of what symbolic AI always aspired to:
| Symbolic AI concept | JEPA equivalent |
|---|---|
| McCarthy’s world model (logical sentences) | JEPA encoder + predictor (learned) |
| Situation calculus: Result(a, s) | JEPA predictor: ŝ(t+1) = f(s(t), a(t)) |
| Pearl’s do-operator: P(Y|do(X)) | JEPA: query the world model under action a |
| Frame axioms: what persists | JEPA representations: persistence is learned |
The difference: symbolic AI required humans to specify the model; JEPA learns it from data.
See also
- maes-2026-leworldmodel-stable-end-to-end-joint-embedding-predictive — LeWorldModel: end-to-end JEPA implementation
- pearl-2000-causality-models-reasoning-and-inference — Causality: the theoretical framework JEPA implements
- mccarthy-1969-some-philosophical-problems-from-the — Frame problem: the symbolic challenge JEPA sidesteps by learning
- brown-2020-language-models-are-few-shot-learners — GPT-3: what LeCun is arguing against