Welcome to the Era of Experience

Authors: David Silver, Richard S. Sutton
Year: 2025
Source: Technical essay (Google DeepMind / University of Alberta)

Contributions

Proposed a paradigm shift from “imitation of human data” to “learning from experience” as the next frontier for AI
Argued that agents interacting with the world, receiving feedback, and improving through trial and error at superhuman scale represent the path beyond current LLM capabilities
Built on AlphaGo/AlphaZero success: learning from self-play without human data achieves superhuman performance where human data is the ceiling
Introduced the concept of “experiential agents” that generate their own training signal through environmental interaction

Historical argument tracing two eras: (1) hand-crafted knowledge, (2) human data imitation, leading to (3) experience-based learning
Comparative analysis of AlphaZero (experience-based) vs. GPT-style (imitation-based) paradigms
Identification of domains where human data creates a ceiling on performance (games, science, mathematics)
Speculative but principled argument for what a third-era system would require: environments, reward signals, long-horizon planning

Argued that LLMs trained on human text are fundamentally limited by the quality and scope of human-generated data
Demonstrated by analogy that AlphaZero’s superhuman performance in chess and Go is only possible because it transcends imitation
Outlined key requirements for experiential AI: rich environments, scalable reward functions, long-horizon credit assignment
The essay became influential in framing the research agenda for post-LLM AI development

Highly speculative: defining “experience” and appropriate reward signals in open-ended real-world domains remains unsolved
Does not address the alignment problem: how to specify rewards that capture human values in experience-based systems
The transition from imitation to experience may require entirely new architectures not yet developed
Critics note that human data may still be crucial for grounding concepts and avoiding degenerate solutions