Contributions
- AlphaGo: first computer program to defeat a professional human Go player (Fan Hui, Oct 2015; Lee Sedol 4-1, March 2016)
- Demonstrated that neuro-symbolic hybrid architecture — neural network value/policy functions combined with symbolic tree search (MCTS) — can solve a problem long considered intractable for computers
- Introduced the combination of: supervised learning from human games → RL self-play → MCTS guided by learned networks
- Showed that evaluation (how good is this position?) and search (what to explore?) can be learned end-to-end from gameplay
Methodology
AlphaGo uses four components:
- Supervised Learning policy network (13-layer CNN, 192 filters): trained on 30M positions from KGS Go Server games; predicts expert moves, accuracy ~57%
- Reinforcement learning policy network: initialized from SL policy, then improved by self-play — gradient from REINFORCE with win/loss reward
- Value network (same CNN architecture): trained on positions from self-play; estimates P(win) from position without search
- Monte Carlo Tree Search: combines all networks — uses RL policy to guide simulation, SL policy for rollouts, value network for evaluation; selects moves by UCB (Upper Confidence Bound)
The MCTS search tree uses: Q(s,a) + u(s,a) where u is exploration bonus proportional to the SL policy prior over action a, decreasing with visits.
Results
- Beat Fan Hui (European champion) 5-0 (October 2015)
- Beat Lee Sedol (world champion) 4-1 (March 2016)
- Estimated Elo ~3500, compared to top human ~3700 at the time
- Go was the last major board game where humans held an advantage — a 20-year benchmark for AI progress
Limitations
- Relies on human expert games for SL initialization — removed in AlphaGo Zero (2017), which learned purely from self-play
- Computationally intensive: used 1202 CPUs and 176 GPUs for distributed search during matches
- Domain-specific: architecture choices (position encoding, symmetry) tailored to Go
- MCTS is a symbolic, non-differentiable component — the full system is a hybrid, not end-to-end differentiable