AI Data Center: Building the AI Factory
Type: Overview — synthesizes industry reports + multiple papers Companion: AI Chip and Memory: From Transistors to HBM Sources: “AI 데이터 센터 동향 및 전망” (2026-03-06), “AI 생태계와 AI 반도체 시장 전망” (2026-03-25), Kim Young-Beom (KIS)
What an AI data center is — and is not
A traditional IDC (Internet Data Center) is a warehouse for CPU servers and databases that serve web traffic. An AIDC (AI Data Center, sometimes called an AI Factory) is a different machine: thousands of GPUs wired into one logical accelerator, optimized end-to-end for training a large neural network and then serving its inference at low latency.
The differences are not incremental.
| Dimension | IDC | AIDC |
|---|---|---|
| Primary compute | CPU servers, DBs | GPU/NPU accelerators + HBM memory |
| Rack power density | 2–20 kW (centers 10–100 MW) | 20–120+ kW (B200 ≈ 1 kW per GPU; centers 100 MW – 1+ GW) |
| Cooling | Air (≈20–30 kW/rack ceiling) | Liquid (thermal conductivity ≈24× air) — direct-to-chip or immersion |
| Network | Ethernet | InfiniBand or RoCE for ultra-low-latency GPU↔GPU |
| Capex split (IT : facility) | ≈ 5 : 5 | ≈ 6 : 4 to 7 : 3 |
| Failure mode | Single-node outages | Job restarts at the cluster level — checkpointing dominates SSD design |
A rack that draws 120 kW is the electrical equivalent of forty homes packed into one cabinet. The heat has nowhere to go through air, so liquid cooling — and the chiller plants, CDUs, and secondary loops that come with it — becomes part of the building, not an option.
This page surveys (1) the AIDC build, (2) the 1 GW economics, (3) the demand pull from multimodal and physical AI, (4) the power and grid constraints, and (5) the industry forecast.
1. Anatomy of a 1 GW AI factory
The reference design in 2026 is a fleet of GB200 NVL72 racks (72 Nvidia Blackwell GPUs + 36 Grace CPUs in one liquid-cooled rack at ≈120 kW each). One gigawatt of contracted power, with PUE ≈ 1.2, yields about 830 MW of usable IT power — enough for roughly 6,900 NVL72 racks, i.e. about half a million GPUs.
| Component | Count in a 1 GW AIDC |
|---|---|
| GB200 NVL72 racks | ≈6,916 |
| Blackwell GPUs (72/rack) | ≈497,952 |
| Grace CPUs (36/rack) | ≈248,976 |
| HBM cubes (8 per GPU) | ≈4 million |
| 12-inch wafers consumed | ≈89,000 |
| HBM total | 95–143 PB |
| CPU DRAM (LPDDR5X) total | ≈120 PB |
| Local NVMe (NAND) total | ≈3.7 EB |
| Capex | ≈$32–36B (≈ ₩50 trillion) |
Two implications. First, a single 1 GW AI factory consumes most of one quarter of global HBM output — supply is genuinely scarce, not artificially constrained. Second, the bill of materials is dominated by IT equipment rather than concrete and chillers. A traditional hyperscale data center is a real-estate play; an AI factory is a semiconductor capital-expenditure play that happens to need a building around it.
2. The 1 GW economics — TCO and break-even
Stack the costs over a 4-year service life of the GPUs:
| Line | Amount |
|---|---|
| Capex (50% facility CSA/MEP, 60–70% IT) | ≈₩50 trillion (≈$35B) |
| Electricity (1 GW × 8,760 h × ≈₩180/kWh) | ≈₩1.58 T/yr |
| Operations + network maintenance (NVL72 ≈ 2% of Capex/yr, vs. H100’s 3–5%) | ≈₩1.0 T/yr |
| 4-year Opex | ≈₩10.3 T |
| 4-year TCO | ≈₩60.3 T (≈$43B) |
Saleable GPU-hours over 4 years: 500,000 GPUs × 24 × 365 × 4 = 17.5 billion hours; at 80% utilization, ≈14 billion sellable hours. The break-even price comes out to:
BEP ≈ ₩60.3 T ÷ 14 billion h ≈ ₩4,300/h ≈ $3/h per GPU
Current AWS on-demand B200 list is roughly **5–7/h: their cost-of-goods is the same $3 the hyperscalers pay, and they capture the spread that hyperscalers reserve for their margin.
This same math is also why hyperscalers shifted from leasing colos to owning purpose-built AI factories. Self-build amortizes the building into TCO; leasing pays the colocator’s own margin on top of TCO.
3. Why demand keeps growing — the token explosion
Why aren’t existing data centers enough? Because each new modality multiplies the tokens that must be processed at training and inference time.
| Workload | Input tokens (relative) | Output tokens (relative) | Total |
|---|---|---|---|
| LLM chatbot / agent | 1× | 1× | 1× |
| Multimodal (video gen, diffusion) | 1× | 1,000× (denoising steps) | ≈1,000× |
| Physical AI (autonomous driving) | 1,000× | 10× | ≈10,000× |
| Physical AI (humanoid robot) | 1,000× | 100× | ≈100,000× |
A4 paper of text is about 200,000 tokens; a self-driving car turns its camera/lidar feed into tokens at orders of magnitude that pace. Compute and memory budgets must rise on the same exponent.
The corresponding training-time table (the report’s worked numbers, assuming H100 100,000 GPUs, BF16 at 989 TFLOPS, 40% MFU):
| Model | Params | Tokens | H100 ×10⁵ time | Rubin ×10⁵ time |
|---|---|---|---|---|
| GPT-3 (2020) | 175B | 300B | ≈1.3 hours | ≈30 minutes |
| GPT-4 (2023) | 1.8T | 15T | ≈55 days | ≈10 days |
| GPT-5.2 (projected) | 5T | 25T | ≈293 days | ≈52 days |
| Sora2 (video diffusion) | 30B | 50T patches | ≈2.4 years | ≈155 days |
| Tesla FSD (autonomous) | 10B | — | — | ≈1.4 years |
| Atlas (humanoid) | 70B | — | — | ≈14 years |
The takeaway is not that humanoid training is impossible — it’s that physical AI requires orders of magnitude more compute than language AI, so every GW-scale data center plan being announced today is anchored on a 5-to-10-year roadmap of these workloads.
4. Power and the grid
Add it all up: AI server power demand grows from ≈10.8 GW (2025) to ≈55.9 GW (2029); installed AIDC capacity from ≈19.8 GW to ≈64.4 GW. At 2026 the utilization is ≈ 17.9 / 29.1 ≈ 61%.
The big-tech plans alone require:
- Microsoft Stargate: 5 GW
- Meta Prometheus (1 GW) + Hyperion (5 GW)
- xAI Colossus: 1 GW
- AWS Cumulus: 1 GW
- Google: undisclosed, but multi-GW
Globally, data center electricity rises from ≈415 TWh in 2024 to a projected ≈945 TWh by 2030 (≈1.5% → ≈3% of world electricity). About 60% of that is servers; cooling is ≈7% in hyperscale and 30%+ in enterprise.
Power supply cannot keep up at this slope. Nuclear is the obvious answer but a 1.4 GW reactor takes 8–10 years and ≈₩7 T to build. SMRs (Small Modular Reactors, ≤300 MW, ≈₩1T each, factory-built and field-assembled, passive cooling) are the more credible 2030s answer; Microsoft, Amazon, and Oracle have all signed SMR PPAs. RE100 (renewables) covers the marketing side but cannot deliver baseload at GW scale.
In Korea specifically: 2024 installed capacity ≈144 GW, peak ≈97 GW. Transmission, not generation, is the binding constraint — most existing capacity is on the south coast (nuclear, coal, LNG) while data-center load is in the Seoul metropolitan area, so HVDC corridor expansion is what unlocks AIDC siting. Industrial electricity prices rose ≈267% over five years, pushing operators to consider Hokkaido, Malaysia, the Middle East, and Nordic siting.
5. The competitive landscape
Where capex actually goes (global AIDC investment, 2025–2026):
- Cloud Big-5 (Microsoft, AWS, Google, Meta, Oracle): 50–60% of global AIDC capex
- Colocation / leasing (Equinix, Digital Realty, KT, etc.): 15–20%
- Neoclouds / AI-native infra (CoreWeave, Nebius, Lambda, Crusoe, IREN): 10–15%
- Sovereign AI (China, EU, UK, Canada, India, Korea, Japan, UAE): 10–15%
Sovereign-AI capex is the new line item — countries treating AI infrastructure the way they treated semiconductors and telecom: as strategic, subsidized, and locally-owned.
Inside the capex envelope, AI silicon’s share is rising from ≈27% of total AI infra spend in 2023 to a projected ≈78% in 2029. That is the math behind Nvidia’s market cap.
Why TSMC and SK hynix sit at the choke point
A 1 GW AIDC = ≈500k GPUs = ≈89k wafers. SK hynix’s projected 2026 4Q capacity is about 205k wafer-starts/month, Samsung’s about 210k — of which only a fraction can be HBM-grade. One mega-build is roughly one quarter of one vendor’s HBM-eligible capacity. Combined with TSMC’s CoWoS-L bottleneck (see the chip overview), the supply chain is genuinely capacity-constrained, not pricing-constrained.
6. Where Korea sits
Korea has three structural strengths: cheap stable electricity (OECD-average industrial price ≈105.8/MWh), domestic HBM and DRAM fabs (SK hynix + Samsung), and proximity to Asian demand. Three structural weaknesses: transmission grid concentrated in the south, water scarcity in some prospective sites, and lower IT-demand density than US/EU primary markets.
Sector-wise:
- Hyperscaler colo at home (Naver Gak, Kakao 안산, NHN 광주): national AI clouds.
- Securities/finance (KIS / 한국투자증권 SKB 가산 IDC, 200-rack footprint as reference): the current pattern is rented colocation racks for inference workloads, with on-prem GPU lines for sensitive data. Training is outsourced to public cloud or Korean GPU service vendors.
- Sovereign AI fund 2025–: government-backed AIDC + HBM-foundry investments, with public-private capex pools for under-30B-parameter Korean foundation models.
Where this connects to the book
- Chapter 8 grounds the book’s “토대와 상부구조” framing in the AIDC build: 1 GW economics, training-time math, the IDC-vs-AIDC comparison, big-tech power plans.
- Chapter 2 supplies the upstream — the value chain (IDM/Fabless/Foundry/OSAT), TSV/MR-MUF/hybrid-bonding packaging, and the path from von Neumann to memory-centric architectures.
- Chapter 10 picks up the downstream — the political economy of AIDC siting, sovereign AI, and the sustainability debate.
- AI Chip and Memory: From Transistors to HBM is the companion overview on the silicon side.
See also
Foundations:
- shannon-1948-a-mathematical-theory-of-communication — Information theory underpins every byte the AIDC moves
- wiener-1948-cybernetics-or-control-and — Feedback loops at the rack/cluster scheduler level
The model side that drives demand:
- vaswani-2017-attention-is-all-you-need — Attention’s quadratic memory drives HBM sizing
- kaplan-2020-scaling-laws-for-neural-language — Scaling laws are the demand-pull curve for AIDC capacity
- wei-2022-chain-of-thought-prompting-elicits-reasoning-in — Chain-of-thought turns every query into many — the inference-time multiplier
- hu-2021-lora-low-rank-adaptation-of-large — LoRA / PEFT relieve some AIDC pressure for fine-tuning
- lewis-2020-retrieval-augmented-generation-for-knowledge-intensive-nlp — RAG shifts cost from training to inference + storage
World models and physical AI (the next wave of AIDC demand):
- lecun-2022-a-path-towards-autonomous-machine — JEPA: the architecture humanoid robotics will scale on
- maes-2026-leworldmodel-stable-end-to-end-joint-embedding-predictive — End-to-end world models at production scale