Training is massive—but it's a one-time cost
Estimates suggest training GPT-4 cost over $100M and used 50 GWh—enough to power San Francisco for 3 days.
Inference dominates energy use
Across AI, ~80–90% of compute is spent on inference rather than training.
Scale of daily inference
One GPT-4o short query ≈ 0.42 Wh. At 700M queries/day, annual impacts add up.
Mistral's environmental disclosure
Mistral discloses training footprint for Large 2 and marginal impacts per 400-token response in Le Chat.
What drives inference energy?
Energy scales with prompt length, model size/architecture, and deployment setup.
How much electricity from data centers is used for AI?
US data centers' 2024 electricity use is comparable to Thailand. AI-specific electricity may rise to 165–326 TWh by 2028.
Rebound effects (Jevons Paradox)
Greater efficiency can increase total consumption when usage explodes.
What can be done?
Set per-inference thresholds (energy, water, CO₂e). Pursue sparsity/quantization, better hardware, cleaner power, cooling & heat reuse.