Training vs Inference Emissions in Large Language Models
Training a frontier large language model emits hundreds of tonnes of CO2-equivalent — {{GPT-3}} at roughly 552 tCO2eq, [[BLOOM]] at about 25 tCO2eq operational plus ~25 tCO2eq {{embodied carbon}} — but for widely deployed models, cumulative {{inference}} emissions from millions of daily queries surpass one-time training within weeks to months.
Two distinct phases dominate the lifetime carbon footprint of a large language model: training (a one-time cost per model version) and inference (the ongoing per-query cost of serving the deployed model). Estimates are noisy because they depend on hardware, data-center Power Usage Effectiveness, grid carbon intensity, and accounting boundaries (operational vs embodied carbon). **Reported training footprints.** Patterson et al. (2021), "Carbon Emissions and Large Neural Network Training," estimated GPT-3's final training run at ~1,287 MWh of electricity and ~552 tCO2eq, computed using the published parameter count, V100 GPU energy, and grid mix of the training site. Luccioni et al. (2022) measured BLOOM — a 176B-parameter open model trained on the Jean Zay supercomputer — at ~24.7 tCO2eq from dynamic compute, rising to ~50.5 tCO2eq once idle power and embodied hardware emissions were included; the much lower operational figure reflects France's largely nuclear grid (~57 gCO2eq/kWh). Meta reported LLaMA-2 pretraining at ~3.3 million A100-80GB GPU-hours and ~539 tCO2eq across all experiments, fully offset under Meta's sustainability program. These numbers are not directly comparable: methodology, what counts as "training" (final run vs all ablations), and whether embodied carbon is included all vary. **The inference crossover.** For a deployed model with millions of queries per day, cumulative inference emissions can exceed the one-time training cost within weeks to months. Wu et al. (2022), "Sustainable AI," used internal telemetry from Meta's production recommender and language workloads and reported inference accounting for roughly 65% of total ML lifecycle emissions versus 35% for training. SemiAnalysis estimated ChatGPT consumed on the order of 564 MWh per day in early 2023 — at that rate, inference electricity alone passes GPT-3's entire training energy budget in about two and a half days. Luccioni measured BLOOM serving at ~0.9 kWh per 1,000 inferences, or roughly 19 kgCO2eq/day on a typical US-grid endpoint. **What drives inference emissions.** Three multiplicative factors: per-query energy (model size, generated token count, batch efficiency, hardware generation), query volume, and grid carbon intensity at the serving region. Quantization, batching, smaller distilled models, and siting in low-carbon regions can swing the per-query figure by an order of magnitude. Comparing models is therefore best done at the lifecycle level, not training alone — but published per-query numbers carry large uncertainty bands because providers rarely disclose hardware utilization, PUE, or regional grid mix.