Power Hungry Processing (Luccioni et al. 2024)
ACM FAccT 2024 paper by Luccioni, Jernite, and Strubell that benchmarked inference-time energy use across 88 models and 10 task types. Found that energy per request varies by more than four orders of magnitude, with general-purpose generative models typically far more costly than task-specific fine-tuned models.
"Power Hungry Processing: Watts Driving the Cost of AI Deployment?" is a 2024 study by Alexandra Sasha Luccioni, Yacine Jernite, and Emma Strubell, presented at the ACM FAccT conference. It is one of the first systematic empirical comparisons of inference-time energy use across a wide range of machine learning tasks. The team measured 88 models across 10 task types — including classification, summarisation, translation, image classification, and image generation — running on standardised hardware via the Hugging Face platform. They found that energy per inference can vary by more than four orders of magnitude across tasks and architectures. Simple text classification used as little as roughly 0.002 Wh per request, while text generation from large language models averaged around 0.05 Wh and image generation with diffusion models averaged about 2.9 Wh, with the least efficient models exceeding 11 Wh per image. A central headline finding is that general-purpose generative architectures are typically orders of magnitude more energy-expensive than task-specific fine-tuned models for the same downstream task. The authors argue this undermines the assumption that one large model should serve all use cases and call for energy and emissions costs to be weighed explicitly against utility when deploying generative AI.