As Large Language Models (LLMs) become increasingly adopted across all disciplines, their environmental impact remains largely unexplored, with limited transparency from major providers. With model sizes reaching hundreds of billions of parameters, training and developing state-of-the-art AI systems generates substantial carbon emissions and strains vital resources like electricity and water, particularly in certain regions of the world.

The scale is staggering, a worst-case estimate suggests that Google’s AI alone consumes ∼29.3 TWh annually, comparable to Ireland’s total energy consumption. ¹

While progress in the field of AI continues apace, addressing sustainability in innovation is crucial not only to limit its ecological footprint and preserve natural resources but also to ensure the development of responsible, ethical and cost effective AI systems that can scale without compromising our societal and environmental future.

This blog post aims to complement the research carried out by the Sustainability Team at Scott Logic as part of the latest update of the Technology Carbon Standard, following a thorough literature review.

Photo by Shantanu Kumar on Pexels

AI impacts on natural resources

To get a more accurate picture of the environmental footprint of AI, we deemed necessary to first examine its embodied carbon. This encompasses both upstream emissions, in other terms carbon emissions generated during the manufactue of hardware, including abiotic resources consumption and the fabrication of server components, as well as downstream emissions which relate to the end-of-life and recycling stages of AI hardware.

AI is increasing demand in AI chips and analysts have estimated that demand for Nvidia’s prized AI chips is exceeding supply by at least 50%. ² In the UK alone, data centres are expected to increase by almost a fifth over the next few years.

The embodied carbon of AI is far from negligeable. A comprehensive AI cradle-to-grave approach ³ estimates that manufacturing emissions represent under 25% of AI carbon emissions and data centre construction emissions under 5%.

We believe that a Life Cycle Assessment approach to the environmental impact of AI is necessary to correctly assess its impact outside operational emissions accounting so as not to lose sight of natural resources depletion, pollution and biodiversity loss associated with the development of AI systems. ⁴

The cumulative cost of inference

State-of-the-art models can generate content across multiple media formats including text, image and video. Each “inference”, the process by which an LLM takes a user’s input and generates a relevant output, carries its own carbon footprint. In their contribution to a global environmental standard for AI released earlier this year, Mistral estimated that a 400-token text response generated 1.14 gCO₂e and 45 mL of water. While this may seem negligible for a single query, the scale becomes staggering when multiplied across billions of daily interactions globally. Indeed, Google reported that 60% of AI-related energy consumption from 2019 to 2021 stemmed from inference. ¹

Inference does not only apply to commercial inference services, but is also increasingly integrated into systems such as search engines. As an example, Alphabet’s chairman indicated in February 2023 that:

Interacting with an LLM could “likely cost 10 times more than a standard keyword search”. ⁵

The hidden impact of pre-training

If inference is the front door through which most of us interact with LLMs, we must also examine what lies behind it: the carbon-intensive phases of data collection, storage, and preprocessing, as well as the pre-training process itself.

To provide a sense of scale, training GPT-3 is estimated to have consumed 1,287 megawatt-hours (MWh) of electricity and emitted over 550 metric tons of CO2e ⁶, and evaporated 700,000 litres of clean freshwater ⁷, enough to fill an Olympic-sized swimming pool by nearly one-third.

AI data centres fundamentally differ from traditional data centres in their infrastructure. The specialised hardware necessary for AI workloads - Graphics Processing Units (GPUs, the chips that process multiple calculations simultaneously) and Tensor Processing Units (TPUs, Google’s custom AI chips) - consumes substantially more power than standard CPUs. Pre-training in general is almost always performed over multiple GPUs which incurs energy costs from communication between GPUs, and often also with gradient accumulation (a technique for processing large amounts of data in smaller chunks) to accommodate large batches.

The less obvious case of fine-tuning

The data on carbon emissions generated by fine-tuning is less well documented than that of pre-training, although fine-tuning accounts for a substantial part of energy consumption. Indeed, while fine-tuning is less computationally expensive than pre-training due to the smaller amount of training data, its carbon footprint may be much bigger due to being intensively performed worldwide. ⁸

As is the case with pre-training, energy consumption depends on the hardware it is run on, the type of task and the type of computation required to carry it out. Additional factors like data center location, energy mix, model complexity, and training duration come into play.

It is important to note that although fine-tuning can be extremely energy-intensive, it can also reduce long-term emissions by making models more efficient during inference.

A path forward

The scale of AI’s environmental impact might seem overwhelming, but our research also revealed reasons for optimism. Across academia and industry, researchers are developing practical strategies to reduce AI’s footprint without sacrificing accuracy.

The future of AI is not yet written

Our research made clear that without addressing the environmental impact of LLMs, there is a risk that the rapid advancements in the field will result in irreversible environmental harm.

The unbridled way AI is currently being developed by big tech companies, which Dr Sasha Luccioni likens to the big oil industry is not sustainable and only benefits a few.

However there are many ways researchers and corporations can collectively work towards a more sustainable AI. In fact, many are already pioneering alternative approaches that prioritise sustainability and responsibility.

Learning from the smaller players

While major tech companies dominate headlines with ever-larger models, smaller AI research groups and novel companies are charting a different course. Organisations like Hugging Face are championing open research into AI’s carbon footprint and demonstrating that effective AI doesn’t always require massive models and infrastructure. Academic institutions, working within resource constraints, have driven innovation in efficient architectures, proving that limitations can foster creativity rather than hinder it. As the poet Charles Baudelaire who said of poetry that because the form is constrained, the idea springs forth more intensely. ⁴ The same principle applies to sustainable AI: sometimes the most elegant solutions emerge not from unlimited resources, but from thoughtful constraints.

Among the papers reviewed, a few observations and actionable recommendations stood out:

Standardised data needs to be available

• The opacity around standardised reporting hinders independent verification and undermines efforts to regulate AI’s true environmental cost. ⁶
• Authors should report training time and sensitivity to hyperparameters ⁹ to enable direct comparison between models, which would enable corporations to make informed and sustainable decisions when training models.
• Academic researchers need equitable access to large-scale compute to foster creativity and prevent the problematic “rich get richer” cycle of research funding. ⁹

Sustainability must be put at the centre of AI innovation

• Cost-effective and sustainable innovation in the context of limited resources should be promoted.
• Frugal AI (a design philosophy emphasising resource-conscious systems) offers a vision of systems that are functional, robust, user-friendly, growing, affordable, and local. ⁴
• Federated Learning (a method where AI models are trained across many devices without centralising data) offers a solution by decentralising the training process and offers several advantages such as reducing the time and bandwidth required for training and inference and lower the energy consumption associated with long-distance data transmission. ¹⁰
• Efficiency should be an evaluation criterion so that ML practitioners compete to increase accuracy. ¹¹ Although this can also lead to a rebound effect whereby the more efficient models become, the more they get used.
• Research should prioritise developing efficient models and hardware. Improvements in state of the art accuracy are possible thanks to industry access to large-scale compute. ⁹

The right AI for the right need at the right time.

Artificial intelligence should only be used in cases where it is the best technique to use. ⁴

• The necessity of using AI should be critically considered in the first place, as it is unlikely that all applications will benefit from AI or that the benefits will always outweigh the costs.
• The Deep Neural Network (DNN) model, processor and data centre should be carefully chosen.
• Existing models should be lightened and faster GPUs used ¹² to both reduce the environmental damage of LLM training while maintaining results. However this comes with financial implications, which necessitates further research to make sustainable AI practices more accessible.
• Short reasoning methods should be used for inference, for accuracy and carbon saving. Long LLM reasoning does not mean accuracy and correct answers are typically shorter than incorrect ones. ¹³

Smaller models for smarter solutions

• Smaller models are sufficiently powerful for many tasks that we entrust AI with, and are considerably less energy-intensive as Small Language Models (SLMs) trained on carefully selected data require less computation power.
• This is particularly relevant in the context of agentic AI where LLMs are excessive and misaligned with the demands of most use cases, like using a sledgehammer to crack a nut. ¹⁴
• The shift to smaller, task-specific models represents perhaps the most immediate opportunity to reduce AI’s environmental impact while maintaining practical utility.

References