Architecture

Structural and design patterns that reduce energy consumption and carbon emissions at the system level.

17 patterns

Adopt serverless architecture for AI/ML workload processes

Building an ML model takes significant computing resources that need to be optimized for efficient utilization.

Choose the region that is closest to users

From an energy-efficiency perspective, it's better to shorten the distance a network packet travels so that less energy is required to transmit it. Similarly, from an embodied-carbon perspective, when a network packet traverses through less computing equipment, we are more efficient with hardware.

Containerize your workloads

Containerizing workloads enables better resource utilisation and bin packing, reducing unnecessary compute allocation and embodied carbon compared to running full virtual machines.

Evaluate other CPU architectures

Applications are built with a software architecture that best fits the business need they are serving. Cloud providers make it easy to evaluate other CPU types

Implement stateless design

Service state refers to the in-memory or on-disk data required by a service to function. State includes the data structures and member variables that the service reads and writes. Depending on how the service is architected, the state might also include files or other resources stored on the disk. Applications that consume large memory or on-disk data require larger VM sizes, especially for cloud computing where you would need larger VM SKUs to support high RAM capacity and multiple data disks.

Optimize the size of AI/ML models

Large-scale AI/ML models require significant storage space and take more resources to run as compared to optimized models.

Queue non-urgent processing requests

All systems have periods of peak and low load. From a hardware-efficiency perspective, we are more efficient with hardware if we minimise the impact of request spikes with an implementation that allows an even utilization of components. From an energy-efficiency perspective, we are more efficient with energy if we ensure that idle resources are kept to a minimum.

Reduce network traversal between VMs

Placing VMs in the same region or availability zone minimises the physical distance data must travel between instances, reducing the energy consumed by network traversal.

Run AI models at the edge

Data computation for ML workloads and ML inference is a significant contributor to the carbon footprint of the ML application. Also, if the ML model is running on the cloud, the data needs to be transferred and processed on the cloud to the required format that can be used by the ML model for inference.

Scale logical components independently

Decomposing applications into independently scalable microservices allows each component to be right-sized for its own demand, reducing overall compute resource consumption and embodied carbon.

Select a more energy efficient AI/ML framework

Training an AI model implies a significant carbon footprint. The underlying framework used for the development, training, and deployment of AI/ML needs to be evaluated and considered to ensure the process is as energy efficient as possible.

Select the right hardware/VM instance types for AI/ML training

Selecting the right hardware/VM instance types for training is one of the choices you should make as part of your energy-efficient AI/ML process.

Use a service mesh only if needed

Service meshes add overhead through additional containers and increased network traffic, so they should only be deployed for applications that genuinely require the capabilities they provide.

Use cloud native processor VMs

Cloud VMs built on energy-efficient processors, such as ARM-based cloud-native chips, can run scale-out workloads with significantly lower energy consumption and embodied carbon than traditional alternatives.

Use efficient file formats for AI/ML development

Efficient storage of the model becomes extremely important to manage the data used for ML model development.

Use energy efficient AI/ML models

Evaluate and use alternative, more energy efficient, models that provide similar functionality.

Use serverless cloud services

Serverless cloud services scale dynamically with demand and share infrastructure across many applications, reducing idle resource consumption and lowering embodied carbon emissions.