Carbon Aware SDK: Shifting Workloads in Time and Space
The ICT sector consumes approximately 460–700 TWh of electricity per year in global data centers, a figure set to double by 2030 driven by the explosion of AI workloads. But not all this energy is equal: one kilowatt-hour produced when the grid is 90% powered by renewables causes one-tenth of the CO₂ emissions compared to one produced during a demand peak covered by gas-fired plants. The same logic applies spatially: running a job in a European cloud region with a clean energy mix can be four times greener than a region predominantly powered by coal.
This concept — running software when and where electricity is cleanest — is called demand shifting or carbon-aware computing, and it is one of the foundational principles of green software engineering. The Green Software Foundation has built a reference open-source toolkit around this principle: the Carbon Aware SDK, available on GitHub under the MIT license and used in production by companies such as UBS, Vestas, and Microsoft itself.
In this article we will explore how the Carbon Aware SDK works from the inside: architecture, data sources, REST API, integration with Python, Kubernetes, and GitHub Actions. We will build concrete examples of time shifting (deferring a job by 6–8 hours when carbon intensity drops) and location shifting (routing workloads to the greenest cloud region in real time).
What You Will Learn
- How the Carbon Aware SDK works: WebAPI, CLI, and client library architecture
- Carbon intensity data sources: WattTime, ElectricityMaps, Ember Climate
- Time shifting: scheduling batch jobs in low carbon intensity windows
- Location shifting: choosing the cloud region with the cleanest energy mix
- Python integration with the Carbon Aware SDK client
- Kubernetes + KEDA: carbon-aware autoscaling without changes to application code
- GitHub Actions: carbon-aware CI/CD pipeline with intelligent scheduling
- Marginal vs average carbon intensity: which signal to use and why
- Limitations, trade-offs, and real-world use cases with measurable emission reductions
Green Software Engineering Series — All Articles
| # | Title | Topic |
|---|---|---|
| 1 | Principles of Green Software Engineering | 8 GSF principles, SCI specification ISO/IEC 21031 |
| 2 | Measuring Emissions with CodeCarbon | CO₂ tracking in Python, MLflow, dashboard |
| 3 | Climatiq API: Carbon Intensity in Cloud Systems | REST API, cloud and supply chain emission calculations |
| 4 | Carbon Aware SDK (this article) | Time shifting, location shifting, Kubernetes, CI/CD |
| 5 | Scope 3 and ESG Pipelines | Upstream/downstream emissions, CSRD data pipeline |
| 6 | Scope 1, 2, 3 Modelling | GHG Protocol accounting frameworks, SBTi |
| 7 | AI Carbon Footprint | LLM training, inference, energy optimization |
| 8 | Sustainable Software Patterns | Green design patterns, efficient architectures |
| 9 | ESG and CSRD for Software | Regulatory compliance, mandatory EU reporting |
| 10 | GreenOps: Sustainable Operations | FinOps+GreenOps, operational metrics, culture shift |
Demand Shifting: The Principle Behind Carbon-Aware Computing
The carbon intensity of a kilowatt-hour of electricity is not fixed: it varies every hour depending on how much renewable energy is available on the grid. On a sunny day in Germany, carbon intensity can drop below 100 gCO₂/kWh during midday hours; at night, with solar off and industrial demand falling, it climbs back above 400 gCO₂/kWh. The same variability exists spatially: Sweden, powered almost entirely by hydroelectric and nuclear, oscillates between 15 and 40 gCO₂/kWh, while Poland, still dependent on coal, often exceeds 700 gCO₂/kWh.
Demand shifting exploits this variability by moving flexible workloads — batch processing, ML training, backups, data analysis, CI/CD tests — to the moments and regions where electricity is cleanest. It is not about giving up computation: it is about choosing the best moment to perform it.
Types of Demand Shifting
| Type | Description | Practical Example | Typical CO₂ Reduction |
|---|---|---|---|
| Time Shifting | Moving the workload to a low carbon intensity time window within the same region | Running ML training at 2:00 AM instead of 2:00 PM | 20–45% |
| Location Shifting | Running the workload in a cloud region with a cleaner energy mix | Routing the job from us-east-1 to eu-north-1 (Stockholm) | 30–70% |
| Demand Shaping | Reducing offered features when carbon intensity is high | Disabling 4K video resolution during emission peaks | 10–25% |
Obviously not all workloads can be shifted: an HTTP user request must be served immediately, a financial transaction cannot wait. But a surprisingly high percentage of enterprise computational workloads is time-flexible: ML model training, nightly ETL pipelines, report generation, backups, security scans, CI/CD pipelines. These are precisely the ideal candidates for carbon-aware scheduling.
Carbon Aware SDK: Architecture and Components
The Carbon Aware SDK is an open-source project from the Green Software Foundation,
released under the MIT license and currently in Graduated Project status (the highest
maturity level in the GSF portfolio). The repository is on GitHub at
Green-Software-Foundation/carbon-aware-sdk and is developed primarily in C# with an
ASP.NET Core WebAPI, but exposes interfaces usable from any language.
The architecture is modular and revolves around a key concept: a single normalized interface for accessing carbon intensity data from different providers, with output always in gCO₂eq/kWh regardless of the source. This is fundamental because each provider uses different units, geographic granularity, and calculation methodologies.
Carbon Aware SDK Components
| Component | Technology | Use | When to Use It |
|---|---|---|---|
| WebAPI | ASP.NET Core (C#), Docker | REST API with Swagger/OpenAPI, deployable as a microservice | Integration with any stack, microservice architectures |
| CLI | dotnet tool, cross-platform | Terminal queries, bash/PowerShell scripting | Deployment scripts, DevOps automation, quick tests |
| SDK Library | NuGet package (C#) | Direct integration in .NET application code | .NET applications wanting embedded carbon-aware logic |
| Python Client | openapi-generator, Python 3.8+ | Auto-generated client from the WebAPI OpenAPI spec | ML pipelines, data engineering scripts, Airflow DAGs |
The typical flow is as follows: run the WebAPI as a Docker container, configured with your carbon intensity provider credentials, and query the REST endpoints from your scheduling application. The WebAPI handles normalizing data, managing provider calls, and returning the optimal time windows or regions.
Data Sources: WattTime, ElectricityMaps, and Ember Climate
The Carbon Aware SDK supports multiple carbon intensity data providers, each with different characteristics and geographic coverage. The choice of provider has significant implications both methodologically (marginal vs average) and practically (coverage, cost, update frequency).
Carbon Intensity Provider Comparison
| Provider | Signal Type | Coverage | Frequency | Forecast | Cost |
|---|---|---|---|---|---|
| WattTime | Marginal (MOER) | Full USA, 50+ countries | 5 minutes | 24–72 hours | Limited free plan, commercial plans |
| ElectricityMaps | Average (LCA) | 85+ countries, full EU | Hourly/sub-hourly | 24 hours | Free developer plan, commercial |
| Ember Climate | Historical average | Global (50+ countries) | Daily/historical | No (historical only) | Open data, free |
| Static JSON | Configurable | Custom | Manual | No | Free (development/testing) |
Marginal vs Average Carbon Intensity: A Crucial Distinction
The choice between marginal and average signal is the most important methodological difference in carbon-aware computing and has concrete implications for achievable results.
The marginal signal (MOER — Marginal Operating Emissions Rate), provided by WattTime, answers the question: "If I increased my consumption by 1 kWh right now, which power plant would come online to cover that additional demand?". The answer is almost always the most flexible gas plant (the so-called "marginal generator"), not the renewables already running at maximum capacity. This signal is most relevant for real-time decisions by those who want to reduce the emissions caused by their incremental consumption.
The average signal (LCA — Life Cycle Average), provided by ElectricityMaps, answers a different question: "What is the average emissions of all electricity produced on the grid right now?". On a grid with 50% solar and 50% gas, the average signal is ~250 gCO₂/kWh, regardless of what happens at the margin. This signal is better suited for ESG reporting and Scope 2 market-based accounting.
Important: ElectricityMaps Approach Change in 2025
In 2025, ElectricityMaps discontinued the marginal signal from its API, citing concerns about data verifiability and alignment with EU and US regulations that prohibit the use of marginal factors in Scope 2 accounting. ElectricityMaps now exclusively offers average signals based on LCA (Life Cycle Assessment) methodology. If your use case requires a marginal signal, WattTime remains the only mainstream provider that supports it. The Carbon Aware SDK handles this difference transparently through provider configuration.
When to Use Which Signal
| Use Case | Recommended Signal | Provider | Why |
|---|---|---|---|
| Time shifting batch jobs | Marginal | WattTime | Maximizes real reduction in caused emissions |
| Multi-region location shifting | Average or marginal | ElectricityMaps or WattTime | Both useful; average more stable cross-region |
| ESG reporting / Scope 2 | Average | ElectricityMaps | Required by GHG Protocol and EU/US regulations |
| Development and testing | Static JSON | Local file | No cost, deterministic data for testing |
Setup and Configuration of the Carbon Aware SDK
The fastest way to start the Carbon Aware SDK locally is with Docker Compose. Here is the complete configuration using ElectricityMaps as the provider (the easiest to configure thanks to the free developer API key).
# docker-compose.yml - Carbon Aware SDK WebAPI
version: '3.8'
services:
carbon-aware-api:
image: ghcr.io/green-software-foundation/carbon-aware-sdk:latest
ports:
- "8080:80"
environment:
# Provider: ElectricityMaps (average LCA signal)
CarbonAwareVars__CarbonIntensityDataSource: "ElectricityMaps"
CarbonAwareVars__ElectricityMapsClient__APITokenHeader: "auth-token"
CarbonAwareVars__ElectricityMapsClient__APIToken: "${ELECTRICITY_MAPS_TOKEN}"
CarbonAwareVars__ElectricityMapsClient__BaseURL: "https://api.electricitymap.org/v3/"
# Or WattTime (marginal MOER signal)
# CarbonAwareVars__CarbonIntensityDataSource: "WattTime"
# CarbonAwareVars__WattTimeClient__Username: "${WATTTIME_USER}"
# CarbonAwareVars__WattTimeClient__Password: "${WATTTIME_PASS}"
# CarbonAwareVars__WattTimeClient__BaseURL: "https://api2.watttime.org/v2/"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:80/health"]
interval: 30s
timeout: 10s
retries: 3For development and testing environments, you can use the static JSON data source, which requires no API key. The SDK includes pre-loaded sample datasets.
# appsettings.json - Configuration with static JSON (dev/test)
{
"CarbonAwareVars": {
"CarbonIntensityDataSource": "Json",
"JsonDataFileLocation": "./data/test-data.json",
"Proxy": {
"UseProxy": false
}
},
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
}
}
Once the container is running, the Swagger documentation is available at
http://localhost:8080/swagger. Let us verify everything is working with a
test call to the CLI:
# Install Carbon Aware CLI (dotnet tool)
dotnet tool install -g GSF.CarbonAware.Cli
# Check current emissions for a location
carbon-aware emissions location --location "westeurope" \
--config ./appsettings.json
# Example output:
# Location: westeurope
# Time: 2025-09-15T14:30:00Z
# Rating: 180.5 gCO2eq/kWh
# Duration: PT1HAPI Endpoints: The Heart of the Carbon Aware SDK
The WebAPI exposes a set of REST endpoints documented with OpenAPI/Swagger. Let us explore the main ones with real call examples and response interpretation.
GET /emissions/bylocations — Current Emissions by Location
Returns carbon intensity data for one or more locations within a time interval. Used to compare current carbon intensity across different regions (location shifting).
# Comparing carbon intensity across cloud regions (location shifting)
GET /emissions/bylocations?locations=westeurope&locations=eastus&locations=northeurope
&time=2025-09-15T14:00:00Z
&toTime=2025-09-15T15:00:00Z
# JSON Response:
[
{
"location": "westeurope",
"timestamp": "2025-09-15T14:00:00Z",
"duration": 60,
"rating": 185.3
},
{
"location": "eastus",
"timestamp": "2025-09-15T14:00:00Z",
"duration": 60,
"rating": 312.7
},
{
"location": "northeurope",
"timestamp": "2025-09-15T14:00:00Z",
"duration": 60,
"rating": 32.1
}
]
# North Europe (Ireland/Stockholm) is almost 10x greener than East US!GET /emissions/bylocation/best — Best Location
Returns directly the location with the lowest carbon intensity in the specified interval. Ideal for automatic workload routing.
GET /emissions/bylocation/best?locations=westeurope&locations=eastus&locations=northeurope
&time=2025-09-15T14:00:00Z
&toTime=2025-09-15T15:00:00Z
# Response:
{
"location": "northeurope",
"timestamp": "2025-09-15T14:00:00Z",
"duration": 60,
"rating": 32.1
}GET /emissions/forecasts/current — Carbon Intensity Forecast
This is the most powerful endpoint for time shifting: it returns the forecast for the next 24–72 hours with the optimal time window in which to run the workload. It allows you to specify the job duration to find the best window of that length.
GET /emissions/forecasts/current?locations=westeurope
&dataStartAt=2025-09-15T16:00:00Z
&dataEndAt=2025-09-16T16:00:00Z
&windowSize=60
# Response with optimal window:
[
{
"generatedAt": "2025-09-15T14:30:00Z",
"location": "westeurope",
"dataStartAt": "2025-09-15T16:00:00Z",
"dataEndAt": "2025-09-16T16:00:00Z",
"windowSize": 60,
"optimalDataPoints": [
{
"location": "westeurope",
"timestamp": "2025-09-16T02:00:00Z",
"duration": 60,
"rating": 95.2
}
],
"forecastData": [
{ "timestamp": "2025-09-15T16:00:00Z", "rating": 195.8 },
{ "timestamp": "2025-09-15T17:00:00Z", "rating": 210.3 },
{ "timestamp": "2025-09-15T18:00:00Z", "rating": 230.1 },
{ "timestamp": "2025-09-15T22:00:00Z", "rating": 155.4 },
{ "timestamp": "2025-09-16T01:00:00Z", "rating": 105.7 },
{ "timestamp": "2025-09-16T02:00:00Z", "rating": 95.2 }, // OPTIMAL
{ "timestamp": "2025-09-16T03:00:00Z", "rating": 98.6 },
{ "timestamp": "2025-09-16T06:00:00Z", "rating": 120.3 }
]
}
]
# The 1-hour job should start at 02:00 UTC (saving ~51% CO2 vs current time)POST /emissions/forecasts/batch — Batch Forecasts
Allows sending a batch of forecast requests to plan multiple jobs simultaneously, useful for scheduling complex pipelines with dependencies.
POST /emissions/forecasts/batch
Content-Type: application/json
[
{
"requestedAt": "2025-09-15T14:00:00Z",
"location": "westeurope",
"dataStartAt": "2025-09-15T20:00:00Z",
"dataEndAt": "2025-09-16T08:00:00Z",
"windowSize": 120
},
{
"requestedAt": "2025-09-15T14:00:00Z",
"location": "westeurope",
"dataStartAt": "2025-09-15T20:00:00Z",
"dataEndAt": "2025-09-16T08:00:00Z",
"windowSize": 30
}
]Python Integration: CarbonAwareClient and Time Shifting
For Python pipelines — typically data engineering, ML training, or batch analytics — the Carbon Aware SDK offers a Python client automatically generated from the OpenAPI specification. Let us build a complete time shifting system for an ML training job.
Installing and Configuring the Python Client
# requirements.txt
carbon-aware-sdk-client>=1.0.0 # Auto-generated client from OpenAPI
requests>=2.31.0
python-dateutil>=2.8.2
apscheduler>=3.10.4 # Job scheduling
pytz>=2023.3
# Install from PyPI (if available) or from source:
# pip install openapi-python-client
# openapi-python-client generate \
# --url http://localhost:8080/swagger/v1/swagger.jsonCarbon Aware Client in Python
"""
carbon_aware_client.py
Python client for the Carbon Aware SDK WebAPI.
Implements time shifting and location shifting.
"""
import requests
from datetime import datetime, timedelta, timezone
from typing import Optional
import logging
logger = logging.getLogger(__name__)
class CarbonAwareClient:
"""
Client for the Carbon Aware SDK WebAPI.
Encapsulates REST calls and provides high-level methods
for time shifting and location shifting.
"""
def __init__(self, base_url: str = "http://localhost:8080"):
self.base_url = base_url.rstrip("/")
self.session = requests.Session()
self.session.headers.update({
"Content-Type": "application/json",
"Accept": "application/json"
})
def get_current_emissions(
self,
locations: list[str],
time: Optional[datetime] = None,
to_time: Optional[datetime] = None
) -> list[dict]:
"""
Retrieves the current carbon intensity for one or more locations.
Args:
locations: List of location names (e.g. ["westeurope", "eastus"])
time: Start of interval (default: current time)
to_time: End of interval (default: time + 1 hour)
Returns:
List of dicts with location, timestamp, rating (gCO2eq/kWh)
"""
now = datetime.now(timezone.utc)
time = time or now
to_time = to_time or (time + timedelta(hours=1))
params = {
"locations": locations,
"time": time.isoformat(),
"toTime": to_time.isoformat()
}
response = self.session.get(
f"{self.base_url}/emissions/bylocations",
params=params
)
response.raise_for_status()
return response.json()
def get_best_location(
self,
locations: list[str],
time: Optional[datetime] = None,
to_time: Optional[datetime] = None
) -> dict:
"""
Finds the location with the lowest carbon intensity.
Returns:
Dict with location, timestamp, rating of the best location
"""
now = datetime.now(timezone.utc)
time = time or now
to_time = to_time or (time + timedelta(hours=1))
params = {
"locations": locations,
"time": time.isoformat(),
"toTime": to_time.isoformat()
}
response = self.session.get(
f"{self.base_url}/emissions/bylocation/best",
params=params
)
response.raise_for_status()
return response.json()
def get_optimal_window(
self,
location: str,
window_size_minutes: int,
search_start: Optional[datetime] = None,
search_end: Optional[datetime] = None
) -> dict:
"""
Finds the optimal time window (lowest carbon intensity)
to run a job of window_size_minutes duration in the given location.
Args:
location: Location name (e.g. "westeurope")
window_size_minutes: Job duration in minutes
search_start: Start of search window (default: current time)
search_end: End of search window (default: search_start + 24 hours)
Returns:
Dict with optimalDataPoints (optimal timestamp) and forecastData
"""
now = datetime.now(timezone.utc)
search_start = search_start or now
search_end = search_end or (search_start + timedelta(hours=24))
params = {
"locations": [location],
"dataStartAt": search_start.isoformat(),
"dataEndAt": search_end.isoformat(),
"windowSize": window_size_minutes
}
response = self.session.get(
f"{self.base_url}/emissions/forecasts/current",
params=params
)
response.raise_for_status()
forecasts = response.json()
if not forecasts:
raise ValueError(f"No forecast available for {location}")
return forecasts[0]
def calculate_carbon_savings(
self,
current_rating: float,
optimal_rating: float,
duration_hours: float,
power_kw: float
) -> dict:
"""
Calculates the CO2 savings from time shifting.
Args:
current_rating: Current carbon intensity (gCO2/kWh)
optimal_rating: Optimal carbon intensity (gCO2/kWh)
duration_hours: Job duration in hours
power_kw: Average job power in kW
Returns:
Dict with current emissions, optimal emissions, and savings
"""
energy_kwh = duration_hours * power_kw
current_emissions_g = current_rating * energy_kwh
optimal_emissions_g = optimal_rating * energy_kwh
savings_g = current_emissions_g - optimal_emissions_g
savings_pct = (savings_g / current_emissions_g) * 100 if current_emissions_g > 0 else 0
return {
"energy_kwh": energy_kwh,
"current_emissions_gco2": round(current_emissions_g, 2),
"optimal_emissions_gco2": round(optimal_emissions_g, 2),
"savings_gco2": round(savings_g, 2),
"savings_percentage": round(savings_pct, 1)
}Time Shifting: Scheduler for ML Training
"""
ml_training_scheduler.py
Carbon-aware scheduler for ML training jobs.
Calculates the optimal window in the next 12 hours
and schedules the training run.
"""
from datetime import datetime, timedelta, timezone
from apscheduler.schedulers.blocking import BlockingScheduler
from carbon_aware_client import CarbonAwareClient
import subprocess
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class MLTrainingScheduler:
"""
Carbon-aware scheduler for ML training.
Finds the lowest carbon intensity window
in the next 12 hours and schedules training.
"""
def __init__(
self,
location: str = "westeurope",
training_duration_minutes: int = 120,
max_wait_hours: int = 12,
carbon_aware_url: str = "http://localhost:8080"
):
self.location = location
self.training_duration_minutes = training_duration_minutes
self.max_wait_hours = max_wait_hours
self.client = CarbonAwareClient(base_url=carbon_aware_url)
self.scheduler = BlockingScheduler(timezone="UTC")
def find_optimal_start(self) -> datetime:
"""
Queries the Carbon Aware SDK to find
the optimal moment within the next max_wait_hours.
"""
now = datetime.now(timezone.utc)
search_end = now + timedelta(hours=self.max_wait_hours)
logger.info(
f"Searching optimal window in {self.location} "
f"for {self.training_duration_minutes}-minute job..."
)
forecast = self.client.get_optimal_window(
location=self.location,
window_size_minutes=self.training_duration_minutes,
search_start=now,
search_end=search_end
)
optimal_points = forecast.get("optimalDataPoints", [])
if not optimal_points:
logger.warning("No optimal window found, using current time")
return now
optimal_timestamp_str = optimal_points[0]["timestamp"]
optimal_timestamp = datetime.fromisoformat(
optimal_timestamp_str.replace("Z", "+00:00")
)
optimal_rating = optimal_points[0]["rating"]
# Get current rating for savings calculation
current_data = self.client.get_current_emissions([self.location])
current_rating = current_data[0]["rating"] if current_data else 300.0
# Calculate savings
savings = self.client.calculate_carbon_savings(
current_rating=current_rating,
optimal_rating=optimal_rating,
duration_hours=self.training_duration_minutes / 60,
power_kw=150 # GPU server: ~150kW estimate
)
wait_minutes = (optimal_timestamp - now).total_seconds() / 60
logger.info(
f"Optimal window found:\n"
f" Start: {optimal_timestamp.strftime('%Y-%m-%d %H:%M UTC')}\n"
f" Carbon intensity: {optimal_rating:.1f} gCO2/kWh\n"
f" Current carbon intensity: {current_rating:.1f} gCO2/kWh\n"
f" Wait: {wait_minutes:.0f} minutes\n"
f" CO2 savings: {savings['savings_gco2']:.0f}g ({savings['savings_percentage']}%)"
)
return optimal_timestamp
def run_training(self):
"""Executes the ML training job."""
logger.info("Starting carbon-optimized ML training...")
result = subprocess.run(
["python", "train_model.py", "--config", "config.yaml"],
capture_output=True,
text=True
)
if result.returncode == 0:
logger.info("Training completed successfully")
else:
logger.error(f"Training failed: {result.stderr}")
def schedule_and_run(self):
"""Finds the optimal window and schedules training."""
optimal_start = self.find_optimal_start()
now = datetime.now(timezone.utc)
if optimal_start <= now + timedelta(minutes=5):
# Immediate start if the optimal moment is imminent
logger.info("Immediate start (optimal window = now)")
self.run_training()
else:
# Schedule the start
self.scheduler.add_job(
self.run_training,
"date",
run_date=optimal_start,
id="ml_training"
)
logger.info(f"Training scheduled for {optimal_start}")
self.scheduler.start()
# Usage:
if __name__ == "__main__":
scheduler = MLTrainingScheduler(
location="westeurope",
training_duration_minutes=120, # 2-hour job
max_wait_hours=12 # Wait maximum 12 hours
)
scheduler.schedule_and_run()Location Shifting: Multi-Region Carbon-Aware Routing
Location shifting is often more effective than time shifting, because the difference in carbon
intensity between cloud regions can be much greater than the temporal variation within a single
region. A job run in
europe-north1 (Finland, ~9 gCO₂/kWh) instead of
asia-east1 (Taiwan, ~550 gCO₂/kWh) reduces emissions by over 98%.
Average Carbon Intensity by Cloud Region (2025)
| Cloud Provider | Region | Location | Average Carbon Intensity (gCO₂/kWh) | Primary Source |
|---|---|---|---|---|
| Google Cloud | europe-north1 | Finland | 9 | Hydroelectric, nuclear |
| Azure | swedencentral | Sweden | 18 | Hydroelectric, nuclear |
| Azure | northeurope | Ireland | 280 | Wind, gas |
| AWS | eu-west-1 | Ireland | 320 | Wind, gas |
| AWS | us-east-1 | Virginia | 380 | Gas, nuclear, coal |
| Google Cloud | asia-east1 | Taiwan | 545 | Coal, gas |
| AWS | ap-southeast-1 | Singapore | 408 | Natural gas |
Python: Automatic Location Shifting for Batch Jobs
"""
location_shifter.py
Carbon-aware location shifting for multi-cloud batch jobs.
Automatically selects the greenest region
to run a Kubernetes job.
"""
from carbon_aware_client import CarbonAwareClient
from datetime import datetime, timezone
import subprocess
import json
import logging
logger = logging.getLogger(__name__)
# Mapping Carbon Aware SDK location -> real cloud region
LOCATION_TO_CLOUD_REGION = {
"northeurope": {
"aws": "eu-west-1",
"azure": "northeurope",
"gcp": "europe-west1"
},
"swedencentral": {
"azure": "swedencentral",
"gcp": "europe-north1"
},
"westeurope": {
"aws": "eu-central-1",
"azure": "westeurope",
"gcp": "europe-west4"
},
"eastus": {
"aws": "us-east-1",
"azure": "eastus",
"gcp": "us-east1"
}
}
CANDIDATE_LOCATIONS = ["swedencentral", "northeurope", "westeurope", "eastus"]
TARGET_CLOUD = "azure" # Target cloud provider
class LocationShifter:
def __init__(self, carbon_aware_url: str = "http://localhost:8080"):
self.client = CarbonAwareClient(base_url=carbon_aware_url)
def select_greenest_region(self) -> tuple[str, str, float]:
"""
Selects the cloud region with the lowest carbon intensity
among the candidates.
Returns:
Tuple (location_name, cloud_region, carbon_intensity_rating)
"""
best = self.client.get_best_location(
locations=CANDIDATE_LOCATIONS
)
best_location = best["location"]
best_rating = best["rating"]
# Map to real cloud region
cloud_regions = LOCATION_TO_CLOUD_REGION.get(best_location, {})
cloud_region = cloud_regions.get(TARGET_CLOUD, "westeurope")
logger.info(
f"Selected region: {best_location} -> {TARGET_CLOUD}:{cloud_region}\n"
f"Carbon intensity: {best_rating:.1f} gCO2/kWh"
)
# Log emissions for all candidates for comparison
all_emissions = self.client.get_current_emissions(CANDIDATE_LOCATIONS)
logger.info("Region comparison:")
for em in sorted(all_emissions, key=lambda x: x["rating"]):
marker = " <- SELECTED" if em["location"] == best_location else ""
logger.info(f" {em['location']:20s} {em['rating']:6.1f} gCO2/kWh{marker}")
return best_location, cloud_region, best_rating
def deploy_job_to_region(self, cloud_region: str, job_config: dict):
"""
Deploys a Kubernetes batch job to the selected region.
Uses kubectl with the target region context.
"""
# Replace region in Kubernetes manifest
manifest = job_config.copy()
manifest["metadata"]["annotations"]["target-region"] = cloud_region
manifest_json = json.dumps(manifest)
logger.info(f"Deploying job to region {cloud_region}...")
result = subprocess.run(
["kubectl", "apply", "-f", "-", "--context", f"aks-{cloud_region}"],
input=manifest_json,
capture_output=True,
text=True
)
if result.returncode == 0:
logger.info(f"Job successfully deployed to {cloud_region}")
else:
raise RuntimeError(f"Deploy failed: {result.stderr}")
def run_carbon_aware_job(self, job_config: dict):
"""Full workflow: select region and deploy."""
location, cloud_region, rating = self.select_greenest_region()
logger.info(f"Starting carbon-aware job in {cloud_region} ({rating:.1f} gCO2/kWh)")
self.deploy_job_to_region(cloud_region, job_config)
# Usage
if __name__ == "__main__":
shifter = LocationShifter()
# Kubernetes job configuration
batch_job = {
"apiVersion": "batch/v1",
"kind": "Job",
"metadata": {
"name": "data-processing-carbon-aware",
"annotations": {
"green-software.io/carbon-aware": "true",
"target-region": "" # Will be populated by the location shifter
}
},
"spec": {
"template": {
"spec": {
"containers": [{
"name": "processor",
"image": "myapp/data-processor:latest",
"resources": {
"requests": {"cpu": "2", "memory": "4Gi"},
"limits": {"cpu": "4", "memory": "8Gi"}
}
}],
"restartPolicy": "Never"
}
}
}
}
shifter.run_carbon_aware_job(batch_job)Kubernetes and KEDA: Carbon-Aware Autoscaling
For production Kubernetes environments, Microsoft has released the Carbon Aware KEDA Operator (open source on Azure GitHub), which integrates the Carbon Aware SDK directly into the KEDA scaling layer. The principle is simple: when carbon intensity is low (clean energy available), KEDA can scale up to the maximum number of replicas; when it is high, the maximum replica count is automatically reduced.
The architecture consists of three elements: the Kubernetes Carbon Intensity Exporter (retrieves data from the Carbon Aware SDK and exposes it as a ConfigMap in the cluster), the Carbon Aware KEDA Operator (reads the ConfigMap and updates KEDA's scaling limits), and the custom resource CarbonAwareKedaScaler that defines the scaling thresholds.
Installing the Carbon Intensity Exporter
# Install the Carbon Intensity Exporter with Helm
helm repo add azure-carbon https://azure.github.io/carbon-aware-keda-operator
helm repo update
# Create secret with WattTime credentials
kubectl create secret generic watttime-credentials \
--from-literal=username=MY_WATTTIME_USER \
--from-literal=password=MY_WATTTIME_PASS \
--namespace kube-system
# Install the exporter
helm install carbon-intensity-exporter azure-carbon/carbon-intensity-exporter \
--namespace kube-system \
--set carbonDataProvider=WattTime \
--set watttime.username=MY_WATTTIME_USER \
--set watttime.password=MY_WATTTIME_PASS \
--set location=westus \
--set forecastIntervalHours=12ConfigMap Generated by the Exporter
apiVersion: v1
kind: ConfigMap
metadata:
name: carbon-intensity
namespace: kube-system
data:
# Updated every 12 hours by the exporter
lastUpdated: "2025-09-15T14:00:00Z"
forecastDateTime: "2025-09-15T14:00:00Z"
message: "Carbon intensity data for westus"
# JSON array with 24-hour forecast
# Format: ISO timestamp + gCO2/kWh
forecastData: |
[
{ "timestamp": "2025-09-15T14:00:00Z", "intensity": 312.5 },
{ "timestamp": "2025-09-15T15:00:00Z", "intensity": 298.3 },
{ "timestamp": "2025-09-15T16:00:00Z", "intensity": 285.1 },
{ "timestamp": "2025-09-15T22:00:00Z", "intensity": 195.7 },
{ "timestamp": "2025-09-16T02:00:00Z", "intensity": 145.2 },
{ "timestamp": "2025-09-16T03:00:00Z", "intensity": 138.9 }
]
# Current intensity
currentIntensity: "312.5"
currentIntensityUnit: "gCO2/kWh"CarbonAwareKedaScaler: Custom Resource Definition
apiVersion: carbonaware.azure.com/v1alpha1
kind: CarbonAwareKedaScaler
metadata:
name: batch-processor-carbon-scaler
namespace: default
spec:
# Reference to the KEDA ScaledJob to control
kedaTargetRef:
apiVersion: keda.sh/v1alpha1
kind: ScaledJob
name: batch-processor-scaledjob
# Carbon intensity data source (exporter's ConfigMap)
carbonIntensityForecastDataSource:
localConfigMap:
name: carbon-intensity
namespace: kube-system
key: forecastData
mockCarbonForecast: false
# Thresholds: define maxReplicas based on carbon intensity
# Ordered by ascending intensity
# - If intensity <= 150: max 20 replicas (very clean energy)
# - If intensity <= 300: max 10 replicas (moderately clean energy)
# - If intensity <= 500: max 4 replicas (less clean energy)
# - If intensity > 500: max 1 replica (dirty energy)
maxReplicasByCarbonIntensity:
- carbonIntensityThreshold: 150
maxReplicas: 20
- carbonIntensityThreshold: 300
maxReplicas: 10
- carbonIntensityThreshold: 500
maxReplicas: 4KEDA ScaledJob for Batch Processing
apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
name: batch-processor-scaledjob
namespace: default
labels:
app: batch-processor
green-software.io/carbon-aware: "true"
spec:
# Trigger source: RabbitMQ, Kafka, Azure Queue, etc.
jobTargetRef:
parallelism: 1
completions: 1
activeDeadlineSeconds: 3600
backoffLimit: 2
template:
metadata:
labels:
app: batch-processor
spec:
containers:
- name: processor
image: myapp/batch-processor:v2.1.0
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "2"
memory: "2Gi"
env:
- name: BATCH_SIZE
value: "1000"
- name: OUTPUT_BUCKET
value: "gs://my-processed-data"
restartPolicy: Never
# Trigger: Azure Service Bus queue
triggers:
- type: azure-servicebus
metadata:
queueName: batch-jobs
namespace: my-servicebus-namespace
messageCount: "50"
# Scaling: min 0, max is controlled by CarbonAwareKedaScaler
pollingInterval: 60 # Check the queue every 60 seconds
successfulJobsHistoryLimit: 5
failedJobsHistoryLimit: 3
minReplicaCount: 0
maxReplicaCount: 20 # Overridden by CarbonAwareKedaScalerHow the Carbon Aware KEDA Operator Works in Practice
Every hour, the Carbon Aware KEDA Operator reads the ConfigMap updated by the exporter and updates
the maxReplicaCount field of the KEDA ScaledJob based on the threshold corresponding
to the current carbon intensity. If there are 200 jobs in the queue and carbon intensity is
312 gCO₂/kWh (300–500 threshold), KEDA can scale up to a maximum of 10 workers
instead of 20. As soon as intensity drops below 150 (e.g. at 2:00 AM with strong winds),
the limit automatically rises to 20 and jobs are processed more quickly.
No changes to application code are required.
Carbon-Aware Kubernetes CronJob with Init Container
For periodically scheduled jobs, a different pattern can be used: an init container that queries the Carbon Aware SDK before the job starts and decides whether to run or defer. This approach does not require KEDA and works with standard Kubernetes CronJobs.
apiVersion: batch/v1
kind: CronJob
metadata:
name: nightly-etl-pipeline
namespace: default
annotations:
green-software.io/carbon-aware: "true"
green-software.io/max-intensity: "200"
spec:
# Scheduled at 00:00 UTC every night
# The init container will decide whether to run or skip
schedule: "0 0 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 7
failedJobsHistoryLimit: 3
jobTemplate:
spec:
template:
spec:
initContainers:
# Init container that checks carbon intensity
- name: carbon-aware-check
image: curlimages/curl:8.5.0
env:
- name: CARBON_AWARE_API
value: "http://carbon-aware-api.monitoring.svc.cluster.local:8080"
- name: LOCATION
value: "westeurope"
- name: MAX_INTENSITY
value: "200"
command:
- sh
- -c
- |
set -e
echo "Checking carbon intensity for ${LOCATION}..."
# Query the Carbon Aware SDK
RESPONSE=$(curl -sf "${CARBON_AWARE_API}/emissions/bylocations?locations=${LOCATION}")
INTENSITY=$(echo "$RESPONSE" | grep -o '"rating":[0-9.]*' | head -1 | cut -d: -f2)
echo "Current carbon intensity: ${INTENSITY} gCO2/kWh (maximum: ${MAX_INTENSITY})"
if [ $(echo "${INTENSITY} > ${MAX_INTENSITY}" | bc -l) -eq 1 ]; then
echo "WARNING: Carbon intensity too high (${INTENSITY} > ${MAX_INTENSITY})"
echo "Job will be deferred to the next scheduling cycle"
exit 1 # Init container fails, job does not start
fi
echo "OK: Carbon intensity acceptable. Starting ETL job..."
exit 0
containers:
- name: etl-pipeline
image: myapp/etl-pipeline:v1.5.0
command: ["python", "run_etl.py"]
env:
- name: PIPELINE_DATE
value: "$(date +%Y-%m-%d)"
resources:
requests:
cpu: "1"
memory: "2Gi"
limits:
cpu: "4"
memory: "8Gi"
restartPolicy: NeverGitHub Actions: Carbon-Aware CI/CD Pipeline
CI/CD tests and build pipelines are among the easiest workloads to make carbon-aware, because many of them are genuinely time-flexible: an integration test suite can wait 2–4 hours without impacting the developer workflow, especially for nightly or scheduled pipelines.
A 2024 study estimated that the carbon footprint of GitHub Actions across the entire ecosystem is in the order of 450–1000 MTCO₂eq/year. With carbon-aware scheduling of even just the heaviest workflows (compilation builds, full test suites, model training), the reduction could be significant.
GitHub Actions Workflow with Carbon-Aware Scheduling
# .github/workflows/carbon-aware-build.yml
# Carbon-aware CI/CD pipeline with intelligent scheduling
name: Carbon-Aware Build and Test
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
# Scheduled: every 6 hours to find the best window
schedule:
- cron: '0 */6 * * *'
workflow_dispatch:
inputs:
force_run:
description: 'Force execution regardless of carbon intensity'
required: false
default: 'false'
env:
CARBON_AWARE_API: