Why Terraform Requires a Dedicated Pipeline

Run terraform apply manually from a developer's laptop is the most dangerous pattern in infrastructure. Without an automated workflow, problems typical are: state file not updated, local credentials different from the CI environment, no review of the plan before applying, no trace of who applied what and when.

A professional Terraform pipeline solves all these problems: the plan comes automatically generated, reviewed in the Pull Request, and the apply occurs only afterwards approval, with permanent logs in the versioning system. In this article We build this pipeline with two approaches: GitHub Actions for teams small and Atlantis for medium and large teams.

What You Will Learn

  • GitHub Actions pipeline: fmt, validate, tflint, plan on PR, and apply on merge
  • Secure cloud credential management with OIDC (no long-lived secrets)
  • Atlantis: installation, atlantis.yaml and collaborative PR workflow
  • Terraform Cloud / HCP Terraform as a managed alternative
  • Drift detection: cron job to detect discrepancies between state and real infrastructure
  • Matrix strategy for multi-environment pipelines (dev/staging/prod)
  • Slack/Teams notifications on plans and drift alerts

Structure of the Terraform Repository

Before building the pipeline, you need a clear repository structure. The pattern the most common is the separation by environment into distinct directories, with modules shared in a separate directory. 

terraform-infra/
  modules/
    vpc/
      main.tf
      variables.tf
      outputs.tf
    eks-cluster/
    rds-postgres/
  environments/
    dev/
      main.tf         # Usa i moduli con variabili dev
      variables.tf
      terraform.tfvars
      backend.tf      # Backend S3 per dev
    staging/
      main.tf
      terraform.tfvars
      backend.tf
    production/
      main.tf
      terraform.tfvars
      backend.tf
  .terraform-version  # tfenv: specifica la versione di Terraform
  .tflint.hcl         # Configurazione tflint
  .github/
    workflows/
      terraform.yml

# .terraform-version
# Usato da tfenv per installare la versione corretta
1.7.5

GitHub Actions: Complete Pipeline

The GitHub Actions pipeline we build has two workflows: one triggered by Pull Requests (lint + plan) and one triggered by merge on main (apply). 

# .github/workflows/terraform.yml
name: Terraform CI/CD

on:
  push:
    branches:
      - main
    paths:
      - 'environments/**'
      - 'modules/**'
  pull_request:
    branches:
      - main
    paths:
      - 'environments/**'
      - 'modules/**'

permissions:
  contents: read
  pull-requests: write  # Per commentare il plan sul PR
  id-token: write       # Per OIDC authentication con AWS

env:
  TF_VERSION: '1.7.5'
  TFLINT_VERSION: 'v0.50.0'

jobs:
  # Job 1: Lint e validazione statica (tutti gli ambienti in parallelo)
  lint-validate:
    name: Lint & Validate (${{ matrix.environment }})
    runs-on: ubuntu-latest
    strategy:
      matrix:
        environment: [dev, staging, production]
      fail-fast: false

    defaults:
      run:
        working-directory: environments/${{ matrix.environment }}

    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Setup TFLint
        uses: terraform-linters/setup-tflint@v4
        with:
          tflint_version: ${{ env.TFLINT_VERSION }}

      - name: Terraform Format Check
        run: terraform fmt -check -recursive
        working-directory: .

      - name: Terraform Init (solo backend locale per lint)
        run: terraform init -backend=false

      - name: Terraform Validate
        run: terraform validate

      - name: TFLint
        run: |
          tflint --init
          tflint --format compact

  # Job 2: Plan su Pull Request
  plan:
    name: Plan (${{ matrix.environment }})
    runs-on: ubuntu-latest
    needs: lint-validate
    if: github.event_name == 'pull_request'
    strategy:
      matrix:
        environment: [dev, staging]  # Non pianifichiamo prod su ogni PR
      fail-fast: false

    defaults:
      run:
        working-directory: environments/${{ matrix.environment }}

    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: ${{ env.TF_VERSION }}

      # OIDC: nessun secret long-lived necessario
      - name: Configure AWS Credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/GitHubActionsterraform-${{ matrix.environment }}
          aws-region: eu-west-1

      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan
        id: plan
        run: |
          terraform plan -no-color -out=tfplan 2>&1 | tee plan_output.txt
          echo "PLAN_EXIT_CODE=${PIPESTATUS[0]}" >> "$GITHUB_ENV"

      # Commenta il plan sul Pull Request
      - name: Comment Plan on PR
        uses: actions/github-script@v7
        if: always()
        with:
          script: |
            const fs = require('fs');
            const planOutput = fs.readFileSync('environments/${{ matrix.environment }}/plan_output.txt', 'utf8');
            const truncated = planOutput.length > 65000
              ? planOutput.substring(0, 65000) + '\n... (truncated)'
              : planOutput;

            const body = `## Terraform Plan - ${{ matrix.environment }}

            \`\`\`
            ${truncated}
            \`\`\`

            *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;

            // Cerca commenti precedenti dello stesso workflow e aggiorna
            const comments = await github.rest.issues.listComments({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
            });

            const existingComment = comments.data.find(c =>
              c.user.login === 'github-actions[bot]' &&
              c.body.includes('Terraform Plan - ${{ matrix.environment }}')
            );

            if (existingComment) {
              await github.rest.issues.updateComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                comment_id: existingComment.id,
                body,
              });
            } else {
              await github.rest.issues.createComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: context.issue.number,
                body,
              });
            }

  # Job 3: Apply su merge in main
  apply:
    name: Apply (${{ matrix.environment }})
    runs-on: ubuntu-latest
    needs: lint-validate
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    strategy:
      # Apply sequenziale: prima dev, poi staging, poi production
      max-parallel: 1
      matrix:
        environment: [dev, staging, production]

    environment:
      name: ${{ matrix.environment }}  # Richiede approvazione manuale per production

    defaults:
      run:
        working-directory: environments/${{ matrix.environment }}

    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Configure AWS Credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/GitHubActionsterraform-${{ matrix.environment }}
          aws-region: eu-west-1

      - name: Terraform Init
        run: terraform init

      - name: Terraform Apply
        run: terraform apply -auto-approve -no-color

OIDC: Credentials Without Secrets Long-Lived

The modern best practice for authenticating GitHub Actions to AWS is OIDC (OpenID Connect): GitHub generates a signed JWT token that AWS accept without you having to memorize AWS_ACCESS_KEY_ID in secrets. 

# Configurazione IAM Role per GitHub Actions OIDC
# Crea questo con Terraform stesso (bootstrap)

# iam.tf
resource "aws_iam_openid_connect_provider" "github" {
  url = "https://token.actions.githubusercontent.com"

  client_id_list = ["sts.amazonaws.com"]

  thumbprint_list = [
    "6938fd4d98bab03faadb97b34396831e3780aea1"
  ]
}

resource "aws_iam_role" "github_actions_terraform" {
  for_each = toset(["dev", "staging", "production"])

  name = "GitHubActionsterraform-${each.key}"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = {
        Federated = aws_iam_openid_connect_provider.github.arn
      }
      Action = "sts:AssumeRoleWithWebIdentity"
      Condition = {
        StringEquals = {
          "token.actions.githubusercontent.com:aud" = "sts.amazonaws.com"
        }
        StringLike = {
          # Permetti solo da questo repo specifico
          "token.actions.githubusercontent.com:sub" = "repo:myorg/terraform-infra:*"
        }
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "terraform_permissions" {
  for_each = aws_iam_role.github_actions_terraform

  role       = each.value.name
  policy_arn = "arn:aws:iam::aws:policy/PowerUserAccess"
  # In produzione usa policy più ristrette con least-privilege
}

Atlantis: Plan/Apply Collaborative via PR

Atlantis is a server that listens to GitHub/GitLab/Bitbucket webhooks and responds to comments in PRs. When you write atlantis plan in a comment, Atlantis executes terraform plan and responds with the output. This creates a workflow fully tracked and reviewable in the PR. 

# atlantis.yaml - Configurazione nella root del repository
version: 3

# Abilita auto-planning: piano automatico quando il PR tocca file .tf
automerge: false
parallel_plan: true
parallel_apply: false

projects:
  - name: dev
    dir: environments/dev
    workspace: default
    autoplan:
      when_modified: ["*.tf", "*.tfvars", "../../modules/**/*.tf"]
      enabled: true
    apply_requirements:
      - approved       # Richiede almeno 1 approvazione
      - mergeable      # Il PR deve essere mergeable

  - name: staging
    dir: environments/staging
    workspace: default
    autoplan:
      when_modified: ["*.tf", "*.tfvars", "../../modules/**/*.tf"]
      enabled: true
    apply_requirements:
      - approved
      - mergeable

  - name: production
    dir: environments/production
    workspace: default
    autoplan:
      enabled: false    # Non pianifica automaticamente in prod
    apply_requirements:
      - approved        # Richiede almeno 2 approvazioni
      - mergeable
    allowed_override_dentists:  # Solo questi utenti possono fare apply in prod
      - alice
      - bob

# Comandi Atlantis nel Pull Request (come commento)

# Esegui plan per il progetto 'dev'
atlantis plan -p dev

# Esegui plan per tutti i progetti modificati
atlantis plan

# Applica dopo approvazione del PR
atlantis apply -p dev

# Annulla un plan in corso
atlantis unlock

# Mostra lo stato attuale
atlantis status

# docker-compose.yml per Atlantis self-hosted
version: '3.8'

services:
  atlantis:
    image: ghcr.io/runatlantis/atlantis:v0.27.0
    ports:
      - "4141:4141"
    environment:
      ATLANTIS_GH_USER: atlantis-bot           # GitHub user del bot
      ATLANTIS_GH_TOKEN: ${GH_TOKEN}            # Personal Access Token del bot
      ATLANTIS_GH_WEBHOOK_SECRET: ${WEBHOOK_SECRET}
      ATLANTIS_REPO_ALLOWLIST: "github.com/myorg/terraform-infra"
      ATLANTIS_AUTOMERGE: "false"
      ATLANTIS_WRITE_GIT_CREDS: "true"

      # Credenziali AWS (oppure usa IAM Instance Profile)
      AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
      AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
      AWS_DEFAULT_REGION: eu-west-1

    volumes:
      - atlantis-data:/home/atlantis
      - ./atlantis.yaml:/home/atlantis/atlantis.yaml:ro

volumes:
  atlantis-data:

Drift Detection: Cron Job to Detect Discrepancies

Il drift it's when the actual infrastructure differs from what is described in the Terraform state file. Drift happens when someone manually edits resources from the AWS console, when a resource is changed by an external process, or when a cloud API changes behavior. 

# .github/workflows/drift-detection.yml
name: Terraform Drift Detection

on:
  schedule:
    - cron: '0 8 * * 1-5'  # Ogni giorno lavorativo alle 8:00 UTC
  workflow_dispatch:          # Esecuzione manuale

permissions:
  contents: read
  issues: write               # Per aprire issue se drift rilevato
  id-token: write

jobs:
  detect-drift:
    name: Detect Drift (${{ matrix.environment }})
    runs-on: ubuntu-latest
    strategy:
      matrix:
        environment: [dev, staging, production]
      fail-fast: false

    defaults:
      run:
        working-directory: environments/${{ matrix.environment }}

    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: '1.7.5'

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/GitHubActionsReadOnly-${{ matrix.environment }}
          aws-region: eu-west-1

      - name: Terraform Init
        run: terraform init -no-color

      - name: Terraform Plan (Drift Detection)
        id: drift
        run: |
          # Esci con codice 0 se no changes, 1 se error, 2 se changes (drift)
          terraform plan -detailed-exitcode -no-color 2>&1 | tee drift_output.txt
          echo "EXIT_CODE=${PIPESTATUS[0]}" >> "$GITHUB_ENV"

      - name: Alert on Drift
        if: env.EXIT_CODE == '2'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const output = fs.readFileSync('environments/${{ matrix.environment }}/drift_output.txt', 'utf8');

            // Crea un issue GitHub per il drift rilevato
            await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: '[DRIFT] Infrastructure drift detected in ${{ matrix.environment }}',
              body: `## Drift rilevato in ambiente: ${{ matrix.environment }}

            Data: ${new Date().toISOString()}

            \`\`\`
            ${output.substring(0, 60000)}
            \`\`\`

            **Azione richiesta:** Analizza le modifiche e riconcilia lo state.
            - Se la modifica è intenzionale: aggiorna il codice Terraform e fai un PR
            - Se è una modifica non autorizzata: ripristina con \`terraform apply\``,
              labels: ['infrastructure', 'drift', '${{ matrix.environment }}'],
            });

      - name: Notify Slack on Drift
        if: env.EXIT_CODE == '2'
        uses: slackapi/slack-github-action@v1.26.0
        with:
          payload: |
            {
              "text": ":warning: Terraform drift rilevato in *${{ matrix.environment }}*!\nControlla il <${{ github.server_url }}/${{ github.repository }}/issues|GitHub Issue> creato.",
              "channel": "#infra-alerts"
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

TFLint: Advanced Linting for HCL

# .tflint.hcl - Configurazione TFLint
plugin "aws" {
  enabled = true
  version = "0.29.0"
  source  = "github.com/terraform-linters/tflint-ruleset-aws"
}

plugin "azurerm" {
  enabled = true
  version = "0.26.0"
  source  = "github.com/terraform-linters/tflint-ruleset-azurerm"
}

# Regole generali
rule "terraform_deprecated_interpolation" {
  enabled = true
}

rule "terraform_documented_outputs" {
  enabled = true
}

rule "terraform_documented_variables" {
  enabled = true
}

rule "terraform_naming_convention" {
  enabled = true
  format  = "snake_case"
}

rule "terraform_required_providers" {
  enabled = true
}

rule "terraform_required_version" {
  enabled = true
}

# Regole AWS specifiche
rule "aws_instance_invalid_type" {
  enabled = true
}

rule "aws_instance_previous_type" {
  enabled = true
}

Terraform Cloud: Managed Alternative

If you don't want to manage Atlantis yourself, HCP Terraform (formerly Terraform Cloud) offers remote plan/apply, state management, SSO and policy (Sentinel) as a service. 

# terraform.tf - Configurazione HCP Terraform come backend
terraform {
  required_version = "~> 1.7"

  # HCP Terraform (gestisce state, plan e apply in cloud)
  cloud {
    organization = "my-organization"

    workspaces {
      # Workspace per environment con tagging
      tags = ["terraform-infra", "aws"]
    }
  }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Con HCP Terraform:
# - Nessun backend S3 da gestire
# - Plan/Apply avvengono nei server HCP (non in CI)
# - Variabili e secrets gestiti nel workspace
# - VCS integration: trigger automatico da PR
# - Team-based access control
# - Run history permanente

Best Practices for Terraform Pipelines in Production

Terraform Pipeline Checklist

  • USA OIDC instead of long-lived AWS credentials in CI secrets
  • Run terraform fmt -check to block PR with unformatted code
  • USA terraform validate before the plan to catch syntax errors
  • Save the binary plan (-out=tfplan) and use that for apply, it doesn't regenerate
  • Run apply in sequential order (dev → staging → prod), not in parallel
  • Request manual approval for production with GitHub Environment protection rules
  • Implement scheduled drift detection to detect manual changes
  • Add prevent_destroy = true on critical stateful resources
  • Log all plans and apply with timestamp and author in the CI log

Conclusions and Next Steps

A well-designed Terraform pipeline is the foundation of an infrastructure safe and traceable. The Plan-Review-Apply workflow in the PR ensures that none infrastructural modification occurs without review, while drift detection Catch unplanned changes before they become problems.

Next Articles in the Series

  • Article 5: IaC Testing — Terratest, Terraform Test Native e Contract Testing: How to write automated tests for your Terraform modules.
  • Article 6: IaC Security — Checkov, Trivy and OPA Policy-as-Code: integrates security scanning into the pre-apply gate of the pipeline.