Dharm Pal

Posted on Oct 9

Infrastructure as Code: Deploying AWS Cost Optimization Engines with Terraform

#aws #devops #terraform #automation

In my previous post, we looked at a Python script designed to automatically clean up idle EBS volumes. However, manually deploying that script, configuring IAM roles, and setting up EventBridge schedules via the AWS Management Console doesn't scale for production.

To achieve true enterprise operational efficiency, everything must be managed as Infrastructure as Code (IaC).

With over 12 years in IT and 6+ years managing scalable AWS infrastructure, I always recommend automating cloud governance architectures from day one. In this article, I will share the complete production-ready Terraform templates to package and deploy our serverless cost optimization engine.

📁 Project Directory Structure

To keep our infrastructure configurations maintainable, we will break our codebase into standard Terraform modules:

aws-cost-optimizer/
├── main.tf        # Core Lambda & EventBridge infrastructure
├── iam.tf         # Strict IAM roles and least-privilege policies
├── variables.tf   # Customizable deployment variables
└── src/
    └── lambda_function.py  # Our optimization script from post #1

🛠️ 1. Defining Variables (`variables.tf`)

First, we establish our variables. This enables different environment setups (like Dev vs. Production) to utilize custom time frames for resource checks.

variable "aws_region" {
  type        = string
  default     = "us-east-1"
  description = "The target AWS region for deployment"
}

variable "retention_days" {
  type        = number
  default     = 7
  description = "Number of days an EBS volume must be idle before deletion"
}

🔒 2. Strict IAM Security Guardrails (`iam.tf`)

As an AWS Certified Solutions Architect, I cannot stress enough the importance of the Principle of Least Privilege. Our Lambda execution role should only have permissions to scan and delete what it absolutely needs.

# Create the IAM Role for Lambda Execution
resource "aws_iam_role" "lambda_cost_optimizer_role" {
  name = "lambda-cost-optimizer-execution-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action    = "sts:AssumeRole"
      Effect    = "Allow"
      Principal = { Service = "lambda.amazonaws.com" }
    }]
  })
}

# Attach Strict Least-Privilege Custom Policies
resource "aws_iam_role_policy" "lambda_ec2_policy" {
  name = "lambda-cost-optimizer-ec2-policy"
  role = aws_iam_role.lambda_cost_optimizer_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Allow"
        Action   = [
          "ec2:DescribeVolumes",
          "ec2:CreateSnapshot",
          "ec2:DeleteVolume"
        ]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ]
        Resource = "arn:aws:logs:*:*:*"
      }
    ]
  })
}

🏗️ 3. Main Automation & Scheduling Engine (`main.tf`)

This file handles packaging the Python script on the fly, creating the Lambda resource, and setting up the weekly EventBridge cron scheduler.

provider "aws" {
  region = var.aws_region
}

# Automatically package the Lambda source directory into a ZIP archive
data "archive_file" "lambda_zip" {
  type        = "zip"
  source_dir  = "${path.module}/src"
  output_path = "${path.module}/lambda_function.zip"
}

# Deploy the Lambda Resource
resource "aws_lambda_function" "cost_optimizer" {
  filename         = data.archive_file.lambda_zip.output_path
  function_name    = "ebs-cost-optimizer-engine"
  role             = aws_iam_role.lambda_cost_optimizer_role.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = "python3.11"
  source_code_hash = data.archive_file.lambda_zip.output_base64sha256
  timeout          = 60

  environment {
    variables = {
      RETENTION_DAYS = var.retention_days
    }
  }
}

# Setup the EventBridge Cron Trigger (Every Friday at 6:00 PM UTC)
resource "aws_cloudwatch_event_rule" "weekly_trigger" {
  name                = "ebs-cleanup-weekly-schedule"
  description         = "Triggers cost optimization engine at the end of the work week"
  schedule_expression = "cron(0 18 ? * FRI *)"
}

# Link EventBridge to Lambda
resource "aws_cloudwatch_event_target" "trigger_lambda" {
  rule      = aws_cloudwatch_event_rule.weekly_trigger.name
  target_id = "TriggerLambdaCostOptimizer"
  arn       = aws_lambda_function.cost_optimizer.arn
}

# Allow EventBridge to Invoke the Lambda Function
resource "aws_lambda_permission" "allow_eventbridge" {
  statement_id  = "AllowExecutionFromEventBridge"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.cost_optimizer.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.weekly_trigger.arn
}

💡 Production Architecture Best Practices

When deploying this inside your organization, implement these two key practices:

Remote State Files: Never store your local terraform.tfstate file on your work machine. Store it safely inside an encrypted Amazon S3 bucket configured with state locking via an Amazon DynamoDB table to prevent concurrent updates.
Environment Isolation: Use Terraform workspaces or distinct folder paths (environments/dev and environments/prod) to thoroughly test code changes in staging environments prior to executing deployments in production data structures.

🚀 Conclusion

By combining AWS serverless computing with Terraform, your organization shifts away from tedious manual monitoring dashboards into an elegant model of fully automated infrastructure lifecycle management.

Do you manage your serverless scripts via cloud consoles, or do you standardize everything through infrastructure as code pipelines? Let's discuss in the comments below!

Top comments (3)

Vlad Z • Oct 10

Moving from a one-off Python cleanup script to a Terraform-deployed engine is the right next step, a script someone runs manually only works until the one time somebody forgets

Is the engine scoped to EBS cleanup specifically still, or has it grown to cover other idle resource types since the last post?

Dharm Pal • Oct 12

Right now, the engine in this post is still focused primarily on EBS volume cleanup (identifying and snapshotting/deleting unattached volumes). However, expanding it to cover other idle resources like unassociated Elastic IPs, orphaned NAT Gateways, and idle RDS instances is definitely the roadmap. I'm planning to cover those expansions in an upcoming post!

Vlad Z • Oct 25

Elastic IPs and orphaned NAT gateways are usually the quieter ones compared to EBS, nobody notices an unused IP or a NAT nobody wired traffic through anymore until someone goes looking specifically

Out of EBS, EIPs, and NAT gateways, which one do you expect to turn up the most total waste once the engine covers all three?

Some comments may only be visible to logged-in visitors. Sign in to view all comments.

📁 Project Directory Structure

🛠️ 1. Defining Variables (variables.tf)

🔒 2. Strict IAM Security Guardrails (iam.tf)

🏗️ 3. Main Automation & Scheduling Engine (main.tf)

💡 Production Architecture Best Practices

🚀 Conclusion

🛠️ 1. Defining Variables (`variables.tf`)

🔒 2. Strict IAM Security Guardrails (`iam.tf`)

🏗️ 3. Main Automation & Scheduling Engine (`main.tf`)