From ClickOps to Terraform: what we measured
How we cut MTTR and why state files deserve respect.
Problem
Manual cloud changes (ClickOps) caused drift, weak audit trails, and hard-to-reproduce incident fixes.
Solution
We moved core AWS resources to Terraform modules, standardized environments, and enforced PR-based infra changes.
Impact
MTTR dropped, rollbacks became predictable, and infra changes became reviewable and repeatable.
The allure of the AWS console is strong. It's visual, immediate, and perfectly designed to make you feel productive. But as any team scaling past their first few services quickly learns, "ClickOps" is the enemy of reliability.
The Hidden Cost of Manual Provisioning
Before our transition, spinning up a new environment was an oral tradition passed down from senior engineers to new hires. We documented it in Confluence, but the documentation was always a step behind reality. When incidents occurred, reproducing the exact state of a production service in a staging environment to debug it was nearly impossible. We were battling configuration drift on a weekly basis.
A manual update to a security group during a sev-1 incident accidentally exposed an internal service. It took us 45 minutes just to realize what had changed because there was no pull request or audit trail outside of CloudTrail logs.
The Migration Strategy
We didn't boil the ocean. We started with the foundational layers: networking (VPCs, Subnets, Route Tables) and IAM. Once the base was solid, we moved our RDS instances and finally, the compute layer (EKS/ECS).
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "5.0.0"
name = "${var.environment}-main-vpc"
cidr = var.vpc_cidr
azs = ["us-east-1a", "us-east-1b", "us-east-1c"]
private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
enable_nat_gateway = true
single_nat_gateway = var.environment != "prod"
}What We Measured (The ROI)
- MTTR Dropped by 60%
Rollbacks became a simple
git revertandterraform applyrather than a frantic click-hunt. - Environment Parity
Staging finally became an exact replica of production, drastically reducing "it works on my machine" bugs.
Takeaway
Terraform isn't just about automation; it's about shifting infrastructure changes into the same review, test, and audit pipelines as your application code. State files require respect and proper remote backends, but the predictability they provide is unmatched.