Enterprise AWS Infrastructure
Multi-environment AWS infrastructure platform built with Terragrunt and Terraform using a hierarchical blueprint pattern — DRY, policy-gated and cost-aware.
- Terragrunt
- Terraform
- AWS
- OPA/Conftest
- GitHub Actions
The problem
Multi-environment Terraform drifts: dev and prod diverge through copy-paste, and nothing stops a change that is syntactically valid but violates security or blows the budget.
Constraints
- Environments must stay structurally identical while differing in scale
- Policy violations have to be caught before apply, not in review
- Cost impact needs to be visible at pull-request time
Architecture
Governance gates — run in parallel
↓ all gates pass ↓
Dev
Prod — protected
Interactive CI/CD Pipeline Simulator
Governance Gate Fan-Out & Policy Engine
CI Job Execution Topology
Review Loop Impact
Modeled from typical gate run-times, not a measured production run.
Key decisions
What was chosen, what it was chosen over, and why.
Chose
Hierarchical Terragrunt blueprints
Instead of
Duplicated Terraform per environment
A generic module library is kept strictly separate from live environment config, which inherits from it. Dev and prod cannot structurally diverge, which is the usual failure mode of per-environment directories.
Chose
OPA and Conftest evaluate the plan
Instead of
Static analysis of the HCL source
The plan shows what will actually be created, with module defaults, variables and computed values resolved. A rule like no-public-buckets is trivially evadable against source, where the offending value can arrive from three modules deep.
Chose
Governance gates fan out in parallel
Instead of
A sequential lint to plan to policy to cost chain
Static analysis, plan, policy and cost are independent. In a chain the first failure hides the rest, so three problems take three round trips to discover; in parallel one run reports every class of failure.
Also decided
Infracost sits alongside the security gatesnotReviewing cost monthly, after the fact
Cost surfaces while the change is still one revert away and in front of the person who made it, rather than weeks later in front of someone who did not.
Nightly drift detectionnotTrusting that applied state stays applied
Out-of-band changes are found on a schedule instead of during the next incident, which is the only way a Git-declared environment stays true over time.
Manual approval gate on prod onlynotUniform automation across environments
Dev applies automatically so the loop stays fast; prod requires a human after dev is stable. The gate is placed where the blast radius is, not everywhere.
Production resilience & failure modes
Failure mode this architecture has to account for
A CI runner is killed mid-apply (a GitHub Actions cancellation, a spot interruption on the runner) before Terraform releases its DynamoDB state lock — the next plan on that environment fails with "Error acquiring the state lock."
How it's handled
The platform doesn't roll back a partial apply — Terraform can't safely destroy resources stuck mid-creation, so the documented recovery is roll-forward: read the lock ID from the error, confirm the runner is actually dead, then `terragrunt force-unlock <LOCK_ID>` and re-apply. The S3 backend means the partial state from the failed run is already durable, so the next plan reconciles from wherever it stopped rather than from scratch.