Robust Infrastructure Testing for Terraform Modules: Terratest and Kitchen-Terraform

As a senior engineer at Khadervali.com, I’ve seen firsthand how critical robust testing is to building reliable and scalable cloud infrastructure. In the world of Infrastructure as Code (IaC), Terraform has become an undisputed champion for provisioning and managing resources across various cloud providers. However, writing Terraform modules alone isn’t enough; we need to ensure they work as expected, every single time. This is where infrastructure testing comes into play, transforming our IaC from a set of scripts into a truly engineered product.

This article dives deep into two powerful tools for infrastructure testing with Terraform: Terratest and Kitchen-Terraform. We’ll explore their philosophies, capabilities, and how to integrate them into your development workflow to build bulletproof Terraform modules.

The Imperative of Infrastructure Testing

In traditional software development, testing is a non-negotiable part of the lifecycle. Unit tests, integration tests, end-to-end tests – they all contribute to code quality and confidence. Why should infrastructure be any different? When infrastructure failures occur, the impact can range from minor service disruptions to catastrophic outages, leading to significant financial losses and reputational damage.

IaC promises consistency and repeatability, but this promise is only as good as the code itself. A bug in a Terraform module can propagate across multiple environments, leading to:

• Configuration Drift: Unintended differences between environments.
• Security Vulnerabilities: Misconfigured resources opening doors for attackers.
• Downtime: Services failing to deploy or operating incorrectly.
• Cost Overruns: Unintentionally provisioned expensive resources.

Automated infrastructure testing acts as a safety net, catching these issues early in the development cycle, long before they impact production. It allows us to refactor with confidence, experiment with new features, and ensure that our infrastructure adheres to desired states and security policies.

The Unique Challenges of Testing Infrastructure

Testing infrastructure isn’t quite like testing application code. Here are some of the key challenges:

1. State Management: Terraform maintains a state file, which tracks the real-world resources. Tests must interact with this state, potentially creating and destroying actual cloud resources.
2. Long Provisioning Times: Creating and destroying cloud resources can take minutes, sometimes even tens of minutes, making fast feedback loops challenging.
3. Cost: Running tests that provision real resources incurs costs from cloud providers.
4. Dependencies: Infrastructure modules often depend on other modules or existing infrastructure. Mocking these dependencies can be complex.
5. Idempotency: Terraform is generally idempotent, but tests must verify that applying the same configuration multiple times yields the same result and that changes are applied correctly.
6. Clean-up: Ensuring all provisioned resources are destroyed after a test is paramount to avoid orphaned resources and unexpected costs.
7. External Integrations: Infrastructure rarely exists in a vacuum. Testing often involves validating interactions with other services, APIs, or even on-premise systems.

These challenges highlight the need for specialized tools and methodologies that can handle the intricacies of cloud environments. Enter Terratest and Kitchen-Terraform.

Terratest: The Go-Native Approach to Infrastructure Testing

Terratest, developed by Gruntwork, is a Go library that provides a comprehensive set of tools for testing infrastructure code. Its philosophy is to treat infrastructure as software and apply standard software testing practices, including unit, integration, and end-to-end tests, directly against real cloud environments.

What is Terratest? Why Go?

Terratest leverages Go’s native testing framework (testing package) to orchestrate infrastructure tests. Why Go?

• Concurrency: Go’s goroutines make it easy to manage concurrent operations, which is useful for tasks like waiting for resources to become available or performing multiple validations.
• Type Safety & Performance: Go is a compiled, statically typed language, offering performance benefits and catching many errors at compile time.
• Simplicity & Readability: Go’s syntax is relatively simple, making test code easier to write and maintain.
• Ecosystem: Go has robust libraries for HTTP, JSON, and interacting with cloud provider APIs, which Terratest uses extensively.

Terratest acts as an orchestration layer. It doesn’t directly run Terraform commands; instead, it wraps the Terraform CLI, allowing you to invoke terraform init, terraform apply, terraform output, and terraform destroy programmatically from your Go tests. Crucially, it then provides helper functions to interact with cloud provider APIs (AWS, Azure, GCP) to assert the state of deployed resources.

Anatomy of a Terratest Suite

A typical Terratest test case follows a pattern:

1. Define Options: Configure the path to your Terraform module, variables, and potentially a unique identifier for the test run.
2. Deploy: Run terraform init and terraform apply to provision the infrastructure.
3. Validate: Use Terratest’s helper functions (e.g., AWS SDK wrappers) to query the cloud provider and assert that the resources were created correctly and have the desired properties.
4. Clean Up: Defer a call to terraform destroy to tear down the infrastructure after the test, regardless of success or failure. This is critical for cost management and avoiding resource sprawl.

Terratest Example: Testing a Simple S3 Bucket Module

Let’s walk through an example. Suppose we have a simple Terraform module that creates an S3 bucket with specific configurations.

Terraform Module (`modules/s3-bucket/main.tf`)

resource "aws_s3_bucket" "this" {
  bucket = var.bucket_name
  acl    = var.acl

  tags = {
    Environment = var.environment
    ManagedBy   = "Terraform"
  }
}

resource "aws_s3_bucket_public_access_block" "this" {
  bucket = aws_s3_bucket.this.id

  block_public_acls       = var.block_public_acls
  block_public_policy     = var.block_public_policy
  ignore_public_acls      = var.ignore_public_acls
  restrict_public_buckets = var.restrict_public_buckets
}

variable "bucket_name" {
  description = "The name of the S3 bucket."
  type        = string
}

variable "acl" {
  description = "The canned ACL to apply to the bucket."
  type        = string
  default     = "private"
}

variable "environment" {
  description = "The environment tag for the bucket."
  type        = string
  default     = "test"
}

variable "block_public_acls" {
  description = "Whether Amazon S3 should block public ACLs for this bucket."
  type        = bool
  default     = true
}

variable "block_public_policy" {
  description = "Whether Amazon S3 should block public bucket policies for this bucket."
  type        = bool
  default     = true
}

variable "ignore_public_acls" {
  description = "Whether Amazon S3 should ignore public ACLs on this bucket."
  type        = bool
  default     = true
}

variable "restrict_public_buckets" {
  description = "Whether Amazon S3 should restrict public bucket policies for this bucket."
  type        = bool
  default     = true
}

output "bucket_id" {
  description = "The ID of the S3 bucket."
  value       = aws_s3_bucket.this.id
}

output "bucket_arn" {
  description = "The ARN of the S3 bucket."
  value       = aws_s3_bucket.this.arn
}

Terratest Go Test (`modules/s3-bucket/test/s3_test.go`)

package test

import (

	"fmt"

	"testing"

	"time"
	"github.com/gruntwork-io/terratest/modules/aws"

	"github.com/gruntwork-io/terratest/modules/random"

	"github.com/gruntwork-io/terratest/modules/terraform"

	"github.com/gruntwork-io/terratest/modules/test-structure"

)
func TestS3BucketModule(t *testing.T) {

	t.Parallel() // Allows tests to run in parallel
	// Specify the path to the Terraform module being tested.

	terraformDir := "../"
	// At the end of the test, run `terraform destroy` to clean up any resources that were created.

	defer test_structure.RunTestStage(t, "teardown", func() {

		terraformOptions := test_structure.LoadTerraformOptions(t, terraformDir)

		terraform.Destroy(t, terraformOptions)

	})
	// Deploy the infrastructure and save the TerraformOptions and test data to disk.

	test_structure.RunTestStage(t, "setup", func() {

		// Generate a unique bucket name to avoid collisions.

		expectedBucketName := fmt.Sprintf("khadervali-test-bucket-%s", random.UniqueId())

		expectedEnvironment := "dev"

		expectedAcl := "private"
		terraformOptions := terraform.With