HashiConfJoin us for our next online community event 8-11 June (CEST) Register Now

Query Data Sources

  • 10 min
  • Products Usedterraform

Cloud infrastructure, applications, and services emit data, which Terraform can query and act on using data sources. Terraform uses data sources to fetch information from cloud provider APIs, such as disk image IDs, or information about the rest of your infrastructure through the outputs of other Terraform configurations.

Data sources allow you to load data from APIs or other Terraform workspaces. You can use this data to make your project’s configuration more flexible, and to connect workspaces that manage different parts of your infrastructure. You can also use data sources to connect and share data between workspaces in Terraform Cloud and Terraform Enterprise.

In this tutorial, you will use Terraform to deploy a workspace containing a VPC and security groups on AWS in the us-east-1 region. Next, you will use the aws_availability_zones data source to configure your VPC’s Availability Zones (AZs), allowing you to deploy this configuration in any AWS region. Then, you will use the terraform_remote_state data source to deploy another workspace containing your application infrastructure to your VPC. Finally, you will use the aws_ami data source to configure the correct AMI for the current region.

»Prerequisites

In order to follow this tutorial, you will need:

»Create infrastructure

The example configuration for this tutorial is hosted in two GitHub repositories.

First, clone the VPC repository.

$ git clone https://github.com/hashicorp/learn-terraform-data-sources-vpc.git

Next, clone the application repository.

$ git clone https://github.com/hashicorp/learn-terraform-data-sources-app.git

Change to the VPC repository directory.

$ cd learn-terraform-data-sources-vpc

The configuration in main.tf defines a VPC and security groups for your application.

Initialize this configuration.

$ terraform init
Initializing modules...

## ...Output truncated...

Terraform has been successfully initialized!

## ...Output truncated...

»Update VPC region

The VPC configuration uses a variable called aws_region with a default value of us-east-1 to set the region.

However, changing the value of the aws_region variable will not successfully change the region because the VPC configuration includes an azs argument to set Availability Zones, which is a hard-coded list of availability zones in the us-east-1 region.

module "vpc" {
# ...
  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
# ...
}

Use the aws_availability_zones data source to load the available AZs for the current region. Add the following to main.tf.

data "aws_availability_zones" "available" {
  state = "available"
}

The aws_availability_zones data source is part of the AWS provider, and its documentation is under its provider in the Terraform registry. Like resources, data source blocks support arguments to specify how they behave. In this case, the state argument limits the availability zones to only those that are currently available.

You can reference data source attributes with the pattern data.<NAME>.<ATTRIBUTE>. Update the VPC configuration to use this data source to set the list of availability zones.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.64.0"

  cidr = var.vpc_cidr_block

-  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
+  azs             = data.aws_availability_zones.available.names

Apply this configuration, setting the value of aws_region to us-west-1. Respond to the confirmation prompt with a yes.

$ terraform apply -var aws_region=us-west-1
## ...Output truncated...

Plan: 34 to add, 0 to change, 0 to destroy.

## ...Output truncated...

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

## ...Output truncated...

Apply complete! Resources: 34 added, 0 changed, 0 destroyed.

Outputs:

app_security_group_ids = [
  "sg-0214d055921c25c8e",
]
lb_security_group_ids = [
  "sg-03f34e1dd93483bd9",
]
private_subnet_ids = [
  "subnet-07b77ef2e9c386a17",
  "subnet-098f226b620943eac",
]
public_subnet_ids = [
  "subnet-034fc6327aeee353f",
  "subnet-0a9a7558a4eaa4640",
]

Before moving on, have the VPC workspace output the region, which the application workspace requires as an input. Add a data source to main.tf to access region information.

data "aws_region" "current" { }

Add an output for the region to outputs.tf.

output "aws_region" {
  description = "AWS region"
  value       = data.aws_region.current.name
}

Run terraform apply to see the region name included in the outputs. Remember to respond to the confirmation prompt with a yes.

$ terraform apply -var aws_region=us-west-1
## ...Output truncated...

Plan: 0 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + aws_region = "us-west-1"

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

app_security_group_ids = [
  "sg-07497935027018f0b",
]
aws_region = "us-west-1"
lb_security_group_ids = [
  "sg-07bebc432da151edf",
]
private_subnet_ids = [
  "subnet-0478ce1fe723c6c1a",
  "subnet-05a921f34e27a88de",
]
public_subnet_ids = [
  "subnet-000a2862729e9bd30",
  "subnet-04bc27821d54b1a56",
]

»Configure Terraform remote state

Now that your VPC workspace is deployed, go to the app workspace directory.

$ cd ../learn-terraform-data-sources-app

This directory contains the Terraform configuration for your application. Like the VPC workspace, this configuration includes hard-coded values for the us-east-1 region. You can use the terraform_remote_state data source to use another Terraform workspace's output data.

Add a terraform_remote_state data source to the main.tf file inside the learn-terraform-data-sources-app directory.

data "terraform_remote_state" "vpc" {
  backend = "local"

  config = {
    path = "../learn-terraform-data-sources-vpc/terraform.tfstate"
  }
}

This remote state block uses the local backend to load state data from the path in the config section. Terraform remote state also supports a remote backend type for use with remote systems, such as Terraform Cloud, Consul, or other systems.

Replace the hard-coded region configuration in main.tf with the region output from the VPC workspace.

provider "aws" {
-  region = "us-east-1"
+  region = data.terraform_remote_state.vpc.outputs.aws_region
}

Configure the load balancer security group and subnet arguments with the corresponding outputs from your VPC workspace.

module "elb_http" {
# ...
-  security_groups = []
-  subnets         = []
+  security_groups = data.terraform_remote_state.vpc.outputs.lb_security_group_ids
+  subnets         = data.terraform_remote_state.vpc.outputs.public_subnet_ids
# ...
}

»Scale EC2 instances

The configuration in main.tf only uses a single EC2 instance. Update the configuration to use multiple EC2 instances per subnet.

resource "aws_instance" "app" {
+  count = var.instances_per_subnet * length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)
+
  ami = "ami-04d29b6f966df1537"

# ...
}

You can use values from data sources just like any other Terraform values, including by passing them to functions. Now when you apply this configuration, Terraform will provision var.instances_per_subnet instances for each private subnet configured in your VPC workspace.

»Configure region-specific AMIs

The AWS instance configuration also uses a hard-coded AMI ID, which is only valid for the us-east-1 region. Use an aws_ami data source to load the correct AMI ID for the current region. Add the following to main.tf.

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

Replace the hard-coded AMI ID with the one loaded from the new data source.

resource "aws_instance" "app" {
  count = var.instances_per_subnet * length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)

-  ami = "ami-04d29b6f966df1537"
+  ami = data.aws_ami.amazon_linux.id

# ...
}

»Configure EC2 subnet and security groups

Finally, update the EC2 instance configuration to use the subnet and security group configuration from the VPC workspace.

resource "aws_instance" "app" {
# ...

-  subnet_id              = ""
-  vpc_security_group_ids = []
+  subnet_id              = data.terraform_remote_state.vpc.outputs.private_subnet_ids[count.index % length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)]
+  vpc_security_group_ids = data.terraform_remote_state.vpc.outputs.app_security_group_ids

# ...
}

Now that your app workspace uses remote state data from the VPC workspace, initialize the app workspace.

$ terraform init
Initializing modules...

## ...Output truncated...

Terraform has been successfully initialized!

## ...Output truncated...

Apply the configuration and Terraform will provision the application infrastructure. Respond to the confirmation prompt with a yes.

$ terraform apply
## ...Output truncated...

Plan: 10 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + lb_url             = (known after apply)
  + web_instance_count = 4

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

## ...Output truncated...

Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

Outputs:

lb_url = "http://lb-DOf-tutorial-example-1971328425.us-west-2.elb.amazonaws.com/"
web_instance_count = 4

After a few minutes, the load balancer health checks will pass, and will return the example response.

$ curl $(terraform output -raw lb_url)
<html><body><div>Hello, world!</div></body></html>

You deployed your application using one workspace for the VPC and security groups, and another for the application infrastructure. You used the terraform_remote_state data source to share data between them. You also replaced region-specific configuration with dynamic values from several other data sources.

»Clean up your infrastructure

Before moving on, destroy the infrastructure you created in this tutorial.

In the application workspace, destroy the application infrastructure. Respond to the confirmation prompt with yes.

$ terraform destroy
## ...Output truncated...

Plan: 0 to add, 0 to change, 10 to destroy.

Changes to Outputs:
  - lb_url             = "http://lb-DOf-tutorial-example-1971328425.us-west-2.elb.amazonaws.com/" -> null
  - web_instance_count = 4 -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

## ...Output truncated...

Destroy complete! Resources: 10 destroyed.

Now move on to the VPC workspace. Once again, respond to the confirmation prompt with yes.

$ cd ../learn-terraform-data-sources-vpc

Destroy this infrastructure as well.

$ terraform destroy -var aws_region=us-west-1
## ...Output truncated...

Plan: 0 to add, 0 to change, 34 to destroy.

Changes to Outputs:
  - app_security_group_ids = [
      - "sg-0214d055921c25c8e",
    ] -> null
  - aws_region             = "us-west-2" -> null
  - lb_security_group_ids  = [
      - "sg-03f34e1dd93483bd9",
    ] -> null
  - private_subnet_ids     = [
      - "subnet-07b77ef2e9c386a17",
      - "subnet-098f226b620943eac",
    ] -> null
  - public_subnet_ids      = [
      - "subnet-034fc6327aeee353f",
      - "subnet-0a9a7558a4eaa4640",
    ] -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

## ...Output truncated...

Destroy complete! Resources: 34 destroyed.

»Next steps

Now that you have used Terraform data sources, check out the following resources for more information.

Back to Collection
HashiCorp
stdin: is not a tty