Query Data Sources | Terraform

Cloud infrastructure, applications, and services emit data, which Terraform can query and act on using data sources. Terraform uses data sources to fetch information from cloud provider APIs, such as disk image IDs, or information about the rest of your infrastructure through the outputs of other Terraform configurations.

Data sources allow you to load data from APIs or other Terraform workspaces. You can use this data to make your project's configuration more flexible, and to connect workspaces that manage different parts of your infrastructure. You can also use data sources to connect and share data between workspaces in Terraform Cloud and Terraform Enterprise.

In this tutorial, you will use Terraform to deploy a workspace containing a VPC and security groups on AWS in the us-east-1 region. Next, you will use the aws_availability_zones data source to configure your VPC's Availability Zones (AZs), allowing you to deploy this configuration in any AWS region. Then, you will use the terraform_remote_state data source to deploy another workspace containing your application infrastructure to your VPC. Finally, you will use the aws_ami data source to configure the correct AMI for the current region.

»Prerequisites

In order to follow this tutorial, you will need:

The Terraform CLI, version 0.13 or later.
AWS Credentials configured for use with Terraform.
The git CLI.

Note: Some of the infrastructure in this tutorial may not qualify for the AWS free tier. Destroy the infrastructure at the end of the guide to avoid unnecessary charges. We are not responsible for any charges that you incur.

»Clone example repositories

The example configuration for this tutorial is hosted in two GitHub repositories.

The VPC workspace contains the configuration to deploy a VPC and security groups for your application.

Clone the VPC repository.

$ git clone https://github.com/hashicorp/learn-terraform-data-sources-vpc.git

$ git clone https://github.com/hashicorp/learn-terraform-data-sources-vpc.git

The application workspace contains the configuration to deploy a VPC and security groups for your application.

Clone the application repository.

$ git clone https://github.com/hashicorp/learn-terraform-data-sources-app.git

$ git clone https://github.com/hashicorp/learn-terraform-data-sources-app.git

»Update VPC region

Change to the VPC repository directory.

$ cd learn-terraform-data-sources-vpc

$ cd learn-terraform-data-sources-vpc

The VPC configuration uses a variable called aws_region with a default value of us-east-1 to set the region.

However, changing the value of the aws_region variable will not successfully change the region because the VPC configuration includes an azs argument to set Availability Zones, which is a hard-coded list of availability zones in the us-east-1 region.

module "vpc" {
  ## ...
  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
  ## ...
}

module "vpc" {
  ## ...
  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
  ## ...
}

Use the aws_availability_zones data source to load the available AZs for the current region. Add the following to main.tf.

data "aws_availability_zones" "available" {
  state = "available"
}

data "aws_availability_zones" "available" {
  state = "available"
}

The aws_availability_zones data source is part of the AWS provider, and its documentation is under its provider in the Terraform registry. Like resources, data source blocks support arguments to specify how they behave. In this case, the state argument limits the availability zones to only those that are currently available.

You can reference data source attributes with the pattern data.<NAME>.<ATTRIBUTE>. Update the VPC configuration to use this data source to set the list of availability zones.

 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
   version = "2.64.0"

   cidr = var.vpc_cidr_block

-  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
+  azs             = data.aws_availability_zones.available.names

   ## ...
 }

 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
   version = "2.64.0"

   cidr = var.vpc_cidr_block

-  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
+  azs             = data.aws_availability_zones.available.names

   ## ...
 }

Initialize this configuration.

$ terraform init
Initializing modules...

## ...

Terraform has been successfully initialized!

## ...

$ terraform init
Initializing modules...

## ...

Terraform has been successfully initialized!

## ...

Apply this configuration, setting the value of aws_region to us-west-1. Respond to the confirmation prompt with a yes.

$ terraform apply -var aws_region=us-west-1
## ...

Plan: 34 to add, 0 to change, 0 to destroy.

## ...

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

  ## ...

Apply complete! Resources: 34 added, 0 changed, 0 destroyed.

Outputs:

app_security_group_ids = [
  "sg-0214d055921c25c8e",
]
lb_security_group_ids = [
  "sg-03f34e1dd93483bd9",
]
private_subnet_ids = [
  "subnet-07b77ef2e9c386a17",
  "subnet-098f226b620943eac",
]
public_subnet_ids = [
  "subnet-034fc6327aeee353f",
  "subnet-0a9a7558a4eaa4640",
]

$ terraform apply -var aws_region=us-west-1
## ...

Plan: 34 to add, 0 to change, 0 to destroy.

## ...

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

  ## ...

Apply complete! Resources: 34 added, 0 changed, 0 destroyed.

Outputs:

app_security_group_ids = [
  "sg-0214d055921c25c8e",
]
lb_security_group_ids = [
  "sg-03f34e1dd93483bd9",
]
private_subnet_ids = [
  "subnet-07b77ef2e9c386a17",
  "subnet-098f226b620943eac",
]
public_subnet_ids = [
  "subnet-034fc6327aeee353f",
  "subnet-0a9a7558a4eaa4640",
]

Before moving on, have the VPC workspace output the region, which the application workspace requires as an input. Add a data source to main.tf to access region information.

data "aws_region" "current" { }

data "aws_region" "current" { }

Add an output for the region to outputs.tf.

output "aws_region" {
  description = "AWS region"
  value       = data.aws_region.current.name
}

output "aws_region" {
  description = "AWS region"
  value       = data.aws_region.current.name
}

Run terraform apply to see the region name included in the outputs. Remember to respond to the confirmation prompt with a yes.

$ terraform apply -var aws_region=us-west-1
## ...

Plan: 0 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + aws_region = "us-west-1"

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

app_security_group_ids = [
  "sg-07497935027018f0b",
]
aws_region = "us-west-1"
lb_security_group_ids = [
  "sg-07bebc432da151edf",
]
private_subnet_ids = [
  "subnet-0478ce1fe723c6c1a",
  "subnet-05a921f34e27a88de",
]
public_subnet_ids = [
  "subnet-000a2862729e9bd30",
  "subnet-04bc27821d54b1a56",
]

$ terraform apply -var aws_region=us-west-1
## ...

Plan: 0 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + aws_region = "us-west-1"

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

app_security_group_ids = [
  "sg-07497935027018f0b",
]
aws_region = "us-west-1"
lb_security_group_ids = [
  "sg-07bebc432da151edf",
]
private_subnet_ids = [
  "subnet-0478ce1fe723c6c1a",
  "subnet-05a921f34e27a88de",
]
public_subnet_ids = [
  "subnet-000a2862729e9bd30",
  "subnet-04bc27821d54b1a56",
]

Tip: In this scenario, you could use the aws_region variable to define the output parameter instead of using the data source. However, there are multiple ways to configure the AWS region. Using the aws_region data source will get the AWS provider's current region no matter how it was configured.

»Configure Terraform remote state

Now that your VPC workspace is deployed, go to the learn-terraform-data-sources-app workspace directory.

$ cd ../learn-terraform-data-sources-app

$ cd ../learn-terraform-data-sources-app

This directory contains the Terraform configuration for your application. Like the VPC workspace, this configuration includes hard-coded values for the us-east-1 region. You can use the terraform_remote_state data source to use another Terraform workspace's output data.

Tip: We recommend using provider-specific data sources when convenient. terraform_remote_state is more flexible, but requires access to the whole Terraform state.

Add a terraform_remote_state data source to the main.tf file inside the learn-terraform-data-sources-app directory.

data "terraform_remote_state" "vpc" {
  backend = "local"

  config = {
    path = "../learn-terraform-data-sources-vpc/terraform.tfstate"
  }
}

data "terraform_remote_state" "vpc" {
  backend = "local"

  config = {
    path = "../learn-terraform-data-sources-vpc/terraform.tfstate"
  }
}

This remote state block uses the local backend to load state data from the path in the config section. Terraform remote state also supports a remote backend type for use with remote systems, such as Terraform Cloud, Consul, or other systems.

Replace the hard-coded region configuration in main.tf with the region output from the VPC workspace.

 provider "aws" {
-  region = "us-east-1"
+  region = data.terraform_remote_state.vpc.outputs.aws_region
 }

 provider "aws" {
-  region = "us-east-1"
+  region = data.terraform_remote_state.vpc.outputs.aws_region
 }

Configure the load balancer security group and subnet arguments with the corresponding outputs from your VPC workspace.

 module "elb_http" {
   ## ...
-  security_groups = []
-  subnets         = []
+  security_groups = data.terraform_remote_state.vpc.outputs.lb_security_group_ids
+  subnets         = data.terraform_remote_state.vpc.outputs.public_subnet_ids
   ## ...
 }

 module "elb_http" {
   ## ...
-  security_groups = []
-  subnets         = []
+  security_groups = data.terraform_remote_state.vpc.outputs.lb_security_group_ids
+  subnets         = data.terraform_remote_state.vpc.outputs.public_subnet_ids
   ## ...
 }

Note: Terraform remote state can only load "root-level" output values from the source workspace, it cannot directly access values from resources or modules in the source workspace. To retrieve those values, you must add a corresponding output to the source workspace.

»Scale EC2 instances

The configuration in main.tf only uses a single EC2 instance. Update the configuration to use multiple EC2 instances per subnet.

 resource "aws_instance" "app" {
   ## ...
+  count = var.instances_per_subnet * length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)
+
   ami = "ami-04d29b6f966df1537"
   ## ...
 }

 resource "aws_instance" "app" {
   ## ...
+  count = var.instances_per_subnet * length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)
+
   ami = "ami-04d29b6f966df1537"
   ## ...
 }

You can use values from data sources just like any other Terraform values, including by passing them to functions. Now when you apply this configuration, Terraform will provision var.instances_per_subnet instances for each private subnet configured in your VPC workspace.

»Configure region-specific AMIs

The AWS instance configuration also uses a hard-coded AMI ID, which is only valid for the us-east-1 region. Use an aws_ami data source to load the correct AMI ID for the current region. Add the following to main.tf.

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

Replace the hard-coded AMI ID with the one loaded from the new data source.

 resource "aws_instance" "app" {
   count = var.instances_per_subnet * length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)

-  ami = "ami-04d29b6f966df1537"
+  ami = data.aws_ami.amazon_linux.id

   ## ...
 }

 resource "aws_instance" "app" {
   count = var.instances_per_subnet * length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)

-  ami = "ami-04d29b6f966df1537"
+  ami = data.aws_ami.amazon_linux.id

   ## ...
 }

»Configure EC2 subnet and security groups

Finally, update the EC2 instance configuration to use the subnet and security group configuration from the VPC workspace.

 resource "aws_instance" "app" {
   ## ...

-  subnet_id              = ""
-  vpc_security_group_ids = []
+  subnet_id              = data.terraform_remote_state.vpc.outputs.private_subnet_ids[count.index % length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)]
+  vpc_security_group_ids = data.terraform_remote_state.vpc.outputs.app_security_group_ids

   ## ...
 }

 resource "aws_instance" "app" {
   ## ...

-  subnet_id              = ""
-  vpc_security_group_ids = []
+  subnet_id              = data.terraform_remote_state.vpc.outputs.private_subnet_ids[count.index % length(data.terraform_remote_state.vpc.outputs.private_subnet_ids)]
+  vpc_security_group_ids = data.terraform_remote_state.vpc.outputs.app_security_group_ids

   ## ...
 }

Now that your app workspace uses remote state data from the VPC workspace, initialize the app workspace.

$ terraform init
Initializing modules...

## ...

Terraform has been successfully initialized!

## ...

$ terraform init
Initializing modules...

## ...

Terraform has been successfully initialized!

## ...

Apply the configuration and Terraform will provision the application infrastructure. Respond to the confirmation prompt with a yes.

$ terraform apply
## ...

Plan: 10 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + lb_url             = (known after apply)
  + web_instance_count = 4

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

  ## ...

Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

Outputs:

lb_url = "http://lb-DOf-tutorial-example-1971328425.us-west-2.elb.amazonaws.com/"
web_instance_count = 4

$ terraform apply
## ...

Plan: 10 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + lb_url             = (known after apply)
  + web_instance_count = 4

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

  ## ...

Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

Outputs:

lb_url = "http://lb-DOf-tutorial-example-1971328425.us-west-2.elb.amazonaws.com/"
web_instance_count = 4

After a few minutes, the load balancer health checks will pass, and will return the example response.

$ curl $(terraform output -raw lb_url)
<html><body><div>Hello, world!</div></body></html>

$ curl $(terraform output -raw lb_url)
<html><body><div>Hello, world!</div></body></html>

Tip: It can take several minutes for the load balancer to become available. If the curl command returns an error, try again after a few minutes.

You deployed your application using one workspace for the VPC and security groups, and another for the application infrastructure. You used the terraform_remote_state data source to share data between them. You also replaced region-specific configuration with dynamic values from several other data sources.

»Clean up your infrastructure

Before moving on, destroy the infrastructure you created in this tutorial.

In the application workspace, destroy the application infrastructure. Respond to the confirmation prompt with yes.

$ terraform destroy
## ...

Plan: 0 to add, 0 to change, 10 to destroy.

Changes to Outputs:
  - lb_url             = "http://lb-DOf-tutorial-example-1971328425.us-west-2.elb.amazonaws.com/" -> null
  - web_instance_count = 4 -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

  ## ...

Destroy complete! Resources: 10 destroyed.

$ terraform destroy
## ...

Plan: 0 to add, 0 to change, 10 to destroy.

Changes to Outputs:
  - lb_url             = "http://lb-DOf-tutorial-example-1971328425.us-west-2.elb.amazonaws.com/" -> null
  - web_instance_count = 4 -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

  ## ...

Destroy complete! Resources: 10 destroyed.

Note: You must destroy the application workspace before the VPC workspace. Since the resources in the application workspace depend on those in the VPC workspace, the AWS API will return an error if you attempt to destroy the VPC first.

Now move on to the VPC workspace. Once again, respond to the confirmation prompt with yes.

$ cd ../learn-terraform-data-sources-vpc

$ cd ../learn-terraform-data-sources-vpc

Destroy this infrastructure as well.

$ terraform destroy -var aws_region=us-west-1
## ...

Plan: 0 to add, 0 to change, 34 to destroy.

Changes to Outputs:
  - app_security_group_ids = [
      - "sg-0214d055921c25c8e",
    ] -> null
  - aws_region             = "us-west-2" -> null
  - lb_security_group_ids  = [
      - "sg-03f34e1dd93483bd9",
    ] -> null
  - private_subnet_ids     = [
      - "subnet-07b77ef2e9c386a17",
      - "subnet-098f226b620943eac",
    ] -> null
  - public_subnet_ids      = [
      - "subnet-034fc6327aeee353f",
      - "subnet-0a9a7558a4eaa4640",
    ] -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

  ## ...

Destroy complete! Resources: 34 destroyed.

$ terraform destroy -var aws_region=us-west-1
## ...

Plan: 0 to add, 0 to change, 34 to destroy.

Changes to Outputs:
  - app_security_group_ids = [
      - "sg-0214d055921c25c8e",
    ] -> null
  - aws_region             = "us-west-2" -> null
  - lb_security_group_ids  = [
      - "sg-03f34e1dd93483bd9",
    ] -> null
  - private_subnet_ids     = [
      - "subnet-07b77ef2e9c386a17",
      - "subnet-098f226b620943eac",
    ] -> null
  - public_subnet_ids      = [
      - "subnet-034fc6327aeee353f",
      - "subnet-0a9a7558a4eaa4640",
    ] -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

  ## ...

Destroy complete! Resources: 34 destroyed.

»Next steps

Now that you have used Terraform data sources, check out the following resources for more information.

Read the Terraform Data Sources documentation.
Connect Terraform Cloud Workspaces with run triggers, and use outputs from one workspace to configure another workspace.
Inject secrets into Terraform using the Vault provider.

Infrastructure

Security

Applications

Networking

cloud