Network Infrastructure Automation with Consul-Terraform-Sync Intro

HashiCorp's Network Infrastructure Automation Integration Program allows your network administrators to build integrations that automatically apply network and security infrastructure changes reacting to changes in the Consul service catalog.

Network Infrastructure Automation is carried out by Consul-Terraform-Sync, a multi-platform tool, that is able to connect to Consul catalog and monitor changes in the services' state and health. The tool leverages Terraform as the underlying automation tool, and utilizes the Terraform provider ecosystem to drive relevant change to your network infrastructure.

Consul-Terraform-Sync can be configured to execute one or more automation tasks that use variables based on the content of the Consul service catalog. Each task consists of a runbook automation written as a compatible Terraform module using resources and data sources for the underlying network infrastructure.

In this tutorial you will learn how to install Consul-Terraform-Sync on a node and to configure it to communicate with your Consul datacenter, to react to service changes, and execute an example task.

»Prerequisites

• Consul v1.11.3+ installed locally. Visit the Install Consul tutorial for detailed steps on how to install Consul.

• A running Consul datacenter or a local Consul agent. Follow the Deployment Guide for guidance on installing Consul.

»Install Consul-Terraform-Sync

You can install Consul-Terraform-Sync manually either by using a pre-compiled binary or by building it yourself from source.

$ apt-get install curl gnupg lsb-release

$ apt-get install curl gnupg lsb-release

$ curl --fail --silent --show-error --location https://apt.releases.hashicorp.com/gpg | \
      gpg --dearmor | \
      sudo dd of=/usr/share/keyrings/hashicorp-archive-keyring.gpg

$ curl --fail --silent --show-error --location https://apt.releases.hashicorp.com/gpg | \
      gpg --dearmor | \
      sudo dd of=/usr/share/keyrings/hashicorp-archive-keyring.gpg

$ echo "deb [arch=amd64 signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
 sudo tee -a /etc/apt/sources.list.d/hashicorp.list

$ echo "deb [arch=amd64 signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
 sudo tee -a /etc/apt/sources.list.d/hashicorp.list

$ sudo apt-get update

$ sudo apt-get update

$ sudo apt-cache policy consul-terraform-sync

consul-terraform-sync:
  Installed: (none)
  Candidate: 0.6.0
  Version table:
     0.6.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.5.2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.5.1 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.5.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.3 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.1 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.0-2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.3.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.2.1 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.2.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.1.3 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.1.2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages

$ sudo apt-cache policy consul-terraform-sync

consul-terraform-sync:
  Installed: (none)
  Candidate: 0.6.0
  Version table:
     0.6.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.5.2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.5.1 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.5.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.3 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.1 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.0-2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.4.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.3.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.2.1 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.2.0 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.1.3 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages
     0.1.2 500
        500 https://apt.releases.hashicorp.com bullseye/main amd64 Packages

$ sudo apt-get install consul-terraform-sync=0.6.0

$ sudo apt-get install consul-terraform-sync=0.6.0

$ sudo apt-get install consul-terraform-sync

$ sudo apt-get install consul-terraform-sync

Once installed, verify the installation works by prompting the help option.

$ consul-terraform-sync -h
Usage of consul-terraform-sync:
  -config-dir
        A directory to load files for configuring Sync. Configuration files require an .hcl or .json file extention in order to specify their format. This option can be specified multiple times to load different directories.
  -config-file
        A file to load for configuring Sync. Configuration file requires an .hcl or .json extension in order to specify their format. This option can be specified multiple times to load different configuration files.
  -inspect false
        Run Sync in Inspect mode to print the proposed state changes for all tasks, and then exits. No changes are applied in this mode.
  -inspect-task
        Run Sync in Inspect mode to print the proposed state changes for the task, and then exits. No changes are applied in this mode.
  -once false
        Render templates and run tasks once. Does not run the process as a daemon and disables buffer periods.
  -version false
        Print the version of this daemon.

$ consul-terraform-sync -h
Usage of consul-terraform-sync:
  -config-dir
        A directory to load files for configuring Sync. Configuration files require an .hcl or .json file extention in order to specify their format. This option can be specified multiple times to load different directories.
  -config-file
        A file to load for configuring Sync. Configuration file requires an .hcl or .json extension in order to specify their format. This option can be specified multiple times to load different configuration files.
  -inspect false
        Run Sync in Inspect mode to print the proposed state changes for all tasks, and then exits. No changes are applied in this mode.
  -inspect-task
        Run Sync in Inspect mode to print the proposed state changes for the task, and then exits. No changes are applied in this mode.
  -once false
        Render templates and run tasks once. Does not run the process as a daemon and disables buffer periods.
  -version false
        Print the version of this daemon.

»Install Envoy (optional)

Consul has first class integration with the proxy Envoy. In this tutorial, you can choose to use Envoy and deploy an instance of Envoy with each service. To install Envoy, use the CLI, func-e.

Start by downloading the func-e CLI. The func-e CLI helps you with installing and updading Envoy in the future.

$ curl https://func-e.io/install.sh | bash -s -- -b /usr/local/bin

$ curl https://func-e.io/install.sh | bash -s -- -b /usr/local/bin

If your machine uses an ARM64 chip (i.e., Apple M1), then you will need to use the command below and use an environment variable before proceeding to the next step. This step is needed due to the Envoy project lacking an arm64 release binary. The amd64 binary will work for Mac with ARM chips thanks to Rosetta.

$ export FUNC_E_PLATFORM=darwin/amd64

$ export FUNC_E_PLATFORM=darwin/amd64

Next, issue the following command to download version 1.20.1 of Envoy.

$ func-e use 1.20.1

$ func-e use 1.20.1

Copy the Envoy binary to a location in your $PATH. This enables Consul to automatically start Envoy without specifying the binary location.

$ sudo cp ~/.func-e/versions/1.20.1/bin/envoy /usr/local/bin/

$ sudo cp ~/.func-e/versions/1.20.1/bin/envoy /usr/local/bin/

Verify that Envoy is in your $PATH by running the following command.

$ envoy --version
envoy  version: bef18019d8fc33a4ed6aca3679aff2100241ac5e/1.20.1/Modified/RELEASE/BoringSSL

$ envoy --version
envoy  version: bef18019d8fc33a4ed6aca3679aff2100241ac5e/1.20.1/Modified/RELEASE/BoringSSL

»Start Consul

In this tutorial you will use a local Consul server in dev mode. You may use an actual deployment of Consul if you have one available.

Issue the following command in a local terminal to start Consul.

$ consul agent -dev

$ consul agent -dev

»Start the services

Download the counting service and the dashboard service from https://github.com/hashicorp/demo-consul-101/releases. Each service is available for different operating systems. Use the filename to identify the correct operating system. Example: counting-service_linux_arm64.zip for Linux.

Next, unzip each downloaded file and extract the binary to a target destination.

Open up two new tabs in your local terminal. In each tab, go ahead and start each service. Make sure you use the correct filename when starting the services. The commands below are for darwin amd64.

$ PORT=9003 ./counting-service_darwin_amd64 

$ PORT=9003 ./counting-service_darwin_amd64 

$ PORT=9002 COUNTING_SERVICE_URL="http://localhost:9003"  ./dashboard-service_darwin_amd64

$ PORT=9002 COUNTING_SERVICE_URL="http://localhost:9003"  ./dashboard-service_darwin_amd64

»Register the services

Open up another tab in your terminal and create two files, one named web.hcl and another namedapi.hcl.

$ touch web.hcl && api.hcl

$ touch web.hcl && api.hcl

You can find the content for each file in the two tabs below.

service {
  name = "web"
  id = "web1"
  port = 9002

  connect {
    sidecar_service {
      proxy {
        upstreams = [
          {
            destination_name = "api"
            local_bind_port  = 9003
          }
        ]
      }
    }
  }
}

 check {
     id =  "web",
     name = "The dashboard service",
     service_id = "web",
     tcp  = "localhost:9002",
     interval = "1s",
     timeout = "1s"
}

service {
  name = "web"
  id = "web1"
  port = 9002

  connect {
    sidecar_service {
      proxy {
        upstreams = [
          {
            destination_name = "api"
            local_bind_port  = 9003
          }
        ]
      }
    }
  }
}

 check {
     id =  "web",
     name = "The dashboard service",
     service_id = "web",
     tcp  = "localhost:9002",
     interval = "1s",
     timeout = "1s"
}

service {
  name = "api"
  id = "api1"
  port = 9003

  connect {
    sidecar_service {}
  }
}

 check {
     id =  "api",
     name = "The counting service",
     service_id = "api",
     tcp  = "localhost:9003",
     interval = "1s",
     timeout = "1s"
}

service {
  name = "api"
  id = "api1"
  port = 9003

  connect {
    sidecar_service {}
  }
}

 check {
     id =  "api",
     name = "The counting service",
     service_id = "api",
     tcp  = "localhost:9003",
     interval = "1s",
     timeout = "1s"
}

$ consul services register web.hcl && consul services register api.hcl

$ consul services register web.hcl && consul services register api.hcl

$ consul connect envoy -admin-bind localhost:19001 -sidecar-for api1 

$ consul connect envoy -admin-bind localhost:19001 -sidecar-for api1 

$ consul connect envoy -admin-bind localhost:19002 -sidecar-for web1

$ consul connect envoy -admin-bind localhost:19002 -sidecar-for web1

»Configure Consul-Terraform-Sync

The Consul-Terraform-Sync daemon is configured using configuration files and supports HashiCorp Configuration Language (HCL) and JSON file formats.

A configuration file for Consul-Terraform-Sync is composed by several blocks, this section will guide you through the different blocks and provide you with example values. For the full list of available options check the documentation.

Start by creating a file named cts-config.hcl

$ touch cts-config.hcl

$ touch cts-config.hcl

Paste the following content into the cts-config.hcl file.

log_level   = "INFO"
working_dir = "sync-tasks"
port        = 8558

syslog {}

buffer_period {
  enabled = true
  min     = "5s"
  max     = "20s"
}

consul {
  address = "localhost:8500"
}

driver "terraform" {
  # version = "0.14.0"
  # path = ""
  log         = false
  persist_log = false

  backend "consul" {
    gzip = true
  }
}

task {
 name        = "learn-cts-example"
 description = "Example task with two services"
 module      = "findkim/print/cts"
 version     = "0.1.0"
 condition "services" {
  names = ["web", "api"]
 }
}

log_level   = "INFO"
working_dir = "sync-tasks"
port        = 8558

syslog {}

buffer_period {
  enabled = true
  min     = "5s"
  max     = "20s"
}

consul {
  address = "localhost:8500"
}

driver "terraform" {
  # version = "0.14.0"
  # path = ""
  log         = false
  persist_log = false

  backend "consul" {
    gzip = true
  }
}

task {
 name        = "learn-cts-example"
 description = "Example task with two services"
 module      = "findkim/print/cts"
 version     = "0.1.0"
 condition "services" {
  names = ["web", "api"]
 }
}

»Global configs

Top level options are reserved for configuring the Consul-Terraform-Sync daemon.

log_level   = "INFO"
working_dir = "sync-tasks"
port        = 8558

syslog {}

buffer_period {
  enabled = true
  min     = "5s"
  max     = "20s"
}

log_level   = "INFO"
working_dir = "sync-tasks"
port        = 8558

syslog {}

buffer_period {
  enabled = true
  min     = "5s"
  max     = "20s"
}

In this section you can configure the log level to use for Consul-Terraform-Sync logging as well as the port used by the demon to offer the API interface.

Other notable sections are:

• buffer_period: configures the default buffer period for all tasks to mitigate the affects of flapping services to downstream network devices.

• syslog: specifies the syslog server for logging. This section can be useful when Consul-Terraform-Sync is configured as a daemon, for example in Linux using systemd.

»Consul block

The consul block is used to configure Consul-Terraform-Sync connection with a Consul agent to perform queries to the Consul catalog and Consul KV pertaining to task execution.

consul {
  address = "localhost:8500"
}

consul {
  address = "localhost:8500"
}

You can use this block to configure connection parameters, such as the Consul address, and security parameters, like TLS certificates or ACL token, to secure the connection with Consul and make sure you provide the Consul-Terraform-Sync daemon with the right privileges to perform the required operations.

»Driver “terraform” block

The driver block configures the subprocess used by Consul-Terraform-Sync to propagate infrastructure change. The Terraform driver is a required configuration for Consul-Terraform-Sync to relay provider discovery and installation information to Terraform.

driver "terraform" {
  # version = "0.14.0"
  # path = ""
  log         = false
  persist_log = false

  backend "consul" {
    gzip = true
  }
}

driver "terraform" {
  # version = "0.14.0"
  # path = ""
  log         = false
  persist_log = false

  backend "consul" {
    gzip = true
  }
}

Using this section you can define the version of Terraform you want to use, in case you want to define strict requirements for it, the path where to find/install the Terraform binary or if you want the logs for Terraform to be included in the Consul-Terraform-Sync logs or persisted on disk. Finally, you can also define the working directory for Terraform to operate in.

»Task block

A task is executed when any change of information for services the task is configured for is detected from the Consul catalog. Execution could include one or more changes to service values, like IP address, added or removed service instance, or tags.

task {
 name        = "learn-cts-example"
 description = "Example task with two services"
 module      = "findkim/print/cts"
 version     = "0.1.0"
 condition "services" {
  names = ["web", "api"]
 }
}

task {
 name        = "learn-cts-example"
 description = "Example task with two services"
 module      = "findkim/print/cts"
 version     = "0.1.0"
 condition "services" {
  names = ["web", "api"]
 }
}

You can check the full list of values that would cause a task to run in the Task Execution documentation.

Consul-Terraform-Sync will attempt to execute each task once upon startup to synchronize infrastructure with the current state of Consul. The daemon will stop and exit if any error occurs while preparing the automation environment or executing a task for the first time.

A task block configures the task to run as automation for the defined services. The services can be explicitly defined in the task's condition block.

You can specify multiple task blocks in case you need to configure multiple tasks.

For example, in the code snippet above, you are defining a task block that will monitor and react to changes on web and api services and run the module defined in the module parameter when any change is detected.

For this tutorial, you will be using the Consul Print Module to configure Consul-Terraform-Sync. This Terraform module creates text files on your local machine containing Consul service information. It is a useful module to use to familiarize yourself with Consul Terraform Sync, without requiring deployed infrastructure and credentials.

»Start Consul-Terraform-Sync

Once the configuration file is created it is possible to start Consul-Terraform-Sync using the consul-terraform-sync binary. The binary requires the configuration to be passed either using the -config-file or -config-dir.

Consul-Terraform-Sync provides different running modes, including some that can be useful to safely test your configuration and the changes that are going to be applied.

The default mode is named daemon mode, in this mode Consul-Terraform-Sync passes through a once-mode phase, in which it will try to run all the tasks once, and then turns into a long running process. During the once-mode phase, the daemon will exit with a non-zero status if it encounters an error. After successfully passing through once-mode phase, errors will be logged and the process is not expected to exit.

You may start Consul-Terraform-Sync as a systemd process. To learn how to configure Consul-Terraform-Sync as a systemd process, checkout the Secure Consul-Terraform-Sync for Production tutorial.

$ consul-terraform-sync start -config-file cts-config.hcl
[INFO]  cli: v0.6.0 (7f0d7ba)
[INFO]  cli: running controller in daemon mode
[INFO]  cli: setting up controller: type=readwrite
[INFO]  ctrl: initializing Consul client and testing connection
[INFO]  driver.terraform: install terraform: install_path=/
[INFO]  driver.terraform: successfully installed terraform
[INFO]  cli: initializing controller
[INFO]  ctrl: initializing driver
[INFO]  ctrl: initializing all tasks
[INFO]  client.terraformcli: Terraform output is muted
[INFO]  driver.terraform: retrieved 0 Terraform handlers for task: task_name=learn-cts-example
[INFO]  ctrl: drivers initialized
[INFO]  ctrl: executing all tasks once through
[INFO]  ctrl: executing task: task_name=learn-cts-example
[INFO]  ctrl: task completed: task_name=learn-cts-example
[INFO]  ctrl: all tasks completed once
[INFO]  api: starting server: port=8558
...

$ consul-terraform-sync start -config-file cts-config.hcl
[INFO]  cli: v0.6.0 (7f0d7ba)
[INFO]  cli: running controller in daemon mode
[INFO]  cli: setting up controller: type=readwrite
[INFO]  ctrl: initializing Consul client and testing connection
[INFO]  driver.terraform: install terraform: install_path=/
[INFO]  driver.terraform: successfully installed terraform
[INFO]  cli: initializing controller
[INFO]  ctrl: initializing driver
[INFO]  ctrl: initializing all tasks
[INFO]  client.terraformcli: Terraform output is muted
[INFO]  driver.terraform: retrieved 0 Terraform handlers for task: task_name=learn-cts-example
[INFO]  ctrl: drivers initialized
[INFO]  ctrl: executing all tasks once through
[INFO]  ctrl: executing task: task_name=learn-cts-example
[INFO]  ctrl: task completed: task_name=learn-cts-example
[INFO]  ctrl: all tasks completed once
[INFO]  api: starting server: port=8558
...

When running in daemon mode Consul-Terraform-Sync will keep running in the foreground and prevent you from performing other operations in the same terminal. Also it will get terminated in case the terminal session is closed. Consider configuring it as a service and to run it at system startup to be able to survive machine reboots.

»Review files created by Consul-Terraform-Sync daemon

After startup, Consul-Terraform-Sync will run Terraform inside the working_directory defined in the configuration. If no directory is defined, it creates a folder, named sync-tasks, in the directory from which the binary is started.

Inside that folder, Terraform will create a workspace for each task defined in the configuration.

$ tree sync-tasks/
sync-tasks/
└── learn-cts-example
    ├── main.tf
    ├── terraform.tfvars
    ├── terraform.tfvars.tmpl
    ├── variables.tf
    └── web.txt

$ tree sync-tasks/
sync-tasks/
└── learn-cts-example
    ├── main.tf
    ├── terraform.tfvars
    ├── terraform.tfvars.tmpl
    ├── variables.tf
    └── web.txt

• main.tf: contains the terraform block, provider blocks, and a module block calling the module configured for the task.

• variables.tf: contains the services input variable which determines module compatibility with Consul-Terraform Sync and optionally the intermediate variables used to dynamically configure providers.

• terraform.tfvars: is where the services input variable is assigned values from the Consul catalog. It is periodically updated to reflect the current state of the configured set of services for the task.

• terraform.tfvars.tmpl: is used to template the information retrieved from Consul catalog into the terraform.tfvars file.

For example, running Consul-Terraform-Sync against a Consul datacenter with a web service matching the task configuration will produce the following terraform.tfvars:

# This file is generated by Consul Terraform Sync.
#
# The HCL blocks, arguments, variables, and values are derived from the
# operator configuration for Sync. Any manual changes to this file
# may not be preserved and could be overwritten by a subsequent update.


services = {
  "web.service-2.dc1" : {
    id              = "web"
    name            = "web"
    address         = "172.19.0.5"
    port            = 9002
    meta            = {}
    tags            = []
    namespace       = null
    status          = "passing"
    node            = "service-2"
    node_id         = "fe3d73af-e099-941c-acfb-93ef5596e01f"
    node_address    = "172.19.0.5"
    node_datacenter = "dc1"
    node_tagged_addresses = {
      lan      = "172.19.0.5"
      lan_ipv4 = "172.19.0.5"
      wan      = "172.19.0.5"
      wan_ipv4 = "172.19.0.5"
    }
    node_meta = {
      consul-network-segment = ""
    }
  }
}

# This file is generated by Consul Terraform Sync.
#
# The HCL blocks, arguments, variables, and values are derived from the
# operator configuration for Sync. Any manual changes to this file
# may not be preserved and could be overwritten by a subsequent update.


services = {
  "web.service-2.dc1" : {
    id              = "web"
    name            = "web"
    address         = "172.19.0.5"
    port            = 9002
    meta            = {}
    tags            = []
    namespace       = null
    status          = "passing"
    node            = "service-2"
    node_id         = "fe3d73af-e099-941c-acfb-93ef5596e01f"
    node_address    = "172.19.0.5"
    node_datacenter = "dc1"
    node_tagged_addresses = {
      lan      = "172.19.0.5"
      lan_ipv4 = "172.19.0.5"
      wan      = "172.19.0.5"
      wan_ipv4 = "172.19.0.5"
    }
    node_meta = {
      consul-network-segment = ""
    }
  }
}

»Review automation results

In this example the module prints the addresses of the matched services into one file for each service matched. In the above example you will have one of those files called web.txt, that refers to the web service, created into the task workspace folder, learn-cts-example in this tutorial.

In a real life scenario your module will react on service changes to trigger Terraform runs and deploy changes to your network infrastructure.

»Clean-up

To stop all services, issue the command Ctrl + C in each tab that you opened up for this tutorial.

»Next steps

In this tutorial you learned the basics of Network Infrastructure Automation with Consul-Terraform-Sync. You got an architectural overview of all the logical blocks involved in the process and what are the advantages of automating Day-2 operations for network administrative tasks. Finally, you tried first hand the Consul-Terraform-Sync binary with an example module that retrieved information from Consul service catalog and printed it on a text file in your working directory.

Specifically you:

• Installed Consul-Terraform-Sync
• Created a working configuration file for the binary
• Started Consul-Terraform-Sync
• Checked the files and data pulled from Consul-Terraform-Sync into the Terraform workspace

In the next tutorial you will learn the different run modes for Consul-Terraform-Sync and how to inspect task status using the REST API, and Terraform state when using Consul as backend for Terraform.

For more information on topics covered in this tutorial, check out the following resources.

• Read more about the Network Infrastructure Automation (NIA) with CTS in the Network Infrastructure Automation documentation
• Configure CTS as a long running service by completing the Secure Consul-Terraform-Sync for Production tutorial
• Learn more about service mesh by completing the Kubernetes or virtual machine tutorials

To learn even more about operating, observing, and monitoring your Consul service mesh, check out the following tutorials and collections.

• Maintenance & Monitoring
• Service Discovery & Health
• Layer 7 Observability with Consul, Prometheus, Grafana, and Kubernetes