Codify Management of Vault Using Terraform

»Challenge

A manual system administration can become a challenge as the scale of infrastructure increases. Often, an organization must manage multiple Vault environments (development, testing, staging, production, etc.). Keeping up with the increasing management demand soon becomes a challenge without some sort of automation.

»Solution

One of the pillars behind the Tao of Hashicorp is automation through codification.

HashiCorp Terraform is an infrastructure as code which enables the operation team to codify the Vault configuration tasks such as the creation of policies. Automation through codification allows operators to increase their productivity, move quicker, promote repeatable processes, and reduce human error.

This tutorial demonstrates techniques for creating Vault policies and configurations using Terraform Vault Provider.

»Prerequisites

• Terraform installed
• A Vault environment to connect

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Start Interactive Lab

»Scenario introduction

Vault administrators must manage multiple Vault environments. The test servers get destroyed at the end of each test cycle and a new set of servers must be provisioned for the next test cycle. To automate the Vault server configuration, you are going to use Terraform to provision the following Vault resources.

»Examine the Terraform files

1. Clone or download the demo assets from the hashicorp/learn-vault-codify GitHub repository to perform the steps described in this tutorial.

   $ git clone https://github.com/hashicorp/learn-vault-codify.git
   

   $ git clone https://github.com/hashicorp/learn-vault-codify.git
   

   Copy

   This repository contains supporting content for all of the Vault learn guides. The content specific to this tutorial can be found within a sub-directory.

2. Change the working directory to learn-vault-codify/oss.

   $ cd learn-vault-codify/oss
   

   $ cd learn-vault-codify/oss
   

   Copy

   The directory contains Terraform files to configure Vault.

   $ tree
   .
   ├── auth.tf
   ├── main.tf
   ├── policies
   │   ├── admin-policy.hcl
   │   └── eaas-client-policy.hcl
   ├── policies.tf
   └── secrets.tf
   

   $ tree
   .
   ├── auth.tf
   ├── main.tf
   ├── policies
   │   ├── admin-policy.hcl
   │   └── eaas-client-policy.hcl
   ├── policies.tf
   └── secrets.tf
   

   Copy

3. Open the main.tf file in your preferred text editor to examine its content.

   provider "vault" {
     # It is strongly recommended to configure this provider through the
     # environment variables:
     #    - VAULT_ADDR
     #    - VAULT_TOKEN
     #    - VAULT_CACERT
     #    - VAULT_CAPATH
     #    - etc.
   }
   

   main.tf

   Copy

   123456789provider "vault" {
     # It is strongly recommended to configure this provider through the
     # environment variables:
     #    - VAULT_ADDR
     #    - VAULT_TOKEN
     #    - VAULT_CACERT
     #    - VAULT_CAPATH
     #    - etc.
   }
   

   Within the file is a vault provider block. You can provide the server connection details inside this block (Vault server address, client tokens, etc.); however, it is strongly recommended to configure those target server specific information using environment variables. And that's what you are going to do in this tutorial.

4. Open the policies.tf file and examine the vault_policy resources.

   # Create admin policy in the root namespace
   resource "vault_policy" "admin_policy" {
     name   = "admins"
     policy = file("policies/admin-policy.hcl")
   }
   
   # Create 'training' policy
   resource "vault_policy" "eaas-client" {
     name   = "eaas-client"
     policy = file("policies/eaas-client-policy.hcl")
   }
   

   policies.tf

   Copy

   1 2 3 4 5 6 7 8 9 1011# Create admin policy in the root namespace
   resource "vault_policy" "admin_policy" {
     name   = "admins"
     policy = file("policies/admin-policy.hcl")
   }
   
   # Create 'training' policy
   resource "vault_policy" "eaas-client" {
     name   = "eaas-client"
     policy = file("policies/eaas-client-policy.hcl")
   }
   

5. Open the auth.tf file and note that it enables userpass auth method and creates a user, "student" with admins and eaas-client policies attached. The password is set to "changeme".

   resource "vault_auth_backend" "userpass" {
     type = "userpass"
   }
   
   # Create a user, 'student'
   resource "vault_generic_endpoint" "student" {
     depends_on           = [vault_auth_backend.userpass]
     path                 = "auth/userpass/users/student"
     ignore_absent_fields = true
   
     data_json = <<EOT
   {
     "policies": ["admins", "eaas-client"],
     "password": "changeme"
   }
   EOT
   }
   

   auth.tf

   Copy

   1 2 3 4 5 6 7 8 9 1011121314151617resource "vault_auth_backend" "userpass" {
     type = "userpass"
   }
   
   # Create a user, 'student'
   resource "vault_generic_endpoint" "student" {
     depends_on           = [vault_auth_backend.userpass]
     path                 = "auth/userpass/users/student"
     ignore_absent_fields = true
   
     data_json = <<EOT
   {
     "policies": ["admins", "eaas-client"],
     "password": "changeme"
   }
   EOT
   }
   

6. Open the secrets.tf file which enables kv-v2 secrets engine (line 2-5).

   # Enable K/V v2 secrets engine at 'kv-v2'
   resource "vault_mount" "kv-v2" {
     path = "kv-v2"
     type = "kv-v2"
   }
   
   # Enable Transit secrets engine at 'transit'
   resource "vault_mount" "transit" {
     path = "transit"
     type = "transit"
   }
   
   # Creating an encryption key named 'payment'
   resource "vault_transit_secret_backend_key" "key" {
     depends_on = [vault_mount.transit]
     backend    = "transit"
     name       = "payment"
     deletion_allowed = true
   }
   

   secrets.tf

   Copy

   1 2 3 4 5 6 7 8 9 10111213141516171819# Enable K/V v2 secrets engine at 'kv-v2'
   resource "vault_mount" "kv-v2" {
     path = "kv-v2"
     type = "kv-v2"
   }
   
   # Enable Transit secrets engine at 'transit'
   resource "vault_mount" "transit" {
     path = "transit"
     type = "transit"
   }
   
   # Creating an encryption key named 'payment'
   resource "vault_transit_secret_backend_key" "key" {
     depends_on = [vault_mount.transit]
     backend    = "transit"
     name       = "payment"
     deletion_allowed = true
   }
   

   It enables transit secrets engine at transit (line 8-10), and create an encryption key named, "payment" (line 14-19).

»Run Terraform to configure Vault

1. Optional: Start a Vault server in development mode with root as the root token if you don't have one running already.

   $ vault server -dev -dev-root-token-id root
   

   $ vault server -dev -dev-root-token-id root
   

   Copy

2. Set the client token in the VAULT_TOKEN environment variable.

   $ export VAULT_TOKEN="root"
   

   $ export VAULT_TOKEN="root"
   

   Copy

   If the token is different, be sure to set it to the correct token value that has permissions to create policies, enable secrets engines, and enable auth methods.

3. Set the target Vault server address in the VAULT_ADDR environment variable if it's not done so already.

   $ export VAULT_ADDR="http://127.0.0.1:8200"
   

   $ export VAULT_ADDR="http://127.0.0.1:8200"
   

   Copy

   If you are connecting to a remote Vault server, be sure to set the VAULT_ADDR value to the correct target Vault API address.

4. Initialize Terraform to pull Vault provider plugin.

   $ terraform init
   
   Initializing the backend...
   ...snip...
   Terraform has been successfully initialized!
   

   $ terraform init
   
   Initializing the backend...
   ...snip...
   Terraform has been successfully initialized!
   

   Copy

   This will download the Vault plugin.

5. Execute the apply command to configure Vault.

   $ terraform apply
   

   $ terraform apply
   

   Copy

   This will display the actions to be performed by Terraform.

   When prompted, enter yes to accept the plan and proceed with Vault configuration.

   Plan: 7 to add, 0 to change, 0 to destroy.
   
   Do you want to perform these actions?
     Terraform will perform the actions described above.
     Only 'yes' will be accepted to approve.
   
     Enter a value: yes
   

   Plan: 7 to add, 0 to change, 0 to destroy.
   
   Do you want to perform these actions?
     Terraform will perform the actions described above.
     Only 'yes' will be accepted to approve.
   
     Enter a value: yes
   

   Once completed, the output similar to the following displays.

   Apply complete! Resources: 7 added, 0 changed, 0 destroyed.
   

   Apply complete! Resources: 7 added, 0 changed, 0 destroyed.
   

»Verify the configuration

»Clean up

When you are done exploring, you can undo the configuration made by Terraform.

1. Make sure that VAULT_TOKEN and VAULT_ADDR environment variables are set.

   $ echo $VAULT_ADDR; echo $VAULT_TOKEN
   

   $ echo $VAULT_ADDR; echo $VAULT_TOKEN
   

   Copy

2. Destroy the Vault resources created by Terraform.

   $ terraform destroy -auto-approve
   
   ...snip...
   
   Destroy complete! Resources: 7 destroyed.
   

   $ terraform destroy -auto-approve
   
   ...snip...
   
   Destroy complete! Resources: 7 destroyed.
   

   Copy

3. Remove the terraform state files.

   $ rm *.tfstate.*
   

   $ rm *.tfstate.*
   

   Copy

4. Unset the VAULT_TOKEN and VAULT_ADDR environment variables.

   $ unset VAULT_TOKEN VAULT_ADDR
   

   $ unset VAULT_TOKEN VAULT_ADDR
   

   Copy

»Next steps

Treat your Terraform files like any other code and manage them through a version control system such as GitHub. You may integrate it with your favorite CI/CD tool (Jenkins, Travis CI, Circle CI, etc.), always review and test the configuration.

Travis CI example:

You can test your Terraform files against a development server that runs locally, or use a Docker image of Vault.

sudo: false

env:
  - VAULT_ADDR=<target_vault_address>  VAULT_TOKEN=<token>

before_install:
  - scripts/install.sh

before_script:
  - vault server -dev -dev-root-token-id="$VAULT_TOKEN"

script:
  - terraform init
  - terraform apply -auto-approve

sudo: false

env:
  - VAULT_ADDR=<target_vault_address>  VAULT_TOKEN=<token>

before_install:
  - scripts/install.sh

before_script:
  - vault server -dev -dev-root-token-id="$VAULT_TOKEN"

script:
  - terraform init
  - terraform apply -auto-approve

»Help and Reference

• Terraform Vault Provider documentation page
• Terraform Provider GitHub repository
• Learn Terraform