Auto-unseal using Transit Secrets Engine

When a Vault server is started, it starts in a sealed state and it does not know how to decrypt data. Before any operation can be performed on the Vault, it must be unsealed. Unsealing is the process of constructing the master key necessary to decrypt the data encryption key.

»Challenge

Vault unseal operation requires a quorum of existing unseal keys split by Shamir's Secret sharing algorithm. This is done so that the "keys to the kingdom" won't fall into one person's hand.

Unseal with Shamir's Secret Sharing

However, this process is manual and can become painful when you have many Vault clusters as there are now many different key holders with many different keys.

»Solution

Vault supports opt-in automatic unsealing via transit secrets engine. This feature enables operators to delegate the unsealing process to a trusted Vault environment to ease operations.

Unseal with Transit

This tutorial demonstrates how to auto-unseal a Vault with Transit secrets engine.

To auto-unseal your Vault with cloud provider's key, refer to the following tutorials:

»Prerequisites

To perform the tasks described in this tutorial, you need to have a Vault 1.1 or later. Refer to the Getting Started tutorial to install Vault. Make sure that your Vault server has been initialized and unsealed.

An interactive tutorial is also available below if you do not have a Vault environment to perform the steps described in this tutorial. Click the Show Terminal button to start.

»Scenario Introduction

For the purpose of demonstration, you are going to run two instances of Vault as described in the following diagram:

Unseal with Transit

In reality, the Vault 1 and Vault 2 are two separate stand-alone Vault clusters where one protecting another's master key. Nonetheless, the steps described in this tutorial directly applies to your clustered environment. The main difference would be the location (address) of Vault 1 and Vault 2.

In this scenario, Vault 1 (http://127.0.0.1:8200) is the encryption service provider, and its transit key protects the Vault 2 server's master key.

Scenario Overview

»Step 1: Configure Auto-unseal Key Provider (Vault 1)

The first step is to enable and configure the transit secrets engine on Vault 1.

Enable an audit device if it hasn't been enabled already so that you can examine the audit log later in Step 3.
```
$ vault audit enable file file_path=audit.log
```
Execute the following command to enable the transit secrets engine.
```
$ vault secrets enable transit
```
Execute the following command to create an encryption key named, "autounseal".
```
# Create a key named 'autounseal'
$ vault write -f transit/keys/autounseal
```
NOTE: To learn more about the transit secrets engine, refer to the Encryption as a Service: Transit Secrets Engine.

Create a policy file named autounseal.hcl which permits update against transit/encrypt/autounseal and transit/decrypt/autounseal paths.

$ tee autounseal.hcl <<EOF
path "transit/encrypt/autounseal" {
   capabilities = [ "update" ]
}

path "transit/decrypt/autounseal" {
   capabilities = [ "update" ]
}
EOF

Create a policy named autounseal.
```
$ vault policy write autounseal autounseal.hcl
```
Create a policy to permit update against transit/encrypt/<key_name> and transit/decrypt/<key_name> where the <key_name> is the name of the encryption key you created in the previous step.

Create a client token with autounseal policy attached and response wrap it with TTL of 120 seconds.

$ vault token create -policy="autounseal" -wrap-ttl=120

Key                              Value
---                              -----
wrapping_token:                  s.qg6HS6WNOYQ2SjnyyMqdSbZ0
wrapping_accessor:               4mHRpfx5oZcHhWAg6VeTR4or
wrapping_token_ttl:              2m
wrapping_token_creation_time:    2019-04-15 15:28:30.598376 -0700 PDT
wrapping_token_creation_path:    auth/token/create
wrapped_accessor:                GlZflsyuG891yw3PYC4b7GrM

Pass the generated wrapping_token value to Vault 2.

»Step 2: Configure Auto-unseal (Vault 2)

Now, start a second Vault instance which listens to port 8100. The server configuration file should define a seal stanza with parameters properly set based on the tasks you performed in Step 1.

Scenario Overview

Execute the following command to unwrap the secrets passed from Vault 1.

$ VAULT_TOKEN=<wrapping_token> vault unwrap

For example, if the wrapping token value was "s.qg6HS6WNOYQ2SjnyyMqdSbZ0", the command would look as follow:

$ VAULT_TOKEN="s.AFqDxN5jdiDQDNuodJxsC6dm" vault unwrap

Key                  Value
---                  -----
token                s.SJj086AW7ZaobvRmNoSjhVaj
token_accessor       6lUHdlwRZfyRnHtNiz8ZB1Jx
token_duration       768h
token_renewable      true
token_policies       ["autounseal" "default"]
identity_policies    []
policies             ["autounseal" "default"]

The revealed token is the client token Vault 2 will use to connect with Vault 1.

Set VAULT_TOKEN environment variable whose value is the client token you just unwrapped.
Example:
```
$ export VAULT_TOKEN="s.SJj086AW7ZaobvRmNoSjhVaj"
```

Create a server configuration file (config-autounseal.hcl) to start a second Vault instance (Vault 2).

disable_mlock = true
ui=true

storage "file" {
  path = "/vault-2/data"
}

listener "tcp" {
  address     = "127.0.0.1:8100"
  tls_disable = 1
}

seal "transit" {
  address = "http://127.0.0.1:8200"
  disable_renewal = "false"
  key_name = "autounseal"
  mount_path = "transit/"
  tls_skip_verify = "true"
}

Notice that the address points to the Vault server listening to port 8200 (Vault 1). The key_name and mount_path match to what you created in Step 1.

NOTE: The seal stanza does not set the token value since it's already set as VAULT_TOKEN environment variable.

Start the vault server with the configuration file.
```
$ vault server -config=config-autounseal.hcl
```
Open another terminal and initialize your second Vault server (Vault 2).
```
$ VAULT_ADDR=http://127.0.0.1:8100 vault operator init -recovery-shares=1 \
        -recovery-threshold=1 > recovery-key.txt
```
By passing the VAULT_ADDR, the subsequent command gets executed against the second Vault server (http://127.0.0.1:8100).
Notice that you are setting the number of recovery key and recovery threshold because there is no unseal keys with auto-unseal. Vault 2's master key is now protected by the transit secrets engine of Vault 1. Recovery keys are used for high-privilege operations such as root token generation. Recovery keys are also used to make Vault operable if Vault has been manually sealed through the "vault operator seal" command.
Check the Vault 2 server status. It is now successfully initialized and unsealed.
```
$ VAULT_ADDR=http://127.0.0.1:8100 vault status

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    1
Threshold                1
# ...snip...
```
Notice that it shows Total Recovery Shares instead of Total Shares. The transit secrets engine is solely responsible for protecting the master key of Vault 2. There are some operations that still requires Shamir's keys (e.g. regenerate a root token). Therefore, Vault 2 server requires recovery keys although auto-unseal has been enabled.

»Step 3: Verify Auto-Unseal

To verify that Vault 2 gets automatically unseal, press Ctrl + C to stop the Vault 2 server where it is running.

...snip...
[INFO]  core.cluster-listener: rpc listeners successfully shut down
[INFO]  core: cluster listeners successfully shut down
[INFO]  core: vault is sealed

Note that Vault 2 is now sealed.

Press the upper-arrow key, and execute the vault server -config=config-autounseal.hcl command again to start Vault 2 and see what happens.

$ vault server -config=config-autounseal.hcl

==> Vault server configuration:

                Seal Type: transit
          Transit Address: http://127.0.0.1:8200
        Transit Key Name: autounseal
      Transit Mount Path: transit/
                      Cgo: disabled
              Listener 1: tcp (addr: "0.0.0.0:8100", cluster address: "0.0.0.0:8101", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
                Log Level: info
                    Mlock: supported: true, enabled: false
                  Storage: file
                  Version: Vault v1.1.0
              Version Sha: 36aa8c8dd1936e10ebd7a4c1d412ae0e6f7900bd

==> Vault server started! Log data will stream in below:

[WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
[INFO]  core: stored unseal keys supported, attempting fetch
[INFO]  core: vault is unsealed
# ...snip...

Notice that the Vault server is already unsealed. The Transit Address is set to your Vault 1 which is listening to port 8200 (http://127.0.0.1:8200).

Check the Vault 2 server status.

$ VAULT_ADDR=http://127.0.0.1:8100 vault status

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    1
Threshold                1
# ...snip...

Now, examine the audit log in Vault 1.

$ tail -f audit.log | jq

# ...snip...
"request": {
  "id": "a46719eb-eee0-92a4-2da6-6c7de77fd410",
  "operation": "update",
  "client_token": "hmac-sha256:ce8613487054dadb36a9d08da1f5a4bbee2fbfc1ef1ec5ebdeec696df7823e69",
  "client_token_accessor": "hmac-sha256:f3b6cb798605835e8a00bafa9e0e16fc0534b8923b31e499f2c8e694f6b69158",
  "namespace": {
    "id": "root",
    "path": ""
  },
  "path": "transit/decrypt/autounseal",
    # ...snip...
  "remote_address": "127.0.0.1",
  "wrap_ttl": 0,
  "headers": {}
},
# ...snip...
}

You should see an update request against the transit/decrypt/autounseal path. The remote_address is 127.0.0.1 in this example since Vault 1 and Vault 2 are both running locally. If the Vault 2 is running on a different host, the audit log will show the IP address of the Vault 2 host.

Infrastructure

Security

Networking

Applications