Day 1: Deploying Your First Vault Cluster

Auto-unseal using Transit Secrets Engine

When a Vault server is started, it starts in a sealed state and it does not know how to decrypt data. Before any operation can be performed on the Vault, it must be unsealed. Unsealing is the process of constructing the master key necessary to decrypt the data encryption key.

» Challenge

Vault unseal operation requires a quorum of existing unseal keys split by Shamir's Secret sharing algorithm. This is done so that the "keys to the kingdom" won't fall into one person's hand.

Unseal with Shamir's Secret Sharing

However, this process is manual and can become painful when you have many Vault clusters as there are now many different key holders with many different keys.

» Solution

Vault supports opt-in automatic unsealing via transit secrets engine. This feature enables operators to delegate the unsealing process to a trusted Vault environment to ease operations.

Unseal with Transit

This guide demonstrates how to auto-unseal a Vault with Transit secrets engine.

To auto-unseal your Vault with cloud provider's key, refer to the following guides:

» Prerequisites

To perform the tasks described in this guide, you need to have a Vault 1.1 or later. Refer to the Getting Started guide to install Vault. Make sure that your Vault server has been initialized and unsealed.

» Steps

For the purpose of demonstration, you are going to run two instances of Vault as described in the following diagram:

Unseal with Transit

In reality, the Vault 1 and Vault 2 are two separate stand-alone Vault clusters where one protecting another's master key. Nonetheless, the steps described in this guide directly applies to your clustered environment. The main difference would be the location (address) of Vault 1 and Vault 2.

You are going to perform the following steps:

  1. Configure Auto-unseal Key Provider (Vault 1)
  2. Configure Auto-unseal (Vault 2)
  3. Audit the incoming request

» Step 1: Configure Auto-unseal Key Provider (Vault 1)

In this scenario, Vault 1 (http://127.0.0.1:8200) is the encryption service provider, and its transit key protects the Vault 2 server's master key. Therefore, the first step is to enable and configure the transit secrets engine on Vault 1.

Scenario Overview

CLI command / API call using cURL / Web UI

» CLI command

  1. Enable an audit device if it hasn't been enabled already so that you can examine the audit log later in Step 3.

    ```plaintext
    $ vault audit enable file file_path=audit.log
    ```
    
  2. Execute the following command to enable the transit secrets engine and create a key named, "autounseal".

    ```shell
    # Enable the transit secrets engine
    $ vault secrets enable transit
    
    # Create a key named 'autounseal'
    $ vault write -f transit/keys/autounseal
    ```
    
    > **NOTE:** To learn more about the `transit` secrets engine, refer to the
    [Encryption as a Service: Transit Secrets
    Engine](/vault/encryption-as-a-service/eaas-transit).
    

1) Create a autounseal policy which permits update against transit/encrypt/autounseal and transit/decrypt/autounseal paths.

    ```shell
    # Create a policy file
    $ tee autounseal.hcl <<EOF
    path "transit/encrypt/autounseal" {
       capabilities = [ "update" ]
    }

    path "transit/decrypt/autounseal" {
       capabilities = [ "update" ]
    }
    EOF

    # Create an 'autounseal' policy
    $ vault policy write autounseal autounseal.hcl
    ```

    Create a policy to permit `update` against `transit/encrypt/<key_name>` and
    `transit/decrypt/<key_name>` where the `<key_name>` is the name of the
    encryption key you created in the previous step.
  1. Create a client token with autounseal policy attached and response wrap it with TTL of 120 seconds.

    ```plaintext
    $ vault token create -policy="autounseal" -wrap-ttl=120
    
    Key                              Value
    ---                              -----
    wrapping_token:                  s.qg6HS6WNOYQ2SjnyyMqdSbZ0
    wrapping_accessor:               4mHRpfx5oZcHhWAg6VeTR4or
    wrapping_token_ttl:              2m
    wrapping_token_creation_time:    2019-04-15 15:28:30.598376 -0700 PDT
    wrapping_token_creation_path:    auth/token/create
    wrapped_accessor:                GlZflsyuG891yw3PYC4b7GrM
    ```
    
    Pass the generated `wrapping_token` value to **Vault 2**.
    

» API call using cURL

  1. Enable an audit device if it hasn't been enabled already so that you can examine the audit log later in Step 3.

    ```plaintext
    $ curl --header "X-Vault-Token: ..." \
           --request PUT \
           --data '{"type":"file", "options":{"file_path":"audit.log"}}' \
           http://127.0.0.1:8200/v1/sys/audit/file
    ```
    
    Be sure to pass your valid token in the `X-Vault-Token` header.
    
  2. Enable transit secret engine and create a key named, "autounseal".

    # Enable the transit secrets engine
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           --data '{"type":"transit"}' \
           https://127.0.0.1:8200/v1/sys/mounts/transit
    
    # Create a key named 'autounseal'
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           https://127.0.0.1:8200/v1/transit/keys/autounseal
    

    NOTE: To learn more about the transit secrets engine, refer to the Encryption as a Service: Transit Secrets Engine.

  3. Create a autounseal policy which permits update against transit/encrypt/autounseal and transit/decrypt/autounseal paths.

    ```shell
    # Create HTTP request payload
    $ tee payload-policy.json <<EOF
    {
      "policy": "path \"transit/encrypt/autounseal\" {\n   capabilities = [ \"update\" ]\n}\n\npath \"transit/decrypt/autounseal\" {\n   capabilities = [ \"update\" ]\n}\n"
    }
    EOF
    
    # Invoke sys/policies/acl endpoint to create a policy
    # Be sure to pass your client token in the X-Vault-Token header
    $ curl --header "X-Vault-Token: ..." --request PUT \
           --data @payload-policy.json \
           http://127.0.0.1:8200/v1/sys/policies/acl/autounseal
    ```
    
    Create a policy to permit `update` against `transit/encrypt/<key_name>` and
    `transit/decrypt/<key_name>` where the `<key_name>` is the name of the
    encryption key you created in the previous step.
    

1) Create a client token with autounseal policy attached and response wrap it with TTL of 120 seconds.

    ```plaintext
    $ curl --header "X-Vault-Wrap-TTL: 120" \
           --header "X-Vault-Token: ..." \
           --request POST \
           --data '{"policies":["autounseal"]}' \
           http://127.0.0.1:8200/v1/auth/token/create | jq
    {
       ...
       "wrap_info": {
         "token": "s.ckhHksIpEl92oeKx6gbVxLJR",
         "accessor": "Ee7s68oZB7QBNwV71LelFpql",
         "ttl": 120,
         "creation_time": "2019-04-15T15:49:57.297275-07:00",
         "creation_path": "auth/token/create",
         "wrapped_accessor": "wBIIlKs2uADdIYRusHi9kug1"
       },
       ...
    }
    ```

    Pass the generated `token` value to **Vault 2**.

» Web UI

Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.

  1. Select Enable new engine.

  2. Select the Transit radio button and click Next. Enable new engine

  3. Click Enable Engine.

  4. Select Create encryption key and enter autounseal in the Name field.

  5. Click Create encryption key to complete.

  6. Click the Policies tab, and then select Create ACL policy.

  7. Enter autounseal in the Name field, and then enter the following policy in the Policy text field.

    ```plaintext
    path "transit/encrypt/autounseal" {
       capabilities = [ "update" ]
    }
    
    path "transit/decrypt/autounseal" {
       capabilities = [ "update" ]
    }
    ```
    
    Create a policy to permit `update` against `transit/encrypt/<key_name>` and
    `transit/decrypt/<key_name>` where the `<key_name>` is the name of the
    encryption key you created in the previous step.
    
  8. Click Create Policy to complete.

  9. Click the Vault CLI shell icon (>_) to open a command shell. Execute vault write auth/token/create policies=autounseal in the CLI shell to create a new token.

    ![Create a token](/assets/images/vault-autounseal-14.png)
    
    Pass the generated `token` value to **Vault 2**.
    
  10. Click the icon (>_) again to hide the shell.

» Step 2: Configure Auto-unseal (Vault 2)

Now, start a second Vault instance which listens to port 8100. The server configuration file should define a seal stanza with parameters properly set based on the tasks you performed in Step 1.

Scenario Overview

  1. Execute the following command to unwrap the secrets passed from Vault 1.

    $ VAULT_TOKEN=<wrapping_token> vault unwrap
    

    For example, if the wrapping token value was "s.qg6HS6WNOYQ2SjnyyMqdSbZ0", the command would look as follow:

    $ VAULT_TOKEN="s.AFqDxN5jdiDQDNuodJxsC6dm" vault unwrap
    
    Key                  Value
    ---                  -----
    token                s.SJj086AW7ZaobvRmNoSjhVaj
    token_accessor       6lUHdlwRZfyRnHtNiz8ZB1Jx
    token_duration       768h
    token_renewable      true
    token_policies       ["autounseal" "default"]
    identity_policies    []
    policies             ["autounseal" "default"]
    

    The revealed token is the client token Vault 2 will use to connect with Vault 1.

  2. Set VAULT_TOKEN environment variable whose value is the client token you just unwrapped.

    **Example:**
    
    ```plaintext
    $ export VAULT_TOKEN="s.SJj086AW7ZaobvRmNoSjhVaj"
    ```
    
  3. Create a server configuration file (config-autounseal.hcl) to start a second Vault instance (Vault 2).

    ```plaintext
    disable_mlock = true
    ui=true
    
    storage "file" {
      path = "/vault-2/data"
    }
    
    listener "tcp" {
      address     = "127.0.0.1:8100"
      tls_disable = 1
    }
    
    seal "transit" {
      address = "http://127.0.0.1:8200"
      disable_renewal = "false"
      key_name = "autounseal"
      mount_path = "transit/"
      tls_skip_verify = "true"
    }
    ```
    
    Notice that the `address` points to the Vault server listening to port
    **8200** (Vault 1).  The `key_name` and `key_name` match to what you created
    in [***Step 1***][step-1].
    
    ~> **NOTE:** The `seal` stanza does not set the `token` value since it's
    already set as `VAULT_TOKEN` environment variable.
    
  4. Start the vault server with the configuration file.

    $ vault server -config=config-autounseal.hcl
    
  5. Open another terminal and initialize your second Vault server (Vault 2).

    $ VAULT_ADDR=http://127.0.0.1:8100 vault operator init -recovery-shares=1 \
             -recovery-threshold=1 > recovery-key.txt
    

    By passing the VAULT_ADDR, the subsequent command gets executed against the second Vault server (http://127.0.0.1:8100).

    Notice that you are setting the number of recovery key and recovery threshold because there is no unseal keys with auto-unseal. Vault 2's master key is now protected by the transit secret engine of Vault 1. Recovery keys are used for high-privilege operations such as root token generation. Recovery keys are also used to make Vault operable if Vault has been manually sealed through the "vault operator seal" command.

1) Check the Vault 2 server status. It is now successfully initialized and unsealed.

    ```plaintext
    $ VAULT_ADDR=http://127.0.0.1:8100 vault status

    Key                      Value
    ---                      -----
    Recovery Seal Type       shamir
    Initialized              true
    Sealed                   false
    Total Recovery Shares    1
    Threshold                1
    ...
    ```

    Notice that it shows `Total Recovery Shares` instead of `Total Shares`. The
    transit secrets engine is solely responsible for protecting the master key
    of **Vault 2**. There are some operations that still requires Shamir's keys
    (e.g. regenerate a root token). Therefore, Vault 2 server requires recovery
    keys although auto-unseal has been enabled.

» Step 3: Verify Auto-Unseal

  1. To verify that Vault 2 gets automatically unseal, press Ctrl + C to stop the Vault 2 server where it is running.

    ```plaintext
    ...
    [INFO]  core.cluster-listener: rpc listeners successfully shut down
    [INFO]  core: cluster listeners successfully shut down
    [INFO]  core: vault is sealed
    ```
    
    Note that Vault 2 is now ***sealed***.
    
  2. Press the upper-arrow key, and execute the vault server -config=config-autounseal.hcl command again to start Vault 2 and see what happens.

    ```plaintext
    $ vault server -config=config-autounseal.hcl
    
    ==> Vault server configuration:
    
                   Seal Type: transit
             Transit Address: http://127.0.0.1:8200
            Transit Key Name: autounseal
          Transit Mount Path: transit/
                         Cgo: disabled
                  Listener 1: tcp (addr: "0.0.0.0:8100", cluster address: "0.0.0.0:8101", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
                   Log Level: info
                       Mlock: supported: true, enabled: false
                     Storage: file
                     Version: Vault v1.1.0
                 Version Sha: 36aa8c8dd1936e10ebd7a4c1d412ae0e6f7900bd
    
    ==> Vault server started! Log data will stream in below:
    
    [WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
    [INFO]  core: stored unseal keys supported, attempting fetch
    [INFO]  core: vault is unsealed
    ...
    ```
    
    Notice that the Vault server is already unsealed.  The **Transit Address**
    is set to your Vault 1 which is listening to port 8200
    (http://127.0.0.1:8200).
    
  3. Check the Vault 2 server status.

    $ VAULT_ADDR=http://127.0.0.1:8100 vault status
    
    Key                      Value
    ---                      -----
    Recovery Seal Type       shamir
    Initialized              true
    Sealed                   false
    Total Recovery Shares    1
    Threshold                1
    ...
    
  4. Now, examine the audit log in Vault 1.

    $ tail -f audit.log | jq
    
    ...
    "request": {
      "id": "a46719eb-eee0-92a4-2da6-6c7de77fd410",
      "operation": "update",
      "client_token": "hmac-sha256:ce8613487054dadb36a9d08da1f5a4bbee2fbfc1ef1ec5ebdeec696df7823e69",
      "client_token_accessor": "hmac-sha256:f3b6cb798605835e8a00bafa9e0e16fc0534b8923b31e499f2c8e694f6b69158",
      "namespace": {
        "id": "root",
        "path": ""
      },
      "path": "transit/decrypt/autounseal",
      ...
      "remote_address": "127.0.0.1",
      "wrap_ttl": 0,
      "headers": {}
    },
    ...
    }
    

    You should see an update request against the transit/decrypt/autounseal path. The remote_address is 127.0.0.1 in this example since Vault 1 and Vault 2 are both running locally. If the Vault 2 is running on a different host, the audit log will show the IP address of the Vault 2 host.

» Help and Reference