Security

[Enterprise] HSM Integration - Entropy Augmentation

The HSM Integration - Seal Wrap guide walked through the master key wrapping, automatic unsealing and seal wrapping functionalities provided by integrating Vault with HSM.

Challenge

Critical to any cryptographic system is its entropy, the randomness of the pseudorandom number generator (or PRNG) used for generating random numbers used in cryptographic operations such as key creation and encryption.

In Vault, random number generation occurs at the operating system level and varies depending on where Vault is running (Linux, Windows, BSD, or WebAssembly). These different PRNGs use different underlying sources of entropy; therefore, different levels of randomness when generating random numbers.

If an organization must conform to entropy requirements which are often dictated by standards such as FIPS 140-2, the Vault Enterprise admins have no control over how the OS generates random numbers.

Solution

Vault Enterprise version 1.3 introduced the Entropy Augmentation function to leverage an external Hardware Security Module (HSM) for augmenting system entropy via the PKCS#11 protocol.

HSM Entropy Augmentation

With Entropy Augmentation enabled, the following keys and tokens leverage the configured external entropy source.

OperationDescription
Master KeyAES key that is encrypted by the seal mechanism. This encrypts the key ring.
Key Ring Encryption KeysThe keys embedded in Vault's keyring which encrypt all of Vault's storage.
Recovery KeyWith auto-unseal, use the recovery keys to regenerate root token, key rotation, etc.
TLS Private KeysFor HA leader, Raft and Enterprise Replications.
MFA TOTP KeysThe keys used for TOTP in Vault Enterprise MFA
JWT Signing KeysThe keys used to sign wrapping token JWTs.
Root TokensSuperuser tokens granting access to all operations in Vault.
DR Operation TokensToken that allows certain actions to be performed on a DR secondary.

The transit secrets engine manages a number of different key types and leverages the keysutil package to generate keys. It will use the external entropy source for key generation.

Prerequisites

This intermediate operations guide assumes that you have:

  • A supported HSM cluster to be integrated with Vault
  • Vault Enterprise version 1.3 or later

Step 1: Configure HSM Integration

When a Vault server is started, it normally starts in a sealed state where a quorum of existing unseal keys is required to unseal it. By integrating Vault with HSM, your Vault server can be automatically unsealed by the trusted HSM key provider.

  1. To enable Entropy Augmentation, define the entropy stanza in your server configuration file. Since Vault will delegate the random number generation to an HSM, be sure to set the seal stanza with your HSM cluster connection information.

    Example: config-hsm.hcl

    # Add the entropy stanza
    entropy "seal" {
      mode = "augmentation"
    }
    
    # This example uses AWS CloudHSM cluster connection information
    seal "pkcs11" {
      lib = "/opt/cloudhsm/lib/libcloudhsm_pkcs11.so"
      slot = "1"
      pin = "vault:Password1"
      key_label = "hsm_demo"
      hmac_key_label = "hsm_hmac_demo"
      generate_key = "true"
    }
    
    # Configure the storage backend for Vault
    storage "consul" {
      address = "0.0.0.0:8500"
      path = "vault/"
    }
    
    # Addresses and ports on which Vault will respond to requests
    listener "tcp" {
      address          = "0.0.0.0:8200"
      tls_disable      = "true"
    }
    
    ui = true
    
  2. If you are using systemd, the ExecStart parameter should point to the correct location of your configuration file. For example, if your configuration file is located at /etc/vault.d/config-hsm.hcl, the vault.service file may look as follow.

    Example: /etc/systemd/system/vault.service

    [Unit]
    Description="HashiCorp Vault - A tool for managing secrets"
    Documentation=https://www.vaultproject.io/docs/
    Requires=network-online.target
    After=network-online.target
    ConditionFileNotEmpty=/etc/vault.d/vault.hcl
    StartLimitIntervalSec=60
    StartLimitBurst=3
    
    [Service]
    User=vault
    Group=vault
    ProtectSystem=full
    ProtectHome=read-only
    PrivateTmp=yes
    PrivateDevices=yes
    SecureBits=keep-caps
    AmbientCapabilities=CAP_IPC_LOCK
    Capabilities=CAP_IPC_LOCK+ep
    CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
    NoNewPrivileges=yes
    ExecStart=/usr/local/bin/vault server -config=/etc/vault.d/config-hsm.hcl
    ExecReload=/bin/kill --signal HUP $MAINPID
    KillMode=process
    KillSignal=SIGINT
    Restart=on-failure
    RestartSec=5
    TimeoutStopSec=30
    StartLimitInterval=60
    StartLimitIntervalSec=60
    StartLimitBurst=3
    LimitNOFILE=65536
    LimitMEMLOCK=infinity
    
    [Install]
    WantedBy=multi-user.target
    
  3. Start the Vault server.

    Example:

    # If starting from the command
    $ vault server -config=/etc/vault.d/config-hsm.hcl
    
    # If starting as a service
    $ sudo systemctl enable vault
    $ sudo systemctl start vault
    
  4. Initialize Vault.

    # Set the VAULT_ADDR environment variable
    $ export VAULT_ADDR="http://127.0.0.1:8200"
    
    # Initialize Vault
    $ vault operator init -recovery-shares=1 -recovery-threshold=1
    
    Recovery Key 1: 2bU2wOfmyMqYcsEYo4Mo9q4s/KAODgHHjcmZmFOo+XY=
    Initial Root Token: s.20JnHBY66EKTj9zyR6SjTMNq
    
    Success! Vault is initialized
    
    Recovery key initialized with 1 key shares and a key threshold of 1. Please
    securely distribute the key shares printed above.
    

    When Vault is initialized while using an HSM, rather than unseal keys being returned to the operator, recovery keys are returned. These are generated from an internal recovery key that is split via Shamir's Secret Sharing, similar to Vault's treatment of unseal keys when running without an HSM. Some Vault operations such as generation of a root token require these recovery keys.

  5. Check to verify that the Vault server is now initialized and auto-unsealed.

    $ vault status
    
    Key                      Value
    ---                      -----
    Recovery Seal Type       shamir
    Initialized              true
    Sealed                   false
    Total Recovery Shares    1
    Threshold                1
    Version                  1.3.0-ent.hsm
    Cluster Name             vault-cluster-a9b467a6
    Cluster ID               fb6aa58b-9cba-c6fd-e2e8-96854fe9573a
    HA Enabled               false
    
  6. Login to the Vault using the generated root token.

    $ vault login s.20JnHBY66EKTj9zyR6SjTMNq
    

Step 2: Enable Entropy Augmentation

To leverage the external entropy source, set the external_entropy_access parameter to true when you enable a secrets engine or auth method.

In this step, you are going to enable external entropy source on a transit secrets engine.

CLI command / API call using cURL

CLI command

  1. Execute the following command to enable transit secrets engine with external entropy source using the -external-entropy-access flag.

    $ vault secrets enable -external-entropy-access transit
    
  2. List the enabled secrets engine with -detailed flag.

    $ vault secrets list -detailed
    
    Path          Plugin       Accessor              ...  External Entropy Access  ...
    ----          ------       --------              ...  -----------------------  ...
    cubbyhole/    cubbyhole    cubbyhole_a4084622    ...  false                    ...
    identity/     identity     identity_b5738cb7     ...  false                    ...
    sys/          system       system_a8b3552e       ...  false                    ...
    transit/      transit      transit_88cd3066      ...  true                     ...
    

    Notice that the External Entropy Access is set to true for transit/.

  3. You can start using the transit secrets engine to encrypt your sensitive data which leverages the HSM as its external entropy source. Regardless, the user experience remains the same as before.

    Example:

    # Create a new encryption key named, "orders"
    $ vault write -f transit/keys/orders
    
    # Send a base64-encoded string to be encrypted by Vault
    $ vault write transit/encrypt/orders \
            plaintext=$(base64 <<< "4111 1111 1111 1111")
    Key           Value
    ---           -----
    ciphertext    vault:v1:AY3ZF2bwGfwZ9dJLSztCLdpPUHkfl/kwaQeRITvKgn74bGYyMI+n34w1CMO8aeg=
    
    # Now, test to verify that you can decrypt
    $ vault write transit/decrypt/orders \
            ciphertext="vault:v1:AY3ZF2bwGfwZ9dJLSztCLdpPUHkfl/kwaQeRITvKgn74bGYyMI+n34w1CMO8aeg="
    
    # Decode to get the original data
    $ base64 --decode <<< Y3JlZGl0LWNhcmQtbnVtYmVyCg==
    4111 1111 1111 1111
    

API call using cURL

  1. To enable the transit secrets engine with external entropy source enabled, set the external_entropy_access parameter to true.

    # Set the seal_wrap parameter to true in the request payload
    $ tee payload.json <<EOF
    {
      "type": "transit",
      "external_entropy_access": true
    }
    EOF
    
    # Enable kv secrets engine at kv-v1/
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           --data @payload.json \
           http://127.0.0.1:8200/v1/sys/mounts/transit
    
  2. List the enabled secrets engine:

    $ curl --header "X-Vault-Token: ..." http://127.0.0.1:8200/v1/sys/mounts | jq
    {
      ...
        "transit/": {
          "accessor": "transit_3a02782b",
          "config": {
            ...
          },
          "description": "",
          "external_entropy_access": true,
          "local": false,
          "options": {},
          "seal_wrap": false,
          "type": "transit",
          "uuid": "2ee33940-2577-f67b-c55b-29aaf8c33f08"
        },
        ...
    

    Notice that the external_entropy_access parameter is set to true for transit/.

  3. You can start using the transit secrets engine to encrypt your sensitive data which leverages the HSM as its external entropy source. Regardless, the user experience remains the same as before.

    Example:

    # Create a new encryption key named, "orders"
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           https://127.0.0.1:8200/v1/transit/keys/orders
    
    # Generate base64-encoded plaintext
    $ base64 <<< "4111 1111 1111 1111"
    NDExMSAxMTExIDExMTEgMTExMQo=
    
    # Pass the base64-encoded plaintext in the request payload
    $ curl --header "X-Vault-Token: $VAULT_TOKEN" \
           --request POST \
           --data '{"plaintext": "NDExMSAxMTExIDExMTEgMTExMQo="}' \
           https://127.0.0.1:8200/v1/transit/encrypt/orders | jq
    {
      ...
      "data": {
        "ciphertext": "vault:v1:/9hdQutaWpZR72s3+VSCLK1JNhV1wKM49hYVjh7RjmuxIy/OvshtgV4L4uVB+aQ="
      }
      ...
    }
    
    # Pass the ciphertext in the request payload to decode
    $ curl --header "X-Vault-Token: $VAULT_TOKEN" \
           --request POST \
           --data '{"ciphertext": "Yvault:v1:/9hdQutaWpZR72s3+VSCLK1JNhV1wKM49hYVjh7RjmuxIy/OvshtgV4L4uVB+aQ="}' \
           https://127.0.0.1:8200/v1/transit/decrypt/orders | jq
    {
       ...
       "data": {
         "plaintext": "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="
       }
       ...
    }
    
    # Decode to get the original data
    $ base64 --decode <<< "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="
    4111 1111 1111 1111
    

Important Note

When the external entropy access is enabled, the connectivity to the HSM is required. If the HSM becomes unreachable for any reason, the transit secrets engine returns an error and you cannot encrypt or decrypt data until the HSM connection gets restored.

Error writing data to transit/encrypt/orders: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/transit/encrypt/orders
Code: 400. Errors:

* error performing token check: failed to read entry: error initializing session
for decryption: error logging in to HSM: pkcs11: 0xE0: CKR_TOKEN_NOT_PRESENT

Help and Reference