The term cubbyhole comes from an Americanism where you get a "locker" or "safe place" to store your belongings or valuables. In Vault, the cubbyhole is your "locker". All secrets are namespaced under your token. If that token expires or is revoked, all the secrets in its cubbyhole are revoked as well.
It is not possible to reach into another token's cubbyhole even as the root user. This is an important difference between the cubbyhole and the key/value secrets engine. The secrets in the key/value secrets engine are accessible to any token for as long as its policy allows it.
»Personas
The end-to-end scenario described in this tutorial involves two personas:
»Challenge
In order to tightly manage the secrets, you set the scope of who can do what using the Vault policy and attach that to tokens, roles, entities, etc.
Think of a case where you have a trusted entity (Chef, Jenkins, etc.) which reads secrets from Vault. This trusted entity must obtain a token. If the trusted entity or its host machine was rebooted, it must re-authenticate with Vault using a valid token.
How can you securely distribute the initial token to the trusted entity?
»Solution
Use Vault's cubbyhole response wrapping where the initial token is stored in the cubbyhole secrets engine. The wrapped secret can be unwrapped using the single-use wrapping token. Even the user or the system created the initial token won't see the original value. The wrapping token is short-lived and can be revoked just like any other tokens so that the risk of unauthorized access can be minimized.
What is cubbyhole response wrapping?
- When response wrapping is requested, Vault creates a temporary single-use token (wrapping token) and insert the response into the token's cubbyhole with a short TTL
- Only the expecting client who has the wrapping token can unwrap this secret
- Any Vault response can be distributed using the response wrapping
Benefits of using the response wrapping:
- It provides cover by ensuring that the value being transmitted across the wire is not the actual secret. It's a reference to the secret.
- It provides malfeasance detection by ensuring that only a single party can ever unwrap the token and see what's inside
- It limits the lifetime of the secret exposure
- The TTL of the response-wrapping token is separate from the wrapped secret's lease TTL
»Prerequisites
To perform the tasks described in this tutorial, you need to have a Vault environment. Refer to the Getting Started tutorial to install Vault. Make sure that your Vault server has been initialized and unsealed.
NOTE: An interactive tutorial is also available if you do not have a Vault environment to perform the steps described in this tutorial. Click the Show Terminal button to start.
»Policy requirements
NOTE: For the purpose of this tutorial, you can use root
token to work
with Vault. However, it is recommended that root tokens are only used for just
enough initial setup or in emergencies. As a best practice, use tokens with
appropriate set of policies based on your role in the organization.
To perform all tasks demonstrated in this tutorial, your policy must include the following permissions:
# Manage tokens
path "auth/token/*" {
capabilities = [ "create", "read", "update", "delete", "sudo" ]
}
# Write ACL policies
path "sys/policies/acl/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
# Manage secret/dev secrets engine - for Verification test
path "secret/dev" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
If you are not familiar with policies, complete the policies tutorial.
»Scenario Introduction
Think of a scenario where apps read secrets from Vault. The apps
need:
- Policy granting "read" permission on the specific path (
secret/dev
) - Valid tokens to interact with Vault
Setting the appropriate policies and token generation are done by the admin
persona. For the admin
to distribute the initial token to the app securely, it
uses cubbyhole response wrapping. In this tutorial, you perform the following:
NOTE: This tutorial demonstrates how the response wrapping works. To learn more about reading and writing secrets in Vault, refer to the Static Secret tutorial.
»Step 1: Create and wrap a token
(Persona: admin)
When a newly created token is wrapped, Vault inserts the generated token into the cubbyhole of a single-use token, returning that single-use wrapping token. Retrieving the secret requires an unwrap operation against this wrapping token.
In this scenario, an admin user creates a token using response wrapping. To perform the steps in this tutorial, first create a policy for the app.
apps-policy.hcl
:
# For testing, read-only on secret/dev path
path "secret/data/dev" {
capabilities = [ "read" ]
}
First create an
apps
policy.$ vault policy write apps apps-policy.hcl Policy 'apps' written.
Enable key/value v2 secrets engine at secret/ if it's not enabled already.
$ vault secrets enable -path=secret kv-v2
Write some test data at
secret/dev
.$ vault kv put secret/dev username="webapp" password="my-long-password"
Generate a token for
apps
persona using response wrapping with TTL of 120 seconds. Also, the generated token should have theapps
policy attached. Use the-wrap-ttl
flag to specify that the response should be wrapped and the wrap TTL indicates the life of the wrapping token which can be either an integer number of seconds or a string duration of minutes (2m) or hours (1h).$ vault token create -policy=apps -wrap-ttl=120 Key Value --- ----- wrapping_token: s.9QFJ8mRxGJD0e7kFfFIbdpDM wrapping_accessor: S0zKNUr2ENbnCtj0YyriO31b wrapping_token_ttl: 2m wrapping_token_creation_time: 2019-12-17 09:45:42.537057 -0800 PST wrapping_token_creation_path: auth/token/create wrapped_accessor: VmBKXoc19ZLZlHGl0nQCvV6r
The response is the wrapping token; therefore the admin user does not even see the generated token from the
token create
command.
»Step 2: Unwrap the secret
(Persona: apps)
The apps
persona receives a wrapping token from the admin
. In order for the
apps
to acquire a valid token to read secrets from secret/dev
path, it must
run the unwrap operation using this token.
NOTE: If a client has been expecting delivery of a response-wrapping token and none arrives, this may be due to an attacker intercepting the token and then preventing it from traveling further. This should cause an alert to trigger an immediate investigation.
The following tasks will be performed to demonstrate the client operations:
- Create a token with
default
policy - Authenticate with Vault using this
default
token (less privileged token) - Unwrap the secret to obtain more privileged token (
apps
persona token) - Verify that you can read
secret/dev
using theapps
token
First, create a token with
default
policy.$ vault token create -policy=default Key Value --- ----- token s.VR3mwdk9miJ6VRLFbQe0Duwq token_accessor vJOhaSjkwaBhlvwPGR5qTNYc token_duration 768h token_renewable true token_policies ["default"] identity_policies [] policies ["default"]
Authenticate using the generated token.
$ vault login s.VR3mwdk9miJ6VRLFbQe0Duwq Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token. Key Value --- ----- token s.VR3mwdk9miJ6VRLFbQe0Duwq token_accessor vJOhaSjkwaBhlvwPGR5qTNYc token_duration 767h59m43s token_renewable true token_policies ["default"] identity_policies [] policies ["default"]
Unwrap the secret by passing the wrapping token you received in Step 1.
$ VAULT_TOKEN="s.3Kf3Xfn58Asr3bSDkRXATHrw" vault unwrap Key Value --- ----- token s.e5XHvU78iR08vAUk4Wq9JOBk token_accessor zvP8uMEVoe9xS6qHQTXpKekJ token_duration 768h token_renewable true token_policies ["apps" "default"] identity_policies [] policies ["apps" "default"]
Verify that this token has
apps
policy attached.NOTE: Remember that the wrapping TTL is set to 120 seconds (2 minutes). If you received the
wrapping token is not valid or does not exist
error, repeat Step 1 to create a token with response wrapping again.To unwrap the wrapped secret, you can do one of the following.
$ vault unwrap
Or
$ VAULT_TOKEN=
vault unwrap Or
$ vault login
$ vault unwrap Once the client acquired the token, future requests can be made using this token.
Example:
$ vault login s.e5XHvU78iR08vAUk4Wq9JOBk
Verify to see if the token can access
secret/dev
.$ vault kv get secret/dev ====== Metadata ====== Key Value --- ----- created_time 2019-12-17T19:44:08.220996Z deletion_time n/a destroyed false version 1 ====== Data ====== Key Value --- ----- password my-long-password username webapp
»Additional Discussion: Cubbyhole
The cubbyhole
secrets engine provides your own private secret storage space
where no one else can read (including root
). This comes handy when you want to
store a password tied to your username that should not be shared with anyone.
The cubbyhole secrets engine is mounted at the cubbyhole/
prefix by
default. The secrets you store in the cubbyhole/
path are tied to your token
and all tokens are permitted to read and write to the cubbyhole
secrets engine
by the default
policy.
$ vault policy read default
...
# Allow a token to manage its own cubbyhole
path "cubbyhole/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
...
To test the cubbyhole secrets engine, perform the following steps. (NOTE: Keep
using the apps
token from Step 2 to ensure that you are logged in with
non-root token.)
Commands to write secrets to the cubbyhole
secrets engine:
$ vault write cubbyhole/<PATH> <KEY>=<VALUE>
Commands to read secrets to the cubbyhole
secrets engine:
$ vault read cubbyhole/<PATH>
Example:
Write some test secrets under cubbyhole/private/
path.
$ vault write cubbyhole/private/access-token token="123456789abcdefg87654321"
Success! Data written to: cubbyhole/private/access-token
Now, read the values from cubbyhole/private/access-token
.
$ vault read cubbyhole/private/access-token
Key Value
--- -----
token 123456789abcdefg87654321
Now, try to access the secret using the root
token. It should NOT return the
secret.
$ VAULT_TOKEN=<ROOT_TOKEN> vault read cubbyhole/private/access-token
No value found at cubbyhole/private/access-token