Vault can be used to store any secret in a secure manner. The secrets may be SSL certificates and keys for your organization's domain, credentials to connect to a corporate database server, etc. Storing such sensitive information in plaintext is not desirable.
NOTE: This tutorial demonstrates the use of Key/Value Secrets Engine Version 1. To learn about Key/Value Secrets Engine Version 2, refer to the Versioned Key/Value Secrets Engine tutorial.
»Personas
The end-to-end scenario described in this tutorial involves two personas:
»Challenge
Consider the following situations:
- Developers use a single admin account to access a third-party app (e.g. Splunk) and anyone who knows the user ID and password can log in as an admin
- SSH keys to connect to remote machines are shared and stored as a plaintext
- API keys to invoke external system APIs are stored as a plaintext
- An app integrates with LDAP, and its configuration information is in a plaintext
Organizations often seek an uniform workflow to securely store this sensitive information.
»Solution
Use Vault as centralized secret storage to secure any sensitive information. Vault encrypts these secrets using 256-bit AES in GCM mode with a randomly generated nonce prior to writing them to its persistent storage. The storage backend never sees the unencrypted value, so even if an attacker gained access to the raw storage, they wouldn't be able to read your secrets.
»Prerequisites
To perform the tasks described in this tutorial, you need to have a Vault environment. Refer to the Getting Started tutorial to install Vault. Make sure that your Vault server has been initialized and unsealed.
NOTE: An interactive tutorial is also available if you do not have a Vault environment to perform the steps described in this tutorial. Click the Show Terminal button to start.
»Policy requirements
NOTE: For the purpose of this tutorial, you can use the root
token to
work with Vault. However, it is recommended that root tokens are only used for
just enough initial setup or in emergencies. As a best practice, use tokens with
appropriate set of policies based on your role in the organization.
To perform all tasks demonstrated in this tutorial, your policy must include the following permissions:
# Enable key/value secrets engine at the kv-v1 path
path "sys/mounts/kv-v1" {
capabilities = [ "update" ]
}
# To list the available secrets engines
path "sys/mounts" {
capabilities = [ "read" ]
}
# Write and manage secrets in key/value secrets engine
path "kv-v1/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
# Create policies to permit apps to read secrets
path "sys/policies/acl/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
# Create tokens for verification & test
path "auth/token/create" {
capabilities = [ "create", "update", "sudo" ]
}
If you are not familiar with policies, complete the policies tutorial.
»Scenario Introduction
This tutorial demonstrates the basic steps to store secrets using Vault. The scenario here is to store the following secrets:
- API key (Google API)
- Root certificate of a production database (MySQL)
To store your API key within the configured physical storage for Vault, use the key/value secrets engine.
The Key/Value secrets engine passes any operation through to the configured storage backend for Vault. For example, if your Vault server is configured with Consul as its storage backend, a "read" operation turns into a read from Consul at the same path.
You will perform the following:
- Start Vault
- Enable KV Secrets Engine
- Store the Google API key
- Store the root certificate for MySQL
- Generate a token for apps
- Retrieve the secrets
»Start Vault
In another terminal, start a Vault dev server with root
as the root token.
$ vault server -dev -dev-root-token-id root
The Vault dev server defaults to running at 127.0.0.1:8200
. The server is
initialized and unsealed.
Insecure operation: Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.
Export an environment variable for the vault
CLI to address the Vault server.
$ export VAULT_ADDR=http://127.0.0.1:8200
Export an environment variable for the vault
CLI to authenticate with the
Vault server.
$ export VAULT_TOKEN=root
NOTE: For these tasks, you can use Vault's root token. However, it is recommended that root tokens are only used for enough initial setup or in emergencies. As a best practice, use an authentication method or token that meets the policy requirements.
The Vault server is ready.
»Enable KV Secrets Engine
Currently, when you start the Vault server in dev
mode, it
automatically enables v2
of the KV secrets engine at secret/
.
This tutorial focuses on key/value v1 secrets engine. The Versioned Key/Value Secret Engin tutorial highlights features that are specific to the key/value v2 secrets engine.
List enabled secrets engines.
$ vault secrets list -detailed
Under the Options, the version number is indicated.
Path Plugin Accessor Default TTL Max TTL Force No Cache Replication Seal Wrap External Entropy Access Options Description UUID
---- ------ -------- ----------- ------- -------------- ----------- --------- ----------------------- ------- ----------- ----
cubbyhole/ cubbyhole cubbyhole_f4543499 n/a n/a false local false false map[] per-token private secret storage 5790b125-c58a-b504-6c9c-7563abaaafcf
identity/ identity identity_4e81662a system system false replicated false false map[] identity store 7919531f-9670-d112-d3c9-d55804dcabdf
secret/ kv kv_ab6587e6 system system false replicated false false map[version:2] key/value secret storage b23eab85-ac15-f976-292a-e93541192f9e
...
Enable the key/value secrets engine v1 at kv-v1/
.
$ vault secrets enable -path="kv-v1" kv
Success! Enabled the kv secrets engine at: kv-v1/
»Store the Google API key
Everything after the kv-v1
path is a key-value pair to write to the
secrets engine. You can specify multiple values. If the value has a space, you
need to surround it with quotes. Having keys with spaces is permitted, but
strongly discouraged because it can lead to unexpected client-side behavior.
Let's assume that the path convention in your organization is
kv-v1/<OWNER>/apikey/<APP>
for API keys. To store the Google API key used
by the engineering team, the path would be kv-v1/eng/apikey/Google
. If you
have an API key for New Relic owned by the DevOps team, the path would look like
kv-v1/devops/apikey/New_Relic
.
To set key/value secrets:
$ vault kv put kv-v1/<PATH> <KEY>=VALUE>
The <PATH>
can be anything you want it to be, and your organization should
decide on the naming convention that makes most sense.
Create a secret at path kv-v1/eng/apikey/Google
with a key
set to
AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI
.
$ vault kv put kv-v1/eng/apikey/Google key=AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI
Success! Data written to: kv-v1/eng/apikey/Google
Read back the secret at path kv-v1/eng/apikey/Google
$ vault kv get kv-v1/eng/apikey/Google
=== Data ===
Key Value
--- -----
key AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI
»Store the root certificate for MySQL
For the purpose of this tutorial, generate a mock certificate using OpenSSL.
$ openssl req -x509 -sha256 -nodes -newkey rsa:2048 -keyout selfsigned.key -out cert.pem
The generated cert.pem
displays something similar to the following.
-----BEGIN CERTIFICATE-----
MIICyjCCAbICCQDrpZYh8et7yTANBgkqhkiG9w0BAQsFADAnMQswCQYDVQQGEwJV
UzELMAkGA1UECAwCQ0ExCzAJBgNVBAcMAlNGMB4XDTE4MTExMjIwNDEwNVoXDTE4
MTIxMjIwNDEwNVowJzELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMQswCQYDVQQH
DAJTRjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJnIdgpml8+xk+Oj
1RGMCyJ1P15RiM6rdtszT+DFBg893Lqsjoyd5YgwELLz0Ux8nviG4L5OXOujEpAP
2cQBxTSLQjBELBZY9q0Qky3+2ewqV6lSfcXrcf/JuDJGR5K8HSqwNG35R3WGnZ+O
JhY0Dmx06IAs/FF8gP88zTQ8M7zuaThkF8MaF4sWPf6+texQwjzk4rewknGBFzar
9wFxVwNCyDD6ewIYPtgDxdJ1bwBVoX3KKKXm8GStl/Zva0aEtbSq/161J4VbTro2
dxArMPKzxjD6NLyF59UNs7vbzyfiw/Wq7BJzU7Kued5KdGt0bEiyWZYO+EvvxGmE
1pHfqysCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAavj4CA+7XFVHbwYMbK3c9tN/
73hkMvkAZWix5bfmOo0cNRuCeJnRIX+o6DmusIc8eXJJJV/20+zoSvUwlsLDPXoN
+c41GfIiEUSaSdSBtETMy8oPga718nIwAvNgYiUHXnV3B0nLYBUpYSnsD00/6VXG
xZUIEVBd7Ib5aRwmK8U5drxoWaBoG5qdvH9iapwTrCcPsRjsLBq7Iza2oBORGlfF
CjqiW2+KJzwRiTQj70yceniGVHM+VSpFYCLJ0mXeyLfITy7joqxr4AGYz+EhpLuf
iDpYDNYlr0JDVQqogskWjrnWOh0YcIJKgVtiTh2HDM5TdQgeXg4wv5IqLok0Tw==
-----END CERTIFICATE-----
NOTE: If you don't have OpenSSL, copy the above certificate and
save it as cert.pem
.
The command is basically the same as the Google API key example. The path
convention for certificates is kv-v1/<ENVIRONMENT>/cert/<SYSTEM>
. To
store the root certificate for production MySQL, the path becomes
kv-v1/prod/cert/mysql
.
Create a secret at path kv-v1/prod/cert/mysql
with a cert
set to file
conents for cert.pem
.
$ vault kv put kv-v1/prod/cert/mysql cert=@cert.pem
NOTE: Any value that begins with @
indicates a file name. Data will be
loaded from this file.
»Generate a token for apps
To read the secrets, apps
persona needs "read" ability on those secrets engine
paths.
Create a policy file named apps-policy.hcl
that permits the read
to the
paths kv-v1/eng/apikey/Google
and kv-v1/prod/cert/mysql
.
$ tee apps-policy.hcl <<EOF
# Read-only permit
path "kv-v1/eng/apikey/Google" {
capabilities = [ "read" ]
}
# Read-only permit
path "kv-v1/prod/cert/mysql" {
capabilities = [ "read" ]
}
EOF
First create
apps
policy$ vault policy write apps apps-policy.hcl Policy 'apps' written.
Create a variable that stores a new token with
apps
policy attached.$ APPS_TOKEN=$(vault token create -policy="apps" -field=token)
Display the token.
$ echo $APPS_TOKEN
Now
apps
can use this token to read the secrets.
NOTE: For the purposes of this tutorial, you created a policy for the
apps
persona, and generated a token for it. However, in the real world, you
may have a dedicated policy author
, or admin
to write policies. Also, the
consumer of the API key may be different from the consumer of the root
certificate. Then each persona would have a policy based on what it needs to
access.
»Retrieve the secrets
Using the token from the Generate a token for apps step, read the Google API key and the root certificate for MySQL.
The command to read a secret is:
$ vault kv get kv-v1/<PATH>
Authenticate with the apps token.
$ vault login $APPS_TOKEN
Read the secret at the path kv-v1/eng/apikey/Google
.
$ vault kv get kv-v1/eng/apikey/Google
Key Value
--- -----
refresh_interval 768h0m0s
key AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI
Read only the key
field at the path kv-v1/eng/apikey/Google
.
$ vault kv get -field=key kv-v1/eng/apikey/Google
AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI
»Root certificate example:
Read only the cert
field at the path kv-v1/prod/cert/mysql
.
$ vault kv get -field=cert kv-v1/prod/cert/mysql
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA6E2Uq0XqreZISgVMUu9pnoMsq+OoK1PI54rsA9vtDE6wiRk0GWhf5vD4DGf1
...
The result displays the certificate.
»Additional Discussion
»Q: How do I enter my secrets without exposing the secret in my shell's history?
As a precaution, you may wish to avoid passing your secret as a part of the CLI command so that the secret won't appear in the history file. Here are a few techniques you can use.
»Option 1: Use a dash "-"
An easy technique is to use a dash "-" and then press Enter. This allows you to
enter the secret on a new line. After entering the secret, press Ctrl+d
to
end the pipe which will write the secret to the Vault.
$ vault kv put kv-v1/eng/apikey/Google key=-
AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI
<Ctrl+d>
»Option 2: Read the secret from a file
The key-value pairs for a secret may be defined in a file.
Create a file named apikey.json
that defines the key
field.
$ tee apikey.json <<EOF
{
"key": "AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI"
}
EOF
Create a secret at path kv-v1/eng/apikey/Google
with keys and values defined
in apikey.json
.
$ vault kv put kv-v1/eng/apikey/Google @apikey.json
»Option 3: Disable all vault command history
The first two options ensure that the contents of the secret do not appear in the shell history. The secret path would still be accessible through the shell history.
You can configure your shell to avoid logging any vault
commands to your
history.
In bash
, set the history to ignore all commands that start with vault
.
$ export HISTIGNORE="&:vault*"
NOTE: This prevents vault
commands from appearing when using the Up
arrow or when searching command history with <Ctrl-r>
.
»Q: How do I save multiple values at once?
The tutorial examples demonstrating a secret with a single key-value pair. Multiple key-value pairs may be provided in a single command.
Create a secret at path kv-v1/dev/config/mongodb
that sets the url
,
db_name
, username
, and password
.
$ vault kv put kv-v1/dev/config/mongodb \
url=foo.example.com:35533 \
db_name=users \
username=admin password=passw0rd
Multiple key-value pairs may be defined in a file provided to the command.
Create a file named mongodb.json
that defines the url
, db_name
, username
,
and password
fields.
$ tee mongodb.json <<EOF
{
"url": "foo.example.com:35533",
"db_name": "users",
"username": "admin",
"password": "pa$$w0rd"
}
EOF
Create a secret at path kv-v1/dev/config/mongodb
with keys and values defined
in mongodb.json
.
»Help and Reference
- Key/Value Secrets Engine
- Key/Value Secrets Engine API
- Client libraries for Vault APIfor commonly used languages