Secrets as a Service: Dynamic Secrets

Vault can generate secrets on-demand for some systems. For example, when an app needs to access an Amazon S3 bucket, it asks Vault for AWS credentials. Vault will generate an AWS credential granting permissions to access the S3 bucket. In addition, Vault will automatically revoke this credential after the TTL is expired.

NOTE: The Getting Started guide walks you through the generation of dynamic AWS credentials. This guide demonstrates the use of database secrets engine to dynamically generate database credentials.

Challenge

Data protection is a top priority which means that the database credential rotation is a critical part of any data protection initiative. Each role has a different set of permissions granted to access the database. When a system is attacked by hackers, continuous credential rotation becomes necessary and needs to be automated.

Solution

Applications ask Vault for database credential rather than setting them as environment variables. The administrator specifies the TTL of the database credentials to enforce its validity so that they are automatically revoked when they are no longer used.

Dynamic Secret Workflow

Each app instance can get unique credentials that they don't have to share. By making those credentials to be short-lived, you reduced the change of the secret to being compromised. If an app was compromised, the credentials used by the app can be revoked rather than changing more global set of credentials.

Personas

The end-to-end scenario described in this guide involves two personas:

admin with privileged permissions to configure secret engines
apps read the secrets from Vault

Prerequisites

To perform the tasks described in this guide, you need to have:

A Vault environment of version 1.2 or later. Refer to the Getting Started guide to install Vault. Make sure that your Vault server has been initialized and unsealed.
A PostgreSQL environment you can connect, or Docker to run a PostgreSQL in a container.

Policy requirements

NOTE: For the purpose of this guide, you can use root token to work with Vault. However, it is recommended that root tokens are only used for just enough initial setup or in emergencies. As a best practice, use tokens with appropriate set of policies based on your role in the organization.

To perform all tasks demonstrated in this guide, your policy must include the following permissions:

# Mount secret engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the database secret engine and create roles
path "database/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

If you are not familiar with policies, complete the policies guide.

Steps

In this guide, you are going to configure PostgreSQL secret engine, and create a read-only database role. The Vault generated PostgreSQL credentials will only have read permission.

Enable the database secret engine
Configure PostgreSQL secret engine
Create a role
Request PostgreSQL credentials
Validation

Step 1 through 3 need to be performed by an admin user. Step 4 describes the commands that an app runs to get a database credentials from Vault.

Step 1: Enable the database secret engine

(Persona: admin)

First step is to enable the database secrets engine at a desired path.

CLI command / API call using cURL / Web UI

CLI command

Execute the following command to enable the database secrets engine at database/ path.

$ vault secrets enable database

If you want to enable the secrets engine at a different path, use the -path parameter to specify your desired path. Otherwise, the secrets engine gets enabled at a path named after its type. Since the type is database in this example, its path becomes database in absence of -path parameter.

API call using cURL

Mount database secret engine using /sys/mounts endpoint:

curl --header "X-Vault-Token: <TOKEN>" \
       --request POST \
       --data <PARAMETERS> \
       <VAULT_ADDRESS>/v1/sys/mounts/<PATH>

Where <TOKEN> is your valid token, and <PARAMETERS> holds configuration parameters of the secret engine.

Example:

The following example mounts database secret engine at sys/mounts/database path, and passed the secret engine type ("database") in the request payload.

$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"type":"database"}' \
       https://127.0.0.1:8200/v1/sys/mounts/database

NOTE: In this guide, the database secret engine is enabled at the /database path in Vault. However, it is possible to enable your secret engines at any location.

Web UI

Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.
Select Enable new engine.
Select Databases from the list, and then click Next.
Click Enable Engine to complete. This sets the path to be database.

NOTE: In this guide, the database secret engine is enabled at the /database path in Vault. However, it is possible to enable your secret engines at any location by entering your desired path in the Path text field.

Step 2: Configure PostgreSQL secret engine

(Persona: admin)

The PostgreSQL secret engine needs to be configured with valid credentials. It is very common to give Vault the superuser credentials and let Vault manage the auditing and lifecycle credentials; it's much better than having one person manage the credentials.

For the purpose of this tutorial, let's run PostgreSQL Docker image in a container.

NOTE: If you already have a running PostgreSQL server that you can connect to, ignore this step.

Execute the following command to start a postgres instance which listens to port 5432, and the superuser (root) password is set to rootpassword.

# For the purpose of this demo, keeping it simple
$ docker run --name postgres -e POSTGRES_USER=root \
         -e POSTGRES_PASSWORD=rootpassword \
         -d -p 5432:5432 postgres

# Verify that the postgres container is running
$ docker ps
CONTAINER ID        IMAGE            ...         PORTS                    NAMES
befcf913da91        postgres         ...         0.0.0.0:5432->5432/tcp   postgres

The following command configures the database secret engine using postgresql-database-plugin. The allowed role is readonly which you will create in Step 3.

NOTE: If your database connection URL is different from this example, be sure to replace the command with correct URL to match your environment.

CLI command / API call using cURL

CLI command

Example:

$ vault write database/config/postgresql
        plugin_name=postgresql-database-plugin \
        allowed_roles=readonly \
        connection_url=postgresql://root:rootpassword@localhost:5432/postgres?sslmode=disable

API call using cURL

Example:

# Create an HTTP request payload
$ tee payload.json <<EOF
{
    "plugin_name": "postgresql-database-plugin",
    "allowed_roles": "readonly",
    "connection_url": "postgresql://root:rootpassword@localhost:5432/postgres?sslmode=disable"
}
EOF

# Invoke the database/config/postgresql endpoint
$ curl --header "X-Vault-Token: ..." --request POST --data @payload.json \
       http://127.0.0.1:8200/v1/database/config/postgresql

Read the Database Root Credential Rotation guide to learn about rotating the root credential immediately after the initial configuration of each database.

Step 3: Create a role

(Persona: admin)

In Step 2, you configured the PostgreSQL secret engine by passing readonly role as an allowed member. The next step is to define the readonly role. A role is a logical name that maps to a policy used to generate credentials.

Vault does not know what kind of PostgreSQL users you want to create. So, supply the information in SQL to create desired users.

Example: readonly.sql

CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";

The values within the {{<value>}} will be filled in by Vault. Notice that VALID_UNTIL clause. This tells PostgreSQL to revoke the credentials even if Vault is offline or unable to communicate with it.

CLI command / API call using cURL

CLI command

Example:

$ vault write database/roles/readonly db_name=postgresql \
        creation_statements=@readonly.sql \
        default_ttl=1h max_ttl=24h

The above command creates a role named readonly with default TTL of 1 hour, and max TTL of the credential is set to 24 hours. The readonly.sql statement is passed as the role creation statement.

API call using cURL

Example:

# Create the HTTP request payload
$ tee payload.json <<EOF
{
    "db_name": "postgres",
    "creation_statements": ["CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';
  GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";"],
    "default_ttl": "1h",
    "max_ttl": "24h"
}
EOF

# Invoke the database/roles API endpoint
$ curl --header "X-Vault-Token: ..." \
       --request POST --data @payload.json \
       http://127.0.0.1:8200/v1/database/roles/readonly

The db_name, creation_statements, default_ttl, and max_ttl are set in the role-payload.json.

Step 4: Request PostgreSQL credentials

(Persona: apps)

Now, you are switching to apps persona. To get a new set of PostgreSQL credentials, the client app needs to be able to read from the readonly role endpoint. Therefore the app's token must have a policy granting the read permission.

apps-policy.hcl

# Get credentials from the database secret engine
path "database/creds/readonly" {
  capabilities = [ "read" ]
}

CLI command / API call using cURL / Web UI

CLI command

First create an apps policy, and generate a token so that you can authenticate as an apps persona.

Example:

# Create "apps" policy
$ vault policy write apps apps-policy.hcl
Policy 'apps' written.

# Create a new token with app policy
$ vault token create -policy="apps"

Key                  Value
---                  -----
token                s.NN5Izfj9ok3VuZiaP9N9QJ1V
token_accessor       l1QxSKs80HfrzR1gfy5zm7ay
token_duration       768h
token_renewable      true
token_policies       ["apps" "default"]
identity_policies    []
policies             ["apps" "default"]

Use the returned token to perform the remaining.

NOTE: AppRole Pull Authentication guide demonstrates more sophisticated way of generating a token for your apps.

# Invoke the vault command with apps token
$ VAULT_TOKEN=s.NN5Izfj9ok3VuZiaP9N9QJ1V vault read database/creds/readonly

Key                Value
---                -----
lease_id           database/creds/readonly/fyF5xDomnKeCHNZNQgStwBKD
lease_duration     1h
lease_renewable    true
password           A1a-ckirtymYaXACpIHn
username           v-token-readonly-6iRIcGv8tLpu816oblPY-1556567086

Re-run the command and notice that Vault returns a different set of credentials each time. This means that each app instance can acquire a unique set of DB credentials.

API call using cURL

First create an apps policy, and generate a token so that you can authenticate as an app persona.

# Create the HTTP request payload
$ tee payload.json <<EOF
{
  "policy": "path \"database/creds/readonly\" {capabilities = [ \"read\" ]}"
}
EOF

# Create "apps" policy
$ curl --header "X-Vault-Token: ..." --request PUT \
       --data @payload.json \
       http://127.0.0.1:8200/v1/sys/policies/acl/apps

# Generate a new token with apps policy
$ curl --header "X-Vault-Token: ..." --request POST \
       --data '{"policies": ["apps"]}' \
       http://127.0.0.1:8200/v1/auth/token/create | jq
{
   ...
   "auth": {
     "client_token": "s.qbloIGiHddxc3chdtE2TrGX7",
     "accessor": "8zrb4SIVJ4DMZUsfr6uRsqdu",
     "policies": [
       "apps",
       "default"
     ],
     "token_policies": [
       "apps",
       "default"
     ],
     ...
   }
}

Be sure to use the returned token to perform the remaining.

NOTE: AppRole Pull Authentication guide demonstrates more sophisticated way of generating a token for your apps.

$ curl --header "X-Vault-Token: s.qbloIGiHddxc3chdtE2TrGX7" \
       --request GET \
       http://127.0.0.1:8200/v1/database/creds/readonly | jq
{
   ...
   "data": {
     "password": "A1a-cxHgJAjdnyDrpBiT",
     "username": "v-token-readonly-4pBp6GCvhixU6PPlbgRw-1556567165"
   },
   ...
}

Re-run the command and notice that Vault returns a different set of credentials each time. This means that each app instance can acquire a unique set of DB credentials.

Web UI

Click the Policies tab, and then select Create ACL policy.
Toggle Upload file sliding switch, and click Choose a file to select your apps-policy.hcl file you authored. This loads the policy and sets the Name to be apps.
Click Create Policy to complete.
Click the Vault CLI shell icon (>_) to open a command shell. Execute vault write auth/token/create policies=apps in the CLI shell to create a new token:
Copy the generated client token value.
Sign out of the Vault UI.
Now, sign into the Vault using the newly generated token you just copied.
Click the Vault CLI shell icon (>_) to open a command shell. Execute vault read database/creds/readonly in the CLI shell.
NOTE: Re-run the command and notice that Vault returns a different set of credentials each time. This means that each app instance can acquire a unique set of DB credentials.

Validation

Generate a new set of credentials.

$ vault read database/creds/readonly

Key                Value
---                -----
lease_id           database/creds/readonly/IQKUMCTg3M5QTRZ0abmLKjTX
lease_duration     1h
lease_renewable    true
password           A1a-T7Ezuy261IBew8H9
username           v-token-readonly-47vOtpF7pZq79Xajx7yq-1556567237

The generated username is v-token-readonly-47vOtpF7pZq79Xajx7yq-1556567237.

Connect to the postgres as an admin user, and run the following psql commands.

$ psql -U postgres

postgres > \du
                                                      List of roles
                   Role name                     |                         Attributes                         | Member of
--------------------------------------------------+------------------------------------------------------------+-----------
postgres                                         | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
v-token-readonly-47vOtpF7pZq79Xajx7yq-1556567237 | Password valid until 2018-01-11 00:37:14+00                | {}

postgres > \q

The \du command lists all users. You should be able to verify that the username generated by Vault exists.

Renew the lease for this credential by passing its lease_id.

$ vault lease renew database/creds/readonly/IQKUMCTg3M5QTRZ0abmLKjTX

Key                Value
---                -----
lease_id           database/creds/readonly/IQKUMCTg3M5QTRZ0abmLKjTX
lease_duration     1h
lease_renewable    true

Revoke the generated credentials.
```
$ vault lease revoke database/creds/readonly/3e8174da-6ca0-143b-aa8c-4c238aa02809
```
NOTE: If you run the command with -prefix=true flag, it revokes all secrets under database/creds/readonly.
Now, when you check the list of users in PostgreSQL, none of the Vault generated user name exists.

Next Step

There are some tools available to help integrate your applications with Vault's database secrets engine. Using those tools, the existing applications may require minimum to no code change to work with Vault.

Refer to the following guides:

Sentinel

Introduction

Static Secrets: Key/Value Secret Engine

Versioned Key/Value Secret Engine

Cubbyhole Response Wrapping

Database Root Credential Rotation

Database Static Roles and Credential Rotation

Azure Secrets Engine

Vault Agent Caching

Build Your Own Certificate Authority (CA)

SSH Secrets Engine: One-Time SSH Password

[Enterprise] KMIP Secrets Engine

Building Plugin Backends

Introduction

Static Secrets: Key/Value Secret Engine

Versioned Key/Value Secret Engine

Cubbyhole Response Wrapping

Secrets as a Service: Dynamic Secrets

Database Root Credential Rotation

Database Static Roles and Credential Rotation

Azure Secrets Engine

Vault Agent Caching

Build Your Own Certificate Authority (CA)

SSH Secrets Engine: One-Time SSH Password

[Enterprise] KMIP Secrets Engine

Building Plugin Backends

Secrets as a Service: Dynamic Secrets

»Challenge

»Solution

»Personas

»Prerequisites

»Policy requirements

»Steps

»Step 1: Enable the database secret engine

»CLI command

»API call using cURL

»Web UI

»Step 2: Configure PostgreSQL secret engine

»CLI command

»API call using cURL

»Step 3: Create a role

»CLI command

»API call using cURL

»Step 4: Request PostgreSQL credentials

»CLI command

»API call using cURL

»Web UI

»Validation

»Next Step

»Help and Reference

Challenge

Solution

Personas

Prerequisites

Policy requirements

Steps

Step 1: Enable the database secret engine

CLI command

API call using cURL

Web UI

Step 2: Configure PostgreSQL secret engine

CLI command

API call using cURL

Step 3: Create a role

CLI command

API call using cURL

Step 4: Request PostgreSQL credentials

CLI command

API call using cURL

Web UI

Validation

Next Step

Help and Reference