Secrets Management

Building Plugin Backends

Plugin backends utilize the plugin system to enable third-party secrets engines and auth methods.

It is worth noting that even though database secrets engines operate under the same underlying plugin mechanism, they are slightly different in design than plugin backends demonstrated in this guide. The database secrets engine manages multiple plugins under the same backend mount point, whereas plugin backends are kv backends that function as either secret or auth methods.

This guide walks through the basic steps to build, register, and enable external plugin backends.

Also, refer to the Custom Plugin Backends and Plugin System documentation.

Step 1: Compile a mock plugin

For the demonstration, this guide walks through the mock plugin available at https://github.com/hashicorp/vault-guides/blob/master/secrets/mock.

  1. Clone or download the vault-guides repository.
$ git clone git@github.com:hashicorp/vault-guides.git

# Change the working directory to where the mock plugin is located
$ cd vault-guides/secrets/mock
  1. Compile the plugin.
$ go build -o vault/plugins/mock cmd/mock/main.go

This compiles the mock plugin and outputs the generated plugin in the vault/plugins/mock folder.

$ tree
.
├── Makefile
├── README.md
├── backend.go
├── cmd
│   └── mock
│       └── main.go
├── go.mod
├── go.sum
└── vault
    └── plugins
        └── mock

Step 2: Start Vault

Start a Vault server in -dev mode for demonstration. Also, set the -dev-plugin-dir to ./vault/plugins which is where the mock plugin is generated in Step 1.

$ vault server -dev -dev-root-token-id=root -dev-plugin-dir=./vault/plugins

==> Vault server configuration:

             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: false, enabled: false
                 Storage: inmem
                 Version: Vault v1.2.3+ent

2019-10-22T17:23:00.991-0700 [WARN]  received Unrecognized remote plugin message:

This usually means that the plugin is either invalid or simply
needs to be recompiled to support the latest protocol. attempting as db plugin, attempting as auth/secret plugin
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variable:

    $ export VAULT_ADDR='http://127.0.0.1:8200'

The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: RqlTtgkl+cvuQNLvpweOWoNPdqDZwFLqANLnwHbAGwk=
Root Token: root

The following dev plugins are registered in the catalog:
    - mock

Development mode should NOT be used in production installations!
...

The output includes the following message:

The following dev plugins are registered in the catalog:
    - mock

Step 3: Test the plugin

The name of the plugin is mock and it is a custom secrets engine.

# Set the VAULT_ADDR if it's not already set
$ export VAULT_ADDR="http://127.0.0.1:8200"

# Login as root
$ vault login root

# Enable the mock plugin
$ vault secrets enable mock
Success! Enabled the mock secrets engine at: mock/

# Now 'mock' custom secrets engine is enabled
$ vault secrets list

Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_4b92ce62    per-token private secret storage
identity/     identity     identity_7e2789e4     identity store
mock/         mock         mock_4947b778         n/a
...

Each plugin responds to read, write, list, and delete as its own behavior. Let's test the mock secrets engine.

# Write some test data in key-value format
$ vault write mock/test message="Hello World"
Success! Data written to: mock/test

# Read the data
$ vault read mock/test
Key        Value
---        -----
message    Hello World

You can disable the custom secrets engine (mock) just like any other secrets engine.

$ vault secrets disable mock
Success! Disabled the secrets engine (if it existed) at: mock/

Setup Vault

The plugin directory is a configuration option of Vault, and can be specified in the configuration file. This setting specifies a directory that all plugin binaries must live. A plugin can not be added to Vault unless it exists in the plugin directory. There is no default for this configuration option, and if it is not set plugins can not be added to Vault.

Set plugin_directory to the desired path in the Vault configuration file. The path should exist and have proper lockdown on access permissions.

Example:

# Location from which plugins are loaded
plugin_directory = "/etc/vault/vault_plugins"

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault"
}

listener "tcp" {
  address     = "127.0.0.1:8200"
  tls_disable = 1
}

...

The Vault server looks for your custom plugin under plugin_directory. Be sure to move your custom plugins under this location.

If the Vault server is already running, you will need to tell it to reload its configuration by sending SIGHUP. If you stop and start the Vault server, you will need to unseal it again.

Register in Plugin Catalog

Calculate the SHA256 sum of the compiled plugin binary, and use that to register the plugin into Vault's plugin catalog:

$ shasum -a 256 /etc/vault/vault_plugins/my-mock-plugin
2c071aafa1b30897e60b79643e77592cb9d1e8f803025d44a7f9bbfa4779d615  /etc/vault/vault_plugins/my-mock-plugin

$ vault write sys/plugins/catalog/secret/my-mock-plugin \
    sha_256=2c071aafa1b30897e60b79643e77592cb9d1e8f803025d44a7f9bbfa4779d615 \
    command=my-mock-plugin
Success! Data written to: sys/plugins/catalog/my-mock-plugin

Enable Plugin

Enabling the plugin varies depending on if it's a secrets engine or an auth method.

If it's a secrets engine:

$ vault secrets enable -path=<mount_path> <plugin_name>

If it's an auth method:

$ vault auth enable -path=<mount_path> <plugin_name>

Help and Reference