Security

[Enterprise] KMIP Secrets Engine

Challenge

The OASIS Key Management Interoperability Protocol (KMIP) standard is a widely adopted protocol for handling cryptographic workloads and secrets management for enterprise infrastructure such as databases, network storage, and virtual/physical servers.

When an organization has services and applications that need to perform cryptographic operations (e.g.: transparent database encryption, full disk encryption, etc.), it often delegates the key management task to an external provider via KMIP protocol. As a result, your organization may have existing services or applications that implement KMIP or use wrapper clients with libraries/drivers that implement KMIP. This makes it difficult for an organization to adopt the Vault API in place of KMIP.

Solution

Vault Enterprise v1.2 introduced KMIP secrets engine which allows Vault to act as a KMIP server for clients that retrieve cryptographic keys for encrypting data via KMIP protocol.

Vault KMIP

Vault's KMIP secrets engine manages its own listener to service KMIP requests which operate on KMIP managed objects. Vault policies do not come into play during these KMIP requests. The KMIP secrets engine determines the set of KMIP operations the clients are allowed to perform based on the roles that are applied to the TLS client certificate.

This enables the existing systems to continue using the KMIP APIs instead of Vault APIs.

Prerequisites

To perform the tasks described in this guide, you need to have a Vault Enterprise v1.2 or later.

A KMIP client system for testing such as:

Policy requirements

To perform all tasks demonstrated in this guide, your policy must include the following permissions:

# Work with kmip secrets engine
path "kmip/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Enable secrets engine
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# List enabled secrets engine
path "sys/mounts" {
  capabilities = [ "read", "list" ]
}

If you are not familiar with policies, complete the policies guide.

Step 1: Enable and configure the KMIP secrets engine

First step is to enable the kmip secrets engine at a desired path. In this guide, you are going to enable the kmip secrets engine at /kmip.

CLI command / API call using cURL / Web UI

CLI command

  1. Execute the following command to enable the kmip secrets engine at kmip/.
$ vault secrets enable kmip
  1. Execute the following command to configure the kmip server to listen to port 5696.
$ vault write kmip/config listen_addrs=0.0.0.0:5696
  1. Execute the following command to read back the kmip secrets engine configuration.
$ vault read kmip/config
Key                            Value
---                            -----
default_tls_client_key_bits    256
default_tls_client_key_type    ec
default_tls_client_ttl         336h
listen_addrs                   [0.0.0.0:5696]
server_hostnames               [localhost]
server_ips                     [127.0.0.1 ::1]
tls_ca_key_bits                256
tls_ca_key_type                ec
tls_min_version                tls12

When the KMIP engine is configured, it generates and stores a key-pair and uses it to sign a root CA which is used to sign an intermediate CA. The intermediate CA is used to sign the server TLS certificate used by the KMIP listener to identify itself during the TLS handshake.

By default, the certificate type (default_tls_client_key_type) is ec (Elliptic Curve). If you need RSA instead, specify the certificate type (tls_ca_key_type) and bits (tls_ca_key_bits) when you configure the kmip server.

vault write kmip/config listen_addrs=0.0.0.0:5696 \
      tls_ca_key_type="rsa" \
      tls_ca_key_bits=2048
  1. Execute the following command to retrieve the generated CA certificate:
$ vault read kmip/ca
Key       Value
---       -----
ca_pem    -----BEGIN CERTIFICATE-----
MIICNzCCAZigAwIBAgIUP8qJ5bh/nsBeAh2V61xuBYgf+8swCgYIKoZIzj0EAwIw
HTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDYxOTE5MTMzMloX
DTI5MDYxNjE5MTQwMlowKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWlu
dGVybWVkaWF0ZTCBmzAQBgcqhkjOPQIBBgUrgQQAIwOBhgAEAckgYpJrCbPGdljc
BfefIRR1xKSBjp6rtudm/fZjiY7Pd7sadsOSTyojvmKZHeQdg/G1dUHMSlE+Lhct
AdEkCRzbAJ00TziUh1Ug+xzXo2PBnuSiRWjVcRzDiGPThgjfojKDpm8EF0V6hJ+z
1Z5lDWAL9eqIwKHJTVsTQtf0QU1D6mQ3o2YwZDAOBgNVHQ8BAf8EBAMCAQYwEgYD
VR0TAQH/BAgwBgEB/wIBCTAdBgNVHQ4EFgQUT5Bgc+xJoZcUltEWkBNkokW94M4w
HwYDVR0jBBgwFoAUM1e6hZBDSLFL/DxUUJqIQVZgvNwwCgYIKoZIzj0EAwIDgYwA
MIGIAkIB6rfGWqfeiFl60Ka/dB1/T3evAibMvy4UFsax8DpnFYME5o15+96LOZvy
t5dj9jH72SCDpKNnwekYDZMWb2NKVzYCQgFS0muzu2wZ69FUmkEQBrNuxnTd+4Nt
ha14Uby4Fgq+J3X4GkQBBhsMkGtwwuXuRiEa0WaViILBE+D1Dc/ifDu2qQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIICKTCCAYugAwIBAgIUAh0QJeKDwBO8hYgRk5tdjiOHeVUwCgYIKoZIzj0EAwIw
HTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDYxOTE5MTMzMloX
DTI5MDYxNjE5MTQwMlowHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MIGb
MBAGByqGSM49AgEGBSuBBAAjA4GGAAQBYODGU1+TYhR11Urm6irXz+75VbdsW8pT
o10hw9TR53F+bKIpEzb9dumnr9P80K0Lf4XCwkoewx6IA6oM64eZlOQBQg3Df35A
ovHRU/kzD5IlwSrqEfhqfs53aVeRrGbv256iO6edHLvftzRmb3Ihtpol9/V4vJIo
HpWj/dkoDbSiLaOjZjBkMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/
AgEKMB0GA1UdDgQWBBQzV7qFkENIsUv8PFRQmohBVmC83DAfBgNVHSMEGDAWgBQz
V7qFkENIsUv8PFRQmohBVmC83DAKBggqhkjOPQQDAgOBiwAwgYcCQgDh5iuDhLHh
vH0xAV3pZwbc5jqE8o3Sb5JzoUnmuTX1ZlBbJdZavkQ4HrYbOhI+bHd+iyu5Zwwb
BiOpisPzu9Rr5wJBDhDzgW1+9dqj7oQF4DD+38hLnZKg+F4pZ47dCxdKzzP5MFxc
/zxa8PYxFi62BpmjIKPsyw4U7l0rJ0JBMn3unS8=
-----END CERTIFICATE-----
  1. Copy and save the CA certificate as ca.pem.

API call using cURL

  1. Enable kmip secrets engine using /sys/mounts endpoint:
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"type":"kmip"}' \
       https://127.0.0.1:8200/v1/sys/mounts/kmip
  1. Use kmip/config endpoint to configure the Vault KMIP server to listen to port 5696.
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"listen_addrs": "0.0.0.0:5696"}' \
       https://127.0.0.1:8200/v1/kmip/config
  1. Read the kmip secrets engine configuration.
$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/kmip/config | jq
{
   ...
   "data": {
      "default_tls_client_key_bits": 256,
      "default_tls_client_key_type": "ec",
      "default_tls_client_ttl": 1209600,
      "listen_addrs": [
        "0.0.0.0:5696"
      ],
      "server_hostnames": [
        "localhost"
      ],
      "server_ips": [
        "127.0.0.1",
        "::1"
      ],
      "tls_ca_key_bits": 256,
      "tls_ca_key_type": "ec",
      "tls_min_version": "tls12"
    },
   ...
}

When the KMIP engine is configured, it generates and stores a key-pair and uses it to sign a root CA which is used to sign an intermediate CA. The intermediate CA is used to sign the server TLS certificate used by the KMIP listener to identify itself during the TLS handshake.

By default, the certificate type (default_tls_client_key_type) is ec (Elliptic Curve). If you need RSA instead, specify the certificate type (tls_ca_key_type) and bits (tls_ca_key_bits) in the API request body.

tee payload.json <<EOF
{
  "listen_addrs": "0.0.0.0:5696",
  "tls_ca_key_type": "rsa",
  "tls_ca_key_bits": 2048
}
EOF

curl --header "X-Vault-Token: ..." \
       --request POST \
       --data @payload.json \
       https://127.0.0.1:8200/v1/kmip/config
  1. Invoke the kmip/ca endpoint to retrieve the generated CA certificate:
$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/kmip/ca | jq
{
  ...
  "data": {
    "ca_pem": "-----BEGIN CERTIFICATE-----\nMIICNzCCAZigAwIBAgIUP8qJ5bh/nsBeAh2V61xuBYgf+8swCgYIKoZIzj0EAwIw\nHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDYxOTE5MTMzMloX\nDTI5MDYxNjE5MTQwMlowKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWlu\ndGVybWVkaWF0ZTCBmzAQBgcqhkjOPQIBBgUrgQQAIwOBhgAEAckgYpJrCbPGdljc\nBfefIRR1xKSBjp6rtudm/fZjiY7Pd7sadsOSTyojvmKZHeQdg/G1dUHMSlE+Lhct\nAdEkCRzbAJ00TziUh1Ug+xzXo2PBnuSiRWjVcRzDiGPThgjfojKDpm8EF0V6hJ+z\n1Z5lDWAL9eqIwKHJTVsTQtf0QU1D6mQ3o2YwZDAOBgNVHQ8BAf8EBAMCAQYwEgYD\nVR0TAQH/BAgwBgEB/wIBCTAdBgNVHQ4EFgQUT5Bgc+xJoZcUltEWkBNkokW94M4w\nHwYDVR0jBBgwFoAUM1e6hZBDSLFL/DxUUJqIQVZgvNwwCgYIKoZIzj0EAwIDgYwA\nMIGIAkIB6rfGWqfeiFl60Ka/dB1/T3evAibMvy4UFsax8DpnFYME5o15+96LOZvy\nt5dj9jH72SCDpKNnwekYDZMWb2NKVzYCQgFS0muzu2wZ69FUmkEQBrNuxnTd+4Nt\nha14Uby4Fgq+J3X4GkQBBhsMkGtwwuXuRiEa0WaViILBE+D1Dc/ifDu2qQ==\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIICKTCCAYugAwIBAgIUAh0QJeKDwBO8hYgRk5tdjiOHeVUwCgYIKoZIzj0EAwIw\nHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDYxOTE5MTMzMloX\nDTI5MDYxNjE5MTQwMlowHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MIGb\nMBAGByqGSM49AgEGBSuBBAAjA4GGAAQBYODGU1+TYhR11Urm6irXz+75VbdsW8pT\no10hw9TR53F+bKIpEzb9dumnr9P80K0Lf4XCwkoewx6IA6oM64eZlOQBQg3Df35A\novHRU/kzD5IlwSrqEfhqfs53aVeRrGbv256iO6edHLvftzRmb3Ihtpol9/V4vJIo\nHpWj/dkoDbSiLaOjZjBkMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/\nAgEKMB0GA1UdDgQWBBQzV7qFkENIsUv8PFRQmohBVmC83DAfBgNVHSMEGDAWgBQz\nV7qFkENIsUv8PFRQmohBVmC83DAKBggqhkjOPQQDAgOBiwAwgYcCQgDh5iuDhLHh\nvH0xAV3pZwbc5jqE8o3Sb5JzoUnmuTX1ZlBbJdZavkQ4HrYbOhI+bHd+iyu5Zwwb\nBiOpisPzu9Rr5wJBDhDzgW1+9dqj7oQF4DD+38hLnZKg+F4pZ47dCxdKzzP5MFxc\n/zxa8PYxFi62BpmjIKPsyw4U7l0rJ0JBMn3unS8=\n-----END CERTIFICATE-----"
  },
  ...
}
  1. Copy and save the CA certificate as ca.pem.

Web UI

  1. Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.

  2. Select Enable new engine.

  3. Select KMIP from the list, and then click Next.

  4. Click Enable Engine to complete. This sets the path to be kmip.

  5. Select the Configuration tab, and then click Configure. Config

  6. Set the Listen addrs value to 0.0.0.0:5696, and leave the rest as their defaulted values.

  7. Click Save. Config

  8. Click Download CA cert and save it as ca.pem for this demonstration. CA Cert

Step 2: Create scopes and roles

Scopes partition KMIP managed object storage into multiple named buckets. (NOTE: Scopes cannot be nested.)

Roles in the KMIP secrets engine determine the set of KMIP operations that KMIP clients are allowed to perform.

Scopes and Roles

CLI command / API call using cURL / Web UI

CLI command

  1. Execute the following command to create a new scope named, "finance":
$ vault write -f kmip/scope/finance
  1. Execute the following command to list the existing scopes:
$ vault list kmip/scope
Keys                                                                             .
----                                                                             .
finance
  1. Execute the following command to create a new role, "accounting" under the finance scope:
$ vault write kmip/scope/finance/role/accounting operation_all=true

By setting the operation_all parameter to true, the created role is granted to perform all KMIP client-server operations. Refer to Step 6 to learn how to modify the allowed operation list.

  1. Execute the following command to list existing roles under finance scope:
$ vault list kmip/scope/finance/role
Keys                                                                            .
----                                                                            .
accounting
  1. Execute the following command to read the accounting role definition:
$ vault read kmip/scope/finance/role/accounting
Key                    Value
---                    -----
operation_all          true
tls_client_key_bits    0
tls_client_key_ttl     0s
tls_client_key_type    n/a

API call using cURL

  1. Invoke the kmip/scope endpoint to create a new scope named, "finance":
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       https://127.0.0.1:8200/v1/kmip/scope/finance
  1. List the existing scopes:
$ curl --header "X-Vault-Token: ..." \
       --request LIST \
       https://127.0.0.1:8200/v1/kmip/scope | jq
 {
   ...
   "data": {
     "keys": [
       "finance"
     ]
   },
   ...
 }
  1. Invoke the kmip/scope/<scope_name>/role/<role_name> endpoint to create a new role, "accounting" under the finance scope:
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"operation_all": true}' \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting

By setting the operation_all parameter to true, the created role is granted to perform all KMIP client-server operations. Refer to Step 6 to learn how to modify the allowed operation list.

  1. List existing roles under finance scope:
$ curl --header "X-Vault-Token: ..." \
       --request LIST \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role | jq
 {
   ...
   "data": {
     "keys": [
       "accounting"
     ]
   },
   ...
 }
  1. Read the accounting role definition:
$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting | jq
{
   ...
    "data": {
      "operation_all": true,
      "tls_client_key_bits": 0,
      "tls_client_key_ttl": 0,
      "tls_client_key_type": ""
    },
   ...
}

Web UI

  1. In the UI (e.g. http://127.0.0.1:8200/ui/vault/secrets), select kmip under Secrets.

  2. Select Create a scope.

  3. Enter finance in the Name field, and click Save.

  4. Select the newly created finance scope, and click Create a role.

  5. Enter accounting in the Name field. Allow this role to perform all operations should be enabled.

  6. Click Save.

Step 3: Client certificate generation

Once the scope was created and a role was created under the scope, the next step is to get the certification for the KMIP client.

CLI command / API call using cURL / Web UI

CLI command

  1. Execute the following command to generate a certificate in PEM format, and save it in a JSON file named, credential.json:
$ vault write -format=json kmip/scope/finance/role/accounting/credential/generate \
        cert_format=pem > creds.json
  1. Extract necessary certificate information from the credential.json using jq tool.
# Extract certificate and save it in cert.pem
$ jq -r .data.certificate < creds.json > cert.pem

# Extract private key and save it in key.pem
$ jq -r .data.private_key < creds.json > key.pem

The resulting cert.pem and key.pem is what your KMIP client will use to establish a connection with Vault KMIP server.

  1. Execute the following command to list the certification serial numbers associated with the accounting role:
$ vault list kmip/scope/finance/role/accounting/credential
Keys                                                                             .
----                                                                             .
693751915900546682090704263335075174345458639865

Copy the returned serial number (e.g. 693751915900546682090704263335075174345458639865).

  1. You can now lookup the client certificate by its serial number as shown in the example. (NOTE: Be sure to pass the serial_number you discovered.)

Example:

$ vault read kmip/scope/finance/role/accounting/credential/lookup \
        serial_number=693751915900546682090704263335075174345458639865
Key              Value
---              -----
ca_chain         [-----BEGIN CERTIFICATE-----
MIIBrDCCAVKgAwIBAgIUA462iIHn2ssIOwZTFDzMaWK8veIwCgYIKoZIzj0EAwIw
HTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX
DTI5MDcyNzE5NDYzOFowKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWlu
dGVybWVkaWF0ZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKpAQgXZZQ5YSXZ7
QiDaSXrbig7AT5xqKw4Cpos1RHNnQtQmFzj4VJdIJfFF3j7+iXjg/4DfQEvsgjfk
OPsR5FSjYzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBTXOnbANc7zQbeXut8z/gW6z1D9+zAfBgNVHSMEGDAWgBRo5cF5kF7WN4Dp
MjlRbvJoRqgNHzAKBggqhkjOPQQDAgNIADBFAiA3W9E5Q40/Ys1CgXgrDx1ywIJm
u7JZ8pg0mahQ60jItwIhALLnHRVXfIXKYGouRCwJ6tZeEYCZXL5SC6W6r5fZcJq7
-----END CERTIFICATE----- -----BEGIN CERTIFICATE-----
MIIBoDCCAUWgAwIBAgIUH/kEhPmsA19HwWyaUe5+6MbSNPwwCgYIKoZIzj0EAwIw
HTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX
DTI5MDcyNzE5NDYzOFowHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MFkw
EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/IIHo7wm0G5ywwsU9I2/fzfcjEac8k+K
satRSL71/SxY4Af4GiBdVHSNqTv/QEq3kfe4ShKQvK0tGo2xjxu39KNjMGEwDgYD
VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFGjlwXmQXtY3
gOkyOVFu8mhGqA0fMB8GA1UdIwQYMBaAFGjlwXmQXtY3gOkyOVFu8mhGqA0fMAoG
CCqGSM49BAMCA0kAMEYCIQCxhqAELYdXfIi7H8yJ6RCaNRntaHbHwqxn6UB4fnEc
HQIhAM5qsuyvbp6U8CH+ejtbHjzzgO5rhXbchx7Um2gWKiEQ
-----END CERTIFICATE-----]
certificate      -----BEGIN CERTIFICATE-----
MIIBszCCAVmgAwIBAgIUT4XoB0JqAPoADTK3lb1NbJKUk3QwCgYIKoZIzj0EAwIw
KjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWludGVybWVkaWF0ZTAeFw0x
OTA3MzAyMDI5MzdaFw0xOTA4MTMyMDMwMDdaMCAxDjAMBgNVBAsTBUtJRUdXMQ4w
DAYDVQQDEwU5akpiZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABPM977vYKmIy
UDTNlWJhQ+3poZrEYt/bH1t0GpUinfHHBSifkG0v/boM85BOLku8S/zURZRQlXXa
D6FONeSHCmWjZzBlMA4GA1UdDwEB/wQEAwIDqDATBgNVHSUEDDAKBggrBgEFBQcD
AjAdBgNVHQ4EFgQUlyQPSXDXzarQ4uD87xIHsQs8BJwwHwYDVR0jBBgwFoAU1zp2
wDXO80G3l7rfM/4Fus9Q/fswCgYIKoZIzj0EAwIDSAAwRQIgRm8doJMK5Wy46fMW
2iqUfn5cykVF0h/78mKts3/Vp5YCIQDJBfh5kGmDZKTCLAZeiiSd07mkF56FzIK1
2HFT4nBZCg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIBrDCCAVKgAwIBAgIUA462iIHn2ssIOwZTFDzMaWK8veIwCgYIKoZIzj0EAwIw
HTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX
DTI5MDcyNzE5NDYzOFowKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWlu
dGVybWVkaWF0ZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKpAQgXZZQ5YSXZ7
QiDaSXrbig7AT5xqKw4Cpos1RHNnQtQmFzj4VJdIJfFF3j7+iXjg/4DfQEvsgjfk
OPsR5FSjYzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBTXOnbANc7zQbeXut8z/gW6z1D9+zAfBgNVHSMEGDAWgBRo5cF5kF7WN4Dp
MjlRbvJoRqgNHzAKBggqhkjOPQQDAgNIADBFAiA3W9E5Q40/Ys1CgXgrDx1ywIJm
u7JZ8pg0mahQ60jItwIhALLnHRVXfIXKYGouRCwJ6tZeEYCZXL5SC6W6r5fZcJq7
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIBoDCCAUWgAwIBAgIUH/kEhPmsA19HwWyaUe5+6MbSNPwwCgYIKoZIzj0EAwIw
HTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX
DTI5MDcyNzE5NDYzOFowHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MFkw
EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/IIHo7wm0G5ywwsU9I2/fzfcjEac8k+K
satRSL71/SxY4Af4GiBdVHSNqTv/QEq3kfe4ShKQvK0tGo2xjxu39KNjMGEwDgYD
VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFGjlwXmQXtY3
gOkyOVFu8mhGqA0fMB8GA1UdIwQYMBaAFGjlwXmQXtY3gOkyOVFu8mhGqA0fMAoG
CCqGSM49BAMCA0kAMEYCIQCxhqAELYdXfIi7H8yJ6RCaNRntaHbHwqxn6UB4fnEc
HQIhAM5qsuyvbp6U8CH+ejtbHjzzgO5rhXbchx7Um2gWKiEQ
-----END CERTIFICATE-----
serial_number    693751915900546682090704263335075174345458639865

API call using cURL

  1. Use the kmip/scope/<scope_name>/role/<role_name>/credential/generate endpoint to generate a certificate in PEM format, and save it in a JSON file named, credential.json:
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"cert_format": "pem"}' \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting/credential/generate \
       > credential.json
  1. Extract necessary certificate information from the credential.json using jq tool.
# Extract certificate and save it in cert.pem
$ jq -r .data.certificate < creds.json > cert.pem

# Extract private key and save it in key.pem
$ jq -r .data.private_key < creds.json > key.pem

The resulting cert.pem and key.pem is what your KMIP client will use to establish a connection with Vault KMIP server.

  1. Use the kmip/scope/<scope_name>/role/<role_name>/credential endpoint to list the certification serial numbers associated with the accounting role:
$ curl --header "X-Vault-Token: ..." \
       --request LIST \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting/credential | jq
{
  ...
  "data": {
    "keys": [
      "592363246796376126217488264600013543918961277098"
    ]
  },
  ...
}

Copy the returned serial number.

  1. You can use the kmip/scope/<scope_name>/role/<role_name>/credential/lookup endpoint to lookup the client certificate by its serial number.

    Supply the serial_number you discovered at the end of the lookup endpoint: kmip/scope/<scope_name>/role/<role_name>/credential/lookup?serial_number=<serial_number>

Example:

$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting/credential/lookup?serial_number=592363246796376126217488264600013543918961277098 | jq
{
  ...
  "data": {
    "ca_chain": [
      "-----BEGIN CERTIFICATE-----\nMIIBrDCCAVKgAwIBAgIUA462iIHn2ssIOwZTFDzMaWK8veIwCgYIKoZIzj0EAwIw\nHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX\nDTI5MDcyNzE5NDYzOFowKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWlu\ndGVybWVkaWF0ZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKpAQgXZZQ5YSXZ7\nQiDaSXrbig7AT5xqKw4Cpos1RHNnQtQmFzj4VJdIJfFF3j7+iXjg/4DfQEvsgjfk\nOPsR5FSjYzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud\nDgQWBBTXOnbANc7zQbeXut8z/gW6z1D9+zAfBgNVHSMEGDAWgBRo5cF5kF7WN4Dp\nMjlRbvJoRqgNHzAKBggqhkjOPQQDAgNIADBFAiA3W9E5Q40/Ys1CgXgrDx1ywIJm\nu7JZ8pg0mahQ60jItwIhALLnHRVXfIXKYGouRCwJ6tZeEYCZXL5SC6W6r5fZcJq7\n-----END CERTIFICATE-----",
      "-----BEGIN CERTIFICATE-----\nMIIBoDCCAUWgAwIBAgIUH/kEhPmsA19HwWyaUe5+6MbSNPwwCgYIKoZIzj0EAwIw\nHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX\nDTI5MDcyNzE5NDYzOFowHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MFkw\nEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/IIHo7wm0G5ywwsU9I2/fzfcjEac8k+K\nsatRSL71/SxY4Af4GiBdVHSNqTv/QEq3kfe4ShKQvK0tGo2xjxu39KNjMGEwDgYD\nVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFGjlwXmQXtY3\ngOkyOVFu8mhGqA0fMB8GA1UdIwQYMBaAFGjlwXmQXtY3gOkyOVFu8mhGqA0fMAoG\nCCqGSM49BAMCA0kAMEYCIQCxhqAELYdXfIi7H8yJ6RCaNRntaHbHwqxn6UB4fnEc\nHQIhAM5qsuyvbp6U8CH+ejtbHjzzgO5rhXbchx7Um2gWKiEQ\n-----END CERTIFICATE-----"
    ],
    "certificate": "-----BEGIN CERTIFICATE-----\nMIIBszCCAVmgAwIBAgIUT4XoB0JqAPoADTK3lb1NbJKUk3QwCgYIKoZIzj0EAwIw\nKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWludGVybWVkaWF0ZTAeFw0x\nOTA3MzAyMDI5MzdaFw0xOTA4MTMyMDMwMDdaMCAxDjAMBgNVBAsTBUtJRUdXMQ4w\nDAYDVQQDEwU5akpiZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABPM977vYKmIy\nUDTNlWJhQ+3poZrEYt/bH1t0GpUinfHHBSifkG0v/boM85BOLku8S/zURZRQlXXa\nD6FONeSHCmWjZzBlMA4GA1UdDwEB/wQEAwIDqDATBgNVHSUEDDAKBggrBgEFBQcD\nAjAdBgNVHQ4EFgQUlyQPSXDXzarQ4uD87xIHsQs8BJwwHwYDVR0jBBgwFoAU1zp2\nwDXO80G3l7rfM/4Fus9Q/fswCgYIKoZIzj0EAwIDSAAwRQIgRm8doJMK5Wy46fMW\n2iqUfn5cykVF0h/78mKts3/Vp5YCIQDJBfh5kGmDZKTCLAZeiiSd07mkF56FzIK1\n2HFT4nBZCg==\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIBrDCCAVKgAwIBAgIUA462iIHn2ssIOwZTFDzMaWK8veIwCgYIKoZIzj0EAwIw\nHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX\nDTI5MDcyNzE5NDYzOFowKjEoMCYGA1UEAxMfdmF1bHQta21pcC1kZWZhdWx0LWlu\ndGVybWVkaWF0ZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKpAQgXZZQ5YSXZ7\nQiDaSXrbig7AT5xqKw4Cpos1RHNnQtQmFzj4VJdIJfFF3j7+iXjg/4DfQEvsgjfk\nOPsR5FSjYzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud\nDgQWBBTXOnbANc7zQbeXut8z/gW6z1D9+zAfBgNVHSMEGDAWgBRo5cF5kF7WN4Dp\nMjlRbvJoRqgNHzAKBggqhkjOPQQDAgNIADBFAiA3W9E5Q40/Ys1CgXgrDx1ywIJm\nu7JZ8pg0mahQ60jItwIhALLnHRVXfIXKYGouRCwJ6tZeEYCZXL5SC6W6r5fZcJq7\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIBoDCCAUWgAwIBAgIUH/kEhPmsA19HwWyaUe5+6MbSNPwwCgYIKoZIzj0EAwIw\nHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MB4XDTE5MDczMDE5NDYwOFoX\nDTI5MDcyNzE5NDYzOFowHTEbMBkGA1UEAxMSdmF1bHQta21pcC1kZWZhdWx0MFkw\nEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/IIHo7wm0G5ywwsU9I2/fzfcjEac8k+K\nsatRSL71/SxY4Af4GiBdVHSNqTv/QEq3kfe4ShKQvK0tGo2xjxu39KNjMGEwDgYD\nVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFGjlwXmQXtY3\ngOkyOVFu8mhGqA0fMB8GA1UdIwQYMBaAFGjlwXmQXtY3gOkyOVFu8mhGqA0fMAoG\nCCqGSM49BAMCA0kAMEYCIQCxhqAELYdXfIi7H8yJ6RCaNRntaHbHwqxn6UB4fnEc\nHQIhAM5qsuyvbp6U8CH+ejtbHjzzgO5rhXbchx7Um2gWKiEQ\n-----END CERTIFICATE-----",
    "serial_number": "453996482527080083111899601311492596313963664244"
  },
  ...
}

Web UI

  1. Under Secrets (e.g. http://127.0.0.1:8200/ui/vault/secrets), select kmip > finance > accounting in the UI.

  2. Select Generate credentials.

  3. The Certificate format is set to pem.

  4. Click Save.

  5. Select Copy certificate to copy the generated certificate. Save it in a file named, cert.pem.

  6. Also, click on the copy to clipboard icon to copy the private key. Save the copied private key in the key.pem file. Generate Certificates

  7. Select Back to role and notice that the serial number of the client certificate you just generated is listed. Copy the serial number.

Step 4: KMIP client configuration

In Step 1, you saved the CA certificate in a file named, ca.pem. You created a role named, "accounting" with all supported KMIP operations allowed. Based on the role, a client certificate was generated and saved in a file, cert.pem and its private key in key.pem in Step 3.

MongoDB Enterprise Advanced

This guide assumes that you already have an instance of MongoDB Enterprise Advanced running.

Use mongod daemon process to configure the KMIP settings.

# First, combine the cert.pem and key.pem and save it as client.pem
$ cat cert.pem key.pem > client.pem

# Execute the mongod command to specify the cert & key file locations
$ mongod --dbpath mongodata --enableEncryption --kmipServerName localhost \
         --kmipPort 5696 --kmipServerCAFile ca.pem \
         --kmipClientCertificateFile client.pem

The command output shows the created KMIP key ID:

...
[initandlisten] options: { security: { enableEncryption: true, kmip: { clientCertificateFile: "client.pem", port: 5696, serverCAFile: "ca.pem", serverName: "localhost" } }, storage: { dbPath: "mongodata" } }
...
[initandlisten] Created KMIP key with id: RklNh5GRRQCZ4PLzPYuV5BUm2HuWwbGv
...
[initandlisten] Encryption key manager initialized using KMIP key with id: RklNh5GRRQCZ4PLzPYuV5BUm2HuWwbGv.

MySQL Enterprise Edition

This guide assumes that you already have an instance of MySQL Enterprise Edition running with your own license.

The keyring_okv keyring plugin uses the KMIP 1.1 protocol to communicate securely as a client of a KMIP back end. Follow the MySQL online documentation to create okvclient.ora file, and an ssl directory to place the certificate and key files for keyring_ova to work properly.

Also, install the keyring_udf plugin and the user-defined functions (UDFs) as described in the MySQL documentation.

Now, generate a new random key with a given ID, type, and length, and store it in the keyring.

mysql> SELECT keyring_key_generate('MyKey1', 'AES', 32);
+-------------------------------------------+
| keyring_key_generate('MyKey1', 'AES', 32) |
+-------------------------------------------+
|                                         1 |
+-------------------------------------------+
1 row in set (0.35 sec)

Step 5: Revoke a client certificate

Vault make it easy to revoke the existing client certificates if necessary.

CLI command / API call using cURL / Web UI

CLI command

Execute the following command to revoke a client certificate by its serial number. (NOTE: Be sure to pass the serial_number for your system.)

Example:

$ vault write kmip/scope/finance/role/accounting/credential/revoke \
        serial_number="187621554749885282282062389325090392369479437576"

# Verify that it no longer exists
$ vault list kmip/scope/finance/role/accounting/credential

No value found at kmip/scope/finance/role/accounting/credential/

API call using cURL

Use the kmip/scope/<scope_name>/role/<role_name>/credential/revoke endpoint to revoke a client certificate by its serial number. (NOTE: Be sure to pass the serial_number for your system.)

Example:

# Revoke a certification
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"serial_number": "187621554749885282282062389325090392369479437576"}' \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting/credential/revoke

# Verification - list serial numbers.  This should return an empty array
$ curl --header "X-Vault-Token: ..." \
      --request LIST \
      https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting/credential | jq
{
 "errors": []
}

Web UI

  1. Under Secrets (e.g. http://127.0.0.1:8200/ui/vault/secrets), select kmip > finance > accounting in the UI.

  2. Select the serial number to view the client certificate.

  3. Select the Revoke credentials. At the confirmation dialog, click Revoke.

Step 6: Manage KMIP roles

In Step 2, the accounting role was permitted to perform all KMIP operations (operation_all). However, you should practice the least privileged and avoid granting more permissions than necessary.

When you grant KMIP operations, the argument convention is operation_<KMIP_operation>. For example, to grant a Create operation, the parameter becomes operation_create.

CLI command / API call using cURL / Web UI

CLI command

  1. Execute the following command to update the accounting role and grant only Activate, Create, and Get operations.
# List a new set of operations to permit
$ vault write kmip/scope/finance/role/accounting \
        operation_activate=true \
        operation_create=true \
        operation_get=true

# Verify the changes
$ vault read kmip/scope/finance/role/accounting
Key                    Value
---                    -----
operation_activate     true
operation_create       true
operation_get          true
tls_client_key_bits    0
tls_client_key_ttl     0s
tls_client_key_type    n/a
  1. To delete the accounting role, execute the following command:
$ vault delete kmip/scope/finance/role/accounting

API call using cURL

  1. Update the accounting role and grant only Activate, Create, and Get operations.
# Create the HTTP request payload
$ tee payload.json <<EOF
{
  "operation_activate": true,
  "operation_create": true,
  "operation_get": true
}
EOF

# Update the accounting role
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data @payload.json \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting

# Verify the changes
$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting | jq
{
   ...
   "data": {
      "operation_activate": true,
      "operation_create": true,
      "operation_get": true,
      "tls_client_key_bits": 0,
      "tls_client_key_ttl": 0,
      "tls_client_key_type": ""
   },
   ...
}
  1. To delete the accounting role, invoke the following endpoint:

    $ curl --header "X-Vault-Token: ..." \
           --request DELETE \
           https://127.0.0.1:8200/v1/kmip/scope/finance/role/accounting
    

Web UI

  1. Under Secrets, select kmip > finance > accounting in the UI.

  2. Select Details and then Edit role.

  3. Deselect the Allow this role to perform all operations check box and then select the check-box for Create, Activate and Get.

  4. Select Save.

  5. To delete the accounting role, select Delete role. At the confirmation dialog, select Delete.

Step 7: Delete scopes

If you need to delete scopes, follow this step.

CLI command / API call using cURL / Web UI

CLI command

Execute the following command to delete scope, finance:

$ vault delete kmip/scope/finance

NOTE: Pass -f or -force flag to allow deleting a scope which has some managed objects created with.

API call using cURL

Delete scope, finance:

$ curl --header "X-Vault-Token: ..." \
       --request DELETE \
       https://127.0.0.1:8200/v1/kmip/scope/finance

Web UI

  1. Under Secrets (e.g. http://127.0.0.1:8200/ui/vault/secrets), select kmip in the UI.

  2. Expand the menu for finance scope, and select Delete scope. At the confirmation dialog, click Delete.

Additional Discussion

As it was discussed, Vault policies do not come into play during these KMIP requests. The KMIP secrets engine determines the set of KMIP operations the clients are allowed to perform based on the roles (accounting).

Let's take a look at the generated client certificate file:

$ openssl x509 -in cert.pem -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            37:78:04:36:13:b2:0a:35:0a:5c:f8:6e:d8:ef:b7:b5:52:ff:ad:8d
    Signature Algorithm: ecdsa-with-SHA256
        Issuer: CN=vault-kmip-default-intermediate
        Validity
            Not Before: Jun 25 17:12:40 2019 GMT
            Not After : Jun 26 17:13:10 2019 GMT
        Subject: OU=5J7Ck, CN=ImYFE
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (521 bit)
                pub:
                    04:01:d3:21:de:6b:b7:ed:36:1e:c4:05:48:39:3d:
                    ce:16:01:e3:11:6e:34:b0:69:bb:aa:35:7f:7e:f5:
                    24:1d:9d:ca:eb:4b:66:41:00:5e:0d:21:91:b5:c7:
                    f4:db:3f:b3:fc:fa:d9:d9:10:10:7c:41:53:45:51:
                    aa:e9:e4:a0:5a:46:25:01:f5:75:69:15:12:0e:bd:
                    4d:5a:9e:a8:f3:19:e1:7a:42:0a:cf:2d:43:bf:b4:
                    43:17:3b:eb:85:55:64:65:c1:7a:19:11:06:33:81:
                    88:0e:49:28:34:d1:a9:76:df:b5:d1:46:a3:16:42:
                    72:25:22:84:cf:ae:55:eb:5e:88:55:e6:fe
                ASN1 OID: secp521r1
                NIST CURVE: P-521
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment, Key Agreement
            X509v3 Extended Key Usage:
                TLS Web Client Authentication
            X509v3 Subject Key Identifier:
                F3:54:4D:DF:F8:CA:DA:80:6D:E1:A5:12:F3:2D:4B:23:42:68:31:64
            X509v3 Authority Key Identifier:
                keyid:B0:DF:DC:07:50:52:F3:67:7F:AA:E3:30:08:CF:62:3C:B9:3E:12:3F

    Signature Algorithm: ecdsa-with-SHA256
         30:81:87:02:41:3f:5a:fc:83:a2:d8:3e:f0:9d:48:aa:7a:8d:
         42:47:b8:d4:dc:74:5d:7b:34:b2:a9:c3:ff:93:8d:30:a8:58:
         64:4a:86:7f:7f:64:7c:25:df:67:ca:2f:f3:d4:50:d4:6d:a9:
         56:35:bc:ae:eb:40:7d:23:4f:00:60:a8:45:32:48:3d:02:42:
         01:12:e1:50:b8:45:5a:55:d8:1f:0a:41:34:47:19:67:01:66:
         22:b8:13:6c:db:d0:65:e0:83:18:ac:e4:28:e1:28:30:1e:94:
         96:43:2d:46:e7:da:a8:51:9e:1f:6f:20:7f:49:d2:bc:7c:ff:
         a9:7b:8e:ad:9d:05:10:b2:41:bd:6f:d5

Notice the Subject properties: Subject: OU=5J7Ck, CN=ImYFE

The OU corresponds to the scope (finance) and CN corresponds to the role (accounting). If you have access to the Vault's storage backend, you would find persisted data with those IDs.

For example:

"key": "vault/logical/<UUID>/scope/5J7Ck/role/ImYFE"
"key": "vault/logical/<UUID>/scope/5J7Ck/info"
"key": "vault/logical/<UUID>/managed-objects/5J7Ck/RklNh5GRRQCZ4PLzPYuV5BUm2HuWwbGv"

You can read the allowed operations for the role using the sys/raw endpoint if it's enabled.

$ vault read sys/raw/logical/<UUID>/scope/5J7Ck/role/ImYFE

Key      Value
---      -----
value    {"id":"ImYFE","allowed_operations":{"operation_all":{}}}

When a situation requires you to delete the client certificate, simply revoke the certificate as described in Step 5.

Help and Reference