Skip to main content
Type '/' to Search

The Learn website is being redesigned to help you find what you are looking for more effectively.

Write a Policy using Audit Logs

Every Vault operation performed through the command-line interface (CLI), API, or web UI require that the authenticated client is granted access; access defined through policies. Everything in Vault is stored at different paths, like a filesystem, and every action in Vault has a corresponding path and capability. Policies provide a declarative way to grant or forbid access to paths and the capabilites at each path.

In this tutorial, you will learn Vault's policy language and how to query an audit log to define policies.

»Prerequisites

  • Vault

  • jq for consuming Vault JSON formatted output and capturing specific fields.

»Start Vault

Start a Vault dev server with root as the root token.

$ vault server -dev -dev-root-token-id root

$ vault server -dev -dev-root-token-id root

The Vault dev server defaults to running at 127.0.0.1:8200. The server is initialized and unsealed.

Insecure operation: Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.

In another terminal, export an environment variable for the vault CLI to address the Vault server.

$ export VAULT_ADDR=http://127.0.0.1:8200

$ export VAULT_ADDR=http://127.0.0.1:8200

Export an environment variable for the vault CLI to authenticate with the Vault server.

$ export VAULT_TOKEN=root

$ export VAULT_TOKEN=root

The Vault server is ready.

»Enable audit logs

Audit devices keep a detailed log of all requests and response to Vault. The file audit device appends these events to a file.

Enable the file audit device at a file named vault_audit.log in the current directory.

$ vault audit enable file file_path=$PWD/vault_audit.log
Success! Enabled the file audit device at: file/

$ vault audit enable file file_path=$PWD/vault_audit.log
Success! Enabled the file audit device at: file/

The file audit device is enabled and immediately appends a test message to the audit log file.

Display the contents of the audit log file.

$ cat vault_audit.log | jq

$ cat vault_audit.log | jq

The output displays two entries similar to these ones.

{
  "time": "2021-07-20T18:12:25.50223Z",
  "type": "request",
  "auth": {
    "token_type": "default"
  },
  "request": {
    "id": "4c4d2b77-4bca-5768-15b7-e1634458e460",
    "operation": "update",
    "namespace": {
      "id": "root"
    },
    "path": "sys/audit/test"
  }
}
{
  "time": "2021-07-20T18:12:25.502867Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2021-07-20T12:44:43-05:00"
  },
  "request": {
    "id": "64d84eaa-cb06-e30a-d9c2-d89186f81e90",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "client_token_accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "namespace": {
      "id": "root"
    },
    "path": "sys/audit/file",
    "data": {
      "description": "hmac-sha256:8a0b71c85fc92abc3945691f4f4dbbfd60b4f31d787e10db4400419f9c0de4aa",
      "local": false,
      "options": {
        "file_path": "hmac-sha256:cde81abb62e6d3bf0f8cf0b3ac29fbf69154d8282580ca27a9a7fd7d7ad39f77"
      },
      "type": "hmac-sha256:e7e1f0730669ddae084fe59712bf7398d983f977ec7acc8c8a1a7c1065f8b889"
    },
    "remote_address": "127.0.0.1"
  },
  "response": {
    "mount_type": "system"
  }
}

{
  "time": "2021-07-20T18:12:25.50223Z",
  "type": "request",
  "auth": {
    "token_type": "default"
  },
  "request": {
    "id": "4c4d2b77-4bca-5768-15b7-e1634458e460",
    "operation": "update",
    "namespace": {
      "id": "root"
    },
    "path": "sys/audit/test"
  }
}
{
  "time": "2021-07-20T18:12:25.502867Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2021-07-20T12:44:43-05:00"
  },
  "request": {
    "id": "64d84eaa-cb06-e30a-d9c2-d89186f81e90",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "client_token_accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "namespace": {
      "id": "root"
    },
    "path": "sys/audit/file",
    "data": {
      "description": "hmac-sha256:8a0b71c85fc92abc3945691f4f4dbbfd60b4f31d787e10db4400419f9c0de4aa",
      "local": false,
      "options": {
        "file_path": "hmac-sha256:cde81abb62e6d3bf0f8cf0b3ac29fbf69154d8282580ca27a9a7fd7d7ad39f77"
      },
      "type": "hmac-sha256:e7e1f0730669ddae084fe59712bf7398d983f977ec7acc8c8a1a7c1065f8b889"
    },
    "remote_address": "127.0.0.1"
  },
  "response": {
    "mount_type": "system"
  }
}

The audit log appends JSON objects to the log file. The first object describes an audit test message. The second object describes the operation that enabled the audit log.

»Enable transit secrets engine

The transit secrets engine handles cryptographic functions on data in-transit through keys that Vault manages.

Enable the transit secrets engine at the default path.

$ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/

$ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/

The transit secrets engine is enabled and the operation that performed it was written to the audit log.

View only the last entry in the audit log file.

$ cat vault_audit.log | jq -s ".[-1]"

$ cat vault_audit.log | jq -s ".[-1]"

The output displays an entry similar to this one.

{
  "time": "2021-07-20T18:20:20.98874Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "display_name": "token",
    "policies": ["root"],
    "token_policies": ["root"],
    "token_type": "service",
    "token_issue_time": "2021-07-20T12:44:43-05:00"
  },
  "request": {
    "id": "ba9474ed-056c-a6fc-2faf-dfaf92e0b933",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "client_token_accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "namespace": {
      "id": "root"
    },
    "path": "sys/mounts/transit",
    "data": {
      "config": {
        "default_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
        "force_no_cache": false,
        "max_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
        "options": null
      },
      "description": "hmac-sha256:8a0b71c85fc92abc3945691f4f4dbbfd60b4f31d787e10db4400419f9c0de4aa",
      "external_entropy_access": false,
      "local": false,
      "options": null,
      "seal_wrap": false,
      "type": "hmac-sha256:81ef11fde94a36a62a12eb704193bb53d4cacfa13f70303600490303a0418d3a"
    },
    "remote_address": "127.0.0.1"
  },
  "response": {
    "mount_type": "system"
  }
}

{
  "time": "2021-07-20T18:20:20.98874Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "display_name": "token",
    "policies": ["root"],
    "token_policies": ["root"],
    "token_type": "service",
    "token_issue_time": "2021-07-20T12:44:43-05:00"
  },
  "request": {
    "id": "ba9474ed-056c-a6fc-2faf-dfaf92e0b933",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
    "client_token_accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
    "namespace": {
      "id": "root"
    },
    "path": "sys/mounts/transit",
    "data": {
      "config": {
        "default_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
        "force_no_cache": false,
        "max_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
        "options": null
      },
      "description": "hmac-sha256:8a0b71c85fc92abc3945691f4f4dbbfd60b4f31d787e10db4400419f9c0de4aa",
      "external_entropy_access": false,
      "local": false,
      "options": null,
      "seal_wrap": false,
      "type": "hmac-sha256:81ef11fde94a36a62a12eb704193bb53d4cacfa13f70303600490303a0418d3a"
    },
    "remote_address": "127.0.0.1"
  },
  "response": {
    "mount_type": "system"
  }
}

The object describes the time, the authorized token, the request, and the response.

Display the request of the last logged object.

$ cat vault_audit.log | jq -s ".[-1].request"

$ cat vault_audit.log | jq -s ".[-1].request"

The output displays an entry similar to this one.

{
  "id": "ba9474ed-056c-a6fc-2faf-dfaf92e0b933",
  "operation": "update",
  "mount_type": "system",
  "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
  "client_token_accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
  "namespace": {
    "id": "root"
  },
  "path": "sys/mounts/transit",
  "data": {
    "config": {
      "default_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
      "force_no_cache": false,
      "max_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
      "options": null
    },
    "description": "hmac-sha256:8a0b71c85fc92abc3945691f4f4dbbfd60b4f31d787e10db4400419f9c0de4aa",
    "external_entropy_access": false,
    "local": false,
    "options": null,
    "seal_wrap": false,
    "type": "hmac-sha256:81ef11fde94a36a62a12eb704193bb53d4cacfa13f70303600490303a0418d3a"
  },
  "remote_address": "127.0.0.1"
}

{
  "id": "ba9474ed-056c-a6fc-2faf-dfaf92e0b933",
  "operation": "update",
  "mount_type": "system",
  "client_token": "hmac-sha256:72f7f5c240aa19aa00dadc393cbdc8279e542aa141efbf0f46448bd824f6ee88",
  "client_token_accessor": "hmac-sha256:c7a07b88f2a8ddc3109e34324a647108c8248def023e0de488174b3587c8669c",
  "namespace": {
    "id": "root"
  },
  "path": "sys/mounts/transit",
  "data": {
    "config": {
      "default_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
      "force_no_cache": false,
      "max_lease_ttl": "hmac-sha256:e656ba34079dc0f3adde346fafff474f29321b7f678c4c98c1584ef8df0f8c99",
      "options": null
    },
    "description": "hmac-sha256:8a0b71c85fc92abc3945691f4f4dbbfd60b4f31d787e10db4400419f9c0de4aa",
    "external_entropy_access": false,
    "local": false,
    "options": null,
    "seal_wrap": false,
    "type": "hmac-sha256:81ef11fde94a36a62a12eb704193bb53d4cacfa13f70303600490303a0418d3a"
  },
  "remote_address": "127.0.0.1"
}

The request describes the operation that was performed on the path.

Display the request's path and the request's operation.

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"sys/mounts/transit"
"update"

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"sys/mounts/transit"
"update"

The output displays that this operation performed an update on the path sys/mounts/transit.

Define a policy that grants the capability to update at path sys/mounts/transit.

path "sys/mounts/transit" {
  capabilities = ["update"]
}

path "sys/mounts/transit" {
  capabilities = ["update"]
}

To learn more about querying the Audit Log, read the Querying Audit Device Logs tutorial.

»Create an encryption key

Create a encryption key named user-data.

$ vault write -f transit/keys/user-data
Success! Data written to: transit/keys/user-data

$ vault write -f transit/keys/user-data
Success! Data written to: transit/keys/user-data

Display the request path and the request operation of the last logged object.

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/keys/user-data"
"update"

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/keys/user-data"
"update"

»Challenge

Define a policy that grants the capability to update at path transit/keys/user-data.

path "transit/keys/user-data" {
  capabilities = ["update"]
}

path "transit/keys/user-data" {
  capabilities = ["update"]
}

»Encrypt plaintext with the key

The user-data key is capable of encrypting base64 encoded plaintext and returns the ciphertext.

Encrypt user sensitive data with the user-data key.

$ vault write transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data")
Key            Value
---            -----
ciphertext     vault:v1:k1nJTMMmYM2nsYitgHRml6ExwWySGtkqgaOz3JI6kV+IT9995O26qOY5Uec/dn+l
key_version    1

$ vault write transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data")
Key            Value
---            -----
ciphertext     vault:v1:k1nJTMMmYM2nsYitgHRml6ExwWySGtkqgaOz3JI6kV+IT9995O26qOY5Uec/dn+l
key_version    1

Display the request path and the request operation of the last logged object.

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/encrypt/user-data"
"update"

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/encrypt/user-data"
"update"

»Challenge

Define a policy that grants the capability to update at path transit/encrypt/user-data.

path "transit/encrypt/user-data" {
  capabilities = ["update"]
}

path "transit/encrypt/user-data" {
  capabilities = ["update"]
}

»Decrypt ciphertext with the key

The same key is used to decrypt the ciphertext and return the base64 encoded plaintext.

Create a variable named USER_SECRET_DATA that stores the cipher text generated.

$ USER_SECRET_DATA=$(vault write -format=json \
   transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data") \
   | jq -r ".data.ciphertext")

$ USER_SECRET_DATA=$(vault write -format=json \
   transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data") \
   | jq -r ".data.ciphertext")

Decrypt the user sensitive data with the user-data key.

$ vault write transit/decrypt/user-data ciphertext=$USER_SECRET_DATA

$ vault write transit/decrypt/user-data ciphertext=$USER_SECRET_DATA

Display the request path and the request operation of the last logged object.

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/decrypt/user-data"
"update"

$ cat vault_audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/decrypt/user-data"
"update"

Define a policy that grants the capability to update at path transit/decrypt/user-data.

path "transit/decrypt/user-data" {
  capabilities = ["update"]
}

path "transit/decrypt/user-data" {
  capabilities = ["update"]
}

»Next steps

In this tutorial, you performed an operation and then queried Vault's audit logs to discover the path and action. Learn more about querying the Audit Log, read the Querying Audit Device Logs tutorial.

To define policies requires understanding the paths and capabilities of each authentication method and secrets engine. Learn additional approaches in the Write a Policy from API documentation tutorial.

stdin: is not a tty