Secrets Management

Azure Secrets Engine

Dynamic secrets are a core feature in Vault. A class of dynamic secrets is on-demand, revocable, time-limited access credentials for cloud providers. For example, the Dynamic Secrets getting started guide demonstrated the AWS secrets engine to dynamically generate AWS credentials (access key ID and secret access key).

Challenge

To consume Azure services (e.g. Azure Kubernetes service), the client must have valid Azure credentials. Azure uses service principal to authenticate its users. An Azure service principal is an identity created for use with applications, hosted services, and automated tools to access Azure resources.

As the number of applications required to access Azure resources, it adds operational overhead to manage those service principals generated for each application.

Solution

Automate the process by integrating your applications with Vault's Azure secrets engine. The applications ask Vault for Azure credential with a time-to-live (TTL) enforcing its validity so that the credentials are automatically revoked when they are no longer used.

Benefits

Each app instance can get unique credentials that they don't have to share. By making those credentials to be short-lived, you reduced the chance of the Azure credentials from being compromised. If an app was compromised, the credentials used by the app can be revoked rather than changing a more global set of credentials.

Personas

The end-to-end scenario described in this guide involves two personas:

  • admin with privileged permissions to configure secrets engines
  • apps read the secrets from Vault

Prerequisites

This guide assumes the following:

  • You have a Microsoft Azure account
  • A Vault environment of version 0.11 or later

Policy requirements

To perform all tasks demonstrated in this guide, your policy must include the following permissions:

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the azure secrets engine and create roles
path "azure/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

If you are not familiar with policies, complete the policies guide.

Step 1: Create an Azure Service Principal

(Persona: admin)

To delegate the credential generation task to Vault, you need to give Vault privileged Azure credentials to perform the task. The following demonstrates the creation of a service principal.

  1. Launch the Microsoft Azure Portal and sign in.

  2. Select Azure Active Directory and select Properties.

  3. Copy and store the Directory ID which is your tenant ID for later use. Tenant ID

  4. Now, select App registrations.

  5. Select New registrations.

  6. Enter your desired application name in the Name text field, and then click Register.

  7. Copy and store the generated Application (client) ID for later use. Client ID

  8. Select Certificate & secrets.

  9. Click New client secret under the Client secrets. When prompted, enter some description and then click Add. Azure
portal

  10. Copy and store the generated secret value which is your client secret.

  11. Click API permissions and Add a permission.

  12. Select Azure Active Directory Graph under the Supported legacy APIs section.

  13. Click Delegated permissions, expand User and then select the check-box for User.Read. Azure Active Directory Graph

  14. Click Application permissions, expand Application and Directory.

  15. Select the check-box for Application.ReadWrite.All and Directory.ReadWrite.All.

  16. Click API permissions.

  17. Click Grant admin consent for azure to grant the permissions. API
permissions

  18. Navigate to the Subscriptions service blade. Copy and store the SUBSCRIPTION ID for later use. Subscription
ID

  19. Now, click into your Subscription name.

  20. Click Access control (IAM) and click Add under the Add a role assignment. Access control

  21. Select Owner under Role, and the Assign Access To Field should be Azure ID, User Group, or Service Principal. In the Select field, enter your application name, or Application (client) ID saved in a previous step to discover the application you created.

  22. Click Save.

Step 2: Enable the Azure secrets engine

First step is to enable the azure secrets engine at a desired path.

CLI command / API call using cURL / Web UI

CLI command

Execute the following command to enable the azure secrets engine at azure/ path.

$ vault secrets enable azure

If you want to enable the secrets engine at a different path, use the -path parameter to specify your desired path. Otherwise, the secrets engine gets enabled at a path named after its type. Since the type is azure in this example, its path becomes azure in absence of -path parameter.

API call using cURL

Mount azure secrets engine using /sys/mounts endpoint:

curl --header "X-Vault-Token: <TOKEN>" \
       --request POST \
       --data <PARAMETERS> \
       <VAULT_ADDRESS>/v1/sys/mounts/<PATH>

Where <TOKEN> is your valid token, and <PARAMETERS> holds configuration parameters of the secrets engine.

The following command example mounts azure secrets engine at sys/mounts/azure path, and passed the secrets engine type ("azure") in the request payload.

$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"type":"azure"}' \
       https://127.0.0.1:8200/v1/sys/mounts/azure

Web UI

  1. Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.

  2. Select Enable new engine.

  3. Select Azure from the list, and then click Next.

  4. Click Enable Engine to complete. This sets the path to be azure.

Step 3: Configure the Azure secrets engine

(Persona: admin)

To configure the Azure secrets engine, be sure that you have the following information you collected in Step 1 ready:

  • Tenant ID (or Directory ID)
  • Client ID
  • Client Secrets
  • Subscription ID

CLI command / API call using cURL

CLI command

Execute the following command to configure the Azure secrets engine to use the service principal you created.

$ vault write azure/config subscription_id=<Subscription_ID>  \
        client_id=<Client_ID> client_secret=<Client_Secret> \
        tenant_id=<Tenant_ID>

API call using cURL

Execute the following command to configure the Azure secrets engine to use the service principal you created.

# Create an HTTP request payload
$ tee payload.json <<EOF
{
  "subscription_id": "<Subscription_ID>",
  "tenant_id": "<Tenant_ID>",
  "client_id": "<Client_ID>",
  "client_secret": "<Client_Secret>",
}
EOF

# Invoke the azure/config endpoint
$ curl --header "X-Vault-Token: ..." --request POST --data @payload.json \
       http://127.0.0.1:8200/v1/azure/config

Step 3: Create a role

(Persona: admin)

The next step is to define what type of service principal you want Vault to generate. Vault roles let you configure either an existing service principal or a set of Azure roles.

CLI command / API call using cURL

CLI command

Execute the following command to create a role named, edu-app with Contributor role on the vault-education resource group. Set the TTL to be 1 hour and the max TTL to be 24 hours.

$ vault write azure/roles/edu-app ttl=1h azure_roles=-<<EOF
    [
      {
        "role_name": "Contributor",
        "scope": "/subscriptions/<Subscription_ID>/resourceGroups/vault-education"
      }
    ]
EOF

Be sure to replace the <Subscription_ID> with your Subscription ID copied in Step 1.

API call using cURL

Execute the following command to create a role named, edu-app with Contributor role on the vault-education resource group. Set the TTL to be 1 hour and the max TTL to be 24 hours.

# Create the HTTP request payload
$ tee payload.json <<EOF
{
  "azure_roles": "[
    {
      \"role_name\": \"Contributor\",
      \"scope\":  \"/subscriptions/<Subscription_ID>/resourceGroup/vault-education\"
    }
  ]",
  "ttl": 3600,
  "max_ttl": "24h"
}
EOF

# Invoke the azure/roles API endpoint
$ curl --header "X-Vault-Token: ..." \
       --request POST --data @payload.json \
       http://127.0.0.1:8200/v1/azure/roles/edu-app

Be sure to replace the <Subscription_ID> with your Subscription ID copied in Step 1.

Step 4: Request Azure credentials

(Persona: apps)

Now, you are switching to apps persona. To get a new set of Azure credentials, the client applications need to be able to read from the edu-app role endpoint. Therefore the app's token must have a policy granting the read permission.

Create a new file called apps-policy.hcl:

$ tee apps-policy.hcl <<EOF
# Get credentials from the azure secrets engine
path "azure/creds/edu-app" {
  capabilities = [ "read" ]
}
EOF

CLI command / API call using cURL / Web UI

CLI command

First create an apps policy, and generate a token so that you can authenticate as an apps persona.

Example:

# Create "apps" policy
$ vault policy write apps apps-policy.hcl
Policy 'apps' written.

# Create a new token with app policy
$ vault token create -policy=apps

Key                  Value
---                  -----
token                s.dVZocexPgYSuMOUYb5xepG7O
token_accessor       ki48gCn3m8TMzAju3Dd8Er4m
token_duration       768h
token_renewable      true
token_policies       ["apps" "default"]
identity_policies    []
policies             ["apps" "default"]

Use the returned token to perform the remaining.

# Invoke the vault command with apps token
$ VAULT_TOKEN=s.dVZocexPgYSuMOUYb5xepG7O vault read azure/creds/edu-app

Key                Value
---                -----
lease_id           azure/creds/edu-app/AeY3ckPy2pByrVtArNeQQDjn
lease_duration     1h
lease_renewable    true
client_id          408bf248-dd4e-XXXX-XXXX-XXXXXXXXXXXX
client_secret      ad06228a-2db9-XXXX-XXXX-XXXXXXXXXXXX

You should be able to discover the application (service principal) in the Azure Portal searching by its client_id.

API call using cURL

First create an apps policy, and generate a token so that you can authenticate as an app persona.

# Create the HTTP request payload
$ tee payload.json <<EOF
{
  "policy": "path \"azure/creds/edu-app\" {capabilities = [ \"read\" ]}"
}
EOF

# Create "apps" policy
$ curl --header "X-Vault-Token: ..." --request PUT \
       --data @payload.json \
       http://127.0.0.1:8200/v1/sys/policies/acl/apps

# Generate a new token with apps policy
$ curl --header "X-Vault-Token: ..." --request POST \
       --data '{"policies": ["apps"]}' \
       http://127.0.0.1:8200/v1/auth/token/create | jq
{
   ...
   "auth": {
     "client_token": "s.qbloIGiHddxc3chdtE2TrGX7",
     "accessor": "8zrb4SIVJ4DMZUsfr6uRsqdu",
     "policies": [
       "apps",
       "default"
     ],
     "token_policies": [
       "apps",
       "default"
     ],
     ...
   }
}

Be sure to use the returned token to perform the remaining.

$ curl --header "X-Vault-Token: s.qbloIGiHddxc3chdtE2TrGX7" \
       --request GET \
       http://127.0.0.1:8200/v1/azure/creds/edu-app | jq
{
  "request_id": "787a8f6e-ccee-f330-de19-2c5ecd752878",
  "lease_id": "azure/creds/edu-app/SPfKsENQ7smTRQxuVhUb3Dfb",
  "renewable": true,
  "lease_duration": 3600,
  "data": {
    "client_id": "2e411b50-2176-XXXX-XXXX-XXXXXXXXXXXX",
    "client_secret": "bf8aa058-c37b-XXXX-XXXX-XXXXXXXXXXXX"
  },
  ...
}

You should be able to discover the application (service principal) in the Azure Portal searching by its client_id.

Web UI

  1. Click the Policies tab, and then select Create ACL policy.

  2. Toggle Upload file sliding switch, and click Choose a file to select your apps-policy.hcl file you authored. This loads the policy and sets the Name to be apps.

  3. Click Create Policy to complete.

  4. Click the Vault CLI shell icon (>_) to open a command shell. Execute vault write auth/token/create policies=apps in the CLI shell to create a new token: Create token

  5. Copy and save the generated client token value.

  6. Sign out of the Vault UI.

  7. Now, sign into the Vault using the newly generated token you just copied.

  8. Click the Vault CLI shell icon (>_) to open a command shell. Execute vault read azure/creds/edu-app in the CLI shell.

You should be able to discover the application (service principal) in the Azure Portal searching by its client_id.

Step 5: Revoke the Azure credentials

(Persona: admin)

The generated Azure credentials get automatically revoked by the Vault server after 1 hour of its creation and not being renewed. If you need to revoke the credentials before reaching its TTL, you can do so by invoking the revoke operation.

CLI command / API call using cURL

CLI command

Execute the following command to revoke the generated Azure credentials before reaching its TTL.

$ vault lease revoke azure/creds/edu-app/SPfKsENQ7smTRQxuVhUb3Dfb
All revocation operations queued successfully!

While azure/creds/edu-app/SPfKsENQ7smTRQxuVhUb3Dfb is the lease_id associated with the generated Azure credentials.

Alternatively, you can revoke all edu-app credentials by executing the following command:

$ vault lease revoke -prefix azure/creds/edu-app
All revocation operations queued successfully!

API call using cURL

Execute the following API endpoint to revoke the generated Azure credentials before reaching its TTL.

$ curl --header "X-Vault-Token: ..." \
       --request PUT \
       --data '{"lease_id": "azure/creds/edu-app/SPfKsENQ7smTRQxuVhUb3Dfb"}'
       http://127.0.0.1:8200/v1/sys/leases/revoke

While azure/creds/edu-app/SPfKsENQ7smTRQxuVhUb3Dfb is the lease_id associated with the generated Azure credentials.

Alternatively, you can revoke all edu-app credentials by executing the following command:

$ curl --header "X-Vault-Token: ..." \
       --request PUT \
       http://127.0.0.1:8200/v1/sys/leases/revoke-prefix/azure/creds/edu-app

Help and Reference