Oct 19-21 HashiConf Global is live. Join Now
Type '/' to Search

Azure Secrets Engine

This tutorial also appears in: Interactive.

Dynamic secrets are a core feature in Vault. A class of dynamic secrets is on-demand, revocable, time-limited access credentials for cloud providers. For example, the Dynamic Secrets getting started tutorial demonstrated the AWS secrets engine to dynamically generate AWS credentials (access key ID and secret access key).

»Challenge

To consume Azure services (e.g. Azure Kubernetes service), the client must have valid Azure credentials. Azure uses service principal to authenticate its users. An Azure service principal is an identity created for use with applications, hosted services, and automated tools to access Azure resources. So each new application adds operational overhead as more service principals are required.

»Solution

Automate the process by integrating your applications with Vault's Azure secrets engine. The applications ask Vault for Azure credential with a time-to-live (TTL) enforcing its validity so that the credentials are automatically revoked when they are no longer used.

»Benefits

Each app instance can request unique, short-lived credentials. Unique credentials ensures isolated, auditable access and enable revocation of a single client. While short-lived reduces the time frame in which they are valid.

This tutorial demonstrates the use of azure secrets engine to dynamically generate azure credentials.

»Personas

The end-to-end scenario described in this tutorial involves two personas:

  • admin with privileged permissions to configure secrets engines
  • apps read the secrets from Vault

»Prerequisites

This tutorial assumes the following:

  • You have a Microsoft Azure account
  • A Vault environment of version 0.11 or later

Online tutorial: An interactive tutorial is also available if you do not wish to install the following resources. Click the Show Terminal button to start.

»Policy requirements

Each persona requires a different set of capabilities. These are expressed in policies. If you are not familiar with policies, complete the policies tutorial.

The admin tasks require these capabilities.

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the azure secrets engine and create roles
path "azure/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

# Mount secrets enginespath "sys/mounts/*" {  capabilities = [ "create", "read", "update", "delete", "list" ]}
# Configure the azure secrets engine and create rolespath "azure/*" {  capabilities = [ "create", "read", "update", "delete", "list" ]}
# Write ACL policiespath "sys/policies/acl/*" {  capabilities = [ "create", "read", "update", "delete", "list" ]}
# Manage tokens for verificationpath "auth/token/create" {  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]}

The apps tasks require these capabilities.

path "azure/creds/edu-app" {
  capabilities = [ "read" ]
}

path "azure/creds/edu-app" {  capabilities = [ "read" ]}

»Create an Azure service principal and resource group

(Persona: admin)

To delegate the credential generation task to Vault, you need to give Vault privileged Azure credentials to perform the task. The following demonstrates the creation of a service principal.

NOTE: Refer to the online Azure documentation for more details.

  1. Launch the Microsoft Azure Portal and sign in.

  2. Select Azure Active Directory and select Properties. Azure Active Directory

  3. Copy the Tenant ID. Tenant ID

  4. In a terminal, set the variable TENANT_ID to the Tenant ID.

    $ TENANT_ID=<Tenant ID>

    $ TENANT_ID=<Tenant ID>

  5. From the side navigation, select App registrations. App registrations

  6. Select New registrations. New App registration

  7. Enter education in the Name field.

  8. Click Register.

  9. Copy the Application (client) ID. Client ID

  10. In a terminal, set the variable CLIENT_ID to the Application (client) ID.

    $ CLIENT_ID=<Client ID>

    $ CLIENT_ID=<Client ID>

  11. From the side navigation, select Certificate & secrets.

  12. Under the Client secrets, click New client secret.

  13. Enter a description in the Description field. Azure portal

  14. Click Add.

  15. Copy the client secret value. Azure directory application secret value

  16. In a terminal, set the variable CLIENT_SECRET to the client secret value.

    $ CLIENT_SECRET=<Client secret>

    $ CLIENT_SECRET=<Client secret>

  17. From the side navigation, click API permissions. Azure directory application API permissions

  18. Under Configured permissions, click Add a permission. Azure directory application add a permission

  19. Under Supported legacy APIs, click Azure Active Directory Graph. Azure directory app perms legacy

  20. Click Delegated permissions. Azure Active Directory Graph

  21. Expand User, select the check-box for User.Read.

  22. Click Application permissions. Azure directory app application permissions

  23. Expand Application, select the check-box for Application.ReadWrite.All.

  24. Expand Directory, select the check-box for Directory.ReadWrite.All.

  25. Click API permissions.

  26. Click Grant admin consent for azure to grant the permissions. Azure directory app permissions grant admin consent

  27. Navigate to the Subscriptions blade.

  28. Copy the Subscription ID. Subscription ID

  29. In a terminal, set the variable SUBSCRIPTION_ID to the Subscription ID.

    $ SUBSCRIPTION_ID=<Subscription ID>

    $ SUBSCRIPTION_ID=<Subscription ID>

  30. Click the name of the subscription.

  31. From the side navigation, click Access control (IAM). Azure subscription access control

  32. Click Add to expand the menu.

  33. Click Add a role assignment. Azure subscription access add role assignment

  34. Choose Owner from the Role select field.

  35. Choose User, group, or service principal from the Assign Access To select field.

  36. Enter the application name or application id in the Select field.

  37. Click the application when it is displayed. Azure subscription access role assignment to app

  38. Click Save.

The application is created with the correct permissions and you have these identifiers and credentials:

  • Tenant ID
  • Client ID
  • Client Secret
  • Subscription ID

The secrets engine generates credentials within an Azure resource group.

  1. Launch the Microsoft Azure Portal and sign in.

  2. Navigate to Resource groups.

  3. Click Create. Resource group create button

  4. Choose the subscription from the Subscription select field.

  5. Enter vault-education in the Resource group field. Create resource group vault-education

  6. Click Review + create.

    The view changes to display the review page.

  7. Click Create.

The resource group vault-education is created.

»Start Vault

In another terminal, start a Vault dev server with root as the root token.

$ vault server -dev -dev-root-token-id root

$ vault server -dev -dev-root-token-id root

The Vault dev server defaults to running at 127.0.0.1:8200. The server is initialized and unsealed.

Insecure operation: Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.

Export an environment variable for the vault CLI to address the Vault server.

$ export VAULT_ADDR=http://127.0.0.1:8200

$ export VAULT_ADDR=http://127.0.0.1:8200

Export an environment variable for the vault CLI to authenticate with the Vault server.

$ export VAULT_TOKEN=root

$ export VAULT_TOKEN=root

NOTE: For these tasks, you can use Vault's root token. However, it is recommended that root tokens are only used for enough initial setup or in emergencies. As a best practice, use an authentication method or token that meets the policy requirements.

The Vault server is ready.

»Enable the Azure secrets engine

The Azure secrets engine dynamically generates Azure service principals.

Enable the azure secrets engine at its default path.

$ vault secrets enable azure

$ vault secrets enable azure

The secrets engine is enabled at the path azure/. To enable the secrets engine at a different path requires that you use the -path parameter and the desired path.

»Configure the Azure secrets engine

(Persona: admin)

The Azure secrets engine requires the credentials you generated in the create an Azure service principal and resource group step to communicate with Azure and generate service principals.

Verify that your Azure subscription ID is stored in the SUBSCRIPTION_ID environment variable.

$ echo $SUBSCRIPTION_ID

$ echo $SUBSCRIPTION_ID

Verify that your client ID is stored in the CLIENT_ID environment variable.

$ echo $CLIENT_ID

$ echo $CLIENT_ID

Verify that your client secret is stored in the CLIENT_SECRET environment variable.

$ echo $CLIENT_SECRET

$ echo $CLIENT_SECRET

Verify that your tenant ID is stored in the TENANT_ID environment variable.

$ echo $TENANT_ID

$ echo $TENANT_ID

If any of those variables are missing their value, refer to the previous step and set them before proceeding.

Configure the Azure secrets engine with the Azure credentials.

$ vault write azure/config \
        subscription_id=$SUBSCRIPTION_ID  \
        client_id=$CLIENT_ID \
        client_secret=$CLIENT_SECRET \
        tenant_id=$TENANT_ID

$ vault write azure/config \        subscription_id=$SUBSCRIPTION_ID  \        client_id=$CLIENT_ID \        client_secret=$CLIENT_SECRET \        tenant_id=$TENANT_ID

»Create a role

(Persona: admin)

A Vault role lets you configure either an existing service principal or a set of Azure roles.

Create a Vault role named, edu-app mapped to the Azure role named, Contributor in the vault-education resource group.

$ vault write azure/roles/edu-app ttl=1h azure_roles=-<<EOF
    [
      {
        "role_name": "Contributor",
        "scope": "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/vault-education"
      }
    ]
EOF

$ vault write azure/roles/edu-app ttl=1h azure_roles=-<<EOF    [      {        "role_name": "Contributor",        "scope": "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/vault-education"      }    ]EOF

The role named edu-app is created.

»Request Azure credentials

The role generates credentials with a time-to-live (TTL) of 1 hour and max TTL of 24 hours.

Read credentials from the edu-app azure role.

$ vault read azure/creds/edu-app

Key                Value
---                -----
lease_id           azure/creds/edu-app/AeY3ckPy2pByrVtArNeQQDjn
lease_duration     1h
lease_renewable    true
client_id          408bf248-dd4e-XXXX-XXXX-XXXXXXXXXXXX
client_secret      ad06228a-2db9-XXXX-XXXX-XXXXXXXXXXXX

$ vault read azure/creds/edu-app
Key                Value---                -----lease_id           azure/creds/edu-app/AeY3ckPy2pByrVtArNeQQDjnlease_duration     1hlease_renewable    trueclient_id          408bf248-dd4e-XXXX-XXXX-XXXXXXXXXXXXclient_secret      ad06228a-2db9-XXXX-XXXX-XXXXXXXXXXXX

The results display the credentials, its TTL, and the lease ID.

For applications (apps persona) to request credentials require a Vault policy that grants access to this role.

Define a policy named apps.

$ vault policy write apps - <<EOF
path "azure/creds/edu-app" {
  capabilities = [ "read" ]
}
EOF
Policy 'apps' written.

$ vault policy write apps - <<EOFpath "azure/creds/edu-app" {  capabilities = [ "read" ]}EOFPolicy 'apps' written.

The apps policy grants the read capability for requests to the path azure/creds/edu-app.

Create a variable named APPS_TOKEN to capture the token created with the apps policy attached.

$ APPS_TOKEN=$(vault token create -policy=apps -field=token)

$ APPS_TOKEN=$(vault token create -policy=apps -field=token)

NOTE: AppRole Pull Authentication tutorial demonstrates a more sophisticated way of generating a token for your apps.

Read credentials from the edu-app azure role with the APPS_TOKEN.

$ VAULT_TOKEN=$APPS_TOKEN vault read azure/creds/edu-app

Key                Value
---                -----
lease_id           azure/creds/edu-app/AeY3ckPy2pByrVtArNeQQDjn
lease_duration     1h
lease_renewable    true
client_id          408bf248-dd4e-XXXX-XXXX-XXXXXXXXXXXX
client_secret      ad06228a-2db9-XXXX-XXXX-XXXXXXXXXXXX

$ VAULT_TOKEN=$APPS_TOKEN vault read azure/creds/edu-app
Key                Value---                -----lease_id           azure/creds/edu-app/AeY3ckPy2pByrVtArNeQQDjnlease_duration     1hlease_renewable    trueclient_id          408bf248-dd4e-XXXX-XXXX-XXXXXXXXXXXXclient_secret      ad06228a-2db9-XXXX-XXXX-XXXXXXXXXXXX

The results display the credentials, its TTL, and the lease ID. The credentials for this application (service principal) in the Azure Portal searching by its client_id.

NOTE: Re-run the command and notice that the role returns a new set of credentials. This means that each app instance acquires a unique set of Azure credentials.

»Manage leases

(Persona: admin)

The credentials are managed by the lease ID and remain valid for the lease duration (TTL) or until revoked. Once revoked the credentials are no longer valid.

List the existing leases.

$ vault list sys/leases/lookup/azure/creds/edu-app

Keys
----
o2F4EA3hU8Fpjgc39XyQpjtU

$ vault list sys/leases/lookup/azure/creds/edu-app
Keys----o2F4EA3hU8Fpjgc39XyQpjtU

All valid leases for Azure credentials are displayed.

Create a variable that stores the first lease ID.

$ LEASE_ID=$(vault list -format=json sys/leases/lookup/azure/creds/edu-app | jq -r ".[0]")

$ LEASE_ID=$(vault list -format=json sys/leases/lookup/azure/creds/edu-app | jq -r ".[0]")

Renew the lease for the Azure credential by passing its lease ID.

$ vault lease renew azure/creds/edu-app/$LEASE_ID
Key                Value
---                -----
lease_id           azure/creds/edu-app/o2F4EA3hU8Fpjgc39XyQpjtU
lease_duration     1h
lease_renewable    true

$ vault lease renew azure/creds/edu-app/$LEASE_IDKey                Value---                -----lease_id           azure/creds/edu-app/o2F4EA3hU8Fpjgc39XyQpjtUlease_duration     1hlease_renewable    true

The TTL of the renewed lease is set to 1h.

Revoke the lease without waiting for its expiration.

$ vault lease revoke azure/creds/edu-app/$LEASE_ID
All revocation operations queued successfully!

$ vault lease revoke azure/creds/edu-app/$LEASE_IDAll revocation operations queued successfully!

List the existing leases.

$ vault list sys/leases/lookup/azure/creds/edu-app
No value found at sys/leases/lookup/azure/creds/edu-app/

$ vault list sys/leases/lookup/azure/creds/edu-appNo value found at sys/leases/lookup/azure/creds/edu-app/

The lease is no longer valid and is not displayed.

Read new credentials from the edu-app role.

$ vault read azure/creds/edu-app

$ vault read azure/creds/edu-app

All leases associated with a path may be removed.

Revoke all the leases with the prefix azure/creds/edu-app.

$ vault lease revoke -prefix azure/creds/edu-app

$ vault lease revoke -prefix azure/creds/edu-app

The prefix flag matches all valid leases with the path prefix of azure/creds/edu-app.

List the existing leases.

$ vault list sys/leases/lookup/azure/creds/edu-app
No value found at sys/leases/lookup/azure/creds/edu-app

$ vault list sys/leases/lookup/azure/creds/edu-appNo value found at sys/leases/lookup/azure/creds/edu-app

All the leases with this path as a prefix have been revoked.

»Clean up

The Azure credentials created to configure the secrets engine should be deleted if they are no longer requried.

  1. Launch the Microsoft Azure Portal and sign in.

  2. Navigate to Azure Active Directory.

  3. From the side navigation, select App registrations.

  4. Click the education application.

  5. From the application overview, click delete.

  6. Select Yes to delete the application.

The application is deleted.

The secrets engine generated credentials within the vault-education Azure resource group. Any credentials that were not revoked should be deleted if they are no longer required.

  1. Launch the Microsoft Azure Portal and sign in.

  2. Navigate to Resource groups.

  3. Click the vault-education resource group.

  4. From the resource group overview, click Delete resource group.

  5. Enter vault-education in the TYPE THE RESOURCE GROUP NAME: field.

  6. Click Delete.

The resource group is deleted.

»Help and Reference

You enabled and configured the Azure secrets engine. Learn more by exploring the documentation or API documentation.

Azure Key Vault can auto-unseal the Vault server. Learn this through this Azure Key Vault Auto-unseal & Dynamic Secrets with HashiCorp Vault demonstation.

stdin: is not a tty