Secrets Management

Active Directory Service Account Check-out

Vault 0.10.2 introduced the Active Directory (AD) secrets engine which was designed to rotate the shared AD passwords dynamically. This allowed companies to reduce the risk of damage from a password leak.

Challenge

A service account is a user account that is created explicitly to provide a security context for services running on Windows Server operating systems. Often, an organization has a limited number of service accounts based on their Client Access License (CAL) which they share among the team. As a result, credential rotation is a cumbersome process and it's difficult to audit who took what actions with those shared credentials.

Solution

Use the AD service account check-in/check-out feature of the AD secrets engine to allow a team to share a select set of service accounts. To use the shared service accounts, the requester first checks out the account. When done using the account, they simply check the service account back in for others to use. Whenever a service account is checked back in, Vault rotates its password.

This minimizes the number of accounts consumed and ensures that as an organization grows, they do not expand beyond the limit of their Client Access License. At the same time, the credentials are valid per check-out and user actions can be tied to whoever had a service account checked out. This lowers the operational burden and helps an organization meet its security requirements around audibility.

Personas

The end-to-end scenario described in this guide involves two personas:

  • admin with privileged permissions to configure secrets engines
  • client requests to check-out service account(s)

Prerequisites

This guide assumes that you have the following environments:

  • Microsoft Active Directory (AD) server with service accounts
  • Vault version 1.3.0 or later

Policy requirements

To perform all tasks demonstrated in this guide, your policy must include the following permissions:

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the ad secrets engine and create roles
path "ad/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage the leases
path "sys/leases/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

If you are not familiar with policies, complete the policies guide.

Scenario Introduction

You are going to peform the following operations:

  1. Enable and configure Active Directory secrets engine
  2. Create a library
  3. Create a client policy
  4. Check-out service accounts

Scenario description

Also, this guide covers the following:

Step 1: Enable and configure Active Directory secrets engine

The first step is to enable and configure the AD secrets engine at a desired path.

CLI command / API call using cURL

CLI command

  1. Execute the following command to enable the AD secrets engine at ad/ path.

    $ vault secrets enable ad
    
    Success! Enabled the ad secrets engine at: ad/
    

    If you want to enable the secrets engine at a different path, use the -path parameter to specify your desired path. Otherwise, the secrets engine gets enabled at a path named after its type. Since the type is ad in this example, its path becomes ad in absence of -path parameter.

  2. To configure the AD secrets engine, you need to set the following parameters.

    parameterDescription
    binddnDistinguished name of object to bind when performing user and group search
    bindpassPassword to use along with binddn when performing user search
    urlThe AD sever URL
    userdnBase DN under which to perform user search

    This is a subset of available parameters. Refer to the API documentation for the full list of configuration parameters.

    Execute the vault write ad/config command with correct AD server configuration information.

    Example:

    $ vault write ad/config binddn='CN=vault-ad-test,CN=Users,DC=example,DC=com' \
            bindpass='pa$$w0rd' \
            url=ldaps://127.0.0.11 \
            userdn='dc=example,dc=com' \
            insecure_tls=true
    
    Success! Data written to: ad/config
    

API call using cURL

  1. The following command example enable ad secrets engine at sys/mounts/ad path.

    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           --data '{"type":"ad"}' \
           http://127.0.0.1:8200/v1/sys/mounts/ad
    

    Where the X-Vault-Token header value is set to your valid token.

    In this guide, the AD secrets engine is enabled at the /ad path in Vault. However, it is possible to enable your secrets engines at any location.

  2. To configure the AD secrets engine, the following parameters will be set.

    parameterDescription
    binddnDistinguished name of object to bind when performing user and group search
    bindpassPassword to use along with binddn when performing user search
    urlThe AD sever URL
    userdnBase DN under which to perform user search

    This is a subset of available parameters. Refer to the API documentation for the full list of configuration parameters.

    Execute the ad/config endpoint to configure the AD secrets engine.

    Example:

    # Create an HTTP request payload
    $ tee payload.json <<EOF
    {
      "binddn": "CN=vault-ad-test,CN=Users,DC=example,DC=com",
      "bindpass": "pa$$w0rd",
      "url": "ldaps://127.0.0.11",
      "userdn": "dc=example,dc=com",
      "insecure_tls": true
    }
    EOF
    
    # Invoke the ad/config endpoint
    $ curl --header "X-Vault-Token: ..." --request POST \
           --data @payload.json \
           http://127.0.0.1:8200/v1/ad/config
    

Step 2: Create a library

(Persona: admin)

A library is a set of service accounts that you want Vault to manage for check-out.

To demonstrate this feature, assume that you have shared service accounts, testMSA@example.com and MSA4SQL@example.com.

In this step, you are going to create a library named, accounting-team where testMSA@example.com and MSA4SQL@example.com are the shared accounts that can be checked out for use.

You can control how long a single check-out can last by setting the ttl and max_ttl associated with the library. Vault will automatically check-in the service account once its TTL expires and not renewed, or its max TTL was reached and no longer renewable.

CLI command / API call using cURL

CLI command

  1. Execute the following command to create a library named accounting-team with a ttl set to 24 hours and max_ttl set to 5 days.

    $ vault write ad/library/accounting-team \
            service_account_names="testMSA@example.com,MSA4SQL@example.com" \
            ttl=24h \
            max_ttl=120h
    
    Success! Data written to: ad/library/accounting-team
    
  2. Execute the following command to list the existing libraries.

    $ vault list ad/library
    
    Keys
    ----
    accounting-team
    
  3. Execute the following command to view the status of the accounts in the library.

    $ vault read ad/library/accounting-team/status
    
    Key                 Value
    ---                 -----
    MSA4SQL@example.com    map[available:true]
    testMSA@example.com    map[available:true]
    

    Both service accounts are available for check-out.

API call using cURL

  1. Execute the ad/library endpoint to create a library named, accounting-team with ttl is set to 24 hours and max_ttl is set to 5 days.

    # Create the HTTP request payload
    $ tee payload.json <<EOF
    {
      "service_account_names": ["testMSA@example.com", "MSA4SQL@example.com"],
      "ttl": "24h",
      "max_ttl": "120h"
    }
    EOF
    
    # Invoke the ad/library API endpoint
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           --data @payload.json \
           http://127.0.0.1:8200/v1/ad/library/accounting-team
    
  2. Execute the ad/library endpoint to list the existing libraries.

    $ curl --header "X-Vault-Token: ..." \
           --request LIST
           http://127.0.0.1:8200/v1/ad/library | jq
    {
      ...
      "data": {
        "keys": [
          "accounting-team"
        ]
      },
      ...
    }
    
  3. Execute the ad/library/accounting-team/status endpoint to view the status of the accounts in the library.

    $ curl --header "X-Vault-Token: ..." \
           http://127.0.0.1:8200/v1/ad/library/accounting-team/status | jq
    {
      ...
      "data": {
        "MSA4SQL@example.com": {
          "available": true
        },
        "testMSA@example.com": {
          "available": true
        }
      },
      ...
    }
    

Step 3: Create a client policy

(Persona: admin)

In order for the client to check-out the service accounts belong to the accounting-team library, the policy must include the following rules.

acct-team.hcl:

# To check-out a service account which is a part of accounting-team library
path "ad/library/accounting-team/check-out" {
  capabilities = [ "update" ]
}

# To allow the extension of TTL
path "sys/leases/renew" {
  capabilities = [ "update" ]
}

# To check back in a service account
path "ad/library/accounting-team/check-in" {
  capabilities = [ "update" ]
}

# If you want to allow the client to see the library status
path "ad/library/accounting-team/status" {
  capabilities = [ "read" ]
}

CLI command / API call using cURL / Web UI

CLI command

Execute the following command to ceate a policy named, acct-team:

$ vault policy write acct-team acct-team.hcl

Success! Uploaded policy: acct-team

API call using cURL

Execute the sys/policies/acl endpoint to ceate a policy named, acct-team:

# Create the API request payload. Use stringified policy expression.
$ tee policy-payload.json <<EOF
{
  "policy": "# To check-out a service account which is a part of accounting-team library\npath \"ad/library/accounting-team/check-out\" {\n  capabilities = [ \"update\" ]\n}\n\n# To allow the extension of TTL\npath \"sys/leases/renew\" {\n  capabilities = [ \"update\" ]\n}\n\n# To check back in a service account\npath \"ad/library/accounting-team/check-in\" {\n  capabilities = [ \"update\" ]\n}\n\n# If you want to allow the client to see the library status\npath \"ad/library/accounting-team/status\" {\n  capabilities = [ \"read\" ]\n}"
}
EOF

# Create admin policy
$ curl --header "X-Vault-Token: ..." \
       --request PUT \
       --data @policy-payload.json \
       http://127.0.0.1:8200/v1/sys/policies/acl/acct-team

Web UI

  1. Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.

  2. Click the Policies tab, and then select Create ACL policy.

  3. Toggle the Upload file sliding switch, and click Choose a file to select your acct-team.hcl file you authored.

  4. Click Create Policy to complete.

Step 4: Check-out service accounts

(Persona: client)

Once the AD secrets engine is configured by the Vault admin, a client (may be a human user, application or system) can request to check-out a service account.

CLI command / API call using cURL

CLI command

  1. First, generate a client token with acct-team policy attached and store it in a VAULT_TOKEN environment variable.

    Example:

    $ vault token create -policy=acct-team
    
    Key                  Value
    ---                  -----
    token                s.1jJFckALyEShvimC7kxizd3Y
    token_accessor       JiFqVsH7PKfGeIX8McFqwGgO
    token_duration       768h
    token_renewable      true
    token_policies       ["acct-team" "default"]
    identity_policies    []
    policies             ["acct-team" "default"]
    
    # Store the generated client token as VAULT_TOKEN env var
    $ export VAULT_TOKEN=s.1jJFckALyEShvimC7kxizd3Y
    
  2. Check-out a service account managed by the accounting-team library.

    $ vault write -f ad/library/accounting-team/check-out
    
    Key                     Value
    ---                     -----
    lease_id                ad/library/accounting-team/check-out/SVNimFfpLKwWThHq1eGBG5mj
    lease_duration          768h
    lease_renewable         true
    password                ?@09AZFIwvunQIThl0RTMI1o3gTGZ7hinlQuUaBJMIPTURin3Ub0n9zkRSgLr0vM
    service_account_name    testMSA@example.com
    

    Now, the testMSA@example.com account has been checked out and its password has been dynamically generated by Vault.

  3. View the library status.

    $ vault read ad/library/accounting-team/status
    
    Key                 Value
    ---                 -----
    MSA4SQL@example.com    map[available:true]
    testMSA@example.com    map[available:false borrower_client_token:49aef0b628c06d77d4237da3cedc6ec29e0e51ae]
    

    It is shown that testMSA@example.com is now checked out.

  4. When you are done, check back in the service account by executing the following command:

    $ vault write -f ad/library/accounting-team/check-in
    
    Key          Value
    ---          -----
    check_ins    [testMSA@example.com]
    

    NOTE: If you have more than one accounts checked out, you must pass the service_account_names parameter to specify which account(s) to check in.

API call using cURL

  1. First, generate a client token with acct-team policy attached and store it in a VAULT_TOKEN environment variable.

    $ curl --request POST --header "X-Vault-Token: ..." \
           --data '{ "policies": "acct-team" }' \
           http://127.0.0.1:8200/v1/auth/token/create
    {
      ...
      "auth": {
        "client_token": "s.7hO8OHNg6YMi4YYAIjMzRo6Y",
        "accessor": "wGqT2H3MM6CVzIF57wCIYCLg",
        "policies": [
          "acct-team",
          "default"
        ],
        "token_policies": [
          "acct-team",
          "default"
        ],
        ...
      }
    }
    
  2. Check-out a service account managed by the accounting-team library.

    $ curl --header "X-Vault-Token: s.7hO8OHNg6YMi4YYAIjMzRo6Y \
           --request POST \
           http://127.0.0.1:8200/v1/ad/library/accounting-team/check-out | jq
    {
      "request_id": "5e468a48-0316-158e-8e03-9e171a8a0f48",
      "lease_id": "ad/library/accounting-team/check-out/HR2AsbSQbNVQCSzx6WTRCLqH",
      "renewable": true,
      "lease_duration": 86400,
      "data": {
        "password": "?@09AZ6Jane6QAjf54kgEpz6R/7ggOKLT93cxpbn2kaY20DfzX7bIKtUbLsl0Y6c",
        "service_account_name": "testMSA@example.com"
      },
      ...
    }
    

    Now, the testMSA@example.com account has been checked out and its password has been dynamically generated by Vault.

  3. View the library status.

    $ curl --header "X-Vault-Token: ..." \
           http://127.0.0.1:8200/v1/ad/library/accounting-team/status | jq
    {
      ...
      "data": {
        "MSA4SQL@example.com": {
          "available": true
        },
        "testMSA@example.com": {
          "available": false,
          "borrower_client_token": "8ee2c9a26f187d88e068f1631a1acd183484287d"
        }
      },
      ...
    }
    

    It is shown that testMSA@example.com is now checked out.

  4. When you are done, check back in the service account by invoking the ad/library/accounting-team/check-in endpoint.

    $ curl --header "X-Vault-Token: s.7hO8OHNg6YMi4YYAIjMzRo6Y" \
             --request POST \
             http://127.0.0.1:8200/v1/ad/library/accounting-team/check-in
    {
      ...
      "data": {
        "check_ins": [
          "testMSA@example.com"
        ]
      },
      ...
    }
    

    NOTE: If you have more than one accounts checked out, you must pass the service_account_names parameter to specify which account(s) to check in.

Step 5: Understanding the TTL

(Persona: admin)

In this step, you are going to test and see how the TTL works for those checked out service accounts.

For testing, update the ttl and max_ttl set on the accounting-team library to be 20 seconds and 40 seconds respectively. Observe what happens when the TTL expires.

CLI command / API call using cURL

CLI command

  1. Log back in with your admin token.

  2. Execute the following command to update the ttl and max_ttl:

    $ vault write ad/library/accounting-team \
            service_account_names="testMSA@example.com,MSA4SQL@example.com" \
            ttl=20 \
            max_ttl=40
    
  3. Check-out a service account, wait for 20 seconds and then try renewing the lease.

    $ vault write -f ad/library/accounting-team/check-out
    
    Key                     Value
    ---                     -----
    lease_id                ad/library/accounting-team/check-out/cp4X5rcwQrp3w8f4lMj2jhTo
    lease_duration          20s
    lease_renewable         true
    password                ?@09AZBbCyiLZtx+DPKYducATJ3HH/kZe2BuzCIC+oDpPpoD71PQW9L5iLaLahah
    service_account_name    testMSA@example.com
    
    # Wait for 20 seconds
    # Try renewing the TTL using the lease_id
    $ vault lease renew ad/library/accounting-team/check-out/cp4X5rcwQrp3w8f4lMj2jhTo
    

    While ad/library/accounting-team/check-out/cp4X5rcwQrp3w8f4lMj2jhTo is the lease_id.

    The lease renew command returns lease not found error.

  4. Check the library status.

    $ vault read ad/library/accounting-team/status
    
    Key                 Value
    ---                 -----
    MSA4SQL@example.com    map[available:true]
    testMSA@example.com    map[available:true]
    

    Both service accounts are available for check-out. This is because Vault automatically checked in testMSA@example.com after the TTL expired.

API call using cURL

  1. Execute the ad/config endpoint to update the ttl and max_ttl set on the accounting-team library to be 20 seconds and 40 seconds respectively.

    # Create the HTTP request payload
    $ tee payload.json <<EOF
    {
      "service_account_names": ["testMSA@example.com", "MSA4SQL@example.com"],
      "ttl": "20s",
      "max_ttl": "40s"
    }
    EOF
    
    # Invoke the ad/library API endpoint
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           --data @payload.json \
           http://127.0.0.1:8200/v1/ad/library/accounting-team
    
  2. Check-out a service account, wait for 20 seconds and then try renewing the lease.

    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           http://127.0.0.1:8200/v1/ad/library/accounting-team/check-out | jq
    {
      "request_id": "b8130438-3651-e17c-bf23-da203a554768",
      "lease_id": "ad/library/accounting-team/check-out/MJJfJyBSG1Wfh1FqiGxA3pWN",
      "renewable": true,
      "lease_duration": 20,
      "data": {
        "password": "?@09AZn1H2P0pUngP7hPeG3QNgo5BrA0/Dx0fI6gDaCBuadfRUw6zmQqZ5MSF9B8",
        "service_account_name": "testMSA@example.com"
      },
      ...
    }
    
    # Wait for 20 seconds
    # Try renewing the TTL using the lease_id
    $ curl --header "X-Vault-Token: ..." \
           --request POST \
           --data '{ "lease_id": "ad/library/accounting-team/check-out/MJJfJyBSG1Wfh1FqiGxA3pWN"}' \
           http://127.0.0.1:8200/v1/sys/lease/renew | jq
    {
      "errors": [
        "lease not found"
      ]
    }
    

    While ad/library/accounting-team/check-out/MJJfJyBSG1Wfh1FqiGxA3pWN is the lease_id.

  3. Execute the ad/library/accounting-team/status endpoint to view the status of the accounts in the library.

    $ curl --header "X-Vault-Token: ..." \
           http://127.0.0.1:8200/v1/ad/library/accounting-team/status | jq
    {
      ...
      "data": {
        "MSA4SQL@example.com": {
          "available": true
        },
        "testMSA@example.com": {
          "available": true
        }
      },
      ...
    }
    

    Both service accounts are available for check-out. This is because Vault automatically checked in testMSA@example.com after the TTL expired.

Considerations for account check-in

During the check-in operation, Vault verifies the identity of the caller to see if it is the same client that checked out the service account. If the client has an associated entity, Vault checks to see if the entity ID matches the borrower's ID. Otherwise, Vault uses the client token to verify its identity. If the identity does not match, the check-in operation fails.

For example, a client authenticated with Vault using AppRole auth method and used the generated token to check-out a shared service account. Then the token expired, so the client re-authenticated to acquire a new token and attempt to invoke the check-in operation. When Vault compares the token value, it does not match; therefore, the check-in operation fails.

The request returns an error: "testMSA@example.com" can't be checked in because it wasn't checked out by the caller

There are three ways to handle this situation in the absence of entity ID:

Disable check-in enforcement

When you configure the accounting-team library, pass the disable_check_in_enforcement parameter and set it to true. This disables the checking of the client identity during the check-in operation.

CLI command

Example:

$ vault write ad/library/accounting-team \
        service_account_names="testMSA@example.com,MSA4SQL@example.com" \
        ttl=24h \
        max_ttl=120h \
        disable_check_in_enforcement=true

API call using cURL

Example:

# Create the HTTP request payload
$ tee payload.json <<EOF
{
  "service_account_names": ["testMSA@example.com", "MSA4SQL@example.com"],
  "ttl": "24h",
  "max_ttl": "120h",
  "disable_check_in_enforcement": true
}
EOF

# Invoke the ad/library API endpoint
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data @payload.json \
       http://127.0.0.1:8200/v1/ad/library/accounting-team

Use the ad/library/manage endpoint

Alternatively, the ad/library/manage endpoint can be used to check-in the service account.

Do not forget that the acct-team policy must include the following:

path "ad/library/managa/accounting-team/check-in" {
  capabilities = [ "update" ]
}

CLI command

$ vault write -f ad/library/manage/accounting-team/check-in

API call using cURL

$ curl --header "X-Vault-Token: ..."  \
       --request POST \
       --data '{ "service_account_names": ["testMSA@example.com"] }'  \
       http://127.0.0.1:8200/v1/ad/library/manage/accounting-team/check-in

Revoke the lease

Revoking the lease has the same effect. Again, do not forget that the acct-team policy must allow this operation:

path "sys/leases/revoke/*" {
    capabilities = [ "update" ]
}

CLI command

$ vault lease revoke ad/library/accounting-team/check-out/zklrSdycrQnwuxvAqCkIJkaV

While ad/library/accounting-team/check-out/zklrSdycrQnwuxvAqCkIJkaV is the lease_id.

API call using cURL

$ curl --header "X-Vault-Token: ..."  \
       --request POST \
       --data '{ "lease_id": "ad/library/accounting-team/check-out/zklrSdycrQnwuxvAqCkIJkaV" }'  \
       http://127.0.0.1:8200/v1/sys/leases/revoke

While ad/library/accounting-team/check-out/zklrSdycrQnwuxvAqCkIJkaV is the lease_id.

Help and Reference