Virtual Event
Join us for HashiConf Digital - product updates, technical sessions, workshops & more in a new online format Register for Free

Access Management

Tokens

Tokens are the core method for authentication within Vault. Tokens can be used directly or dynamically generated by the auth methods. Regardless, the clients need valid tokens to interact with Vault.

As of Vault 1.0, there are two types of tokens: service tokens and batch tokens. The service tokens are persisted; therefore, they can be renewed or revoked before reaching its time-to-live (TTL). On the other hand, batch tokens are not persisted. They are encrypted binary large objects (blobs) that carry enough information for them to be used for Vault actions. Therefore, batch tokens are extremely lightweight and scalable; however, they lack most of the flexibility and features of service tokens.

Service Tokens vs. Batch Tokens

As the number of machines and apps using Vault for secret management scales, Vault must manage the growing number of client tokens. The creation of service tokens can start affecting the Vault performance since they must be replicated across the primary and secondary clusters. On the other hand, batch tokens are neither persisted to disk nor live in memory, they are not a part of the data replication process.

Due to its lack of features with batch tokens (covered in Create batch tokens), it's preferable to use the service tokens in most use cases. However, think of a scenario where you have a large number of containers (e.g. 100,000s) start up and all request a token from Vault. The use of batch token can reduce the stress on the storage backend and improve the overall performance.

Service Token Lifecycle

Every non-root token has a time-to-live (TTL) associated with it. When a token expires and it's not renewed, the token is automatically revoked by Vault.

When a new token or secret is created, it is a child of its creator. If the parent is revoked or expires, so do all its children regardless of their own TTLs.

Suppose a hierarchy exists with respective TTL as follows:

    s.b519c6aa... (3h)
    |___ s.6a2cf3e7... (4h)
    |___ s.1d3fd4b2... (1h)
          |___ s.794b6f2f... (2h)

In this scenario, the token s.1d3fd4b2.. will expire in an hour. If a token or secret with a lease is not renewed before reaching its TTL, it will be revoked by the Vault server. When it's revoked, it takes its child (s.794b6f2f...) although the child has one more hour before it expires. Then, two hours later, s.b519c6aa... will be revoked and takes its child (s.6a2cf3e7...) with it.

Refer to Tokens documentation for more details.

Prerequisites

To perform the tasks described in this guide, you need to have a Vault environment. Refer to the Getting Started guide to install Vault.

NOTE: An interactive tutorial is also available if you do not have a Vault environment to perform the steps described in this guide. Click the Show Tutorial button to launch the tutorial.

Policy requirements

To perform all tasks demonstrated in this guide, your policy must include the following permissions:

# List available auth method
path "sys/auth" {
  capabilities = [ "read" ]
}

# Read default token configuration
path "sys/auth/token/tune" {
  capabilities = [ "read", "sudo" ]
}

# Create and manage tokens (renew, lookup, revoke, etc.)
path "auth/token/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

# For Advanced Features - list available secrets engines
path "sys/mounts" {
  capabilities = [ "read" ]
}

# For Advanced Features - tune the database secrets engine TTL
path "sys/mounts/database/tune" {
  capabilities = [ "update" ]
}

If you are not familiar with policies, complete the policies guide.

Create service tokens with use limit

In addition to TTL and max TTL, tokens may be limited to a number of uses. Use limit tokens expire at the end of their last use regardless of their remaining TTLs. On the same note, use limit tokens expire at the end of their TTLs regardless of their remaining uses.

To create tokens with a use limit, set the number of uses when you create them.

Create a token with the -use-limit property argument.

Example:

$ vault token create -policy=default -use-limit=2

Key                  Value
---                  -----
token                s.OZZFOsivUeOMeDyFtRz7cOmE
token_accessor       OOHblEiweXu6ozNUk6GIOl5C
token_duration       768h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

This creates a token with the default policy and a use limit of 2.

Verification

  1. Set the VAULT_TOKEN value to the token you just generated, and invoke any CLI command.

    Example:

    $ VAULT_TOKEN=s.OZZFOsivUeOMeDyFtRz7cOmE vault token lookup
    
    Key                 Value
    ---                 -----
    accessor            OOHblEiweXu6ozNUk6GIOl5C
    creation_time       1558033998
    creation_ttl        768h
    display_name        token
    entity_id           n/a
    expire_time         2019-06-17T12:13:18.297964-07:00
    explicit_max_ttl    0s
    id                  s.OZZFOsivUeOMeDyFtRz7cOmE
    issue_time          2019-05-16T12:13:18.297963-07:00
    meta                <nil>
    num_uses            1
    # ...snip...
    

    Notice that the num_uses is now 1.

  2. Run another CLI command.

    $ VAULT_TOKEN=s.OZZFOsivUeOMeDyFtRz7cOmE vault write cubbyhole/token value=1234567890
    Success! Data written to: cubbyhole/token
    
  3. Try to read the value now using the same token.

    $ VAULT_TOKEN=s.OZZFOsivUeOMeDyFtRz7cOmE vault read cubbyhole/token
    Error reading cubbyhole/token: Error making API request.
    
    URL: GET http://127.0.0.1:8200/v1/cubbyhole/token
    Code: 403. Errors:
    
    permission denied
    

The first command read the token's properties and then wrote a value to the cubbyhole secrets engine. This exhausted the use limit of 2 for this token. Therefore, the attempt to read the secret from the cubbyhole failed.

Periodic service tokens

Root or sudo users have the ability to generate periodic tokens. Periodic tokens have a TTL (validity period), but no max TTL; therefore, they may live for an infinite duration of time so long as they are renewed within their TTL. This is useful for long-running services that cannot handle regenerating a token.

In this example, create a token role named "zabbix" with its period set to 24 hours.

  1. Create a token role named zabbix with period set to 24 hours.

    $ vault write auth/token/roles/zabbix allowed_policies="default" period="24h"
    
  2. Now, generate a token for a role, zabbix.

    $ vault token create -role=zabbix
    
    Key                  Value
    ---                  -----
    token                s.nRucA9Gtb3yNVmLUK22yzVKA
    token_accessor       4Nm9BvIVS4HWCgLATc3rIoiW
    token_duration       24h
    token_renewable      true
    token_policies       ["default"]
    identity_policies    []
    policies             ["default"]
    

Renew service tokens

You can renew the service token's TTL as long as it has not been expired.

To renew a token's TTL, execute the vault token renew command.

$ vault token renew <TOKEN>

If you want to renew and extend the service token's TTL, pass the desired extension.

$ vault token renew -increment=<EXTENSION> <TOKEN>

The extension value can be an integer number of seconds (e.g. 3600) or a string duration (e.g. "1h").

Revoke service tokens

If a user or machine needs a temporal access to Vault, you can set a short TTL or a number of uses to a service token so the token is automatically revoked at the end of its life. But if any suspicious activity was detected, Vault has built-in support for revocation of service tokens before reaching its TTL.

To revoke a specific token, execute the vault token revoke command.

$ vault token revoke <TOKEN>

Example:

Revoke a service token, s.idqBWD6zx2eFJQKfDvNuSH3B.

$ vault token revoke s.idqBWD6zx2eFJQKfDvNuSH3B

Instead of revoking using a token value, revoke tokens with a token accessor, 5mus4HTbvVpI1VPaLK6eoQxy.

$ vault token revoke -accessor 5mus4HTbvVpI1VPaLK6eoQxy

Create short-lived tokens

Create a new service token with TTL of 60 seconds which means that the token gets automatically revoked after 60 seconds.

To view optional parameters to create tokens, run the command with -help flag.

$ vault token create -help

There are a number of parameters you can set. To specify the token TTL, pass the value using -ttl parameter.

Example:

Create a token with TTL of 60 seconds.

$ vault token create -ttl=60s
Key                  Value
---                  -----
token                s.6XhDGuPwiJgCbQUIIei4uA1Z
token_accessor       ykEJhiAP6KsP55IBQp6UaQqt
token_duration       1m
token_renewable      true
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Lookup the token details.

$ vault token lookup s.6XhDGuPwiJgCbQUIIei4uA1Z
Key                 Value
---                 -----
accessor            ykEJhiAP6KsP55IBQp6UaQqt
creation_time       1558034505
creation_ttl        1m
display_name        token
entity_id           n/a
expire_time         2019-05-16T12:22:45.318019-07:00
explicit_max_ttl    0s
id                  s.6XhDGuPwiJgCbQUIIei4uA1Z
issue_time          2019-05-16T12:21:45.318019-07:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [root]
renewable           true
ttl                 38s
type                service

NOTE: The vault token lookup command returns the token's properties. In this example, it shows that this token has 8 more seconds before it expires.

When you execute a Vault command using the new token immediately following its creation, it should work. Wait for 60 seconds and try again. It returns Code: 403. Errors: which indicates a forbidden API call due to expired token usage.

Orphan tokens

Orphan tokens are not children of their parent; therefore, orphan tokens do not expire when their parent does.

NOTE: Orphan tokens still expire when their own max TTL is reached.

The following CLI command requires root token or sudo capability on the auth/token/create path.

$ vault token create -orphan

Create batch tokens

Batch tokens are designed to be lightweight with limited flexibility. The following table highlights the difference.

Service TokensBatch Tokens
Can be root tokensYesNo
Can create child tokensYesNo
RenewableYesNo
Can be periodicYesNo
Can have explicit Max TTLYesNo (always uses a fixed TTL)
Has accessorsYesNo
Has CubbyholeYesNo
Revoked with parent (if not orphan)YesStops Working
Dynamic secrets lease assignmentSelfParent (if not orphan)
Can be used across Performance Replication clustersNoYes
Creation scales with performance standby node countNoYes
CostHeavyweight; multiple storage writes per token creationLightweight; no storage cost for token creation

  1. To create a batch token, you need to explicitly set the token type to be batch and cannot be root. Create a batch token with test policy attached.

    $ vault token create -type=batch -policy=test
    
    Key                  Value
    ---                  -----
    token                b.AAAAAQL_tyer_gNuQqvQYPVQgsNxjap_YW1NB2m4CDHHadQo7rF2XLFGdw-NJplAZNKbfloOvifrbpRCGdgG1taTqmC7D-a_qftN64zeL10SmNwEoDTiPzC_1aS1KExbtVftU3Sx16cBVqaynwsYRDfVnfTAffE
    token_accessor       n/a
    token_duration       768h
    token_renewable      false
    token_policies       ["test"]
    identity_policies    []
    policies             ["test"]
    

    Notice that the token value is much longer than the service tokens. This is because batch tokens are encrypted by the Vault's barrier.

  2. Lookup the token details.

    $ vault token lookup <batch_token>
    
    Key                 Value
    ---                 -----
    accessor            n/a
    creation_time       1539823898
    creation_ttl        768h
    display_name        token
    entity_id           n/a
    expire_time         2018-11-18T16:51:38-08:00
    explicit_max_ttl    0s
    id                  b.AAAAAQKEhArH0qEZze1AHljUq-ujPxQiuZS5VVTrPHtE9mthVu261PpyfcU28StDdLzNBmm5Tf7u5BC3oaiIyUxTxVq1B9SQgnPYnviuWaqlKJ4wzPkQcIyNTg2YdGobdlhaM8AIqbnSPGir5xQnsNRjve48w2c
    issue_time          2018-10-17T17:51:38-07:00
    meta                <nil>
    num_uses            0
    orphan              false
    path                auth/token/create
    policies            [default]
    renewable           false
    ttl                 767h59m49s
    type                batch
    

    Notice that the renewable is set to false.

  3. Log in with the generated batch token.

    $ vault login <batch_token>
    
  4. Test to see if the token has a Cubbyhole.

    $ vault write cubbyhole/token value="1234567890"
    Error writing data to cubbyhole/token: Error making API request.
    
    URL: PUT http://127.0.0.1:8200/v1/cubbyhole/token
    Code: 400. Errors:
    
     cubbyhole operations are only supported by "service" type tokens
    
  5. Batch token cannot create child tokens even if its policy grants permission. Test to verify.

    $ vault token create -policy=default
    Error creating token: Error making API request.
    
    URL: POST http://127.0.0.1:8200/v1/auth/token/create
    Code: 400. Errors:
    
     batch tokens cannot create more tokens
    
  6. Log in with a highly privileged token such as root.

    $ vault login root
    
  7. Try to revoke batch token.

    $ vault token revoke <batch_token>
    Error revoking token: Error making API request.
    
    URL: PUT http://127.0.0.1:8200/v1/auth/token/revoke
    Code: 400. Errors:
    
     batch tokens cannot be revoked
    

Token settings

Token is a core authentication method in Vault with some default settings. This step demonstrates the following:

  1. View the current token settings
  2. Modify the default TTL and maximum TTL
  1. Execute the following command to view the current settings on the token auth method.

    $ vault auth list -detailed
    
    Path         Plugin      Accessor                  Default TTL    Max TTL    Token Type         ...
    ----         ------      --------                  -----------    -------    ----------         ...
    token/       token       auth_token_86f81f5c       2764800        2764800    default-service    ...
    ...snip...
    

    Alternatively, you can run the following command:

    $ vault read sys/mounts/auth/token/tune
    
    Key                  Value
    ---                  -----
    default_lease_ttl    768h
    force_no_cache       false
    max_lease_ttl        768h
    token_type           default-service
    

    When no specific TTL was provided with the vault token create command, the generated token will inherit the default TTL which is 2764800 seconds (32 days). The same for the maximum TTL.

  2. Execute the following command to change the default TTL to 6 minutes and the max TTL to 24 hours.

    $ vault write sys/mounts/auth/token/tune default_lease_ttl=6m max_lease_ttl=24h
    
  3. Verify the token configuration.

    $ vault read sys/mounts/auth/token/tune
    
    Key                  Value
    ---                  -----
    default_lease_ttl    6m
    force_no_cache       false
    max_lease_ttl        24h
    token_type           default-service
    
  4. Let's execute the following command to examine the resulting behavior. First, update the zabbix role setting - the period is 72 hours.

    $ vault write auth/token/roles/zabbix allowed_policies="default" period="72h"
    
  5. Now, generate a token and see what happens.

    $ vault token create -role=zabbix
    
    WARNING! The following warnings were returned from Vault:
    
       period of "72h0m0s" exceeded the effective max_ttl of "24h0m0s"; period
      value is capped accordingly
    
    Key                  Value
    ---                  -----
    token                s.AqJXvQsQiT2iqAJWKb4EDARs
    token_accessor       onWpEZ0Ll9a4nkEBDO1x1ON5
    token_duration       24h
    token_renewable      true
    token_policies       ["default"]
    identity_policies    []
    policies             ["default"]
    

    Notice the warning message. The max TTL configured on the token auth method is the hard max TTL.

Apply token types

You've learned how you can fine tune the token's lifecycle. Now, let's talk about tokens for your applications. Vault clients first authenticate with Vault using an auth method to acquire a token.

There are auth methods aimed to authenticate applications or machines. Once its identity was verified, Vault server will return a token with appropriate policies attached.

Let's talk about AppRole auth method configuration.

Generate periodic tokens

  1. To configure the approle auth method to generate a periodic token for your app, first enable the approle auth method.

    $ vault auth enable approle
    
  2. Create a role for your app specifying that the generated token should be periodic.

    $ vault write auth/approle/role/jenkins policies="jenkins" period="72h"
    

    This example defines a role named, "jenkins". The tokens generated for this role will be a periodic token with jenkins policy attached.

Generate batch tokens

To configure the approle auth method to generate a batch token for your app, execute the following command.

$ vault write auth/approle/role/shipping policies="shipping" \
        token_type="batch" \
        token_ttl="60s"

This example defines a role named, "shipping". The tokens generated for this role will be a batch token with TTL of 1 minute.

Tune the default TTLs

When you create tokens or leases with no specific TTL values, the default value applies to them.

$ vault auth list -detailed

Path         Type        Accessor                  Plugin    Default TTL    Max TTL    ...
----         ----        --------                  ------    -----------    -------
approle/     approle     auth_approle_9592c1db     n/a       system         system
ldap/        ldap        auth_ldap_38571baa        n/a       system         system
token/       token       auth_token_47eac1f8       n/a       system         system
userpass/    userpass    auth_userpass_6804c90e    n/a       system         system

The system max TTL is 32 days, but you can override it to be longer or shorter in Vault's configuration.

Another option is to tune the mount configuration to override the system defaults by calling the /sys/auth/<METHOD>/tune endpoint.

Read the default TTL settings for token auth method.

$ vault read sys/auth/token/tune

Key                  Value
---                  -----
default_lease_ttl    768h
force_no_cache       false
max_lease_ttl        768h
token_type           default-service

Notice that the token_type is default-service.

Get the token count

To find out how many service tokens exist on a Vault environment, invoke the sys/internal/counters/tokens endpoint.

Example:

$ vault read sys/internal/counters/tokens
Key         Value
---         -----
counters    map[service_tokens:map[total:5]]

Help and Reference