NewHCP Vault is now Generally Available Sign up today

Tokens

Tokens are the core method for authentication within Vault. You can use tokens to log in with Vault. If you log in with Vault via an auth method, a successful authentication generates a token. Regardless, clients need valid tokens to read secrets from Vault.

There are two types of Vault tokens: service token and batch token. Vault persists the service tokens in its storage backend. You can renew a service token or revoke it as necessary. On the other hand, Vault does not persist the batch tokens. Batch tokens are encrypted binary large objects (blobs) that carry enough information to perform Vault actions. Therefore, batch tokens are extremely lightweight and scalable; however, they lack most of the flexibility and features of service tokens.

»Service token lifecycle

Every non-root token has a time-to-live (TTL). When a token expires, Vault automatically revokes it. If you create a new token, the token you used to create the token becomes the parent token. Once the parent token expires, so do all its children regardless of their own TTLs.

Suppose a hierarchy exists with respective TTL as follows:

    s.b519c6aa... (1h)
    |___ s.6a2cf3e7... (4h)
    |___ s.1d3fd4b2... (2h)
          |___ s.794b6f2f... (3h)

In this scenario, the token s.1d3fd4b2.. will expire in two hour. Although its child token (s.794b6f2f...) has TTL of three hours, Vault will revoke the child token when its parent expires.

When the top-level token (s.b519c6aa...) expires, Vault will revoke all tokens under the tree (s.6a2cf3e7..., s.1d3fd4b2..., and s.794b6f2f...) regardless of their TTL.

»TTL and max TTL

If the token is renewable, you can use vault token renew command to extend the token's TTL before it expires. You can repeatedly renew a token until it reaches its maximum TTL.

For example, if a token's TTL is 30 minutes and the maximum TTL is 24 hours, you can renew the token before reaching the 30 minutes. You can renew the token multiple times if you are using it. However, once the token reaches the 24 hours of its first creation, you can no longer renew the token.

»Prerequisites

To perform the tasks described in this tutorial, you need to have a Vault environment. Refer to the Getting Started tutorial to install Vault.

NOTE: An interactive tutorial is also available if you do not have a Vault environment to perform the steps described in this tutorial. Click the Show Terminal button to start.

»Start Vault

Start a Vault dev server with root as the root token.

$ vault server -dev -dev-root-token-id root

Export an environment variable for the vault CLI to address the Vault server.

$ export VAULT_ADDR=http://127.0.0.1:8200

Login with the root token.

$ vault login root

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                root
token_accessor       n9CYvD0GK3iV6nwAOZQAy9Md
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

The Vault server is ready for you to explore service tokens.

»Token types

Tokens allow Vault clients to read or write secrets from Vault; therefore, it is critical to use a specific type of token based on the use case.

In this tutorial, you will learn the lifecycle of different token types.

Token typeUsage
Use limit tokensTokens that are only good to invoke a specific number of operations.
Periodic service tokensTokens that can be renewed indefinitely.
Short-lived tokensTokens that are valid for a short time to avoid keeping unused tokens.
Orphan tokensTokens that are root of their own token tree.

Then, you will learn how to manage tokens.

»Tokens with use limit

In addition to TTL and max TTL, you can set the number of uses for tokens. The tokens with a use limit expire at the end of their last use regardless of their remaining TTLs. On the same note, use limit tokens expire at the end of their TTLs regardless of their remaining uses.

To create tokens with a use limit, set the number of uses when you create them.

To view optional parameters to create tokens, run the command with -help flag.

$ vault token create -help

There are a number of parameters you can set.

Create a token with TTL of 1 hour and a use limit of 2. Attach the default policy.

$ vault token create -ttl=1h -use-limit=2 -policy=default

Example output:

Key                  Value
---                  -----
token                s.f7Ea3C3ojOYE0GRLzmhSGNkE
token_accessor       Rw2YoxPxH0R3fCp3wgUvq4tC
token_duration       1h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

»Verification

  1. Store the generated token in an environment variable, USE_LIMIT_TOKEN.

    Example:

    $ export USE_LIMIT_TOKEN="s.f7Ea3C3ojOYE0GRLzmhSGNkE"

  2. Set the VAULT_TOKEN value to the token you just generated, and invoke any CLI command.

    Example:

    $ VAULT_TOKEN=$USE_LIMIT_TOKEN vault token lookup

    Example output:

    Key                 Value
---                 -----
accessor            Rw2YoxPxH0R3fCp3wgUvq4tC
creation_time       1613092241
creation_ttl        1h
display_name        token
entity_id           n/a
expire_time         2021-02-11T18:10:41.950833-08:00
explicit_max_ttl    0s
id                  s.f7Ea3C3ojOYE0GRLzmhSGNkE
issue_time          2021-02-11T17:10:41.950851-08:00
meta                
num_uses            1
orphan              false
path                auth/token/create
policies            [default]
renewable           true
ttl                 56m
type                service

    Notice that the num_uses is now 1.

  3. Run another CLI command using the token.

    $ VAULT_TOKEN=$USE_LIMIT_TOKEN vault write cubbyhole/token value=1234567890
Success! Data written to: cubbyhole/token

  4. Try to read the value now using the same token.

    $ VAULT_TOKEN=$USE_LIMIT_TOKEN vault read cubbyhole/token
Error reading cubbyhole/token: Error making API request.

URL: GET http://127.0.0.1:8200/v1/cubbyhole/token
Code: 403. Errors:

permission denied

    The first command read the token's properties and then wrote a value to the cubbyhole secrets engine. This exhausted the use limit of 2 for this token. Therefore, the attempt to read the secret from the cubbyhole failed.

»Periodic service tokens

Root or sudo users have the ability to generate periodic tokens. Periodic tokens have a TTL (validity period), but no max TTL; therefore, they may live for an infinite duration of time so long as they are renewed within their TTL. This is useful for long-running services that cannot handle regenerating a token.

Create a token with 24 hours period.

$ vault token create -policy="default" -period=24h

Key                  Value
---                  -----
token                s.ogElkUdIRwhpBJA4kTUWoZms
token_accessor       OixDYsVfPMH5VuthPG7yVq5V
token_duration       24h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

You can renew the generated token indefinitely for as long as it does not expire. If you do not renew, the token expires after 24 hours.

»Short-lived tokens

Create a new service token with TTL of 60 seconds which means that the token gets automatically revoked after 60 seconds.

Create a token with TTL of 60 seconds.

$ vault token create -ttl=60s

Key                  Value
---                  -----
token                s.6XhDGuPwiJgCbQUIIei4uA1Z
token_accessor       ykEJhiAP6KsP55IBQp6UaQqt
token_duration       1m
token_renewable      true
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Lookup the token details.

$ vault token lookup s.6XhDGuPwiJgCbQUIIei4uA1Z
Key                 Value
---                 -----
accessor            ykEJhiAP6KsP55IBQp6UaQqt
creation_time       1558034505
creation_ttl        1m
display_name        token
entity_id           n/a
expire_time         2019-05-16T12:22:45.318019-07:00
explicit_max_ttl    0s
id                  s.6XhDGuPwiJgCbQUIIei4uA1Z
issue_time          2019-05-16T12:21:45.318019-07:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [root]
renewable           true
ttl                 38s
type                service

NOTE: The vault token lookup command returns the token's properties. In this example, it shows that this token has 38 more seconds before it expires.

When you execute a Vault command using the new token immediately following its creation, it should work. Wait for 60 seconds and try again. It returns Code: 403. Errors: which indicates a forbidden API call due to expired token usage.

»Orphan tokens

Orphan tokens are not children of their parent; therefore, orphan tokens do not expire when their parent does.

NOTE: Orphan tokens still expire when their own max TTL is reached.

The following CLI command requires root token or sudo capability on the auth/token/create path.

$ vault token create -orphan

»Token role

Instead of passing a number of parameters, you can create a role with a set of parameter values set.

Create a token role named zabbix.

$ vault write auth/token/roles/zabbix \
    allowed_policies="policy1, policy2, policy3" \
    orphan=true \
    token_ttl=8h

Create a token for zabbix role.

$ vault token create -role=zabbix

Key                  Value
---                  -----
token                s.Jirk7VY2XsKUR7yUHVGSwaYF
token_accessor       zf7HTloUOmgFZ91e4CHMzI7W
token_duration       8h
token_renewable      true
token_policies       ["default" "policy1" "policy2" "policy3"]
identity_policies    []
policies             ["default" "policy1" "policy2" "policy3"]

The generated token is valid for 8 hours and it is renewable, and multiple policies are attached.

»Renew service tokens

You can renew the service token's TTL as long as it has not expired.

  1. Create a token and save its value in a file, test_token.txt.

    $ vault token create -ttl=45 -explicit-max-ttl=120 -policy=default -format=json \
    | jq -r ".auth.client_token" > test_token.txt

    The generated token has a TTL of 45 seconds, and max TTL of 2 minutes (120 seconds).

  2. Renew the token's TTL before the token expires.

    $ vault token renew $(cat test_token.txt)

    Example output:

    Key                  Value
---                  -----
token                s.bETmMLCJfYwCg5r7SxgNJ3NZ
token_accessor       ZVd00AFdYzVRzG5MSjbXgTe6
token_duration       45s
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

  3. Renew and extend the token's TTL to 60 seconds.

    $ vault token renew -increment=60 $(cat test_token.txt)

    Example output:

    Key                  Value
---                  -----
token                s.bETmMLCJfYwCg5r7SxgNJ3NZ
token_accessor       ZVd00AFdYzVRzG5MSjbXgTe6
token_duration       1m
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

    Notice that the token TTL (token_duration) is now 1 minute instead of 45 seconds.

Because the explicit max TTL is set to 2 minutes, you will not be able to renew the token after 2 minutes.

As time passes, Vault returns a message such as TTL of "26s" exceeded the effective max_ttl of "10s"; TTL value is capped accordingly to indicate that the token TTL cannot exceed 2 minutes from its creation time. Eventually, the token expires and Vault automatically revokes it. Once the token expires, the renew command returns token not found message.

»Revoke service tokens

If a user or machine needs a temporal access to Vault, you can set a short TTL or a number of uses to a service token so the token is automatically revoked at the end of its life. But if any suspicious activity was detected, Vault has built-in support for revocation of service tokens before reaching its TTL.

You can revoke service tokens using the vault token revoke command or the auth/token/revoke API endpoint.

In this section, you are going to create tokens with the following hierarchy and inspect the token lifecycle.

 parent_token (1 minute)
    |___ child_token (3 minutes)
    |___ orphan_token (3 minutes)

  1. Create a test policy.

    $ vault policy write test -<<EOF
path "auth/token/create" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
EOF

    If you are not familiar with policies, complete the policies tutorial.

  2. Create a token and save its value in a file, parent_token.txt.

    $ vault token create -ttl=60 -policy=test -format=json \
    | jq -r ".auth.client_token" > parent_token.txt

    The generated token has a TTL of 1 minute (60 seconds).

  3. Create a token using the parent token and save its value in a file, child_token.txt.

    $ VAULT_TOKEN=$(cat parent_token.txt) \
   vault token create -ttl=180 -policy=default -format=json \
    | jq -r ".auth.client_token" > child_token.txt

    The generated token has a TTL of 3 minute (180 seconds) while its parent token's TTL is 1 minute.

  4. Create an orphan token using the parent token and save its value in a file, orphan_token.txt.

    $ VAULT_TOKEN=$(cat parent_token.txt) \
   vault token create -orphan -ttl=180 -policy=default -format=json \
    | jq -r ".auth.client_token" > orphan_token.txt

    The generated token is an orphan token with a TTL of 3 minute (180 seconds).

  5. Revoke the parent token.

    $ vault token revoke $(cat parent_token.txt)
Success! Revoked token (if it existed)

  6. Verify that the token no longer exists by looking it up.

    $ vault token lookup $(cat parent_token.txt)

    Vault returns an error message.

    Error looking up token: Error making API request.

URL: POST http://127.0.0.1:8200/v1/auth/token/lookup
Code: 403. Errors:

* bad token

  7. Look up the child token.

    $ vault token lookup $(cat child_token.txt)

    Vault returns the bad token error because Vault revoked the child token along with its parent token.

  8. Look up the orphan token.

    $ vault token lookup $(cat orphan_token.txt)

    Because each orphan token is the root of its own token tree, it exists until it expires. Therefore, the command displays the detail information about the orphan token.

»Apply token types

You learned how you can set the token's lifecycle. The next step is to apply this to generate tokens for your applications. Vault clients first authenticate with Vault using an auth method to acquire a token. There are auth methods aimed to authenticate applications or machines. Once its identity was verified, Vault server will return a token with appropriate policies attached.

Use the AppRole auth method to demonstrate this.

  1. Enable the approle auth method.

    $ vault auth enable approle

  2. Create a role for your app specifying that the generated token type is periodic and expires after 24 hours if not renewed.

    $ vault write auth/approle/role/jenkins policies="jenkins" period="24h"

    This example defines a role named, "jenkins". The tokens generated for this role will be a periodic token with jenkins policy attached.

»Verification

  1. Retrieve the RoleID for the jenkins role and save it in a file, role_id.txt.

    $ vault read -format=json auth/approle/role/jenkins/role-id \
    | jq -r ".data.role_id" > role_id.txt

  2. Generate a SecretID for the jenkins role and save it in a file, secret_id.txt.

    $ vault write -f -format=json auth/approle/role/jenkins/secret-id \
    | jq -r ".data.secret_id" > secret_id.txt

  3. Authenticate with Vault using the generated role_id and secret_id.

    $ vault write auth/approle/login role_id=$(cat role_id.txt) \
     secret_id=$(cat secret_id.txt)

    Example output:

    Key                     Value
---                     -----
token                   s.Hebs2ofuKL3RKjAVLoVkIsWq
token_accessor          CruzydNdIuct1OYYHXb86wwI
token_duration          24h
token_renewable         true
token_policies          ["default" "jenkins"]
identity_policies       []
policies                ["default" "jenkins"]
token_meta_role_name    jenkins

  4. View the token details.

    $ vault token lookup <returned_token>

    Example:

    $ vault token lookup s.Hebs2ofuKL3RKjAVLoVkIsWq

Key                 Value
---                 -----
accessor            CruzydNdIuct1OYYHXb86wwI
creation_time       1614129261
creation_ttl        24h
display_name        approle
entity_id           109c9401-b2bb-c1b2-9166-216d7d16fe75
expire_time         2021-02-24T17:14:21.895619-08:00
explicit_max_ttl    0s
id                  s.Hebs2ofuKL3RKjAVLoVkIsWq
issue_time          2021-02-23T17:14:21.895622-08:00
meta                map[role_name:jenkins]
num_uses            0
orphan              true
path                auth/approle/login
period              24h
policies            [default jenkins]
renewable           true
ttl                 23h58m12s
type                service

    The output shows the period of 24 hours, and the jenkins policy is attached.

»Next steps

This tutorial walked through the Vault token fundametal. You rarely create tokens using the vault token create commands or the /auth/token/create endpoint. In most cases, you specify the type of token in the context of auth methods as demonstrated in the Apply token type section.

Integrating your application to read or write secrets to Vault may require:

  • Authenticate with Vault using an auth method
  • Maintain the token
  • Renew or revoke the token if necessary

Vault Agent can help to simplify the introduction of Vault to your applications. The App Integration collection lists tutorials that introduce different approaches.

But first, go through the Batch tokens tutorial to understand the difference between the service tokens and batch tokens so that you can decide which token type is best suited for your use case.

»Reference materials

NextBatch Tokens
HashiCorp
stdin: is not a tty