»What are tokens
Tokens are the core method for authenticate and validate Vault clients; therefore, nearly all requests to Vault must be accompanied by a token. Vault clients authenticate with Vault using a configured auth method (Okta, Kubernetes, etc.). Upon successful authentication, Vault generates a token managed by the token backend and returns it to the client.
The token auth method is built-in and is at the core of client authentication. A client can authenticate with Vault through the token auth method. For example, a Vault admin logs in with Vault via token auth method using the initial root token (or admin token if you are running HCP Vault) so that the admin can configure other auth methods.
»Token types
There are two types of Vault tokens: service token and batch token. Vault persists the service tokens in its storage backend. You can renew a service token or revoke it as necessary. On the other hand, Vault does not persist the batch tokens. Batch tokens are encrypted binary large objects (blobs) that carry enough information to perform Vault actions. Therefore, batch tokens are extremely lightweight and scalable; however, they lack most of the flexibility and features of service tokens.
In addition, there is a recovery token which is a special token used when Vault is running in recovery mode. To learn more, read the Operate Vault in Recovery Mode tutorial.
»Token prefix
Tokens have a prefix that indicates their type. As of Vault 1.10, the token format has changed. The following table lists the prefix differences.
| Token Type | Vault 1.9.x or earlier | Vault 1.10 and later |
|---|---|---|
| Service tokens | s. | hvs. |
| Batch tokens | b. | hvb. |
| Recovery tokens | r. | hvr. |
When you upgrade your existing Vault cluster to Vault 1.10 or later, the tokens generated prior to Vault 1.10 will still work if they have not expired.
This tutorial focuses on the service tokens. Read the batch tokens tutorial to learn the usage of batch tokens.
»Service token lifecycle
Every non-root token has a time-to-live (TTL). When a token expires, Vault automatically revokes it. If you create a new token, the token you used to create the token becomes the parent token. Once the parent token expires, so do all its children regardless of their own TTLs.
Suppose a hierarchy exists with respective TTL as follows:
In this scenario, the token hvs.1d3fd4b2.. will expire in two hours. Although
its child token (hvs.794b6f2f...) has TTL of three hours, Vault will revoke
the child token when its parent expires.
When the top-level token (hvs.b519c6aa...) expires, Vault will revoke all
tokens under the tree (hvs.6a2cf3e7..., hvs.1d3fd4b2..., and
hvs.794b6f2f...) regardless of their TTL.
»TTL and max TTL
If the token is renewable, you can use vault token renew command to extend the
token's TTL before it expires. You can repeatedly renew a token until it reaches
its maximum TTL.
For example, if a token's TTL is 30 minutes and the maximum TTL is 24 hours, you can renew the token before reaching the 30 minutes. You can renew the token multiple times if you are using it. However, once the token reaches the 24 hours of its first creation, you can no longer renew the token.
Important: If you do not explicitly set the token's TTL or maximum TTL, it takes the system max TTL which is 32 days by default. (You can change the system default in the Vault server configuration file.) This means that Vault stores the token in its storage backend for 32 days even if you are not using it.
»What you are going to learn
This tutorial explores the lifecycle of service tokens.
- Token with use limit: Tokens that are only good to invoke a specific number of operations.
- Periodic service tokens: Tokens that can be renewed indefinitely.
- Short-lived tokens: Tokens that are valid for a short time to avoid keeping unused tokens.
- Orphan tokens: Tokens that are root of their own token tree.
- Token role: Create a role which defines a set of policies and token lifecycle specification.
- Renew service tokens: Renew a token's TTL before it expires.
- Revoke service tokens: Revoke a token before its expiration.
- Apply token types: Configure an auth method so that it generates a token with specified lifecycle.
»Prerequisites
To perform the tasks described in this tutorial, you need to have a Vault environment. Refer to the Getting Started tutorial to install Vault.
Launch Terminal
This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.
»Lab setup
Open a terminal and start a Vault dev server with root as the root token.
The Vault dev server defaults to running at 127.0.0.1:8200. The server is
initialized and unsealed.
Insecure operation: Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.
Export an environment variable for the vault CLI to address the Vault server.
Export an environment variable for the vault CLI to authenticate with the
Vault server.
NOTE: For these tasks, you can use Vault's root token. However, it is recommended that root tokens are only used for enough initial setup or in emergencies. As a best practice, use an authentication method or token that meets the policy requirements.
The Vault server is ready.
»Tokens with use limit
In addition to TTL and max TTL, you can set the number of uses for tokens. The tokens with a use limit expire at the end of their last use regardless of their remaining TTLs. On the same note, use limit tokens expire at the end of their TTLs regardless of their remaining uses.
To create tokens with a use limit, set the number of uses when you create them.
To view optional parameters to create tokens, run the command with -help flag.
There are a number of parameters you can set.
Create a token with a TTL of 1 hour and a use limit of 2. Attach the
default policy.
Example output:
»Verification
Store the generated token in an environment variable,
USE_LIMIT_TOKEN.Example:
Set the
VAULT_TOKENvalue to the token you just generated, and invoke any CLI command.Example output:
Notice that the
num_usesis now1.Run another CLI command using the token.
Try to read the value now using the same token.
The first command read the token's properties and then wrote a value to the cubbyhole secrets engine. This exhausted the use limit of 2 for this token. Therefore, the attempt to read the secret from the cubbyhole failed.
»Periodic service tokens
Root or sudo users have the ability to generate periodic tokens. Periodic tokens have a TTL (validity period), but no max TTL; therefore, they may live for an infinite duration of time so long as they are renewed within their TTL. This is useful for long-running services that cannot handle regenerating a token.
NOTE: When you set period, it becomes the token renewal period (TTL).
When a period and an explicit max TTL were both set on a token, it behaves as a
periodic token. However, once the explicit max TTL is reached, the token will be
revoked. Refer to the renew service tokens to learn
more about the period and the maximum TTL.
Create a token with 24 hours period.
Generate a periodic token again and store the token value in a
periodic_token.txt file.
Display the generated token's metadata.
Example output:
You can renew the generated token indefinitely for as long as it does not expire. If you do not renew, the token expires after 24 hours.
Token Renewal: Jump to the Renew service tokens section to learn how to renew the generated token.
»Short-lived tokens
Create a new service token with a TTL of 60 seconds which means that the token gets automatically revoked after 60 seconds.
Create a token with a TTL of 60 seconds, and store the generated token value in
a short-lived_token.txt file.
Lookup the generated token's metadata.
Example output:
NOTE: The vault token lookup command returns the token's properties.
In this example, it shows that this token has 38 more seconds before it expires.
When you execute a Vault command using the new token immediately following its
creation, it should work. Wait for 60 seconds and try again. It returns
Code: 403. Errors: which indicates a forbidden API call due to expired
token usage.
»Orphan tokens
Orphan tokens are not children of their parent; therefore, orphan tokens do not expire when their parent does.
NOTE: Orphan tokens still expire when their own max TTL is reached.
Root privilege: A user must use a root token or token with sudo
capability on the auth/token/create endpointto generate an orphan token via
CLI.
Generate an orphan token, and store the generated token value in a
orphan_token.txt file.
Lookup the generated token's metadata.
Example output:
Validation: The revoke service tokens section tests the lifecycle of orphan tokens.
»Token role
Instead of passing a number of parameters, you can create a role with a set of parameter values set.
Create a token role named zabbix.
Create a token for zabbix role.
Example output:
The generated token is valid for 8 hours and it is renewable, and multiple policies are attached.
»Renew service tokens
You can renew the service token's TTL as long as it has not expired.
Create a token and save its value in a file,
test_token.txt.The generated token has a TTL of 45 seconds, and max TTL of 2 minutes (120 seconds).
Renew the token's TTL before the token expires.
Example output:
Renew and extend the token's TTL to 60 seconds.
Example output:
Notice that the token TTL (
token_duration) is now 1 minute instead of 45 seconds.
Because the explicit max TTL is set to 2 minutes, you will not be able to renew the token after 2 minutes.
As time passes, Vault returns a message such as TTL of "26s" exceeded the effective max_ttl of "10s"; TTL value is capped accordingly to indicate that
the token TTL cannot exceed 2 minutes from its creation time. Eventually, the
token expires and Vault automatically revokes it. Once the token expires, the
renew command returns token not found message.
»Revoke service tokens
If a user or machine needs a temporal access to Vault, you can set a short TTL or a number of uses to a service token so the token is automatically revoked at the end of its life. But if any suspicious activity was detected, Vault has built-in support for revocation of service tokens before reaching its TTL.
You can revoke service tokens using the vault token revoke command or the
auth/token/revoke API endpoint.
In this section, you are going to create tokens with the following hierarchy and inspect the token lifecycle.
Create a
testpolicy.If you are not familiar with policies, complete the policies tutorial.
Create a token and save its value in a file,
parent_token.txt.The generated token has a TTL of 1 minute (60 seconds).
Create a token using the parent token and save its value in a file,
child_token.txt.The generated token has a TTL of 3 minute (180 seconds) while its parent token's TTL is 1 minute.
Create an orphan token using the parent token and save its value in a file,
orphan_token.txt.The generated token is an orphan token with a TTL of 3 minute (180 seconds).
Revoke the parent token.
Verify that the token no longer exists by looking it up.
Vault returns an error message.
Look up the child token.
Vault returns the
bad tokenerror because Vault revoked the child token along with its parent token.Look up the orphan token.
Because each orphan token is the root of its own token tree, it exists until it expires. Therefore, the command displays the detail information about the orphan token.
Instead of revoking using a token value, revoke tokens with a token
accessor
using the -accessor flag.
»Apply token types
You learned how you can set the token's lifecycle. The next step is to apply this to generate tokens for your applications. Vault clients first authenticate with Vault using an auth method to acquire a token. There are auth methods aimed to authenticate applications or machines. Once its identity was verified, Vault server will return a token with appropriate policies attached.
Use the AppRole auth method to demonstrate this.
Enable the
approleauth method.Create a role for your app specifying that the generated token type is periodic and expires after 24 hours if not renewed.
This example defines a role named, "jenkins". The tokens generated for this role will be a periodic token with
jenkinspolicy attached.
»Verification
NOTE: If you are not familiar with the AppRole auth method, read the AppRole Pull Authentication tutorial.
Retrieve the RoleID for the
jenkinsrole and save it in a file,role_id.txt.Generate a SecretID for the
jenkinsrole and save it in a file,secret_id.txt.Authenticate with Vault using the generated
role_idandsecret_id.Example output:
View the token details.
Example:
The output shows the
periodof 24 hours, and thejenkinspolicy is attached.
»Clean up
If you wish to clean up your environment after completing the tutorial, follow the steps in this section.
Unset the
VAULT_TOKENenvironment varialbe.Unset the
VAULT_ADDRenvironment varialbe.Delete the files used to test tokens.
If you are running Vault locally in
-devmode, you can stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.
»Next steps
This tutorial walked through the lifecycle of service tokens. You learned how to
generate a token using the vault token create command or the
/auth/token/create endpoint. Then, you learned how to apply the desired token
lifecycle when you configure an auth method in the Apply token
type section.
Integrating your application to read or write secrets to Vault may require:
- Authenticate with Vault using an auth method
- Maintain the token
- Renew or revoke the token if necessary
Vault Agent can help to simplify the introduction of Vault to your applications. The App Integration tutorials introduce different approaches.
But first, go through the Batch tokens tutorial to understand the difference between the service tokens and batch tokens so that you can decide which token type is best suited for your use case.
