Enterprise Only: The namespaces feature requires Vault Enterprise Platform.
Everything in Vault is path-based, and often uses the terms path
and
namespace
interchangeably. The application namespace pattern is a useful
construct for providing Vault as a service to internal customers, giving them
the ability to implement secure multi-tenancy within Vault in order to provide
isolation and ensure teams can self-manage their own environments.
»Personas
The scenario described in this tutorial introduces the following personas:
operations
is the cluster-level administrator with privileged policiesorg-admin
is the organization-level administratorteam-admin
is the team-level administrator
»Challenge
When Vault is primarily used as a central location to manage secrets, multiple organizations within a company may need to be able to manage their secrets in a self-serving manner. This means that a company needs to implement a Vault as a Service model allowing each organization (tenant) to manage their own secrets and policies. The most importantly, tenants should be restricted to work only within their tenant scope.
»Solution
Create a namespace dedicated to each team, organization, or app where they can perform all necessary tasks within their tenant namespace.
Each namespace can have its own:
- Policies
- Auth Methods
- Secrets Engines
- Tokens
- Identity entities and groups
Tokens are locked to a namespace or child-namespaces. Identity groups can pull in entities and groups from other namespaces.
»Prerequisites
To perform the tasks described in this tutorial, you need to have a Vault Enterprise environment.
NOTE: The creation of namespaces should be performed by a user with a
highly privileged token such as root
to set up isolated environments for
each organization, team, or application.
»Scenario introduction
In this tutorial, you are going to create a namespace dedicated to the Education organization which has Training and Certification teams. Delegate operational tasks to the team admins so that the Vault cluster operators won't have to be involved.
he following steps are demonstrated:
- Create namespaces
- Write policies
- Setup entities and groups
- Test the organization admin user
- Test the team admin user
- Audit ambient credentials
»Create namespaces
(Persona: operations)
This tutorial focuses on the use of Vault namespaces. Refer to Vault Limits and Maximums to understand the known upper limits on namespaces.
To create a new namespace, run: vault namespace create <namespace_name>
Create a namespace dedicated to the
education
organizations.$ vault namespace create education
Create child namespaces called
training
under theeducation
namespace.$ vault namespace create -namespace=education training
Create child namespaces called
certification
under theeducation
namespace.$ vault namespace create -namespace=education certification
List the existing namespaces on the root.
$ vault namespace list education/
List the existing namespace on the
education
namespace.$ vault namespace list -namespace=education certification/ training/
»Write policies
(Persona: operations)
In this scenario, there is an organization-level administrator who is a
superuser within the scope of the education
namespace. Also, there is a
team-level administrator for training
and certification
.
»Policy for education admin
Requirements:
- Create and manage namespaces
- Create and manage policies
- Enable and manage secrets engines
- Create and manage entities and groups
- Manage tokens
edu-admin.hcl
# Manage namespaces
path "sys/namespaces/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage policies
path "sys/policies/acl/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# List policies
path "sys/policies/acl" {
capabilities = ["list"]
}
# Enable and manage secrets engines
path "sys/mounts/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
# List available secrets engines
path "sys/mounts" {
capabilities = [ "read" ]
}
# Create and manage entities and groups
path "identity/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
# Manage tokens
path "auth/token/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage secrets at 'edu-secret'
path "edu-secret/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
»Policy for training admin
Requirements:
- Create and manage child-namespaces
- Create and manage policies
- Enable and manage secrets engines
training-admin.hcl
# Manage namespaces
path "sys/namespaces/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage policies
path "sys/policies/acl/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# List policies
path "sys/policies/acl" {
capabilities = ["list"]
}
# Enable and manage secrets engines
path "sys/mounts/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
# List available secrets engines
path "sys/mounts" {
capabilities = [ "read" ]
}
# Manage secrets at 'team-secret'
path "team-secret/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
Now, let's deploy the policies!
Also, refer to the Additional Discussion section to learn more about policy authoring with namespaces.
To target a specific namespace, you can do one of the following:
Option 1: Set VAULT_NAMESPACE
so that all subsequent CLI commands will be executed against that particular namespace.**
$ export VAULT_NAMESPACE=<namespace_name>
$ vault policy write <policy_name> <policy_file>
Option 2: Specify the target namespace with -namespace
flag
$ vault policy write -namespace=<namespace_name> <policy_name> <policy_file>
Since you have to deploy policies onto education
and education/training
namespaces, use -namespace
flag instead of environment variable.
Create
edu-admin
policy undereducation
namespace.$ vault policy write -namespace=education edu-admin edu-admin.hcl
Create
training-admin
policy undereducation/training
namespace.$ vault policy write -namespace=education/training training-admin \ training-admin.hcl
To codify the creation of namespaces and policies under each namespaces, read the Codify Management of Vault Enterprise tutorial.
»Setup entities and groups
(Persona: operations)
In the education
namespace, create an entity, Bob Smith with edu-admin
policy attached. Add a userpass
user, bob as an entity alias. By default, Bob Smith has no visibility into the education/training
namespace since the bob
user was defined in the education
namespace.
You are going to create an internal group named, Training Admin in the
education/training
namespace with training-admin
policy attached. To grant
the training-admin
policy for bob
, add the Bob Smith
entity to the
Training Admin
group as a member entity.
This step only demonstrates CLI commands and Web UI to create entities and groups. Refer to the Identity - Entities and Groups tutorial if you need the full details. Also, read the Additional Discussion section for an example of setting up external groups.
Enable the
userpass
auth method.$ vault auth enable -namespace=education userpass
Create a user
bob
under theeducation
namespace.$ vault write -namespace=education \ auth/userpass/users/bob password="training"
Create an entity for
Bob Smith
withedu-admin
policy attached. Save the generated entity ID in a file namedentity_id.txt
.$ vault write -namespace=education -format=json identity/entity name="Bob Smith" \ policies="edu-admin" | jq -r ".data.id" > entity_id.txt
Get the mount accessor for userpass auth method and save it in
accessor.txt
.$ vault auth list -namespace=education -format=json \ | jq -r '.["userpass/"].accessor' > accessor.txt
Create an entity alias for
Bob Smith
to attachbob
.$ vault write -namespace=education identity/entity-alias name="bob" \ canonical_id=$(cat entity_id.txt) mount_accessor=$(cat accessor.txt)
Create a group, "Training Admin" in
education/training
namespace withBob Smith
entity as its member.$ vault write -namespace=education/training identity/group \ name="Training Admin" policies="training-admin" \ member_entity_ids=$(cat entity_id.txt)
»Test the Bob Smith entity
(Persona: org-admin)
Log in as bob
into the education
namespace.
$ vault login -namespace=education -method=userpass \
username="bob" password="training"
Key Value
--- -----
token s.zLNbFJQFWaMR7R5tgobn6xHg.1Vi61
token_accessor xcCNEsRStvQcPEp4AKrkWikk.1Vi61
token_duration 768h
token_renewable true
token_policies ["default"]
identity_policies ["edu-admin"]
policies ["default" "edu-admin"]
token_meta_username bob
Notice that the generated token contains the namespace ID which was created in
(e.g. s.zLNbFJQFWaMR7R5tgobn6xHg
.1Vi61
). User bob
only has default
policy attached to his token (token_policies
); however, he inherited the
edu-admin
policy from the Bob Smith
entity (identity_policies
).
Test to make sure that bob
can create a namespace, enable secrets engine, and
whatever else that you want to verify against the education
namespace.
$ export VAULT_NAMESPACE="education"
Verify that you can create a new namespace called web-app
.
$ vault namespace create web-app
Success! Namespace created at: education/web-app/
Verify that you can enable key/value v2 secrets engine at edu-secret
.
$ vault secrets enable -path=edu-secret kv-v2
Success! Enabled the kv-v2 secrets engine at: edu-secret/
Optionally, you can create new policies to test that bob
can perform the
operations as expected. When you are done testing, unset the VAULT_NAMESPACE
environment variable.
$ unset VAULT_NAMESPACE
»Test the training admin group
(Persona: team-admin)
Stay logged in as bob
and look up the token details.
$ vault token lookup
Key Value
--- -----
# ...snip...
external_namespace_policies map[ygcTv:[training-admin]]
id s.zLNbFJQFWaMR7R5tgobn6xHg.1Vi61
identity_policies [edu-admin]
issue_time 2019-09-26T16:29:33.960232-07:00
meta map[username:bob]
namespace_path education/
num_uses 0
orphan true
path auth/userpass/login/bob
policies [default]
renewable true
ttl 767h53m55s
type service
Notice that the external_namespace_policies
parameter lists
training-admin
policy. The user bob
inherited this policy from the
Training Admin
group defined in the education/training
namespace although
bob
user was created in the education
namespace.
Verify that bob
can perform the operations permitted by the
training-admin
policy.
Set the target namespace as an env variable.
$ export VAULT_NAMESPACE="education/training"
Create a new namespace called vault-training
.
$ vault namespace create vault-training
Key Value
--- -----
id nQjeG
path education/training/vault-training/
Enable key/value v1 secrets engine at team-secret
.
$ vault secrets enable -path=team-secret -version=1 kv
Success! Enabled the kv secrets engine at: team-secret/
When you are done testing, unset the VAULT_NAMESPACE environment variable.
$ unset VAULT_NAMESPACE
Summary:
As this tutorial demonstrated, each namespace you created behaves as an
isolated Vault environment. By default, there is no visibility into other
namespaces regardless of its hierarchical relationship. In order for Bob to
operate in education/training
namespace, you can enable an auth method in the education/training
namespace so that he can log in. Or, as demonstrated in this tutorial, you can use Vault identity to associate entities and groups defined in different namespaces.
NOTE: Bob still needs to log into the education
namespace since his
token is tied to the education
namespace and it is invalid to log into the
education/training
namespace.
»Audit ambient credentials
(Persona: operator)
Many auth and secrets providers, such as AWS, Azure, GCP, and AliCloud, use ambient credentials to authenticate API calls. For example, AWS may:
- Use an access key and secret key configured in Vault.
- If not present, check for environment variables such as "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY".
- If not present, load credentials configured in "~/.aws/credentials".
- If not present, use instance metadata.
This becomes a problem if these ambient credentials are not intended to be used within a particular namespace.
For example, suppose that your Vault server is running on an AWS EC2 instance. You give the owner of a namespace a particular set of permissions to use for AWS. However, that owner does not configure them. So, Vault falls back to using the credentials available in instance metadata, leading to a privilege escalation.
To handle this:
- Ensure no environment variables are available that could grant a privilege escalation.
- Ensure that any privileges granted through instance metadata (in this example) or other ambient identity info represent a loss of privilege.
- Directly configure the correct credentials in namespaces, and restrict access to that endpoint so credentials can't later be edited to use ambient credentials.
Summary: As this tutorial demonstrated, each namespace you created behaves as an isolated Vault environment. Once you sign into a namespace, there is no visibility into other namespaces regardless of its hierarchical relationship. Tokens, policies, and secrets engines are tied to its namespace; therefore, each client must acquire a valid token for each namespace to access their secrets.
»Additional discussion
»Leveraging identity for auth methods with external groups
For simplicity, this tutorial used the username and password (userpass
) auth
method which was enabled in the education
namespace as well as the
education/training
namespace.
However, most likely, your organization uses an auth method such as ldap
or
okta
and map appropriate policies to those externally defined groups. For
these types of auth methods, you have two options:
- Enable the auth method in each namespace the same way you performed in the setup entities and groups step with
userpass
. - Enable the auth method in the root namespace and use Identity Groups to pull in external groups and map policies in each namespace.
NOTE: As mentioned in the Solution section, identity groups can pull in entities and groups from other namespaces.
The second option enables an auth method in the root namespace rather than enable in multiple namespaces. The following steps demonstrate the second option to create the "Training Admin" group as described in this tutorial.
First, enable and configure the desired auth method (e.g. LDAP) in the root namespace.
$ vault auth enable ldap
Configure the LDAP auth method.
$ vault write auth/ldap/config \
url="ldap://ldap.example.com" \
userdn="ou=Users,dc=example,dc=com" \
groupdn="ou=Groups,dc=example,dc=com" \
groupfilter="(&(objectClass=group)(member:1.2.840.113556.1.4.1941:={{.UserDN}}))" \
groupattr="cn" \
upndomain="example.com" \
certificate=@ldap_ca_cert.pem \
insecure_tls=false \
starttls=true
For each LDAP group to which you want to map policies, create an external group in the root namespace with an alias whose name exactly matches the LDAP group name.
Get the mount accessor for ldap auth method and save it in accessor.txt
.
$ vault auth list -format=json \
| jq -r '.["ldap/"].accessor' > accessor.txt
Create an external group and save the generated group ID in group_id.txt
.
$ vault write -format=json identity/group name="training_admin_root" \
type="external" \
| jq -r ".data.id" > group_id.txt
Create a group alias whose name exactly matches the LDAP group name. The
following example uses a group name in LDAP is ops_training
.
$ vault write -format=json identity/group-alias name="ops_training" \
mount_accessor=$(cat accessor.txt) \
canonical_id=$(cat group_id.txt)
In the education/training
namespace, create an internal group which has
the external group (training_admin_root
) as its member. Attach the
training-admin
policy to this internal group. Remember that you created this
policy inside of the education/training
namespace in the write policies step.
$ vault write -namespace=education/training identity/group \
name="Training Admin" \
policies="training-admin" \
member_group_ids=$(cat group_id.txt)
NOTE: Remember that the auth methods are enabled in an explicit
namespace. Therefore, the users must authenticate into the namespace where the
auth method is configured (in this case, the ldap
auth method is configured in
the root namespace).
Let's assume the user, "bob_smith" belongs to the LDAP ops_training
group.
$ vault login -method=ldap username=bob_smith
Password (will be hidden):
Success! You are now authenticated.
# ...snip...
When you check the returned token's properties, the output should display
training-admin
as its external_namespace_policies
.
$ vault token Lookup
Key Value
--- -----
...snip...
external_namespace_policies map[9dKXw:[training-admin]]
id s.10LCp6O5xnweGVm1eyJbp127
identity_policies <nil>
...snip...
policies [default]
renewable true
ttl 767h59m11s
type service
Since you did not assign any policy when you created the training_admin_root
external group, the identity_policies
shows nil
. Therefore, bob_smith's
token only has the default
policy in the root namespace. However, it has
training-admin
policy in the external namespace of ID, 9dKXw
(in this
example, it is the ID of education/training
namespace).
Set the target namespace as an environment variable for convenience, and
bob_smith
is ready to operate against the education/training
namespace.
$ export VAULT_NAMESPACE="education/training"
The namespaces feature is designed to create an isolation around each namespace so that Vault tenants can operate independently by having a mini-Vault environment of their own. However, the second option leaves some dependency on the root namespace. The root-level admin is responsible for creating the mapping between the LDAP groups and the Vault's identity groups. The users log into the root namespace instead of logging in the target namespace.
»Policy with namespaces
In this tutorial, you created policies in each namespace (education
and
education/training
). Therefore, you did not have to specify the target
namespace in the policy paths.
If you want to create policies in the root namespace to control education
and
education/training
namespaces, prepend the namespace in the paths.
For example:
# Manage policies in the 'education' namespace
path "education/sys/policies/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage tokens in the 'education' namespace
path "education/auth/token/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage policies under 'education/training' namespace
path "education/training/sys/policies/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage tokens in the 'education/training' namespace
path "education/training/auth/token/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
...
In the write policies step, you deployed the training-admin
policy in
the education/training
namespace. The path is relative to the working
namespace. So, if you want to create the training-admin
policy in the
education
namespace instead, the paths starts with training/
rather than
education/training/
.
# Manage namespaces
path "training/sys/namespaces/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
# Manage policies
path "training/sys/policies/*" {
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
...
NOTE: Policies are tied to its namespace. Therefore, if you created the
training_admin
policy in the education
namespace, it is available only
to those clients authenticated in the education
namespace.