April 6 & 7
Learn about Vault, Consul, & more at HashiDays Sydney in Australia Register Now

Secure Nomad with Access Control

Nomad ACL Policy Concepts

ACL policies are written using HashiCorp Configuration Language (HCL). This language is designed for human readability. The HCL interpreter also parses JSON which facilitates the use of machine-generated configuration.

An ACL policy contains one or more rules. Rules contain coarse-grained policy dispositions. Rules typically have several policy dispositions:

  • read: allow the resource to be read but not modified

  • write: allow the resource to be read and modified

  • deny: do not allow the resource to be read or modified. Deny takes precedence when multiple policies are associated with a token.

Some rules, such as namespace and host_volume, also allow the policy designer to specify a policy in terms of a coarse-grained policy disposition, fine-grained capabilities, or a combination of the two.

ACL policy specification

An ACL policy is a composition of one or more rules. Its specification in the HCL format looks like:

# Allow read only access to the default namespace
namespace "default" {
  policy = "read"
}

# Allow writing to the `foo` namespace
namespace "foo" {
  policy = "write"
}

agent {
  policy = "read"
}

node {
  policy = "read"
}

quota {
  policy = "read"
}

The JSON representation of the same policy:

{
  "namespace": {
    "default": {
      "policy": "read"
    },
    "foo": {
      "policy": "write"
    }
  },
  "agent": {
    "policy": "read"
  },
  "node": {
    "policy": "read"
  },
  "quota": {
    "policy": "read"
  }
}

The ACL Policy API allows either HCL or JSON to be used to define the content of the rules section. Because HCL is designed to be more operator-friendly and allows comments, this guide will use HCL for all examples and snippets.

Namespace rules

While Nomad open-source only offers the "default" namespace, Nomad Enterprise allows operators to create multiple namespaces to provide granular access to cluster resources.

The namespace rule controls access to a namespace. Namespaces contain the majority of the active work of the cluster Jobs API, Deployments API, Allocations API, and Evaluations API.

namespace "default" {
  policy = "write"
}

namespace "sensitive" {
  policy = "read"
}

Namespace rules are keyed by the namespace name they apply to. When no namespace is specified, the "default" namespace is the one used. For example, the above policy grants write access to the default namespace, and read access to the sensitive namespace. In addition to the coarse-grained policy disposition, the namespace stanza allows setting a more fine grained list of capabilities. This includes:

  • deny - When multiple policies are associated with a token, deny will take precedence and prevent any capabilities.

  • list-jobs - Allows listing the jobs and seeing coarse grain status.

  • read-job - Allows inspecting a job and seeing fine grain status.

  • submit-job - Allows jobs to be submitted or modified.

  • dispatch-job - Allows jobs to be dispatched

  • read-logs - Allows the logs associated with a job to be viewed.

  • read-fs - Allows the filesystem of allocations associated to be viewed.

  • alloc-exec - Allows an operator to connect and run commands in running allocations.

  • alloc-node-exec - Allows an operator to connect and run commands in allocations running without filesystem isolation, for example, raw_exec jobs.

  • alloc-lifecycle - Allows an operator to stop individual allocations manually.

  • sentinel-override - Allows soft mandatory policies to be overridden.

The coarse-grained policy dispositions are shorthand for the following fine- grained namespace capabilities:

PolicyCapabilities
denydeny
readlist-jobs
read-job
writelist-jobs
read-job
submit-job
dispatch-job
read-logs
read-fs
alloc-exec
alloc-lifecycle

When both the policy shorthand and a capabilities list are provided, the capabilities are merged. This policy adds the submit-job capability to the read policy disposition, which provide the list-job and read-job capabilities:

# Allow reading jobs and submitting jobs, without allowing access
# to view log output or inspect the filesystem
namespace "default" {
  policy       = "read"
  capabilities = ["submit-job"]
}

This policy could be expressed as:

# Allow reading jobs and submitting jobs, without allowing access
# to view log output or inspect the filesystem
namespace "default" {
  capabilities = ["submit-job","list-jobs","read-job"]
}

Namespace definitions may also include globs, allowing a single policy definition to apply to a set of namespaces. For example, the below policy allows read access to most production namespaces, but allows write access to the "production-api" namespace, and rejects any access to the "production-web" namespace.

namespace "production-*" {
  policy = "read"
}

namespace "production-api" {
  policy = "write"
}

namespace "production-web" {
  policy = "deny"
}

Namespaces are matched to their rules first by performing a lookup on any exact match, before falling back to performing a glob based lookup. When looking up namespaces by glob, the matching policy with the greatest number of matched characters will be chosen. For example:

namespace "*-web" {
  policy = "deny"
}

namespace "*" {
  policy = "write"
}

Will evaluate to deny for production-web, because it is 9 characters different from the "*-web" rule, but 13 characters different from the "*" rule.

Node rules

The node rule controls access to the Node API such as listing nodes or triggering a node drain. Node rules are specified for all nodes using the node key:

node {
  policy = "read"
}

There's only one node rule allowed per Nomad ACL Policy, and its value is set to one of the policy dispositions.

Agent rules

The agent rule controls access to the utility operations in the Agent API, such as join and leave. Agent rules are specified for all agents using the agent key:

agent {
  policy = "write"
}

There's only one agent rule allowed per Nomad ACL Policy, and its value is set to one of the policy dispositions.

Operator rules

The operator rule controls access to the Operator API. Operator rules look like:

operator {
  policy = "read"
}

There's only one operator rule allowed per Nomad ACL Policy, and its value is set to one of the policy dispositions. In the example above, the token could be used to query the operator endpoints for diagnostic purposes but not make any changes.

Quota rules

The quota policy controls access to the quota specification operations in the Quota API, such as quota creation and deletion. Quota rules are specified for all quotas using the quota key:

quota {
  policy = "write"
}

There's only one quota rule allowed per Nomad ACL Policy, and its value is set to one of the policy dispositions.

Host Volume rules

The host_volume policy controls access to mounting and accessing host volumes.

host_volume "*" {
  policy = "write"
}

host_volume "prod-*" {
  policy = "deny"
}

host_volume "prod-ca-certificates" {
  policy = "read"
}

Host volume rules are keyed to the volume names that they apply to. As with namespaces, you may use wildcards to reuse the same configuration across a set of volumes. In addition to the coarse grained policy specification, the host_volume stanza allows setting a more fine grained list of capabilities. This includes:

  • deny - Do not allow a user to mount a volume in any way.

  • mount-readonly - Only allow the user to mount the volume as readonly

  • mount-readwrite - Allow the user to mount the volume as readonly or readwrite if the host_volume configuration allows it.

The course grained policy permissions are shorthand for the fine grained capabilities:

  • deny policy - ["deny"]

  • read policy - ["mount-readonly"]

  • write policy - ["mount-readonly", "mount-readwrite"]

When both the policy short hand and a capabilities list are provided, the capabilities are merged.

Next Steps

Now that you have learned the basic building blocks of Nomad's ACL policies, the next section allows you to put this knowledge into practice.