Configuration entries are Consul resources used to define datacenter-wide configuration
defaults for various aspects of the service mesh, or to create service specific
configurations. As of Consul 1.9, configuration entries
can be created as Custom Resource Defintions (CRDs), and managed using
In this tutorial you will:
- Learn how to work with config entries as Custom Resource Definitions (CRDs)
- Download the latest helm chart
- Install or upgrade Consul to enable CRDs
- Deploy an application workload
- Configure a service intention
- Modify a service intention
- Delete a service intention
To complete this tutorial you will need:
This guide was tested with:
- Kubernetes v1.18.2
- Minikube v1.10.1
- kind v0.8.1
- Helm v3.2.1
»Working with config entry Custom Resource Definitions (CRDs)
Prior to Consul 1.9, when using configuration entries with Consul in Kubernetes,
an operator would either need to
exec into a running container, or configure
a host that could interact with the datacenter using a local Consul binary.
In versions older than 1.9, configuration entries have to be managed with the Consul CLI,
the HTTP API, or provided to agents during startup as configuration files.
As of Consul 1.9, most configuration entries can be managed as Kubernetes Custom
Resource Definitions (CRDs). You can now define most configuration entries as YAML,
and register them with Consul using the familiar
kubectl apply command.
The configuration entries currently available as CRDs for Consul on Kubernetes are:
- proxy-defaults - controls proxy configuration
- service-defaults - configures defaults for all the instances of a given service
- service-resolver - matches service instances with a specific Connect upstream discovery requests
- service-router - defines where to send layer 7 traffic based on the HTTP route
- service-splitter - defines how to divide requests for a single HTTP route based on percentages
- service-intentions - defines restrictions for specific service to service interactions
»Download Helm chart
If you have not already done so, download the latest official consul-helm chart now.
$ helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" has been added to your repositories
»Verify chart version
To ensure you have version
0.27.0 of the Helm chart, search your local repo.
$ helm search repo hashicorp/consul
NAME CHART VERSION APP VERSION DESCRIPTION hashicorp/consul 0.27.0 1.9.0 Official HashiCorp Consul Chart
If you do not see that you have the correct version of the chart locally, try updating your Helm repo.
$ helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "hashicorp" chart repository
»Enable Consul CRDs
CRDs can be enabled in a Consul on Kubernetes datacenter by adding the following top level stanza to the Helm chart configuration file, and then installing or upgrading the datacenter using the chart with the updated configuration file.
controller: enabled: true
No other changes to the Helm chart configuration is required.
Note: This feature is only available using the official consul-helm chart versions 0.25 and higher.
You can use the command below to create a minimal, unsecured, development datacenter for testing this tutorial. This configuration is not suitable for production use. See the Secure Consul and Registered Services on Kubernetes tutorial for instructions on how to configure a production datacenter.
$ cat > config.yaml <<EOF global: name: consul datacenter: dc1 server: # use 1 server replicas: 1 bootstrapExpect: 1 disruptionBudget: enabled: true maxUnavailable: 0 connectInject: enabled: true # inject an envoy sidecar into every new pod, # except for those with annotations that prevent injection default: true # enable CRDs controller: enabled: true EOF
Once you have created the configuration file, install Consul to your Kubernetes cluster using helm.
$ helm install -f ./config.yaml consul hashicorp/consul --version "0.27.0" --wait
NAME: consul ...OMITTED... $ helm status consul $ helm get all consul
»Deploy a demo application workload
Now that you have a datacenter with CRDs enabled, you will deploy an demo application to test the features. First, from the command line, clone the GitHub repository that contains the configuration you will use with this tutorial.
$ git clone https://github.com/hashicorp/learn-consul-kubernetes.git
Next, change into the directory containing the repository you just cloned.
$ cd learn-consul-kubernetes
Now, checkout the tagged version verified for this tutorial.
git checkout tags/v0.0.3
Deploy the demo application using the
kubectl apply command.
$ kubectl apply -f custom-resource-definitions/hashicups
You should receive the following output.
service/frontend-service created configmap/nginx-configmap created deployment.apps/frontend created service/products-api-service created serviceaccount/products-api created configmap/db-configmap created deployment.apps/products-api created service/postgres created deployment.apps/postgres created service/public-api-service created deployment.apps/public-api created
HashiCups may take a minute or more to deploy and start. Use the following command to watch your deployment.
$ watch kubectl get pods
The output of that command will be similar to the following. The application is
ready for use when all pods show a status of
NAME READY STATUS RESTARTS AGE consul-6gjth 1/1 Running 0 9m18s consul-connect-injector-webhook-deployment-7f74c84444-b89m9 1/1 Running 0 9m18s consul-controller-69c8b5f65d-kw76c 1/1 Running 0 9m18s consul-server-0 1/1 Running 0 9m16s consul-webhook-cert-manager-665b8d7997-qllc7 1/1 Running 0 9m18s frontend-cf78c4879-5rjk8 3/3 Running 0 3m28s postgres-7d774895db-7xq7v 3/3 Running 0 3m28s product-api-66d6865657-br2fp 3/3 Running 1 3m28s public-api-69fb4d6ffc-8mqlx 3/3 Running 0 3m29s
Type CTRL-C to stop watching the pods once all pods have a status of
You can test the application by viewing the user interface. Do this by forwarding the frontend deployment's port 80 to your development host.
$ kubectl port-forward deploy/frontend 8081:80
Forwarding from 127.0.0.1:8081 -> 80 Forwarding from [::1]:8081 -> 80
Note: Since the port forwarding process is running in the foreground, open a new terminal window, and navigate to the same directory to complete the rest of the tutorial.
localhost:8081 in a browser window. You should observe the following screen.
»Configure a service intention
Also new as of Consul 1.9, is the ability to define permissions based on application-layer (i.e. layer 7) request attributes such as the HTTP method, path, or the presence/absence of a header.
To create the service intention for this tutorial, you first need to define the protocols for the frontend and public-api services as HTTP because application aware intentions can only operate on HTTP services. To do so, you need to create ServiceDefaults config entries for the frontend and public-api services:
apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: frontend spec: protocol: 'http' --- apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: public-api spec: protocol: 'http'
Next, you need to define a ServiceIntentions config entry that manages the traffic between the frontend and public-api services. ServiceIntentions can have only a single destination, but can have multiple sources. In this example, the destination will be public-api and frontend will be the only source:
apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions metadata: name: public-api spec: # Name of the destination service affected by this ServiceIntentions entry destination: name: public-api # The set of traffic sources affected by this ServiceIntentions entry sources: - name: frontend
Next, you need to define what actions are allowed or denied using the permissions stanza. First, deny all traffic from the frontend service to the health route. The health route is only for internal use and could be abused by external users, so traffic from the frontend service to this exact path should be denied.
- action: deny http: pathExact: "/health"
Next, you need to define the primary allow action. In the HashiCups application,
it is required that the client passes an
Authorization header. Also, the
application only allows the GET, PUT, POST, and DELETE HTTP methods.
- action: allow http: pathPrefix: '/' methods: - GET - PUT - POST - DELETE header: - name: 'Authorization' present: true
Finally, you need to define a catch-all deny action. In this particular datacenter, there is no default deny policy defined, so you will need to define a deny action for this specific destination service.
- action: deny http: pathPrefix: '/'
By placing this at the end of the list, any traffic that doesn't match either of the previous actions will trigger the action, and be denied.
Putting it all together, your manifest should resemble the following:
# Application aware intentions can only be applied to services that use the HTTP protocol # so you must first define service default config entries for the targetted # services. apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: frontend spec: protocol: 'http' --- apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: public-api spec: protocol: 'http' --- apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions metadata: name: public-api spec: # Name of the destination service affected by this ServiceIntentions entry destination: name: public-api # The set of traffic sources affected by this ServiceIntentions entry sources: # The first affected traffic source-maps to service name of inbound traffic source - name: frontend # The set of permissions to apply when frontend is the traffic source # The first permission to match in the list is terminal and stops further evaluation. permissions: # Add this to always deny traffic from the frontend service to /health route - action: deny http: pathExact: '/health' # This permission now defines the conditions that should be allowed # Allow traffic to all paths for the GET, PUT, POST, DELETE verbs as long as an # Authorization header is present - action: allow http: pathPrefix: '/' methods: - GET - PUT - POST - DELETE header: - name: 'Authorization' present: true # Define a deny intention for all other traffic - action: deny http: pathPrefix: '/'
The repository you downloaded earlier in the tutorial contains a
directory at the path
custom-resource-definitions. In that directory you will find a file
service-intentions.yaml that is ready to be applied.
kubectl apply to register the service intention resource with Kubernetes.
$ kubectl apply -f custom-resource-definitions/service-intentions.yaml
servicedefaults.consul.hashicorp.com/frontend created servicedefaults.consul.hashicorp.com/public-api created serviceintentions.consul.hashicorp.com/public-api created
Now you can interact with the service intention using
kubectl just like you can
any other kube-native resource. For example, you can get a list of all service
intentions with this command:
$ kubectl get serviceintentions
NAME SYNCED public-api True
You can also inspect details about the service intention using
kubectl describe like so:
$ kubectl describe serviceintentions/public-api
Abbreviated example output:
Name: public-api Namespace: default Labels: <none> Annotations: API Version: consul.hashicorp.com/v1alpha1 Kind: ServiceIntentions ...OMITTED... Spec: Destination: Name: public-api Sources: Name: frontend Permissions: Action: deny Http: Path Exact: /health Action: allow Http: Header: Name: Authorization Present: true Methods: GET PUT POST DELETE Path Prefix: / Status: Conditions: Last Transition Time: 2020-10-08T15:13:15Z Status: True Type: Synced Events: <none>
This configuration should allow traffic, because the frontend service passes
the required header and uses the GET HTTP method. Revisit the application in
a browser tab at
localhost:8081, and verify you can still visit the web page.
»Modify a service intention
Next, modify the service intention you created a moment ago.
$ sed -i '' 's/Authorization/api-token/g' custom-resource-definitions/service-intentions.yaml
This script updates the service intention. It changes the name of the required header to a value the application does not provide. Review the file and notice the "allow" action has been altered. The header key "Authorization" has been changed to "api-token".
header: - name: 'api-token' present: true
kubectl apply to apply the modified configuration.
$ kubectl apply -f custom-resource-definitions/service-intentions.yaml
servicedefaults.consul.hashicorp.com/frontend unchanged servicedefaults.consul.hashicorp.com/public-api unchanged serviceintentions.consul.hashicorp.com/public-api configured
Now, navigate to
localhost:8081 in a browser window. You should observe that you
are no longer able to successfully visit the page. If you inspect the network call
in the browser's developer tools, you should observe that the request was denied with
the following error message:
RBAC: access denied
»View the service intention in the Consul UI
To access the Consul UI, forward the
consul-server-0 pod's port
8500 to the
development host, so that you can access the Consul UI in a browser.
$ kubectl port-forward consul-server-0 8500:8500
Forwarding from 127.0.0.1:8500 -> 8500 Forwarding from [::1]:8500 -> 8500
http://localhost:8500 in a new browser tab. Click on "Intentions" in
the top navigation bar, and you will observe a list of intentions. The screen
have one entry as illustrated in this screenshot.
Notice there is a badge next to the
frontend entry indicating that the service
L7 intentions applied. Also, noticed that there is a badge that says
Managed by CRDs. Click on the row in the list, and you will be redirected to a
page that contains the service intentions configuration details and looks like this:
»Delete a service intention
kubectl delete to remove the service intention.
$ kubectl delete -f custom-resource-definitions/service-intentions.yaml
servicedefaults.consul.hashicorp.com "frontend" deleted servicedefaults.consul.hashicorp.com "public-api" deleted serviceintentions.consul.hashicorp.com "public-api" deleted
If you navigate back to the "Intentions" screen in the Consul UI, you will see that the list is now empty.
localhost:8081 in a browser window. You should once again be
able to visit the page.
In this tutorial you:
- Learned how to work with config entries as Custom Resource Definitions (CRDs)
- Downloaded the latest helm chart
- Installed/upgraded a cluster to enable Consul CRDs
- Deployed an application workload
- Configured a service intention
- Modified a service intention
- Deleted a service intention
To learn more about how to manage Consul config entries using Consul on Kubernetes visit the documentation.
To learn more about securing your service mesh, check out our Securing Agents and Registered Services tutorial.