The Amazon Elastic Kubernetes Service (EKS) is the AWS service for deploying, managing, and scaling containerized applications with Kubernetes.
In this tutorial, you will deploy an EKS cluster using Terraform. Then, you will configure kubectl
using Terraform output to deploy a Kubernetes dashboard on the cluster.
Warning! AWS charges
$0.10
per hour for each EKS cluster.
As a result, you may be charged to run these examples. The most you should be
charged should only be a few dollars, but we're not responsible for any charges
that may incur.
»Why deploy with Terraform?
While you could use the built-in AWS provisioning processes (UI, CLI, CloudFormation) for EKS clusters, Terraform provides you with several benefits:
Unified Workflow - If you are already deploying infrastructure to AWS with Terraform, your EKS cluster can fit into that workflow. You can also deploy applications into your EKS cluster using Terraform.
Full Lifecycle Management - Terraform doesn't only create resources, it updates, and deletes tracked resources without requiring you to inspect the API to identify those resources.
Graph of Relationships - Terraform understands dependency relationships between resources. For example, if an AWS Kubernetes cluster needs a specific VPC and subnet configurations, Terraform won't attempt to create the cluster if the VPC and subnets failed to create with the proper configuration.
»Prerequisites
The tutorial assumes some basic familiarity with Kubernetes and kubectl
but does
not assume any pre-existing deployment.
It also assumes that you are familiar with the usual Terraform plan/apply workflow. If you're new to Terraform itself, refer first to the Getting Started tutorial.
For this tutorial, you will need:
- an AWS account with the IAM permissions listed on the EKS module documentation,
- a configured AWS CLI
- AWS IAM Authenticator
kubectl
- wget (required for the
eks
module)
In order for Terraform to run operations on your behalf, you must install and configure the AWS CLI tool. To install the AWS CLI, follow these instructions or choose a package manager based on your operating system.
Use the package manager homebrew
to install the AWS CLI.
$ brew install awscli
After you've installed the AWS CLI, configure it by running aws configure
.
When prompted, enter your AWS Access Key ID, Secret Access Key, region and output format.
$ aws configure
AWS Access Key ID [None]: YOUR_AWS_ACCESS_KEY_ID
AWS Secret Access Key [None]: YOUR_AWS_SECRET_ACCESS_KEY
Default region name [None]: YOUR_AWS_REGION
Default output format [None]: json
If you don't have an AWS Access Credentials, create your AWS Access Key ID and Secret Access Key by navigating to your service credentials in the IAM service on AWS. Click "Create access key" here and download the file. This file contains your access credentials.
Your default region can be found in the AWS Web Management Console beside your username. Select the region drop down to find the region name (eg. us-east-1) corresponding with your location.
»Set up and initialize your Terraform workspace
In your terminal, clone the following repository. It contains the example configuration used in this tutorial.
$ git clone https://github.com/hashicorp/learn-terraform-provision-eks-cluster
You can explore this repository by changing directories or navigating in your UI.
$ cd learn-terraform-provision-eks-cluster
In here, you will find six files used to provision a VPC, security groups and an EKS cluster. The final product should be similar to this:
vpc.tf
provisions a VPC, subnets and availability zones using the AWS VPC Module. A new VPC is created for this tutorial so it doesn't impact your existing cloud environment and resources.security-groups.tf
provisions the security groups used by the EKS cluster.eks-cluster.tf
provisions all the resources (AutoScaling Groups, etc...) required to set up an EKS cluster in the private subnets and bastion servers to access the cluster using the AWS EKS Module.On line 14, the AutoScaling group configuration contains three nodes.
worker_groups = [ { name = "worker-group-1" instance_type = "t2.small" additional_userdata = "echo foo bar" asg_desired_capacity = 2 additional_security_group_ids = [aws_security_group.worker_group_mgmt_one.id] }, { name = "worker-group-2" instance_type = "t2.medium" additional_userdata = "echo foo bar" additional_security_group_ids = [aws_security_group.worker_group_mgmt_two.id] asg_desired_capacity = 1 }, ]
outputs.tf
defines the output configuration.versions.tf
sets the Terraform version to at least 0.12. It also sets versions for the providers used in this sample.
»Initialize Terraform workspace
Once you have cloned the repository, initialize your Terraform workspace, which will download and configure the providers.
$ terraform init
Initializing modules...
Downloading terraform-aws-modules/eks/aws 9.0.0 for eks...
- eks in .terraform/modules/eks/terraform-aws-modules-terraform-aws-eks-908c656
- eks.node_groups in .terraform/modules/eks/terraform-aws-modules-terraform-aws-eks-908c656/modules/node_groups
Downloading terraform-aws-modules/vpc/aws 2.6.0 for vpc...
- vpc in .terraform/modules/vpc/terraform-aws-modules-terraform-aws-vpc-4b28d3d
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "template" (hashicorp/template) 2.1.2...
- Downloading plugin for provider "kubernetes" (hashicorp/kubernetes) 1.10.0...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.52.0...
- Downloading plugin for provider "random" (hashicorp/random) 2.2.1...
- Downloading plugin for provider "local" (hashicorp/local) 1.4.0...
- Downloading plugin for provider "null" (hashicorp/null) 2.1.2...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
»Provision the EKS cluster
In your initialized directory, run terraform apply
and review the planned actions.
Your terminal output should indicate the plan is running and what resources will be created.
$ terraform apply
module.eks.data.aws_partition.current: Refreshing state...
module.eks.data.aws_caller_identity.current: Refreshing state...
module.eks.data.aws_ami.eks_worker: Refreshing state...
data.aws_availability_zones.available: Refreshing state...
module.eks.data.aws_iam_policy_document.cluster_assume_role_policy: Refreshing state...
module.eks.data.aws_ami.eks_worker_windows: Refreshing state...
module.eks.data.aws_iam_policy_document.workers_assume_role_policy: Refreshing state...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
<= read (data resources)
Terraform will perform the following actions:
## output truncated ...
Plan: 51 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value:
You can see this terraform apply will provision a total of 51 resources (VPC,
Security Groups, AutoScaling Groups, EKS Cluster, etc...).
If you're comfortable with this, confirm the run with a yes
.
This process should take approximately 10 minutes. Upon successful application,
your terminal prints the outputs defined in outputs.tf
.
Apply complete! Resources: 51 added, 0 changed, 0 destroyed.
Outputs:
cluster_endpoint = https://A1ADBDD0AE833267869C6ED0476D6B41.gr7.us-east-2.eks.amazonaws.com
cluster_security_group_id = sg-084ecbab456328732
kubectl_config = apiVersion: v1
preferences: {}
kind: Config
clusters:
- cluster:
server: https://A1ADBDD0AE833267869C6ED0476D6B41.gr7.us-east-2.eks.amazonaws.com
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01ETXdPVEU0TXpVeU1sb1hEVE13TURNd056RTRNelV5TWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTThkClZaN1lmbjZmWm41MEgwL0d1Qi9lRmVud2dydXQxQlJWd29nL1JXdFpNdkZaeStES0FlaG5lYnR5eHJ2VVZWMXkKTXVxelBiMzgwR3Vla3BTVnNTcDJVR0ptZ2N5UVBWVi9VYVZDQUpDaDZNYmIvL3U1bWFMUmVOZTBnb3VuMWlLbgpoalJBYlBJM2JvLzFPaGFuSXV1ejF4bmpDYVBvWlE1U2N5MklwNnlGZTlNbHZYQmJ6VGpESzdtK2VST2VpZUJWCjJQMGd0QXJ3alV1N2MrSmp6OVdvcGxCcTlHZ1RuNkRqT1laRHVHSHFRNEpDUnRsRjZBQXpTUVZ0cy9aRXBnMncKb2NHakd5ZE9pSmpMb1NsYU9weDIrMTNMbHcxMDAvNmY4Q0F2ajRIbFZUZDBQOW5rN1UyK04xNSt5VjRpNjFoQgp3bHl4SXFUWEhDR0JvYmRNNE5VQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFIbEI3bGVMTnJYRXZnNksvNUdtR2s5Tlh4SUkKRDd0Y1dkTklBdnFka1hWK3ltVkxpTXV2ZGpQVjVKV3pBbEozTWJqYjhjcmlQWWxnVk1JNFJwc0N0aGJnajMzMwpVWTNESnNxSmZPUUZkUnkvUTlBbHRTQlBqQldwbEtaZGc2dklxS0R0eHB5bHovOE1BZ1RldjJ6Zm9SdzE4ZnhCCkI2QnNUSktxVGZCNCtyZytVcS9ULzBVS1VXS0R5K2gyUFVPTEY2dFVZSXhXM2RncWh0YWV3MGJnQmZyV3ZvSW8KYitSOVFDTk42UHRQNEFFQSsyQnJYYzhFTmd1M2EvNG9rN3lPMjZhTGJLdC9sbUNoNWVBOEdBRGJycHlWb3ZjVgpuTGdyb0FvRnVRMCtzYjNCTThUcEtxK0YwZ2dwSFptL3ZFNjh5NUk1VFlmUUdHeEZ6VEVyOHR5NHk1az0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
name: eks_training-eks-TNajBRIF
contexts:
- context:
cluster: eks_training-eks-TNajBRIF
user: eks_training-eks-TNajBRIF
name: eks_training-eks-TNajBRIF
current-context: eks_training-eks-TNajBRIF
users:
- name: eks_training-eks-TNajBRIF
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws-iam-authenticator
args:
- "token"
- "-i"
- "training-eks-TNajBRIF"
region = us-east-2
»Configure kubectl
Now that you've provisioned your EKS cluster, you need to configure kubectl
.
Run the following command to retrieve the access credentials for your cluster
and automatically configure kubectl
.
$ aws eks --region $(terraform output -raw region) update-kubeconfig --name $(terraform output -raw cluster_name)
The Kubernetes cluster name and region correspond to the output variables showed after the successful Terraform run.
»Deploy and access Kubernetes Dashboard
To verify that your cluster is configured correctly and running, you will deploy the Kubernetes dashboard and navigate to it in your local browser.
While you can deploy the Kubernetes metrics server and dashboard using Terraform, kubectl
is used in this tutorial so you don't need to configure your Terraform Kubernetes Provider.
»Deploy Kubernetes Metrics Server
The Kubernetes Metrics Server, used to gather metrics such as cluster CPU and memory usage over time, is not deployed by default in EKS clusters.
Download and unzip the metrics server by running the following command.
$ wget -O v0.3.6.tar.gz https://codeload.github.com/kubernetes-sigs/metrics-server/tar.gz/v0.3.6 && tar -xzf v0.3.6.tar.gz
Deploy the metrics server to the cluster by running the following command.
$ kubectl apply -f metrics-server-0.3.6/deploy/1.8+/
Verify that the metrics server has been deployed. If successful, you should see something like this.
$ kubectl get deployment metrics-server -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 1/1 1 1 4s
»Deploy Kubernetes Dashboard
The following command will schedule the resources necessary for the dashboard.
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created
Now, create a proxy server that will allow you to navigate to the dashboard
from the browser on your local machine. This will continue running until you
stop the process by pressing CTRL + C
.
$ kubectl proxy
You should be able to access the Kubernetes dashboard here
(http://127.0.0.1:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
).
»Authenticate the dashboard
To use the Kubernetes dashboard, you need to create a ClusterRoleBinding
and
provide an authorization token. This gives the cluster-admin
permission to
access the kubernetes-dashboard
.
Authenticating using kubeconfig
is not an option. You can read more about
it in the Kubernetes documentation.
In another terminal (do not close the kubectl proxy
process), create the
ClusterRoleBinding
resource.
$ kubectl apply -f https://raw.githubusercontent.com/hashicorp/learn-terraform-provision-eks-cluster/master/kubernetes-dashboard-admin.rbac.yaml
Then, generate the authorization token.
$ kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep service-controller-token | awk '{print $1}')
Name: service-controller-token-46qlm
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: service-controller
kubernetes.io/service-account.uid: dd1948f3-6234-11ea-bb3f-0a063115cf22
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6I...
Select "Token" on the Dashboard UI then copy and paste the entire token you receive into the dashboard authentication screen to sign in. You are now signed in to the dashboard for your Kubernetes cluster.
Navigate to the "Cluster" page by clicking on "Cluster" in the left navigation bar. You should see a list of nodes in your cluster.
»Clean up your workspace
Congratulations, you have provisioned an EKS cluster, configured kubectl
,
and deployed the Kubernetes dashboard.
If you'd like to learn how to manage your EKS cluster using the Terraform Kubernetes Provider, leave your cluster running and continue to the Kubernetes provider Learn tutorial.
If not, remember to destroy any resources you create once you are done with this
tutorial. Run the destroy
command and confirm with yes
in your terminal.
$ terraform destroy
»Next steps
For more information on the EKS provider, visit the AWS provider documentation.
For steps on how to manage Kubernetes resources your EKS cluster or any other already created Kubernetes cluster, visit the Kubernetes provider Learn tutorial.
For a more in-depth Kubernetes example, Deploy Consul and Vault on a Kubernetes Cluster using Run Triggers (this tutorial is GKE based).