Virtual Event
Join us for the next HashiConf Digital October 12-15, 2020 Register for Free

Provision Infrastructure Deployed with Terraform

Provision Infrastructure with Packer

Packer is HashiCorp's open-source tool for creating machine images from source configuration. You can configure Packer images with an operating system and software for your specific use-case.

Terraform configuration for a compute instance can use a Packer image to provision your instance without manual configuration.

In this tutorial, you will create a Packer image with a user group, a new user with authorized SSH keys, and a Go web app. Then, you will deploy this image using Terraform. Finally, you will access the instance via SSH to deploy the Go web app.

»Prerequisites

To follow along with this tutorial, you will need:

Clone the example repository here.

$ git clone -b packer https://github.com/hashicorp/learn-terraform-provisioning

Change into your cloned repo directory.

$ cd learn-terraform-provisioning

»Create a local SSH key

For this tutorial, create a local SSH key to pair with the new terraform user you create on this instance.

Generate a new SSH key called tf-packer. The argument provided with the -f flag creates the key in the current directory and creates two files called tf-packer and tf-packer.pub. Change the placeholder email address to your email address.

$ ssh-keygen -t rsa -C "your_email@example.com" -f ./tf-packer

When prompted, press enter to leave the passphrase blank on this key.

»Review the shell script

Packer's configuration will pass it a shell script to run when it builds the image. For more information on the other methods of delivering provisioning instructions to your image, visit the Packer provisioners documentation.

The script for this tutorial will update the default software on your instance to the latest versions, install necessary app software (git, go, etc) and create your user & groups so you can SSH into the machine with the username and SSH key you created above.

Change directories into the scripts directory.

$ cd scripts

Open setup.sh in your file editor and review the provisioning instructions. This script installs the necessary dependencies, adds the terraform user to the sudo group, installs the previously created SSH key, and downloads the sample GoLang webapp. Use the comments in setup.sh to verify these steps before building the image.

#!/bin/bash
set -e

# Install necessary dependencies
sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
sudo apt-get -y -qq install curl wget git vim apt-transport-https ca-certificates
sudo add-apt-repository ppa:longsleep/golang-backports -y
sudo apt -y -qq install golang-go

# Setup sudo to allow no-password sudo for "hashicorp" group and adding "terraform" user
sudo groupadd -r hashicorp
sudo useradd -m -s /bin/bash terraform
sudo usermod -a -G hashicorp terraform
sudo cp /etc/sudoers /etc/sudoers.orig
echo "terraform  ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/terraform

# Installing SSH key
sudo mkdir -p /home/terraform/.ssh
sudo chmod 700 /home/terraform/.ssh
sudo cp /tmp/tf-packer.pub /home/terraform/.ssh/authorized_keys
sudo chmod 600 /home/terraform/.ssh/authorized_keys
sudo chown -R terraform /home/terraform/.ssh
sudo usermod --shell /bin/bash terraform

# Create GOPATH for Terraform user & download the webapp from github

sudo -H -i -u terraform -- env bash << EOF
whoami
echo ~terraform

cd /home/terraform

export GOROOT=/usr/lib/go
export GOPATH=/home/terraform/go
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
go get -d github.com/hashicorp/learn-go-webapp-demo
EOF

»Review the Packer image

The Packer configuration language is based on JSON. Some properties of Packer can be written in HCL2, but this feature is in beta. Your Packer configuration defines the parameters of the image you want to build.

Change directories into images.

$ cd ../images

Open the image.json file in your file editor.

First, review the variables block. This determines the environment variables Packer will use for your AWS account. This region must match the region where Terraform will deploy your infrastructure, so if you customize it you will need to customize the sample Terraform configuration to match (later on in the tutorial).

{
  "variables": {
      "aws_access_key": "{{env `AWS_ACCESS_KEY`}}",
      "aws_secret_key": "{{env `AWS_SECRET_KEY`}}",
      "aws_region": "us-east-1"
  },
##...

The builders block creates an AMI named learn-packer {{timestamp}} that's based on a t2.micro Ubuntu image with Elastic Block Storage (EBS).

##...
  "builders": [
      {
          "type": "amazon-ebs",
          "access_key": "{{user `aws_access_key`}}",
          "secret_key": "{{user `aws_secret_key`}}",
          "region": "{{user `aws_region`}}",
          "source_ami_filter": {
              "filters": {
                  "virtualization-type": "hvm",
                  "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
                  "root-device-type": "ebs"
              },
              "owners": [
                  "099720109477"
              ],
              "most_recent": true
          },
          "instance_type": "t2.micro",
          "ssh_username": "ubuntu",
          "ami_name": "learn-packer {{timestamp}}"
      }
  ],
##...

Finally, the provisioners block builds out your instances with specific scripts or files.

  1. The first provisioner is a file type provisioner and copies your newly created public SSH key to a temporary directory on the image.
  2. The second provisioner is a shell type which points to the relative path to your bash script. Packer uses these provisioners to customize the image to your specifications.
##...
  "provisioners": [
    {
        "type": "file",
        "source": "../tf-packer.pub",
        "destination": "/tmp/tf-packer.pub"
      },
    {
        "type": "shell",
        "script": "../scripts/setup.sh"
    }
]
}

»Build your Packer image

Run the Packer build command providing your image template file.

$ packer build image.json

==> amazon-ebs: Prevalidating any provided VPC information
==> amazon-ebs: Prevalidating AMI Name: learn-packer 1594419525
    amazon-ebs: Found Image ID: ami-0fcf65d66fe3e1f92
==> amazon-ebs: Creating temporary keypair: packer_5f08e945-d87e-c0e0-9c06-127cb40d51f4
==> amazon-ebs: Creating temporary security group for this instance: packer_5f08e947-d29d-8982-903a-f1532932d397
==> amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
    amazon-ebs: Adding tag: "Name": "Packer Builder"
    amazon-ebs: Instance ID: i-0208315d85b166c4b
==> amazon-ebs: Waiting for instance (i-0208315d85b166c4b) to become ready...
==> amazon-ebs: Using ssh communicator to connect: 54.81.101.149
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Uploading ../tf-packer.pub => /tmp/tf-packer.pub
newkey.pub 574 B / 574 B [===============================================] 100.00% 0s
==> amazon-ebs: Provisioning with shell script: ../scripts/setup.sh
    amazon-ebs: Reading package lists...
    amazon-ebs: Building dependency tree...
    amazon-ebs: Reading state information...
==> amazon-ebs: Stopping the source instance...
    amazon-ebs: Stopping instance
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating AMI learn-packer 1594419525 from instance i-0208315d85b166c4b
    amazon-ebs: AMI: ami-06cb92a18a30fec16
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-06cb92a18a30fec16

The final line of the output is the AMI ID you will pass into your Terraform configuration in the next step.

»Deploy your Packer image with Terraform

The AMI is your artifact from the Packer run and is available in your AWS account in the EC2 Images section. You can visit the AWS Web Console to view this AMI ID again.

To use this AMI in your Terraform environment, navigate to the instances directory.

$ cd ../instances

Open the main.tf file and navigate to the aws_instance resource. Edit the ami attribute with the AMI ID you received from your Packer build.

##...
resource "aws_instance" "web" {
-  ami                         = "ami-YOUR-AMI-ID"
+  ami                         = "ami-06cb92a18a30fec16"
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.subnet_public.id
  vpc_security_group_ids      = [aws_security_group.sg_22_80.id]
  associate_public_ip_address = true

  tags = {
    Name = "Learn-Packer"
  }
}
##...

Save your configuration.

Create a new file called terraform.tfvars and add the region where the Packer image is stored as the variable definition. If you customized the region you gave to Packer you must change this region to match, or Terraform won't be able to access your image.

region = "us-east-1"

Save this file and then initialize and apply your configuration.

$ terraform init && terraform apply

Type yes when you are prompted to to create your instance. Your final output is your instance IP address. In the next section, you will SSH into this instance with your local key.

Your instance in this tutorial already contains preferred SSH key, because it uses the AMI you previously packaged with Packer. Using a Packer-packaged AMI makes deploying mass instances faster and more consistent than configuring the instances manually.

»Verify your instance

Connect to your instance via SSH by piping the aws_instance.web.public_ip resource attribute to the terraform console command.

$ ssh terraform@$(echo "aws_instance.web.public_ip" | terraform console) -i ../tf-packer

Now you have SSH access to your AWS instances without creating an SSH key in AWS. This is useful if your organization maintains keypairs outside of AWS.

Navigate to the Go directory.

$ cd go/src/github.com/hashicorp/learn-go-webapp-demo

Launch the demo webapp.

$ go run webapp.go

In your web browser, navigate to the IP address of your instance and port 8080 to see the app you deployed.

»Destroy your image

Avoid unnecessary charges in your AWS account by destroying your instance in Terraform.

$ terraform destroy

Type yes when you are prompted in your terminal to delete your infrastructure.

This will not destroy your Packer image. Your Packer image will not incur costs in your AWS account. Most base Linux distributions have free image versions, but be sure to check on the cost of deploying and maintaining your images.

»Next Steps

In this tutorial, you created a Packer image with your desired configuration and deployed it using Terraform.