Virtual Event
Join us for the next HashiConf Digital October 12-15, 2020 Register for Free

Call APIs with Terraform Providers

Implement Complex Read

In this tutorial, you will retrieve an order, a protected endpoint, to verify whether a Terraform provider that interacts with the API of a fictional coffee-shop application, HashiCups, has been authenticated to the API successfully. The order schema contains nested objects, making it a more complicated data structure than the previously defined coffee data source. To do this, you will:

  1. Create a new order via API
    There must be an existing order for you to retrieve it.
  2. Define order data source.
    You will add a scaffold that defines an empty schema and functions to retrieve order information.
  3. Define order schema.
    The schema reflects the response from invoking the /order/{orderId} endpoint. The order schema is considered complex because it nests a list of OrderItems.
  4. Implement complex read.
    This read functionality uses multiple nesting functions to flatten the response from /order/{orderId} and map it to the order schema.
  5. Add order data source to the provider schema.
    This allows you to use the data source in your configuration.

»Prerequisites

To follow this tutorial, you need:

Navigate to your terraform-provider-hashicups directory. Then, checkout the auth-configuration branch. This step is optional but recommended to insure that you've accurately completed the previous steps.

$ git checkout auth-configuration

Your directory should have the following structure.

$ tree -L 2
.
├── Makefile
├── README.md
├── docker_compose
│   ├── conf.json
│   └── docker-compose.yml
├── examples
│   ├── coffee
│   ├── main.tf
├── go.mod
├── go.sum
├── hashicups
│   ├── data_source_coffee.go
│   └── provider.go
├── main.go
└── vendor

If you’re stuck, refer to the implement-complex-read branch to see the changes implemented in this tutorial.

»Create an order via API

Before you can query a user's order, you must first create an order. Replace the authorization token with the token you generated in the sign in step.

$ curl -X POST -H "Authorization: ${HASHICUPS_TOKEN}" localhost:19090/orders -d '[{"coffee": { "id":1 }, "quantity":4}, {"coffee": { "id":3 }, "quantity":3}]'
{
  "id": 1,
  "items": [
    {
      "coffee": {
        "id": 1,
        "name": "Packer Spiced Latte",
        "teaser": "Packed with goodness to spice up your images",
        "description": "",
        "price": 350,
        "image": "/packer.png",
        "ingredients": null
      },
      "quantity": 4
    },
    {
      "coffee": {
        "id": 3,
        "name": "Nomadicano",
        "teaser": "Drink one today and you will want to schedule another",
        "description": "",
        "price": 150,
        "image": "/nomad.png",
        "ingredients": null
      },
      "quantity": 3
    }
  ]
}

Query the order using the order ID you received in the response above as a HTML parameter (localhost:19090/orders/{orderID}). The response should be the same as the response above.

$ curl -X GET -H "Authorization: ${HASHICUPS_TOKEN}" localhost:19090/orders/1

»Define order data resource

Now, create a file named data_source_order.go in your hashicups directory and add the following snippet.

package hashicups

import (
  "context"
  "strconv"
  
  hc "github.com/hashicorp-demoapp/hashicups-client-go"
  "github.com/hashicorp/terraform-plugin-sdk/v2/diag"
  "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

func dataSourceOrder() *schema.Resource {
  return &schema.Resource{
    ReadContext: dataSourceOrderRead,
    Schema: map[string]*schema.Schema{
      "id": &schema.Schema{
        Type:     schema.TypeInt,
        Required: true,
      },
      "items": &schema.Schema{
        Type:     schema.TypeList,
        Computed: true,
        Elem: &schema.Resource{
          Schema: map[string]*schema.Schema{
            "coffee_id": &schema.Schema{
              Type:     schema.TypeInt,
              Computed: true,
            },
            "coffee_name": &schema.Schema{
              Type:     schema.TypeString,
              Computed: true,
            },
            "coffee_teaser": &schema.Schema{
              Type:     schema.TypeString,
              Computed: true,
            },
            "coffee_description": &schema.Schema{
              Type:     schema.TypeString,
              Computed: true,
            },
            "coffee_price": &schema.Schema{
              Type:     schema.TypeInt,
              Computed: true,
            },
            "coffee_image": &schema.Schema{
              Type:     schema.TypeString,
              Computed: true,
            },
            "quantity": &schema.Schema{
              Type:     schema.TypeInt,
              Computed: true,
            },
          },
        },
      },
    },
  }
}

Notice how the schema mirrors the JSON response — only id and items exist on the top level.

  • The id is required because an order ID must be set so the data source knows which order to retrieve.

  • The items schema is a computed list (schema.TypeList) of objects (schema.Resource) containing coffee attributes and quantity.

    The coffee attributes is flattened and mapped accordingly (coffee.id is mapped to coffeee_id, etc...)

You will learn an alternative method to nest maps in the Implement Create tutorial, when you create new orders.

»Implement complex read

Now that you have defined the order schema, add the dataSourceOrderRead function to data_source_order.go. This function will retrieve the order and map its values to the order schema defined above.

func dataSourceOrderRead(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
  c := m.(*hc.Client)
  
  // Warning or errors can be collected in a slice type
  var diags diag.Diagnostics
  
  orderID := strconv.Itoa(d.Get("id").(int))
  
  order, err := c.GetOrder(orderID)
  if err != nil {
    return diag.FromErr(err)
  }
  
  orderItems := flattenOrderItemsData(&order.Items)
  if err := d.Set("items", orderItems); err != nil {
    return diag.FromErr(err)
  }
  
  d.SetId(orderID)
  
  return diags
}

Notice the m (meta) input parameter contains the HashiCups API Client set by the ConfigureContextFunc defined above. If provider uses an unauthenticated API Client, this function will fail and return an error message.

The API Client's GetOrder function returns an order object. However, you must flatten this response to accurately map the response to the order schema. An order consists of an order ID and a list of coffee objects and their respective quantities. As a result, the order object must go through one flattening function to populate the list of coffee objects and their quantities.

Add the flattenOrderItemsData function to your data_source_order.go file. This returns a list of order items.

func flattenOrderItemsData(orderItems *[]hc.OrderItem) []interface{} {
  if orderItems != nil {
    ois := make([]interface{}, len(*orderItems), len(*orderItems))
    
    for i, orderItem := range *orderItems {
      oi := make(map[string]interface{})
      
      oi["coffee_id"] = orderItem.Coffee.ID
      oi["coffee_name"] = orderItem.Coffee.Name
      oi["coffee_teaser"] = orderItem.Coffee.Teaser
      oi["coffee_description"] = orderItem.Coffee.Description
      oi["coffee_price"] = orderItem.Coffee.Price
      oi["coffee_image"] = orderItem.Coffee.Image
      oi["quantity"] = orderItem.Quantity
      
      ois[i] = oi
    }
    
    return ois
  }
  
  return make([]interface{}, 0)
}

The flattenOrderItemsData function takes an *[]hc.OrderItem as orderItems. If orderItems is not nil, it will iterate through the slice and map its values into a map[string]interface{}. Notice how the function assigns the coffee attributes directly to its corresponding flattened attribute (orderItem.Coffee.ID -> coffee_id).

»Add data source to provider

Now that you’ve defined the order data source, you can add it to your provider.

In your provider.go file, add the order data source to the DataSourcesMap of your Provider() function.

// Provider -
func Provider() *schema.Provider {
  return &schema.Provider{
    Schema: map[string]*schema.Schema{
      "username": &schema.Schema{
        Type:        schema.TypeString,
        Optional:    true,
        DefaultFunc: schema.EnvDefaultFunc("HASHICUPS_USERNAME", nil),
      },
      "password": &schema.Schema{
        Type:        schema.TypeString,
        Optional:    true,
        Sensitive:   true,
        DefaultFunc: schema.EnvDefaultFunc("HASHICUPS_PASSWORD", nil),
      },
    },
    ResourcesMap: map[string]*schema.Resource{},
    DataSourcesMap: map[string]*schema.Resource{
        "hashicups_coffees":     dataSourceCoffees(),
+       "hashicups_order":       dataSourceOrder(),
    },
    ConfigureFunc: providerConfigure,
  }
}

»Test the provider

Now that you’ve implemented read and created the coffees data source, verify that it works.

First, navigate to the terraform-provider-hashicups root directory.

Then, build the binary and move it into your user Terraform plugins directory. This allows you to sideload and test your custom providers.

$ make install
go build -o terraform-provider-hashicups
mv terraform-provider-hashicups ~/.terraform.d/plugins/hashicorp.com/edu/hashicups

Navigate to the examples directory. This contains a sample Terraform configuration for the Terraform HashiCups provider.

$ cd examples

Add the following Terraform configuration to main.tf.

data "hashicups_order" "order" {
  id = 1
}

output "order" {
  value = data.hashicups_order.order
}

Finally, initialize your workspace to refresh your HashiCups provider, then apply. This should return the first order in your output.

Notice how the coffee attributes are flattened.

$ terraform init && terraform apply --auto-approve
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

order = {
  "id" = 1
  "items" = [
    {
      "coffee_description" = ""
      "coffee_id" = 1
      "coffee_image" = "/packer.png"
      "coffee_name" = "Packer Spiced Latte"
      "coffee_price" = 350
      "coffee_teaser" = "Packed with goodness to spice up your images"
      "quantity" = 4
    },
    {
      "coffee_description" = ""
      "coffee_id" = 3
      "coffee_image" = "/nomad.png"
      "coffee_name" = "Nomadicano"
      "coffee_price" = 150
      "coffee_teaser" = "Drink one today and you will want to schedule another"
      "quantity" = 3
    },
  ]
}

»Next steps

Congratulations! You have implemented a nested read function. This will be useful when you will create a resource using the HashiCups provider in the Implement Create tutorial.

If you were stuck during this tutorial, checkout the implement-complex-read branch to see the changes implemented in this tutorial.