Terraform: Introduction to for_each

Using for_each with resources in Terraform

 Posted on November 28, 2020  |  Carlos Robles

In the course of using Terraform, you will undoubtedly encounter the need to build more than one of something. Be it an instance group, cloud function, or VPC, Terraform has a couple of options for specifying multiple instances of a resource: count and for_each.

This post will discuss the use of for_each.

For information on count, see this post.

Note: count and for_each are only available when using Terraform 0.12 or higher.
 

Creating Multiple Resources

To use for_each with a resource, simply add the for_each argument. Because the identifier for each resource created with for_each must be unique, we can only use for_each with a data type that does not allow duplicates:

  • map - Each key in map is unique and has not duplicate
  • set - Each element in a set is unique and cannot have duplicates

All Terraform resources support the for_each argument. For demo purposes, we will use the null_resource resource as this doesn’t create anything - very useful for testing and debugging!
 

Using Maps

We can create multiple resources with for_each and a map. A map is most commonly used when one wants to store more than one piece of information for each resource.
 

# Create a map that represents primary and secondary resources
locals {
    my_resources = {
        primary = "us-east-1"
        secondary = "us-west-2"
    }
}

# Create a single "null_resource" resource
resource "null_resource" "my_single_resource" {}

# Create two "null_resource" resources using a map
resource "null_resource" "my_multi_resources" {
    for_each = local.my_resources
}

# Output the map that makes up the single "my_single_resource" resource
output "my_single_resource" {
    value = null_resource.my_single_resource
}

# Output the map of maps that makes up the "my_multi_resources" resource
output "my_multi_resource_map" {
    value = null_resource.my_multi_resources
}

# terraform apply -auto-approve
null_resource.my_single_resource: Creating...
null_resource.my_multi_resources["secondary"]: Creating...
null_resource.my_multi_resources["primary"]: Creating...
null_resource.my_multi_resources["primary"]: Creation complete after 0s [id=7456215554446454609]
null_resource.my_single_resource: Creation complete after 0s [id=7291314958864433749]
null_resource.my_multi_resources["secondary"]: Creation complete after 0s [id=2946853653739811989]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

my_multi_resource_map = {
  "primary" = {
    "id" = "7456215554446454609"
  }
  "secondary" = {
    "id" = "2946853653739811989"
  }
}
my_single_resource = {
  "id" = "7291314958864433749"
}

 

Using Sets

We can create multiple resources with for_each and a set. A set is most commonly used when one only needs to store one piece of information for each resource (i.e. the key name)
 

locals {
    # Create a list that represents primary and secondary resources
    # We will convert it into a set below
    my_resources = [
        "primary",
        "secondary"
    ]
}

# Create a single "null_resource" resource
resource "null_resource" "my_single_resource" {}

# Create two "null_resource" resources using a list that we turn into a set
resource "null_resource" "my_multi_resources" {
    for_each = toset(local.my_resources)
}

# Output the map that makes up the single "my_single_resource" resource
output "my_single_resource" {
    value = null_resource.my_single_resource
}

# Output the map of maps that makes up the "my_multi_resources" resource
output "my_multi_resource_map" {
    value = null_resource.my_multi_resources
}

# terraform apply -auto-approve
null_resource.my_single_resource: Creating...
null_resource.my_multi_resources["secondary"]: Creating...
null_resource.my_multi_resources["primary"]: Creating...
null_resource.my_single_resource: Creation complete after 0s [id=1585597877195778047]
null_resource.my_multi_resources["primary"]: Creation complete after 0s [id=3851632300015385142]
null_resource.my_multi_resources["secondary"]: Creation complete after 0s [id=429123151351701121]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

my_multi_resource_map = {
  "primary" = {
    "id" = "3851632300015385142"
  }
  "secondary" = {
    "id" = "429123151351701121"
  }
}
my_single_resource = {
  "id" = "1585597877195778047"
}

 
In these examples, because the keys in the map are the same as the elements in the list (which is then turned into a set), the resources have the same name designations regardless of whether a map or set is used.

A single resource is represented with a map; adding for_each creates a map of maps. Each map contains the exported attributes for a single instance of the resource. In this case, only one attribute is exported: id. We can retrieve the null_resource map of maps by specifying the name of the created resource.
 

Static Resource Lookup

If you want to retrieve the map of maps that represents the resource, you can simply refer to the resource name as shown above. If you want to retrieve a specific resource in the map, you can access it by using its key.
 

locals {
    my_resources = [
        "primary",
        "secondary"
    ]
}

resource "null_resource" "my_resource" {
    for_each = toset(local.my_resources)
}

# Output the map of maps that make up the "my_resource" resource
output "my_resource" {
    value = null_resource.my_resource
}

# Output the "primary" resource map in the map
output "my_primary_resource" {
    value = null_resource.my_resource["primary"]
}

# Output the "secondary" resource map in the map
output "my_secondary_resource" {
    value = null_resource.my_resource["secondary"]
}

# Output the id attribute of the first resource
output "my_primary_resource_id" {
    value = null_resource.my_resource["primary"].id
}

# Output the id attribute of the second resource
output "my_secondary_resource_id" {
    value = null_resource.my_resource["secondary"].id
}

# terraform apply -auto-approve
null_resource.my_resource["secondary"]: Creating...
null_resource.my_resource["primary"]: Creating...
null_resource.my_resource["secondary"]: Creation complete after 0s [id=262194844307873449]
null_resource.my_resource["primary"]: Creation complete after 0s [id=7985352624881011885]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

my_primary_resource = {
  "id" = "7985352624881011885"
}
my_primary_resource_id = 7985352624881011885
my_resource = {
  "primary" = {
    "id" = "7985352624881011885"
  }
  "secondary" = {
    "id" = "262194844307873449"
  }
}
my_secondary_resource = {
  "id" = "262194844307873449"
}
my_secondary_resource_id = 262194844307873449

 

Dynamic Resource Lookup

The above example used a static lookup for each item by specifying a key:
 

null_resource.my_resource["primary"]
null_resource.my_resource["primary"].id

 
When iterating through multiple resources with for_each, the current key can be retrieved with each.key and the key value can be retrieved with each.value. The for expression can be used to loop through resources and generate new maps or lists from retrieved items.
 

locals {
  my_resources = {
      primary   = "us-east-1",
      secondary = "us-west-2",
      tertiary  = "us-east-2"
  }
}

# We create as many resources as there are keys in my_map
# and add a trigger that is named after each key in the map
resource "null_resource" "my_resource" {
    for_each = local.my_resources

    triggers = {
        name = "${each.key}: ${each.value}"
    }
}

# Output a list of the maps that make up the "my_resource" resource
output "my_resource_list" {
    value = [
        for resource in null_resource.my_resource:
      resource
    ]
}

# Output a map of the maps that make up the "my_resource" resource
# by using the unique id as the key
output "my_resource_map" {
    value = {
        for resource in null_resource.my_resource:
            resource.id => resource
    }
}

# Output a list of strings of all the "my_resource" ids
output "my_resource_ids" {
    value = [
        for resource in null_resource.my_resource:
            resource.id
    ]
}

# terraform apply -auto-approve
null_resource.my_resource["secondary"]: Creating...
null_resource.my_resource["tertiary"]: Creating...
null_resource.my_resource["primary"]: Creating...
null_resource.my_resource["tertiary"]: Creation complete after 0s [id=8781964496827544111]
null_resource.my_resource["secondary"]: Creation complete after 0s [id=900182135080969641]
null_resource.my_resource["primary"]: Creation complete after 0s [id=5013816299053157416]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

my_resource_ids = [
  "5013816299053157416",
  "900182135080969641",
  "8781964496827544111",
]
my_resource_list = [
  {
    "id" = "5013816299053157416"
    "triggers" = {
      "name" = "primary: us-east-1"
    }
  },
  {
    "id" = "900182135080969641"
    "triggers" = {
      "name" = "secondary: us-west-2"
    }
  },
  {
    "id" = "8781964496827544111"
    "triggers" = {
      "name" = "tertiary: us-east-2"
    }
  },
]
my_resource_map = {
  "5013816299053157416" = {
    "id" = "5013816299053157416"
    "triggers" = {
      "name" = "primary: us-east-1"
    }
  }
  "8781964496827544111" = {
    "id" = "8781964496827544111"
    "triggers" = {
      "name" = "tertiary: us-east-2"
    }
  }
  "900182135080969641" = {
    "id" = "900182135080969641"
    "triggers" = {
      "name" = "secondary: us-west-2"
    }
  }
}

 

Considerations

Because for_each creates a map of resources, the usual considerations associated with maps apply:

  • Each key must be unique (no duplicates)
  • Each map value can be looked up by its key

Thus, maps are a good way to identify unique resources. Also, because for_each uses keys and not indices like count, removing a key and its value from a map will not affect any resource instances created by other map keys.
 

# Initial map
locals {
  my_resources = {
      primary   = "us-east-1",
      secondary = "us-west-2",
      tertiary  = "us-east-2"
  }
}

resource "null_resource" "my_resource" {
    for_each = local.my_resources

    triggers = {
        name = "${each.key}: ${each.value}"
    }
}

# Output the map of maps that make up the "my_resource" resource
output "my_resource" {
    value = null_resource.my_resource
}

# terraform apply -auto-approve
null_resource.my_resource["tertiary"]: Creating...
null_resource.my_resource["primary"]: Creating...
null_resource.my_resource["secondary"]: Creating...
null_resource.my_resource["tertiary"]: Creation complete after 0s [id=574024423946004547]
null_resource.my_resource["primary"]: Creation complete after 0s [id=7920307120447949203]
null_resource.my_resource["secondary"]: Creation complete after 0s [id=6388992819329330829]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

my_resource = {
  "primary" = {
    "id" = "7920307120447949203"
    "triggers" = {
      "name" = "primary: us-east-1"
    }
  }
  "secondary" = {
    "id" = "6388992819329330829"
    "triggers" = {
      "name" = "secondary: us-west-2"
    }
  }
  "tertiary" = {
    "id" = "574024423946004547"
    "triggers" = {
      "name" = "tertiary: us-east-2"
    }
  }
}

 
If we then remove the second key and value from the list, none of the other resources are affected, which is good!
 

# Updated map
locals {
  my_resources = {
      primary   = "us-east-1",
      tertiary  = "us-east-2"
  }
}

resource "null_resource" "my_resource" {
    for_each = local.my_resources

    triggers = {
        name = "${each.key}: ${each.value}"
    }
}

# Output the map of maps that make up the "my_resource" resource
output "my_resource" {
    value = null_resource.my_resource
}

# terraform apply -auto-approve
null_resource.my_resource["primary"]: Refreshing state... [id=7920307120447949203]
null_resource.my_resource["tertiary"]: Refreshing state... [id=574024423946004547]
null_resource.my_resource["secondary"]: Refreshing state... [id=6388992819329330829]
null_resource.my_resource["secondary"]: Destroying... [id=6388992819329330829]
null_resource.my_resource["secondary"]: Destruction complete after 0s

Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

Outputs:

my_resource = {
  "primary" = {
    "id" = "7920307120447949203"
    "triggers" = {
      "name" = "primary: us-east-1"
    }
  }
  "tertiary" = {
    "id" = "574024423946004547"
    "triggers" = {
      "name" = "tertiary: us-east-2"
    }
  }
}