The Inside Story: Terraform Module Handling

Francesc, a backend developer here at Cycloid and talented maintainer of TerraCognita, tells us what led to our recent launch of Terraform Module handling and what exciting developments it's paving the way for.

What did we add?

In the newest version of Terracognita (0.6.1) we added the ability to import a running infrastructure into a Terraform Module. This is a more manageable and standard way to structure your HCL.

Terraform Modules allow you to create a reusable and customizable definition of the infrastructure which will help you better define your standard infrastructure components. Also, the use of Terraform Modules is considered good practice and we always try to follow best practices.

How does it work?

Before telling you how it works, let's go over how it used to work so we can see the difference.

Before, we generated one HCL file with all the information imported from your infrastructure. To be honest, big configuration files are not manageable at all, and they're certainly not reusable, as what you see is what you get. To change it, you would need to dive into the generated file (potentially +500 lines of code) and change things manually or try to generate a module from it yourself. Honestly, that's not anyone's idea of fun and it can lead to errors in your IaC creation process. This is pretty silly when you bear in mind that the reason you'd use TerraCognita in the first place is to avoid mistakes in your IaC creation!

As we can see, this is not the best use case. Sure, it's usable and it gets you started in Terraform and HCL quickly - you would have the infrastructure in HCL and wouldn't need to write it from scratch. That said, it would require a lot of manual work to have something more refined and actually reusable.

Now, with the new version, we can do that work for you! The documentation explains how to do it but we'll go through it, the benefits it provides, and how is it done.

First of all, use:

 --module module/path/name flag 

...to enable the feature and we'll import everything as a module! Once it finishes, if you check your module/path/name/, you'll have a Terraform Module inside! The structure is the following:

• module.tf: is where the Terraform Module is declared and where it is configured
• module-name/: is where the module configuration resides (name being the directory you specified)
• module-name/variables.tf: is where all the variables that we define (so the module can be configured) are placed (more about that later).

name/

├── module-name

│   ├── autoscaling.tf

│   ├── cloud_front.tf

│   ├── cloud_watch.tf

│   ├── ec2.tf

│   ├── elastic_load_balancing_v2_alb_nlb.tf

│   ├── iam.tf

│   ├── rds.tf

│   ├── route53_resolver.tf

│   ├── route53.tf

│   ├── s3.tf

│   ├── ses.tf

│   └── variables.tf

└── module.tf

So, the first benefit is that we no longer have to import everything in one single HCL file. Instead, we do it in multiple separate ones based on the "category" of the resources. We can get this information from another OSS library we have named tfdocs, which, in a nutshell, parses the documentation from the Terraform Providers and provides a normalized output so we can check it in different places in the Cycloid codebase, like InfraView and StackCraft.

Now, back to TerraCognita!

This generate module is a bit raw - it's the default and most basic configuration that we provide and what it'll do is replace all the attribute values from your HCL for variables with the value that they had as a default of the variable and declare those variables into a variables.tf which will be inside of your module-name/variables.tf.

Then if you check the module.tf you'll see it has all the variables added to the Module block commented with their values, so you know which value they have by default. If you wanted to change them, you could uncomment and make the modification you need.  

module "name" {

  # aws_instance_front_instance_type = "t2.small"

  [...]

  source = "module-name"

}

The second benefit we see is that modules make modifying your configuration much easier, but wait, there's more...

We can make that module even better by adding --module-varibles path/to/file flag which will allow you to define a JSON/YAML file with all the resource + attributes that would be important for you to use in the module so only those specific ones (instead of ALL of them) would be defined as variables and added to the Module block

{

  "aws_instance": [

    "instance_type",

    "cpu_threads_per_core",

    "cpu_core_count"

  ]

}

This file format is basically a Key Values in which the Key is the name of the Terraform Resource and the Values are the list of all the attributes of that resource that you would like to import.

Conclusion

Using TerraCognita's Modules feature will help you better import your infrastructure into Terraform Modules. You'll be able to import in a more customizable way that will improve the reusability and configurability of your infrastructure. This provides an easy way to adopt Terraform best practice.

Stay up to date with everything coming from the Cycloid stable - sign up for the quarterly  newsletter.