This week I attended the DevOps Glasgow meetup entitled “Episode 5. Hashicorp”, here we were introduced to Terraform, Hashicorp’s tool for implementing Infrastructure as Code (IaC), by Hashicorp engineer Radek Simko. Here is the link to Radek’s slide deck.
This was my first time seeing Terraform put to work. I’ve been a fan of an other Hashicorp tools such as Vagrant for building local test environments with Virtualbox, Consul for service discovery and Vault for secrets management.
In this post I’ll give a brief overview of what Infrastructure As Code (IaC) is and touch on how Terraform comes into play when implementing IaC and the problems it can help solve. In the follow up I’ll go over putting Terraform into action to build small environment.
Infrastructure As Code, Automation, do we need it?
Due to the increased adoption of server virtualisation and cloud service technologies the number of servers being used in IT projects is constantly growing, making the traditional ways of managing the IT infrastructure less relevant. Manually building servers maybe fine in a small environment. But as the infrastructure grows doing things in a manual way is always going to be both time consuming and inconsistent.
Having an automated approach to provisioning infrastructure, when implementing IaC, allows us to work in a much more efficient, effective and consistent manner. The modern data centre is becoming more and more “software defined”, this includes networking, storage as well as compute, through the use of API’s.
Today API enabled platforms are driving the change of how we integrate and manage IT infrastructure, and how we are moving quickly from a process driven, reactive approach to a more automated driven, proactive approach.
Through automation, we start to remove human error and unnecessary operational processes. Even with a well documented procedure, a repeatable manual task when carried out by human workers is likely to be completed without the same consistency. However, if that same task were to be automated, it would be executed the same way every time, improving reliability and operational efficiency.
As we start to codifying our infrastructure we can start to apply SDLC principles. By turning our infrastructure into code we now have the following benefits:-
- Can be stored in a version control system, such as git
- This in turns provides a change history
- Easier collaboration through sharing and reuse
- Improves documentation (removes tribal knowledge)
- Allows for more efficient and predictable deployments
- Enables Test-Driven deployments
- Reduces the mean time to recovery with issues.
A benefit of using a tool like Terraform is that it allows Ops to focus on providing quality and value to other teams while managing risk through standardising resources. Terraform enables ops to configure those standardised resources, such as creating standard app stacks that can then be provided to consumers as “self-service infrastructure”.
Terraform
Terraform is an open source product from HashiCorp. Like I mentioned before, this is the same company that brought us other DevOps tools such Vagrant, Packer, Consul, and Vault. Written in Go, Terraform was first released in July 2014 and in recent months has increased in popularity with the DevOps community.
Here is a quote from the Terraform site:
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.
Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied.
The infrastructure Terraform can manage includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.
Terraform providers understand and interact with the API’s to expose available resources to configure and update our infrastructure. You can think of Providers as the platforms, or service provider that we wish to provision our infrastructure on. Providers range from large cloud service providers like AWS, Google Cloud, Azure and DigitalOcean, as well to on premise platforms like VMware vSphere, Docker and smaller services and PaaS such as SimpleDNS and Heroku.
Terraform is vendor agnostic. However, it doesn’t completely remove transparency when using it to build infrastructure. Each provider allows different resources to be provisioned through that providers APIs. Such as AWS will have differing resources than say Docker will. It does though, allow for single configurations to contain multiple providers, allowing a hybrid infrastructure to be contained within a single configuration rather than having a bunch of scripts for each platform. Another benefit is the removal of vendor lock in, removing the need to learn and maintain vendor proprietary tools, like AWS CloudFormation or Azure Resource Manager.
In the follow up I give a practical example of using Terraform.