Creating a CI/CD pipeline with a git repository in ~30 seconds using Terraform and AWS

May 15, 2018 0 Comments

Creating a CI/CD pipeline with a git repository in ~30 seconds using Terraform and AWS



Terraform is a powerful tool for provisioning and managing cloud infrastructure. I recently wrote a blog post that provisions a CI/CD pipeline integrated with GitHub repositories to continuously deploy static websites. After going through the exercise of creating that template and blog post I realized I could abstract all of this.

So I created a new Terraform module that provisions an entire CI/CD pipeline and AWS CodeCommit Git repository in roughly 30 seconds.

To leverage the module we can create a new template called and add the following to it.

module "codecommit-cicd" { source = "git::" reponame = "learn-by-doing-newsletter" # Required organizationname = "kylegalbraith" # Required repodefaultbranch = "master" # Default value awsregion = "us-west-2" # Default value chardelimiter = "-" # Default value environment = "dev" # Default value buildtimeout = "5" # Default value buildcomputetype = "BUILDGENERAL1SMALL" # Default value buildimage = "aws/codebuild/nodejs:6.3.1" # Default value testbuildspec = "buildspectest.yml" # Default value packagebuildspec = "buildspec.yml" # Default value forceartifactdestroy = "false" # Default value 
} output "repo
url" { dependson = ["module.codecommit-cicd"] value = "${module.codecommit-cicd.clonerepohttps}" }

There are quite a few variables that can be passed into the module. Here is a brief explanation of what each one does:

  • repo
    name: The name of our new Git repository.

  • organizationname: The organization name provisioning the template.

  • repo
    defaultbranch: The default branch of our Git repository.

  • aws
    region: The AWS region we want to provision our infrastructure in.

  • chardelimiter: The delimiter to use for unique names.

  • environment: What our development team calls this environment (i.e. dev v.s. prod).

  • build
    timeout: How long CodeBuild will wait for a build to complete in minutes.

  • buildcomputetype: The instance type CodeBuild will use to run our builds on.

  • buildimage: The base AMI our build instances will use to run our builds on.

  • test
    buildspec: The path to the test runner buildspec.yml file.

  • packagebuildspec: The path to the package & deploy buildspec.yml file.

  • force
    artifactdestroy: Specify whether or not we want to forcibly delete our build artifacts when this template is destroyed.

To provision the resources in our AWS account we run the following commands from our command line.

$>terraform init 
.....this will initialize terraform template..... $>terraform plan
.....this will show you what the module is going to provision..... $>terraform apply
.....this will create all of the resources.....

Once the apply command is complete we will have the following resources created in our Amazon Web Services account.

  • AWS CodeCommit git repository for source code.

  • AWS CodePipeline with a build and test stage.

  • AWS CodeBuild projects, one that is looking for a buildspectest.yml file and one for the build process that is looking for a buildspec.yml file.

An important thing to note here is that our new repository is not yet initialized. We need to clone the repository using the URL from our template output, add a file, and then push.

$>git clone 
$>cd learn-by-doing-newsletter
$>echo 'hello world' > touch.txt
$>git commit -a -m 'init master'
$>git push -u origin master

With this module and about 30 seconds on our command line, we have created a new git repository and provisioned a CI/CD pipeline all in AWS.

It might not seem like much on the surface but infrastructure as code like this is a huge time saver. Take a look at this accompanying blog post that has manual steps for creating our CI/CD pipeline. There are 21 different steps to create the pipeline. Here in our template we create the pipeline and a new repository in 20 lines of code.

This comes in very handy when we need to provision new micro-services frequently. We could turn this into a template that other teams use to quickly stand up their own CI/CD pipelines and repositories. Terraform is a great tool for provisioning our infrastructure and there is more power in encapsulating common infrastructure into its own module.

If you are looking to start treating your infrastructure as code, start thinking about using Terraform. Give this module a try, and let us know if there is more you would like to see us add to it!

Learn AWS by actually using it

If you enjoyed this post and are hungry to start learning more about Amazon Web Services, I have created a new course on how to host, secure, and deliver static websites on AWS! It is a book and video course that cuts through the sea of information to accelerate your learning of AWS. Giving you a framework that enables you to learn complex things by actually using them.

Head on over to the landing page to get a copy for yourself!

Tag cloud