GitHub Actions for Automated CI/CD Deployment with AWS CDK

Developers all have to deal with deployments. You can deal with these manually or you can address this with CI/CD automation. Most developers and organizations start out with manual deployments. However, manual deployments can be a painful process. Those pain points fall into a number of categories. 

The Pains of Manual Deployments

First off, manual deployments typically mean inconsistent deployments. Inconsistently releasing many changes at once, as opposed to continually deploying smaller commits, can make rollbacks a very painful experience. It also introduces the risk of files sometimes being forgotten, or even whole sections of an application falling by the wayside. 

Doing deployments manually also often means less testing. Since testing is then also manual, it becomes more likely that it will not be done uniformly across all features before being pushed to production.

Feature lag between environments is another common pitfall of manual deployments. As production environment configurations fall behind the dev environments, the dev team has to account for an ever-growing list of related issues. This leads to longer and longer development cycles. If deployments are painful, many folks try to draw them out until there are more to be deployed or until things are more complete.

Finally, manual deployments require a certain amount of tribal knowledge. If only a small handful of folks, or a specific individual, know how to successfully deploy in your environments, then employee turnover becomes an even more disruptive event. The smaller the team managing deployments, typically means many small steps won’t be documented, just known by those team members. When a person leaves the company, or if they’re on vacation and somebody else needs to deploy, only then do you discover that information is lost, possibly resulting in failed deployment.

Benefits of Automated CI/CD

On the other end of the deployment spectrum, you have fully automated CI/CD deployments. This approach brings benefits for improving time to production and quality of code. It also brings benefits for each team member and the team as a whole.

For the developer, automated deployments bring single-click dev environment setups. When onboarding new developers or replacing older dev environments, you can spin up new environments with one command, making it much faster to get to work on new features and improvements. 

Automating environment build steps also allows you to automate testing. Each commit that is pushed can trigger the build process and automatically run any defined tests. The logs and output of that process then become something to look at only when something fails, rather than the UI for performing manual testing.

Automated deployments, and the associated automatic build steps, bring more consistency across environments. Keeping the development, testing, staging, and production environments consistent saves a lot of headaches for devs. Consistency gives developers confidence; if code and configuration worked in one environment, it should also work in the production environment. 

Automated CI/CD deployments bring externalized configurations. What is an externalized config? As your push is happening, a system can pull the specs and secrets needed for the specific environment from a secrets manager, or another place outside the codebase. This relieves you, the developer, from needing to worry about configurations being included in your code and can free you up to focus on features, rather than manually juggling deployment details.

This approach results in Infrastructure as Code. Automating a process and needed infrastructure means you are most likely maintaining configuration as a part of your codebase. That codebase is able to be viewed by anybody, including auditors and managers, and makes it much easier to automatically test each setup. It also allows you to easily version control your CI/CD process.

Another benefit of this approach is that it lets you add automated triggers into the process, all the way up to production. Automated triggers can fire, for example, whenever you’re doing code commits, or whenever you’re creating new tags, or whenever you’re creating new versions. Each action can trigger a push to a specific environment or fire off a set of tests. 

State of the CI/CD Industry

86% of organizations say that speed is vital to their application. It’s vital to their business to be fast and to be able to do things in a faster way. However, only 77% of companies are actually using DevOps and automating things. Of that 77% that say they’re using it, only 10% say that they’re successful at it.

So why isn’t everyone doing it and doing it successfully? Of the companies employing DevOps, 49% cite that legacy applications, with their legacy codebases and processes, are the biggest challenge to adopting better DevOps practices. From that same group, 71% say that they’re using containers. That doesn’t mean they’re using only containers, just that they’re using containers as some part of their DevOps strategy.

A few examples of what this looks like in the real world can be seen at CapitalOne, which deploys about 50 times per day. On the other hand, at an extreme scale, companies like Amazon, Google, and Netflix are deploying thousands of times per day. There would be no way that these companies would be able to deploy at that frequency without automating. But how do they go about automating their CI/CD processes?

GitHub Actions for Automating CI/CD

One approach for automating CI/CD is leveraging GitHub Actions, which gives developers the opportunity to automate your workflows, from idea to production. The nice thing is that all the code you need to accomplish GitHub Actions exists within your code repository on GitHub. GitHub Actions is very easy to use and get on the path to automation. It gives you an easy path to Infrastructure as Code as well.

GitHub Actions are project-based, meaning that the configuration of the created workflow is stored in the individual project repo. This reduces the number of files and processes to manage for any given workflow.

There’s also a very handy marketplace within GitHub Actions. If you’re worried about getting started, companies and individual developers have shared their workflows, which provide a quick way to get up and working quickly. Chances are there’s already something there related to what you would like to do.

GitHub Actions help bring consistency across all environments. No matter if it’s Dev, Test, or Production or what dependencies are needed, the automation process can be monitored and adjusted pretty easily. This all also means that performing things that require human intervention, like code reviews, enforcing Git branching strategies, and handling issues, can all be done from within the single platform of GitHub. 

Amazon Elastic Container Service (AWS ECS)

The next thing to consider in your workflow is the production side of things. Amazon’s Elastic Container Service (AWS ECS), gives users a way to be able to manage containers on Amazon and deploy code into them. The nice thing about AWS ECS, like most things at Amazon, is security is “job zero”. They’ve really put a lot of thought into security into ECS and made it easy to do things like adding security rules into your workflow processes.

AWS ECS is fully managed, meaning you don’t have to worry about managing the infrastructure behind the containers. All you need to do is use the containers, letting Amazon handle the back end of it. As a container orchestration service, it saves you a lot of heavy lifting. 

AWS ECS integrates with most other AWS services, like AWS Identity & Access Management and Amazon S3, letting you use all these services with your containers. Going serverless on Amazon most often means leveraging Amazon Fargate. All of these services add up to being able to quickly and easily deploy and scale all manner of applications.

The Amazon Cloud Development Kit (AWS CDK)

At the heart of all automating with AWS ECS is the Amazon Cloud Development Kit (AWS CDK). The CDK is basically the core of Infrastructure as Code, giving you a way of creating code that builds and ramps up your infrastructure. It’s open source and available on GitHub. It lets you build using the programming language of your choice, such as JavaScript, TypeScript, Python, Java, C#, or even GoLang. 

Since Amazon CDK makes calls based on your code, even this part of the workflow process can be tracked using version control. As you add or change things, you’re tracking those changes in your source control. You can also test the code that makes those calls, just as you would test, just as you would test anything else written in your preferred programming language.

There’s also a handy command line interface that comes with AWS CDK. You can address the system directly and run many different commands, such as deploy, destroy and doctor.

Amazon CodeBuild

Amazon CodeBuild is part of the code pipeline and is used to build the package that will be deployed with AWS CDK. CodeBuild compiles source code and gets it ready to go out and deploy somewhere, delivering the artifact it produces to you as an archive for easy storage and transport. CodeBuild can also be set to run your tests.

As with many other Amazon services, CodeBuild is very scalable. It allows you to create multiple builds from within the application. You can create as many builds as you need, and can also have them all running concurrently, which is often needed for bigger projects.

CodeBuild is completely customizable. Every bit of your deployment can be tailored to your exact needs since you can add any custom code needed. 

Tying it All Together with GitHub Actions

So how do we get from pushing our code up to GitHub to building with CodeBuild on AWS? Click here to watch an example using a GitHub repository, social-saver.

In each GitHub repo, you can find a GitHub Actions tab. Opening this tab on any of your repositories will let you just click a button to start setting up your workflow. You will also see that there’s a marketplace right at your fingertips. You do all the needed setup through GitHub.com’s interface. The service creates a file for you in your repository because GitHub Actions are repo specific. 

You can leverage workflow templates that are found in the GitHub Actions marketplace. These pre-made templates can interact with just about any AWS service to deploy to Amazon ECS. Chances are that if you want to do anything using GitHub Actions, there’s probably a beginning template created by a company or an individual out there for you to use. 

Selecting a template from the marketplace will create the workflow file in your code repository inside the .github directory. Inside that folder, you will find a workflows subdirectory where you’ll keep your workflows. Workflows exist as .yml files as part of the version controlled repository.

Inside the workflow’s .yml file, you define the needed elements for the workflow. First is the trigger for the event, starting with something such as on:. The .yml file also defines the environment where the workflow should deploy. After declaring the trigger event and the target, you need to define the action steps. These steps can define things like checking out code, configuring AWS credentials, logging into Amazon ECS, and anything else you need to accomplish your workflow.

You can, and should, also leverage a secrets manager for any service you are working with. Instead of adding your secrets and keys directly to your code, you instead declare where the secrets are stored. In this case, they are stored safely behind AWS’ security layer.

Declaring Task Definitions

Task definitions for Amazon CodeBuild are also needed to complete this workflow. This is very important because the task-definition.json file defines how the workflow is set up. This goes beyond the steps in the workflow that live in your .yml file. These task definitions create the names for resources and needed containers, set up port mappings, define which machine images are to be used, and anything else along those lines.

When Amazon CodeBuild runs, you should end up with an AWS ECS cluster that was implemented with all the details and test definitions you specified. If you want to use Kubernetes instead of Amazon’s flavor of containers, you can define it to do that. Again, every detail and option is completely customizable.

Meanwhile, GitHub is waiting for the results of the build process. After everything’s been executed, GitHub workflows will update to tell you how things went. If you see green checkmarks, then you know things went as expected. You can click on those checkmarks to see the details for any part of the process. The logging is quite in-depth, showing details about each individual step. 

Get Started with GitHub Actions and AWS CDK

Check out this video to see Adam Culp, senior principal SW architect at Learning A-Z, walk through an example of deploying to AWS from GitHub leveraging GitHub Actions and the AWS CDK, as well as a live Q&A from the 2021 GitKon Git conference!