github Self-Hosted Runners on AWS CodeBuild

👋 Hey there!

I’ve written before about establishing Self-Hosted Runners within github.com here. However, this involves deploying API endpoints and integrating with github via a WebHook. It’s not hard to establish, but it’s extra work to look after, update, and manage.

That has been made much easier with the following feature I stumbled across by chance in the AWS Console.

Official Documentation:- Set up self-hosted github-actions runners in AWS CodeBuild

This makes things simple to set up but also gives flexibility to the github-actions workflow author with some of the following benefits:-

• Ephemeral runners, you don’t need to patch!
• Ephemeral runners that can spin up within your VPC, in your network!
• Set the compute architecture type directly within the workflow definition
• Access to native AWS services right from within github-actions
• There are no secrets to manage if configured via OAuth

Setup

Connect with an OAuth app

Establish an OAuth AWS CodeBuild connection to your github Account. You can do this via a Personal Access Token (PAT); however, I’d recommend you don’t! Using OAuth and a github App is a much better way, providing fine-grained access to only what’s required in github.

To set this up head over to CodeBuild, Build projects, Create build project

Select github as the Source.

Click on Connect using OAuth. This will open the following window which will request access to your github Account. From here you can approve access to just your account or to any github Organisations you have.

.

Finally, CodeBuid will ask for confirmation to establish the connection.

Once the connection is successfully established, you will be able to list the repositories within your github Account or Organisations.

NOTE:- If you are using different credentials for your AWS & github Accounts, then you may need to mess about logging in with your github credentials in your browser prior to hitting the Connect to github button.

Create a CodeBuild Project

With the connection established, we now create our CodeBuild Project following the steps from the AWS documentation here

The key thing that enables our integration is the WORKFLOW_JOB_QUEUED event trigger. This will trigger the codebuild project from these events within your github repo.

Reference your CodeBuild Project in github-actions

Now, back over in github, we just need to update our runs-on value to trigger our CodeBuild Project.

name: Hello World
on: [push]
jobs:
  Hello-World-Job:
    runs-on: codebuild-myProject-${{ github.run_id }}-${{ github.run_attempt }}
    steps:
      - run: echo "Hello World!"

What’s more is, if you want to use a different architecture, you can just override it right from within your workflow! Very powerful!

name: Hello World
on: [push]
jobs:
  Hello-World-Job:
    runs-on: codebuild-myProject-${{ github.run_id }}-${{ github.run_attempt }}-arm-3.0-small
    steps:
      - run: echo "Hello World!"

Here is an example of the standard output from github-actions running the Super-Linter on CodeBuild

Here is the CloudWatch output log from the CodeBuild Project

Summary

This integration from AWS gives users a simple and flexible way to establish ephemeral Self-Hosted runners in AWS for github-actions without the headaches of previous solutions.

It opens up so many possibilities to further integrate github-actions Workflows with AWS native services. I can’t wait to play with it more!

Hope someone finds this helpful!

Cheers