Creating shared GitHub Actions

Creating shared GitHub Actions

ยท

7 min read

๐Ÿ‘‹ Hey there!

๐Ÿ‘ If you use GitHub Actions for your CI/CD workflows, you may reach a point where you need to maintain similar workflows across multiple repositories. At this stage, it's a good idea to consider creating a "Reusable" / "Shared" Action.

In this article, the author explains how to turn a Terraform workflow into a reusable, versioned workflow that can be referenced across many repositories.

NOTE:- I'd recommend making sure the workflow has matured to a point where you want to switch to the shared action

If you just want to see the code, see the following gists:-

Workflow Before

Workflow After

The Workflow

Before pulling this apart into a shared workflow, let's first examine what the workflow is performing.

๐Ÿ” To begin with, we give it a name and schedule when it should run. As the name suggests, it runs a Terraform Plan whenever a pull request is made, but it can also be triggered manually. It also ensures our README is kept up to date using terraform-docs.

---
name: Terraform Plan

# yamllint disable-line rule:truthy
on:
  pull_request:
    branches: [master, main]

  workflow_dispatch:

Our Jobs section is divided into two parts - Validation and Docs. Validation performs the Terraform Plan and Docs runs terraform-docs on our HCL code.

The permissions key sets various permissions required for our action to run correctly. These will be assigned to the GITHUB_TOKEN context in which our workflow runs.

As we are running Terraform commands, we need the binary available. The container key will pull down the latest version of Terraform from Docker Hub. The environment variables are set with the repository secrets for our GitHub App used to authenticate Terraform to GitHub using the GitHub Provider.

jobs:
  Validation:
    name: Terraform Plan Workflow
    runs-on: ubuntu-latest
    needs: docs
    permissions:
      id-token: write
      contents: read
      issues: write
      pull-requests: write

    container:
      image: hashicorp/terraform:latest
      env:
        GITHUB_APP_ID: ${{ secrets.TF_APP_ID }}
        GITHUB_APP_INSTALLATION_ID: ${{ secrets.TF_APP_INSTALLATION_ID }}
        GITHUB_APP_PEM_FILE: ${{ secrets.TF_APP_PEM_FILE }}
        GITHUB_ORGANIZATION: 'github-organisation'

Now, we move to the steps within our Validation job. We check out our code and configure our AWS Credentials via OIDC. These are necessary as we use S3 & Dynamodb to maintain our Terraform state.

For more details on configuring OIDC, see my other article - Terraform, GitHub Actions & OIDC on AWS

We also need to generate a JWT to allow Terraform to access the modules in another organisation; this step requires additional environment variables to support this.

steps:
  - name: Checkout Code
    uses: actions/checkout@v4

  - name: Configure AWS Credentials
    uses: aws-actions/configure-aws-credentials@v4
    with:
      audience: https://github.com/github-organisation
      aws-region: ap-southeast-2
      role-to-assume: arn:aws:iam::123456789123:role/app-github-organisation-123456789123
      role-session-name: github-organisation

  - uses: actions/create-github-app-token@v1
    id: app-token
    with:
      app-id: ${{ secrets.TERRAFORM_MODULES_APP_ID }}
      private-key: ${{ secrets.TERRAFORM_MODULES_APP_PRIVATE_KEY }}
      owner: another-github-organisation

Now that the previous authentication steps have taken place we need to inject these credentials into the git config so that Terraform will use these to authenticate to pull down the modules defined within the HCL code.

  - name: Terraform Initialization
    run: |
      git config --global url."https://oauth2:${{ steps.app-token.outputs.token }}@github.com".insteadOf https://github.com
      terraform init  -backend-config="terraform_state.tfvars" -upgrade
    id: terraform_init

Finally, we perform the standard Terraform workflow commands to validate & format the HCL and run & output the plan.

  - name: Terraform Validation
    run: terraform validate
    id: terraform_validate

  - name: Terraform Format and Style
    run: terraform fmt -check -diff -recursive
    id: terraform_fmt

  - name: Terraform Plan
    id: terraform_plan
    # if: github.event_name == 'pull_request'
    run: terraform plan -no-color --out tfplan.binary
    continue-on-error: true

  - name: Terraform JSON output
    id: terraform_json
    if: steps.terraform_plan.outcome == 'success'
    run: terraform show -json tfplan.binary > tfplan.json
    continue-on-error: true

The final section of the validate workflow will inform the user of the workflow status by updating the PR using github-script.

  - name: Update Pull Request
    uses: actions/github-script@v6
    if: github.event_name == 'pull_request'
    with:
      script: |
        const output = `${ "${{ steps.terraform_fmt.outcome }}" == "success" ? "โœ”๏ธ" : "โŒ" } Terraform Format and Style ๐Ÿ–Œ
        ${ "${{ steps.terraform_init.outcome }}" == "success" ? "โœ”๏ธ" : "โŒ" } Terraform Initialization โš™๏ธ
        ${ "${{ steps.terraform_plan.outcome }}" == "success" ? "โœ”๏ธ" : "โŒ" } Terraform Plan ๐Ÿ“–
        ${ "${{ steps.terraform_validate.outcome }}" == "success" ? "โœ”๏ธ" : "โŒ" } Terraform Validation ๐Ÿค–

        *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;

        github.rest.issues.createComment({
          issue_number: context.issue.number,
          owner: context.repo.owner,
          repo: context.repo.repo,
          body: output
        });

  - name: Terraform Plan Status
    if: steps.terraform_plan.outcome == 'failure'
    run: exit 1

The docs job, uses terraform-doc to autogenerate out documentation. See more examples here.

  docs:
    name: Terraform Docs
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.ref }}

      - name: Render terraform docs inside the README.md and push changes back to PR branch
        uses: terraform-docs/gh-actions@main
        with:
          working-dir: .
          output-file: README.md
          output-method: inject
          git-push: "true"

Creating a shared (reusable) workflow

Workflow Repository

The first step is to create a repository that will host our workflow. Make sure you give it a meaningful name and follow some conventions as it's going to be referenced by this in your calling repositories.

The visibility settings for your repository depend on how widely you plan to share these workflows. If you intend to reuse workflows across multiple repositories within the same organisation, "private" should be fine as long as all relevant repositories belong to the same organisation and have permission to access the shared workflows repository.

This approach keeps your CI/CD logic private and secure, accessible only to members of your organisation, and gives you more control over who can access it.

If you want to share your workflows with repositories outside of your organisation, you need to set the repository to "public". Regardless of affiliation, any GitHub user or repository can reference and use your shared workflows in their projects.

NOTE: Please don't host anything sensitive in public repositories, such as secrets or access keys.

Within this repository, we create a structure for our workflow

/
/.github/workflows/
/.github/workflows/terraform-plan.yml

Adapt the workflow for reuse

Shared workflows are different, so this is where our modifications take place. The main difference is how you define the on triggers and inputs for the workflow.

Shared workflows typically use the workflow_call event. Documentation is available here.

In our normal workflow, we can reference the secret variables defined in our repository. When we create a reusable workflow these all need to be defined as inputs & secrets.

The following shows our changes, defining all the previous secrets as variables required for the workflow.

---
name: 'Terraform GitHub Plan Shared Action'

# yamllint disable-line rule:truthy
on:
  workflow_call:
    inputs:
      aws_region:
        description: 'The target AWS Region for access to S3 & DynamoDB'
        required: true
        type: string
      role_to_assume:
        description: 'The target IAM Role to assume to grant access to AWS S3 & DynamoDB or other resources required'
        required: true
        type: string
      aws_audience:
        description: 'Audience value used to restrict access to the IAM Role'
        required: true
        type: string
      role_session_name:
        description: 'Session name for the Assumed IAM Role'
        required: false
        type: string
      terraform_backend_config:
        description: 'The Terraform State Variable file defining the location of state settings such as S3, DynamoDb etc'
        default: 'terraform_state.tfvars'
        required: false
        type: string
    secrets:
      github_app_id:
        description: 'The GitHub App ID that provides the necessary access for Terraform to configure GitHub resources'
        required: true
      github_app_installation_id:
        description: 'The GitHub App Installation ID of the GitHub App ID'
        required: true
      github_app_pem_file:
        description: 'The contents of the GitHub App Private Key file'
        required: true
      terraform_app_id:
        description: 'The GitHub App ID that provides the necessary access to the Terraform Community Repositories'
        required: true
      terraform_app_pem_file:
        description: 'The contents of the GitHub App Private Key file that provides the necessary access to the Terraform Community Repositories'
        required: true

The next modification is to adjust how we reference the inputs & secrets for the various parts through the workflow steps.

jobs:
  Validation:
    name: Terraform Plan Workflow
    runs-on: ubuntu-latest
    needs: docs
    permissions:
      id-token: write
      contents: read
      issues: write
      pull-requests: write

    container:
      image: hashicorp/terraform:latest
      env:
        GITHUB_APP_ID: ${{ secrets.github_app_id }}
        GITHUB_APP_INSTALLATION_ID: ${{ secrets.github_app_installation_id }}
        GITHUB_APP_PEM_FILE: ${{ secrets.github_app_pem_file }}

    steps:
      - name: Checkout Code
        uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          audience: ${{ inputs.aws_audience }}
          aws-region: ${{ inputs.aws_region }}
          role-to-assume: ${{ inputs.role_to_assume }}
          role-session-name: ${{ inputs.role_session_name }}

      - uses: actions/create-github-app-token@v1
        id: app-token
        with:
          app-id: ${{ secrets.terraform_app_id }}
          private-key: ${{ secrets.terraform_app_pem_file }}
          owner: 'another-github-organisation'

      - name: Terraform Initialization
        run: |
          git config --global url."https://oauth2:${{ steps.app-token.outputs.token }}@github.com".insteadOf https://github.com
          terraform init  -backend-config="${{ inputs.terraform_backend_config }}" -upgrade
        id: terraform_init

Calling the shared workflow

Now we have our reusable workflow we can just call it with the various inputs and secrets required. Note these are not shared with the repository hosting the action and remain private to the calling repository workflow.

Please see the documentation here for more information on inputs and secrets.

---
name: Shared Terraform Plan

# yamllint disable-line rule:truthy
on:
  pull_request:
    branches: [master, main]
  workflow_dispatch:

jobs:
  call-shared-action:
    uses: myactions/terraform-github-provider-workflows/.github/workflows/terraform-plan.yml@v1.0.0
    with:
      aws_region: 'ap-southeast-2'
      aws_audience: 'https://github.com/github-organisation'
      role_to_assume: 'arn:aws:iam::123456789123:role/app-github-organisation-123456789123'
      terraform_backend_config: 'terraform_state.tfvars'
    secrets:
      github_app_id: ${{ secrets.TF_APP_ID }}
      github_app_installation_id: ${{ secrets.TF_APP_INSTALLATION_ID }}
      github_app_pem_file: ${{ secrets.TF_APP_PEM_FILE }}
      terraform_app_id: ${{ secrets.TERRAFORM_MODULE_APP_ID }}
      terraform_app_pem_file: ${{ secrets.TERRAFORM_MODULE_APP_PRIVATE_KEY }}

Summary

This post explains how to convert a GitHub Actions workflow to be used across many repositories. This has several benefits of avoiding duplication and introduction of best practices such as semantic versioning for your workflows.

I hope this helps someone else!

Cheers

Did you find this article valuable?

Support sjramblings by becoming a sponsor. Any amount is appreciated!

ย