Installing Self-hosted Runners for GitHub Actions

Objective

Install self-hosted runners to leverage your Panfactum clusters for executing GHA workflows.

Background and Limitations

Please ensure you have read this section and the associated links thoroughly before continuing.

GitHub Actions are not the recommended method for setting up CI / CD in Panfactum clusters. We provide a full set of guides for our recommended architecture here.

Additionally, GitHub Actions have many shortcomings compared to a more full-featured, Kubernetes-native workflow engine such as Argo Workflows. See our detailed analysis and list of limitations here.

Install the Runner Controller

To begin using self-hosted runners, you must first install the Actions Runner Controller (ARC) which is a Kubernetes operator that orchestrates and scales the runners based on Kubernetes custom resources.

We provide a module to deploy this controller: kube_gha.

Let's deploy it now:

Create a new directory adjacent to your aws_eks module called kube_gha.
Add a terragrunt.hcl to that directory that looks like this.
Run pf-tf-init in the directory to enable the required providers.
Run terragrunt apply.

You should now see a new Deployment running in the gha namespace called gha-controller.

Install the Runners

The controller allows you to create various "runners" which are the pods that will actually execute your GHA code.

To create runners, you will use our kube_gha_runners module.

Let's deploy one type of runner now:

Create a new directory adjacent to your kube_gha module called kube_gha_runners.
Add a terragrunt.hcl to that directory that looks like this.
1. Follow GitHub's documentation to provision a Classic Personal Access Token (PAT) with the correct permissions. Provide this token as an input via github_token. Make sure that the token is encrypted with sops.
2. The runners input is a key-value map that defines the different runner groups that your workflow jobs can target. The keys are the names of the groups and the values are their configuration. See the module documentation for information on the various configuration values.
  
  The names of your runners must be globally unique to your organization. So if you install runners in multiple clusters, they cannot have identical names.
3. The only required configuration value is the github_config_url which defines where in GitHub your runners will be registered for use. You can register runners to either (a) a single repository, (b) an entire organization, or (c) an entire enterprise. The github_config_url is simply the URL to one of those entities (e.g., https://github.com/Panfactum/stack). We recommend registering runners to your organization.
Run pf-tf-init in the directory to enable the required providers.
Run terragrunt apply.

Assuming you have registered runners to your organization, you should now be able to see your runners in your GitHub organization's settings:

Runners successfully registered with GitHub

Test the Runner

Let's create a test workflow in on of your repositories to verify that self-hosted runners are working as expected.

In your repository, create a file at .github/workflows/example.yaml.

Add the following configuration:

name: Example
on:
  workflow_dispatch: # This enables you to launch the workflow manually in the GitHub UI
jobs:
  test:
    runs-on: default # This should be the name of your runner in the `runners` input when you created it in the previous section
    steps:
      - name: "Hello world"
        shell: bash
        run: 'echo "Hello World"'

Commit and push this new .github/workflows/example.yaml.
Open the GitHub repository in a web browser. Navigate to the Actions tab. You should see the "Example" workflow in the left-hand sidebar:
Select the workflow. You should now see an option to run the workflow:
Run the workflow. Within a minute, the workflow job should begin, and you should see the runner pod in your cluster. It should complete successfully after one to two minutes.

If the workflow is stuck in a pending state, double check that your runner is available on the repository by visiting Actions > Runners > Self-hosted runners. Additionally, make sure that the runs-on value in your .github/workflows/example.yaml matches the name of at least one runner in this view.

Congratulations! You have successfully installed self-hosted GHA runners and can now run GHA workflows inside your Panfactum clusters.