CDK constructs for self-hosted GitHub Actions runners
Use this CDK construct to create ephemeral self-hosted GitHub runners on-demand inside your AWS account.
Self-hosted runners in AWS are useful when:
Ephemeral (or on-demand) runners are the recommended way by GitHub for auto-scaling, and they make sure all jobs run with a clean image. Runners are started on-demand. You don't pay unless a job is running.
The best way to browse API documentation is on Constructs Hub. It is available in all supported programming languages.
A runner provider creates compute resources on-demand and uses actions/runner to start a runner.
EC2 | CodeBuild | Fargate | ECS | Lambda | |
---|---|---|---|---|---|
Time limit | Unlimited | 8 hours | Unlimited | Unlimited | 15 minutes |
vCPUs | Unlimited | 2, 4, 8, or 72 | 0.25 to 4 | Unlimited | 1 to 6 |
RAM | Unlimited | 3gb, 7gb, 15gb, or 145gb | 512mb to 30gb | Unlimited | 128mb to 10gb |
Storage | Unlimited | 50gb to 824gb | 20gb to 200gb | Unlimited | Up to 10gb |
Architecture | x86_64, ARM64 | x86_64, ARM64 | x86_64, ARM64 | x86_64, ARM64 | x86_64, ARM64 |
sudo | ✔ | ✔ | ✔ | ✔ | ❌ |
Docker | ✔ | ✔ (Linux only) | ❌ | ✔ | ❌ |
Spot pricing | ✔ | ❌ | ✔ | ✔ | ❌ |
OS | Linux, Windows | Linux, Windows | Linux, Windows | Linux, Windows | Linux |
The best provider to use mostly depends on your current infrastructure. When in doubt, CodeBuild is always a good choice. Execution history and logs are easy to view, and it has no restrictive limits unless you need to run for more than 8 hours.
You can also create your own provider by implementing IRunnerProvider
.
pip install cloudsnorkel.cdk-github-runners
from cloudsnorkel.cdk_github_runners import GitHubRunners
GitHubRunners(self, "runners")
npm i @cloudsnorkel/cdk-github-runners
import { GitHubRunners } from '@cloudsnorkel/cdk-github-runners';
new GitHubRunners(this, "runners");
<dependency>
<groupId>com.cloudsnorkel</groupId>
<artifactId>cdk.github.runners</artifactId>
</dependency>
import com.cloudsnorkel.cdk.github.runners.GitHubRunners;
GitHubRunners.Builder.create(this, "runners").build();
go get github.com/CloudSnorkel/cdk-github-runners-go/cloudsnorkelcdkgithubrunners
import "github.com/CloudSnorkel/cdk-github-runners-go/cloudsnorkelcdkgithubrunners"
NewGitHubRunners(this, jsii.String("runners"))
dotnet add package CloudSnorkel.Cdk.Github.Runners
using CloudSnorkel;
new GitHubRunners(this, "runners");
GitHubRunners
construct in your code (starting with default arguments is fine)aws --region us-east-1 lambda invoke --function-name status-XYZ123 status.json
✅ github-runners-test
✨ Deployment time: 260.01s
Outputs:
github-runners-test.runnersstatuscommand4A30F0F5 = aws --region us-east-1 lambda invoke --function-name github-runners-test-runnersstatus1A5771C0-mvttg8oPQnQS status.json
--profile
too) and open the resulting status.json
filegithub.setup.url
from status.json
or manually setup GitHub integration as an app or with personal access tokengithub.auth.status
and github.webhook.status
are OKself-hosted
label with runs-on: [self-hosted, linux, codebuild]
or similarThe default providers configured by GitHubRunners
are useful for testing but probably not too much for actual production work. They run in the default VPC or no VPC and have no added IAM permissions. You would usually want to configure the providers yourself.
For example:
let vpc: ec2.Vpc;
let runnerSg: ec2.SecurityGroup;
let dbSg: ec2.SecurityGroup;
let bucket: s3.Bucket;
// create a custom CodeBuild provider
const myProvider = new CodeBuildRunnerProvider(this, 'codebuild runner', {
labels: ['my-codebuild'],
vpc: vpc,
securityGroups: [runnerSg],
});
// grant some permissions to the provider
bucket.grantReadWrite(myProvider);
dbSg.connections.allowFrom(runnerSg, ec2.Port.tcp(3306), 'allow runners to connect to MySQL database');
// create the runner infrastructure
new GitHubRunners(this, 'runners', {
providers: [myProvider],
});
Another way to customize runners is by modifying the image used to spin them up. The image contains the runner, any required dependencies, and integration code with the provider. You may choose to customize this image by adding more packages, for example.
const myBuilder = FargateRunnerProvider.imageBuilder(this, 'image builder');
myBuilder.addComponent(
RunnerImageComponent.custom({ commands: ['apt install -y nginx xz-utils'] }),
);
const myProvider = new FargateRunnerProvider(this, 'fargate runner', {
labels: ['customized-fargate'],
imageBuilder: myBuilder,
});
// create the runner infrastructure
new GitHubRunners(this, 'runners', {
providers: [myProvider],
});
Your workflow will then look like:
name: self-hosted example
on: push
jobs:
self-hosted:
runs-on: [self-hosted, customized-fargate]
steps:
- run: echo hello world
Windows images can also be customized the same way.
const myWindowsBuilder = FargateRunnerProvider.imageBuilder(this, 'Windows image builder', {
architecture: Architecture.X86_64,
os: Os.WINDOWS,
});
myWindowsBuilder.addComponent(
RunnerImageComponent.custom({
name: 'Ninja',
commands: [
'Invoke-WebRequest -UseBasicParsing -Uri "https://github.com/ninja-build/ninja/releases/download/v1.11.1/ninja-win.zip" -OutFile ninja.zip',
'Expand-Archive ninja.zip -DestinationPath C:\\actions',
'del ninja.zip',
],
}),
);
const myProvider = new FargateRunnerProvider(this, 'fargate runner', {
labels: ['customized-windows-fargate'],
imageBuilder: myWindowsBuilder,
});
new GitHubRunners(this, 'runners', {
providers: [myProvider],
});
The runner OS and architecture is determined by the image it is set to use. For example, to create a Fargate runner provider for ARM64 set the architecture
property for the image builder to Architecture.ARM64
in the image builder properties.
new GitHubRunners(this, 'runners', {
providers: [
new FargateRunnerProvider(this, 'fargate runner', {
labels: ['arm64', 'fargate'],
imageBuilder: FargateRunnerProvider.imageBuilder(this, 'image builder', {
architecture: Architecture.ARM64,
os: Os.LINUX_UBUNTU,
}),
}),
],
});
Runners are started in response to a webhook coming in from GitHub. If there are any issues starting the runner like missing capacity or transient API issues, the provider will keep retrying for 24 hours. Configuration issue related errors like pointing to a missing AMI will not be retried. GitHub itself will cancel the job if it can't find a runner for 24 hours. If your jobs don't start, follow the steps below to examine all parts of this workflow.
runs-on
in the workflow matches the expected labels set in the runner providertroubleshooting.stepFunctionUrl
from status.json
troubleshooting.webhookHandlerUrl
from status.json
workflow_job
eventsgithub.auth.app.installations
All logs are saved in CloudWatch.
status.json
for each provider, image builder, and other parts of the systemGitHubRunners.createLogsInsightsQueries()
To get status.json
, check out the CloudFormation stack output for a command that generates it. The command looks like:
aws --region us-east-1 lambda invoke --function-name status-XYZ123 status.json
There are two important ways to monitor your runners:
GitHubRunners.metricFailed()
to get a metric for the number of failed runner starts. You should use this metric to trigger an alarm.GitHubRunners.failedImageBuildsTopic()
to get SNS topic that gets notified of failed runner image builds. You should subscribe to this topic.Other useful metrics to track:
GitHubRunners.metricJobCompleted()
to get a metric for the number of completed jobs broken down by labels and job success.GitHubRunners.metricTime()
to get a metric for the total time a runner is running. This includes the overhead of starting the runner.