Conversation
…per month for public repositories with 4 vCPU runners. Could try 3000 free with 2 vCPU or 750 mins free with 8 vCPU too but build apparently heavy so better start with 2 or 4 and went with the middle option). See https://docs.blacksmith.sh/introduction/quickstart GitHub's plans to ask 0.002 USD per minute GitHub Actions for self-hosted runners would have applied to Blacksmith too. Were it not those plans were postponed and more importantly, runners will remain free for public repositories (see https://github.blog/changelog/2025-12-16-coming-soon-simpler-pricing-and-a-better-experience-for-github-actions/). Our Action runs are currently also free for the public repo with the standard GitHub-hosted runners, but those are slower (see https://docs.github.com/en/billing/concepts/product-billing/github-actions). Blacksmith caches dependencies and build outputs by default when running jobs with them. Blacksmith's docker build layer caching isn't free (0.50 USD/GB/month, see https://docs.blacksmith.sh/blacksmith-caching/docker-builds#pricing). While GHCR currently is free (https://docs.github.com/en/billing/concepts/product-billing/github-packages#free-use-of-github-packages), so it's better to stick with GHCR for now for that.
VariableVince
added
the
Devops
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughUpdated runner environment across three GitHub Actions workflows (deploy, release, and ci) from Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/release.yml (1)
15-202:⚠️ Potential issue | 🟠 MajorAdd actionlint configuration to allow the custom runner label.
The custom runner label
blacksmith-4vcpu-ubuntu-2404is used in lines 15, 50, 100, 151, and 202 of.github/workflows/release.yml, but no actionlint configuration exists in the repository. When actionlint runs, it will fail on this unknown label and block CI validation.Create an
actionlint.yamlfile to register the runner label:actionlint.yaml
self-hosted-runner: labels: - blacksmith-4vcpu-ubuntu-2404🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/release.yml around lines 15 - 202, Add an actionlint configuration file to register the custom runner label used in the workflow; create a new file named actionlint.yaml containing a top-level key (e.g., self-hosted-runner) with a labels array that includes the exact label "blacksmith-4vcpu-ubuntu-2404" so actionlint recognizes that runner used in the release.yml runs-on fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In @.github/workflows/release.yml:
- Around line 15-202: Add an actionlint configuration file to register the
custom runner label used in the workflow; create a new file named
actionlint.yaml containing a top-level key (e.g., self-hosted-runner) with a
labels array that includes the exact label "blacksmith-4vcpu-ubuntu-2404" so
actionlint recognizes that runner used in the release.yml runs-on fields.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5d8901fc-fe8f-4a53-90e7-4a0f4cdae995
📒 Files selected for processing (2)
.github/workflows/deploy.yml.github/workflows/release.yml
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/pr-author.yml:
- Line 13: The workflow currently sets runs-on to a third-party runner
("blacksmith-4vcpu-ubuntu-2404") which weakens security for a privileged job
triggered by pull_request_target; change the runs-on value back to
"ubuntu-latest" (or another GitHub-hosted runner) for the job that uses
pull_request_target/write permissions so the workflow retains GitHub-hosted
runner protections (look for the runs-on entry and the pull_request_target job
definition in the workflow to locate and update it).
In @.github/workflows/pr-stale.yml:
- Around line 9-11: The workflow uses an external runner for a job that has
write-scoped permissions (the runs-on setting "blacksmith-4vcpu-ubuntu-2404"
together with the permissions "pull-requests: write"); change the runs-on to a
GitHub-hosted runner (e.g., "ubuntu-latest") or split the job so that any step
requiring "pull-requests: write" runs only on a trusted runner, and ensure the
job that retains "permissions: pull-requests: write" is the one updated to use
the GitHub-hosted runner (adjust the runs-on and/or job boundaries accordingly).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5dcf6f52-41e3-477f-bfc5-77fbbc50e4b6
📒 Files selected for processing (4)
.github/workflows/ci.yml.github/workflows/pr-author.yml.github/workflows/pr-description.yml.github/workflows/pr-stale.yml
github-project-automation
bot
moved this from Triage
to Development
in OpenFront Release Management
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/deploy.yml (1)
50-50: Add Blacksmith runner label to actionlint config to suppress false positives.The
blacksmith-4vcpu-ubuntu-2404label is valid for the Blacksmith runner service, but actionlint does not recognize third-party runner labels by default. To fix the static analysis errors across all workflows, add an.github/actionlint.yamlconfig file.📝 Proposed actionlint config
Create
.github/actionlint.yaml:self-hosted-runner: labels: - blacksmith-4vcpu-ubuntu-2404Security note: This workflow handles sensitive secrets (SSH keys, API tokens) on a third-party runner. Blacksmith is a reputable provider, but verify their security posture meets your compliance requirements for handling deployment credentials.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/deploy.yml at line 50, Create an .github/actionlint.yaml file to tell actionlint the third-party runner label is valid: add a top-level key self-hosted-runner with labels containing "blacksmith-4vcpu-ubuntu-2404" so actionlint recognizes the runner and suppresses false positives for the runs-on value; ensure the file includes the exact label string and is committed to the repo root so workflows using runs-on: blacksmith-4vcpu-ubuntu-2404 validate cleanly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In @.github/workflows/deploy.yml:
- Line 50: Create an .github/actionlint.yaml file to tell actionlint the
third-party runner label is valid: add a top-level key self-hosted-runner with
labels containing "blacksmith-4vcpu-ubuntu-2404" so actionlint recognizes the
runner and suppresses false positives for the runs-on value; ensure the file
includes the exact label string and is committed to the repo root so workflows
using runs-on: blacksmith-4vcpu-ubuntu-2404 validate cleanly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e37d19ec-4a2f-4a43-af02-76b34d52f850
📒 Files selected for processing (6)
.github/workflows/ci.yml.github/workflows/deploy.yml.github/workflows/pr-author.yml.github/workflows/pr-description.yml.github/workflows/pr-stale.yml.github/workflows/release.yml
pr-stale and pr-author: back to default github runners ubuntu-latest. They have write rights on PRs and are 1s slower on Blacksmith 4vCPU anyway, so better keep them as they are. pr-description: test on 2vCPU to see if there's a difference in build time that has an impact, since on 4vCPU it is 1s slower too. Could move it back to default github runners ubuntu-latest too, or keep at 2vCPU
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/ci.yml (1)
16-16: Configure actionlint to recognize Blacksmith runner labels.The actionlint tool flags these runner labels as unknown. To suppress these false positives, add a
.github/actionlint.yamlconfiguration file.📝 Sample actionlint.yaml configuration
Create
.github/actionlint.yaml:self-hosted-runner: labels: - blacksmith-2vcpu-ubuntu-2404 - blacksmith-4vcpu-ubuntu-2404 - blacksmith-8vcpu-ubuntu-2404This tells actionlint that these are valid custom runner labels.
Do you want me to create a complete actionlint.yaml file with the proper configuration?
Also applies to: 35-35, 51-51, 64-64
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/ci.yml at line 16, Add a .github/actionlint.yaml file that tells actionlint these custom self-hosted runner labels are valid: include a top-level key self-hosted-runner with a labels array containing "blacksmith-2vcpu-ubuntu-2404", "blacksmith-4vcpu-ubuntu-2404", and "blacksmith-8vcpu-ubuntu-2404" (and add any other blacksmith-* labels you use); this will stop actionlint from flagging the runs-on: blacksmith-8vcpu-ubuntu-2404 occurrences (and the similar labels on other lines) as unknown.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In @.github/workflows/ci.yml:
- Line 16: Add a .github/actionlint.yaml file that tells actionlint these custom
self-hosted runner labels are valid: include a top-level key self-hosted-runner
with a labels array containing "blacksmith-2vcpu-ubuntu-2404",
"blacksmith-4vcpu-ubuntu-2404", and "blacksmith-8vcpu-ubuntu-2404" (and add any
other blacksmith-* labels you use); this will stop actionlint from flagging the
runs-on: blacksmith-8vcpu-ubuntu-2404 occurrences (and the similar labels on
other lines) as unknown.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 76698e27-51a8-48f6-a468-e6904daf3bcb
📒 Files selected for processing (4)
.github/workflows/ci.yml.github/workflows/deploy.yml.github/workflows/pr-description.yml.github/workflows/release.yml
…slower doesn't really hurt but it seems Github runners have a little faster access to pull requests or so. Plus this makes all PR workflows back on ubuntu-latest which looks more organized
1 participant
Description:
Use Blacksmith for faster and free action runs.
Chose their 2vCPU runners to fit our usage best, and the workflows/steps it would fit us best within their free plan.
Now we can choose between a combination of CI/Test and Release, or Deploy and Release to migrate within the free plan. Release profits from more than 1 minute faster deploy to the production instance alone.
2vCPU (see full performance comparisons below):
CI/Test: ~26% faster (1m 21s > 52s = 29s)
Deploy: ~19.5% faster (2m 39s > 2m 8s = 31s)
Release: cannot measure without real release, but its Deploy steps could be somewhat comparable to deploy in % faster. Which could mean that its Deploy to Green step for v0.30.12 would have taken 4m 44s instead of 5m 53s.
INFORMATION AND CHOICES
See https://docs.blacksmith.sh/introduction/quickstart
3000 free free per month for public repositories with 2 vCPU runners, 1500 minutes free for 4 vCPU, 750 minutes for 8 vCPU. See https://www.blacksmith.sh/pricing or https://docs.blacksmith.sh/blacksmith-runners/overview#is-there-a-free-tier
GitHub's plans to ask 0.002 USD per minute GitHub Actions for self-hosted runners would have applied to Blacksmith too. Were it not those plans were postponed and more importantly, runners will remain free for public repositories (see https://github.blog/changelog/2025-12-16-coming-soon-simpler-pricing-and-a-better-experience-for-github-actions/).
Our Action runs are currently also free for the public repo with the standard GitHub-hosted runners, but those are slower.
Blacksmith caches dependencies and build outputs by default when running jobs with them.
Blacksmith's docker build layer caching isn't free (0.50 USD/GB/month). While GHCR currently is free, so it's better to stick with GHCR for now for that.
PERFORMANCE MEASUREMENTS
With 4vCPU Blacksmith against the default GitHub runners:
Gave the Deploy job 3 runs, to let Blacksmith be able to build up and use cache.
Compared this PR (with the Blacksmith runners, and the Node 24 commits with new Action versions merged from main. Includes Deploy action.). CI/Prettier: 42s, CI/Build: 44s, CI/ESLint: 37s, CI/Test: 40s, PR/Has Milestone: 4s, PR/Validate Description: 5s, Deploy: 2m10s
with:
PR 3475 (1 file, Github runners, also has the Node 24 commits with new Action versions merged from main. Includes Deploy action.). CI/Prettier: 1m2s, CI/Build: 59s, CI/ESLint: 55s, CI/Test: 1m22s, PR/Has Milestone: 5s, PR/Validate Description: 2s, Deploy: 2m42s
PR 3450 (6 files, Github runners, also has the Node 24 commits with new Action versions merged from main. Includes Deploy action.)
CI/Prettier: 1m2s, CI/Build: 57s, CI/ESLint: 54s, CI/Test: 1m18s, PR/Has Milestone: 5s, PR/Validate Description: 5s, Deploy: 2m36s
PR 3439 (10 files, Github runners, Node 20 workflows. No Deploy action.)
CI/Prettier: 1m6s, CI/Build: 58s, CI/ESLint: 56s, CI/Test: 1m23s, PR/Has Milestone: 3s, PR/Validate Description: 4s
PR 3438 (1 file, Github runners, Node 20 workflows. No Deploy action.)
CI/Prettier: 1m3s, CI/Build: 58s, CI/ESLint: 52s, CI/Test: 1m21s, PR/Has Milestone: 3s, PR/Validate Description: 5s
4vCPU:
CI/Prettier: ~34% faster on average (1m 3.25s > 42s = 21.25s)
CI/Build: ~24% faster on average (58s > 44s = 14s)
CI/ESLint: ~32% faster (54.25s > 37s = 17.25s)
CI/Test: ~51% faster (1m 21s > 40s = 41s)
Deploy: ~18% faster (2m 39s > 2m10s = 29s)
PR/Has Milestone AND PR/Validate Description: 1s slower on all Blacksmith machines so better left on Github runners (maybe slightly longer startup time for Blacksmith which negates faster script run, or less fast access to the PR).
PR/Stale AND PR/author: not measured. The first is a cron job so not measured and has write access, second has write access too (CodeRabbit advises to use GitHub runners for PR/Has Milestone and PR/Stale since they have write rights on the PR anyway, so better keep the PR ones on ubuntu-latest)
Release: cannot measure without real release, but its Deploy steps could be somewhat comparable to deploy in % faster. Which could mean that its Deploy to Green step for v0.30.12 would have taken 4m 49s instead of 5m 53s.
Overall: 31.8% (18-51%) or 24s (14-41s) faster per workflow, on average. For Release action step Deploy to Green/Blue, an estimated 1m 4s faster release to Prod based on the performance gains for the Deploy action.
With 8vCPU Blacksmith against the default GitHub runners:
Gave the Deploy job 3 runs, to let Blacksmith be able to build up and use cache.
Compared this PR with the same PRs as for the 4vCPU comparison above.
This PR: CI/Prettier: 39s, CI/Build: 37s, CI/ESLint: 36s, CI/Test: 37s, PR/Has Milestone: 5s, PR/Validate Description: 5s, Deploy: 2m7s (fastest was 1m 28s with the 3rd deploy of the same commit, after Blacksmith had probably been able to cache things)
8vCPU:
CI/Prettier: ~38.5% faster on average (1m 3.25s > 39s = 24.25s)
CI/Build: ~36.5% faster on average (58s > 37s = 21s)
CI/ESLint: ~33.5% faster (54.25s > 36s = 18.25s)
CI/Test: ~54.5% faster (1m 21s > 37s = 44s)
Deploy: ~20% faster (2m 39s > 2m 7s = 32s)
Release: cannot measure without real release, but its Deploy steps could be somewhat comparable to deploy in % faster. Which could mean that its Deploy to Green step for v0.30.12 would have taken 4m 42s instead of 5m 53s.
Overall: 36.6% (20-54.5%) or 34.9s (18.25-44s) faster per workflow, on average. For Release action step Deploy to Green/Blue, an estimated 1m 11s faster release to Prod based on the performance gains for the Deploy action.
With 2vCPU Blacksmith against the default GitHub runners:
Compared PR 3474 (with the Blacksmith runners, and the Node 24 commits with new Action versions merged from main. Includes Deploy action.). CI/Prettier: 58s, CI/Build: 37s, CI/ESLint: 36s, CI/Test: 37s, PR/Has Milestone: 5s, PR/Validate Description: 5s, Deploy: 2m7s (fastest was 1m 28s with the 3rd deploy of the same commit, after Blacksmith had probably been able to cache things)
Only ran on 3 Steps/Workflows:
CI/Test: ~26% faster (1m 21s > 52s = 29s)
Deploy: ~19.5% faster (2m 39s > 2m 8s = 31s)
Release: cannot measure without real release, but its Deploy steps could be somewhat comparable to deploy in % faster. Which could mean that its Deploy to Green step for v0.30.12 would have taken 4m 44s instead of 5m 53s.
Please complete the following:
Please put your Discord username so you can be contacted if a bug or regression is found:
tryout33