-
-
Notifications
You must be signed in to change notification settings - Fork 0
Add previews/v1 service for preview build lookup #195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
35f9ebc
Add previews/v1 service for preview build lookup
admdly 51a38d2
Match previews/v1 resolver to the real artifact naming/metadata
admdly 6accfc4
Deslop previews/v1: dedupe route dispatch, drop dead defensive check
admdly a0c427b
Cache-share /download routes to cut GitHub API calls per hit
admdly f0bacce
Fix pre-existing typecheck failure: bump @hono/zod-openapi to 1.5.3
admdly 905c673
Longer cache TTL for immutable commit lookups
admdly d499766
Update previews/v1 README for the commit-lookup TTL change
admdly e8fee1d
Add GET /previews/v1/main/download for uniform addressing
admdly d7e29c5
Enrich /main with GitHub Actions build metadata for shape parity
admdly 9b25f9e
Fix review findings: fork-PR resolution, TTL/cache correctness, docs
admdly 9c21738
Fix second review pass: TTL edge cases and fallback pagination
admdly 48a09fd
Raise fallback pagination cap from a correctness bound to a circuit b…
admdly 40116de
Deslop: extract repeated ArtifactMatch type, merge overlapping tests
admdly 5fbc8dc
Modify R2 bucket binding and name in wrangler.jsonc
admdly 9d6ca7e
Rename preview R2 binding to DOWNLOAD_BUCKET
admdly 886dbd9
Stop duplicating the generated OpenAPI contract in previews/v1's README
admdly c4434b4
Add field-level OpenAPI descriptions to previews/v1's response schemas
admdly File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # Previews Service | ||
|
|
||
| **Base Path:** `/previews/v1` | ||
|
|
||
| Read-only lookup of FOSSBilling preview builds. GitHub Actions is the source | ||
| of truth for PR/commit previews - `FOSSBilling/FOSSBilling`'s `ci.yml` | ||
| uploads one artifact per commit, named `FOSSBilling-preview-{short_sha}.zip` | ||
| (`archive: false`, so the zip itself is the artifact - no extra wrapping), | ||
| for every PR build, non-main branch push, and main push. This service | ||
| resolves by querying that exact name rather than listing every preview | ||
| artifact and filtering. The `main` preview's `download_url`/`digest` are | ||
| answered from R2 instead, sourced from `digest`/`commit-sha` custom object | ||
| metadata the same CI job sets on the R2 upload - kept separate from the | ||
| GitHub-artifact path because the R2 zip and the GitHub artifact zip for a | ||
| given commit are two independently-built files (a `cp` of the same bytes, | ||
| in the current CI job, but not guaranteed to stay that way), so whichever | ||
| one is reported as the digest has to match the bytes `main` actually | ||
| serves. `GET /main` does still cross-reference that commit's GitHub Actions | ||
| artifact for enrichment - see Resource Model below - but only as | ||
| best-effort, never as a dependency. | ||
|
|
||
| There is no publish/write endpoint: nothing pushes data into this service, | ||
| it only resolves and redirects. | ||
|
|
||
| ## Endpoints | ||
|
|
||
| Endpoints are not listed here. The service publishes its own contract: | ||
|
|
||
| - **OpenAPI document:** `GET /previews/v1/openapi.json` | ||
| - **Reference UI:** `GET /previews/v1/docs` | ||
|
|
||
| ## Resource Model | ||
|
|
||
| - `GET /main` and `GET /pr/{number}` are **pointers** - they always resolve | ||
| to whatever is current. | ||
| - `GET /commit/{sha}` is a **fixed point** - one commit, one build, | ||
| permanently addressable (until GitHub's artifact retention expires it). | ||
| - `pr/{number}`'s handler resolves the PR to its head SHA | ||
| (`GET /pulls/{number}`) and delegates to the same resolver `commit/{sha}` | ||
| uses - one GitHub-facing code path, not two. | ||
| - `download_url` differs in kind depending on the resource. `main`'s is the | ||
| permanent public `download.fossbilling.org` URL, embedded directly, since | ||
| it never expires. `pr`/`commit`'s is self-referential - it points back at | ||
| their own `/download` sub-route rather than GitHub's actual signed URL, | ||
| because that URL expires in ~60s and can't be baked into a response with | ||
| any longer cache lifetime; `/download` resolves the real one live on | ||
| each hit. | ||
| - `source` on `/main` stays `"r2"` regardless of whether the GitHub Actions | ||
| enrichment below resolves - it describes where `download_url`/`digest` | ||
| come from, which never changes. | ||
| - `main`'s `run_id`/`artifact_id`/`created_at`/`expires_at` are enrichment, | ||
| resolved from that commit's GitHub Actions artifact (the same lookup | ||
| `commit/{sha}` uses) purely for shape parity with the PR/commit response, | ||
| so a client reading either doesn't have to special-case field | ||
| availability. It's best-effort and never load-bearing: a miss (no known | ||
| artifact yet, the artifact aged out of GitHub's 14-day retention, GitHub | ||
| unavailable) just leaves those four fields `null` - it's never the reason | ||
| a request to `/main` fails, since `download_url`/`digest` are R2-sourced | ||
| and don't depend on it. | ||
|
|
||
| ## Notes | ||
|
|
||
| - Responses are cached in `CACHE_KV`, only for successful lookups - a | ||
| not-yet-built PR or a transient GitHub error always re-resolves on the | ||
| next request. `GET /pr/{number}` (`preview:pr:{number}`, also used by | ||
| `/pr/{number}/download` to avoid re-resolving what the metadata route | ||
| already cached) and `GET /main` (`preview:main`) use the 60s default, | ||
| matching how often a moving pointer can realistically change. | ||
| `GET /commit/{sha}` (`preview:commit:{sha}`, likewise shared with | ||
| `/commit/{sha}/download`) uses 3600s instead - a commit's build never | ||
| changes once it exists, so there's no correctness reason to re-check it | ||
| every minute. That 3600s is capped at the artifact's own remaining | ||
| GitHub retention (minus a small safety margin for the cache write | ||
| itself), so a lookup resolved near the end of an artifact's 14-day life | ||
| is never cached longer than the artifact actually exists. Within roughly | ||
| the final minute of that life the capped value falls under KV's 60s | ||
| minimum TTL, so those requests (and any more before the artifact expires | ||
| or a request refreshes it) are just served live instead of cached - a | ||
| short burst of extra GitHub calls right at the end, never stale data. | ||
| - `GET /pr/{number}/download` and `GET /commit/{sha}/download` always | ||
| resolve GitHub's signed redirect URL live, never cached - it expires in | ||
| about a minute, and Cloudflare KV's 60s minimum TTL leaves no safe margin | ||
| to cache it without risking handing out an already-expired URL. | ||
| - `GITHUB_TOKEN` is required for GitHub API access (shared with | ||
| `versions/v1`). | ||
| - `DOWNLOAD_BUCKET` (R2 binding) backs `/main` - see `wrangler.jsonc` for the | ||
| bucket this points at and why. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| import { GithubLookupResult } from "./github/artifacts"; | ||
|
|
||
| // Default for anything that moves (main, pr/{number}) - previews churn | ||
| // often (a new commit on a PR supersedes the last build within minutes), | ||
| // so a short TTL keeps CACHE_KV useful without serving meaningfully stale | ||
| // data - matches download-worker's existing choice for the same trade-off. | ||
| // Callers addressing something immutable (a fixed commit/artifact) pass a | ||
| // longer ttlSeconds explicitly - see routes/commit.ts and routes/respond.ts. | ||
| export const DEFAULT_CACHE_TTL_SECONDS = 60; | ||
|
|
||
| // Cloudflare KV's own floor - a shorter expirationTtl is a 400 at the API | ||
| // level, not just an app-level policy choice. | ||
| const KV_MIN_TTL_SECONDS = 60; | ||
|
|
||
| // Only "found" results are cached. "not_found"/"unavailable" always | ||
| // re-resolve, so a transient GitHub hiccup or a not-yet-built PR doesn't | ||
| // get stuck negative for the TTL window. | ||
| // | ||
| // ttlSeconds may be a function of the resolved data instead of a fixed | ||
| // number - see routes/commit.ts, which caps the cache lifetime at the | ||
| // artifact's own remaining GitHub retention so a lookup resolved just | ||
| // before expiry doesn't outlive it and keep serving a 200 after GitHub | ||
| // itself has started 404ing. If the computed TTL is under KV's 60s floor, | ||
| // the result is returned but not cached at all - better to re-resolve | ||
| // live for the rest of that final minute than to either violate the floor | ||
| // or round up and cache something past its real expiry. | ||
| export async function cachedLookup<T>( | ||
| kv: KVNamespace, | ||
| key: string, | ||
| resolve: () => Promise<GithubLookupResult<T>>, | ||
| ttlSeconds: number | ((data: T) => number) = DEFAULT_CACHE_TTL_SECONDS | ||
| ): Promise<GithubLookupResult<T>> { | ||
| const cached = await kv.get(key); | ||
| if (cached !== null) { | ||
| try { | ||
| return { status: "found", data: JSON.parse(cached) as T }; | ||
| } catch { | ||
| // Corrupt cache entry - fall through to a fresh resolve. | ||
| } | ||
| } | ||
|
|
||
| const result = await resolve(); | ||
| if (result.status === "found") { | ||
| const ttl = | ||
| typeof ttlSeconds === "function" ? ttlSeconds(result.data) : ttlSeconds; | ||
| if (ttl >= KV_MIN_TTL_SECONDS) { | ||
| await kv.put(key, JSON.stringify(result.data), { | ||
| expirationTtl: ttl | ||
|
admdly marked this conversation as resolved.
|
||
| }); | ||
| } | ||
| } | ||
| return result; | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.