diff --git a/.github/workflows/publish-kustomize-bundle.yaml b/.github/workflows/publish-kustomize-bundle.yaml index 9b026f5..349ae09 100644 --- a/.github/workflows/publish-kustomize-bundle.yaml +++ b/.github/workflows/publish-kustomize-bundle.yaml @@ -1,3 +1,7 @@ +# Single-image, dedicated-job publisher. For stamping MULTIPLE images (each with +# its own build-step tag) into one bundle from within the caller's job, see the +# composite action `publish-kustomize-bundle/action.yml` +# (`uses: datum-cloud/actions/publish-kustomize-bundle@`). name: Publish Kustomize Bundle on: diff --git a/docs/README.md b/docs/README.md index ad74ef9..2a633f2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -12,7 +12,8 @@ available for use across the organization. ### Publishing & Distribution - [**Publish Docker Images**](./publish-docker/) - Build and push Docker images to GitHub Container Registry -- [**Publish Kustomize Bundle**](./publish-kustomize-bundle/) - Build and push Kustomize bundles to GitHub Container Registry +- [**Publish Kustomize Bundle**](./publish-kustomize-bundle/) - Build and push Kustomize bundles to GitHub Container Registry (reusable workflow, single-image) +- [**Publish Kustomize Bundle (composite)**](../publish-kustomize-bundle/) - Stamp multiple images into one bundle and push it, from within the caller's job - [**Update Plugin Index**](./update-plugin-index/) - Open a PR against a datumctl plugin catalog to bump a plugin manifest to a new release, refreshing version, URLs, and checksums ### Validation & Linting diff --git a/publish-kustomize-bundle/README.md b/publish-kustomize-bundle/README.md new file mode 100644 index 0000000..3958b07 --- /dev/null +++ b/publish-kustomize-bundle/README.md @@ -0,0 +1,105 @@ +# Publish Kustomize Bundle (composite action) + +Stamp **one or more** image references into a Kustomize bundle and push it to +GitHub Container Registry with Flux — as a **step inside the caller's job**. + +This is the multi-image, in-job alternative to the reusable workflow +[`.github/workflows/publish-kustomize-bundle.yaml`](../.github/workflows/publish-kustomize-bundle.yaml). +Because it runs in the caller's job, it reuses the caller's existing checkout and +can read outputs from earlier build-and-push steps (one tag per image), then +stamp them all into a single bundle before pushing. + +## Usage + +```yaml +jobs: + publish: + runs-on: ubuntu-latest + permissions: + contents: read + packages: write + steps: + - uses: actions/checkout@v4 + + - name: Log in to GitHub Container Registry + uses: docker/login-action@v4.2.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - id: provider # build + push the provider image → outputs.tag + uses: datum-cloud/actions/.github/workflows/publish-docker.yaml@v1 + # ... + - id: runtime # build + push the runtime image → outputs.tag + uses: datum-cloud/actions/.github/workflows/publish-docker.yaml@v1 + # ... + + - uses: datum-cloud/actions/publish-kustomize-bundle@v1.20.0 + with: + bundle-name: ghcr.io/datum-cloud/unikraft-provider-kustomize + bundle-path: config + images: | + - { path: config/default, name: ghcr.io/datum-cloud/unikraft-provider, tag: "${{ steps.provider.outputs.tag }}" } + - { path: config/dependencies/ukp-runtime, name: ghcr.io/datum-cloud/ukp-runtime, tag: "${{ steps.runtime.outputs.tag }}" } +``` + +> Reference a tagged version to avoid unexpected breaking changes. Use a floating +> major (`@v1`) to track the latest compatible release, or pin an exact tag +> (`@v1.20.0`) for full reproducibility. + +## Inputs + +| Input | Required | Default | Description | +| ------------- | -------- | ------- | ----------- | +| `bundle-name` | yes | — | Full bundle name including registry and repository (e.g. `ghcr.io/datum-cloud/unikraft-provider-kustomize`). | +| `bundle-path` | yes | — | Path to the bundle to push, relative to the repo root (e.g. `config`). | +| `images` | yes | — | Multi-line YAML list of `{ path, name, tag }` entries to stamp before pushing (see below). | +| `debug` | no | `false` | Print the stamped `images:` blocks and bundle structure before pushing. | + +### The `images` list + +Each entry stamps one image into one kustomization directory: + +- **`path`** — kustomization directory (relative to the repo root) to run + `kustomize edit set image` in. Must contain a `kustomization.yaml`. +- **`name`** — image reference to rewrite (the `name=newName` target). +- **`tag`** — *optional*. When provided it is used verbatim. When omitted the + entry falls back to the **bundle's first computed tag** (the same tag logic the + reusable workflow uses), so images without their own build step still get a + sensible version. + +The bundle itself is stamped **once** and then pushed under **every** computed +tag (PR/branch/semver/sha variants), matching the reusable workflow's push loop. + +## Outputs + +| Output | Description | +| -------- | ----------- | +| `tags` | Newline-separated tags computed for the bundle. | +| `labels` | Labels computed for the bundle. | +| `digest` | Digest of the pushed bundle (from the final push). | + +## Prerequisites + +Composite actions run in the caller's job and **cannot** take a `secrets:` block +or set job `permissions:`. The caller's job must therefore provide, before this +step: + +1. `actions/checkout` — the bundle and kustomization files are read from the + working tree. +2. `docker/login-action` against `ghcr.io` — `flux push` authenticates using the + caller's Docker credentials; there is no separate login here. +3. Job `permissions:` including `packages: write` and `contents: read`. +4. A runner with `kustomize`, `yq`, and `jq` available (GitHub-hosted Ubuntu + runners have all three). Flux is installed by this action. + +## Relationship to the reusable workflow + +The reusable workflow +[`publish-kustomize-bundle.yaml`](../.github/workflows/publish-kustomize-bundle.yaml) +still exists and is unchanged. Use it when a **single** image (or none) needs +stamping in a **dedicated job**; use this composite action when several images — +each with its own build-step tag — must be stamped into **one** bundle from +**within** the caller's job. Whether to keep both long-term or deprecate the +reusable workflow is an open question for review. diff --git a/publish-kustomize-bundle/action.yml b/publish-kustomize-bundle/action.yml new file mode 100644 index 0000000..85936de --- /dev/null +++ b/publish-kustomize-bundle/action.yml @@ -0,0 +1,173 @@ +name: Publish Kustomize Bundle +description: >- + Stamp one or more image references into a Kustomize bundle and push it to an + OCI registry (ghcr.io) with Flux. Runs inside the caller's job so it reuses + the caller's checkout and prior build-step outputs. + +# NOTE: This is a composite action, not a reusable workflow. It runs as a step +# inside the caller's existing job, so it can consume outputs from earlier +# build-and-push steps (e.g. per-image tags) and stamp several images into one +# bundle. The reusable workflow `.github/workflows/publish-kustomize-bundle.yaml` +# remains available for the single-image, dedicated-job use case. +# +# Prerequisites the caller's job MUST satisfy (a composite action cannot): +# - `actions/checkout` has already run. +# - `docker/login-action` has logged in to the target registry (ghcr.io) so +# the `flux push` below authenticates via the caller's Docker credentials. +# - Job `permissions:` include `packages: write` and `contents: read`. +# - The runner provides `kustomize`, `yq`, and `jq` (GitHub-hosted Ubuntu +# runners do). Flux is installed by this action. + +inputs: + bundle-name: + required: true + description: >- + Full name of the bundle including the registry and repository + (e.g. `ghcr.io/datum-cloud/unikraft-provider-kustomize`). + bundle-path: + required: true + description: >- + Path to the bundle to push, relative to the repository root + (e.g. `config`). + images: + required: true + description: >- + Multi-line YAML list of images to stamp before pushing. Each entry is a + `{ path, name, tag }` mapping: `path` is a kustomization directory + (relative to the repo root) to run `kustomize edit set image` in, `name` + is the image to rewrite, and `tag` is optional — when omitted the entry + falls back to the bundle's first computed tag. Example: + images: | + - { path: config/default, name: ghcr.io/datum-cloud/unikraft-provider, tag: "v0.0.0-pr-1-abc" } + - { path: config/dependencies/ukp-runtime, name: ghcr.io/datum-cloud/ukp-runtime } + debug: + required: false + default: "false" + description: "Print the stamped `images:` blocks and bundle structure before pushing." + +outputs: + tags: + description: "Newline-separated tags computed for the bundle." + value: ${{ steps.meta.outputs.tags }} + labels: + description: "Labels computed for the bundle." + value: ${{ steps.meta.outputs.labels }} + digest: + description: "Digest of the pushed bundle (from the final push)." + value: ${{ steps.publish.outputs.digest }} + +runs: + using: composite + steps: + - name: Setup Flux CLI + uses: fluxcd/flux2/action@v2.7.5 + + - name: Extract bundle metadata + id: meta + uses: docker/metadata-action@v6 + with: + images: ${{ inputs.bundle-name }} + tags: | + type=ref,event=pr,suffix=-{{commit_date 'YYYYMMDD-HHmmss'}},prefix=v0.0.0- + type=ref,event=pr,prefix=v0.0.0- + type=ref,event=branch,suffix=-{{commit_date 'YYYYMMDD-HHmmss'}},prefix=v0.0.0- + type=ref,event=branch,prefix=v0.0.0- + type=semver,pattern=v{{version}} + type=semver,pattern=v{{major}}.{{minor}} + type=semver,pattern=v{{major}} + type=sha,prefix=v0.0.0- + + - name: Stamp image tags into the bundle + shell: bash + env: + IMAGES: ${{ inputs.images }} + BUNDLE_TAGS: ${{ steps.meta.outputs.tags }} + BUNDLE_PATH: ${{ inputs.bundle-path }} + DEBUG: ${{ inputs.debug }} + run: | + set -euo pipefail + + # Default tag for entries that don't pin their own: the version portion + # of the bundle's first computed tag (mirrors the reusable workflow). + DEFAULT_TAG="$(printf '%s\n' "$BUNDLE_TAGS" | head -n1 | cut -d':' -f2)" + echo "Default image tag (bundle first tag): ${DEFAULT_TAG}" + + STAMPED_PATHS=() + # yq turns the YAML list into one compact-JSON object per line; jq reads + # each field. `.tag // ""` yields "" when the entry omits a tag. + while IFS= read -r entry; do + [ -n "$entry" ] || continue + + path="$(printf '%s' "$entry" | jq -r '.path')" + name="$(printf '%s' "$entry" | jq -r '.name')" + tag="$(printf '%s' "$entry" | jq -r '.tag // ""')" + [ -n "$tag" ] || tag="$DEFAULT_TAG" + + full_path="${GITHUB_WORKSPACE}/${path}" + if [ ! -f "${full_path}/kustomization.yaml" ]; then + echo "::error::${full_path}/kustomization.yaml not found" + exit 1 + fi + + echo "Setting image ${name}:${tag} in ${path}" + ( cd "${full_path}" && kustomize edit set image "${name}=${name}:${tag}" ) + STAMPED_PATHS+=("${full_path}") + done < <(printf '%s\n' "$IMAGES" | yq -o=json -I=0 '.[]') + + if [ "${#STAMPED_PATHS[@]}" -eq 0 ]; then + echo "::error::no images were parsed from the 'images' input" + exit 1 + fi + + # Ensure the edits are flushed before the push step reads them. + sync + + if [ "${DEBUG}" = "true" ]; then + echo "=== Stamped images: blocks ===" + for full_path in "${STAMPED_PATHS[@]}"; do + echo "--- ${full_path}/kustomization.yaml ---" + yq '.images' "${full_path}/kustomization.yaml" + done + echo "=== Bundle structure at ${BUNDLE_PATH} ===" + find "${GITHUB_WORKSPACE}/${BUNDLE_PATH}" -type f | sort + fi + + - name: Publish bundle + id: publish + shell: bash + env: + BUNDLE_TAGS: ${{ steps.meta.outputs.tags }} + BUNDLE_PATH: ${{ inputs.bundle-path }} + run: | + set -euo pipefail + + FULL_BUNDLE_PATH="${GITHUB_WORKSPACE}/${BUNDLE_PATH}" + echo "Publishing bundle from: ${FULL_BUNDLE_PATH}" + + DIGEST="" + PUSH_COUNT=0 + # Push the (already stamped) bundle once per computed tag, mirroring the + # reusable workflow. A small delay avoids ghcr rate limiting on bursts. + while IFS= read -r tag; do + [ -n "$tag" ] || continue + PUSH_COUNT=$((PUSH_COUNT + 1)) + if [ "${PUSH_COUNT}" -gt 1 ]; then + sleep 3 + fi + + tag_version="${tag#*:}" + echo "Push #${PUSH_COUNT}: oci://${tag}" + + RESULT="$(flux push artifact \ + "oci://${tag}" \ + --path="${FULL_BUNDLE_PATH}" \ + --source="https://github.com/${GITHUB_REPOSITORY}" \ + --revision="${tag_version}@sha1:$(git -C "${GITHUB_WORKSPACE}" rev-parse HEAD)" \ + --output=json)" + + DIGEST="$(printf '%s' "$RESULT" | jq -r '.digest')" + echo " digest: ${DIGEST}" + done < <(printf '%s\n' "$BUNDLE_TAGS") + + echo "Total pushes: ${PUSH_COUNT}" + echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"