Skip to content

fix: use distro-based images for credential plugin integration#191

Open
porridge wants to merge 5 commits into
masterfrom
distro
Open

fix: use distro-based images for credential plugin integration#191
porridge wants to merge 5 commits into
masterfrom
distro

Conversation

@porridge
Copy link
Copy Markdown
Collaborator

@porridge porridge commented May 6, 2026

Why

  • Turns out that feat: initial image cred plugin integration #184 never actually worked in practice, since the FROM scratch image does not contain libraries that the GKE credential plugin needs: plugin execution failed: fork/exec /tmp/credential-provider-bin/auth-provider-gcp: no such file or directory, stderr:
  • Looks like the testing done for the above PR in fix(ci): work around preloaded image use problem in recent k8s stackrox#19287 where the gke-latest-... job succeeded must have been some kind of fluke 🤷🏻 In fact that problematic test case didn't seem to run at all:
    15:10:32 | INFO  | ImageScanningTest         | Starting testcase: Image metadata from registry test - quay-auto
    15:10:32 | INFO  | Helpers                   | Won't collect logs for: org.opentest4j.TestAbortedException: Ignored via @IgnoreIf
    15:10:32 | INFO  | ImageScanningTest         | Post test cleanup:
    15:10:32 | INFO  | ImageScanningTest         | Ending testcase
    

What this does

  • This PR adds a parallel set of images differing only in base image, they are about twice the size of the FROM scratch ones. It also changes the deploy tool to use them transparently when credential plugin integration is enabled.
    image

  • Additionally, it changes the auth data passing to resemble the way kubelet does it more - in particular it drops the server endpoint piece which somehow prevents CRI from fetching the image:

    msg="image failed to pull" image=us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12 authNum=3
    error="rpc error: code = Unknown desc = failed to pull and unpack image \"us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12\": failed to resolve reference \"us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12\": failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://us.gcr.io/v2/token?scope=repository%3Aacs-san-stackroxci%2Fqa-multi-arch%3Apull&scope=repository%3Aacs-san-stackroxci%2Fus.gcr.io%2Fqa-multi-arch%3Apull&service=us.gcr.io: 403 Forbidden"
    

Testing

time=2026-05-06T13:11:41.995Z level=INFO source=/home/runner/work/image-prefetcher/image-prefetcher/internal/main.go:187 msg="image pulled successfully" image=us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12 authNum=0 authServer="" authUsername=_token response="image_ref:\"sha256:4037a5562b030fd80ec889bb885405587a52cfef898ffb7402649005dfda75ff\"" elapsed=28.646953919s
time=2026-05-06T13:11:50.222Z level=INFO source=/home/runner/work/image-prefetcher/image-prefetcher/internal/main.go:187 msg="image pulled successfully" image=us.gcr.io/acs-san-stackroxci/qa/registry-image:0.3 authNum=1 authServer="" authUsername=_token response="image_ref:\"sha256:e03ee8c409b34496c09c261194dd3d0d825f0a67350d49c8812d7dd65a95dfdc\"" elapsed=36.874397134s

@porridge porridge changed the title Distro fix: use distro-based images for credential plugin integration May 6, 2026
porridge and others added 5 commits May 11, 2026 06:59
The credential provider binary needs a real OS (not scratch), so
select the -distro image variant automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-Off-By: Marcin Owsiany <porridge@redhat.com>
docker manifest push "${tag}"
done

# Distro-full "FROM debian:stable-slim" images.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why not ubi?

Comment thread deploy/main.go
Comment on lines +60 to +61
// If distro is true, appends "-distro" to the result.
func processVersion(version string, distro bool) string {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// If distro is true, appends "-distro" to the result.
func processVersion(version string, distro bool) string {
// If useDistroImage is true, appends "-distro" to the result.
func processVersion(version string, useDistroImage bool) string {

nit: thought that distro is a bit cryptic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants