test(e2e): run compute e2e on the shared test-infra federation foundation - #181
Draft
scotwells wants to merge 5 commits into
Draft
test(e2e): run compute e2e on the shared test-infra federation foundation#181scotwells wants to merge 5 commits into
scotwells wants to merge 5 commits into
Conversation
Replace compute's bespoke Kind + Karmada standup with a remote Taskfile include of the shared test-infra federation foundation, so a laptop and CI run the identical path. The foundation now owns creating the three Kind clusters, installing and tuning Karmada, and joining the POP cells; compute keeps only its service-specific layer (Karmada federation config, CRD installs, the management/cell federation credentials, image build, and the deploy overlays). Drop the absorbed plumbing: the karmadactl/helm-repo tool installs, the cluster-create and kubeconfig-export tasks, the Karmada install/tune/join tasks, and the four hack/e2e config and kubeconfig-rewrite scripts. Read the federation kubeconfigs the foundation writes under .test-infra/kubeconfigs/federation, and source the cell operators' hub address from the foundation-provided karmada-internal.yaml instead of re-deriving the hub container IP. Gitignore the foundation self-clone and Task's remote cache.
Mechanical path substitution only — the shared foundation writes the cluster kubeconfigs under .test-infra/kubeconfigs/federation with new names, so the hardcoded --kubeconfig paths inside the Chainsaw suite steps move: tmp/e2e/kubeconfigs/control-plane.yaml -> .test-infra/kubeconfigs/federation/compute-control-plane.yaml tmp/e2e/kubeconfigs/downstream.yaml -> .test-infra/kubeconfigs/federation/karmada.yaml tmp/e2e/kubeconfigs/pop-dfw.yaml -> .test-infra/kubeconfigs/federation/compute-pop-dfw.yaml tmp/e2e/kubeconfigs/pop-ord.yaml -> .test-infra/kubeconfigs/federation/compute-pop-ord.yaml No test assertions, workload specs, or step logic change; verify with `git diff --word-diff`. The relative depth (../../../ from test/e2e/<suite>/) is unchanged because .test-infra sits at the repo root just like tmp did.
…onfigs Update the chainsaw configuration's named clusters (downstream, pop-dfw, pop-ord) to the foundation-written kubeconfigs under .test-infra/kubeconfigs/federation, and refresh the referenced-data README to describe the new layout: the foundation writes the hub kubeconfig as karmada.yaml and there is no longer a downstream.yaml copy or a build-kubeconfig task.
Drive the e2e job through the remote federation include: opt into Task's remote-taskfile support at the job level and add --yes to each task step for the one-time trust prompt. Drop the manual kind install (the foundation's ensure-tools provides it) and the PyYAML shim (no harness python parses YAML any more). Point diagnostics collection at the foundation kubeconfig directory and cluster filenames.
A remote Taskfile include's root var clobbers a same-named var in the
consumer Taskfile. Compute and the test-infra foundation both defined
LOCALBIN, so compute's {{.ROOT_DIR}}/bin was overridden by the foundation's
REPO_DIR-based definition; REPO_DIR is undefined in the compute scope, so the
value rendered as a literal "%!s(<nil>)/bin" — chainsaw would have installed
to a bogus directory and e2e:test could not have found it. Rename the var to
E2E_BIN, which does not collide. KARMADA_API_NODEPORT also shares a name with
a foundation var, but it is passed through the include by design and resolves
to the same value (override included), so it is unaffected.
This was referenced Jul 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Compute's end-to-end tests now run on the shared test-infra federation foundation (datum-cloud/test-infra#32) instead of a bespoke, compute-only Kind + Karmada harness. Standing up three Kind clusters, installing and tuning Karmada, and joining the POP cells is no longer compute's code to maintain — it's a reusable foundation any Datum service can consume the same way. Compute keeps only what is genuinely service-specific: its Karmada federation config, CRD installs, the management/cell federation credentials, the operator image build, and the deploy overlays. Local
task e2e:upon a laptop and the CI job run the identical path, because both pull the foundation in as a remote Taskfile include.This builds directly on the in-cluster e2e harness from #179 (the deployability work for #149): the operators, overlays, federation credentials, and all nine Chainsaw suites are unchanged — only the cluster/Karmada provisioning underneath them moves to the shared foundation.
Net result: ~620 lines of Karmada/Kind plumbing and four hack/e2e scripts deleted from this repo, the same nine Chainsaw suites still exercising the full federated topology.
What moved where
includeofdatum-cloud/test-infra(pinned viaTEST_INFRA_REF) and delegates cluster/Karmada lifecycle to it.e2e:env:upbecomese2e:tools→infra:federation-up→e2e:karmada:configure→e2e:crds:install;e2e:down→infra:federation-down;e2e:image:load→infra:federation-load-image. The cell-credential step reads the hub's docker-reachable address from the foundation-providedkarmada-internal.yamlrather than re-deriving the hub container IP. Deleted: the karmadactl/helm-repo tool installs, cluster-create/kubeconfig-export, and the Karmada install/tune/join tasks..test-infra/kubeconfigs/federation/ascompute-control-plane.yaml,karmada.yaml,compute-pop-dfw.yaml,compute-pop-ord.yaml(replacingtmp/e2e/kubeconfigs/*and thedownstream.yamlcopy). The chainsaw config's named clusters and the referenced-data README are repointed accordingly. The hardcoded--kubeconfigpaths inside the Chainsaw suite steps are a path-substitution-only change isolated in its own commit (test(e2e): repoint suite kubeconfig paths…) — no assertion, workload spec, or step logic changes; verify withgit diff --word-diff. The relative depth (../../../fromtest/e2e/<suite>/) is unchanged because.test-infra/sits at the repo root just liketmp/did.TASK_X_REMOTE_TASKFILES=1at job level, runstask --yes …, drops the manual kind install (the foundation's ensure-tools provides it) and the PyYAML shim (no harness Python parses YAML any more). Diagnostics repointed at the foundation kubeconfig directory/filenames.Deviations inherited from the foundation
These are intentional consequences of moving onto the shared foundation, each validated by the compute e2e CI acceptance gate:
kindest/node:v1.32.0(foundation'sK8S_VERSION)>= v0.30.0)sedsubstitution of the NodePortenvsubstof${KARMADA_API_NODEPORT}helm repo addthen installhelm install --repo …kubectl config(unset-CA-then-set)Var-naming note: the e2e tool-bin var is
E2E_BIN, not the conventionalLOCALBIN. A remote Taskfile include's root var clobbers a same-named var in the consumer, and the foundation already definesLOCALBIN— sharing the name rendered compute's value as a nil path.KARMADA_API_NODEPORTis also shared, but it is passed through the include by design and resolves to the same value (local override included, verified 32443→32643).Verification
Full e2e CI is green on this branch with the foundation pinned to
feat/federation-topology: E2E Tests run 29112915347 — foundation-provisioned Kind + Karmada environment, operator image build and deploy, and all nine Chainsaw suites pass. This run doubles as the cross-repo validation for datum-cloud/test-infra#32.Pinning chain — DO NOT MERGE until re-pinned
TEST_INFRA_REFis currently pinned to the test-infra branchfeat/federation-topologyso CI can exercise the foundation before it merges. This must be flipped to the tagged foundation release before this PR merges:v0.7.0.TEST_INFRA_REFtov0.7.0here (one var moves both the include URL and the foundation's self-clone ref), re-run compute CI green.Stacked on #179
Base branch is
feat/e2e-in-cluster(PR #179, which closes #149). This PR is stacked on it and should be reviewed/merged after #179 lands.