diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..cf74daa --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,138 @@ +# HyperFleet E2E — Agent Instructions + +Black-box E2E testing framework for HyperFleet cluster lifecycle management. Tests hit the HyperFleet API, create ephemeral clusters, verify adapter execution and K8s resource creation, then clean up. Built with Go 1.25, Ginkgo v2, Gomega, and an OpenAPI-generated client. + +Test suites: `e2e/cluster/`, `e2e/nodepool/`, `e2e/adapter/`. + +## Verification + +Run `make check` before declaring work done. It runs everything in order: + +| Target | What it does | +|---|---| +| `make check` | `generate` → `fmt-check` → `vet` → `lint` → `test` (all-in-one) | +| `make build` | `generate` → compile binary to `bin/hyperfleet-e2e` | +| `make fmt` | Format code and imports (`golangci-lint fmt`) | +| `make test` | Unit tests only (`./pkg/...`) | +| `make lint` | `golangci-lint` (config: `.golangci.yml`) | +| `make generate` | Regenerate OpenAPI client from spec | + +Pre-flight order: `make check` then `make build`. + +## Source of Truth + +| Topic | Location | +|---|---| +| Getting started | `docs/getting-started.md` | +| Architecture | `docs/architecture.md` | +| Test writing guide | `docs/development.md` | +| Debugging | `docs/debugging.md` | +| Local kind setup | `docs/local-kind-setup.md` | +| Runbook | `docs/runbook.md` | +| Contributing | `CONTRIBUTING.md` | +| Test case templates | `test-design/templates/` | +| Test case documents | `test-design/testcases/` | +| User journey maps | `test-design/user-journeys/` | +| Config defaults | `pkg/config/defaults.go` | +| Config struct & validation | `pkg/config/config.go` | +| Pollers | `pkg/helper/pollers.go` | +| Custom matchers | `pkg/helper/matchers.go` | +| Helper core (New, TestDataPath, CleanupTestCluster) | `pkg/helper/helper.go` + `pkg/helper/suite.go` | +| Synchronous validators (HasResourceCondition, AdapterNameToConditionType) | `pkg/helper/validation.go` | +| Payload template vars | `pkg/client/payload.go` (`templateVars` struct) | +| Labels | `pkg/labels/labels.go` | +| Condition type constants | `pkg/client/constants.go` | +| Config file | `configs/config.yaml` | + +## Test Conventions + +### File naming and structure + +- **IMPORTANT:** Test files use `.go` extension, NOT `_test.go`. E2E tests are compiled into the binary, not run via `go test`. +- Location: `e2e/{suite}/descriptive-name.go` (package matches directory name) +- Test name format: `[Suite: component][category] Description` (e.g., `[Suite: cluster][baseline] Cluster Resource Type Lifecycle`). Known categories: `baseline`, `update`, `delete`, `concurrent`, `negative`. +- Test suites auto-register via blank import in `e2e/e2e.go` + +### Labels + +Every test MUST have exactly one severity label from `pkg/labels`: + +- `labels.Tier0` — critical path, blocks release +- `labels.Tier1` — important features +- `labels.Tier2` — edge cases, can defer + +Optional: `labels.Negative`, `labels.Performance`, `labels.Upgrade`, `labels.Disruptive`, `labels.Slow` + +### Async operations — pollers + custom matchers + +**IMPORTANT:** Use pollers with custom matchers. Do NOT create `WaitFor*` wrapper functions that hide `Eventually` inside helpers. + +```go +// Wait for resource condition +Eventually(h.PollCluster(ctx, clusterID), h.Cfg.Timeouts.Cluster.Reconciled, h.Cfg.Polling.Interval). + Should(helper.HaveResourceCondition(client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue)) + +// Wait for all adapters at generation +Eventually(h.PollClusterAdapterStatuses(ctx, clusterID), h.Cfg.Timeouts.Adapter.Processing, h.Cfg.Polling.Interval). + Should(helper.HaveAllAdaptersAtGeneration(h.Cfg.Adapters.Cluster, expectedGen)) + +// Wait for hard-delete (404) +Eventually(h.PollClusterHTTPStatus(ctx, clusterID), timeout, h.Cfg.Polling.Interval). + Should(Equal(http.StatusNotFound)) + +// Wait for namespace cleanup +Eventually(h.PollNamespacesByPrefix(ctx, clusterID), timeout, h.Cfg.Polling.Interval). + Should(BeEmpty()) +``` + +Available pollers: `PollCluster`, `PollNodePool`, `PollClusterAdapterStatuses`, `PollNodePoolAdapterStatuses`, `PollClusterHTTPStatus`, `PollNodePoolHTTPStatus`, `PollNamespacesByPrefix`. + +Available matchers: `HaveResourceCondition`, `HaveAllAdaptersWithCondition`, `HaveAllAdaptersAtGeneration`. + +For one-off complex assertions, use `Eventually(func(g Gomega) { ... }).Should(Succeed())` with `g.Expect()` (not bare `Expect()`). + +### Cleanup + +Every test MUST clean up resources with `ginkgo.DeferCleanup` inline right after resource creation. + +### Payload templates + +Resolve payload paths via `h.TestDataPath()` — never hardcode `testdata/` as a prefix (breaks when `TESTDATA_DIR` is overridden, e.g., in CI): + +```go +h.Client.CreateClusterFromPayload(ctx, h.TestDataPath("payloads/clusters/cluster-request.json")) +``` + +Payloads in `testdata/payloads/` support Go templates. Available variables (defined in `pkg/client/payload.go`): + +- `.Random` — 8-char random hex +- `.UUID` — full UUID v4 +- `.Timestamp` — Unix seconds +- `.TimestampMs` — Unix milliseconds + +### Step markers + +Use `ginkgo.By()` for major steps. **IMPORTANT:** Never use `ginkgo.By()` inside `Eventually` closures. + +### Timeouts and intervals + +Always use config values: `h.Cfg.Timeouts.Cluster.Reconciled`, `h.Cfg.Timeouts.NodePool.Reconciled`, `h.Cfg.Timeouts.Adapter.Processing`, `h.Cfg.Polling.Interval`. Never hardcode durations. + +## Boundaries + +### DON'T + +- Use `_test.go` suffix for E2E test files +- Hardcode timeout durations — use `h.Cfg.Timeouts.*` +- Skip cleanup (`DeferCleanup`) +- Use `ginkgo.By()` inside `Eventually` closures +- Import `e2e/*` packages from `pkg/` code + +## Gotchas + +- `Validate()` in `pkg/config/config.go` returns `error`, does not panic — only checks that `API.URL` is non-empty +- `helper.New()` calls `log.Fatalf` if config is nil — tests must call `SetSuiteConfig` before running +- Config priority: CLI flags > env vars (`HYPERFLEET_*` prefix) > `configs/config.yaml` > built-in defaults (see `pkg/config/defaults.go`) +- Config file path priority: `--config` flag > `HYPERFLEET_CONFIG` env > `./configs/config.yaml` auto-detect +- Adapter names come from `h.Cfg.Adapters.Cluster` and `h.Cfg.Adapters.NodePool` at runtime — never hardcode adapter names. Values in `configs/config.yaml` (e.g., `cl-namespace`) override compiled defaults in `pkg/config/defaults.go` (e.g., `clusters-namespace`) +- `e2e-ci` Makefile target sets `TESTDATA_DIR` to absolute path and writes JUnit XML to `output/` diff --git a/CLAUDE.md b/CLAUDE.md index 64045e2..43c994c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,336 +1 @@ -# HyperFleet E2E - AI Agent Instructions - -This file provides context for AI agents working with the HyperFleet E2E testing framework. - -## Project Overview - -Black-box E2E testing framework for HyperFleet cluster lifecycle management. Built with Go, Ginkgo, and OpenAPI-generated clients. Tests create ephemeral clusters for complete isolation. - -## Build and Test Commands - -### Build Binary - -```bash -make build -``` - -Binary output: `bin/hyperfleet-e2e` - -### Run All Checks - -```bash -make check -``` - -This runs: format check, vet, lint, and unit tests. - -### Individual Quality Checks - -```bash -make fmt # Format code -make fmt-check # Verify formatting -make vet # Run go vet -make lint # Run golangci-lint -make test # Run unit tests -``` - -### Generate API Client - -Required after OpenAPI schema updates: - -```bash -make generate -``` - -Extracts schema from `hyperfleet-api-spec` Go module (pinned via `hack/tools.go`) and regenerates `pkg/api/openapi/`. - -### Run E2E Tests - -```bash -export HYPERFLEET_API_URL=https://api.hyperfleet.example.com -make build -./bin/hyperfleet-e2e test --label-filter=tier0 -``` - -### Clean Build Artifacts - -```bash -make clean -``` - -## Validation Checklist - -Before submitting changes, verify: - -1. **Format**: `make fmt` -2. **Generate**: `make generate` (if OpenAPI schema or config changed) -3. **Lint**: `make lint` (must pass with zero errors) -4. **Vet**: `make vet` (must pass) -5. **Unit Tests**: `make test` (all tests must pass) -6. **Build**: `make build` (binary must compile) -7. **E2E Tests**: Optional, but recommended for test changes - -## Code Conventions - -### Test Files - -- **Extension**: Use `.go` NOT `_test.go` -- **Location**: `e2e/{resource-type}/descriptive-name.go` -- **Package**: Match directory name (e.g., package `cluster` for `e2e/cluster/`) -- **Test Name**: Format as `[Suite: component] Description` (e.g., `[Suite: cluster] Create Cluster via API`) - -Example: -```go -package cluster - -var testName = "[Suite: cluster] Create Cluster via API" -``` - -### Labels - -Every test MUST have exactly one severity label: - -```go -import "github.com/openshift-hyperfleet/hyperfleet-e2e/pkg/labels" - -var _ = ginkgo.Describe(testName, - ginkgo.Label(labels.Tier0), // Critical severity - func() { ... } -) -``` - -Available labels: -- **Severity** (required): `Tier0`, `Tier1`, `Tier2` -- **Optional**: `Negative`, `Performance`, `Upgrade`, `Disruptive`, `Slow` - -### Test Structure - -Required structure: - -```go -var _ = ginkgo.Describe(testName, ginkgo.Label(...), func() { - var h *helper.Helper - var resourceID string - - ginkgo.BeforeEach(func() { - h = helper.New() - }) - - ginkgo.It("description", func(ctx context.Context) { - ginkgo.By("step description") - // test logic - }) - - ginkgo.AfterEach(func(ctx context.Context) { - if h == nil || resourceID == "" { - return - } - if err := h.CleanupTestCluster(ctx, resourceID); err != nil { - ginkgo.GinkgoWriter.Printf("Warning: cleanup failed: %v\n", err) - } - }) -}) -``` - -### Step Markers - -Use `ginkgo.By()` for major steps ONLY. Do NOT use inside `Eventually` closures: - -```go -// CORRECT -ginkgo.By("waiting for cluster to become Reconciled") -Eventually(h.PollCluster(ctx, clusterID), timeout, h.Cfg.Polling.Interval). - Should(helper.HaveResourceCondition(client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue)) - -// INCORRECT - never do this -Eventually(func() { - ginkgo.By("checking status") // ❌ Wrong - // ... -}).Should(Succeed()) -``` - -### Async Operations — Pollers + Custom Matchers - -Use **pollers** (thin functions returning current state) with **custom matchers** (reusable assertions). This keeps `Eventually` visible at the call site and avoids combinatorial helper function explosion. - -**Wait for a resource condition** (cluster or nodepool): -```go -Eventually(h.PollCluster(ctx, clusterID), h.Cfg.Timeouts.Cluster.Reconciled, h.Cfg.Polling.Interval). - Should(helper.HaveResourceCondition(client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue)) - -Eventually(h.PollNodePool(ctx, clusterID, npID), h.Cfg.Timeouts.NodePool.Reconciled, h.Cfg.Polling.Interval). - Should(helper.HaveResourceCondition(client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue)) -``` - -**Wait for adapter conditions** (works for both cluster and nodepool adapters): -```go -Eventually(h.PollClusterAdapterStatuses(ctx, clusterID), timeout, h.Cfg.Polling.Interval). - Should(helper.HaveAllAdaptersWithCondition(h.Cfg.Adapters.Cluster, client.ConditionTypeFinalized, openapi.AdapterConditionStatusTrue)) - -Eventually(h.PollNodePoolAdapterStatuses(ctx, clusterID, npID), timeout, h.Cfg.Polling.Interval). - Should(helper.HaveAllAdaptersAtGeneration(h.Cfg.Adapters.NodePool, expectedGen)) -``` - -**Wait for hard-delete** (resource returns 404): -```go -Eventually(h.PollClusterHTTPStatus(ctx, clusterID), timeout, h.Cfg.Polling.Interval). - Should(Equal(http.StatusNotFound)) -``` - -**Wait for namespace cleanup**: -```go -Eventually(h.PollNamespacesByPrefix(ctx, clusterID), timeout, h.Cfg.Polling.Interval). - Should(BeEmpty()) -``` - -**For one-off complex assertions**, use `Eventually` with `func(g Gomega)` and `g.Expect()` (not `Expect()`): -```go -Eventually(func(g Gomega) { - statuses, err := h.Client.GetClusterStatuses(ctx, clusterID) - g.Expect(err).NotTo(HaveOccurred()) - // complex multi-field validation... -}, timeout, h.Cfg.Polling.Interval).Should(Succeed()) -``` - -Available pollers: `PollCluster`, `PollNodePool`, `PollClusterAdapterStatuses`, `PollNodePoolAdapterStatuses`, `PollClusterHTTPStatus`, `PollNodePoolHTTPStatus`, `PollNamespacesByPrefix` — see `pkg/helper/pollers.go`. - -Available matchers: `HaveResourceCondition`, `HaveAllAdaptersWithCondition`, `HaveAllAdaptersAtGeneration` — see `pkg/helper/matchers.go`. - -**Do NOT** create `WaitFor*` wrapper functions that hide `Eventually` inside helpers. - -### Resource Cleanup - -ALWAYS implement cleanup in `AfterEach`: - -```go -ginkgo.AfterEach(func(ctx context.Context) { - if h == nil || clusterID == "" { - return - } - if err := h.CleanupTestCluster(ctx, clusterID); err != nil { - ginkgo.GinkgoWriter.Printf("Warning: failed to cleanup cluster %s: %v\n", clusterID, err) - } -}) -``` - -### Payload Templates - -Test payloads in `testdata/payloads/` support Go templates for dynamic values: - -```json -{ - "name": "cluster-{{.Random}}", - "labels": { - "created-at": "{{.Timestamp}}" - } -} -``` - -Available variables: `.Random`, `.Timestamp`. See `pkg/client/payload.go`. - -## Boundary Statements - -### DO NOT - -- **Modify generated code**: Never edit files in `pkg/api/openapi/` - these are generated by `make generate` -- **Use `_test.go` suffix**: Test files must use `.go` extension -- **Hardcode timeouts**: Use `h.Cfg.Timeouts.*` values from config -- **Skip cleanup**: Always implement `AfterEach` cleanup -- **Commit without checks**: Always run `make check` before committing -- **Use `ginkgo.By()` in `Eventually`**: Only use at top-level test steps -- **Import test packages**: Do NOT import `e2e/*` packages in production code -- **Edit OpenAPI schema**: Schema is maintained in hyperfleet-api-spec repo -- **Create `WaitFor*` wrapper functions**: Use pollers + custom matchers instead (see Async Operations) - -### DO - -- **Use pollers + matchers**: Prefer `Eventually(h.PollCluster(...)).Should(helper.HaveResourceCondition(...))` over raw `Eventually` with inline closures -- **Use config values**: `h.Cfg.Timeouts.*` for timeouts, `h.Cfg.Polling.*` for intervals -- **Store resource IDs**: Save IDs in variables for cleanup -- **Check errors**: Use `Expect(err).NotTo(HaveOccurred())` -- **Use context**: All test functions receive `context.Context` - -## Development Workflow - -### Adding a New Test - -1. Create file: `e2e/{suite}/descriptive-name.go` -2. Copy structure from existing test -3. Update test name, labels, and logic -4. Run validation checklist -5. Test locally before submitting PR - -### Updating API Client - -When hyperfleet-api-spec changes: - -```bash -# Bump spec module version -go get github.com/openshift-hyperfleet/hyperfleet-api-spec@vX.Y.Z - -# Regenerate client code -make generate - -# Verify changes compile -make build -``` - -### Local Testing - -```bash -# Build and run specific tests -make build -./bin/hyperfleet-e2e test --focus "\[Suite: cluster\]" - -# Run critical tests only -./bin/hyperfleet-e2e test --label-filter=tier0 - -# Debug mode -./bin/hyperfleet-e2e test --log-level=debug -``` - -## Configuration - -Priority (highest to lowest): -1. CLI flags: `--api-url`, `--log-level` -2. Environment variables: `HYPERFLEET_API_URL` -3. Config file: `configs/config.yaml` -4. Built-in defaults - -## Common Patterns - -### Create Resource - -```go -cluster, err := h.Client.CreateClusterFromPayload(ctx, "testdata/payloads/clusters/cluster-request.json") -Expect(err).NotTo(HaveOccurred()) -clusterID = *cluster.Id -``` - -### Wait for Condition - -```go -Eventually(h.PollCluster(ctx, clusterID), h.Cfg.Timeouts.Cluster.Reconciled, h.Cfg.Polling.Interval). - Should(helper.HaveResourceCondition(client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue)) -``` - -### Wait for All Adapters - -```go -Eventually(h.PollClusterAdapterStatuses(ctx, clusterID), h.Cfg.Timeouts.Adapter.Processing, h.Cfg.Polling.Interval). - Should(helper.HaveAllAdaptersAtGeneration(h.Cfg.Adapters.Cluster, expectedGen)) -``` - -### Verify Conditions (synchronous) - -```go -hasReconciled := h.HasResourceCondition(cluster.Status.Conditions, client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue) -Expect(hasReconciled).To(BeTrue()) -``` - -## Documentation - -- [Getting Started](docs/getting-started.md) - Run first test in 10 minutes -- [Architecture](docs/architecture.md) - Framework design -- [Development](docs/development.md) - Detailed test writing guide -- [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines +@AGENTS.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3b88632..630c333 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -54,6 +54,7 @@ hyperfleet-e2e/ │ ├── labels/ - Test label definitions │ └── logger/ - Structured logging (slog) ├── e2e/ - Test suites +│ ├── adapter/ - Adapter failover and Maestro tests │ ├── cluster/ - Cluster lifecycle tests │ └── nodepool/ - NodePool management tests ├── testdata/ - Test payloads and fixtures @@ -111,7 +112,7 @@ Key points: - Test files use `.go` extension (NOT `_test.go`) - All tests must have labels (Tier0, Tier1, or Tier2) - Use `ginkgo.By()` to mark major test steps -- Clean up resources in `AfterEach` +- Clean up resources with `DeferCleanup` - Use helper functions from `pkg/helper` ## Common Tasks @@ -121,11 +122,11 @@ Key points: When the HyperFleet API OpenAPI schema changes: ```bash -# Regenerate from main branch (default) -make generate +# 1. Bump the spec module version in go.mod +go get github.com/openshift-hyperfleet/hyperfleet-api-spec@vX.Y.Z -# Or from a specific branch/tag -OPENAPI_SPEC_REF=release-0.2 make generate +# 2. Regenerate client code from the new spec +make generate ``` ### Build and Test Workflow diff --git a/docs/architecture.md b/docs/architecture.md index fe79b3b..c80ca17 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -49,11 +49,9 @@ Test ends ```yaml api: url: https://api.hyperfleet.example.com -resources: - keep: false timeouts: cluster: - reconciled: 5m + reconciled: 5m # Default: 30m (see pkg/config/defaults.go) ``` ### pkg/config @@ -202,7 +200,7 @@ Built-in Defaults (lowest priority) - Suite timeout management **Key Functions**: -- `RunTests(cfg)` - Main entry point for test execution +- `RunTests(ctx)` - Main entry point for test execution - Configures Ginkgo reporters, timeouts, and filters - Handles suite-level setup and teardown @@ -228,7 +226,7 @@ Configuration values are applied in this priority order: ↓ ┌──────────────────────────────────────────────────────────────────┐ │ 4. Built-in Defaults (lowest priority) │ -│ Example: timeout: 30m, poll: 5s │ +│ Example: timeout: 30m, poll: 10s │ └──────────────────────────────────────────────────────────────────┘ ``` @@ -291,16 +289,11 @@ The framework uses code generation to maintain type-safe API clients. The generated client is wrapped by `pkg/client.HyperFleetClient` to provide: - Simplified error handling - Test-friendly method signatures -- Automatic retry logic (future) - Request/response logging **Example**: ```go -// Generated client (low-level) -apiClient := openapi.NewClient(...) -resp, httpResp, err := apiClient.ClustersAPI.GetCluster(ctx, clusterID).Execute() - -// Wrapped client (test-friendly) +// Wrapped client (test-friendly) — used in tests client, _ := client.NewHyperFleetClient(apiURL, nil) cluster, err := client.GetCluster(ctx, clusterID) ``` diff --git a/docs/development.md b/docs/development.md index db5609f..86e3ec5 100644 --- a/docs/development.md +++ b/docs/development.md @@ -81,7 +81,7 @@ import ( "github.com/openshift-hyperfleet/hyperfleet-e2e/pkg/labels" ) -var testName = "[Suite: cluster] Create Cluster via API" +var testName = "[Suite: cluster][baseline] Create Cluster via API" var _ = ginkgo.Describe(testName, ginkgo.Label(labels.Tier0), @@ -98,20 +98,16 @@ var _ = ginkgo.Describe(testName, cluster, err := h.Client.CreateClusterFromPayload(ctx, h.TestDataPath("payloads/clusters/cluster-request.json")) Expect(err).NotTo(HaveOccurred()) clusterID = *cluster.Id + ginkgo.DeferCleanup(func(ctx context.Context) { + if err := h.CleanupTestCluster(ctx, clusterID); err != nil { + ginkgo.GinkgoWriter.Printf("Warning: cleanup failed: %v\n", err) + } + }) ginkgo.By("waiting for cluster to become Reconciled") Eventually(h.PollCluster(ctx, clusterID), h.Cfg.Timeouts.Cluster.Reconciled, h.Cfg.Polling.Interval). Should(helper.HaveResourceCondition(client.ConditionTypeReconciled, openapi.ResourceConditionStatusTrue)) }) - - ginkgo.AfterEach(func(ctx context.Context) { - if h == nil || clusterID == "" { - return - } - if err := h.CleanupTestCluster(ctx, clusterID); err != nil { - ginkgo.GinkgoWriter.Printf("Warning: failed to cleanup cluster %s: %v\n", clusterID, err) - } - }) }, ) ``` @@ -121,11 +117,12 @@ var _ = ginkgo.Describe(testName, ### 1. Test Name ```go -var lifecycleTestName = "[Suite: cluster] Full Cluster Creation Flow" +var lifecycleTestName = "[Suite: cluster][baseline] Full Cluster Creation Flow" ``` -- Format: `[Suite: component] Description` -- Suite represents the HyperFleet component being tested (cluster, nodepool, api, adapter, etc.) +- Format: `[Suite: component][category] Description` +- Suite represents the HyperFleet component being tested (cluster, nodepool, adapter) +- Category describes the test type: `baseline`, `update`, `delete`, `concurrent`, `negative` - Use clear, descriptive names ### 2. Labels @@ -147,7 +144,7 @@ All tests must use labels for categorization. See `pkg/labels/labels.go` for com ```go import "github.com/openshift-hyperfleet/hyperfleet-e2e/pkg/labels" -var testName = "[Suite: cluster] Full Cluster Creation Flow" +var testName = "[Suite: cluster][baseline] Full Cluster Creation Flow" var _ = ginkgo.Describe(testName, ginkgo.Label(labels.Tier0), func() { ... } @@ -192,21 +189,23 @@ ginkgo.By("verifying adapter conditions") - Makes test output readable - **DO NOT** use `ginkgo.By()` inside `Eventually` closures -### 5. AfterEach Cleanup +### 5. Resource Cleanup + +Prefer `ginkgo.DeferCleanup` inline right after resource creation: ```go -ginkgo.AfterEach(func(ctx context.Context) { - if h == nil || clusterID == "" { - return - } +clusterID, err := h.GetTestCluster(ctx, h.TestDataPath("payloads/clusters/cluster-request.json")) +Expect(err).NotTo(HaveOccurred()) +ginkgo.DeferCleanup(func(ctx context.Context) { if err := h.CleanupTestCluster(ctx, clusterID); err != nil { - ginkgo.GinkgoWriter.Printf("Warning: failed to cleanup cluster %s: %v\n", clusterID, err) + ginkgo.GinkgoWriter.Printf("Warning: cleanup failed: %v\n", err) } }) ``` -- Clean up resources after test -- Skip cleanup if helper not initialized or no cluster created +- Register cleanup inline right after creating the resource +- `DeferCleanup` runs in LIFO order and is scoped to the current node +- Guard against empty IDs to avoid unnecessary cleanup calls - Log cleanup failures as warnings ## Writing Assertions @@ -297,8 +296,7 @@ Available pollers: see `pkg/helper/pollers.go`. Available matchers: see `pkg/hel - Don't use `_test.go` suffix (use `.go`) - Don't use `ginkgo.By()` inside `Eventually` closures - Don't hardcode timeouts (use config values) -- Don't skip cleanup (unless debugging) -- Don't ignore errors +- Don't skip cleanup - Don't create `WaitFor*` wrapper functions that hide `Eventually` — use pollers + matchers instead ## Adding New Tests @@ -330,6 +328,7 @@ Tests are automatically registered via the package import in `e2e/e2e.go`: package e2e import ( + _ "github.com/openshift-hyperfleet/hyperfleet-e2e/e2e/adapter" _ "github.com/openshift-hyperfleet/hyperfleet-e2e/e2e/cluster" _ "github.com/openshift-hyperfleet/hyperfleet-e2e/e2e/nodepool" ) @@ -356,7 +355,7 @@ make build ### Create Resource from Payload ```go -cluster, err := h.Client.CreateClusterFromPayload(ctx, "testdata/payloads/clusters/cluster-request.json") +cluster, err := h.Client.CreateClusterFromPayload(ctx, h.TestDataPath("payloads/clusters/cluster-request.json")) Expect(err).NotTo(HaveOccurred()) ``` @@ -428,6 +427,6 @@ SENTINEL_BROKER_TYPE=rabbitmq \ ## Next Steps - **Architecture**: Understand the framework design in [Architecture](architecture.md) -- **Configuration**: Customize behavior in [Configuration Reference](config.md) -- **Debug Tests**: Learn debugging techniques in [Troubleshooting Guide](troubleshooting.md) -- **CLI Reference**: Full command documentation in [CLI Reference](cli-reference.md) +- **Configuration**: See detailed comments in `configs/config.yaml` +- **Debug Tests**: Learn debugging techniques in [Debugging Guide](debugging.md) +- **Runbook**: Step-by-step operational guide in [Runbook](runbook.md) diff --git a/docs/getting-started.md b/docs/getting-started.md index 6eed3fd..606427a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -37,7 +37,7 @@ export HYPERFLEET_API_URL=https://api.hyperfleet.example.com **Step 2**: Run tests ```bash -./bin/hyperfleet-e2e test --label-filter=critical +./bin/hyperfleet-e2e test --label-filter=tier0 ``` **What happens**: @@ -59,20 +59,20 @@ The framework: ```bash # Run critical tests only -./bin/hyperfleet-e2e test --label-filter=critical +./bin/hyperfleet-e2e test --label-filter=tier0 # Run all cluster suite tests ./bin/hyperfleet-e2e test --focus "\[Suite: cluster\]" -# Run cluster lifecycle tests -./bin/hyperfleet-e2e test --label-filter="lifecycle && critical" +# Run cluster tier0 tests only +./bin/hyperfleet-e2e test --label-filter="tier0" --focus "\[Suite: cluster\]" # Deep debug mode (add API calls and framework internals) ./bin/hyperfleet-e2e test --log-level=debug ``` -**Note**: The default output already shows detailed test execution steps. If a test fails, you can usually diagnose the issue from the logs without re-running in debug mode. Use `--log-level=debug` when you need to see API calls and framework internals. See [Troubleshooting](troubleshooting.md) for more debugging techniques. +**Note**: The default output already shows detailed test execution steps. If a test fails, you can usually diagnose the issue from the logs without re-running in debug mode. Use `--log-level=debug` when you need to see API calls and framework internals. See [Debugging Guide](debugging.md) for more debugging techniques. ## Common Commands @@ -95,8 +95,10 @@ echo $HYPERFLEET_API_URL curl -I $HYPERFLEET_API_URL ``` -**Test timeouts**: +**Test timeouts**: Increase timeouts via environment variables: ```bash +HYPERFLEET_TIMEOUTS_CLUSTER_RECONCILED=45m make e2e +``` **Configuration not taking effect**: