Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8431 +/- ##
==========================================
- Coverage 88.91% 88.90% -0.02%
==========================================
Files 286 286
Lines 63103 63103
Branches 7908 7908
==========================================
- Hits 56110 56100 -10
- Misses 4735 4739 +4
- Partials 2258 2264 +6 🚀 New features to boost your workflow:
|
e1f8bc6 to
6a27d5c
Compare
0a86a6b to
723789d
Compare
d6e1c53 to
b44d601
Compare
added 9 commits
February 6, 2026 15:44
0642595 to
edf377e
Compare
onurctirtir
approved these changes
Feb 10, 2026
Member
There was a problem hiding this comment.
LGTM, optionally, we can make CI happier by adding this into multi_name_resolution.sql
SET client_min_messages TO WARNING;before the following line
DROP SCHEMA multi_name_resolution CASCADE;Also, I got confused a bit - won't we add a workflow that can be manually triggered via Actions tab within this PR? Sth that takes Citus version and "n-1 mode" to test against as inputs?
--
Also, it'd be nicer if we can document how to run those tests locally / on CI in src/test/regress/README.md etc.
eaydingol
added a commit
that referenced
this pull request
Mar 3, 2026
Adds support for running regression tests in mixed‑version scenarios This PR updates the regression test runner script, test schedules, and some regression tests to support running Citus in mixed‑version setups. With these changes, regression test schedules can be executed in **N/N‑1 and mixed‑version scenarios**, including: - SQL version is N‑1 - Library version is N‑1 - A worker is N‑1 - Coordinator is N‑1 For potential usage, see the workflow jobs removed in commit (0642595) and the corresponding test runs at (https://github.com/citusdata/citus/actions/runs/20746079664). **Details** **Schedule refactoring** To enable N‑1 testing for the remaining multi‑node tests, this PR separates tests that drop/create the Citus extension from the existing multi_1_schedule into a new schedule `multi_1_create_citus_schedule` This allows the remaining multi‑node tests to be reused safely in N‑1 scenarios. **Test changes** - Cleanup steps after tests. - Minor adjustments to account for the updated schedule structure. **Perl script changes** This PR extends src/test/regress/pg_regress_multi.pl to support: - Creating the Citus extension at a specific version for the coordinator (the original PR that introduced versioned extension creation updated only worker logic and missed the coordinator #8361) - Changing the Citus library (citus.so) for specific nodes (worker, coordinator, or all) - Creating the Citus extension at a given version for specific nodes (worker, coordinator, or all) These changes enable running N/N‑1 compatibility tests in multiple configurations. Sample scenarios are listed below. **Sample scenarios** To demonstrate potential usage sample jobs are added to git workflow but excluded from this PR (see previous comment on the commit and runs). **Test Citus Lib N‑1** All nodes load citus.so from version 13.2. **Test Citus SQL N‑1** All nodes create the Citus extension at version 13.2‑1. **Test Citus Worker N‑1** Only worker 1 loads citus.so from 13.2 and creates the extension at 13.2‑1. **Test Citus Coordinator N‑1** Only the coordinator loads citus.so from 13.2 and creates the extension at 13.2‑1. Note that the following schedules from “Test Citus” are not included in N‑1 scenarios: `check-multi-1-create-citus, check-multi-mx` Some tests in these schedules drop and recreate the Citus extension using the default version, which is incompatible with N‑1 setups. `check-vanilla` The test preparation steps have not yet been adapted for N‑1 workflows. **Local testing** Local testing can be performed using sql/versions.sql and test_versions.sh. (removed in previous commit) Steps: - Install the citus 13.2 - Copy citus.so files from 13.2.0 to ~/citus-libs/17/v13.2.0 - Install the version from head - Run ./test_versions.sh See version_test_results.txt for example results. Sample test runs: CITUSLIBDIR=~/citus-libs/17/v13.2.0 CITUSVERSION=13.2-1 N1MODE=workeronly EXTRA_TESTS=versions make check-minimal CITUSVERSION=13.2-1 N1MODE=all EXTRA_TESTS=versions make check-minimal
eaydingol
added a commit
that referenced
this pull request
Mar 3, 2026
Adds support for running regression tests in mixed‑version scenarios This PR updates the regression test runner script, test schedules, and some regression tests to support running Citus in mixed‑version setups. With these changes, regression test schedules can be executed in **N/N‑1 and mixed‑version scenarios**, including: - SQL version is N‑1 - Library version is N‑1 - A worker is N‑1 - Coordinator is N‑1 For potential usage, see the workflow jobs removed in commit (0642595) and the corresponding test runs at (https://github.com/citusdata/citus/actions/runs/20746079664). **Details** **Schedule refactoring** To enable N‑1 testing for the remaining multi‑node tests, this PR separates tests that drop/create the Citus extension from the existing multi_1_schedule into a new schedule `multi_1_create_citus_schedule` This allows the remaining multi‑node tests to be reused safely in N‑1 scenarios. **Test changes** - Cleanup steps after tests. - Minor adjustments to account for the updated schedule structure. **Perl script changes** This PR extends src/test/regress/pg_regress_multi.pl to support: - Creating the Citus extension at a specific version for the coordinator (the original PR that introduced versioned extension creation updated only worker logic and missed the coordinator #8361) - Changing the Citus library (citus.so) for specific nodes (worker, coordinator, or all) - Creating the Citus extension at a given version for specific nodes (worker, coordinator, or all) These changes enable running N/N‑1 compatibility tests in multiple configurations. Sample scenarios are listed below. **Sample scenarios** To demonstrate potential usage sample jobs are added to git workflow but excluded from this PR (see previous comment on the commit and runs). **Test Citus Lib N‑1** All nodes load citus.so from version 13.2. **Test Citus SQL N‑1** All nodes create the Citus extension at version 13.2‑1. **Test Citus Worker N‑1** Only worker 1 loads citus.so from 13.2 and creates the extension at 13.2‑1. **Test Citus Coordinator N‑1** Only the coordinator loads citus.so from 13.2 and creates the extension at 13.2‑1. Note that the following schedules from “Test Citus” are not included in N‑1 scenarios: `check-multi-1-create-citus, check-multi-mx` Some tests in these schedules drop and recreate the Citus extension using the default version, which is incompatible with N‑1 setups. `check-vanilla` The test preparation steps have not yet been adapted for N‑1 workflows. **Local testing** Local testing can be performed using sql/versions.sql and test_versions.sh. (removed in previous commit) Steps: - Install the citus 13.2 - Copy citus.so files from 13.2.0 to ~/citus-libs/17/v13.2.0 - Install the version from head - Run ./test_versions.sh See version_test_results.txt for example results. Sample test runs: CITUSLIBDIR=~/citus-libs/17/v13.2.0 CITUSVERSION=13.2-1 N1MODE=workeronly EXTRA_TESTS=versions make check-minimal CITUSVERSION=13.2-1 N1MODE=all EXTRA_TESTS=versions make check-minimal
eaydingol
added a commit
that referenced
this pull request
Mar 6, 2026
Adds support for running regression tests in mixed‑version scenarios This PR updates the regression test runner script, test schedules, and some regression tests to support running Citus in mixed‑version setups. With these changes, regression test schedules can be executed in **N/N‑1 and mixed‑version scenarios**, including: - SQL version is N‑1 - Library version is N‑1 - A worker is N‑1 - Coordinator is N‑1 For potential usage, see the workflow jobs removed in commit (0642595) and the corresponding test runs at (https://github.com/citusdata/citus/actions/runs/20746079664). **Details** **Schedule refactoring** To enable N‑1 testing for the remaining multi‑node tests, this PR separates tests that drop/create the Citus extension from the existing multi_1_schedule into a new schedule `multi_1_create_citus_schedule` This allows the remaining multi‑node tests to be reused safely in N‑1 scenarios. **Test changes** - Cleanup steps after tests. - Minor adjustments to account for the updated schedule structure. **Perl script changes** This PR extends src/test/regress/pg_regress_multi.pl to support: - Creating the Citus extension at a specific version for the coordinator (the original PR that introduced versioned extension creation updated only worker logic and missed the coordinator #8361) - Changing the Citus library (citus.so) for specific nodes (worker, coordinator, or all) - Creating the Citus extension at a given version for specific nodes (worker, coordinator, or all) These changes enable running N/N‑1 compatibility tests in multiple configurations. Sample scenarios are listed below. **Sample scenarios** To demonstrate potential usage sample jobs are added to git workflow but excluded from this PR (see previous comment on the commit and runs). **Test Citus Lib N‑1** All nodes load citus.so from version 13.2. **Test Citus SQL N‑1** All nodes create the Citus extension at version 13.2‑1. **Test Citus Worker N‑1** Only worker 1 loads citus.so from 13.2 and creates the extension at 13.2‑1. **Test Citus Coordinator N‑1** Only the coordinator loads citus.so from 13.2 and creates the extension at 13.2‑1. Note that the following schedules from “Test Citus” are not included in N‑1 scenarios: `check-multi-1-create-citus, check-multi-mx` Some tests in these schedules drop and recreate the Citus extension using the default version, which is incompatible with N‑1 setups. `check-vanilla` The test preparation steps have not yet been adapted for N‑1 workflows. **Local testing** Local testing can be performed using sql/versions.sql and test_versions.sh. (removed in previous commit) Steps: - Install the citus 13.2 - Copy citus.so files from 13.2.0 to ~/citus-libs/17/v13.2.0 - Install the version from head - Run ./test_versions.sh See version_test_results.txt for example results. Sample test runs: CITUSLIBDIR=~/citus-libs/17/v13.2.0 CITUSVERSION=13.2-1 N1MODE=workeronly EXTRA_TESTS=versions make check-minimal CITUSVERSION=13.2-1 N1MODE=all EXTRA_TESTS=versions make check-minimal
eaydingol
added a commit
that referenced
this pull request
Mar 9, 2026
## Backport mixed-version test infrastructure to release-13.2 This PR cherry-picks a series of commits from `main` that introduce mixed-version testing support and related fixes to the `release-13.2` branch. These changes enable running regression tests against different Citus versions (N/N-1 scenarios), which is essential for validating backward compatibility during phased upgrades. ### Cherry-picked commits 1. **Relax version check ([#8356](#8356 — `c125b9bec563` Relaxes the Citus version compatibility check to allow the installed extension to differ by at most one minor version from the loaded library. This enables replicas to lag by a single minor version during phased upgrades without triggering a version mismatch for distributed queries. 2. **Workflow refactor ([#8361](#8361 — `b8ab8aac554c` Refactors `.github/workflows/build_and_test.yml` to use a shared reusable workflow (`run_tests.yml`) for the main, failure, and CDC test suites. Introduces the `CITUSVERSION` environment variable so tests can create the Citus extension at a specific version (e.g., `CITUSVERSION=13.2-1 make check`). 3. **Test refactor for columnar separation ([#8420](#8420 — `b7a5f698ab77` Minor test adjustments to prepare for columnar module separation, adding necessary setup/teardown steps in several test files. 4. **Alternative lib directory for tests ([#8390](#8390 — `2e64eb020268` Adds support for loading `citus.so` and `columnar.so` from an alternative directory during tests via the `CITUSLIBDIR` environment variable (e.g., `CITUSLIBDIR=/path/to/v13.2.0 make check-minimal`). 5. **Store/restore exit status ([#8436](#8436 — `1c5b304feeef` Fixes regression test exit status being lost after test cleanup steps, ensuring the correct exit code is propagated. 6. **Mixed version tests for citus ([#8431](#8431 — `08a27ddc0579` The core mixed-version testing infrastructure. Extends `pg_regress_multi.pl` to support: - Creating the Citus extension at a specific version for coordinator and/or workers - Swapping the Citus shared library for specific nodes - Running N/N-1 compatibility tests in multiple configurations (SQL N-1, Lib N-1, Worker N-1, Coordinator N-1) Separates tests that drop/create the Citus extension into a new `multi_1_create_citus_schedule` to allow safe reuse of remaining multi-node tests in N-1 scenarios. 7. **Fix IsTenantSchema when version checks are disabled ([#8480](#8480 — `546f20661a3b` Fixes `IsTenantSchema()` incorrectly returning `false` when `citus.enable_version_checks` was off. The previous guard completely disabled schema-based sharding during mixed-version testing, causing 5 test failures. The fix replaces the guard with a `CheckCitusVersion()` call, which returns `true` unconditionally when version checks are disabled. ### Conflicts resolved - **`build_and_test.yml`** (commits 2 and 6): Resolved by accepting the incoming reusable workflow pattern and the new `check-multi-1-create-citus` / `check-tap` make targets. - **`Makefile`** (commit 6): Resolved by accepting the new `check-multi-1-create-citus` and `check-tap` targets in `check-full`. ### Files changed 29 files changed, +699 / -210 lines across CI workflows, test infrastructure (`pg_regress_multi.pl`, `Makefile`), core version checking logic (`metadata_cache.c`, `columnar_tableam.c`), tenant schema handling, and regression test SQL/expected output files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for running regression tests in mixed‑version scenarios
This PR updates the regression test runner script, test schedules, and some regression tests to support running Citus in mixed‑version setups.
With these changes, regression test schedules can be executed in N/N‑1 and mixed‑version scenarios, including:
For potential usage, see the workflow jobs removed in commit (0642595) and the corresponding test runs at (https://github.com/citusdata/citus/actions/runs/20746079664).
Details
Schedule refactoring
To enable N‑1 testing for the remaining multi‑node tests, this PR separates tests that drop/create the Citus extension from the existing multi_1_schedule into a new schedule
multi_1_create_citus_scheduleThis allows the remaining multi‑node tests to be reused safely in N‑1 scenarios.
Test changes
Perl script changes
This PR extends src/test/regress/pg_regress_multi.pl to support:
These changes enable running N/N‑1 compatibility tests in multiple configurations. Sample scenarios are listed below.
Sample scenarios
To demonstrate potential usage sample jobs are added to git workflow but excluded from this PR (see previous comment on the commit and runs).
Test Citus Lib N‑1
All nodes load citus.so from version 13.2.
Test Citus SQL N‑1
All nodes create the Citus extension at version 13.2‑1.
Test Citus Worker N‑1
Only worker 1 loads citus.so from 13.2 and creates the extension at 13.2‑1.
Test Citus Coordinator N‑1
Only the coordinator loads citus.so from 13.2 and creates the extension at 13.2‑1.
Note that the following schedules from “Test Citus” are not included in N‑1 scenarios:
check-multi-1-create-citus, check-multi-mxSome tests in these schedules drop and recreate the Citus extension using the default version, which is incompatible with N‑1 setups.check-vanillaThe test preparation steps have not yet been adapted for N‑1 workflows.Local testing
Local testing can be performed using sql/versions.sql and test_versions.sh. (removed in previous commit)
Steps:
See version_test_results.txt for example results.
Sample test runs:
CITUSLIBDIR=~/citus-libs/17/v13.2.0 CITUSVERSION=13.2-1 N1MODE=workeronly EXTRA_TESTS=versions make check-minimal
CITUSVERSION=13.2-1 N1MODE=all EXTRA_TESTS=versions make check-minimal