Feature/825 split up workflows unit tests and periodic runs by ArBridgeman · Pull Request #826 · exasol/python-toolbox

ArBridgeman · 2026-05-06T12:12:09Z

closes #825
relates to #337
relates to #730

Checklist

Note: If any of the items in the checklist are not relevant to your PR, just check the box.

For any Pull Request

Is the following correct:

the title of the Pull Request?
the title of the corresponding issue?
there are no other open Pull Requests for the same update/change?
that the issue which this Pull Request fixes ("Fixes...") is mentioned?

When Changes Were Made

Did you:

update the changelog?
update the cookiecutter-template?
update the implementation?
check coverage and add tests: unit tests and, if relevant, integration tests?
update the User Guide & other documentation?
resolve any failing CI criteria (incl. Sonar quality gate)?

When Preparing a Release

Have you:

thought about version number (major, minor, patch)?
checked Exasol packages for updates and resolved open vulnerabilities, if easily possible?

ArBridgeman · 2026-05-06T12:13:01Z

Modify docs
Should be able to modify the .gitattributes in the template to include all of these, but should discuss if that makes sense
Need to add code coverage for

ckunki · 2026-05-06T12:20:57Z

@@ -0,0 +1,26 @@
+name: Fast-Tests-Extension


Is there no template for this workflow by intention?
Why?

Ah! I think I understood:

default workflow (generated from template) is fast-tests.yml

The yaml renderer has been enhanced to allow (%if sections

Such a section includes a call to an individual workflow file if such exists.

Maybe add a comment to this file?
Or user guide?

All good suggestions. I was planning on adding the doc-related points in this PR, just wanted to get an initial response first. Think I'd do both the user guide and a comment in one of the workflow files, like fast-tests.yml

ckunki · 2026-05-06T12:25:12Z

@@ -0,0 +1,37 @@
+name: Periodic-Validation


The name is excellent! 💯

Ah, I think it comes from you and/or @tkilias 😄
I took a look at #337 to see what had been discussed & with what I had in mind 😉

ckunki · 2026-05-06T12:25:56Z

+
+  run-slow-checks:
+    name: Slow Checks
+    uses: ./.github/workflows/slow-checks.yml


Should we keep the name "slow-checks" or would "slow-tests" be better as pendant to "fast-tests"?

OK looking below, I think you already came to a conclusion here.

- run-fast-checks - run-fast-tests - run-slow-checks

Either way works for me -> I'd like slow-tests a bit more.
Though, right now, it's not clear what we'll allow in the current slow-checks.
I like the comments from @tkilias:

Feature/825 split up workflows unit tests and periodic runs #826 (comment)

Feature/825 split up workflows unit tests and periodic runs #826 (comment)

If we allowed slow-checks to not be controlled by the PTB, this would reduce how much infrastructure we need to accommodate and would be easier for the downstream projects. It's a question then of what separates slow-checks from other slower tests or checks. Like @tkilias put in this comment:

Feature/825 split up workflows unit tests and periodic runs #826 (comment)

We might want to make the distinction as to what file options are put into the periodic-validation.yml for the Sonar statistics.

ckunki · 2026-05-06T12:28:09Z

+class WorkflowExtension(BaseModel):
+    """
+    Used to define which workflow extensions are active.
+    The corresponding `*-extension.yml` must be defined in the project.


Suggested change

The corresponding `*-extension.yml` must be defined in the project.

The corresponding files `*-extension.yml` are optional

but must be added in the individual projects.

ckunki · 2026-05-06T12:29:37Z

+    The corresponding `*-extension.yml` must be defined in the project.
+    """
+
+    fast_tests: bool = Field(


Couldn't the renderer look for the files and include them if they exist?
In this case we wouldn't need an additional configuration in the noxconfig.

There could also be options when calling nox session workflow:generate, e.g. --fast-tests-extension and --slow-checks-extension to generate an initial empty skeleton.

I like your point about having the BaseConfig detect the files (instead of providing a configuration). I think that's a better/simpler way to go.

I'll think about it some more. Initially, I'm not so sure about adding on to the CLI with empty skeletons. I like @tkilias's more general suggestion to include/link to examples. We would already have 2 cases covered in the python-toolbox too, so those could be linked.

tkilias · 2026-05-06T13:28:31Z


      - name: Run Integration Tests
        id: run-integration-tests
        run: poetry run -- nox -s test:integration -- --coverage --db-version ${{ matrix.exasol-version }}


Having this by default is a problem, because for most projects that would cause most slow tests to run sequentially. I think the good thing is that the slow-tests don't need to implement the merge-gate anymore. So, we could think about giving this as an example, but with the intention that this will be changed. And, for things like installing Python, creating matrix builds, or uploading coverage, we provide tools and jinja templates/macros.

I really like this idea and think it would simplify the potential work greatly.

tkilias · 2026-05-06T13:39:01Z

@@ -45,3 +45,10 @@
          name: coverage-python${{ matrix.python-version }}-exasol${{ matrix.exasol-version }}-slow


We could assume that slow-tests run coverage for a matrix or in sub-workflows, so we probably need a more robust way to define the name of the coverage file name. Maybe we should add the workflow name, if this is possible. And, if available, the matrix. This might need a jinja macro with parameters to achieve

With Sonar, we only need the coverage file to look like coverage-python3.10-*. If we do the wild west variants, then I think we need only to communicate this clearly.

If we decide we need more information and deliberation, I'm ok to move the slow-checks modifications out. I wanted to get the conversation started in pieces, as when we try the whole, sometimes it can lead to longer and harder discussions.

tkilias · 2026-05-06T13:40:47Z

          name: coverage-python${{ matrix.python-version }}-exasol${{ matrix.exasol-version }}-slow
          path: .coverage
          include-hidden-files: true
+  (% if workflow_extension.slow_checks %)


If we assume the Wild West in this file, we probably don't need this

tkilias · 2026-05-06T13:45:48Z

    secrets: inherit
    permissions:
      contents: read



We probably need here an extension workflow call for approval-protected tests that are independent of the slow tests. For example, GPU or SaaS, we might want long-term to split up from the slow tests because they are not always needed for integration tests and they are even more expensive. For GPU tests, we already do this in some repos. An equivalent we probably need in the periodic validation. The idea is that the project can duplicate the structure for slow tests for their own manual approvals.

ArBridgeman added 10 commits May 6, 2026 12:50

Move fast unit tests to fast-tests.yml

81d8ec2

Apply changes to the PTB

40ffdb7

Extract periodic-validation code

0f8abdd

Apply changes to the PTB for periodic-validation

c163f0d

Add way to extend fast-tests.yml

e5e6bc0

Fix unit tests

4b351f7

Apply changes to the PTB for fast-tests-extension

70ec171

Add way to extend slow-checks.yml

85b3be8

Apply changes to the PTB for slow-checks-extension

ceda85f

Fix spacing in extensions

f823761

ArBridgeman requested a deployment to manual-approval May 6, 2026 12:12 — with GitHub Actions Waiting

ArBridgeman marked this pull request as draft May 6, 2026 12:12

Add to .gitattributes

6193139

ArBridgeman force-pushed the feature/825_split_up_workflows_unit_tests_and_periodic_runs branch from 83e2a12 to 6193139 Compare May 6, 2026 12:13

ArBridgeman requested a deployment to manual-approval May 6, 2026 12:13 — with GitHub Actions Waiting

ckunki requested changes May 6, 2026

View reviewed changes

tkilias reviewed May 6, 2026

View reviewed changes

	The corresponding `*-extension.yml` must be defined in the project.
	The corresponding files `*-extension.yml` are optional
	but must be added in the individual projects.

		@@ -45,3 +45,10 @@
		name: coverage-python${{ matrix.python-version }}-exasol${{ matrix.exasol-version }}-slow

Conversation

ArBridgeman commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

For any Pull Request

When Changes Were Made

When Preparing a Release

Uh oh!

ArBridgeman commented May 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tkilias May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tkilias May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ArBridgeman commented May 6, 2026 •

edited

Loading

tkilias May 6, 2026 •

edited

Loading

tkilias May 6, 2026 •

edited

Loading