NETOBSERV-2387: Backend tests migration by oliver-smakal · Pull Request #2586 · netobserv/netobserv-operator

oliver-smakal · 2026-03-24T13:27:15Z

Description

This PR is intended to show how the e2e backend migration of tests could look like and where we can discuss more details.

Changes compare to the openshift-tests-private repo:

using ginkgo directly without wrapper
using BeforeSuite possible, but we need to be carefull about not using compat_otp.NewCLI and similar utilities which have g.BeforeEach calls
yaml testdata files are not being compiled in and are just referenced from the file tree

Once agreed on the implementation, the testcases will need to be ported over one more time to reflect the latest changes and add guards to skip some of the testcases for certain openshift versions. Then I would put this PR from Draft status.

How to run locally

Either:

go run github.com/onsi/ginkgo/v2/ginkgo #to run all
go run github.com/onsi/ginkgo/v2/ginkgo --focus="VM" #use regex to filter

or if the ginkgo cli is installed:

ginkgo #to run all
ginkgo --focus="VM" #use regex to filter

Other standard flags of ginkgo such as --dry-run or -v also work.

Example output

The following is an example output of of a run:

[1776324075] Backend Suite - 3/7488 specs Running Suite: Backend Suite - /home/osmakal/Repos/network-observability-operator/integration-tests/backend
==========================================================================================================
Random Seed: 1776324075

Will run 3 specs
[sig-netobserv] Network_Observability with Loki with Kafka Author:aramesha-NonPreRelease-Longduration-Critical-56362-High-53597-High-56326-Verify network flows are captured with Kafka with TLS [Serial][Slow]
/home/osmakal/Repos/network-observability-operator/integration-tests/backend/test_flowcollector.go:2601
• PASSED [0.000 seconds]
•[sig-netobserv] Network_Observability with Loki with Kafka Author:aramesha-NonPreRelease-Longduration-High-57397-High-65116-Verify network-flows export with Kafka and netobserv installation without Loki[Serial]
/home/osmakal/Repos/network-observability-operator/integration-tests/backend/test_flowcollector.go:2654
• PASSED [0.000 seconds]
•[sig-netobserv] Network_Observability with Loki with Kafka Author:aramesha-NonPreRelease-High-64880-High-75340-Verify secrets copied for Loki and Kafka when deployed in NS other than flowcollector pods and networkPolicy enabled[Serial]
/home/osmakal/Repos/network-observability-operator/integration-tests/backend/test_flowcollector.go:2780
• PASSED [0.000 seconds]
•------------------------------

Backend Suite - 3/3 specs • SUCCESS! [0.136 seconds]

Ran 3 tests
Passed: 3, Failed: 0, Skipped: 0
 SUCCESS! 136.307854ms PASS

Ginkgo ran 1 suite in 12.165523429s
Test Suite Passed

This implementation cannot control the [1776103444] Backend Suite - 3/7488 specs line. I don't think it will be issue in any way, as the rest of the report can make it really clear to us what is actually being run. For running it in prow it should also not be a problem if we preserve the way it works with the openshift-test-private repo and use junit report.

More examples:

How it could be used in CI.

Currently the openshift-test-private implementation uses junit to handle the result.
See lines 398-459 that junit result is generated and parsed and later is used by the openshift-e2e-test-qe-report step to fail the prow job if necessary). For the nice output in prow, it seems like the function handle_result in the openshift-extended-test step is used as it seems to be formatting and renaming the junit file with the help of handleresult.py python script.

Though this is probably not directly reusable for implementation as the junit in openshift-test-private does not use default ginkgo implementation, we could use the same idea and use the junit output with minor transformation in prow to report result in a nice way.

Alternative approaches

We could create something a bit more custom using more low level ginkgo functionality like in commit fbb6fc36b17bc37b6eb847b8e51f42d26a9b29d8, to remove things like the [1776103444] Backend Suite - 3/7488 specs line. However, I don't think it would be worth the tradeoff of not using the default ginkgo runner via RunSpecs() as we would use the possibility to run tests in parallel and some other features.

This reverts commit 05de826.

…tion/NETOBSERV-2387

openshift-ci · 2026-03-24T13:27:20Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci-robot · 2026-03-24T13:27:20Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-03-24T13:27:26Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign stleerh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

DOWNSTREAM_OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2026-03-24T13:35:23Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.28%. Comparing base (877e2e9) to head (037d406).
⚠️ Report is 91 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2586      +/-   ##
==========================================
- Coverage   72.65%   72.28%   -0.37%     
==========================================
  Files         104      107       +3     
  Lines       10615    11354     +739     
==========================================
+ Hits         7712     8207     +495     
- Misses       2429     2659     +230     
- Partials      474      488      +14

Flag	Coverage Δ
unittests	`72.28% <ø> (-0.37%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 42 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

openshift-ci-robot · 2026-03-24T13:35:50Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-03-25T11:57:35Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

memodi · 2026-03-27T14:25:24Z

@@ -0,0 +1,44 @@
+package netobserv


should we change the package name to say, may be "integration-tests" instead?

Sure, what about just integration, testsuite or tests?

memodi · 2026-03-27T14:32:23Z

+		ensureNetObservOperatorDeployed(oc, NO, NOSource, deployedUpstreamCatalogSource)
+	})
+
+	g.It("Author:aramesha-High-64156-Verify IPFIX-exporter [Serial]", func() {


I am not sure if labels like [Serial] will have any meaning here. Most of the tests that deals with flowcollector resource are Serial, we could use ginkgo Ordered decorator to make tests Serial.

We should also probably evaluate what tests could run in parallel, for e.g. FlowMetrics ones perhaps?

I am not sure if labels like [Serial] will have any meaning here. Most of the tests that deals with flowcollector resource are Serial, we could use ginkgo Ordered decorator to make tests Serial.

Unless we do -p flag, than it does not run in parallel. But I will definitionally cleanup the [Serial] flags once I merge the latest test changes.

We should also probably evaluate what tests could run in parallel, for e.g. FlowMetrics ones
perhaps?

Agree, I will make sure it is possible to run tests in parallel and do some brief exploration and suggestion on which tests to run in parallel. It is nice that with the Ordered we could even run tests in stages, if a few tests can run in parallel together but cannot run parallel with others.

cool, all of these can be done as follow ups, no rush to get them as part of migration.

openshift-ci-robot · 2026-04-13T18:16:28Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:16:00Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:16:20Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:19:52Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:20:55Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:24:26Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:26:26Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T19:27:47Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-13T20:09:18Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

coderabbitai · 2026-04-13T20:09:24Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 13af035a-d238-4ee7-984e-8e63fa5f74cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-ci-robot · 2026-04-14T12:12:06Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-14T12:16:43Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-14T12:18:28Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-14T12:56:39Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-04-16T07:23:10Z

@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

…tion/NETOBSERV-2387

Oliver Smakal added 16 commits January 13, 2026 22:09

gingko miniman test project

cb1aaee

succesfully compiling port

05de826

Revert "succesfully compiling port"

2dbe8a9

This reverts commit 05de826.

Merge remote-tracking branch 'upstream/main' into backend-tests-migra…

8809971

…tion/NETOBSERV-2387

adjust packgage&module naming

284b61b

gomod copied from previous commit

c67680d

add notes on the migration

d151e73

add migration notes

ce354f3

copy files from upstream main

87a1d24

rm unsued deps

9a73014

fix print formatting issues

5a30bdf

fix gomod issues

364238d

rm unused deps

31d8f75

run only netobserv tests

469fd05

add testdata; don't use bindata

b359c73

add example of beforesuite usage

ff52909

openshift-ci bot added the do-not-merge/work-in-progress label Mar 24, 2026

openshift-ci-robot added the jira/valid-reference label Mar 24, 2026

docs cleanup

606a641

add customer wrapper to correctly show skipped tests

fbb6fc3

memodi reviewed Mar 27, 2026

View reviewed changes

use more standard ginkgo functionallity

73f75de

adjust skipped test print

3a28386

Oliver Smakal added 4 commits April 16, 2026 09:24

output cleanup

7d94bcc

rm local origin replace

aeaac2b

make the go.mod as close to the openshift-test-private one as possible

037d406

merge go.mod of tests and vendor

3163e02

openshift-ci bot added the needs-rebase label Apr 16, 2026

Merge remote-tracking branch 'upstream/main' into backend-tests-migra…

5a78e6a

…tion/NETOBSERV-2387

openshift-ci bot removed the needs-rebase label Apr 17, 2026

Oliver Smakal added 2 commits April 19, 2026 21:06

fixtest

ba741f1

fix test

32a4a2d

Conversation

oliver-smakal commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How to run locally

Example output

How it could be used in CI.

Alternative approaches

Uh oh!

openshift-ci bot commented Mar 24, 2026

Uh oh!

openshift-ci-robot commented Mar 24, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

openshift-ci bot commented Mar 24, 2026

Uh oh!

codecov bot commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

openshift-ci-robot commented Mar 24, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

openshift-ci-robot commented Mar 25, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

memodi Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

oliver-smakal Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

memodi Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

memodi Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

oliver-smakal Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

memodi Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Apr 13, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

openshift-ci-robot commented Apr 13, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Example output

How it could be used in CI.

Alternative approaches

Uh oh!

openshift-ci-robot commented Apr 13, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Example output

How it could be used in CI.

Alternative approaches

Uh oh!

openshift-ci-robot commented Apr 13, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Example output

How it could be used in CI.

Alternative approaches

Uh oh!

openshift-ci-robot commented Apr 13, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Example output

How it could be used in CI.

Alternative approaches

oliver-smakal commented Mar 24, 2026 •

edited

Loading

openshift-ci-robot commented Mar 24, 2026 •

edited by openshift-ci bot

Loading

codecov bot commented Mar 24, 2026 •

edited

Loading

openshift-ci-robot commented Mar 24, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Mar 25, 2026 •

edited by openshift-ci bot

Loading

oliver-smakal Apr 1, 2026 •

edited

Loading

oliver-smakal Apr 2, 2026 •

edited

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 13, 2026 •

edited by openshift-ci bot

Loading

coderabbitai bot commented Apr 13, 2026 •

edited

Loading

openshift-ci-robot commented Apr 14, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 14, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 14, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Apr 14, 2026 •

edited by openshift-ci bot

Loading