Skip to content

CI: Add GitHub Actions workflow for conformance tests and performance benchmarking#61

Merged
kiritigowda merged 1 commit into
KhronosGroup:openvx_1.3from
simonCatBot:ci/github-actions-workflow
Jun 20, 2026
Merged

CI: Add GitHub Actions workflow for conformance tests and performance benchmarking#61
kiritigowda merged 1 commit into
KhronosGroup:openvx_1.3from
simonCatBot:ci/github-actions-workflow

Conversation

@simonCatBot

Copy link
Copy Markdown
Contributor

Summary

Replaces legacy Travis CI and GitLab CI with a GitHub Actions workflow modeled after kiritigowda/rustVX/.github/workflows/conformance.yml.

What this PR does

  • Deletes .travis.yml and .gitlab-ci.yml
  • Adds .github/workflows/ci.yml with three phases:

Phase 1 — Build (parallel)

  • build-debug: Debug build with all extensions + coverage flags (-fprofile-arcs -ftest-coverage)
  • build-release: Release build with all extensions (for benchmarking)
  • build-cts: Build Khronos CTS against the debug install

Phase 2 — Test (parallel, needs build-cts)

  • cts-baseline: Smoke tests + code coverage collection
  • cts-enhanced-vision, cts-neural-networks, cts-ix, cts-graph-features, cts-data-objects, cts-user-kernels: Stage-based CTS runs with --filter, matching the rustVX conformance pattern

Phase 3 — Performance gate (PR only, needs build-release + build-main)

  • build-main: Build the merge target (main / openvx_1.3) release for comparison
  • perf-gate\: Clone openvx-mark`, build against PR and main releases, run benchmarks, compare results

Design decisions

  • Separate debug/release builds: Debug is instrumented for coverage and CTS; Release is optimized for openvx-mark benchmarking
  • Artifact reuse: Build once, test/benchmark many times — avoids rebuilding the library N times
  • chmod +x after artifact download: actions/download-artifact@v4 strips executable permissions
  • continue-on-error: true for known pre-existing failures (not regressions):
    • cts-neural-networks: AlexNetTestNetwork fails to load weights (missing test_data file)
    • cts-graph-features: GraphPipeline.* fails (pipelining incomplete in C model target)
  • Excluded: --conf_nnef due to NNEF-Tools C++ parser GCC 11 incompatibility

Notes

This PR is intentionally separate from #60 (bug-fix PR) so CI infrastructure can be reviewed and merged independently.

cc: @kiritigowda — modeled after rustVX conformance workflow, thanks for the reference!

… benchmarking

Modeled after kiritigowda/rustVX conformance.yml workflow.

Phase 1 (parallel):
- build-debug: Debug build with all extensions + coverage flags
- build-release: Release build with all extensions (for benchmarking)
- build-cts: Build Khronos CTS against debug install

Phase 2 (parallel, needs build-cts):
- cts-baseline: Smoke tests + code coverage collection
- cts-enhanced-vision, cts-neural-networks, cts-ix,
  cts-graph-features, cts-data-objects, cts-user-kernels:
  Stage-based CTS runs with --filter, matching rustVX pattern

Phase 3 (PR only, needs build-release + build-main):
- build-main: Build target branch release for comparison
- perf-gate: Clone openvx-mark, build against PR and main,
  run benchmarks, compare results

Known issues (marked continue-on-error):
- cts-neural-networks: TensorNetworks.AlexNetTestNetwork fails
  to load weights (missing test_data file, pre-existing)
- cts-graph-features: GraphPipeline.* fails (pipelining incomplete
  in C model target, pre-existing)

Removed legacy .travis.yml and .gitlab-ci.yml.
Excluded --conf_nnef due to NNEF-Tools GCC 11 incompatibility.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Simon seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@kiritigowda kiritigowda merged commit a98dc8e into KhronosGroup:openvx_1.3 Jun 20, 2026
1 check was pending
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants