Conversation
Two runs that failed the same way rarely have byte-identical error text (paths, line numbers, addresses, ids, timestamps differ), which defeats 'is this the same failure?' and 'which tests fail together?'. Strip the variable parts of an error to a canonical form and hash it (SHA-256) so the same kind of failure gets the same short signature across runs - the join key run diffing and flake clustering group on. group_failures buckets a list of errors by signature. Pure stdlib.
…ature-batch Add failure_signature: normalise + hash errors to stable signatures
A run history says a run failed but not what changed from the run that passed. Align two step sequences with an LCS walk (so an inserted or removed step shifts the rest into place instead of mis-pairing) and classify the differences: added/removed steps, status flips (with the new failure's signature), and timing regressions. summarize_run_diff renders a one-line summary. Pure stdlib over step dicts.
Add run_diff: LCS-aligned diff of two run step-traces
…ard) Flaky tests are rarely independent - a wobbly fixture or noisy dependency makes a group fail in the same runs (~75% of flaky tests cluster). Ranking tests one-by-one by flip rate misses that shared root cause. Measure how often each pair fails in the same runs (Jaccard over their failing-run sets) and group tests above a threshold into connected clusters with a cohesion score. Pure stdlib over a list of failed-test sets.
…r-batch Add flake_cluster: cluster tests that flake together (co-failure Jaccard)
The action profiler aggregates timings by step name across runs, which
can't explain why one specific run was slow. Turn a single run's ordered
steps into a waterfall (each step's offset, duration and share of the
total) with the bottleneck step and a parallelism ratio, and rank the
dominant steps. A step is any {name, duration, start?} dict; explicit
start times place overlapping steps on an absolute timeline. Pure
stdlib.
…e-batch Add step_timeline: per-run step waterfall + bottleneck steps
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 152 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release: test-robustness lane (v191–v194)
Bundles the completed test-robustness lane — four pure-stdlib analytics features over run / failure data, each merged to
devCI-green (Codacy 0 / SonarCloud OK / all matrices + Docker).failure_signature(v191, Add failure_signature: normalise + hash errors to stable signatures #410) — normalise an error (strip paths / addresses / line numbers / timestamps / ids) and SHA-256 it, so the same kind of failure matches across runs;group_failuresbuckets by signature.run_diff(v192, Add run_diff: LCS-aligned diff of two run step-traces #411) — LCS-align two run step-traces → added/removed steps, status flips (with the new failure's signature), timing regressions;summarize_run_diff.flake_cluster(v193, Add flake_cluster: cluster tests that flake together (co-failure Jaccard) #412) — cluster tests that fail together by co-failure Jaccard, so you chase one root cause instead of N symptoms.step_timeline(v194, Add step_timeline: per-run step waterfall + bottleneck steps #413) — turn one run's ordered steps into a waterfall (offset / duration / share, bottleneck, parallelism) and rank the dominant steps.This completes ROUND-14 (4 lanes: native-UI control depth, cross-app clipboard, vision, test-robustness). All logic is headless-testable pure stdlib; EN/Zh docs (v191–v194) + WHATS_NEW entries included.