Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 3 additions & 8 deletions backend/kernelCI_app/management/commands/notifications.py
Original file line number Diff line number Diff line change
Expand Up @@ -778,18 +778,18 @@ def generate_hardware_summary_report(


def _fmt_change(cur: int, prev: int, show_percentage: bool = True) -> str:
"""Return a signed change string, e.g. '-246 (-17%)' or '-5'."""
"""Return a signed change string, e.g. '-246 (-17%)' or '-5'."""
diff = cur - prev
if diff == 0:
return "0 (0%)"
return "0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question: is there any reason we are not just following the show_percentage parameter here? to decide if we should show or not?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can we return a tuple of int instead? Then the template can align them separately, and we further separate data processing from templating

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure tbh, but even still the lab changes are currently using percentages but they would be better off if the zeroes didn't have the percentage IMO


abs_formatted_diff = f"{abs(diff):,}"
signed_diff = f"+{abs_formatted_diff}" if diff > 0 else f"-{abs_formatted_diff}"

if show_percentage and prev != 0:
percentage = round((diff / prev) * 100)
percentage_sign = "+" if percentage > 0 else ""
return f"{signed_diff} ({percentage_sign}{percentage}%)"
return f"{signed_diff} ({percentage_sign}{percentage}%)"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: perhaps we could use f"{diff:+} {percentage:+}" here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I didn't get it. The difference between f"{signed_diff} ({percentage_sign}{percentage}%)" and f"{diff:+} {percentage:+}" would be from +100 (+200%) to +100 +200 for diff = 100 and prev = 50

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only meant to use '+' format modifier, but is not necessary.


return signed_diff

Expand Down Expand Up @@ -864,18 +864,13 @@ def generate_metrics_report(

deltas = compute_metrics_deltas(data)

# Compute the text spacing for the labs so it isn't too big or too small
lab_spacing = max((len(lab_key) for lab_key in data.lab_maps.keys()), default=0)
lab_spacing += 7 # add spacing for leading tab and possible * mark for new labs

report = {}
template = setup_jinja_template("metrics_report.txt.j2")
report["content"] = template.render(
**data.model_dump(),
start_datetime=start_datetime.strftime("%Y-%m-%d %H:%M %Z"),
end_datetime=end_datetime.strftime("%Y-%m-%d %H:%M %Z"),
deltas=deltas,
lab_spacing=lab_spacing,
)

report["title"] = "KernelCI Metrics Report - %s" % now.strftime("%Y-%m-%d %H:%M %Z")
Expand Down
164 changes: 112 additions & 52 deletions backend/kernelCI_app/management/commands/templates/metrics_report.txt.j2
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,42 @@ Period: {{ start_datetime }} to {{ end_datetime }}

COVERAGE
--------
{{ "{:<22}".format("") -}}{{ "{:<13}".format("This week") -}}{{ "{:<13}".format("Last week") }}Change
{{ "{:<22}".format("") -}}{{ "{:<13}".format("─────────") -}}{{ "{:<13}".format("─────────") }}──────
{{ "{:<22}".format(" Trees monitored") -}}{{ "{:<13}".format(fmt(n_trees)) -}}{{ "{:<13}".format(fmt(prev_n_trees)) }}{{ deltas.n_trees }}
{{ "{:<22}".format(" Checkouts") -}}{{ "{:<13}".format(fmt(n_checkouts)) -}}{{ "{:<13}".format(fmt(prev_n_checkouts)) }}{{ deltas.n_checkouts }}
{{ "{:<22}".format(" Builds") -}}{{ "{:<13}".format(fmt(n_builds)) -}}{{ "{:<13}".format(fmt(prev_n_builds)) }}{{ deltas.n_builds }}
{{ "{:<22}".format(" Tests") -}}{{ "{:<13}".format(fmt(n_tests)) -}}{{ "{:<13}".format(fmt(prev_n_tests)) }}{{ deltas.n_tests }}
{%- set static_spacing = 2 -%}
{%- set current_week_space = [n_builds|string|length, n_boots|string|length, n_tests|string|length, "This week"|length] | max + static_spacing %}
{%- set previous_week_space = [prev_n_builds|string|length, prev_n_boots|string|length, prev_n_tests|string|length, "Last week"|length] | max + static_spacing %}
{%- set change_space = [deltas.n_builds|string|length, deltas.n_boots|string|length, deltas.n_tests|string|length, "Change"|length] | max + static_spacing -%}
{%- set label_space = 15 %} {#- length of the longest label -#}

{#- Each group is a row, each line is a column #}
{{ "{:<{width}}".format("", width=label_space) -}}
{{ "{:>{width}}".format("This week", width=current_week_space) -}}
{{ "{:>{width}}".format("Last week", width=previous_week_space) -}}
{{ "{:>{width}}".format("Change", width=change_space) }}
{#-#}
{{ "{:<{width}}".format("", width=label_space) -}}
{{ "{:>{width}}".format("─" * (current_week_space-static_spacing), width=current_week_space) -}}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we make this horizontal rule consistent with the other tables'?

{{ "{:>{width}}".format("─" * (previous_week_space-static_spacing), width=previous_week_space) -}}
{{ "{:>{width}}".format("─" * (change_space-static_spacing), width=change_space) }}
{#-#}
{{"{:<{width}}".format("Trees monitored", width=label_space) -}}
{{ "{:>{width}}".format(fmt(n_trees), width=current_week_space) -}}
{{ "{:>{width}}".format(fmt(prev_n_trees), width=previous_week_space) -}}
{{ "{:>{width}}".format(deltas.n_trees, width=change_space) }}
{#-#}
{{ "{:<{width}}".format("Checkouts", width=label_space) -}}
{{ "{:>{width}}".format(fmt(n_checkouts), width=current_week_space) -}}
{{ "{:>{width}}".format(fmt(prev_n_checkouts), width=previous_week_space) -}}
{{ "{:>{width}}".format(deltas.n_checkouts, width=change_space) }}
{#-#}
{{ "{:<{width}}".format("Builds", width=label_space) -}}
{{ "{:>{width}}".format(fmt(n_builds), width=current_week_space) -}}
{{ "{:>{width}}".format(fmt(prev_n_builds), width=previous_week_space) -}}
{{ "{:>{width}}".format(deltas.n_builds, width=change_space) }}
{#-#}
{{ "{:<{width}}".format("Tests", width=label_space) -}}
{{ "{:>{width}}".format(fmt(n_tests), width=current_week_space) -}}
{{ "{:>{width}}".format(fmt(prev_n_tests), width=previous_week_space) -}}
{{ "{:>{width}}".format(deltas.n_tests, width=change_space) }}


BUILD REGRESSIONS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BUILD REGRESSIONS should also be right aligned for consistency

@MarceloRobert MarceloRobert Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not needed to change build regressions since the scale there is smaller, eg. https://groups.io/g/kernelci-results/message/61332

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe leave the auto-width out, but align the numbers right? the report you linked to still spans 3 orders of magnitude on Affected Builds

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works, done.

Expand All @@ -24,30 +54,37 @@ Period: {{ start_datetime }} to {{ end_datetime }}

{% if build_incidents_by_origin -%}

{{ "{:<12}".format(" Origin") -}}
{{ "{:<18}".format("Regressions") -}}
{{ "{:<19}".format("Affected") -}}
Affected builds by top issues
{{ "{:<12}".format("") -}}
{{ "{:<18}".format("(known + new)") -}}
{{ "{:<19}".format("Builds (total)") -}}
{{ "{:<8}".format("#1") -}}{{ "{:<8}".format("#2") -}}#3
─────────────────────────────────────────────────────────────────────────
{% set origin_space = 12 -%}
{% set known_reg_space = 6 -%}
{% set new_reg_space = 6 -%}
{% set total_reg_space = 6 -%}
{% set reg_space = known_reg_space + new_reg_space + total_reg_space -%}
{% set affected_space = 16 -%}
{% set ind_issues_space = 10 -%}
{{ "{:<{width}}".format(" Origin", width=origin_space) -}}
{{ "{:>{width}}".format("Regressions", width=reg_space) -}}
{{ "{:>{width}}".format("Affected", width=affected_space) -}}
{{ "{:>{width}}".format("Affected builds by top issues", width=ind_issues_space*3 + 1) }} {#- +1 to get to a 2-space gap, same for the #1 issue and the space after total incidents #}
{{ "{:<{width}}".format("", width=origin_space) -}}
{{ "{:>{width}}".format("(known + new)", width=reg_space) -}}
{{ "{:>{width}}".format("Builds (total)", width=affected_space) -}}
{{ "{:>{width}}".format("#1", width=ind_issues_space + 1) -}}{{ "{:>{width}}".format("#2", width=ind_issues_space) -}}{{ "{:>{width}}".format("#3", width=ind_issues_space) }}
───────────────────────────────────────────────────────────────────────────
{% for origin, data in build_incidents_by_origin.items() | sort -%}
{{ "{:<12}".format(" " + origin) -}}
{{ "{:<4}".format(fmt(data['n_existing_issues'])) -}} +{{" "}}
{{- "{:<4}".format(fmt(data['n_new_issues'])) -}} ={{" "}}
{{- "{:<6}".format(fmt(data['n_total_issues'])) -}}
{{ "{:<19}".format(fmt(data['total_incidents'])) -}}
{{ "{:<{width}}".format(" " + origin, width=origin_space) -}}
{{ "{:>{width}}".format(fmt(data['n_existing_issues']) + " +", width=known_reg_space) }}
{{- "{:>{width}}".format(fmt(data['n_new_issues']) + " =", width=new_reg_space) }}
{{- "{:>{width}}".format(fmt(data['n_total_issues']), width=total_reg_space) -}}
{{ "{:>{width}}".format(fmt(data['total_incidents']), width=affected_space) -}} {{ " " -}}
{% for issue_key, data in top_issues_by_origin.get(origin, {}).items() -%}
{{ "{:<8}".format(fmt(data['total_incidents'])) -}}
{{ "{:>{width}}".format(fmt(data['total_incidents']), width=ind_issues_space) -}}
{% endfor %}
{% endfor %} ─────────────────────────────────────────────────────────────────────────
{{ "{:<12}".format(" Total") -}}
{{ "{:<4}".format(fmt(build_incidents_by_origin.values() | map(attribute='n_existing_issues') | sum)) -}} +{{" "}}
{{- "{:<4}".format(fmt(build_incidents_by_origin.values() | map(attribute='n_new_issues') | sum)) -}} ={{" "}}
{{- "{:<6}".format(fmt(build_incidents_by_origin.values() | map(attribute='n_total_issues') | sum)) -}}
{{ fmt(build_incidents_by_origin.values() | map(attribute='total_incidents') | sum) }}
{% endfor %} ───────────────────────────────────────────────────────────────────────────
{{ "{:<{width}}".format(" Total", width=origin_space) -}}
{{ "{:>{width}}".format(fmt(build_incidents_by_origin.values() | map(attribute='n_existing_issues') | sum) + " +", width=known_reg_space) }}
{{- "{:>{width}}".format(fmt(build_incidents_by_origin.values() | map(attribute='n_new_issues') | sum) + " =", width=new_reg_space) }}
{{- "{:>{width}}".format(fmt(build_incidents_by_origin.values() | map(attribute='n_total_issues') | sum), width=total_reg_space) -}}
{{ "{:>{width}}".format(fmt(build_incidents_by_origin.values() | map(attribute='total_incidents') | sum), width=affected_space) }}
{%- else %} No build regressions to show in this period. {%- endif %}


Expand Down Expand Up @@ -80,34 +117,57 @@ Affected builds by top issues

Labs marked with an asterisk (*) are new.

{% if n_labs -%}
{{ "{:<{width}}".format(" Lab", width=lab_spacing) -}}
{{ "{:<16}".format("Covered builds") -}}
{{ "{:<9}".format("Boots") -}}
{{ "{:<15}".format("Tests") -}}
Change (tests)
────────────────────────────────────────────────────────────────────────────────
{% if n_labs and lab_maps -%}
{#- Compute the text spacing in jinja instead of python to consider the `fmt` macro -#}
{%- set static_spacing = 3 -%}
{%- set lab_names_len = lab_maps.keys() | map('length') | max -%}
{%- set lab_names_space = [lab_names_len, "Lab"|length ] | max + static_spacing + 2 -%} {#- +2 for the possible ` *` -#}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: group all *_space for a table together

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wdym by grouping? just remove the empty lines between the variables? I think it helps to discern which steps are happening and which order they'll appear


{#- a namespace is required to store the values outside of the loop -#}
{#- initialize with the label length so we consider them as well -#}
{%- set ns = namespace(lab_builds_len="Covered builds"|length, lab_boots_len="Boots"|length, lab_tests_len="Tests"|length) -%}
{%- for v in lab_maps.values() -%}
{%- set ns.lab_builds_len = [ns.lab_builds_len, (fmt(v.builds))|length] | max -%}
{%- set ns.lab_boots_len = [ns.lab_boots_len, (fmt(v.boots))|length] | max -%}
{%- set ns.lab_tests_len = [ns.lab_tests_len, (fmt(v.tests))|length] | max -%}
{%- endfor -%}
{%- set lab_builds_space = ns.lab_builds_len + static_spacing -%}
{%- set lab_boots_space = ns.lab_boots_len + static_spacing -%}
{%- set lab_tests_space = ns.lab_tests_len + static_spacing -%}

{%- set lab_change_len = deltas.labs.values() | map('length') | max -%}
{%- set lab_change_space = [lab_change_len, "Change (tests)"|length] | max + static_spacing -%}

{%- set hrule_length = lab_names_space + lab_builds_space + lab_boots_space + lab_tests_space + lab_change_space - 2 -%}

{{ "{:<{width}}".format(" Lab", width=lab_names_space) -}}
{{ "{:>{width}}".format("Covered builds", width=lab_builds_space) -}}
{{ "{:>{width}}".format("Boots", width=lab_boots_space) -}}
{{ "{:>{width}}".format("Tests", width=lab_tests_space) -}}
{{ "{:>{width}}".format("Change (tests)", width=lab_change_space) }}
{{ "─" * hrule_length}}
{% for lab_key, lab_values in lab_maps.items() | sort -%}
{%- set display_name = lab_key + " *" if lab_key in deltas.new_lab_keys else lab_key -%}
{{ "{:<{width}}".format(" " + display_name, width=lab_spacing) -}}
{{ "{:<16}".format(fmt(lab_values["builds"])) -}}
{{ "{:<9}".format(fmt(lab_values["boots"])) -}}
{{ "{:<15}".format(fmt(lab_values["tests"])) -}}
{{ deltas.labs.get(lab_key, "") }}
{%- set display_name = lab_key + " *" if lab_key in deltas.new_lab_keys else lab_key -%}
{{ "{:<{width}}".format(" " + display_name, width=lab_names_space) -}}
{{ "{:>{width}}".format(fmt(lab_values["builds"]), width=lab_builds_space) -}}
{{ "{:>{width}}".format(fmt(lab_values["boots"]), width=lab_boots_space) -}}
{{ "{:>{width}}".format(fmt(lab_values["tests"]), width=lab_tests_space) -}}
{{ "{:>{width}}".format(deltas.labs.get(lab_key, ""), width=lab_change_space) }}
{% endfor %}
{%- for lab_key in deltas.extinct_lab_keys | sort -%}
{%- set lab_values = prev_lab_maps[lab_key] -%}
{{ "{:<{width}}".format(" " + lab_key, width=lab_spacing) -}}
{{ "{:<16}".format(fmt(0)) -}}
{{ "{:<9}".format(fmt(0)) -}}
{{ "{:<15}".format(fmt(0)) -}}
{{ deltas.labs.get(lab_key, "") }}
{% endfor %} ────────────────────────────────────────────────────────────────────────────────
{{ "{:<{width}}".format(" Total", width=lab_spacing) -}}
{{ "{:<16}".format(fmt(lab_maps.values() | map(attribute='builds') | sum)) -}}
{{ "{:<9}".format(fmt(lab_maps.values() | map(attribute='boots') | sum)) -}}
{{ "{:<15}".format(fmt(lab_maps.values() | map(attribute='tests') | sum)) -}}
{{ deltas.n_total_lab_activity }}
{%- set lab_values = prev_lab_maps[lab_key] -%}
{{ "{:<{width}}".format(" " + lab_key, width=lab_names_space) -}}
{{ "{:>{width}}".format(fmt(0), width=lab_builds_space) -}}
{{ "{:>{width}}".format(fmt(0), width=lab_boots_space) -}}
{{ "{:>{width}}".format(fmt(0), width=lab_tests_space) -}}
{{ "{:>{width}}".format(deltas.labs.get(lab_key, ""), width=lab_change_space) }}
{% endfor %}
{{- " " + "─" * hrule_length}}
{{ "{:<{width}}".format(" Total", width=lab_names_space) -}}
{{ "{:>{width}}".format(fmt(lab_maps.values() | map(attribute='builds') | sum), width=lab_builds_space) -}}
{{ "{:>{width}}".format(fmt(lab_maps.values() | map(attribute='boots') | sum), width=lab_boots_space) -}}
{{ "{:>{width}}".format(fmt(lab_maps.values() | map(attribute='tests') | sum), width=lab_tests_space) -}}
{{ "{:>{width}}".format(deltas.n_total_lab_activity, width=lab_change_space) }}

{%- endif %}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,25 @@ KernelCI Metrics Summary

COVERAGE
--------
This week Last week Change
───────── ───────── ──────
Trees monitored 105 100 +5
Checkouts 1,000 1,000 0 (0%)
Builds 11,000 10,000 +1,000 (+10%)
Tests 1,000,000 1,500,000 -500,000 (-33%)
This week Last week Change
───────── ───────── ───────────────
Trees monitored 105 100 +5
Checkouts 1,000 1,000 0
Builds 11,000 10,000 +1,000 (+10%)
Tests 1,000,000 1,500,000 -500,000 (-33%)


BUILD REGRESSIONS
-----------------
A "regression" is defined as a reported problem affecting 0 or multiple builds.

Origin Regressions Affected Affected builds by top issues
(known + new) Builds (total) #1 #2 #3
─────────────────────────────────────────────────────────────────────────
maestro 1 + 1 = 2 70 50 20
redhat 1 + 0 = 1 5 5
─────────────────────────────────────────────────────────────────────────
Total 2 + 1 = 3 75
Origin Regressions Affected Affected builds by top issues
(known + new) Builds (total) #1 #2 #3
───────────────────────────────────────────────────────────────────────────
maestro 1 + 1 = 2 70 50 20
redhat 1 + 0 = 1 5 5
───────────────────────────────────────────────────────────────────────────
Total 2 + 1 = 3 75


TOP REGRESSIONS PER ORIGIN
Expand All @@ -44,12 +44,12 @@ KernelCI Metrics Summary

Labs marked with an asterisk (*) are new.

Lab Covered builds Boots Tests Change (tests)
────────────────────────────────────────────────────────────────────────────────
lava-broonie 0 25,000 475,000 -175,000 (-27%)
lava-collabora 0 50,000 450,000 -250,000 (-36%)
────────────────────────────────────────────────────────────────────────────────
Total 0 75,000 925,000 -425,000 (-31%)
Lab Covered builds Boots Tests Change (tests)
───────────────────────────────────────────────────────────────────────
lava-broonie 0 25,000 475,000 -175,000 (-27%)
lava-collabora 0 50,000 450,000 -250,000 (-36%)
───────────────────────────────────────────────────────────────────────
Total 0 75,000 925,000 -425,000 (-31%)

--
This is an experimental report format. Please send feedback in!
Expand Down
Loading
Loading