Skip to content

ControllerReconcileErrorRatio flaps with unrelated traffic volume and omits the cluster #326

Description

@aflor024

What needs to happen

ControllerReconcileErrorRatioHigh and ControllerReconcileErrorRatioCritical should fire when a controller is genuinely failing to make progress, and should stay quiet otherwise. Today a controller with a steady, known set of failures flaps in and out of critical as unrelated healthy traffic rises and falls, and the notification doesn't say which cluster it came from.

Why this matters

Measured on the prod infra control plane for dnsrecordset-powerdns, per 10-minute window:

result count
error 193
success 604
requeue 0
requeue_after 0
total 797

Ratio at time of measurement: 24.2%.

Six customer records cannot be written because PowerDNS rejects them as coexistence conflicts (datum-cloud/infra#3716, #315). Those retries produce a roughly constant error floor. The ratio therefore tracks the denominator — when legitimate DNS work is busy, 600-odd successes dilute the floor below threshold and the alert resolves; when real activity goes quiet, the same unchanged error floor reads as 96.77% or 100% and it fires again.

The result is an alert that flips on traffic patterns rather than on a change in controller health, firing and clearing roughly 120 times per 24h into slack-critical. On 2026-07-29 it paged the on-call engineer seven times before being silenced, and re-fired seven minutes after the silence lapsed.

Separately, sum by (controller) discards cluster, namespace and pod. The notification names the reconciler but not where it is running, and both prod and staging fire the same alert — today the only way to tell them apart is the Alertmanager source URL.

Note

An earlier version of this issue claimed the alert fires on negligible reconcile volume, citing "30 of 31 reconciles". That was wrong — it was inferred from the reported percentage rather than measured. Real volume is ~800 reconciles per 10-minute window. A minimum-volume guard would not have prevented any of the pages described above. See the comment below.

Desired outcome

A controller with a stable set of known-failing objects does not flap in and out of critical as unrelated traffic varies. Either the alert reflects sustained inability to progress, or terminal failures stop contributing to it.

The notification identifies the cluster and namespace, so whoever is paged knows where to look without inferring it from the source link.

The promtool fixtures under test/prometheus-rules/controllers/ cover the flapping case — a fixed error rate with a varying success rate — so a future change cannot reintroduce it.

Related

Important

dns-operator#75 will make this specific alert go quiet by eliminating the error floor. It does not fix the alert's behaviour — any controller that accumulates a steady set of permanently-failing objects will flap the same way. Worth keeping this open on its own merits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions