Record the login SLI rebuild and the Ktor instrumentation gap - #3141
Closed
StylianosGakis wants to merge 1 commit into
Closed
Record the login SLI rebuild and the Ktor instrumentation gap#3141StylianosGakis wants to merge 1 commit into
StylianosGakis wants to merge 1 commit into
Conversation
…n gap The `Auth: login (Android)` SLO fired at 534.351% of its 7-day error budget with no real login outage behind it. Its numerator counted `@type:error` events while its denominator counted `@type:resource` events, 96% of which were apollo-router calls that merely happened while the login screen was on top, and both sides were keyed on `@view.name`, which changed underneath the SLI during the Nav2 to Nav3 migration. Now that the auth client produces resource events, the fix is a filter change with no app code: both sides become `@type:resource` on `auth.prod.hedvigit.com` scoped to an exact `/member-login` path, `@view.name` drops out entirely, and the window moves to 30 days so the error budget is roughly 15 events rather than one to three. Also exempts the two login metrics from the pending `OR`-branch cleanup. No 14.4.x build emits a login-view resource event, so removing the legacy branch would take their denominator to zero and make monitor 93408872 fire permanently.
StylianosGakis
force-pushed
the
docs/datadog-login-slo-findings
branch
from
September 8, 2026 12:48
54c68cc to
7a11565
Compare
StylianosGakis
changed the title
StylianosGakis
removed this pull request from stack #3142
September 9, 2026 08:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI description:
Two records from the investigation behind the PR below this one, plus a blocker note on an existing doc. No code, because the remaining work is Datadog configuration rather than app changes.
2026-09-08-android-login-sli-rebuild.mdis the definition for rebuilding theAuth: login (Android)SLI. It fired at 534.351% of a 7-day budget with no login outage: the numerator counted@type:errorevents while the denominator counted@type:resourceevents, 96% of which were apollo-router calls that merely happened while the login screen was on top, and both were keyed on@view.name, which changed underneath the SLI during the Nav2 to Nav3 migration.Because the PR below makes auth calls produce resource events, the fix is a filter change and nothing more. Both sides become
@type:resourceonauth.prod.hedvigit.comscoped to an exact/member-loginpath, so the bad set is a subset of the total set by construction and@view.namedrops out entirely. The window moves to 30 days, which at 1,547 login-screen impressions a month gives an error budget of roughly 15 events instead of one to three.2026-09-07-ktor-datadog-instrumentation-gap.mdrecords what the OkHttp to Ktor move cost us and what is recoverable, including two things worth not rediscovering: no version of the Datadog Ktor plugin has ever supported resource timings, and addingDatadogEventListeneralongside it cannot work, because the plugin keys resources with its ownuuid4()in a Ktor attribute that the listener cannot see.2026-08-27-datadog-android-metric-recovery.mdgains a blocker note. It schedules removal of the legacyORbranches once the pre-14.3.6 install base drains, and for the two login metrics that removal would take the denominator to zero and make monitor93408872fire permanently, since no 14.4.x build emits a login-view resource event at all.Why write these down
RUM raw event retention is 30 days, so every measurement the argument rests on stops being verifiable around 7 October. This follows what #3104 established with the 2026-08-27 doc.
An earlier draft of this PR proposed rebuilding the SLI as a RUM operation, which would have meant app code and another release. That plan only made sense while auth calls were invisible. It has been dropped.