feat(buzz-agent): surface stop reason and silent-turn WARN in telemetry - #7038
Merged
Conversation
Add two telemetry improvements to make the silent-death failure mode visible in harness logs. 1. Emit `stop` on the existing `llm: call completed` INFO line in `llm.rs`. The `ProviderStop` value was already parsed and carried on `LlmResponse` but never logged, leaving "model chose end_turn" vs "gateway truncated/refused" indistinguishable from telemetry alone. 2. Log a WARN in `agent.rs` when a turn ends with no tool calls and output tokens below a conservative threshold (< 10). This is the exact signature observed in the silent-death incidents: one LLM call, 2–12 output tokens, no actions, no message posted. The WARN carries the `stop` reason and token count so a single re-ping in an affected channel identifies the failure class directly from the log. Neither change alters turn-handling logic, retry behavior, or stop- reason mapping semantics. The WARN is diagnostic only. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
github-actions
Bot
added
the
codex-security-review-current
🔐 Codex Security Review
|
…e boundary, None separation Three review fixes (all agent.rs): 1. Turn-level gate: add per-run `any_tool_call_seen` flag, updated unconditionally when tool calls execute. Gate the WARN on `!any_tool_call_seen` so a normal multi-round turn (tools ran in prior rounds, short final completion is expected) is never mislabeled as a silent death. 2. Inclusive threshold: change predicate to `output_tokens <= SILENT_TURN_TOKEN_THRESHOLD` (was `<`) and raise the constant from 10 to 12 to include the full observed failure range (2–12). The `is_silent_turn` signature changes from `Option<u64>` to `u64` — caller now handles None explicitly. 3. Separate None branch: providers that omit usage entirely are a distinct diagnostic case, not "near-zero tokens". Emit a separate WARN message for the None arm rather than equating it with low output counts; fixes the false-positive on any normal no-tool response from a usage-omitting provider. Tests updated: boundary tests now cover 0/12 (inclusive edge) and 13/large (above); new negative test proves the combined gate is silent when any_tool_call_seen is true even with zero final-round tokens. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
github-actions
Bot
removed
the
codex-security-review-current
github-actions
Bot
added
the
codex-security-review-current
…eam tests Four review fixes (agent.rs): 1. Publish-aware gate: replace any_tool_call_seen with unconditional buzz_reply_call_seen (via existing is_buzz_reply_call). Read-only tool calls no longer suppress the WARN — a turn that ran tools but never published then died at 3 tokens is a silent death. Only a genuine publish suppresses it. 2. Text check: capture response.text.trim().is_empty() before the move into history.push, and add it as a gate on warn_if_silent_turn so a terse reply like "OK" (non-empty text, ≤12 tokens) is not labeled a silent death. 3. Extracted warn_if_silent_turn: pull the WARN emit block into a pure synchronous free function taking (buzz_reply_call_seen, text_is_empty, output_tokens, stop). The run() call site becomes a single function call; the seam is directly testable. 4. Seam tests: replace the tautology negative test with 4 tests using a scoped tracing_subscriber layer (same pattern as the existing stall-warn tests in llm.rs) that counts WARNs from warn_if_silent_turn at the actual emission site: - canonical signature (no publish, no text, 4 tokens) → 1 WARN - non-empty text present → 0 WARNs - publish seen → 0 WARNs - None usage (no publish, no text) → 1 WARN (distinct message) Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
github-actions
Bot
removed
the
codex-security-review-current
wpfleger96
had a problem deploying
to
codex-review
August 29, 2026 19:06 — with 
GitHub Actions
Error
GitHub Actions
Error
… "no tool calls" The gate is publish-aware, not tool-call-aware, so the message strings were inaccurate for the read-only-tool → 3-token-death shape the WARN now covers. Wording only; no logic change. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96
had a problem deploying
to
codex-review
August 29, 2026 19:13 — with
GitHub Actions
Error
GitHub Actions
Error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two telemetry additions to make a known silent-death failure mode visible in harness logs.
1.
stopfield onllm: call completedINFO (crates/buzz-agent/src/llm.rs)The
ProviderStopvalue was already parsed and stored onLlmResponsebut never emitted in the log line. Without it, "model choseend_turn" vs "gateway truncated/refused" is indistinguishable from telemetry alone.2. WARN on silent-turn signature (
crates/buzz-agent/src/agent.rs)Emits a
WARNwhen a turn produces no publish, no visible assistant text, and either near-zero or absent output tokens. The WARN logic is extracted intowarn_if_silent_turn(pure synchronous function) so the seam is testable without the async run loop.Three independent gates before the WARN fires:
!buzz_reply_call_seen— no publish attempt in any round, tracked unconditionally via the existingis_buzz_reply_callmatcher. Read-only tool calls do NOT suppress the WARN; a turn that ran tools but never published and died at 3 tokens is still a silent death.text_is_empty— no visible assistant text in the final round. A terse reply like "OK" (≤12 tokens, non-empty) is not a silent death.Some(t) where t <= 12: near-zero token count, the observed failure signature (2–12 tokens)Noneusage: provider omitted token counts entirely, separately diagnosticTests use a scoped
tracing_subscriberlayer (same pattern as the existing stall-warn tests inllm.rs) to exercise the WARN seam directly:Noneusage (no publish, no text) → 1 WARNWhy
Recurring silent-death incident in a specific agent×channel combination: sessions die with 1 LLM call, 2–12 output tokens, no tool calls, no message, no error — recorded as a "successful" turn. The harness log shows the token count but not the
stop_reason, leaving the root cause undiagnosable without request-level tracing. The observed shape also includes tool-step-then-3-token-death (one tool call, then silence) — the publish-aware gate catches both shapes.Context thread: buzz://message?channel=91fd9ca1-cf04-4ef7-b18f-aa2aee55692b&id=e3f1693f2e29f26a0c840f8054d592270c1504beacdc1d9c2063d8ab82960a06
Scope
Logging and telemetry only. No behavior change, no retry-logic change, no stop-reason mapping change.