Skip to content

Record the conversation history as each generation's input - #36

Merged
bezbac merged 3 commits into
langfuse:mainfrom
milanagm:fix/issue-24-via-session-messages
Sep 4, 2026
Merged

Record the conversation history as each generation's input#36
bezbac merged 3 commits into
langfuse:mainfrom
milanagm:fix/issue-24-via-session-messages

Conversation

@milanagm

@milanagm milanagm commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Addresses the conversation-history part of #24. Builds on the analysis in #29, thank you @DavidTraina.
This PR takes a different route to the same goal, described below.

Fixes LFE-15253

Problem

A generation's input held only what arrived since the previous one: the new user message, or the previous assistant message plus the tool results after it. Earlier turns were never repeated, so opening a generation in Langfuse never showed the prompt the model actually received.

What this does

The conversation now comes from OpenCode itself. client.session.messages() serves the whole session, buildSessionHistory maps it to ChatML, and each generation takes the messages before its own assistant message as its input.

  • An assistant message is split at its step-start parts, so the real order survives: tool calls, their results, then the next step — instead of collapsing a multi-step turn into one message.
  • The steps of the running assistant message still come from the live events, which know their order exactly. The store is the source for everything before it.
  • Tool definitions describe the request, not the conversation, so they ride on the newest user message only. Carrying them on every past message would repeat the whole tool list once per earlier turn.
  • The snapshot is only a cache. It is dropped with the rest of the per-session state on session.idle and refetched, so nothing needs releasing by hand.
  • One fetch per step. A generation that arrives without a step-start fetches only when the cache is cold.
  • A failed fetch keeps the previous snapshot and leaves the trace intact rather than stopping it.

Why read the store instead of accumulating in the plugin

#29 accumulates the history in plugin state, which is smaller and needs no I/O. Both produce byte-identical inputs in normal operation. The store wins in the cases where the plugin did not watch the session from its first message:

  • After a restart, an accumulator starts empty. Measured: the next generation's input holds 1 message instead of 3 — and the trace looks correct, just shorter, which is the dangerous kind of wrong for prompt debugging.
  • On a resumed session or a fork, same gap.
  • After session.revert, which removes messages from the store: the plugin handles 14 event types and none of them is revert-related, so an accumulator cannot un-remember. A refresh drops them by construction.

Tests

pnpm run test:integration, 20 passing. Two new cases: a later turn's generation carrying the prior turn, and the history staying complete when the plugin restarts mid-session. The test harness now models OpenCode's message store, which is what makes the restart case testable at all. Existing multi-generation expectations were updated to include the history. format, lint and knip are clean.

Reference PRs - Other integrations:

See Claude Cloud Plugin: langfuse/claude-observability-plugin#80

@bezbac bezbac changed the title feat: record the conversation history as each generation's input Record the conversation history as each generation's input Sep 4, 2026
@bezbac
bezbac enabled auto-merge (squash) September 4, 2026 11:17
@bezbac
bezbac merged commit 8dd173b into langfuse:main Sep 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants