-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
feat(chat): hand run() a streamText with the managed options already applied #4884
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
c6c35d5
test: cover the head-start accumulator seed without hydrateMessages
ericallam 5fef13f
fix(chat): put injected steering messages into the accumulator
ericallam 4b0f52a
fix(chat): persist history an action rolled back
ericallam e399b60
fix(chat): make a response streamed from onAction part of the convers…
ericallam 52bd98d
fix(chat): route a system-role injection to the instructions lane
ericallam 47f2f8f
fix(chat): address review on the accumulator, instructions and action…
ericallam 52772b5
fix(chat): report a failed action stream instead of committing it as …
ericallam c6b0dbd
docs(chat): note the instructions delivery path and one-shot injectio…
ericallam e3318cf
docs(ai-chat): say what an action persists under each persistence model
ericallam 02e0a70
docs(ai-chat): delete the replaced answer in the regenerate example
ericallam e6618cf
docs(ai-chat): style-guide pass on the injection and action sections
ericallam 1f7fb5e
docs(ai-chat): drop em dashes from the docs and changesets
ericallam 3246d12
docs(ai-chat): drop the remaining em dashes from the actions and inje…
ericallam 86d67fa
docs(chat): warn about the two upgrade hazards in the changesets
ericallam e2737f6
fix(chat): consume injected instructions per turn, not per options build
ericallam 692c060
fix(chat): keep an injection made during the turn that consumed the lane
ericallam 900418d
fix(chat): rebuild the model messages after a steering injection
ericallam 83fe5ef
test(chat): cover the model-lane rebuild on a turn that captures no r…
ericallam 6683e4b
fix(chat): keep a steer in the lanes on the createSession surface
ericallam bf82221
fix(chat): append a steer to the model lane instead of rebuilding it
ericallam dd06745
fix(chat): report a drained steer from a turn that fails
ericallam a0a07bb
test(chat): pin a one-shot instruction across an action
ericallam c9fe897
test(chat): use the spread form in the action-instruction test
ericallam 4f535ac
fix(chat): keep a steer's prepared form for later turns
ericallam 39ea541
fix(chat): reconcile a steer into the model lane when the turn fails
ericallam 920c76d
fix(chat): keep a failed action from counting as a turn
ericallam e348d61
fix(chat): move the snapshot cursor on a failed turn
ericallam 0ef1caa
docs(chat): cover the second review round in the changesets
ericallam ae72c70
fix(chat): report a steer in the turn delta, and once after a history…
ericallam 62fd77c
fix(chat): keep a prepared steer through a history edit and a failed …
ericallam 6482650
fix(chat): deliver plain onAction replies, and keep compaction throug…
ericallam 472eaf4
fix(chat): keep an instruction injected after an action for the next …
ericallam 023432d
feat(chat): send actions through useChat so their turns render
ericallam d332a2a
feat(chat): let an action become a turn with chat.turn()
ericallam 0cf59a2
fix(chat): persist an action's edit before its turn, and label that turn
ericallam 8d5d466
fix(chat): merge an action's metadata over the transport's clientData
ericallam ee8f449
docs(chat): correct three changesets after the chat.turn() redesign
ericallam cd39f8b
feat(chat): hand run() a streamText that already has the managed options
ericallam 447c910
fix(sdk): fall back to the agent's tools in the managed streamText
ericallam 53b588e
fix(chat): apply agent-level options in chat.toStreamTextOptions
ericallam 977c22a
refactor(chat): drop the streamText handed to onAction
ericallam 0b66e8d
docs(chat): describe actions as edits that can become turns
ericallam d4458ea
docs(chat): the turn an action requests carries trigger action-turn
ericallam 5728314
fix(chat): publish the agent options for hydrateMessages agents, and …
ericallam 6db5dcb
docs(chat): describe the action-turn branch, and fix two Head Start t…
ericallam bf66457
docs(chat): list prompt among the Head Start owned options in the cha…
ericallam File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions can now become turns. `onAction` edits history with `chat.history`; to answer after the edit, return `chat.turn()` and a turn runs on the edited history with everything a turn has: the agent's system prompt and tools, steering, compaction, injected instructions, `onTurnStart` and `onTurnComplete`, and persistence. A regenerate is `chat.history.slice(0, -1); return chat.turn();`. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action }) => { | ||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| return chat.turn(); | ||
| } | ||
| if (action.type === "undo") chat.history.slice(0, -2); // edit only | ||
| }, | ||
| ``` | ||
|
|
||
| Returning a `StreamTextResult`, `string` or `UIMessage` from `onAction` is no longer supported and now fails with an error pointing to `chat.turn()`. A response produced that way skipped every turn guarantee, and its delivery to the browser was unreliable: the frontend never read the stream `transport.sendAction` returned, so a regenerate that appeared to work on the server did not render. | ||
|
|
||
| History edits made by an action are still persisted as before: platform-managed snapshots are written after the edit, and apps with their own store mirror the edit themselves. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages are now kept in the conversation when you drive turns yourself with `chat.createSession()` or `chat.MessageAccumulator`. Previously a message that arrived mid-answer shaped that answer and then existed nowhere: it was missing from `turn.uiMessages`, so an app persisting from there never stored it, missing from `turn.messages`, so every later turn answered as though it had never been sent, and it was not queued as its own turn either. It now lands in both, the same way it does on `chat.agent`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Injected system context is merged into a single instruction block, so it works on every supported AI SDK version. Note that a cached system prompt gives up its cache entry for as long as an injection is live, since the cached prefix has changed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| `chat.inject()` with `role: "system"` now works. It previously put the system message into the conversation, which AI SDK 7 rejects for every provider: the next turn died with a generic "An error occurred." and persisted an empty assistant message, so the agent looked like it had stopped answering. System-role context is now appended to the model's instructions, which is also the only way to inject context the agent treats as trusted. | ||
|
|
||
| Two things to know. Instructions are delivered by `chat.toStreamTextOptions()`, so a `run()` that calls `streamText` without spreading it does not receive a system-role injection. The conversational lane has no such requirement. And an injection applies to the next turn only, rather than repeating on every turn that follows it. Every inference call in that turn sees it, so a `run()` that builds options more than once gets the same instructions each time. An instruction injected after an action has run, and before the next message, reaches that next turn rather than the one after it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| `run()` now receives a `streamText` with your agent's managed options already applied, so they cannot be lost by leaving out the spread: | ||
|
|
||
| ```ts | ||
| run: async ({ messages, signal, streamText }) => | ||
| streamText({ model, messages, abortSignal: signal }); | ||
| ``` | ||
|
|
||
| Spreading `chat.toStreamTextOptions()` still works and is equivalent. The difference is what happens when your options collide with the managed ones. Passing `tools` after the spread replaces the skill tools, and passing your own `prepareStep` replaces the managed one, which silently switches off steering, compaction and injected context. The managed `streamText` merges tools and composes `prepareStep` instead, so neither can be turned off by accident. | ||
|
|
||
| `system` can be set at the call site, on `chat.agent({ system })`, or through `chat.prompt.set()`, but only in one of them: setting it in two places throws, because no single shape merges two system values across every supported AI SDK version, and dropping one silently is the failure this seam exists to prevent. Injected instructions append to whichever one is in play. | ||
|
|
||
| `chat.agent()` also takes `registry`, `cacheControl` and `systemProviderOptions` now, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site. `chat.toStreamTextOptions()` applies them as well, so spreading it into the `streamText` imported from `ai` stays equivalent to the one `run()` receives. | ||
|
|
||
| `chat.headStart` and `chat.startHeadStart` hand their `run` the same thing, carrying the options the handover protocol depends on. There it matters more: re-setting `messages`, `prompt`, `stopWhen` or `abortSignal` after a spread breaks the handover rather than degrading a feature, and nothing caught it. On the managed one those four keys are a type error; `tools` is yours to pass. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Undo, edit and regenerate now survive a run ending. History rolled back from `onAction` was only kept in the running worker's memory, so the rollback held while that worker stayed warm and then reverted on the next continuation. The undone messages came back, minutes later, with no error. This also holds when the turn before the action failed: the rollback used to be written against the cursor from before that turn, so a continuation could replay output the failed turn had already superseded. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages injected mid-answer are now part of the conversation, both for your hooks and for the model on later turns. Previously they reached the model for the answer they steered and reached the browser, but nothing else: `onTurnComplete` never saw them, so an app storing its own transcript lost the instruction the answer was shaped by, and it vanished from the conversation on reload. The model also forgot the instruction from the next turn onwards, answering as though the message had never been sent, while the chat UI still showed it. This holds when the steered turn fails part-way, and when `pendingMessages.prepare` reshapes the message: later turns now see the same form the steered turn did, not the original message. | ||
|
|
||
| Approving a tool call no longer undoes compaction. A tool-approval continuation used to rebuild the model's context from the full conversation, so a chat that had been summarised to fit the context window was sent the whole transcript again on the next call, and could go over the limit it had just been compacted to avoid. | ||
|
|
||
| If you worked around this by saving steering messages as they arrive, in `pendingMessages.onReceived` for example, that write now duplicates the one you get from `newUIMessages`. Drop it, or skip messages you have already stored. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions are sent through `useChat` so a turn that follows one renders like any turn. `TriggerChatTransport` recognises `body.action` on a `useChat` request and sends it as an action, so `sendMessage(undefined, { body: { action } })` or `regenerate({ body: { action } })` sends the action and `useChat` owns the response: it streams into the message list, `status` and `error` behave as for a message, and `stop` works. `useChatActions({ sendMessage })` in `@trigger.dev/sdk/chat/react` is a two-line convenience over that. | ||
|
|
||
| ```tsx | ||
| const { sendMessage } = useChat({ id: chatId, transport }); | ||
| const { sendAction } = useChatActions({ sendMessage }); | ||
| sendAction({ type: "regenerate" }); | ||
| ``` | ||
|
|
||
| Previously the frontend docs said `useChat` consumed the stream `transport.sendAction` returns; it never did, so an action's answer was never rendered by an app following them. `transport.sendAction` still returns a stream that callers outside `useChat` must read, and now accepts `{ abortSignal, metadata }`, with per-action metadata merged over the transport's `clientData`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.