Skip to content

Tell the LangGraph Bot's model a refused tool call was refused, not that it returned nothing - #518

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:fix/langgraph-refused-tool-call
Sep 13, 2026
Merged

Tell the LangGraph Bot's model a refused tool call was refused, not that it returned nothing#518
davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:fix/langgraph-refused-tool-call

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

What happens

A LangGraph Bot (agent-langgraph) whose tool call the deployment will not run — it holds an agent token the deployment no longer accepts, or a token issued to a different Bot — asks a governed tool something:

Do we have a PRD for this in Drive?

and answers:

No files were returned from Google Drive for that search.

The Drive has the files. The call never reached it.

Why

/api/agent-tools/call refuses a callback it cannot verify before any grant is consulted (app.ts):

return context.json({ error: verdict.reason }, verdict.status);
// 401 { "error": "Not authorised." }
// 403 { "error": "That token is not for this Bot." }

callTool in agent-langgraph/src/index.ts reads every answer the same way, whatever its status:

const body = (await response.json()) as { text?: string };
return body.text ?? "The tool returned nothing.";

A refusal has no text, so the model is handed The tool returned nothing. and reports an absence.

This is the exact failure #135 found and #136 addressed on the server side: that PR added the mcp.callback_refused audit row so the trail stops agreeing nothing happened, and its CHANGELOG entry describes the Bot "returned nothing to its own model, and the model told the person there were no results". The trail was fixed; what the TypeScript Bot tells its model was not.

The Python LangGraph Bot already handles this response. agent-langgraph-agui/src/tool_runtime.py:

if not response.is_success:
    return (f"Refused. Tool callback returned HTTP {response.status_code}.", True)

The change

The reading of the response moves out of index.ts into agent-langgraph/src/tool-answer.ts, for the reason history.ts, deltas.ts and stream.ts are their own modules: index.ts calls serve() at module scope, so a test cannot import it.

  • A response that is not a success becomes Refused. Tool callback returned HTTP <status>., the sibling's wording, followed by the deployment's own error when it sent one: Refused. Tool callback returned HTTP 401. Not authorised.
  • It leads with Refused., the marker the transcript reads (app/src/lib/plugins/tool-result.ts), which callTool's other two refusals in the same function already start with. So the transcript draws it as a refusal rather than a result.
  • A success is read exactly as before, byte for byte.

callTool keeps its try/catch, so a network failure or an unreadable success body is still reported as That tool could not be called: ….

Where it runs

In the agent-langgraph process, once per tool call, on the answer from the deployment.

  • New state that outlives a request? None. A pure function of one response.
  • What happens on the second replica? Nothing different. Nothing is held.
  • Anything serialised? No.
  • Anything fanned out to a browser? No new fan-out. The tool result already reaches the transcript through the existing TOOL_CALL_RESULT event; only its text changes for a refused call.
  • New listener, port, or schedule? None.

Boundary and audit

  • Every acting call still goes through the gateway. Nothing about the call changes, only how the Bot reads the refusal.
  • New refusals and new failures each write a row. No new refusal: the server already writes mcp.callback_refused for this one.
  • Nothing new is trusted from the client. The reason shown is the deployment's own sentence.

Changelog

A line at the top of Unreleased, because a person asking this Bot now hears that the call was refused instead of that nothing was found.

Proof

bun test tests/tool-answer.test.ts in agent-langgraph, with the tests applied and tool-answer.ts holding callTool's current reading moved verbatim (absolute paths shortened to the repository root, nothing else edited):

tests\tool-answer.test.ts:
31 |   test("a callback the deployment would not accept is a refusal, not an empty result", async () => {
32 |     // The shape of a Bot holding a token the deployment no longer accepts: every call answers 401
33 |     // with the reason under `error` and no `text`. Told "the tool returned nothing", the model tells
34 |     // the person nothing was found.
35 |     const answer = await toolAnswer(json({ error: "Not authorised." }, 401));
36 |     expect(answer).not.toBe("The tool returned nothing.");
                            ^
error: expect(received).not.toBe(expected)

Expected: not "The tool returned nothing."

      at <anonymous> (agent-langgraph\tests\tool-answer.test.ts:36:24)
(fail) what a tool call came back with > a callback the deployment would not accept is a refusal, not an empty result [10.85ms]
41 | 
42 |   test("a token issued to another Bot is a refusal that says so", async () => {
43 |     const answer = await toolAnswer(
44 |       json({ error: "That token is not for this Bot." }, 403),
45 |     );
46 |     expect(answer.startsWith("Refused.")).toBe(true);
                                               ^
error: expect(received).toBe(expected)

Expected: true
Received: false

      at <anonymous> (agent-langgraph\tests\tool-answer.test.ts:46:43)
(fail) what a tool call came back with > a token issued to another Bot is a refusal that says so [1.58ms]
1 | export async function toolAnswer(response: Response): Promise<string> {
2 |   const body = (await response.json()) as { text?: string };
                                   ^
SyntaxError: Failed to parse JSON
      at toolAnswer (agent-langgraph\src\tool-answer.ts:2:32)
      at <anonymous> (agent-langgraph\tests\tool-answer.test.ts:51:26)
(fail) what a tool call came back with > a failure with no readable body still says it failed [1.08ms]

 2 pass
 3 fail
 4 expect() calls
Ran 5 tests across 1 file. [67.00ms]

The 401 and 403 failures are the bug. The third, a 502 with an HTML body, is not a user-facing failure on main: the parse error it throws is caught by callTool and reported as That tool could not be called: Failed to parse JSON. It is there to pin that the moved function answers a non-JSON failure without throwing.

With the change: 5 pass, 0 fail.

Not a widening. The first two tests pass before and after. One pins that a successful result reaches the model as the deployment wrote it; the other pins that a refusal the store already made (a 200 whose text starts with Refused.) is passed on untouched. So the only answers this changes are the ones that were not a success.

Whole package (bun test in agent-langgraph):

main this PR
agent-langgraph 33 pass, 0 fail 38 pass, 0 fail

The delta is exactly the five tests added.

bunx @biomejs/biome check is clean on tool-answer.ts and its test. On index.ts it reports one import-order error, on import { readReasoningEffort } from "./model-options" sitting above ./model-key. That is on main too (checked against an unmodified copy), and the one import this PR adds is already in order, so I have left that line alone. bunx prettier --check is clean on both new files; on index.ts and CHANGELOG.md it reports the same style issues it reports on main, none in lines added here.

🤖 Generated with Claude Code

When /api/agent-tools/call would not run a call from the LangGraph Bot,
it answered 401 or 403 with the reason under `error` and no `text`.
callTool read that as a result and told the model "The tool returned
nothing.", so the model told the person nothing was found.

A response that is not a success is now a refusal, worded the way
agent-langgraph-agui already words it, with the deployment's reason
after it. The reading moves to its own module so it can be tested
without binding a port. A successful answer is passed on unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep-reviewed clean (validation, no secret leak, fail-closed, agrees with existing layers). CI green.

@davidmckayv
davidmckayv merged commit a3e7abd into CopilotKit:main Sep 13, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants