fix: Route thread-pool and live-mode tool dispatch through run_async (v1) - #6799
Open
GWeale wants to merge 2 commits into
Open
fix: Route thread-pool and live-mode tool dispatch through run_async (v1)#6799GWeale wants to merge 2 commits into
GWeale wants to merge 2 commits into
Conversation
When `RunConfig.tool_thread_pool_config` was set, a synchronous `FunctionTool` was not run through `BaseTool.run_async` at all. The thread-pool helper re-implemented a subset of it inside the worker thread: it converted arguments, injected the tool context, filtered parameters, and then called `tool.func(...)` directly. Everything else `run_async` does was skipped, so a tool declared with `require_confirmation=True` executed without a confirmation ever being requested, and a call missing a mandatory argument raised `TypeError` from the worker thread instead of returning the standard error dict. The helper now calls `tool.run_async` on the caller's event loop and offloads only the synchronous callable. A module-level context variable in `function_tool.py` holds a runner that `FunctionTool._invoke_callable` uses for that offload; the worker rebinds it to `None` so a nested `FunctionTool` call does not reuse the caller loop's runner. Caller context variables are copied across the offload as before. Behaviour change: a synchronous tool that exposes a `func` attribute but is not a `FunctionTool` — `SetModelResponseTool` is the in-tree example — used to run on the caller's event loop. It is now dispatched to a worker thread running its own event loop, like every other non-`FunctionTool`. Ported from google/adk-python main.
In a live session, a streaming tool (one whose wrapped function is an async generator) was executed by `FunctionTool._call_live`, which built its own argument dict and called the wrapped function directly. It never went through `BaseTool.run_async`, so everything `run_async` does was skipped: the confirmation check added in #6575, Pydantic argument coercion, mandatory-argument validation, and dispatch to a subclass that overrides `run_async`. A tool declared with `require_confirmation=True` ran unattended. `_call_live` and its `__call_tool_live` wrapper are removed. The live streaming branch now calls `__call_tool_async`, which calls `tool.run_async`, and inspects what comes back: an async generator is drained into the live request queue as before, while a plain value — the error dict `run_async` returns when confirmation is required or rejected, or when a mandatory argument is missing — is relayed as a single FunctionResponse. The `input_stream` injection that `_call_live` performed moves into `FunctionTool._prepare_invocation_args`, and `_get_mandatory_args` no longer counts the framework-supplied `tool_context` and `input_stream` parameters as mandatory. Behaviour changes a live-mode caller will see: - A tool behind `require_confirmation` is refused instead of executed, and a confirmation request is recorded on the tool context. Confirmation is still not answerable in live mode: the flow does not emit an `adk_request_confirmation` function call and the live path does not accept a `ToolConfirmation`, so the tool cannot be approved and resumed. TODOs mark the three sites that have to change to close that loop. - Arguments annotated with a Pydantic model now arrive as model instances rather than raw dicts. - A `BaseTool` subclass that overrides `run_async` is now dispatched through that override instead of having its `func` called directly. - A streaming tool that raises now sends an error FunctionResponse. Previously the exception escaped the background task and the session waited forever for a response that never came. The exception text is logged, not sent to the model. - A streaming tool invoked outside live mode, where there is no active stream to inject, now raises `TypeError` for the missing `input_stream` argument instead of returning the "mandatory input parameters are not present" error dict. Such a tool was already broken there; only the shape of the failure changed. `FunctionTool._call_live` is private and had one caller, but a third-party subclass that overrides it will silently stop being called. Adapted from google/adk-python main: `LiveRequestQueue.send_content` on this branch takes only a content argument, so the upstream `partial=` keywords are dropped. The two comment-only changes upstream made to `base_llm_flow.py` are also omitted, because one of them anchors on a line that does not exist here.
GWeale
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports two tool-dispatch fixes to
v1, following #6575, so every dispatch sitein
functions.pygoes throughBaseTool.run_async.Thread-pool dispatch (
0156bc5a)_call_tool_in_thread_poolcallstool.run_asyncon the caller's eventloop and offloads only the synchronous callable.
RunConfig.tool_thread_pool_configset, confirmation is enforced anda call missing a mandatory argument returns the standard error dict.
FunctionToolexposingfunc(in-tree:SetModelResponseTool) runs on a worker thread with its own event loop.Live-mode dispatch (
8b9d2222)run_async, so arun_asyncoverride ishonoured and Pydantic-annotated arguments arrive as model instances.
FunctionResponse.require_confirmation=Trueis refused in live mode and cannotyet be approved there;
TODOs infunctions.pymark the remaining work.FunctionTool._call_liveis removed; a subclass overriding it is notcalled.