Skip to content

feat: tool robustness + usability (INVALID_PARAM, uniform response envelope, execute_code skip_refresh, screenshot auto-file)#34

Open
dehuaichendragonplus wants to merge 1 commit into
FunplayAI:mainfrom
dehuaichendragonplus:feat/tool-robustness-and-usability
Open

feat: tool robustness + usability (INVALID_PARAM, uniform response envelope, execute_code skip_refresh, screenshot auto-file)#34
dehuaichendragonplus wants to merge 1 commit into
FunplayAI:mainfrom
dehuaichendragonplus:feat/tool-robustness-and-usability

Conversation

@dehuaichendragonplus

Copy link
Copy Markdown
Contributor

Why this change is needed

Four independent reliability / ergonomics problems an AI agent driving Unity through this MCP hits in practice:

  1. Silent wrong writes on malformed arguments. FunctionInvokerController.ConvertValue wraps all parsing in try { ... } catch { return default(T); }, and GameObjectFunctions.ParseVector3 returns Vector3.zero when the string doesn't split into 3 parts. So set_transform(position:"1,2") silently moves the object to the origin and reports success; a non-numeric int silently becomes 0. The caller gets a success response for an operation that did the wrong thing, with no signal. This is the most dangerous class — it produces incorrect state invisibly.
  2. Errors reported as success. Several Profiler/memory tools return precondition/error states as bare human-readable strings ("target is required.", "No snapshots found.", ...). The plugin's own ToolResultFormatter.IsError() only recognizes failure by parsing {success:false}, so these read as success — a polling caller can loop forever or mis-report a failure as data.
  3. Non-uniform response shape. Roughly half the tool families return a bare string on success but structured {success:false,code} JSON on failure, so a client cannot uniformly branch on a success field — the success path is unparseable prose.
  4. Two usability papercuts. execute_code always runs AssetDatabase.Refresh + wait-for-ready before compiling, which in a shared editor (or during Play Mode) can trigger a domain reload that wipes the runtime state you were inspecting. And high-resolution screenshots emit a multi-MB base64 payload that reliably drops the MCP client socket, putting the burden on the caller to know to pass save_to_file.

Change approach

  • Parse failures now propagate as a structured INVALID_PARAM naming the parameter, provided value, and expected format — both in the reflection invoker (a ToolArgumentException carrying context, thrown from BuildArguments and mapped to INVALID_PARAM) and in set_transform / create_primitive's own vector parsing (TryParseVector3; create_primitive validates before creating the object, so a bad value leaves no half-created primitive).
  • The bare-string error returns in ProfilerFunctions become ToolResultFormatter.Error(code, { hint }) (TARGET_REQUIRED, NO_SNAPSHOTS, FRAME_TIMING_UNAVAILABLE, NO_FRAME_DEBUGGER_EVENTS).
  • SerializeResult wraps a legacy tool's bare success string in {success:true,message} — but passes through (a) image data: URIs (still rendered as image content downstream) and (b) strings already in a {success:...} envelope (no double-wrap). Result: every tool response is uniformly {success,...}.
  • execute_code gains an optional skip_refresh (default false) that bypasses the pre-compile refresh.
  • Screenshot FinishCapture auto-falls back to a file when the PNG exceeds ~768 KB (returns {path, bytes, fell_back_to_file:true}); small captures still inline; save_to_file remains an explicit override.

Benefits

  • No more silent wrong writes — the agent gets an actionable error instead of a bad result reported as success.
  • Callers (and the plugin's own IsError) can reliably detect failures across the whole tool surface.
  • Every response is machine-parseable as {success,...} — removes a whole class of client-side parse fragility — while images and existing envelopes are untouched.
  • skip_refresh makes read-only inspection and live Play-Mode debugging safe in a shared editor.
  • Screenshots no longer drop the socket by default on large captures.

Test results

Verified live in a Unity 6000.3.13f1 project (embedded package, compiles with zero errors):

  • create_primitive(position:"1,2"){success:false, code:"INVALID_PARAM", data:{param:"position", provided:"1,2", expected:"Vector3 'x,y,z'", detail:"...got 2"}}, and get_hierarchy confirms no object was created
  • get_object_memory(target:""){success:false, code:"TARGET_REQUIRED"} (was a bare string) ✔
  • get_compilation_errors / memory_list_snapshots success now returned as {success:true,message:"..."} envelopes ✔
  • execute_code(skip_refresh:true) runs a snippet without a refresh ✔
  • FinishCapture via reflection: 800 KB payload → {..., fell_back_to_file:true} saved to file; 16 B → inline data: URI ✔; live captures under the threshold still return inline images ✔

Compatibility testing

  • All changes are at the tool-call response/parameter layer; the MCP wire protocol, initialize handshake, and tool schemas are untouched → compatible with all MCP clients (Claude Code, Cursor, LM Studio, VS Code, Trae, Kiro, Codex).
  • skip_refresh is a new optional parameter (default false) → existing callers unaffected.
  • The envelope wrapping changes a legacy tool's success text from prose to {success:true,message} JSON: still a valid text content block for every client, now additionally machine-parseable; image data: URIs are preserved so image rendering is unchanged; error envelopes are never double-wrapped.
  • The screenshot auto-fallback only triggers above the size at which the base64 path would otherwise drop the socket — strictly better than the previous behavior.

Checklist

  • Tested in a clean Unity 2022.3+ project (Unity 6000.3.13f1)
  • Verified Funplay > MCP Server opens and starts correctly
  • Did not commit local junk such as .idea/ or .DS_Store
  • Updated CHANGELOG.md

…velope, execute_code skip_refresh, screenshot auto-file)

Reliability:
- Malformed tool arguments no longer silently coerce to default(T)/Vector3.zero and run
  with a wrong value. Unparseable values return a structured INVALID_PARAM naming the
  parameter, provided value, and expected format -- in the reflection-based invoker AND in
  set_transform/create_primitive's own vector parsing (create_primitive validates before
  creating the object, leaving no half-created primitive on a bad value).
- Several Profiler/memory tools reported error/precondition states (get_object_memory empty
  target, memory_list_snapshots with none, get_frame_timing unavailable, frame_debugger_get_events
  with none) as bare strings on the success channel, so IsError() and callers treated the
  failure as success. Now structured {success:false,code} errors.
- Every tool response is uniformly parseable as {success,...}: legacy bare-string successes are
  wrapped in {success:true,message} by the serializer, while image data URIs and existing
  {success:...} envelopes pass through untouched (screenshots still render; no double-wrap).

Usability:
- execute_code skip_refresh option bypasses the pre-compile AssetDatabase.Refresh + wait-for-ready
  for read-only snippets or a live Play Mode session you must not disturb (the default refresh can
  trigger a reload that wipes Play state, especially in a shared editor).
- Screenshot captures auto-fall back to a file above ~768KB instead of emitting an oversized
  base64 payload that drops the client socket; returns {path,bytes,fell_back_to_file}. Small
  captures still inline; save_to_file remains an explicit override.

Verified live in a Unity 6000.3.13f1 project (embedded package): compiles clean; malformed
create_primitive position -> INVALID_PARAM with no object created; get_object_memory empty target
-> TARGET_REQUIRED envelope; legacy string successes now returned as envelopes; execute_code
skip_refresh=true runs; FinishCapture(800KB) -> fell_back_to_file=true, (16B) -> inline.

(The CLAUDE.md/AGENTS.md managed-block change is split into its own PR.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant