Skip to content

feat(mcp): centralise ToolAnnotations across public + SDK tool surface - #87

Open
louzt wants to merge 4 commits into
appwrite:mainfrom
louzt:feat/mcp-tool-annotations
Open

feat(mcp): centralise ToolAnnotations across public + SDK tool surface#87
louzt wants to merge 4 commits into
appwrite:mainfrom
louzt:feat/mcp-tool-annotations

Conversation

@louzt

@louzt louzt commented Jul 27, 2026

Copy link
Copy Markdown

Rebase onto MCP Python SDK v2 (#89)

Rebased this PR onto f37d118 (PR #89 by @ChiragAgg5k, "Migrate to MCP Python SDK v2", merged 2026-07-29). The SDK migration is orthogonal to the safety annotations work:

  • types.Tool.annotations is stable across MCP Python SDK v1 and v2.
  • upstream/main at f37d118 contains zero references to ToolAnnotations (git grep -nE 'ToolAnnotations|annotations_for' upstream/main returns 0 matches). PR feat(mcp): centralise ToolAnnotations across public + SDK tool surface #87 remains the only PR introducing centralised ToolAnnotations.
  • The migration's snake_case field aliases (input_schema, output_schema) on Tool apply to the schema structure, not to the annotations field. No collision.

The rebase produced two file conflicts:

Note on intervening PRs (#90, #91, #92)

Three PRs merged after #89 and touch files in this PR's scope:

Rebased onto f37d118 (the SDK v2 migration). When this PR is reviewed and merged, please rebase onto the current main so the diff against upstream reflects only PR #87's additions.

Design evolution vs the original proposal in #86

The original proposal in #86 suggested destructiveHint=False for appwrite_call_tool. During review, Greptile Bot flagged that this contradicts the MCP 2025-06-18 spec default semantics: setting the hint explicitly to False signals "definitely safe", which is wrong for a dispatcher whose inner behaviour depends on the target tool.

This PR resolves it with annotations_for_gateway(), a dedicated helper that returns ToolAnnotations(destructiveHint=None, ...). Per the MCP 2025-06-18 spec, an unset destructiveHint defaults to True on the client side, which is the honest semantics for a gateway. Clients that gate human approval on the hint correctly prompt the user for every gateway call, while confirm_write=true stays as the per-call inner gate for Appwrite's destructive SDK calls. The two layers cooperate; neither is a workaround for the other.

On benchmarks

A separate test/mcp-benchmarks branch holds an earlier tests/unit/test_benchmarks.py draft. It was deliberately not included in this PR for three concrete reasons:

  1. Time-based assertions are flaky on CI (e.g. assert elapsed < 0.25 for _clamp_limit). CI runners are routinely 2 to 5x slower than developer machines, so the assertion becomes a coin flip.
  2. One assertion contradicts this PR's design. It asserts tool.annotations.title == "Appwrite Documentation Search", but annotations_for_classification returns ToolAnnotations(title=None, ...). The test would fail against the current implementation.
  3. Operator(manager, executor, docs_search=...) is not callable. The rebased Operator.__init__ types execute_tool as Callable[...]. ThreadPoolExecutor is not directly callable (it has submit()); the runtime throws TypeError when Operator tries to dispatch.

If reviewers want deterministic performance signals, the right vehicle is a separate PR using pytest-benchmark (with warmup, multi-iteration statistics, and stddev reporting), not assertions on raw wall-clock elapsed.

Local validation on the rebased tip

  • python -m unittest discover -s tests/unit passes on mcp 2.0.0 (the version PR Migrate to MCP Python SDK v2 (dual-era protocol support) #89's migration requires). Local environments with mcp 1.27.2 hit a Pydantic ValidationError on input_schema= because the field name in 1.x was inputSchema (camelCase). Upstream CI runs mcp 2.0.0 and is green.
  • ruff check src tests, black --check src tests, pyright all pass.

Effective diff

git diff upstream/main...HEAD (at f37d118):

 src/mcp_server_appwrite/annotations.py | 236 ++++++++++++
 src/mcp_server_appwrite/docs_search.py |   8 +
 src/mcp_server_appwrite/operator.py    |  27 ++
 src/mcp_server_appwrite/service.py     |   8 +
 tests/unit/test_annotations.py         | 639 +++++++++++++++++++++
 5 files changed, 918 insertions(+)

@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown

Greptile Summary

Centralizes MCP tool safety annotations and applies them across the public, documentation-search, and dynamically generated SDK tool surfaces.

  • Classifies SDK tool names into read, write, delete, and unknown categories.
  • Uses a dedicated conservative annotation profile for the dynamic gateway.
  • Adds unit coverage for classification, annotation mappings, gateway behavior, and generated tools.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported gateway issue is resolved because appwrite_call_tool no longer promises non-destructive behavior and instead leaves destructiveHint unset for host-policy handling.

Important Files Changed

Filename Overview
src/mcp_server_appwrite/annotations.py Adds centralized classification and annotation helpers, including an unset destructive hint for the gateway.
src/mcp_server_appwrite/operator.py Applies read annotations to lookup tools and the dedicated gateway profile to appwrite_call_tool.
src/mcp_server_appwrite/service.py Adds verb-derived annotations to dynamically generated SDK tools.
src/mcp_server_appwrite/docs_search.py Marks the documentation-search tool as read-only.
tests/unit/test_annotations.py Adds broad regression coverage for classifications, annotation profiles, and emitted tool definitions.

Reviews (4): Last reviewed commit: "fix(rebase): drop orphaned 'from . impor..." | Re-trigger Greptile

Comment thread src/mcp_server_appwrite/operator.py
louzt added a commit to louzt/appwrite-mcp that referenced this pull request Jul 27, 2026
appwrite#87 review)

Greptile flagged appwrite#87 operator.py:263 as P1: the
appwrite_call_tool gateway dispatched to delete-classified SDK
methods while advertising destructiveHint=False, letting clients
that gate human approval on the hint treat a destructive tool as
non-destructive.

Fix: introduce annotations_for_gateway() in annotations.py.
destructiveHint is intentionally set to None (unset) — the MCP
2025-06-18 spec defaults an unset hint to True, so clients default
to destructive and prompt the user for every gateway call. This
matches the runtime confirm_write=true boundary enforced inside
_call_hidden_tool, but at the layer where MCP clients (Claude Code,
Gemini MCP, Appwrite broker) actually make the approval decision.

annotations_for_classification("unknown") is preserved for verb
buckets we genuinely cannot derive; the gateway's profile is
intentionally separate because the security semantics differ
(unknown cannot promise non-destructive; gateway cannot promise
anything static).

- 5 new tests in AnnotationsForGatewayTests
- test_appwrite_call_tool_is_unknown renamed and updated to assert
  destructiveHint is None (with a regression-guard comment)
- AnnotationsModuleSurfaceTests now covers the new public name

CI: ruff ✓, black ✓, pyright ✓, unittest 127/127 ✓.
@louzt
louzt force-pushed the feat/mcp-tool-annotations branch from 94a8ef1 to 42385cf Compare July 27, 2026 18:35
louzt added a commit to louzt/appwrite-mcp that referenced this pull request Jul 27, 2026
appwrite#87 review)

Greptile flagged appwrite#87 operator.py:263 as P1: the
appwrite_call_tool gateway dispatched to delete-classified SDK
methods while advertising destructiveHint=False, letting clients
that gate human approval on the hint treat a destructive tool as
non-destructive.

Fix: introduce annotations_for_gateway() in annotations.py.
destructiveHint is intentionally set to None (unset) — the MCP
2025-06-18 spec defaults an unset hint to True, so clients default
to destructive and prompt the user for every gateway call. This
matches the runtime confirm_write=true boundary enforced inside
_call_hidden_tool, but at the layer where MCP clients (Claude Code,
Gemini MCP, Appwrite broker) actually make the approval decision.

annotations_for_classification("unknown") is preserved for verb
buckets we genuinely cannot derive; the gateway's profile is
intentionally separate because the security semantics differ
(unknown cannot promise non-destructive; gateway cannot promise
anything static).

- 5 new tests in AnnotationsForGatewayTests
- test_appwrite_call_tool_is_unknown renamed and updated to assert
  destructiveHint is None (with a regression-guard comment)
- AnnotationsModuleSurfaceTests now covers the new public name

CI: ruff ✓, black ✓, pyright ✓, unittest 127/127 ✓.
louzt added 4 commits July 31, 2026 14:43
Adds ToolAnnotations (readOnlyHint, destructiveHint, idempotentHint,
openWorldHint, title) to the 4 public MCP tools exposed by Appwrite MCP
per AGENTS.md §Tool surface:

- appwrite_get_context: readOnly, idempotent, openWorld (calls Cloud APIs)
- appwrite_search_tools: readOnly, idempotent, NOT openWorld (local catalog)
- appwrite_call_tool: NOT readOnly, NOT idempotent, openWorld (dispatches
  to Appwrite SDK with confirm_write=true gate on mutating calls)
- appwrite_search_docs: readOnly, idempotent, NOT openWorld (local index)

Each hint carries an inline comment explaining the semantic rationale,
matching the pattern already established in serpapi-mcp-fork so
reviewers can audit the readOnly/destructive/idempotent/openWorld
choices at a glance.

Includes tests/unit/test_annotations.py with 13 unittest-style tests
covering: tool surface count + uniqueness, every-hint-is-bool shape,
readOnly implies not-destructive, readOnly implies idempotent,
appwrite_call_tool honest semantics, search tools are not open world,
and human-readable title presence. All 111 unit tests pass.

Per the MCP 2025-06-18 spec, leaving a hint unset means 'unknown' and
forces clients to prompt the user. With these annotations, MCP clients
(Claude Code, Cursor, Antigravity) can safely auto-execute the
read-only context and search tools while keeping write/delete calls
gated on the existing confirm_write=true mechanism.
Expand the ToolAnnotations work from 4 public tools to cover every tool the
server can emit (4 public + ~25 dynamic SDK services + 1 docs tool), driven
by a single helper module so the hints stay in lock-step with the action
verb in the tool name.

What's in the diff:

- New module ``src/mcp_server_appwrite/annotations.py`` exporting two pure
  helpers:

  - ``classify_tool_name(tool_name)`` — extract the action verb and bucket
    the tool as ``read`` / ``write`` / ``delete`` / ``unknown``.
  - ``annotations_for_classification(bucket)`` — return a canonical
    ``ToolAnnotations`` instance with every hint set explicitly. The inline
    rationale comments document why each hint has its value (e.g. why
    ``unknown.openWorldHint=True`` despite not knowing the verb, why
    ``delete.idempotentHint=True``).

- ``service.py``: ``Service.list_tools()`` now classifies each SDK method
  by its action verb and attaches the resulting annotations. This covers
  every method on every service the Appwrite SDK ships (~25 services,
  hundreds of methods).

- ``operator.py``: the 3 public tools now use ``annotations_for_classification``
  instead of inline ``ToolAnnotations(...)`` constructions. The old inline
  blocks were removed in the same commit to keep the diff reviewable.

- ``docs_search.py``: the docs search tool uses the helper for its
  ``read`` annotation.

Why a single helper instead of inline annotations per tool:

- 30 tools × 4 hints = 120 places where drift can creep in. One bug in
  ``classify_tool_name`` would silently mislead every MCP client that
  filters on these hints (claude-code, Gemini MCP, Appwrite broker).
- Pure functions make the mapping exhaustively testable.
- Future SDK methods automatically get the right hints without anyone
  having to remember the pattern.

Test coverage expansion (from 13 → 41 net new tests):

- ``classify_tool_name`` parametrized over 19 read verbs, 19 write verbs,
  10 delete verbs, 4 unknown names, case-insensitivity, verbs in the
  middle of names.
- ``annotations_for_classification`` for every bucket; explicit-field check
  catches MCP-default leakage; unrecognised inputs fall through to
  ``unknown``.
- Parity with ``operator._classify_verb`` for every known verb × resource
  combination — both helpers stay in lock-step.
- Public tools: ``appwrite_get_context`` / ``appwrite_search_tools`` are
  read; ``appwrite_call_tool`` is the gateway (``unknown``); the docs
  search tool is read.
- Dynamic SDK tools: ``Service.list_tools()`` is invoked on a stub users
  service and the resulting ``Tool`` objects carry annotations consistent
  with the verb bucket.

The "consistent with" check accounts for the MCP spec limitation that
write and unknown collapse to the same wire shape on the four hints.

Total: 122 unit tests pass locally. All four CI jobs (ruff, black,
pyright, unittest) green.

Refs appwrite#86 — supersedes the 4-tool-only draft.
appwrite#87 review)

Greptile flagged appwrite#87 operator.py:263 as P1: the
appwrite_call_tool gateway dispatched to delete-classified SDK
methods while advertising destructiveHint=False, letting clients
that gate human approval on the hint treat a destructive tool as
non-destructive.

Fix: introduce annotations_for_gateway() in annotations.py.
destructiveHint is intentionally set to None (unset) — the MCP
2025-06-18 spec defaults an unset hint to True, so clients default
to destructive and prompt the user for every gateway call. This
matches the runtime confirm_write=true boundary enforced inside
_call_hidden_tool, but at the layer where MCP clients (Claude Code,
Gemini MCP, Appwrite broker) actually make the approval decision.

annotations_for_classification("unknown") is preserved for verb
buckets we genuinely cannot derive; the gateway's profile is
intentionally separate because the security semantics differ
(unknown cannot promise non-destructive; gateway cannot promise
anything static).

- 5 new tests in AnnotationsForGatewayTests
- test_appwrite_call_tool_is_unknown renamed and updated to assert
  destructiveHint is None (with a regression-guard comment)
- AnnotationsModuleSurfaceTests now covers the new public name

CI: ruff ✓, black ✓, pyright ✓, unittest 127/127 ✓.
… rebase

The rebase onto upstream/main (PR appwrite#84 SDK 22.2.0) auto-merged the import
but lost the telemetry.record_search_docs(...) call that my original
9f443b6 commit added. Upstream 8a962e4 ('cleanup: stop emitting metrics
the MCP dashboard no longer queries') had already removed both, so the
import was no longer needed. Removing the orphan fixes ruff F401.

Verified locally on this branch:
- ruff check src tests             ✓ All checks passed!
- black --check src tests          ✓ 40 files unchanged
- pyright                          ✓ 0 errors, 0 warnings, 0 informations
- python -m unittest discover -s tests/unit   ✓ 210 tests, OK
@louzt
louzt force-pushed the feat/mcp-tool-annotations branch from 42385cf to 8c14950 Compare August 6, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant