Skip to content

Add centralized OpenTelemetry tracing for MCP tool execution#3111

Open
Copilot wants to merge 19 commits intomainfrom
copilot/add-telemetry-for-custom-tools
Open

Add centralized OpenTelemetry tracing for MCP tool execution#3111
Copilot wants to merge 19 commits intomainfrom
copilot/add-telemetry-for-custom-tools

Conversation

Copy link
Contributor

Copilot AI commented Feb 6, 2026

Why make this change?

MCP tool executions (both built-in DML tools and dynamic custom stored-procedure tools) currently lack explicit OpenTelemetry tracing. While ILogger logging exists, there are no tracing spans emitted for MCP tool invocations. This makes it difficult to observe and diagnose tool execution in distributed tracing systems.

What is this change?

Added centralized OpenTelemetry tracing at the MCP dispatcher level (where tools are resolved and ExecuteAsync is invoked), ensuring consistent spans for all MCP tools without code duplication.

Span Attributes

Attribute Description
mcp.tool.name Tool name
dab.entity Entity alias/name (when applicable)
dab.operation Operation type: read/create/update/delete/describe/execute
db.procedure Stored procedure name, schema-qualified if available

Operation Inference

  • Built-in tools: Mapped by exact tool name match (type-safe, no string parsing)
    • read_recordsread
    • create_recordcreate
    • update_recordupdate
    • delete_recorddelete
    • describe_entitiesdescribe
    • execute_entityexecute
    • Unknown built-in tools → execute (fallback)
  • Custom tools (stored procedures): Always use execute operation (determined by ToolType.Custom)

Outcome Recording

  • Success: ActivityStatusCode.Ok
  • Failure: ActivityStatusCode.Error with RecordException() and error.code/error.message tags
  • Error extraction: When tools return IsError=true, error code/message extracted from JSON in CallToolResult.Content and set as span tags for diagnosability

Error Codes

Code Usage
EXECUTION_FAILED Generic execution failure
AUTHENTICATION_FAILED Authentication failure (DataApiBuilderException.AuthenticationChallenge)
AUTHORIZATION_FAILED Authorization failure (DataApiBuilderException.AuthorizationCheckFailed, UnauthorizedAccessException)
DATABASE_ERROR Database exceptions (DbException)
INVALID_REQUEST Invalid arguments (ArgumentException)
OPERATION_CANCELLED Cancellation (OperationCanceledException)

Coverage

Spans cover all paths:

  • Successful execution
  • Auth/validation failures (with extracted error details from CallToolResult)
  • Database/execution errors (with exception recording)

Implementation Details

  • Uses existing TelemetryTracesHelper.DABActivitySource ("DataApiBuilder")
  • Follows the established pattern from ExecutionHelper where DABActivitySource.StartActivity() wraps query execution
  • Instrumentation is centralized via McpTelemetryHelper.ExecuteWithTelemetryAsync wrapper at the MCP dispatcher level
  • Best-effort metadata extraction ensures telemetry cannot break tool execution (catches all exceptions including DataApiBuilderException)
  • Separate authentication vs authorization error codes for clear diagnostics
  • Tool type-based operation inference (no fragile string matching)

How was this tested?

  • Integration Tests
  • Unit Tests (30 tests covering all scenarios including all 6 DML tools, custom tools, error handling, and exception mapping)

Sample Request(s)

N/A - Telemetry instrumentation is transparent to API consumers. Spans are emitted to configured OpenTelemetry exporters with the attributes and error codes documented above.

Example telemetry span for a successful built-in tool:

Activity: mcp.tool.execute
  Tags:
    - mcp.tool.name: read_records
    - dab.entity: books
    - dab.operation: read
  Status: Ok

Example telemetry span for a custom stored procedure:

Activity: mcp.tool.execute
  Tags:
    - mcp.tool.name: get_book
    - dab.entity: GetBook
    - dab.operation: execute
    - db.procedure: dbo.GetBookById
  Status: Ok

Example telemetry span for an authorization failure:

Activity: mcp.tool.execute
  Tags:
    - mcp.tool.name: read_records
    - dab.entity: books
    - dab.operation: read
    - error.type: DataApiBuilderException
    - error.code: AuthorizationFailed
    - error.message: Authorization Failure: Access Not Allowed.
  Status: Error

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 5 commits February 6, 2026 08:11
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
…EntityName property

Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Copilot AI changed the title [WIP] Add OpenTelemetry tracing for custom tools Add OpenTelemetry tracing for MCP tool execution Feb 6, 2026
Copilot AI requested a review from souvikghosh04 February 6, 2026 08:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds centralized OpenTelemetry tracing around MCP tool execution so both built-in and dynamic custom MCP tools emit consistent spans across stdio and HTTP transports.

Changes:

  • Added McpTelemetryHelper.ExecuteWithTelemetryAsync to start/finish spans, set tags, and record exceptions with MCP-specific error codes.
  • Wired telemetry wrapper into both MCP dispatch entry points (McpStdioServer and HTTP CallToolHandler).
  • Added unit tests for telemetry helpers and exposed DynamicCustomTool.EntityName to avoid reflection.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/Azure.DataApiBuilder.Mcp/Utils/McpTelemetryHelper.cs Central telemetry wrapper + operation inference + error-code mapping + custom-tool metadata extraction
src/Azure.DataApiBuilder.Mcp/Utils/McpTelemetryErrorCodes.cs Defines MCP telemetry error-code constants
src/Core/Telemetry/TelemetryTracesHelper.cs Adds Activity extension methods for MCP tool tags/status/exception recording
src/Azure.DataApiBuilder.Mcp/Core/McpStdioServer.cs Wraps stdio tool execution with telemetry helper
src/Azure.DataApiBuilder.Mcp/Core/McpServerConfiguration.cs Wraps HTTP tool execution with telemetry helper
src/Azure.DataApiBuilder.Mcp/Core/DynamicCustomTool.cs Exposes EntityName property and updates internal usage
src/Service.Tests/UnitTests/McpTelemetryTests.cs Adds unit tests validating MCP telemetry tagging/status/error mapping
src/Service.Tests/Azure.DataApiBuilder.Service.Tests.csproj Adds project reference to MCP project for unit testing
src/Azure.DataApiBuilder.Mcp/Azure.DataApiBuilder.Mcp.csproj Adds InternalsVisibleTo for Azure.DataApiBuilder.Service.Tests

Copilot AI changed the title Add OpenTelemetry tracing for MCP tool execution Restrict /openapi endpoint to development mode only Feb 12, 2026
@souvikghosh04 souvikghosh04 changed the title Restrict /openapi endpoint to development mode only Add centralized OpenTelemetry tracing for MCP tool execution Feb 13, 2026
@souvikghosh04 souvikghosh04 linked an issue Feb 13, 2026 that may be closed by this pull request
@souvikghosh04
Copy link
Contributor

@copilot please go through the open review comments which are in Unresolved state and work on the fixes or implementation.

  • make sure to check and validate the comment and whether a fix or change really needed to address that
  • for each specific, if fix is needed, implement the fix and respond back to the comment appropriately
  • if there is no fix or changes needed, make sure to respond that as well appropriately
  • when in doubt or need more context, ask your question as a reply to the specific open review comment
  • do not Resolve the comments yourself
  • make sure the changes are accurate and the functionality is not broken
  • add supporting tests to check and validate the changes including any edge cases
  • keep the code changes modular and reusable including the test cases
  • changes should be within a single commit in this same PR branch
  • keep the PR description updated but do not divert from the requirements. in case there is any diversion or confusion, put a comment with your summary note or clarifications

- Add AUTHORIZATION_FAILED error code separate from AUTHENTICATION_FAILED
- Use tool type to infer operations: built-in tools mapped by name, custom tools always "execute"
- Map describe_entities to "describe" operation (not "read")
- Extract error.code and error.message from CallToolResult when IsError=true
- Catch all exceptions in ExtractCustomToolMetadata for best-effort metadata
- Update MapExceptionToErrorCode to handle DataApiBuilderException auth/authz cases
- Make _recordedActivities readonly in tests
- Use McpTelemetryErrorCodes constants in tests
- Rename FakeMcpTool to MockMcpTool
- Test all DML tools (read, create, update, delete, describe, execute)
- Focus tests on ExecuteWithTelemetryAsync rather than individual TrackXX methods
- Custom tools (stored procedures) now correctly return "execute" operation

Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
@souvikghosh04
Copy link
Contributor

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@souvikghosh04
Copy link
Contributor

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@souvikghosh04
Copy link
Contributor

@copilot I want to run the DAB locally in Aspire. the project already supports as it has inbuilt references. could you run it and make some MCP calls so that we can see the logs and telemetry info in Aspire?

@souvikghosh04
Copy link
Contributor

@copilot I want to run the DAB locally in Aspire. the project already supports as it has inbuilt references. could you run it here in my VS code locally and make some MCP calls so that we can see the logs and telemetry info in Aspire?

Copy link
Contributor Author

Copilot AI commented Feb 17, 2026

@copilot I want to run the DAB locally in Aspire. the project already supports as it has inbuilt references. could you run it and make some MCP calls so that we can see the logs and telemetry info in ...

I've reviewed the Aspire setup in src/Aspire.AppHost/. The current dab-config.json doesn't have MCP enabled. To demonstrate the telemetry with Aspire, we would need to:

  1. Add MCP runtime configuration to src/Service/dab-config.json
  2. Add a stored procedure entity with custom tool enabled
  3. Run dotnet run --project src/Aspire.AppHost/Aspire.AppHost.csproj --launch-profile aspire-mssql
  4. Make MCP tool calls via stdio or HTTP
  5. View the telemetry spans in the Aspire dashboard

However, this would require:

  • Setting up a database (Docker container or connection string)
  • Modifying the DAB config (which is shared and might affect other developers)
  • Running in interactive mode which isn't suitable for this automated environment

Would you prefer:

  1. A documented example of the expected telemetry output?
  2. Instructions for running this manually?
  3. An integration test that validates the telemetry is emitted correctly?

The unit tests in commit 98105c3 already verify that ExecuteWithTelemetryAsync correctly sets all span attributes, operations, and error codes for both built-in and custom tools across success and failure scenarios.

@souvikghosh04
Copy link
Contributor

@copilot this is not a follow to any comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Review In Progress

Development

Successfully merging this pull request may close these issues.

MCP Custom Tool Telemetry

3 participants