Smoke OTEL - 26056780450

[github-actions]

Executive Summary

Overall: FAIL

Run 26056780450 was verified end-to-end for three of four backends. The local OTEL mirror contains current-run spans. Grafana/Tempo received and indexed the trace. Sentry received spans without export error. Datadog failed with HTTP 404 on the OTLP write path, and the Datadog MCP exposes zero tools so the read path cannot be validated.

Status Matrix

Backend	Write Config Present	Write Export Succeeded	Read Config Present	Read Query Succeeded	Overall
Local OTLP	✅	✅	—	—	✅
Sentry	✅	✅	✅	🔶	🔶
Grafana	✅	✅	✅	✅	✅
Datadog	✅	❌	❌	❌	❌

Evidence

Local OTLP

• OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, GH_AW_OTLP_ENDPOINTS all set.
• OTEL_SERVICE_NAME=gh-aw.smoke-otel-backends
• /tmp/gh-aw/otel.jsonl contains 1 line with span gh-aw.agent.setup, traceId=7b74fc957c7c71f272a171b166d93200, run_id=26056780450.
• Configured backend hosts: o205451.ingest.us.sentry.io, otlp-gateway-prod-eu-west-2.grafana.net, otlp-intake.***REDACTED***.
• OTLP export errors: 1 — {"host":"otlp-intake.***REDACTED***","status":404,"reason":"Not Found"}.

Sentry

• MCP connection: ✅ (github org, gh-aw project found at https://github.sentry.io).
• No OTLP export errors attributed to Sentry (o205451.ingest.us.sentry.io not in error log).
• Write export: presumed succeeded (no error entry).
• Read: list_events returned no results — the available Sentry MCP tools only query error events, not OTLP traces/spans. Cannot confirm trace 7b74fc957c7c71f272a171b166d93200 is visible in Sentry's trace explorer.
• Representative link: https://github.sentry.io/explore/discover/homepage/?project=4511347087179777

Grafana

• MCP connection: ✅.
• Datasource used: grafanacloud-traces (Tempo, uid grafanacloud-traces).
• TraceQL query {resource.service.name="gh-aw.smoke-otel-backends"} returned trace 7b74fc957c7c71f272a171b166d93200 — exact match for the current run.
• Spans visible: 3 spans, root span gh-aw.pre_activation.setup, duration ~29 s.
• Write export: no error entry in /tmp/gh-aw/otlp-export-errors.jsonl for Grafana.

Datadog

• Configured backend host: otlp-intake.***REDACTED*** ✅ present in GH_AW_OTLP_ENDPOINTS.
• Write export: ❌ — /tmp/gh-aw/otlp-export-errors.jsonl contains {"host":"otlp-intake.***REDACTED***","status":404,"reason":"Not Found"}.
• Datadog MCP: ❌ — datadog --help reports 0 available tools; no span or trace queries possible.

Failure Analysis

Sentry — Read Query Inconclusive

Failing step: Confirming current-run spans in Sentry via MCP.
Evidence: list_events queries error events only and returned no results. No tool in the Sentry MCP server queries the Trace Explorer or OTLP span index.
Likely root cause: The Sentry MCP skill does not expose a trace/span search tool. This is a tooling gap, not a Sentry ingest failure.
Path: Read path / tooling.
Next step: Use Sentry's Trace Explorer UI directly at https://github.sentry.io/performance/ or add a search_spans / get_trace tool to the Sentry MCP configuration.

Datadog — Write Export Failed (HTTP 404)

Failing step: OTLP export to otlp-intake.***REDACTED***.
Evidence: {"host":"otlp-intake.***REDACTED***","status":404,"reason":"Not Found"} in /tmp/gh-aw/otlp-export-errors.jsonl.
Likely root cause: The endpoint path is incorrect or the Datadog EU OTLP ingestion endpoint has moved. otlp-intake.***REDACTED*** should accept traces at /api/v0.2/traces or /v1/traces (OTLP HTTP). A 404 means the URL path is wrong, or OTLP HTTP ingestion is not enabled for this Datadog EU site/org.
Path: Write path — endpoint configuration.
Next step: Verify the correct OTLP endpoint for ***REDACTED***. The expected path is (otlpintake/redacted)***REDACTED***:443 with the agent or (otlpintake/redacted)***REDACTED***/v1/traces for HTTP. Check the GH_AW_OTEL_DATADOG_ENDPOINT secret value and confirm OTLP ingestion is enabled in the Datadog EU org settings.

Datadog — Read Path Unavailable

Failing step: Querying Datadog for current-run or recent gh-aw spans.
Evidence: datadog --help reports (tool list unavailable) — zero tools exposed by the Datadog MCP server.
Likely root cause: The Datadog MCP bridge is misconfigured or the Datadog MCP server is not responding to the tool-list request. The bridge URL resolves but returns an empty tool set.
Path: Read path — MCP configuration.
Next step: Check the Datadog MCP server process at (host.docker.internal/redacted) verify the DD_API_KEYandDD_APPLICATION_KEY` secrets are injected, and restart the bridge if needed.

---

Run URL: https://github.com/github/gh-aw/actions/runs/26056780450

Generated by 🧪 Smoke OTEL for issue #33116 · ● 3.6M · ◷

• expires on May 18, 2026, 9:58 PM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #33147

View newer issue

---

This action was performed automatically by the Smoke OTEL workflow.