Propagate LLM Observability context (session, ml_app, agent attribution) across process boundaries

[ncybul]

Summary

Adds an explicit, manual API for propagating LLM Observability context across process boundaries that automatic instrumentation doesn't cover — e.g. an SQS worker reading its own message attributes. Mirrors dd-trace-py's inject_distributed_headers/activate_distributed_headers model (Python/Node/Go already support this; Java did not).

// Producer side
Map<String, String> headers = new HashMap<>();
LLMObs.injectDistributedHeaders(span, headers);
// ... put headers into SQS message attributes ...

// Consumer side
try (Closeable scope = LLMObs.activateDistributedHeaders(headers)) {
  // spans started here join the producer's trace and inherit session_id / agent attribution
}

Design

• Explicit/manual, not automatic instrumentation. The customer calls this per message in their own consumer loop. This sidesteps a pre-existing TracingIterator batch-context-reuse bug in APM's automatic SQS instrumentation rather than fixing it — that's a separate, not-yet-filed issue.
• Wire format: standard APM trace context (trace id, parent id, sampling) is injected/extracted via the normal Propagators.defaultPropagator(). LLMObs-specific values (ml_app, session_id, agent id/name) are carried as dedicated PropagationTags fields — _dd.p.llmobs_ml_app, _dd.p.llmobs_sid, _dd.p.llmobs_pagent_span_id, _dd.p.llmobs_pagent_name — mirroring how dm/tid/ts/opm already ride inside x-datadog-tags/tracestate t.*. These key names were checked against dd-trace-py's _constants.py and match exactly, so a mixed-language pipeline can still join a trace across this hop.
• Why PTags instead of hand-rolled string manipulation: an earlier draft appended key=value pairs directly onto the x-datadog-tags carrier string from agent-llmobs, self-contained to that module. This PR replaces that with proper PropagationTags fields in dd-trace-core (both DatadogPTagsCodec and W3CPTagsCodec), wired through a new AgentSpanContext seam (getLLMObsMlApp/updateLLMObsMlApp/etc., default no-ops) so agent-llmobs still doesn't need a direct dd-trace-core dependency. This is the same seam pattern already used for dm/tid/ts/opm.
• Truncation/graceful-degradation parity with dd-trace-py: Python writes ml_app, session_id, and agent id/name unbounded (no fixed length cap) and relies on the codec's own overflow handling. Java originally added a self-invented 64-char truncation cap on these fields; that's been removed so all 4 fields pass through unbounded, matching Python. One gap is intentionally not replicated: Python proactively computes whether the full tag budget fits within 485 of the 512-byte x-datadog-tags limit and gracefully truncates/drops parent_agent_name (then the id) to avoid the whole header being dropped on overflow. Java relies on the existing codec-level behavior instead — the Datadog codec drops the whole x-datadog-tags header on overflow (all-or-nothing), while the W3C codec already gracefully drops individual overlong tags without invalidating the rest. Implementing full parity would require threading a "does this fit" check from PTagsFactory up through AgentSpanContext to DDLLMObsPropagator, which is out of scope for this PR — flagging as a known, deliberate gap for a fast-follow.
• API surface: LLMObs.injectDistributedHeaders/activateDistributedHeaders (dd-trace-api), delegating to a new LLMObs.LLMObsPropagator interface, following the same static-field delegation pattern as SPAN_FACTORY/EVAL_PROCESSOR/FEEDBACK_PROCESSOR. Real implementation is DDLLMObsPropagator (agent-llmobs), wired in via LLMObsInternal.setPropagator(...); defaults to a no-op when the agent isn't active.

Bug fixed along the way

ExtractedContext (the context produced by Propagators.defaultPropagator().extract(...)) didn't override the new AgentSpanContext.getLLMObs*() getters, so they fell back to the interface's null-returning defaults instead of reading ExtractedContext's own PropagationTags field. This silently dropped session_id/ml_app/agent-attribution on the consumer side of activateDistributedHeaders even though the values were correctly on the wire. Fixed by adding the four delegating overrides in ExtractedContext.

Known scope limitations

• ml_app rides on the wire for cross-SDK compatibility but is not yet auto-applied to spans started after activateDistributedHeaders — callers must still pass ml_app explicitly when starting a span.
• agent_version is not propagated cross-process (no SDK sends it today).
• parent_agent_name budget-check/graceful-truncation parity with dd-trace-py (see Design above) is deferred.
• No cross-language system test yet — this is a Java-only manual API, not a shared automatic-instrumentation contract, so it's scoped as a fast-follow.

Test plan

• DDLLMObsPropagatorTest (JUnit 5) — round-trips a plain Map<String, String> carrier (the shape an SQS message-attribute map would take): trace-id join across inject/activate, session_id/agent-attribution propagation, no-op on missing trace context, null-arg validation, ml_app always present even without session/attribution. All 5 tests pass, including the session_id/agent-attribution propagation case that caught the ExtractedContext bug above.
• :dd-java-agent:agent-llmobs:test passes (no regressions).
• :dd-trace-core:test (propagation package) passes (no regressions).
• spotlessApply/spotlessCheck pass.
• New unit tests for the new PTags fields' serialize/parse round-trip in both DatadogPTagsCodec and W3CPTagsCodec — tracked as a follow-up on this branch.

Claude session: 15543c2c-2abe-408e-b16e-05ddbe972287
Resume: claude --resume 15543c2c-2abe-408e-b16e-05ddbe972287

[datadog-prod-us1-5]

🎯 Code Coverage (details)
• Patch Coverage: 41.67%
• Overall Coverage: 57.38% (-1.63%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fb3651c | Docs | View more details | Give us feedback!}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.81 s	14.70 s	[-0.1%; +1.5%] (no difference)
startup:insecure-bank:tracing:Agent	13.58 s	13.72 s	[-1.7%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent	17.01 s	16.89 s	[-0.3%; +1.6%] (no difference)
startup:petclinic:iast:Agent	16.86 s	16.94 s	[-1.3%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	16.75 s	16.72 s	[-1.0%; +1.3%] (no difference)
startup:petclinic:sca:Agent	16.90 s	16.59 s	[+0.8%; +3.0%] (maybe worse)
startup:petclinic:tracing:Agent	16.05 s	15.63 s	[-1.7%; +7.1%] (no difference)

Commit: fb3651c0 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.