Skip to content

Trace latency: add Postgres trace bounds and storage state #286

Description

@thorrester

Parent: #281

Goal

Store bounded trace lookup metadata and trace storage state in Postgres so read paths can avoid unnecessary Delta/object-store work.

Postgres should own transactional metadata and coordination state here, not span payloads or broad analytical scans.

Scope

  • Add Postgres tables or rows for trace bounds and storage state.
  • Track enough information to bound trace-id lookups by time/partition.
  • Track storage freshness state such as latest observed Delta version and a monotonic refresh epoch.
  • Keep updates idempotent and safe under concurrent writers/pods.
  • Expose the state needed by query cache keys and maintenance coordination.

High-level design

This is control-plane metadata. The row count and payload size should stay bounded relative to trace volume. DataFusion/Delta remains the source of span data; Postgres helps the system decide which Delta partitions and versions are worth reading.

The storage state should become the common freshness marker for result-cache invalidation and request-path refresh detection.

Acceptance criteria

  • Trace bounds can be written and read through Postgres.
  • Query code can obtain a refresh epoch or equivalent storage freshness marker.
  • Bounds updates tolerate duplicate or concurrent writes.
  • No broad span payloads or analytical query results are stored in Postgres.
  • Tests cover bounds lookup, missing bounds, and concurrent update behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions