Skip to content

buildsignal republishes to speculate with a batch-ID partition key, inconsistent with queue-partitioned publishers #531

Description

@behinddwalls

Every publisher to the speculate topic uses the queue name as the partition key (e.g. speculate.go, cancel.go, mergesignal.go all pass batch.Queue), except the buildsignal controller: submitqueue/orchestrator/controller/buildsignal/buildsignal.go:173 republishes the batch ID to speculate with msg.PartitionKey — which on the buildsignal topic is the batch ID (set in build.go:196), not the queue.

Consequence: the same queue's speculate messages land on different partitions depending on who published them, so per-queue ordering/serialization of speculate processing does not hold for the buildsignal-triggered re-evaluations, and each batch ID becomes an additional short-lived partition on the speculate topic.

If speculate's consumer is idempotent and order-independent this may be benign, but the partitioning should be made consistent either way — likely by resolving the batch's queue before publishing (as every other publisher does).

Found while auditing partition-key cardinality across topics (context: idle-lease release work).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions