Skip to content

Bulk publish: per-message metadata in publish_events #1214

Description

@MyMirelHub

Describe the feature

publish_events does not populate BulkPublishRequestEntry.metadata, so per-message metadata (partition key, session id, Kafka key) cannot be set. Entries are built from entry_id, event and content_type only, in both dapr/clients/grpc/client.py#L544 and dapr/aio/clients/grpc/client.py#L493.

The runtime supports it, and go-sdk (PublishEventsEvent.Metadata), java-sdk (BulkPublishEntry) and js-sdk (PubSubBulkPublishMessage.metadata) all expose it. Same request against dotnet-sdk in dapr/dotnet-sdk#1198.

Side effect: the runtime only merges request-level metadata into entries that already have metadata, so publish_metadata never reaches the envelope either. On 1.18.4 with ttlInSeconds, publish_event sets expiration on the cloudevent and publish_events does not.

Proposed API, mirroring go-sdk:

d.publish_events(
    pubsub_name='pubsub',
    topic_name='TOPIC_A',
    data=[
        BulkPublishEntry(event='{"n": 1}', metadata={'partitionKey': 'tenant-a'}),
        BulkPublishEntry(event='{"n": 2}', metadata={'partitionKey': 'tenant-b'}),
        '{"n": 3}',
    ],
)

Existing bytes/str entries keep working, and publish_metadata should be copied onto every entry.

Release Note

RELEASE NOTE: ADD Per-message metadata support in publish_events bulk publish.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions