Skip to content

Conversation

@vaibhavtiwari33
Copy link
Contributor

@vaibhavtiwari33 vaibhavtiwari33 commented Jan 26, 2026

Fixes: #205

Notes

Change in OnSuccess message metadata

With the introduction of a shared UserMetadata class, the userMetadata member in Message class for OnSuccess message has been changed from:

   private final HashMap<String, KeyValueGroup> userMetadata;

to

    private final UserMetadata userMetadata;

This is done so that the users can utilize the same UserMetadata for onSuccess message without conversion.

Furthermore, the new UserMetadata has been implemented in its current form since it allows parity with other SDKs (inner data holds Map of Map, instead of Map of KeyValueGroup) and also prevents the extra step of converting proto KeyValueGroup object to local KeyValueGroup class object and back.

Format of userMetadata and systemMetadata

Both UserMetadata and SystemMetadata classes have inner data in the format:

Map<String, Map<String, byte[]>> data;

Signed-off-by: vtiwari5 <vaibhav_tiwari1@intuit.com>
@codecov
Copy link

codecov bot commented Jan 26, 2026

Codecov Report

❌ Patch coverage is 87.56757% with 23 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@3180f88). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...ain/java/io/numaproj/numaflow/sourcer/Message.java 23.07% 9 Missing and 1 partial ⚠️
...o/numaproj/numaflow/sourcetransformer/Message.java 28.57% 4 Missing and 1 partial ⚠️
...java/io/numaproj/numaflow/shared/UserMetadata.java 96.51% 0 Missing and 3 partials ⚠️
...java/io/numaproj/numaflow/mapper/HandlerDatum.java 50.00% 1 Missing ⚠️
.../java/io/numaproj/numaflow/mapper/MapperActor.java 80.00% 0 Missing and 1 partial ⚠️
.../numaproj/numaflow/sourcer/OutputObserverImpl.java 0.00% 0 Missing and 1 partial ⚠️
...aproj/numaflow/sourcetransformer/HandlerDatum.java 50.00% 1 Missing ⚠️
...j/numaflow/sourcetransformer/TransformerActor.java 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #210   +/-   ##
=======================================
  Coverage        ?   61.83%           
  Complexity      ?      588           
=======================================
  Files           ?      154           
  Lines           ?     3590           
  Branches        ?      255           
=======================================
  Hits            ?     2220           
  Misses          ?     1193           
  Partials        ?      177           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vaibhavtiwari33 vaibhavtiwari33 changed the title chore: Metadata propagation in source, source transformer, mapper, sink feat: Metadata propagation in source, source transformer, mapper, sink Jan 26, 2026
@vaibhavtiwari33 vaibhavtiwari33 self-assigned this Jan 26, 2026
@vaibhavtiwari33 vaibhavtiwari33 added the enhancement New feature or request label Jan 26, 2026
vtiwari5 and others added 3 commits January 26, 2026 11:00
Signed-off-by: vtiwari5 <vaibhav_tiwari1@intuit.com>
Signed-off-by: vtiwari5 <vaibhav_tiwari1@intuit.com>
… transform, mapper and sinker

Signed-off-by: Vaibhav Tiwari <vaibhav.tiwari33@gmail.com>
@vaibhavtiwari33 vaibhavtiwari33 marked this pull request as ready for review January 26, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Java SDK Changes to expose Metadata

2 participants