Skip to content

Fix broken data sourcing benchmark#1389

Merged
llucax merged 1 commit intofrequenz-floss:v1.x.xfrom
llucax:fix-datasourcing-benchmark
Apr 21, 2026
Merged

Fix broken data sourcing benchmark#1389
llucax merged 1 commit intofrequenz-floss:v1.x.xfrom
llucax:fix-datasourcing-benchmark

Conversation

@llucax
Copy link
Copy Markdown
Contributor

@llucax llucax commented Apr 20, 2026

The benchmark stopped working after multiple breaking changes accumulated over time:

  • Channel type mismatch: ChannelRegistry.get_or_create() was called with ComponentMetricRequest but DataSourcingActor internally creates channels typed as Sample[Quantity].

    Fix: use Sample[Quantity].

  • Missing api_client_streaming flag: MockMicrogrid defaults to not streaming, so no mock data was ever sent.

    Fix: pass api_client_streaming=True.

  • Actor lifecycle: The old @actor decorator auto-started in __init__, but the new Actor class requires async-with. The benchmark sent requests before the actor was running, so subscriptions were missed.

    Fix: restructure to send requests inside the async-with block and manually start streaming tasks after the actor has subscribed.

  • Cleanup failure: mock_grid.cleanup() failed because init_mock_client doesn't set mock_resampler.

    Fix: reset _CONNECTION_MANAGER directly.

  • Consume tasks never stopped: Broadcast channels were never closed, so ReceiverStoppedError never fired.

    Fix: cancel consume tasks after streaming completes.

@llucax llucax requested a review from a team as a code owner April 20, 2026 11:21
@llucax llucax requested review from florian-wagner-frequenz and removed request for a team April 20, 2026 11:21
@llucax llucax added this to the v1.0.0-rc2206 milestone Apr 20, 2026
@llucax llucax requested a review from simonvoelcker April 20, 2026 11:21
@llucax llucax force-pushed the fix-datasourcing-benchmark branch from 9ba4886 to 136b7d8 Compare April 20, 2026 11:22
@llucax
Copy link
Copy Markdown
Contributor Author

llucax commented Apr 20, 2026

I guess for this to not break again (soon) we should probably run the benchmarks as part of the CI...

The benchmark stopped working after multiple breaking changes
accumulated over time:

- Channel type mismatch: ChannelRegistry.get_or_create() was called
  with ComponentMetricRequest but DataSourcingActor internally creates
  channels typed as Sample[Quantity]. Fix: use Sample[Quantity].

- Missing api_client_streaming flag: MockMicrogrid defaults to not
  streaming, so no mock data was ever sent. Fix: pass
  api_client_streaming=True.

- Actor lifecycle: The old @Actor decorator auto-started in __init__,
  but the new Actor class requires async-with. The benchmark sent
  requests before the actor was running, so subscriptions were missed.
  Fix: restructure to send requests inside the async-with block and
  manually start streaming tasks after the actor has subscribed.

- Cleanup failure: mock_grid.cleanup() failed because init_mock_client
  doesn't set mock_resampler. Fix: reset _CONNECTION_MANAGER directly.

- Consume tasks never stopped: Broadcast channels were never closed, so
  ReceiverStoppedError never fired. Fix: cancel consume tasks after
  streaming completes.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
@llucax llucax force-pushed the fix-datasourcing-benchmark branch from 136b7d8 to aaed9cd Compare April 20, 2026 13:37
@llucax
Copy link
Copy Markdown
Contributor Author

llucax commented Apr 21, 2026

@simonvoelcker maybe we can merge this before your PR so you can actually run the benchmark before and after the changes to see how they compare?

@simonvoelcker
Copy link
Copy Markdown
Contributor

@llucax good idea and thank you for fixing these issues. I had identified a few myself but was in disbelief because the benchmarks didn't even work for me in the oldest versions, right after they were introduced.

@llucax
Copy link
Copy Markdown
Contributor Author

llucax commented Apr 21, 2026

@llucax good idea and thank you for fixing these issues. I had identified a few myself but was in disbelief because the benchmarks didn't even work for me in the oldest versions, right after they were introduced.

Yeah, that's very weird. To be honest, I didn't investigated myself, I just delegated to AI, so I'm not sure how it actually happened that it doesn't work for old versions. But it could be wrong/lack of deps pinning.

@llucax llucax added this pull request to the merge queue Apr 21, 2026
Merged via the queue into frequenz-floss:v1.x.x with commit 50883ff Apr 21, 2026
7 checks passed
@llucax llucax deleted the fix-datasourcing-benchmark branch April 21, 2026 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants