Fix broken data sourcing benchmark#1389
Conversation
9ba4886 to
136b7d8
Compare
|
I guess for this to not break again (soon) we should probably run the benchmarks as part of the CI... |
The benchmark stopped working after multiple breaking changes accumulated over time: - Channel type mismatch: ChannelRegistry.get_or_create() was called with ComponentMetricRequest but DataSourcingActor internally creates channels typed as Sample[Quantity]. Fix: use Sample[Quantity]. - Missing api_client_streaming flag: MockMicrogrid defaults to not streaming, so no mock data was ever sent. Fix: pass api_client_streaming=True. - Actor lifecycle: The old @Actor decorator auto-started in __init__, but the new Actor class requires async-with. The benchmark sent requests before the actor was running, so subscriptions were missed. Fix: restructure to send requests inside the async-with block and manually start streaming tasks after the actor has subscribed. - Cleanup failure: mock_grid.cleanup() failed because init_mock_client doesn't set mock_resampler. Fix: reset _CONNECTION_MANAGER directly. - Consume tasks never stopped: Broadcast channels were never closed, so ReceiverStoppedError never fired. Fix: cancel consume tasks after streaming completes. Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
136b7d8 to
aaed9cd
Compare
|
@simonvoelcker maybe we can merge this before your PR so you can actually run the benchmark before and after the changes to see how they compare? |
|
@llucax good idea and thank you for fixing these issues. I had identified a few myself but was in disbelief because the benchmarks didn't even work for me in the oldest versions, right after they were introduced. |
Yeah, that's very weird. To be honest, I didn't investigated myself, I just delegated to AI, so I'm not sure how it actually happened that it doesn't work for old versions. But it could be wrong/lack of deps pinning. |
The benchmark stopped working after multiple breaking changes accumulated over time:
Channel type mismatch:
ChannelRegistry.get_or_create()was called withComponentMetricRequestbutDataSourcingActorinternally creates channels typed asSample[Quantity].Fix: use
Sample[Quantity].Missing
api_client_streamingflag:MockMicrogriddefaults to not streaming, so no mock data was ever sent.Fix: pass
api_client_streaming=True.Actor lifecycle: The old
@actordecorator auto-started in__init__, but the newActorclass requires async-with. The benchmark sent requests before the actor was running, so subscriptions were missed.Fix: restructure to send requests inside the async-with block and manually start streaming tasks after the actor has subscribed.
Cleanup failure:
mock_grid.cleanup()failed because init_mock_client doesn't set mock_resampler.Fix: reset
_CONNECTION_MANAGERdirectly.Consume tasks never stopped: Broadcast channels were never closed, so
ReceiverStoppedErrornever fired.Fix: cancel consume tasks after streaming completes.