fix(aws): keep Nova Sonic replies alive with IGNORE_ON_ENTER - #6813
fix(aws): keep Nova Sonic replies alive with IGNORE_ON_ENTER#6813uuzzrm wants to merge 8 commits into
Conversation
Coalesce pending tool-set session recycles and wait for an active generation to finish before rebuilding the stream. This keeps repeated generate_reply calls from racing session initialization when IGNORE_ON_ENTER temporarily changes the tools.
|
Addressed the two new review findings in commit 642106d.
Added deterministic regression coverage for both cases. Verification:
AI assistance was used and the final diff and test results were reviewed. |
Signed-off-by: Ruiming Zhao <uuzzrm@gmail.com>
|
Addressed the latest Devin Review finding in The recycle task now propagates cancellation when shutdown interrupts the wait for its response or audio child task. Previously those Added a regression test that exercises cancellation during the child-task wait and asserts that stream initialization is not restarted. Verification:
AI assistance was used and the final diff and test results were reviewed. |
Closes #6812
What changed
Amazon Nova Sonic declares tools when a realtime session starts.
ToolFlag.IGNORE_ON_ENTERtemporarily changes the tool set aroundgenerate_reply, and the old path could queue overlapping session recycles. A later reply could then race the stream restart and stop producing audio.This change:
The regression coverage exercises consecutive replies, restored tool sets, active generations, stuck generations, generation replacement, prompt snapshot races, audio delivery, and reply gating without requiring AWS credentials.
Validation
Local validation on Windows, using the isolated worktree virtual environment:
pytest livekit-plugins/livekit-plugins-aws/tests -q(25 passed)pytest tests/test_realtime_reply_chat_ctx.py tests/test_agent_session.py -q(82 passed)ruff check livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/experimental/realtime/realtime_model.py livekit-plugins/livekit-plugins-aws/tests/test_nova_sonic_tool_recycle.py(passed)ruff format --check livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/experimental/realtime/realtime_model.py livekit-plugins/livekit-plugins-aws/tests/test_nova_sonic_tool_recycle.py(passed)git diff --check(passed)Remote CI is running for the follow-up commit. The repository's current mypy checks and the existing AWS typing baseline are tracked by CI; no package manifests or changelog files are part of this change.
AI assistance
Codex assisted with investigating the reported behavior, drafting the regression coverage, and implementing the patch. I reviewed the resulting diff and ran the validation commands above.