Skip to content

fix(voice): preserve turns across activity handoffs - #6844

Draft
IanCollection wants to merge 2 commits into
livekit:mainfrom
IanCollection:agent/preserve-agent-task-handoff-turn
Draft

fix(voice): preserve turns across activity handoffs#6844
IanCollection wants to merge 2 commits into
livekit:mainfrom
IanCollection:agent/preserve-agent-task-handoff-turn

Conversation

@IanCollection

Copy link
Copy Markdown

What changed

  • retain a pipeline user turn accepted while its source activity is being paused/replaced
  • transfer the turn to the successor activity and run it through the existing user-turn scheduler
  • persist an accepted queued turn exactly once if the successor cannot start or the session closes

Why

During an AgentTask or update_agent() handoff, the source activity blocks scheduling before the successor is fully active. An EOU landing in that window currently follows the scheduling_paused branch, returns True, and clears the recognition transcript without adding it to either activity's history. The user turn disappears.

The queue is owned by the activity and moves across the concrete activity boundary. Successful successors consume it through _user_turn_completed_task; terminal paths use the same history/metrics semantics as session close. Realtime models are intentionally excluded because their input commit lifecycle is remote and is covered separately by #6662.

Impact

Cascade/AgentTask and serialized update_agent() handoffs retain a crossing user turn exactly once instead of silently dropping it.

Design feedback requested

This PR implements the existing TODO in on_end_of_turn by transferring ownership to the successor activity. Feedback is especially welcome on whether activity-owned buffering is the preferred framework boundary for chained handoffs.

Validation

  • fail-before real VAD/STT/EOU regression timed out with skipping user input, speech scheduling is paused
  • covers AgentTask, consecutive update_agent(), successor close rejection, successor startup failure, and source close
  • pytest tests/test_agent_task_handoff_turn.py tests/test_nested_agent_task.py tests/test_update_agent_long_on_enter.py tests/test_agent_task_close_race.py tests/test_foreground_run_tracking.py tests/test_agent_session.py --unit -q — 98 passed
  • Ruff check and format check passed
  • mypy (livekit.agents) — 206 files passed

Related but distinct: #6662 fixes Realtime paused-speech/overlap behavior; this PR handles local pipeline turn ownership across activity replacement.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


ian seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants