Skip to content

feat(voice): dynamic settings voice picker with safe fallback + preview#690

Open
heavygee wants to merge 3 commits into
tiann:mainfrom
heavygee:feat/voice-picker-upstream
Open

feat(voice): dynamic settings voice picker with safe fallback + preview#690
heavygee wants to merge 3 commits into
tiann:mainfrom
heavygee:feat/voice-picker-upstream

Conversation

@heavygee
Copy link
Copy Markdown
Contributor

Summary

  • add a Settings voice picker (Default + selectable ElevenLabs voices)
  • load voices dynamically from a new authenticated hub route: GET /api/voice/voices
  • include preview controls in the picker:
    • enabled when preview_url is available
    • visible but disabled with tooltip when unavailable (including no-key fallback)
  • label cloned voices with a clone badge
  • keep no-key behavior safe:
    • static fallback voice list remains available
    • starting voice session without key still returns clear 400 error
  • route voice selection via per-voice agent/token selection (no fragile runtime voice override)
  • localize fallback voice display names for zh-CN
  • add/extend tests for settings picker and hub voice routes

Refs #686

Test plan

  • cd web && bun run test src/routes/settings/index.test.tsx src/lib/voices.test.ts
  • cd hub && bun test src/web/routes/voice.test.ts
  • Manual:
    • default voice path works
    • non-default premade voice (Alice) works
    • cloned voice works
    • no API key path shows static list, disabled preview controls, and clear 400 toast on voice start

@heavygee heavygee force-pushed the feat/voice-picker-upstream branch from c3d8e89 to 3b07716 Compare May 25, 2026 17:31
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] Selected voice is ignored when ELEVENLABS_AGENT_ID is configured — voiceId reaches the token route, but process.env.ELEVENLABS_AGENT_ID wins before the per-voice auto-create path runs, and the client no longer sends a runtime TTS override. Any install with a configured default agent will keep using that agent's voice for non-default picker selections. Evidence hub/src/web/routes/voice.ts:185.
  • [Minor] Preview audio can keep playing after leaving Settings — the new Audio instance is stored in a ref, but nothing pauses it on unmount, so navigating away while a preview is playing leaves audio running with no visible stop control. Evidence web/src/routes/settings/index.tsx:440.

Summary

  • Review mode: initial
  • The PR needs changes for the configured-agent voice selection path. Residual risk: I could not execute local automation in this runner because bun is not installed.

Testing

  • Not run (automation; bun: command not found)

HAPI Bot

Comment thread hub/src/web/routes/voice.ts Outdated
let agentId = customAgentId || process.env.ELEVENLABS_AGENT_ID
const voiceAgentMap = parseVoiceAgentMap()
const mappedAgentId = voiceId ? voiceAgentMap[voiceId] : undefined
let agentId = customAgentId || mappedAgentId || process.env.ELEVENLABS_AGENT_ID
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This makes the picker ineffective for the common configured-agent setup. When ELEVENLABS_AGENT_ID is set, agentId is resolved here even if voiceId is present, so the request never reaches getOrCreateAgentIdForVoice(...). Since RealtimeVoiceSession only passes the voice to the token endpoint and does not include a TTS override in startSession, non-default voice selections will keep using the configured agent's existing voice.

Suggested fix:

const voiceAgentMap = parseVoiceAgentMap()
const mappedAgentId = voiceId ? voiceAgentMap[voiceId] : undefined
const defaultAgentId = process.env.ELEVENLABS_AGENT_ID
let agentId = customAgentId || mappedAgentId || (!voiceId ? defaultAgentId : undefined)

if (!agentId) {
    agentId = await getOrCreateAgentIdForVoice(apiKey, voiceId) ?? undefined
    if (!agentId) {
        return c.json({
            allowed: false,
            error: 'Failed to create ElevenLabs agent automatically'
        }, 500)
    }
}

}

currentAudioRef.current?.pause()
const audio = new Audio(previewUrl)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] A playing preview is not stopped when the Settings page unmounts. currentAudioRef owns the Audio object, but there is no cleanup effect, so users can navigate away from Settings and the preview keeps playing without the stop button still being available.

Suggested fix:

useEffect(() => {
    return () => {
        currentAudioRef.current?.pause()
        currentAudioRef.current = null
    }
}, [])

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • No high-confidence issues found.

Summary

  • Review mode: follow-up after new commits
  • Full latest diff reviewed. The prior bot findings appear addressed: selected voiceId now resolves a voice-specific agent before falling back to ELEVENLABS_AGENT_ID, and Settings preview audio is paused on unmount.
  • Residual risk: local automation was not run because bun is not installed in this runner.

Testing

  • Not run (automation; bun not found)

HAPI Bot

@heavygee
Copy link
Copy Markdown
Contributor Author

Linking related upstream Discussion for roadmap context: #691

@heavygee
Copy link
Copy Markdown
Contributor Author

Happy to rebase this onto post-#692 main once that lands — the two PRs are complementary (voice picker + multi-backend are separate concerns) but share enough files that a clean rebase makes more sense than trying to merge both simultaneously. No action needed here until #692 merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant