Skip to content

feat: add finalized live subtitles for Discord meetings - #266

Open
kr1shap wants to merge 4 commits into
stagingfrom
feat/224/live-subtitle-stream
Open

kr1shap wants to merge 4 commits into
stagingfrom
feat/224/live-subtitle-stream

Conversation

@kr1shap

@kr1shap kr1shap commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

/record start now posts finalized live subtitles in a dedicated Discord thread by default, with subtitles:false to opt out. Subtitles reuse the meeting's authenticated WebSocket, batch every two seconds, and flush an ended marker before archiving their own thread; failures are isolated from recording and minutes generation.

Why

Related to #224. Members can follow finalized meeting text live and search retained history through Discord, without repeatedly polling the cumulative transcript.

The API key and subtitle_events:true travel together in the first JSON authentication message. Events are sent only after authentication and scope validation. Sequence numbers describe delivery order; speech timestamps sort each batch. Late AWS results can appear below later speech across batches, and posted messages are not edited. The HTTP/PDF transcript retains its existing chronological ordering.

Zone

services/meeting · discord-bot · docs · root

This deliberately spans the meeting service and its Discord consumer because both sides implement the same optional event protocol. Shared recording documentation and README command summaries describe the resulting behavior.

Changed-file justification
File Why necessary
services/meeting/src/stt/transcribe.py Observe finalized results after storing words; isolate observer failures and expose incomplete flush state.
services/meeting/src/sessions.py Map final chunks to meeting time, use current speaker names, and complete after speaker flushes.
services/meeting/src/api/subtitles.py Bound outgoing events and serialize per-session sequence allocation and FIFO sends.
services/meeting/src/api/routers/meetings.py Read opt-in alongside first-frame authentication, attach/detach delivery, and specify the wire contract.
services/meeting/tests/test_transcribe.py Verify finals-only callbacks, observer isolation, and aborted-stream incomplete status.
services/meeting/tests/test_subtitles.py Exercise chunks, ordering, completion, overflow, and slow transport against fakes.
services/meeting/tests/test_meetings_api.py Exercise authenticated HTTP/WS orchestration, omitted opt-in, and rejected credentials.
discord-bot/src/commands/record.js Declare the optional subtitle boolean.
discord-bot/src/adapters/discord/recording.js Read the choice with default-on semantics and preserve explicit false.
discord-bot/src/meeting/meetingClient.js Request subtitles and validate incoming session-scoped JSON.
discord-bot/src/meeting/meetingSurface.js Bind subtitle state to each session and integrate start, stop, salvage, and cleanup.
discord-bot/src/meeting/subtitles.js Batch/sort chunks, split Unicode-safe messages, bound pending text, and coordinate completion/archive.
discord-bot/src/adapters/discord/subtitles.js Create launch-channel/sibling threads, suppress mentions, and archive the exact created thread.
discord-bot/src/adapters/discord/meetingPosts.js Reuse the existing Toronto timestamp formatter for thread names.
discord-bot/src/context.js Inject the subtitle adapter into orchestration.
discord-bot/src/index.js Wire the real Discord subtitle adapter.
discord-bot/test/meetingClient.test.js Verify combined first-frame authentication/subscription and incoming event validation.
discord-bot/test/adapters-discord.test.js Verify default-on and explicit-false command behavior.
discord-bot/test/subtitles.test.js Cover batching, tail/session isolation, sibling threads, mention suppression, Unicode, failures, and deadlines.
services/meeting/docs/API.md Document handshake fields, event schemas, ordering, completion, and queue limits.
services/meeting/docs/ARCHITECTURE.md Explain FIFO concurrency, speech/delivery order, polling trade-offs, and no replay.
services/meeting/README.md Describe live events and the combined first authentication message.
discord-bot/README.md Document the option, handshake, visible thread behavior, and ordering limitation.
docs/MEETING-RECORDING.md Document lifecycle, permissions, retention, examples, failures, and the out-of-order edge case.
README.md Update the command summary and link the recording guide.

How to verify

Run from the repository root:

(cd services/meeting && uv run pytest && uv run ruff check . && uv run ruff format --check .)
(cd discord-bot && npm test && npm run lint && npm run format:check)
git diff --check

Both service suites, lint, formatting, and whitespace checks passed locally. Python source paths were verified against this checkout. Automated tests use fake sockets/providers and make no AWS, Google, or LLM calls.

On staging:

  1. Start with default subtitles, alternate speakers, and pause. Confirm a new thread, finalized names/timestamps, and no edits to published chunks. Start with subtitles:false and confirm recording/PDF without a subtitle thread.
  2. Launch from an existing thread and confirm a sibling in the parent channel. Say “at everyone” / “at here”; confirm no pings. Literal mention suppression is also covered by automated adapter tests.
  3. Speak a final sentence and stop manually; repeat with everyone leaving voice. Confirm tail delivery, ended marker, archive of the created subtitle thread only, and PDF delivery. Search retained subtitle text after indexing.
  4. Remove thread creation/send permissions. Confirm one subtitle notice while recording/minutes continue.
  5. Record in two servers concurrently, then overlap old-session finalization with a new session. Confirm captions and archive operations remain isolated.
  6. Interrupt the meeting-service connection. Confirm incomplete-history handling and the existing HTTP PDF recovery attempt.

Checklist

  • Branched off staging and targeting staging (or this is a deliberate staging → main promotion).
  • Ran the service's test suite locally — the full one, including Postgres if the service has a database. Meeting has no database; the bot suite also passed.
  • uv run ruff check . and uv run ruff format --check . clean — CI gates both on every Python job. Bot ESLint and Prettier also passed.
  • Read and followed the service's docs/CONTRIBUTING.md pre-push checklist.
  • Docs updated in this PR where the change makes them wrong — API contract, architecture trade-offs, command summaries, and end-to-end recording guide. No new settings, scopes, or deployment variables.

Deployment notes

  • Command registration — /record start adds the optional subtitles boolean. The bot's existing preDeployCommand registers it; confirm exit 0 and both staging Registered … lines after deployment.

No migrations, environment variables, or new API keys are required. The protocol is opt-in: existing clients still ingest audio without receiving events. A new bot against an older meeting service disables subtitles after the ready timeout while recording continues, so no strict deploy order is required.

Anything you're unsure about

Live Discord/AWS integration has not been exercised. Staging should confirm permissions, recognition latency, retained-history search, and recovery behavior. Docker image checks were unavailable because the local Docker daemon was not running.

Finalized results have no fixed AWS latency bound. Ordering is guaranteed for event delivery and within each Discord batch, not across already-posted batches. Discord operation timeouts are local; a timed-out remote request may still complete, so cleanup is best-effort during outages. No durable queue or replay is added.

Subscribe in the first authenticated WebSocket message and serialize finalized subtitle events with per-session sequence numbers. Batch captions into dedicated Discord threads, suppress mentions, and flush/archive each session independently of minutes generation.

Document the wire contract, handshake, lifecycle, and out-of-order speech edge case. Cover authentication, queue failures, completion, session isolation, and Discord rendering with offline tests.
@github-actions github-actions Bot added size/xl >= 500 lines changed zone: discord-bot Owned by the discord-bot zone (docs/CODE-OWNERSHIP.md) zone: services/meeting Owned by the services/meeting zone (docs/CODE-OWNERSHIP.md) zone: docs Owned by the docs zone (docs/CODE-OWNERSHIP.md) zone: root Owned by the root zone (docs/CODE-OWNERSHIP.md) and removed size/xl >= 500 lines changed labels Oct 10, 2026
@kr1shap kr1shap self-assigned this Oct 10, 2026
@kr1shap
kr1shap marked this pull request as ready for review October 10, 2026 17:26

@andrei-akopian andrei-akopian left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed with Gemini, specifically looking for logic soundness and error / failure handling. Things like network relatex problems are all looking good, but there are some UX issues.

Findings:


B. Orphaned Starter Message in Text Channel

• In makeSubtitleAdapter, Discord requires a message to attach a public thread to:

  const starter = await parent.send({
    content: 'Live subtitles for this meeting.',
    allowedMentions: { parse: [] },
  });

• User confusion:
• Every meeting started with default options leaves a standalone message: "Live subtitles for this meeting." in the main text
channel.
• When the meeting ends, Misty archives the thread, but the starter message remains in chat forever with no update, no
timestamp, and no link to the final PDF minutes.
• Over a busy week, text channels will accumulate multiple stray "Live subtitles for this meeting." messages.


Voice Channel Text Chat Pitfall

• Discord voice channels now have their own integrated text chats (ChannelType.GuildVoice).
• Discord does not support creating public threads on messages inside voice channel text chats.
• User confusion: If a user joins voice and types /record start in that voice channel's chat, thread creation fails immediately.
Misty immediately sends the ⚠ Live subtitles are unavailable... warning, even though the user did nothing obviously wrong.


Incomplete Subtitle History vs Pristine PDF

• If the WebSocket drops momentarily, the bot's salvage path cleanly recovers and renders the full PDF report over HTTP.
• However, the subtitle thread will post:
Subtitles ended — history may be incomplete.
• User confusion: Users who see both the thread and the PDF might question whether the PDF minutes are also incomplete or
trustworthy, even though the PDF actually has the complete transcript.

Missing Subtitles Info in /record status

• Running /record status reports only the elapsed recording time and voice channel. It does not mention whether subtitles are
running, nor does it link to the subtitle thread for late-joiners.


Suggested Improvements for Better UX

  1. Clarify the Warning Notice:
    • Change:
    ⚠ Live subtitles are unavailable or incomplete. Recording and minutes are handled separately.
    • To:
    ⚠ Live subtitles have stopped (or could not be started in this channel). Voice recording is still active, and full minutes
    will be posted when the meeting ends.
  2. Update the Starter Message on Stop:
    • When archiving, edit the starter message in the main channel (e.g., starter.edit({ content: 'Live subtitles for this meeting
    (ended).' })) so users know the thread is closed without opening it.
  3. Link Subtitle Thread in /record status:
    • Include the subtitle thread link/mention in /record status output if active.

@kr1shap

kr1shap commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

Added starter message updating, so that when the thread is archived under that msg, it will say (archived) at the end.
/record status should link the thread link.
calling /record start will fail if done in a voice channel thread thingy, regardless if subtitles flag is set to false or not. Keep consistent behaviour so for subtitles to be able to create a thread in its parent channel.

Mark each subtitle starter archived after successful thread archive, reject recording starts from voice-channel text chat, and link active subtitles from record status. Update recording documentation and cover channel validation, lifecycle visibility, starter ownership, and failed archives with offline tests.
@github-actions github-actions Bot added the size/xl >= 500 lines changed label Oct 10, 2026

This branch was successfully deployed

1 active deployment
Misty / dev — 73820b4e Deployed Oct 11, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl >= 500 lines changed zone: discord-bot Owned by the discord-bot zone (docs/CODE-OWNERSHIP.md) zone: docs Owned by the docs zone (docs/CODE-OWNERSHIP.md) zone: root Owned by the root zone (docs/CODE-OWNERSHIP.md) zone: services/meeting Owned by the services/meeting zone (docs/CODE-OWNERSHIP.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants