Context: we ship a production product (FrameOS) built on the classic OpenCut web editor (MIT — thank you). On top of it we built a full caption system — visual style picker, transcript import, word-level karaoke highlights, per-range restyling — and shipped it across ~10 releases. Since the rewrite is plugin-first and will need a first-class captions story, here's the data model that survived production and the five places it fought us, which together are an argument for captions as a typed core primitive rather than a convention. (Filed as architecture-phase input per the README; related: #811.)
The model that worked
Captions as a group of ordinary timeline elements: every element of a caption run carries a captionGroupId plus a captionRole (chunk — the visible text block — or highlight — the karaoke word overlay). Word-level karaoke is just per-word overlay elements on their own track, each one's window running to the next word's start. Style edits fan out group-wide with role-aware rules (e.g. color/background apply to chunks only, because a highlight's color is the karaoke; size changes rescale highlight offsets proportionally).
Building on plain elements bought us rendering, transforms, dragging, and keyframe animation for free.
Where it fought us — the case for a first-class primitive
- Split: the generic split command silently no-ops on elements not containing the playhead, so "split captions at playhead" (a bread-and-butter captioning action) needed special-casing.
- Drag: retiming a caption chunk must carry its karaoke companions; group-move semantics had to be bolted onto the interaction controller.
- Replace vs. coexist: "add captions" must find existing caption runs to replace — stamped groups, legacy caption-named tracks, word-shaped highlight tracks. Identity-by-convention is exactly as fragile as it sounds.
- Timeline rendering: caption cues want to render as one continuous ribbon of flush segments (the Resolve/Premiere convention — zero-inset, hairline separators). We tried collapsing them into a single quiet bar and reverted: it broke split and selection, because the bar wasn't the thing the commands operated on.
- Restyling: mutating styled elements in place lost information. What worked: stamp a
captionSource (pre-preset text + word timings) on insert, and rebuild cues from it on restyle. Selection is the styling scope: one cue selected = whole-group fan-out; several = range-only.
Two robustness notes from production incidents
- Caption sizes must come from the render/export spec, not preview-space values. Ours looked fine in the picker and rendered invisible on canvas until the two agreed.
- Font loading must be non-fatal.
FontFace.load() rejects with a stack-less DOMException("NetworkError") when, e.g., an ad-blocker eats a Google Fonts request — and if project load awaits fonts inside its fatal path, a cosmetic failure kills the whole project open. Fonts are cosmetic; degrade, don't abort.
Distilled suggestion for the rewrite
Make caption group + role + rebuild-source a typed primitive in the core model (with karaoke windows as derived data), and give split/drag/replace group-aware semantics there — rather than free elements plus conventions. A plugin API then gets captions "for free", and every editor UI (desktop, mobile, browser) renders the same ribbon.
Happy to elaborate on any of these, including the exact fan-out rules and the failure modes we hit.
Context: we ship a production product (FrameOS) built on the classic OpenCut web editor (MIT — thank you). On top of it we built a full caption system — visual style picker, transcript import, word-level karaoke highlights, per-range restyling — and shipped it across ~10 releases. Since the rewrite is plugin-first and will need a first-class captions story, here's the data model that survived production and the five places it fought us, which together are an argument for captions as a typed core primitive rather than a convention. (Filed as architecture-phase input per the README; related: #811.)
The model that worked
Captions as a group of ordinary timeline elements: every element of a caption run carries a
captionGroupIdplus acaptionRole(chunk— the visible text block — orhighlight— the karaoke word overlay). Word-level karaoke is just per-word overlay elements on their own track, each one's window running to the next word's start. Style edits fan out group-wide with role-aware rules (e.g. color/background apply to chunks only, because a highlight's color is the karaoke; size changes rescale highlight offsets proportionally).Building on plain elements bought us rendering, transforms, dragging, and keyframe animation for free.
Where it fought us — the case for a first-class primitive
captionSource(pre-preset text + word timings) on insert, and rebuild cues from it on restyle. Selection is the styling scope: one cue selected = whole-group fan-out; several = range-only.Two robustness notes from production incidents
FontFace.load()rejects with a stack-lessDOMException("NetworkError")when, e.g., an ad-blocker eats a Google Fonts request — and if project load awaits fonts inside its fatal path, a cosmetic failure kills the whole project open. Fonts are cosmetic; degrade, don't abort.Distilled suggestion for the rewrite
Make caption group + role + rebuild-source a typed primitive in the core model (with karaoke windows as derived data), and give split/drag/replace group-aware semantics there — rather than free elements plus conventions. A plugin API then gets captions "for free", and every editor UI (desktop, mobile, browser) renders the same ribbon.
Happy to elaborate on any of these, including the exact fan-out rules and the failure modes we hit.