feat(backend): Phase-1 to Signal migration path - #533
Merged
codebestia merged 3 commits intoAug 3, 2026
Conversation
feat(backend): Phase-1 to Signal migration path Defines how existing sealed-box conversations move to Signal without losing history. Clients update at their own pace, so a conversation holds devices on both sides of the line until the slowest one catches up. Three rules, enforced server side: 1. History is never re-encrypted. message_envelopes.protocol is added NOT NULL DEFAULT sealed_box, which labels every pre-existing envelope with the construction that actually encrypted it in the same statement. A client always knows which decryption path to use instead of inferring it from the ciphertext or from a timestamp. 2. A conversation cuts over only when every active device on every side advertises Signal support. One un-upgraded device holds the whole conversation on sealed box, because the sender has to produce something that device can open. 3. After cutover a sealed-box envelope is refused, so a patched client cannot quietly keep everyone on the weaker construction. - devices.supports_signal is the capability flag, advertised via PATCH /devices/:id/capabilities or at registration. It is monotonic: a device may turn it on but not off, since withdrawal would be a downgrade lever indistinguishable from a genuine rollback. A device that lost its Signal state re-registers under a new identity key - GET /conversations/:id/e2ee-protocol reports the negotiated protocol and which devices are blocking the cutover, so the UI can name them - GET /conversations/:id/devices carries the negotiated protocol alongside the device set the client is about to encrypt for - both send paths reject a Signal envelope aimed at a device that cannot read it (400) and a sealed-box downgrade after cutover (409), each carrying the negotiated protocol and the offending device ids - sync, delivery and history read paths report protocol per envelope, so catching up across the cutover returns a mix and loses nothing Adds docs/signal-migration.md with the data model, endpoints, cutover timeline and rollout order, linked from docs/signal-integration.md.
|
@Olorunfemi20 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Owner
|
Please resolve the conflicts. |
Merge upstream/main into feat/signal-migration-path Rebuilds the Phase-1 to Signal migration on the device capability model that landed on main while this branch was open, and repairs merge damage in the files it touches. Reworked onto devices.capabilities: - drops the bespoke devices.supports_signal flag. main gained a capabilities jsonb document listing the protocols a device can decrypt, plus selectProtocol() to pick the strongest one two devices share. Two sources of truth for "can this device do Signal" is exactly the kind of divergence the repo already fixed once when user_devices was merged into devices, so the flag is gone and negotiation reads capabilities - drops PATCH /devices/:id/capabilities. POST /auth/verify already treats a newer capabilities document as the upgrade path - drops GET /conversations/:id/e2ee-protocol. GET /conversations/:id/devices already returns each device capabilities and the negotiated protocol - negotiation is now per device pair rather than per conversation. Since each recipient device already gets its own envelope, an upgraded pair can use Signal immediately instead of waiting for the slowest device in the conversation, which in a large group may never happen What this branch still adds, because main has no equivalent: - message_envelopes.protocol, NOT NULL DEFAULT sealed_box, backfilling every existing row. Capabilities describe what a device can do now, which cannot interpret an envelope written months ago; recording the construction per envelope is what keeps pre-cutover history decryptable - send-path enforcement on both REST and socket: an envelope naming a protocol the recipient does not advertise is refused with 400, and one weaker than both sides support with 409, each listing the offending envelopes - protocol reported on every read path so a client catching up across the cutover decrypts each envelope with the matching routine Migration renumbered to 0004_envelope_protocol.sql; existing journal entries left byte-identical. Repairs to files this branch touches, which do not compile on main after earlier merges: - schemas/message.schemas.ts declared SendMessageSchema twice - routes/messages.ts destructured the request body twice - socket/messaging.ts had the sibling-coverage check duplicated with an unbalanced brace, and send_file_message lost its payload destructuring - db/schema.ts had the mlsKeyPackages and deviceKeyHistory tables spliced into one another - services/deliveryPipeline.ts referenced welcomeTransport without declaring it routes/messages.ts now uses the shared insertMessageEnvelopes helper the socket paths already use, so the per-envelope protocol default cannot drift between the two implementations again.
fix(backend): repair merge damage in routes broken on main These five files do not parse on main after earlier conflict resolutions dropped closing braces and spliced handlers together. Backend CI cannot type-check the package while they are in this state. - conversations.ts: the group-invite rate-limit guard lost its return and closing brace, swallowing the rest of POST /:id/members - devices.ts: duplicate db/schema import, an import statement whose opening line was lost, and the device_linked audit event hoisted above the signature check where row and existing are not yet in scope - files.ts: two GET /:fileId route openers, one unclosed, and the membership guard fused with the audit-event block - uploads.ts: the MLS group check lost its closing brace, swallowing the daily upload quota - users.ts: the key-bundle prekey-low signal fused with the audit block and referenced an undeclared variable; GET /:id/key-fingerprint had lost its implementation entirely and carried key-bundle code instead; PATCH /me was truncated mid-statement and fused with a second, bogus key-fingerprint route referencing undefined identifiers The fingerprint implementation is restored to the contract its existing test pins: sorted identity keys, SHA-256, two 30-digit segments from non-overlapping halves, returned raw and in groups of five. uploads.test.ts had an unterminated vi.mock factory from the same cause. Each repair keeps both sides of the original merge; no feature is dropped.
Contributor
Author
|
@codebestia CI on this PR cannot be fixed from inside it, and the same failure is affecting All four jobs fail at "Install dependencies", before any code in this PR runsFormat, lint, test and build never execute. This PR changes 19 files under The lockfile cannot simply be regenerated
It appears zero times in I have deliberately not removed or substituted it. Picking a different MLS WASM binding is a product and security decision, and Not specific to this PRBackend CI has failed on What unblocks it
Happy to open that PR if you tell me which way you want Related: several backend files do not parse on mainSeparately from the install failure, some earlier merges landed conflict resolutions that dropped closing braces and spliced handlers together, so
Each repair keeps both sides of the original merge; no feature is dropped. Roughly 164 type errors remain elsewhere in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #364
Summary
Defines how existing Phase-1 sealed-box conversations transition to Signal without losing history. Clients update at their own pace, so a conversation contains devices on both sides of the line for as long as the slowest one takes.
This has been reworked onto the
devices.capabilitiesmodel that landed onmainwhile the PR was open — see "Rebased onto the capability model" below for what was dropped and why.What this builds on
mainalready has capability advertisement:devices.capabilitieslists theprotocolsa device can decrypt, andselectProtocol(src/lib/capabilities.ts) picks the strongest one two devices share, falling back to thesealed_boxbaseline.GET /conversations/:id/devicesalready returns each device's capabilities and the negotiated protocol.This PR adds the two things negotiation alone does not provide.
1. A record of what actually encrypted each envelope
message_envelopes.protocol—e2ee_protocol NOT NULL DEFAULT 'sealed_box'(enum mirroringKNOWN_PROTOCOLS). Adding it with that default backfills every existing row in the same statement.This is the no-history-loss guarantee. Capabilities describe what a device can do now, which says nothing about what encrypted a message last month — so capability alone cannot be used to pick a decryption path for old data. Recording the construction per envelope is what keeps pre-cutover history decryptable forever.
Every read path reports it (
GET /sync,message_envelopesocket events, conversation history), so a client catching up across the cutover receives a mix ofsealed_boxandsignalenvelopes in one page and decrypts each with the matching routine. Nothing is migrated or re-encrypted.2. Enforcement on the way in
A sender can claim any protocol on an envelope. Both send paths (REST
POST /messagesand socketsend_message) now reject two things:400unsupported_by_recipient409downgradeBoth list every offending envelope with
declared,expectedandreason, so the client re-fetches the device set and rebuilds rather than retrying blind. When a batch contains both kinds, the undecryptable one decides the status code — it is the more specific failure.MLS group messages carry a single group ciphertext and no per-device envelopes (#372), so the check does not run for them.
Rebased onto the capability model
Dropped from the original version of this PR, because
mainnow covers them:devices.supports_signal. A bespoke boolean alongsidecapabilities.protocolswould be two sources of truth for "can this device do Signal" — exactly the divergence the repo already fixed once whenuser_deviceswas merged intodevices.PATCH /devices/:id/capabilities.POST /auth/verifyalready treats a newercapabilitiesdocument as the upgrade path.GET /conversations/:id/e2ee-protocol.GET /conversations/:id/devicesalready answers this per device.Changed:
main's pairwiseselectProtocolis the better primitive and this now enforces exactly what it returns.Net effect: the PR is smaller and composes with what shipped, instead of competing with it.
Migration
Renumbered to
drizzle/0004_envelope_protocol.sql(0001–0003are taken onmain). Existing_journal.jsonentries are left byte-identical — only a new entry is appended.Merge repairs
Several files this branch touches do not compile on
mainafter earlier merges — duplicated blocks and lost declarations from conflict resolutions. They are fixed here because the branch cannot build otherwise:schemas/message.schemas.ts—SendMessageSchemadeclared twiceroutes/messages.ts— the request body destructured twicesocket/messaging.ts— sibling-coverage check duplicated with an unbalanced brace;send_file_messagelost its payload destructuringdb/schema.ts—mlsKeyPackagesanddeviceKeyHistoryspliced into one anotherservices/deliveryPipeline.ts—welcomeTransportreferenced without being declaredEach repair preserves both merged features; nothing from
mainis dropped. The removals in thesocket/messaging.tsdiff are exactly the duplicated and now-dead code.mainis also broken inroutes/users.ts,routes/conversations.ts,routes/files.ts,routes/uploads.tsandroutes/devices.ts, which this branch does not touch. Those are left alone deliberately —users.tsin particular has aPATCH /mehandler fused with a bogus duplicate route referencing undefined variables, and involves privacy-settings work I was not part of. That needs its own PR; I'm happy to open one.Refactor
routes/messages.tsnow uses the sharedinsertMessageEnvelopeshelper that the socket paths already use, instead of its own copy of the resolve-devices-and-insert logic. The per-envelope protocol default lives in one place, so the two implementations cannot drift apart again — which is what produced the duplication above.Tests
e2eeProtocol.test.ts— 13 cases: sealed box accepted for a Phase-1 recipient, Signal accepted when both advertise it, Signal-to-Phase-1 rejected, post-cutover downgrade rejected, a mixed batch where one pair uses Signal and another stays on sealed box (the per-pair property), undecryptable reported ahead of downgrade, and null/absent capability documents treated as baseline at both ends.signalMigration.routes.test.ts— 8 cases overPOST /messages: protocol defaulting, persistence, both rejection codes with their violations, unknown protocol rejected at the schema layer, membership checked first, and the check skipped for envelope-free MLS messages.75 tests pass across my four suites.
tscandeslintare clean on every file this branch touches.Docs
apps/backend/docs/signal-migration.mdrewritten for the capability model: what it builds on, the three rules, why negotiation is per-pair, the data model, advertising support via re-verify, both rejection codes, the read contract, a per-pair cutover diagram, and the rollout order — including that dual-path decryption must ship before Signal encryption.