Skip to content

Commit ffb2560

Browse files
d-csclaude
andcommitted
feat(run-ops): automatically migrate the dedicated run-ops database
Adds the ability to migrate the NEW dedicated run-ops database automatically, matching how every other DB in the system is migrated. - New `@internal/run-ops-database` migrate runner (scripts/migrate.mjs) exposed as `db:migrate:deploy` / `db:migrate:status`. Resolves the connection the same way the webapp does (RUN_OPS_DATABASE_URL, falling back to TASK_RUN_DATABASE_URL; direct endpoint preferred for migrations) and expands ${VAR} refs like Prisma's dotenv. - docker/scripts/entrypoint.sh runs the run-ops migration on boot when the DB is configured, gated by SKIP_RUN_OPS_MIGRATIONS. Single-DB installs are a no-op. - Unify the NEW-DB connection URL on a single canonical `runOpsNewDatabaseUrl` (RUN_OPS_DATABASE_URL ?? TASK_RUN_DATABASE_URL) across every consumer (connect path, split-decision predicates, replication source) so migrations always target the DB the app connects to. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 4ab8c7d commit ffb2560

10 files changed

Lines changed: 129 additions & 16 deletions

File tree

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: feature
4+
---
5+
6+
Automatically migrate the dedicated run-ops database on deploy (entrypoint + `@internal/run-ops-database` deploy/status scripts) and resolve its connection through one canonical `RUN_OPS_DATABASE_URL` (falling back to `TASK_RUN_DATABASE_URL`) so migrations always target the DB the app connects to.

apps/webapp/app/db.server.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import {
1010
import { RunOpsPrismaClient } from "@internal/run-ops-database";
1111
import invariant from "tiny-invariant";
1212
import { z } from "zod";
13-
import { env } from "./env.server";
13+
import { env, runOpsNewDatabaseUrl } from "./env.server";
1414
import { logger } from "./services/logger.server";
1515
import { isValidDatabaseUrl } from "./utils/db";
1616
import {
@@ -237,7 +237,7 @@ export function selectRunOpsTopology(
237237
// nothing new. The builders apply the SAME wrapper pair the control-plane
238238
// singletons use (captureInfrastructureErrors(tagDatasource(role, raw))).
239239
const runOpsTopology: RunOpsTopology = singleton("runOpsTopology", () => {
240-
const newUrl = env.TASK_RUN_DATABASE_URL;
240+
const newUrl = runOpsNewDatabaseUrl;
241241
// Gate on the opt-in flag too: the distinct-DB sentinel only runs when the flag is on.
242242
const splitEnabled = env.RUN_OPS_SPLIT_ENABLED && !!newUrl && !!env.TASK_RUN_LEGACY_DATABASE_URL;
243243

@@ -278,7 +278,7 @@ export const runOpsSplitReadEnabled: boolean = computeRunOpsSplitReadEnabled({
278278
newReplica: runOpsNewReplicaClient,
279279
controlPlaneWriter: prisma,
280280
controlPlaneReplica: $replica,
281-
hasNewUrl: !!env.TASK_RUN_DATABASE_URL,
281+
hasNewUrl: !!runOpsNewDatabaseUrl,
282282
hasLegacyUrl: !!env.TASK_RUN_LEGACY_DATABASE_URL,
283283
});
284284

apps/webapp/app/env.server.ts

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -132,12 +132,19 @@ const EnvironmentSchema = z
132132
// Explicit positive opt-in. Split behavior is unreachable unless this is true
133133
// AND the distinct-DB sentinel confirms the two URLs are physically distinct DBs.
134134
RUN_OPS_SPLIT_ENABLED: BoolEnv.default(false),
135-
// Datasource URL for the dedicated run-ops Prisma schema (migrations/generation).
136-
// The webapp runtime pool is driven by TASK_RUN_DATABASE_URL, not this var.
135+
// Canonical connection URL for the dedicated run-ops DB — drives the runtime pool, the split
136+
// decision, replication, and migrations (resolved via runOpsNewDatabaseUrl). Takes precedence
137+
// over TASK_RUN_DATABASE_URL, which remains as the compatibility fallback.
137138
RUN_OPS_DATABASE_URL: z
138139
.string()
139140
.refine(isValidDatabaseUrl, "RUN_OPS_DATABASE_URL is invalid")
140141
.optional(),
142+
// Direct/unpooled endpoint for the run-ops DB, consumed by the migration runner (poolers break
143+
// Prisma's advisory locks). Falls back to TASK_RUN_DATABASE_DIRECT_URL then the pooled URL.
144+
RUN_OPS_DATABASE_DIRECT_URL: z
145+
.string()
146+
.refine(isValidDatabaseUrl, "RUN_OPS_DATABASE_DIRECT_URL is invalid")
147+
.optional(),
141148
// The NEW dedicated run-ops DB writer. Optional so single-DB installs never set it.
142149
TASK_RUN_DATABASE_URL: z
143150
.string()
@@ -1724,8 +1731,8 @@ const EnvironmentSchema = z
17241731

17251732
// --- Run-ops DB split — second replication source (the NEW dedicated run-ops DB). ---
17261733
// Cloud-only; only consulted when isSplitEnabled() is true. Self-host never sets these.
1727-
// The NEW source's connection URL is TASK_RUN_DATABASE_URL; these add
1728-
// the NEW source's replication slot/publication and an explicit per-source enable so it can be
1734+
// The NEW source's connection URL is runOpsNewDatabaseUrl (RUN_OPS_DATABASE_URL ?? TASK_RUN_DATABASE_URL);
1735+
// these add the NEW source's replication slot/publication and an explicit per-source enable so it can be
17291736
// brought up independently of the legacy source during the transition.
17301737
RUN_REPLICATION_NEW_SLOT_NAME: z.string().default("task_runs_to_clickhouse_v2"),
17311738
RUN_REPLICATION_NEW_PUBLICATION_NAME: z
@@ -2102,3 +2109,10 @@ const EnvironmentSchema = z
21022109

21032110
export type Environment = z.infer<typeof EnvironmentSchema>;
21042111
export const env = EnvironmentSchema.parse(process.env);
2112+
2113+
// Canonical connection URL for the NEW dedicated run-ops DB. RUN_OPS_DATABASE_URL is the canonical
2114+
// name; TASK_RUN_DATABASE_URL is the compatibility fallback. Every NEW-DB consumer (connect path,
2115+
// split-decision predicates, replication source, migration runner) resolves through this single
2116+
// value so they can never disagree about which physical database the split targets.
2117+
export const runOpsNewDatabaseUrl: string | undefined =
2118+
env.RUN_OPS_DATABASE_URL ?? env.TASK_RUN_DATABASE_URL;

apps/webapp/app/services/runsReplicationInstance.server.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import invariant from "tiny-invariant";
2-
import { env } from "~/env.server";
2+
import { env, runOpsNewDatabaseUrl } from "~/env.server";
33
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
44
import { singleton } from "~/utils/singleton";
55
import { isSplitEnabled } from "~/v3/runOpsMigration/splitMode.server";
@@ -172,7 +172,7 @@ function initializeRunsReplicationInstance() {
172172
const sources = buildReplicationSources({
173173
splitEnabled,
174174
legacyUrl: DATABASE_URL,
175-
newUrl: env.RUN_OPS_DATABASE_URL ?? env.TASK_RUN_DATABASE_URL,
175+
newUrl: runOpsNewDatabaseUrl,
176176
newSourceOverride: env.RUN_REPLICATION_NEW_ENABLED === "disabled" ? false : undefined,
177177
legacySlotName: env.RUN_REPLICATION_SLOT_NAME,
178178
legacyPublicationName: env.RUN_REPLICATION_PUBLICATION_NAME,

apps/webapp/app/v3/runOpsMigration/controlPlaneResolver.server.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ import type {
55
RuntimeEnvironmentType,
66
} from "@trigger.dev/database";
77
import { prisma, $replica } from "~/db.server";
8-
import { env } from "~/env.server";
8+
import { env, runOpsNewDatabaseUrl } from "~/env.server";
99
import {
1010
ControlPlaneCache,
1111
DEFAULT_CP_CACHE_MAX_ENTRIES,
@@ -449,7 +449,7 @@ export class ControlPlaneResolver {
449449
// run-ops topology factory uses); the async isSplitEnabled() distinct-DB sentinel is enforced
450450
// at boot elsewhere and is never awaited on a resolver hot path.
451451
const SPLIT_ENABLED =
452-
env.RUN_OPS_SPLIT_ENABLED && !!env.TASK_RUN_DATABASE_URL && !!env.TASK_RUN_LEGACY_DATABASE_URL;
452+
env.RUN_OPS_SPLIT_ENABLED && !!runOpsNewDatabaseUrl && !!env.TASK_RUN_LEGACY_DATABASE_URL;
453453

454454
export const controlPlaneResolver = new ControlPlaneResolver({
455455
controlPlanePrimary: prisma,

apps/webapp/app/v3/runOpsMigration/splitMode.server.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
* infer split-vs-single from URL string-equality — distinctness is proven by the
55
* runtime sentinel.
66
*/
7-
import { env } from "~/env.server";
7+
import { env, runOpsNewDatabaseUrl } from "~/env.server";
88
import { logger } from "~/services/logger.server";
99
import { probeDistinctDatabases as defaultProbe } from "./distinctDbSentinel.server";
1010

@@ -71,7 +71,7 @@ export function isSplitEnabled(): Promise<boolean> {
7171
{
7272
flagEnabled: env.RUN_OPS_SPLIT_ENABLED,
7373
legacyUrl: env.TASK_RUN_LEGACY_DATABASE_URL,
74-
newUrl: env.TASK_RUN_DATABASE_URL,
74+
newUrl: runOpsNewDatabaseUrl,
7575
},
7676
{ logger }
7777
);

apps/webapp/app/v3/runStore.server.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import {
1010
runOpsNewPrismaClient,
1111
runOpsNewReplicaClient,
1212
} from "~/db.server";
13-
import { env } from "~/env.server";
13+
import { env, runOpsNewDatabaseUrl } from "~/env.server";
1414
import { singleton } from "~/utils/singleton";
1515

1616
type BuildRunStoreDeps = {
@@ -76,7 +76,7 @@ export function buildRunStore(deps: BuildRunStoreDeps): RunStore {
7676
// RUN_OPS_SPLIT_ENABLED. Reads must fan out across both DBs so a run that lives on the new
7777
// DB stays visible even with the flag off (matches the db.server topology factory). The flag
7878
// governs write/mint residency + migration via isSplitEnabled(), not read visibility.
79-
const ROUTING_ENABLED = !!env.TASK_RUN_DATABASE_URL && !!env.TASK_RUN_LEGACY_DATABASE_URL;
79+
const ROUTING_ENABLED = !!runOpsNewDatabaseUrl && !!env.TASK_RUN_LEGACY_DATABASE_URL;
8080

8181
// Resolve the run-ops handles, tolerating contexts where they are absent — tests that mock
8282
// ~/db.server minimally omit them, and accessing a missing export under vi.mock throws. A

docker/scripts/entrypoint.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,20 @@ else
1313
echo "SKIP_POSTGRES_MIGRATIONS=1, skipping Postgres migrations."
1414
fi
1515

16+
# Run-ops split: migrate the dedicated NEW run-ops database only when it is configured. Single-DB
17+
# installs never set the URL, so this is a no-op there.
18+
if [ -n "$RUN_OPS_DATABASE_URL" ] || [ -n "$TASK_RUN_DATABASE_URL" ]; then
19+
if [ "$SKIP_RUN_OPS_MIGRATIONS" != "1" ]; then
20+
echo "Running run-ops migrations"
21+
pnpm --filter @internal/run-ops-database db:migrate:deploy
22+
echo "Run-ops migrations done"
23+
else
24+
echo "SKIP_RUN_OPS_MIGRATIONS=1, skipping run-ops migrations."
25+
fi
26+
else
27+
echo "RUN_OPS_DATABASE_URL/TASK_RUN_DATABASE_URL not set, skipping run-ops migrations."
28+
fi
29+
1630
if [ "$SKIP_DASHBOARD_AGENT_MIGRATIONS" != "1" ]; then
1731
echo "Running dashboard agent migrations"
1832
pnpm --filter @internal/dashboard-agent-db db:migrate:deploy

internal-packages/run-ops-database/package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,8 @@
2727
"clean": "rimraf dist",
2828
"generate": "prisma generate",
2929
"db:migrate:dev:create": "prisma migrate dev --create-only",
30-
"db:migrate:deploy": "prisma migrate deploy",
30+
"db:migrate:deploy": "node scripts/migrate.mjs deploy",
31+
"db:migrate:status": "node scripts/migrate.mjs status",
3132
"db:push": "prisma db push",
3233
"test": "vitest run",
3334
"typecheck": "tsc --noEmit",
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// Run Prisma migrations against the dedicated NEW run-ops database (the second physical DB in the
2+
// split). It owns its own migration history, so it is migrated independently of the control-plane
3+
// DB. The connection is resolved the same way the webapp resolves it (RUN_OPS_DATABASE_URL, falling
4+
// back to TASK_RUN_DATABASE_URL) so migrations always target the DB the app connects to.
5+
//
6+
// Usage: node scripts/migrate.mjs [deploy|status] (defaults to deploy)
7+
import { spawnSync } from "node:child_process";
8+
import { readFileSync } from "node:fs";
9+
import { dirname, resolve } from "node:path";
10+
import { fileURLToPath } from "node:url";
11+
12+
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
13+
14+
// Read from local .env files so dev works without an exported env; deploy environments inject vars directly.
15+
function readFromEnvFiles(key) {
16+
for (const file of [resolve(packageRoot, ".env"), resolve(packageRoot, "../../.env")]) {
17+
let contents;
18+
try {
19+
contents = readFileSync(file, "utf8");
20+
} catch {
21+
continue;
22+
}
23+
for (const line of contents.split("\n")) {
24+
const match = line.match(/^\s*([A-Z0-9_]+)\s*=\s*(.*?)\s*$/);
25+
if (!match || match[1] !== key) continue;
26+
let value = match[2];
27+
if (
28+
(value.startsWith('"') && value.endsWith('"')) ||
29+
(value.startsWith("'") && value.endsWith("'"))
30+
) {
31+
value = value.slice(1, -1);
32+
}
33+
if (value) return value;
34+
}
35+
}
36+
return undefined;
37+
}
38+
39+
// Expand `${VAR}` refs (e.g. the repo .env's RUN_OPS_DATABASE_DIRECT_URL=${RUN_OPS_DATABASE_URL});
40+
// our manual .env reader loads them literally, unlike Prisma's dotenv-expand.
41+
const expand = (value) =>
42+
value?.replace(/\$\{(\w+)\}/g, (_, k) => process.env[k] ?? readFromEnvFiles(k) ?? "");
43+
const resolveVar = (key) => expand(process.env[key] || readFromEnvFiles(key));
44+
const redact = (url) => url.replace(/:\/\/[^@]*@/, "://***@");
45+
46+
const subcommand = process.argv[2] === "status" ? "status" : "deploy";
47+
48+
const databaseUrl = resolveVar("RUN_OPS_DATABASE_URL") || resolveVar("TASK_RUN_DATABASE_URL");
49+
// Prefer the direct/unpooled endpoint for migrations (poolers break Prisma's advisory locks).
50+
const directUrl =
51+
resolveVar("RUN_OPS_DATABASE_DIRECT_URL") ||
52+
resolveVar("TASK_RUN_DATABASE_DIRECT_URL") ||
53+
databaseUrl;
54+
55+
if (!databaseUrl) {
56+
// Single-DB installs never set these — safe no-op. A genuinely-expected DB is gated on by the caller.
57+
console.log(
58+
`run-ops migrate ${subcommand}: neither RUN_OPS_DATABASE_URL nor TASK_RUN_DATABASE_URL is set ` +
59+
"(checked env and .env). No dedicated run-ops database configured — skipping."
60+
);
61+
process.exit(0);
62+
}
63+
64+
console.log(
65+
`Running \`prisma migrate ${subcommand}\` against the run-ops database (${redact(databaseUrl)})`
66+
);
67+
68+
const result = spawnSync("prisma", ["migrate", subcommand, "--schema", "prisma/schema.prisma"], {
69+
cwd: packageRoot,
70+
stdio: "inherit",
71+
env: {
72+
...process.env,
73+
RUN_OPS_DATABASE_URL: databaseUrl,
74+
RUN_OPS_DATABASE_DIRECT_URL: directUrl,
75+
},
76+
});
77+
78+
process.exit(result.status ?? 1);

0 commit comments

Comments
 (0)