Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .claude/ways/kg/schema/way.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,21 @@ scope: agent, subagent

```
schema/
├── 00_baseline.sql # Core schema (AGE, auth, API tables)
└── migrations/ # Sequential migrations (NNN_name.sql)
├── 00_baseline.sql # Checkpoint baseline (migrations 001-080 consolidated)
├── migrations/ # Post-checkpoint migrations (081+)
│ └── archived/ # 001-080, superseded by baseline — never edit
├── migrations-warm/ # Migrations needing a running graph engine
└── scripts/checkpoint.sh # Regenerate the checkpoint (replay + dump + verify)
```

## Creating Migrations

1. Create `schema/migrations/NNN_descriptive_name.sql`
1. Create `schema/migrations/NNN_descriptive_name.sql` (next number after 080/latest)
2. Use next sequential number
3. Migrations auto-apply on `./operator.sh restart postgres`

`00_baseline.sql` is generated by `checkpoint.sh` — regenerate it, don't hand-edit.

## Query Language

**Apache AGE uses openCypher** (not Neo4j Cypher):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,19 @@ COMMIT;
- Merge all migrations into new baseline: `schema/migrations/stable_v2_baseline.sql`
- Archive old migrations in `schema/migrations/archived/`

> **Executed 2026-07-01 (checkpoint v080):** migrations 001–080 were
> consolidated into `schema/00_baseline.sql` and moved to
> `schema/migrations/archived/`. The consolidation is generated, not
> hand-merged: `schema/scripts/checkpoint.sh` replays the old baseline plus
> every migration into a throwaway Apache AGE container via raw psql, dumps
> the resting state (relational DDL + seed data via pg_dump; graph-seeding
> cypher migrations carried verbatim, since AGE graphs cannot round-trip
> through pg_dump), and verifies the candidate by building a second container
> from it alone and diffing schema, AGE labels, row counts, and normalized
> data against the replayed original. The baseline records versions 1–80 in
> `schema_migrations`; the runner scans `archived/` before `migrations/`, so
> pre-checkpoint databases keep their incremental upgrade path.

---

## Consequences
Expand Down
1,446 changes: 797 additions & 649 deletions docs/reference/schema.md

Large diffs are not rendered by default.

16 changes: 11 additions & 5 deletions operator/database/migrate-db.sh
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,12 @@ if [ ! -d "$MIGRATIONS_DIR" ]; then
exit 0
fi

# Scan for migration files
for migration_file in "$MIGRATIONS_DIR"/*.sql; do
# Scan for migration files. archived/ holds migrations consolidated into the
# checkpoint baseline (schema/00_baseline.sql): fresh databases have all of
# them recorded by the baseline and skip them; databases created before the
# checkpoint still find their pending versions there. Scanned first so
# archived versions sort ahead of post-checkpoint migrations.
for migration_file in "$MIGRATIONS_DIR"/archived/*.sql "$MIGRATIONS_DIR"/*.sql; do
[ -f "$migration_file" ] || continue

# Extract version from filename (001_baseline.sql → 001)
Expand Down Expand Up @@ -292,9 +296,11 @@ for pending in "${PENDING_MIGRATIONS[@]}"; do
echo -e "${GRAY}----------------------------------------${NC}"
fi

# Apply migration (capture output to detect errors)
MIGRATION_OUTPUT=$(run_psql_file "$FILE" 2>&1)
MIGRATION_EXIT_CODE=$?
# Apply migration (capture output to detect errors). The || arm is
# required: under set -e a bare command-substitution assignment aborts
# the whole script on psql failure, making the skip-and-retry branch
# below unreachable.
MIGRATION_OUTPUT=$(run_psql_file "$FILE" 2>&1) && MIGRATION_EXIT_CODE=0 || MIGRATION_EXIT_CODE=$?

# Show output in verbose mode
if [ "$VERBOSE" = true ]; then
Expand Down
Loading
Loading