Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions apps/cli/docs/supabase/inspect/db-toast-sizes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# db-toast-sizes

This command displays TOAST table sizes and dead chunk counts for every user table that has a TOAST relation. When a column value exceeds ~2 kB (TEXT, JSONB, bytea), Postgres stores it out-of-line in a companion TOAST table. Autovacuum runs on the TOAST table independently from the main heap, so it can accumulate dead chunks even when the parent table looks healthy by its own dead-tuple count.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · documentation · source: claude

The documentation incorrectly describes the approximately 2 kB TOAST threshold as applying to an individual column value rather than the row tuple.

Evidence: docs/supabase/inspect/db-toast-sizes.md:3 says Postgres moves a value out-of-line when that value exceeds approximately 2 kB. PostgreSQL instead applies the threshold to the tuple and compresses or moves eligible attributes until it fits.

Suggested fix: Explain that when a row exceeds the threshold, PostgreSQL compresses and/or moves eligible variable-length attributes out of line.


A table that appears fine by `vacuum-stats` or `bloat` alone can still have significant TOAST bloat that wastes disk space and slows reads. High `TOAST Dead %` values indicate that autovacuum is not keeping up with the TOAST table and a manual `VACUUM` may be needed.

```
TABLE │ TOTAL SIZE │ HEAP SIZE │ TOAST SIZE │ TOAST LIVE CHUNKS │ TOAST DEAD CHUNKS │ TOAST DEAD % │ LAST AUTOVACUUM │ LAST VACUUM
────────────────────────┼────────────┼───────────┼────────────┼───────────────────┼───────────────────┼──────────────┼────────────────────┼─────────────
public.documents │ 4200 MB │ 800 MB │ 3400 MB │ 250000 │ 18000 │ 6.7 │ 2024-03-01 04:12 │
public.media_assets │ 890 MB │ 120 MB │ 770 MB │ 80000 │ 200 │ 0.2 │ 2024-03-02 01:45 │
public.messages │ 340 MB │ 280 MB │ 60 MB │ 95000 │ 0 │ 0.0 │ 2024-03-02 03:10 │
```
5 changes: 3 additions & 2 deletions apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# `supabase inspect db <subcommand>`

Single shared side-effect document for all 13 active `inspect db` subcommands and
Single shared side-effect document for all 14 active `inspect db` subcommands and
their 12 deprecated aliases. Every subcommand has the same surface — it resolves a
Postgres connection from `--db-url` / `--linked` / `--local`, runs one read-only
`SELECT`, and renders the result as a Glamour ASCII table. They differ only in the
Expand Down Expand Up @@ -46,7 +46,7 @@ no new config reads.
## Database Queries

Each subcommand runs one read-only `SELECT` (the embedded Go `<name>.sql`). The
5 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list;
6 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list;
`db-stats` additionally takes `$2` = the database name.

| Subcommand | SQL file | InternalSchemas param? |
Expand All @@ -64,6 +64,7 @@ Each subcommand runs one read-only `SELECT` (the embedded Go `<name>.sql`). The
| long-running-queries | long_running_queries.sql | no |
| role-stats | role_stats.sql | no |
| traffic-profile | traffic_profile.sql | no |
| toast-sizes | toast-sizes.query.ts | yes (`$1`) |

Deprecated aliases run an active subcommand's query: `cache-hit`→db-stats;
`index-usage`/`total-index-size`/`index-sizes`/`unused-indexes`/`seq-scans`/`table-record-counts`→index-stats;
Expand Down
2 changes: 2 additions & 0 deletions apps/cli/src/commands/inspect/db/db.command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import { inspectDbTableStatsCommand } from "./table-stats/table-stats.command.ts
import { inspectDbTotalIndexSizeCommand } from "./total-index-size/total-index-size.command.ts";
import { inspectDbTotalTableSizesCommand } from "./total-table-sizes/total-table-sizes.command.ts";
import { inspectDbTrafficProfileCommand } from "./traffic-profile/traffic-profile.command.ts";
import { inspectDbToastSizesCommand } from "./toast-sizes/toast-sizes.command.ts";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · style · source: claude

The new imports do not follow the established ordering in either the command registry or sibling command files.

Evidence: db.command.ts:22-27 places toast-sizes after unused-indexes rather than before the total-* imports in the otherwise grouped ordering. toast-sizes.command.ts:1-4 places its handler import last, while every other sibling imports its handler immediately after Effect's Command import.

Suggested fix: Move the registry import before total-index-size and move the handler import to the second line of toast-sizes.command.ts.

import { inspectDbUnusedIndexesCommand } from "./unused-indexes/unused-indexes.command.ts";
import { inspectDbVacuumStatsCommand } from "./vacuum-stats/vacuum-stats.command.ts";

Expand All @@ -42,6 +43,7 @@ export const inspectDbCommand = Command.make("db").pipe(
inspectDbVacuumStatsCommand,
inspectDbTableStatsCommand,
inspectDbTrafficProfileCommand,
inspectDbToastSizesCommand,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · documentation · source: claude

Adding toast-sizes raises the inspect-db leaf count to 26, leaving four comments that still say 25.

Evidence: apps/cli/src/commands/inspect/db/db.command.ts:32-59 registers 26 leaves. Stale counts remain at inspect-db-command.ts:10 and :40, db.layers.ts:13, and db.layers.unit.test.ts:9.

Suggested fix: Change the four counts from 25 to 26.

inspectDbCacheHitCommand,
inspectDbIndexUsageCommand,
inspectDbTotalIndexSizeCommand,
Expand Down
2 changes: 1 addition & 1 deletion apps/cli/src/commands/inspect/db/db.layers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { inspectBaseLayer } from "../inspect.layers.ts";
* deprecated alias like `"cache-hit"`) and is appended to `["inspect", "db"]`. This
* path is what `withCommandTelemetry` records as the PostHog
* `cli_command_executed` `command` property: the inspect tree is a real 3-level
* hierarchy, so each of the 25 leaves emits a distinct command name. A shared
* hierarchy, so each of the 26 leaves emits a distinct command name. A shared
* `["inspect", "db"]` path would collapse them all into one event, so each leaf must
* pass its own name — and a deprecated alias records the alias the user typed, not
* the backend command it delegates to.
Expand Down
4 changes: 2 additions & 2 deletions apps/cli/src/commands/inspect/db/inspect-db-command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import type { InspectConnectionFlags } from "./inspect-query.ts";

/**
* The `inspect` persistent flag set, inherited by every `inspect db` subcommand.
* Shared verbatim across all 25 commands
* Shared verbatim across all 26 commands
* so the flag names and descriptions live in one place. `Command.make` reads this
* immutable descriptor without mutating it, so a single instance is safe to reuse.
*/
Expand Down Expand Up @@ -37,7 +37,7 @@ export const INSPECT_DB_FLAGS = {
* Wraps an `inspect db` handler with the standard command-level pipeline: legacy
* telemetry instrumentation (the Go-shape `cli_command_executed` event, with the
* three connection flags) and the machine-format JSON error envelope. Shared by
* all 25 command files so the wiring is defined once.
* all 26 command files so the wiring is defined once.
*/
export function inspectDbCommandHandler<E, R>(
handler: (flags: InspectConnectionFlags) => Effect.Effect<void, E, R>,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { outliersSpec } from "./outliers/outliers.query.ts";
import { replicationSlotsSpec } from "./replication-slots/replication-slots.query.ts";
import { roleStatsSpec } from "./role-stats/role-stats.query.ts";
import { tableStatsSpec } from "./table-stats/table-stats.query.ts";
import { toastSizesSpec } from "./toast-sizes/toast-sizes.query.ts";
import { trafficProfileSpec } from "./traffic-profile/traffic-profile.query.ts";
import { vacuumStatsSpec } from "./vacuum-stats/vacuum-stats.query.ts";

Expand Down Expand Up @@ -248,11 +249,27 @@ const cases: ReadonlyArray<Case> = [
},
expect: ["public", "100", "50", "12.0", "1:1 (Balanced)"],
},
{
spec: toastSizesSpec,
params: "schemas1",
row: {
name: "public.events",
total_size: "120 kB",
heap_size: "80 kB",
toast_size: "40 kB",
toast_live_chunks: 1200,
toast_dead_chunks: 80,
toast_dead_pct: "6.3",
last_autovacuum: "2025-01-15 03:00",
last_vacuum: "",
},
expect: ["public.events", "120 kB", "40 kB", "1200", "80", "6.3", "2025-01-15 03:00"],
},
Comment on lines +253 to +267

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MINOR · test-coverage · source: claude

toast-sizes is omitted from the inspect cli-e2e subcommand matrix and has no PostgreSQL fixture, unlike every other active inspect-db subcommand.

Evidence: apps/cli-e2e/src/tests/inspect.e2e.test.ts:31-45 lists 13 active subcommands and drives success and connection-failure subprocess tests at lines 66-90. apps/cli-e2e/fixtures/pg contains matching fixtures for those 13 commands but none for toast-sizes. The added integration case exercises runInspectQuery directly, not command registration or subprocess wiring.

Suggested fix: Add a toast-sizes fixture and entry to the cli-e2e SUBCOMMANDS table.

];

describe("inspect db specs (per-subcommand correctness)", () => {
it("covers all 13 active subcommands", () => {
expect(cases).toHaveLength(13);
it("covers all 14 active subcommands", () => {
expect(cases).toHaveLength(14);
});

for (const testCase of cases) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { Command } from "effect/unstable/cli";
import { INSPECT_DB_FLAGS, inspectDbCommandHandler } from "../inspect-db-command.ts";
import { inspectDbRuntimeLayer } from "../db.layers.ts";
import { inspectDbToastSizes } from "./toast-sizes.handler.ts";

export const inspectDbToastSizesCommand = Command.make("toast-sizes", INSPECT_DB_FLAGS).pipe(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MINOR · documentation · source: claude

The new command lacks a docs default override, so its published --linked default will incorrectly appear as false even though omitting target flags selects the linked project.

Evidence: apps/cli/src/commands/inspect/db/inspect-db-command.ts:21-24 declares the primitive default as false, while apps/cli/src/commands/inspect/db/inspect-query.ts:193-196 defaults execution to linked. apps/cli/src/docs/docs-spec.ts:254 falls back to the primitive default, and apps/cli/src/docs/docs-spec.tables.ts:167-179 overrides every other active inspect-db command but not toast-sizes.

Suggested fix: Add "supabase-inspect-db-toast-sizes linked": "true" to DOCS_DEFAULT_OVERRIDES.

Command.withDescription(
"Displays TOAST table sizes and dead chunk counts for every user table that has overflow storage. " +
"Autovacuum runs on TOAST relations independently, so dead chunks can accumulate even when the " +
"main heap looks healthy.",
),
Comment on lines +7 to +11

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · consistency · source: claude

The command's five-sentence help description is substantially longer than every sibling inspect-db description.

Evidence: toast-sizes.command.ts:7-13 contains five sentences. The other 25 command files each use a single-sentence description; examples include vacuum-stats.command.ts:7 and traffic-profile.command.ts:10-12.

Suggested fix: Use a concise one-sentence CLI description and retain the detailed explanation in the docs overlay.

Command.withShortDescription("Show TOAST table sizes and dead chunk counts"),
Command.withHandler(inspectDbCommandHandler(inspectDbToastSizes)),
Command.provide(inspectDbRuntimeLayer("toast-sizes")),
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import { makeInspectDbHandler } from "../inspect-query.ts";
import { toastSizesSpec } from "./toast-sizes.query.ts";

export const inspectDbToastSizes = makeInspectDbHandler(toastSizesSpec, "inspect.db.toast-sizes");
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import {
inspectFloat1,
inspectInt,
inspectPlainText,
inspectText,
type InspectQuerySpec,
} from "../inspect-query.ts";
import { INTERNAL_SCHEMAS, likeEscapeSchema } from "../inspect-schemas.ts";

const SQL = `
SELECT
FORMAT('%I.%I', n.nspname, main.relname) AS name,
pg_size_pretty(pg_total_relation_size(main.oid)) AS total_size,
pg_size_pretty(pg_relation_size(main.oid)) AS heap_size,
pg_size_pretty(pg_total_relation_size(main.reltoastrelid)) AS toast_size,
COALESCE(ts.n_live_tup, 0) AS toast_live_chunks,
COALESCE(ts.n_dead_tup, 0) AS toast_dead_chunks,
COALESCE(
round(100.0 * ts.n_dead_tup / nullif(ts.n_live_tup + ts.n_dead_tup, 0), 1),
0.0
) AS toast_dead_pct,
COALESCE(to_char(ts.last_autovacuum, 'YYYY-MM-DD HH24:MI'), '') AS last_autovacuum,
COALESCE(to_char(ts.last_vacuum, 'YYYY-MM-DD HH24:MI'), '') AS last_vacuum
FROM pg_class main
JOIN pg_namespace n ON n.oid = main.relnamespace
LEFT JOIN pg_stat_all_tables ts ON ts.relid = main.reltoastrelid
WHERE main.relkind = 'r'
AND main.reltoastrelid <> 0
AND pg_total_relation_size(main.reltoastrelid) > 0
AND NOT n.nspname LIKE ANY($1)
ORDER BY pg_total_relation_size(main.reltoastrelid) DESC`;

export const toastSizesSpec: InspectQuerySpec = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · consistency · source: claude

toastSizesSpec is the only active inspect-db query spec without the explanatory JSDoc used by all sibling specs.

Evidence: toast-sizes.query.ts:32 declares the spec without JSDoc. Each of the other 13 active query files has a JSDoc immediately before its exported InspectQuerySpec, such as table-stats.query.ts:32-37 and vacuum-stats.query.ts:68-73.

Suggested fix: Add a short JSDoc describing the report and its notable size and timestamp semantics.

name: "toast-sizes",
sql: SQL,
params: () => [likeEscapeSchema(INTERNAL_SCHEMAS)],
headers: [
"Table",
"Total Size",
"Heap Size",
"TOAST Size",
"TOAST Live Chunks",
"TOAST Dead Chunks",
"TOAST Dead %",
"Last Autovacuum",
"Last Vacuum",
],
project: (row) => [
inspectText(row["name"]),
inspectText(row["total_size"]),
inspectText(row["heap_size"]),
inspectText(row["toast_size"]),
inspectInt(row["toast_live_chunks"]),
inspectInt(row["toast_dead_chunks"]),
inspectFloat1(row["toast_dead_pct"]),
inspectPlainText(row["last_autovacuum"]),
inspectPlainText(row["last_vacuum"]),
],
};
Original file line number Diff line number Diff line change
Expand Up @@ -190,12 +190,12 @@ describe("inspect report", () => {
return Effect.gen(function* () {
yield* inspectReport(flags({ outputDir: base }));
const { dir, files } = dateFolderContents(base);
expect(files.length).toBe(14);
expect(files.length).toBe(15);
expect(files).toContain("db_stats.csv");
expect(files).toContain("unused_indexes.csv");
expect(files).not.toContain("db-stats.csv");
// Every query was copied with both placeholders substituted.
expect(connection.copiedSql.length).toBe(14);
expect(connection.copiedSql.length).toBe(15);
expect(
connection.copiedSql.every(
(s) => s.startsWith("COPY (") && s.endsWith("TO STDOUT WITH CSV HEADER"),
Expand Down Expand Up @@ -466,10 +466,10 @@ describe("inspect report", () => {
const data = (
success as { data?: { files?: Array<unknown>; outputDir?: string; rules?: Array<unknown> } }
).data;
expect(data?.files?.length).toBe(14);
expect(data?.files?.length).toBe(15);
expect(typeof data?.outputDir).toBe("string");
expect(data?.rules?.length).toBe(13);
expect(dateFolderContents(base).files.length).toBe(14);
expect(dateFolderContents(base).files.length).toBe(15);
expect(out.stderrText).toBe("");
}).pipe(Effect.provide(layer));
});
Expand Down Expand Up @@ -556,7 +556,7 @@ describe("inspect report", () => {
return Effect.gen(function* () {
yield* inspectReport(flags({ outputDir: "reports" }));
const { files } = dateFolderContents(join(cwd, "reports"));
expect(files.length).toBe(14);
expect(files.length).toBe(15);
}).pipe(Effect.provide(layer));
});

Expand All @@ -567,7 +567,7 @@ describe("inspect report", () => {
return Effect.gen(function* () {
yield* inspectReport(flags({ outputDir: base }));
// Written under the absolute base, not under the CWD.
expect(dateFolderContents(base).files.length).toBe(14);
expect(dateFolderContents(base).files.length).toBe(15);
expect(readdirSync(cwd).length).toBe(0);
}).pipe(Effect.provide(layer));
});
Expand Down
4 changes: 3 additions & 1 deletion apps/cli/src/commands/inspect/report/report.queries.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { outliersSpec } from "../db/outliers/outliers.query.ts";
import { replicationSlotsSpec } from "../db/replication-slots/replication-slots.query.ts";
import { roleStatsSpec } from "../db/role-stats/role-stats.query.ts";
import { tableStatsSpec } from "../db/table-stats/table-stats.query.ts";
import { toastSizesSpec } from "../db/toast-sizes/toast-sizes.query.ts";
import { trafficProfileSpec } from "../db/traffic-profile/traffic-profile.query.ts";
import { vacuumStatsSpec } from "../db/vacuum-stats/vacuum-stats.query.ts";

Expand Down Expand Up @@ -50,7 +51,7 @@ export interface ReportQuery {
}

/**
* The 14 report queries. Reuses the 13 `inspect db` specs' `.sql` verbatim
* The 15 report queries. Reuses the 14 `inspect db` specs' `.sql` verbatim
* (byte-identical COPY input → byte-identical CSVs) plus the standalone
* `unused_indexes` query.
*/
Expand All @@ -68,6 +69,7 @@ export const REPORT_QUERIES: ReadonlyArray<ReportQuery> = [
{ fileName: "table_stats", sql: tableStatsSpec.sql },
{ fileName: "traffic_profile", sql: trafficProfileSpec.sql },
{ fileName: "unused_indexes", sql: UNUSED_INDEXES_REPORT_SQL },
{ fileName: "toast_sizes", sql: toastSizesSpec.sql },
{ fileName: "vacuum_stats", sql: vacuumStatsSpec.sql },
];

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ describe("reportIgnoreSchemas", () => {
});

describe("REPORT_QUERIES", () => {
it("has the 14 underscore CSV basenames Go embeds", () => {
it("has the 15 underscore CSV basenames Go embeds", () => {
expect(REPORT_QUERIES.map((q) => q.fileName)).toEqual([
"bloat",
"blocking",
Expand All @@ -53,6 +53,7 @@ describe("REPORT_QUERIES", () => {
"table_stats",
"traffic_profile",
"unused_indexes",
"toast_sizes",
"vacuum_stats",
]);
});
Expand Down
Loading