Skip to content

feat(tables): add automatic row expiration with TTL - #7071

Open
j15z wants to merge 10 commits into
refactor/table-column-type-extension-pointsfrom
feat/table-row-ttl
Open

feat(tables): add automatic row expiration with TTL#7071
j15z wants to merge 10 commits into
refactor/table-column-type-extension-pointsfrom
feat/table-row-ttl

Conversation

@j15z

@j15z j15z commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary\n\nTables can now expire rows automatically with one TTL column. Expiration values are stored as Unix epoch seconds, edited with the existing date UI, and removed by a bounded background cleanup job that notifies realtime clients.\n\nTTL CSV values are handled by an added TTL case in the existing import switch. This keeps the established import structure intact instead of adding a new registry hook.\n\nThis is the TTL feature PR in a three-PR stack. Parent PR #7119 adds the generic column-limit and conversion extension points; child PR #7072 consolidates and hardens the timezone conversion used by TTL and date cells.\n\n## Type of Change\n\n- [ ] Bug fix\n- [x] New feature\n- [ ] Breaking change\n- [ ] Documentation\n- [ ] Other: ___________\n\n## Testing\n\n- Ran focused tests across TTL coercion, CSV import, table schema validation, column limits, cleanup batching, cron dispatch, UI column selection, and cell rendering.\n- Verified expired rows are deleted in bounded batches while future, missing, and non-numeric TTL values remain.\n- Verified the cleanup route authenticates, deduplicates each five-minute window, and dispatches through the configured async-job backend.\n- Type-check, lint, API validation, and client-boundary validation pass.\n\n## Checklist\n\n- [x] Code follows project style guidelines\n- [x] Self-reviewed my changes\n- [x] Tests added/updated and passing\n- [x] No new warnings introduced\n- [ ] I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)\n\n## Screenshots/Videos\n\nNo recording is included. Component tests cover the TTL selector, disabled one-column state, and cell rendering.\n\n## Post-Deploy Monitoring & Validation\n\n- Owner and window: Tables on-call for the first 24 hours after deploy.\n- Logs: search for Table row TTL cleanup dispatched, Table row TTL cleanup completed, and Failed to dispatch table row TTL cleanup.\n- Metrics: watch cron/API success rate and async-job failures for cleanup-table-row-ttl.\n- Healthy signals: the cron route returns 2xx every five minutes; jobs complete without retries; limitReached is normally false; seeded expired rows disappear and future rows remain.\n- Failure signals: repeated 500s, repeated job failures, sustained limitReached true, or any row deleted before its stored epoch second.\n- Mitigation: disable the TTL cleanup cron first to stop further deletion, then roll back this PR and inspect stored TTL values and cleanup logs.\n\n---\n\nCompound Engineering\n

@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 26, 2026 10:51pm

Request Review

@j15z j15z changed the title feat/table row ttl feat(tables): add automatic row expiration with TTL Aug 25, 2026
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a TTL table-column type backed by Unix epoch seconds, including validation, timezone-aware editing, import and conversion support, and documentation.

  • Adds scheduled, serialized background cleanup of expired table rows.
  • Enforces one TTL column per table across schema validation and table mutation flows.
  • Adds TTL rendering, editing, picker availability, tests, deployment schedules, and generated tool metadata.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking cleanup-fairness issue that can leave later tables behind under sustained high-volume expiration workloads.

The TTL representation and mutation paths are internally consistent, but the cleanup worker's fixed ordering and shared batch budget allow an early table to repeatedly monopolize scheduled cleanup capacity.

Files Needing Attention: apps/sim/background/cleanup-table-row-ttl.ts

Important Files Changed

Filename Overview
apps/sim/background/cleanup-table-row-ttl.ts Implements locked, batched TTL deletion; the global budget and fixed table ordering can create unbounded lag under sustained backlog.
apps/sim/lib/table/column-types/ttl.ts Defines TTL epoch-second coercion, validation, conversion, and timezone-aware formatting.
apps/sim/lib/table/columns/service.ts Applies source-aware value conversion and validates the resulting schema before retyping columns.
apps/sim/lib/table/schema-invariants.ts Extends shared schema invariants with registry-declared per-table column limits.
apps/sim/app/api/cron/cleanup-table-row-ttl/route.ts Adds an authenticated cron endpoint that dispatches one deduplicated, serialized cleanup job.
apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/table-grid/cells/inline-editors.tsx Normalizes date-editor saves through timezone-aware column coercion so TTL cells persist epoch seconds.
helm/sim/values.yaml Enables the TTL cleanup endpoint on a five-minute non-overlapping cron schedule.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Cron[Five-minute cron] --> Route[Authenticated cleanup route]
  Route --> Queue[Serialized cleanup job]
  Queue --> List[List tables with expired TTL rows]
  List --> Lock[Acquire table advisory lock]
  Lock --> Batch[Delete up to 500 expired rows]
  Batch -->|More rows and budget remains| Lock
  Batch -->|Table complete| Signal[Signal table rows changed]
  Signal -->|More tables and budget remains| Lock
Loading

Reviews (1): Last reviewed commit: "feat(tables): add row TTL expiration" | Re-trigger Greptile

Comment thread apps/sim/background/cleanup-table-row-ttl.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread apps/sim/lib/table/column-types/ttl.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant