feat(observability): extend audit log, PostHog, and storage metering coverage by waleedlatif1 · Pull Request #5269 · simstudioai/sim

waleedlatif1 · 2026-06-29T20:44:51Z

Summary

Extends the three observability layers — audit log, PostHog product analytics, and usage metering — to cover resources and actions that were previously uninstrumented. Found via a codebase-wide coverage audit; every gap below was a real blind spot.

Audit log

Exfiltration trail (none existed): file downloads (workspace, public-share, v1 API), and table / workflow / workspace exports.
Credential access audited at the token-issuance boundary (CREDENTIAL_ACCESSED), success-only.
Full revenue/financial trail: invoice paid/failed, overage billed, charge disputes, credit fulfillment, subscription create/cancel/transfer, enterprise provisioning, plan/seat changes.
Session/account lifecycle via Better Auth hooks: login, blocked sign-in, logout, session revoke, account delete.
The entire v1/admin programmatic surface (member/role/export ops) and copilot tool handlers (skills, custom tools, tables) — both previously bypassed audit entirely.
Wired several defined-but-dead AuditAction constants (lock/unlock, table update/delete, etc.).

PostHog

Revenue events (payments, overage, credits, disputes, plan conversion, enterprise), org/seat lifecycle, KB search, copilot/skill/tool events.
Client-side workspace + organization group association so events roll up by account.

Metering

Storage now counted for KB documents and copilot files (was workspace-files only). Connector-synced KB docs remain correctly unmetered on both ingest and delete.

Hardening

Audit package nulls the actor FK for system actors (admin-api) with a readable label instead of silently FK-failing; adds an awaitable recordAuditNow for pre-delete hooks.
Every billing-webhook actor lookup is guarded so instrumentation can never fail payment processing.
All instrumentation is fire-and-forget and never blocks or breaks the primary operation. Ephemeral execution-file storage is intentionally not metered (no decrement path → would leak).

Type of Change

New feature (observability coverage)

Testing

Full typecheck, biome, and check:api-validation pass.
2,617 touched-area tests + 22 audit-package tests pass.
Validated by a multi-agent adversarial review across every changed domain (FK safety, fire-and-forget, no double-emit, storage symmetry, behavior-preserving refactors).

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-06-29T20:44:58Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 9, 2026 3:44am

cursor · 2026-06-29T20:45:01Z

PR Summary

Medium Risk
Touches auth/credential, billing webhooks, and storage accounting alongside broad API instrumentation; instrumentation is designed not to block primary flows, but billing and metering paths warrant careful review.

Overview
Adds audit log and matching PostHog events across previously uninstrumented paths: credential token issuance, file downloads/exports (including public-share with anonymous actors), table and admin workflow/workspace exports, org membership and seat reconciliation, billing (invoices, overage, disputes, credits, subscriptions), copilot chat and tool-driven skills/custom tools/tables, and realtime debounced workflow variable updates (tracking the latest writer).

Billing webhooks gain idempotent instrumentation where needed, best-effort actor resolution, and threshold-billing refactors so overage settlement can be audited after commit without blocking blocks. Org seat changes audit/analytics move into reconcileOrganizationSeats (with actorId plumbed from invite accept and member removal).

Knowledge base documents now enforce storage quota on ingest and adjust usage counters on upload; hard deletes decrement storage inside the same transaction via new decrementStorageUsageInTx. Client PostHog workspace/org groups wait for workspace metadata so org association stays consistent.

^{Reviewed by Cursor Bugbot for commit 914e2ee. Configure here.}

greptile-apps · 2026-06-29T20:49:50Z

Greptile Summary

This PR extends three observability layers — audit log, PostHog analytics, and storage metering — to cover resources and operations that were previously uninstrumented. It fills gaps identified in a codebase-wide coverage audit across billing webhooks, file downloads, copilot tool handlers, knowledge-base documents, and admin API surfaces.

Audit log gains new actions for file downloads, table/workflow/workspace exports, credential access, the full revenue lifecycle (invoices, disputes, subscriptions, enterprise provisioning), and org/session lifecycle events; the actor FK is now safely nulled for system and anonymous actors.
PostHog gains typed revenue, org, KB search, and copilot events, plus client-side workspace + organization group association in workspace-scope-sync.tsx.
Storage metering is extended to KB documents and copilot files, using an atomic decrementStorageUsageInTx that makes the counter update transactional with the document delete.

Confidence Score: 5/5

Safe to merge — all instrumentation is fire-and-forget, actor FK handling is correct, and all billing-critical operations remain protected from instrumentation failures.

Every new billing webhook instrumentation block uses functions that are definitively fire-and-forget (recordAudit swallows via .catch(), captureServerEvent wraps in try/catch). Actor resolution helpers have their own internal error guards. The one inconsistency — handleSubscriptionDeleted cancellation paths lack an explicit try/catch guard that handleSubscriptionCreated has — is safe in practice because none of the helpers can throw today.

apps/sim/lib/billing/webhooks/subscription.ts — the two cancellation audit blocks in handleSubscriptionDeleted could benefit from the same try/catch guard applied to handleSubscriptionCreated.

Important Files Changed

Filename	Overview
packages/audit/src/log.ts	Actor FK is now correctly nulled for system actors and on lookup failure; `recordAudit` remains fire-and-forget. Both branches (not-found and catch) mirror each other.
packages/audit/src/types.ts	New AuditAction constants (billing, subscription, credential, file download, export, and seat/table/workflow events) and SUBSCRIPTION resource type added without removing or renaming existing values.
apps/sim/lib/billing/webhooks/subscription.ts	handleSubscriptionCreated instrumentation is properly wrapped in try-catch. The two cancellation audit blocks in handleSubscriptionDeleted are not wrapped, but are safe today since all called helpers (resolveSubscriptionActorId, recordAudit, captureServerEvent) are internally guarded or fire-and-forget.
apps/sim/lib/billing/webhooks/invoices.ts	payment_failed instrumentation is wrapped in try-catch so it cannot block user-blocking logic. payment_succeeded and credit-purchase instrumentation uses fire-and-forget helpers and is correctly positioned after critical operations.
apps/sim/lib/billing/webhooks/disputes.ts	Dispute handling refactored to use idempotency keys and audit both opened/closed outcomes. Lost disputes are now correctly audited (remain blocked) but no longer silently dropped.
apps/sim/lib/billing/webhooks/enterprise.ts	Enterprise provisioning audit and PostHog events are deduplicated via executeWithIdempotency using the Stripe event ID, preventing double-emit on redelivery.
apps/sim/lib/knowledge/documents/service.ts	Storage quota check (inside TX) and increment (outside TX) introduce a TOCTOU race for concurrent uploads — already flagged as P2 in a previous outside-diff comment. hardDeleteDocuments atomically decrements storage inside the transaction using the new decrementStorageUsageInTx.
apps/sim/lib/billing/storage/tracking.ts	New decrementStorageUsageInTx correctly runs the counter update inside an existing transaction, making the decrement atomic with the document delete and preventing counter drift on rollback.
apps/sim/app/api/auth/oauth/token/route.ts	CREDENTIAL_ACCESSED audit and PostHog event now fire after refreshTokenIfNeeded succeeds in both GET and POST handlers, so a failed refresh never records a spurious credential access.
apps/sim/app/workspace/[workspaceId]/providers/workspace-scope-sync.tsx	Organization group is set alongside workspace group; when no org exists, resetGroups() clears any stale org group and the workspace group is re-applied on the next line. Waits for activeWorkspace metadata to avoid transient null mismatches.
apps/sim/lib/billing/organizations/seats.ts	reconcileOrganizationSeats now accepts an optional actorId and emits both seat-provisioned and seat-deprovisioned audit/analytics events with correct attribution; syncSubscriptionUsageLimits failure is caught so the audit still fires for an already-committed seat change.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Stripe Webhook Event] --> B{Event Type}
    B --> C[invoice.payment_succeeded]
    B --> D[invoice.payment_failed]
    B --> E[customer.subscription.created]
    B --> F[customer.subscription.deleted]
    B --> G[charge.dispute.created/closed]
    B --> H[manual_enterprise_subscription]

    C --> C1[executeWithIdempotency]
    C1 --> C2[Business logic renewal / usage reset]
    C2 --> C3[resolveBillingActorId internal catch]
    C3 --> C4[recordAudit fire-and-forget captureServerEvent fire-and-forget]

    D --> D1[executeWithIdempotency]
    D1 --> D2[Business logic]
    D2 --> D3[try/catch guard]
    D3 --> D4[resolveBillingActorId recordAudit captureServerEvent]
    D2 --> D5[User blocking IF attemptCount ge 1]

    E --> E1[executeWithIdempotency via Better Auth hook]
    E1 --> E2[free to paid usage reset]
    E2 --> E3[try/catch guard]
    E3 --> E4[resolveSubscriptionActorId recordAudit captureServerEvent]

    F --> F1[executeWithIdempotency]
    F1 --> F2[Overage billing Org dormancy Pro restore]
    F2 --> F3[resolveSubscriptionActorId internal catch]
    F3 --> F4[recordAudit fire-and-forget captureServerEvent fire-and-forget]

    G --> G1[executeWithIdempotency]
    G1 --> G2[Block / unblock user or org]
    G2 --> G3[getOrganizationOwnerId internal catch]
    G3 --> G4[recordDisputeInstrumentation fire-and-forget]

    H --> H1[executeWithIdempotency for audit only]
    H1 --> H2[resolveActor via stripeCustomerId]
    H2 --> H3[recordAudit captureServerEvent]

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Stripe Webhook Event] --> B{Event Type}
    B --> C[invoice.payment_succeeded]
    B --> D[invoice.payment_failed]
    B --> E[customer.subscription.created]
    B --> F[customer.subscription.deleted]
    B --> G[charge.dispute.created/closed]
    B --> H[manual_enterprise_subscription]

    C --> C1[executeWithIdempotency]
    C1 --> C2[Business logic renewal / usage reset]
    C2 --> C3[resolveBillingActorId internal catch]
    C3 --> C4[recordAudit fire-and-forget captureServerEvent fire-and-forget]

    D --> D1[executeWithIdempotency]
    D1 --> D2[Business logic]
    D2 --> D3[try/catch guard]
    D3 --> D4[resolveBillingActorId recordAudit captureServerEvent]
    D2 --> D5[User blocking IF attemptCount ge 1]

    E --> E1[executeWithIdempotency via Better Auth hook]
    E1 --> E2[free to paid usage reset]
    E2 --> E3[try/catch guard]
    E3 --> E4[resolveSubscriptionActorId recordAudit captureServerEvent]

    F --> F1[executeWithIdempotency]
    F1 --> F2[Overage billing Org dormancy Pro restore]
    F2 --> F3[resolveSubscriptionActorId internal catch]
    F3 --> F4[recordAudit fire-and-forget captureServerEvent fire-and-forget]

    G --> G1[executeWithIdempotency]
    G1 --> G2[Block / unblock user or org]
    G2 --> G3[getOrganizationOwnerId internal catch]
    G3 --> G4[recordDisputeInstrumentation fire-and-forget]

    H --> H1[executeWithIdempotency for audit only]
    H1 --> H2[resolveActor via stripeCustomerId]
    H2 --> H3[recordAudit captureServerEvent]

_{Reviews (32): Last reviewed commit: "fix(billing): guard the new subscription..." | Re-trigger Greptile}

waleedlatif1 · 2026-06-29T20:56:07Z

@greptile

waleedlatif1 · 2026-06-29T20:56:10Z

@cursor review

waleedlatif1 · 2026-06-29T21:04:52Z

@greptile

waleedlatif1 · 2026-06-29T21:04:55Z

@cursor review

waleedlatif1 · 2026-06-29T21:16:41Z

@greptile

waleedlatif1 · 2026-06-29T21:16:44Z

@cursor review

waleedlatif1 · 2026-06-29T21:30:06Z

@greptile

waleedlatif1 · 2026-06-29T21:30:09Z

@cursor review

waleedlatif1 · 2026-06-29T21:42:39Z

@greptile

waleedlatif1 · 2026-06-29T21:42:42Z

@cursor review

waleedlatif1 · 2026-06-29T21:46:37Z

@greptile

Address final audit flags: - Table import: move the 'columns added' audit OUT of the import transaction into a post-commit auditTableColumnsAdded() helper called by the three tx-owning callers, so a mid-import row-batch rollback no longer logs a false 'added N columns' (matches the PR's success-only discipline). - Omit empty-string analytics dimensions on workflow_lock_toggled (workspace_id) and organization_created (name), consistent with file_downloaded/copilot_chat_sent. - Restore an unrelated capacity-check comment removed incidentally. - Move copilot_chat_sent emit above the traceparent comment so the comment sits with the code it documents.

Address Cursor review: addImportColumns (async createColumns import path) now threads the importing userId into auditTableColumnsAdded instead of falling back to table.createdBy, so column additions are attributed to the actual member who ran the import rather than the table creator.

Per design decision: copilot files are working/conversational artifacts, so gating their materialization on storage quota would fail an agent operation mid-flow when a user is over limit, and metering them would inflate usage enough to block KB/workspace uploads indirectly. Remove copilot quota gates + copilot ingest metering entirely (revert copilot-file-manager, metadata, and the upload route's copilot branch to baseline) and drop the now-unused releaseDeletedFileStorage. KB document metering + quota enforcement (the deliberate, persistent storage path) is unchanged.

Address Cursor review: gate the group-sync effect on workspace metadata being loaded (activeWorkspace present) so the workspace and organization groups always update atomically. Acting during the load window paired the new workspace group with the previous workspace's org group; until metadata loads, events stay consistently attributed to the previous workspace.

Address Cursor review: async exports only emitted TABLE_EXPORTED when the background job reached 'ready', so an authorized export whose job later failed or was abandoned left no audit trail — inconsistent with the sync export route, which audits before streaming. Move the audit + analytics to the async route's authorization point (after the job is claimed/dispatched) and remove it from the runner. Drop the now-unused userId from TableExportPayload.

…king Address Greptile P1: the payment_failed audit hoisted an unguarded isSubscriptionOrgScoped DB read (for entity_type) directly before the attempt-count user-blocking block. A transient failure of that read would throw out of the handler and skip blocking. Wrap the whole audit/analytics block in try/catch (best-effort), and let the blocking compute its own isSubscriptionOrgScoped as it did originally — instrumentation can no longer abort payment processing.

…n in Stripe webhook idempotency recordAudit/captureServerEvent calls added for charge disputes, enterprise subscription provisioning, and free->paid subscription creation ran unconditionally with no idempotency guard, unlike their sibling handlers in the same files. Stripe redelivers webhooks at-least-once, so a retry would double-record the audit row and PostHog event even though the underlying DB writes were already idempotent.

…, guard concurrent table delete The org-scoped self-service audit-log endpoint matches org-level rows via metadata.organizationId (or resourceType=organization). Six recordAudit call sites (subscription create/cancel, admin credit issuance, threshold overage billing, charge disputes, credit purchase, invoice payment succeeded/failed) tagged org-scoped events with a differently-named key (referenceId/entityId/targetOrgId), making them invisible to org admins querying their own audit trail despite being stored in the DB. Add the missing organizationId key everywhere the pattern was missed. Also guard deleteTable's archive UPDATE with isNull(archivedAt) so a concurrent duplicate delete request is a no-op instead of re-archiving and re-firing a duplicate TABLE_DELETED audit row. Adds test coverage for the ORG_MEMBER_ADDED audit/analytics emission in acceptInvitation, which previously had none.

waleedlatif1 · 2026-07-09T03:18:47Z

@greptileai review

waleedlatif1 · 2026-07-09T03:18:49Z

@cursor review

…ure email test handleInvoicePaymentFailed's new payment_failed audit instrumentation resolves the billing actor via an extra db.select before sendPaymentFailureEmails runs. The test's fixed select-response queue didn't account for it, so the org-admin lookup consumed the wrong queued row and the assertion saw zero email sends. Production behavior is unaffected — each query is independent; this was a mock-queue ordering issue only.

…ss the seat audit Cursor Bugbot flagged: reconcileOrganizationSeats committed the seat change and Stripe outbox enqueue in a transaction, then called syncSubscriptionUsageLimits outside it before recording the audit/ PostHog events. A thrown sync left a genuinely-changed seat count with no ORG_SEAT_PROVISIONED/DEPROVISIONED trail. Wrap the sync in its own try/catch (log + continue) so the events always fire for a committed change, matching the fire-and-forget instrumentation pattern used elsewhere in this PR.

waleedlatif1 · 2026-07-09T03:33:02Z

@greptileai review

waleedlatif1 · 2026-07-09T03:33:04Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e4cc7da. Configure here.}

Greptile P1: the org-scope check I added for the audit-metadata fix (isSubscriptionOrgScoped) was a raw DB call with no error guard, unlike resolveSubscriptionActorId next to it. Since it ran inside the idempotency lambda with an unconditional rethrow, a transient DB error would abort and retry the whole webhook after the free -> paid usage reset had already committed. Wrap the actor/org-scope resolution + audit + analytics block in try/catch, matching the same guarded instrumentation pattern already used in handleInvoicePaymentFailed.

waleedlatif1 · 2026-07-09T03:44:33Z

@greptileai review

waleedlatif1 · 2026-07-09T03:44:36Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 914e2ee. Configure here.}