feat(api): make the platform operable headless over v2 - #6912
Open
waleedlatif1 wants to merge 60 commits into
Open
feat(api): make the platform operable headless over v2#6912waleedlatif1 wants to merge 60 commits into
waleedlatif1 wants to merge 60 commits into
Conversation
Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the
async-run loop for headless callers. A run's output carries UserFile URLs
pointing at /api/files/serve/..., which rejects x-api-key outright, so an
async run that produces a file previously had no byte path out for an API
key at all.
The file is addressed by the id the run reported and resolved against the
run's own recorded execution data, from which the storage key is read. The
request never supplies a storage key, so the endpoint cannot be aimed at
bytes the run did not produce. Resolution deliberately reads the
materialized-but-undisplayed recording, because the display projection
strips exactly the `key`/`context` fields a byte read needs.
Also hardens normalizeStartFile to derive a file's storage key only from a
validated internal serve URL, discarding any caller-supplied `key`/`context`.
A workspace API key has no human subject, so the executor resolves its actor
to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution);
verifyFileAccess then authorizes a workspace-context key as that owner, whose
reach is not bounded by the key's workspace. Accepting an attacker-authorable
key made that substitution exploitable as a confused deputy. Normalization is
all-or-nothing, so a forged file now drops the whole files input.
…, and v2 authoring endpoints
Extract replaceWorkflowNormalizedState as the single persistence primitive for a
workflow graph replace and route both the internal editor save and the Copilot
edit tool through it, so neither can skip state preparation, the row lock, the
lastSynced stamp, or custom-tool extraction by choosing a different entry point.
Derive the audit source from the acting principal instead of hardcoding
'copilot', then widen workflows.variables.apply_operations and
workflows.bulk.move to every principal kind.
Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate,
/restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral
application use cases; move the edit engine to lib/workflows/editing.
…he v2 authoring surface Pin the two-doors fix (preparation runs, the row is locked, custom-tool extraction is post-commit and best-effort) and the false-audit fix (a session principal writes source: 'session', a delegated one writes its service). Both were verified to fail with the fix reverted. Add the application matrix for replaceWorkflowState, applyWorkflowOperations, readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection before canonical load, asserted-scope concealment, lock, validation, atomic conflict, plan gate, and audit-then-notify ordering — plus route tests for every new endpoint.
Characterize saveWorkflowNormalizedState's statuses, messages, and notification after the persistence extraction, and cover the new scope filter on GET /api/v2/workflows including a cursor replayed under a different scope.
Adds six read endpoints under /api/v2 that publish Sim's code-defined
catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools,
GET /tools/{toolId}, GET /connector-types, and GET /enrichments.
These read like static reference data and are not. What a caller may
place is decided per workspace by its permission-group integration
allowlist, per organization by which unreleased blocks have been
revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace
again by the workflows it has deployed as blocks. So all six are plain
defineWorkspaceOperation reads at minimumRole 'read' with
workspaceApiKey 'allow' — the exact policy of credentials.providers.list
— and every response keeps Cache-Control: private, no-store, because an
unrevealed preview block's existence must not leak across organizations
through a shared cache.
Trigger blocks ride as ?capability=trigger rather than a second
endpoint, and workspace custom blocks ride inside /blocks discriminated
by `source`, so "what may I place?" stays a one-call question.
The block projection is extracted out of the Copilot get_blocks_metadata
tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs.
That cuts the tool's own @/tools/registry edge as a side effect: its
module graph drops from 6,756 to 1,318, and the new routes land at
1,673-1,734, next to the shipped /v2/credentials/providers baseline of
1,668.
Supporting changes:
- scripts/sync-tool-metadata.ts derives hostedApiKey ('always' |
'conditional' | 'none') from each tool's `hosting`. The config itself
stays excluded because it holds closures, but "does Sim host the key"
is a first-order authoring question, so the answer is emitted.
- getCopilotToolDescription takes hostedApiKey as an option instead of
reading `hosting` off the tool, so both an executable ToolConfig and
the generated metadata can answer it through one shared derivation.
- principalUserId / allowedIntegrationTypes move out of
lib/credentials/application/provider-catalog.ts into
lib/integrations/principal-scope.server.ts. Two copies of the
workspace integration gate would diverge first on the workspace-key
path, which has no user for permission groups to key on.
- scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx
under app/workspace only, so a route importing the executable registry
passed green. It now walks a list of entry sources, seeded with the
four catalog route subtrees and the shared projection barrel. Routes
are covered per subtree rather than wholesale because 122 of ~1,130
route files legitimately execute tools.
Registry sweeps parse every block, tool, connector type, and enrichment
through its published response schema and compare against the wire
round-trip. They caught a real drift while being written: an operation's
inputs were typed as a union of the tool-param and block-input shapes,
and the union resolved to whichever member matched first, silently
dropping a block input's `schema`.
Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a
caller that lost track of a transfer could abort it but could not ask
whether the session was still alive, already finalized, or failed — the
resume story was missing.
Runs on a new files.upload.read operation at minimumRole 'read' rather than
reusing uploadCancel, which is a 'write': asking about a session must not
require permission to destroy it. The GET is a control leg like every other,
so it carries the signed upload token and re-authorizes the caller's present
workspace permission through reauthorizeWorkspaceUploadPurpose instead of
resolving the session on its id alone.
Merging the catalog and workflow-authoring branches surfaced four issues that neither produced in isolation. - Route and OpenAPI counters were bumped to the same value on both branches, so git merged them as one change while the merged tree holds the sum. Corrects the route ratchet to 1142 and the workflows document to 29 operations (152 total), then regenerates the OpenAPI documents and the CLI surface from the reconciled contracts. - The seven new workflow operations were published in the spec but absent from the workflow API reference groups, which `check:openapi` rejects. - `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through `apply-workflow-operations`, dragging the edit engine — and its diff and comparison dependencies, which reach a client OAuth hook — into every route that uses the shared workflow error policies. The class moves to its own leaf module, mirroring `WorkflowImportError`, and each importer now takes it from there. - The operations route test shadowed that class inside its module mock, so `instanceof` matched a fake and the assertion pinned a message the production class never emits. It now uses the real class and asserts the real message.
Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive
from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal
and workspace API keys at the unchanged 'write' role.
The widening is an authorization change, so the justification lives in the
operation's TSDoc: extraction grants no capability an API key lacks, since
every file it writes could be created one at a time through files.create and
files.upload.create, both already 'allow' at the same role. It only collapses
many calls into one. The previous ['session'] restriction read as an artifact
of the UI having been the only caller. Delegated services stay out — no
copilot or executor caller exists and admitting one is a separate decision.
The response is counts plus the destination folderPath, never the extracted
files: a large archive would otherwise materialize thousands of objects into
one body. Callers page GET /api/v2/files?folderPath=... instead. The use case
returns the internal display path and the adapter projects it to a v2 path,
keeping the use case surface-neutral.
Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind
checkInternalAuth on /api/files/parse, a route that also mixes in external-URL
fetching, execution-file upload, and multi-file aggregation, so it could not be
reused. The parse call is lifted into a thin application use case instead.
Runs on the existing files.read_content operation unchanged — it is already
workspaceApiKey 'allow' at the read role, and turning bytes it already
authorizes into text grants no further reach.
`degraded` is a required, non-optional boolean on the response. The legacy doc
and ppt parsers deliberately return best-effort or placeholder content rather
than throwing, so an omittable flag would let a client that never checks it
treat guessed text as extracted text. It is reported honestly rather than
converted into an error, because the parsers' behaviour is deliberate and
characterization-tested.
The read is bounded on its input at 25 MiB before extraction rather than on its
output after, given the parsers' documented DoS history; a caller may lower the
ceiling but never raise it.
DELETE /api/v2/files/folders archives recursively, so a recursive delete was unrecoverable over the API: the archived files stayed visible through GET /api/v2/files?scope=archived, but nothing could rebuild the folder structure. Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the v2 folder family, and a `scope` selector on the folder list so a caller can find the archived path to hand it. `scope` extends the files folder-list query rather than the shared v2ListFoldersQuerySchema: only workspace files have an archived folder set, so adding it to the shared schema would give tables, workflows, and knowledge a parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged, so no cursor binding changes — list-pagination.test.ts passes unchanged. Restore resolves the archived folder from its path by scanning the archived set rather than walking the live tree, which by definition does not contain the folder being restored. The folder-restored analytics hook now reports the folder actually restored rather than the requested selector, which carries no id on a path-addressed surface.
Adds GET /api/v2/files/bulk-download, an adapter over the existing downloadWorkspaceFileItems use case and its internal binary route. Path collision: a static segment beside [fileId] permanently shadows a file whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+. Rather than invent a new shape, this follows the existing bulk-delete sibling: the hyphenated form cannot be produced by either minted id shape (UUID v4 or wf_<shortId>), so the shadowed id is unreachable in practice. Documented on the contract so the reasoning is not lost. Folders are addressed by path, matching the rest of the v2 file surface. The paths resolve against the folder set the selection already loads, so it costs no extra query, and a path matching no folder is rejected rather than silently dropped — a misspelled folder must not yield a zip of whatever else was selected. The empty-selection and folder-count guards now account for folderPaths, which a path-only selection would otherwise have tripped. Selections are comma-separated only: v2 rejects a query parameter sent more than once, so a repeated-parameter form would never reach the schema. Pinned by a test so the contract cannot advertise a form the boundary rejects.
…ns read
GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced,
each with the downloadPath that fetches its bytes, and can inline them as
base64 on request.
Gated by includeOutput, matching `output`'s nullability: a caller that did not
ask for output does not receive a file list it did not request. The async
execute request's rejection of includeFileBase64 is deliberately left alone —
at submit time the run has not happened, so there is nothing to inline; reading
a finished run is the first moment the question means anything.
Inlining is capped per file at the executor's 16 MiB inline ceiling, which a
caller may lower but never raise. A file above it answers 413 naming that
file's downloadPath, so the caller is told exactly how to get the bytes rather
than being left stuck.
The descriptor deliberately omits the storage key — files are addressed by id
and the key is re-derived from the run's recording — and omits an expiry, which
the recording does not carry and which would be fabricated if published.
The route becomes headSafe: false, since inlining reads object storage. The
builder enforces that this requires the use case to expose authorize(), so HEAD
still answers from a real authorization rather than from authentication alone.
DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes
are never removed — so there was no way to actually destroy a file over the API.
Adds the repository primitive, application use case, operation, and
DELETE /api/v2/files/{fileId}/permanent.
A distinct path rather than a flag on the ordinary delete: a query parameter
that turns a recoverable archive into an irreversible destruction is set by
accident, and the two acts carry different minimum roles, which one route
declaration cannot express. The file must already be archived; a live file
answers 409 naming the archive step, so no single request can turn a live file
into lost bytes.
minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key
ceiling is 'write' — the desired policy anyway: unattended credentials should
not destroy bytes.
Row first, then object. The two legs commit independently, so one can survive a
crash between them: deleting the row first leaves at most an orphaned object for
the storage sweep, while the reverse would leave a live row pointing at bytes
that no longer exist — a file that lists and opens but can never be read. A
failed object delete is therefore reported as objectDeleted: false rather than
thrown, because the request has genuinely succeeded once the row is gone. Both
directions are pinned by failure-injection tests, verified to fail when the
order is reversed.
Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of
FILE_DELETED, which records the recoverable archive step.
Adds the aggregate and rich-read halves of the public logs surface, and fixes three defects the existing reads carry. Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run counts, success rate, error count, mean latency, and the window bounds, per workflow and for the workspace. The first-party route was a raw handler with inline SQL and inline aggregation, so it is split into a repository (`lib/logs/stats-queries.ts`), a pure aggregator (`lib/logs/stats.ts`), and an application use case. That route keeps its legacy authorization — it answers a caller without workspace access with a zeroed 200, where v2 conceals the workspace as a 404 — and consumes only the two surface-neutral halves. `segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by zero and `1e9` allocated two billion-element arrays: both caller-reachable 500s. Bounded on both contracts. `workflows` is capped, with the workspace totals still computed from every workflow and the cut reported as `workflowsTruncated`. Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are distinct answers and both reachable) and `workflowInput`, restoring a v1→v2 regression. The list gains `workflowName` and `status` filters, and `includeJobRuns`, which unions Chat and Sim-agent job runs into the sequence behind a new `kind` discriminator — without it a job run is indistinguishable from a run whose workflow was deleted. A filter no job row can answer drops the branch outright rather than meaning two things across the union. `POST /api/v2/logs/query` carries the additional sort columns. `GET /logs` is untouched: its single `order` param rests on there being exactly one sortable column, and both escapes from that are ruled out, so the rich read gets its own endpoint — the split the table surface already ships. It uses the shared keyset scheme with the two nullable sort columns read through a sentinel, since a keyset cannot compare against null. `folderPaths` now covers a folder's whole subtree on the public path, as it already did everywhere else; it previously omitted every nested run with no error. The path strings did not change, so a folder-scope version is stamped into the cursor and in-flight tokens restart rather than silently skipping rows. Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the clause above it — and so searched workflow names instead of folders. `buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and pinned: under `NULLS LAST` the null block is only reachable through it, so removing it as a duplicate-row fix makes those runs unpageable. Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total operations 152 -> 154.
… archive
Closes the headless gaps on the v2 tables surface.
- Per-cell run state is now readable through an opt-in `includeRunState` on
`GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection
is byte-identical; a page whose sidecar outgrows its byte budget is a 413
rather than a silent truncation.
- Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}`
publishes the column's full four-state domain so polling a finished run is
not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight.
- `POST /rows/batch-update` takes one distinct patch per row. Its transaction
moved out of the Copilot-only module into a surface-neutral use case both
surfaces now call.
- `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and
timing behind one enrichment cell.
- `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use
cases, which now accept folders by canonical path and resolve them inside
the application layer.
- `DELETE` is recoverable: `scope=archived` on the table list plus
`POST /tables/{tableId}/restore`.
Both branches bumped the route ratchet and OpenAPI operation counts from the same 1142/152 base, so their values conflict rather than compose. Recomputes every counter from the merged tree — 1150 routes, 161 operations — and regenerates the OpenAPI documents and CLI surface from the reconciled contracts.
Closes the knowledge cluster's remaining public-surface gaps.
Chunks: list/read/create/update/delete/bulk under
`/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains
an `id` tiebreaker on every sort so the list pages on a keyset rather than an
offset — `tokenCount` and `enabled` are both non-unique, so a page boundary
inside a run of equal values used to repeat or drop the tied rows. The
internal offset caller is unchanged; the two positioning schemes share one
read.
Tag definitions: create, update, delete, next-slot, usage, and the
document-scoped save and cleanup. Without them a caller could write a tag
value into a slot with no definition and then had no way to name it, so
tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema`
gains `id`, without which PATCH and DELETE are unaddressable. The
document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'`
deletes the whole knowledge base's tag vocabulary from a document path.
Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather
than a `scope` param — the two reads bind different operations and a v2 route
declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore`
is a new workspace operation carrying `delete`'s policy, since an operation's
inverse must not be harder to reach; the internal session route now delegates
its workspace branch to the shared use case and keeps only the legacy personal
one.
Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`,
so a file already in workspace storage no longer has to be re-uploaded
byte-for-byte to be indexed; the `chunkingConfig` write widens to the
first-party five-key schema with its refines and separator bounds, while the
response stays `.catchall` so a legacy JSONB row cannot 500; and
`CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that
the bare 403 on connector-managed chunk writes is wire-reachable.
Document upsert is deliberately not included.
Six operations added across the workflow, tables and logs work take a folder path or a comma-split list filter, and none had a CLI contract entry. Two consequences, neither caught by `check:audits`: - `folderPaths`, `folderPath` and `targetFolderPath` went unmarked for per-segment encoding, so a folder typed by the name the app shows it under was rejected on exactly those commands. - The derived flag names diverged from the ones the contract already uses, spelling one concept `--folder` on one log command and `--folder-paths` on the next. Puts the three shared log list filters on one constant that `listLogs`, `getLogStats` and `queryLogs` all spread, and does the same for the folder-path list and the move destination, so a future operation picks up the spelling and the encoding marker by construction rather than by being remembered.
Combines the additive halves the merge could not: both sides added an import to the workflows OpenAPI module and a slice to the cross-cutting sweep, so each conflict needed both, not one. Recomputes the route ratchet (1165), the slice sweep (95) and the per-document operation counts (184) from the merged tree, and lists the run-file download in the Workflow Runs reference group so the spec and the published groups agree.
…endpoint
PATCH /api/v2/credentials/{credentialId} rotates service-account secret
material or renames a credential in place, preserving the credential id so
existing workflow, deployment, paused-run, connector, and webhook references
keep working. Re-posting to POST /api/v2/credentials answers 409, and
delete-and-recreate mints a new id, so rotation previously had no door.
The route is adapter-only: updateWorkspaceCredentialUseCase already owned the
rotation, its audit projection, and credentials.update. It gains one additive
assertedWorkspaceId field for the v2 workspace assertion, and the per-principal
credential-type table that deleteCredentialUseCase already applied is lifted
into requireManageableCredentialType so both operations share it. Without it a
personal API key could rename an env_workspace row and toV2Credential's throw
would surface as a caller-reachable 500.
CredentialProviderOperationError now maps to 503 with Retry-After when the
provider is unreachable, instead of the 400 its OrchestrationError('validation')
base projected. A transient outage rendered as a permanent input error invites a
caller to revoke a working credential.
GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry.
It is the one route declaring the new typed gate: 'exempt' option, because the
rollout gate and the unknown-path catch-all answer byte-identical 404s and a
gated /api/v2/meta could never resolve that ambiguity. Authentication still runs
first, so the only fact disclosed is one about the caller's own credential.
Recomputes the route ratchet (1166) and per-document operation counts (186) from the merged tree and regenerates the OpenAPI documents and CLI surface.
Adds the four deployment-lifecycle operations v2 was missing, and the
workflow-as-MCP publishing surface, both as adapters over application use
cases that already existed.
Deployment lifecycle:
- PATCH /api/v2/workflows/{id}/versions/{version} relabels a version.
Deliberately not the internal route's body-shape dispatch between
"rename" and "promote to live".
- POST .../versions/{version}/activate promotes a version. Same use case
as rollback under a different transition, on its own path because the
two mean opposite things to a caller.
- POST .../versions/{version}/revert overwrites the draft. Accepts the
literal `active` alongside a version number.
- PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public
execution.
`workflows.public_api.update` widens from session-only to session plus
personal API key: it is an admin-role change the same accountable human
may make from a script. Workspace keys stay denied. Its EE refusal now
carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden.
Workflow MCP servers:
- /api/v2/workflow-mcp-servers list, create, update, delete, plus
publish and unpublish of a workflow as a tool. Named apart from
/api/v2/mcp-servers, which registers the external servers Sim calls.
- The six mcp_servers.workflow_deployments operations widen from
['delegated'] to admit sessions and personal API keys; roles and the
workspace-key denial are unchanged.
- The server list gains keyset pagination, matching its external
sibling, since nothing caps how many a workspace publishes.
- Server, tool, and workflow reads move out of the use case into
lib/mcp/queries.
Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171.
Chat deployment was a shipped module with no public API and two authorization systems: `lib/workflows/application/chat-deployments.ts` had deploy/undeploy extracted, but only Copilot used them — the REST routes reimplemented workflow authorization inline, and `PATCH /api/chat/manage/[id]` additionally owned password encryption, the auth-type field-clearing matrix, identifier uniqueness, the redeploy-gating protocol with two 409s, a raw db.update, and a manual recordAudit. New `lib/chat-deployments` domain: - `chat_deployments.list/read/update/delete`, keyed on the deployment whose workspace is derived by joining its workflow. Creation stays `workflows.chat.deploy`, which is keyed on the workflow. - The PATCH extraction, including the field-clearing matrix and the asynchronous-cutover invariant the route had hand-mirrored from `performChatDeploy`. - One `buildChatDeploymentUrl`, replacing three constructions that had already drifted onto two different host helpers. There is no chat subdomain, so nothing publishes a host. - Repository reads moved out of the use cases into `lib/chat-deployments/queries`. Internal routes are now adapters over those use cases. `GET /api/chat` is deliberately not migrated: it scopes by `chat.userId` while every other chat operation authorizes by workspace admin, and reconciling the two is a product decision. `PATCH` keeps its 400 for an identifier collision through a typed `ChatIdentifierInUseError`; v2 reports the 409 the condition actually is. v2 surface at `/api/v2/chat-deployments`: list, create, read, update, delete. Workspace-scoped, keyset-paged, and a stored password is never readable — reads carry `hasPassword` only, and the session-only reveal endpoint deliberately has no v2 counterpart. Also: an email- or SSO-gated chat with an empty allow-list is now refused in the use case rather than only at the internal boundary, since it is unenterable; and the doc comment on `processHostedKeyCost` claiming a `usageLog` write is corrected — no such write exists. Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176.
Keeps both new forbidden detail codes, recomputes the route ratchet (1174) and per-document operation counts (201) from the merged tree, and regenerates the OpenAPI documents and CLI surface. Three regressions the branch checks could not see, because each needs two branches to exist at once: - The audit mock lost sync when the files work added a new `AuditAction`, which only `packages/audit` asserts. - Six operations added across the files, chat and workflow-MCP work had no CLI contract entry: the folder restore turned the `files restore` leaf back into a group holding a lone `create` — the exact shape its rename exists to remove — three destructive DELETEs carried no confirmation, and two folder inputs went unmarked for per-segment encoding, so a folder typed by the name the app shows it under was rejected there. - A CLI fixture predates the `kind` discriminator the log list now requires, which only a full-workspace type-check reaches.
Rebase was the wrong tool here: the route ratchet and the generated OpenAPI, CLI and tool-metadata artifacts are touched by most of the 26 commits, so replaying them re-conflicted on files that are regenerated from the tree anyway. Merging resolves each once. Conflicts were additive on both sides and both sides are kept: `workspace-vfs` gains staging's deployed-state loader alongside the lint imports, `logs/types` keeps the stale-sweep vocabulary next to the persisted-status guard, and the relocated edit engine takes staging's new `resolveBlockRetryUpdate` coverage. `SkippedItemType` keeps its derived form and gains `retry_not_supported`, which staging added — verified against the pre-merge base so the extraction is known to have dropped nothing. Regenerates tool metadata, the registry-boundary baseline, the OpenAPI documents and the CLI surface, and recomputes the counters from the merged tree: 1176 routes, 201 operations.
- Run output files are filtered to keys under the run's own execution prefix. The recording they came from is not a trustworthy key source: the start block copies every caller-supplied input field verbatim into its output and `collectUserFilesById` accepts anything carrying the `UserFile` shape, so a caller could name any storage key and have the download and base64 paths — neither of which authorizes per file — serve it back. - `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal, so `constructor`, `toString` and friends returned inherited functions that every consumer then treated as a block, turning a path segment into a 500. `getToolMetadata` already guarded this way. - A folder-scoped log page no longer unions in every job run in the workspace. The guard read `filters.folderIds`, which the public surface never sets — it carries the folder filter in `folderScope` — so the page contradicted the contract's promise that job runs are dropped whenever a filter they cannot answer is set. Also: the log cursor stamps `includeJobRuns` only when it is on, so its `.default(false)` no longer puts a constant in every fingerprint and rejects cursors minted before it existed; and the `folderName` subquery is scoped to the workspace and to workflow folders instead of scanning the whole `folder` table.
- `workflows.operations.apply` no longer admits a workspace API key. The
use case authorizes against three per-user policies — the EE permission
config, block visibility, and credential reachability — and all three
take a human subject. An actorless key has none, and both substitutes
fail open: attributing to the workspace billing owner evaluates the
batch as the least-restricted account in the workspace, and passing no
user makes `getUserPermissionConfig` return `null`, which every caller
reads as unrestricted. Either way a workspace constrained by an
allowlist was edited as though it were not. Personal keys keep the
capability, so headless editing is unaffected for a credential that
names a human.
- `GET /workflows/{id}/state` reads its variables through
`parseWorkflowVariables`, and the stored variable response schema drops
the two assertions the column cannot honour. The column has carried a
JSON string and a legacy array as well as the current record, the
realtime `variable.add` op types `type` as `z.any()`, and the parser
writes `name` through verbatim — so the input bounds on the read turned
a stored workflow into a 500 on the endpoint that opens it. The write
schema keeps them, which is where they can still be honoured.
- `GET /workflows?scope=archived` projects folder paths tolerantly.
Archiving a folder cascades onto the workflows inside it but leaves
their `folderId` dangling — which is why restore has to null it — so the
strict projector threw a bare `Error` and took the whole page down with
no cursor able to step past the row.
The Start block derived a file's storage key by parsing the caller's own `url`, which `isInternalFileUrl` matches on any host and `extractStorageKey` returns verbatim — so a request body could name any tenant's bytes. The key is now accepted only when its own layout names the workspace the execution runs in, and every file is dropped when the execution carries no workspace. Also bounds `includeFileBase64` with an aggregate response ceiling and a worker pool instead of an unbounded `Promise.all`, scopes the bulk download's authorization resource to the workspace when folder paths are requested, makes the folder-restore selector mutually exclusive at the type level, and names the bound in the `maxBytes` validation message.
`getLatestBlockForViewer` took the newest version and then hid it, which inverted the contradiction it was written to close: `slack_v2` and `table_v2` are preview-gated while their v1 deliberately stays in the toolbar, so an unrevealed viewer got a `404` on a detail read for a type `GET /api/v2/blocks` was listing in the same breath. It now walks versions newest-first and answers with the first one visible to that viewer. Also: - The chat password guard ran after `performFullDeploy`, so a request that could never succeed burned a real workflow deployment version and then answered 400. Its two sibling gate guards already refuse ahead of the deploy; this one now does too. - The Copilot sub-block serializer published the registry's own `options` and `dependsOn` arrays by reference. Pre-existing, but the catalog projection this parallels copies every array it publishes precisely because they are process-global and shared by every request.
Staging landed its own v2 work — resource-management endpoints, KB connector management, and four new connectors — in the same contract and OpenAPI modules this branch rewrites, so the automatic resolution was not trustworthy anywhere it succeeded either. Both sides' route definitions were spliced by operationId rather than by text, because the conflicting hunks share an opening and a closing and are alternative *interiors*: concatenating them cut definitions in half, and picking a side dropped a surface wholesale. The same applied to the contract declarations, which were spliced by export name. Three things the automatic merge silently dropped and the checks caught: `v2ServiceAccountSecretFieldsShape` (kept its use, lost its definition), the chunk and tag route modules (whole families absent from the generated document until `check:openapi` cross-referenced the registry), and the `chunkingConfig` write widening along with the `.extend` it depends on — `.safeExtend` cannot override a refined field, which is why it was `.extend` to begin with. Recomputes every counter from the merged tree: 1182 routes, 213 operations, 107 request slices.
…ator Two findings verified as real regressions against staging, out of ten checked — the rest were pre-existing, latent, or false. `setWorkflowBlockEnabled` read the graph outside the row lock and wrote it back inside a later transaction. The editor's own save takes that same lock, so an autosave committing in the window was silently discarded: this operation writes a whole graph, not a delta. The persistence primitive now accepts a reader that runs after the lock is taken, and the toggle re-reads and re-decides there. Its lock predicate is also scoped to the workspace and to a live row again, so a workflow archived mid-flight is refused rather than written. The v2 chat-deployment contracts inlined their own password rule twice instead of using `chatDeploymentPasswordSchema`, losing the refusal of a whitespace-only password — which the internal contract rejects precisely because it strands the deployment behind a password the visitor form will not submit. Both sites use the canonical validator now.
- `toV2Folder` existed twice, and the second copy had been written without the name/path invariant — so a row the list read refuses loudly would have been served with a mismatched pair by the restore read. One definition, guard included. - The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had drifted by one member. Derived from the canonical list instead. - The tables reads documented a `413` for run state that they cannot emit — the budget became opt-in, and the row limit is the bound now — so the claim is removed rather than declared. The workflow run read has the opposite problem: it genuinely emits one, on a single file *or* the run's inlined total, and declared neither. Now declared, and the sentence covers both. - Reclassifies the operations staging added into the destructive sweep, so the triage stays exhaustive.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryLow Risk Overview Adds CLI pages and command-group links for API reference Reviewed by Cursor Bugbot for commit 139e4f6. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Regenerates the tool metadata and recomputes the route ratchet from the merged tree (1183 routes, 213 operations); both conflicts were a generated artifact and a counter, neither hand-mergeable.
Collaborator
Author
Collaborator
Author
|
@cursor review |
Contributor
Greptile SummaryThe PR substantially expands the public v2 API and CLI so workflows and related platform resources can be managed headlessly, while moving protected behavior into shared authorized application operations.
Confidence Score: 5/5The PR appears safe to merge based on the reviewed authorization, persistence, and public-contract paths. No concrete changed-code failure remained after checking the shared authorization boundary, workflow graph persistence, variable contract, and Start-block file ownership behavior.
|
| Filename | Overview |
|---|---|
| apps/sim/lib/api/server/routes/v2-json-route.ts | Centralizes v2 authentication, contract parsing, use-case invocation, response validation, and error projection. |
| apps/sim/lib/core/application/workspace-operation.ts | Defines consistent workspace operation policies for principal kinds, workspace API keys, and minimum roles. |
| apps/sim/lib/workflows/application/apply-workflow-operations.ts | Adds an authorized, validated, optionally atomic entry point for workflow graph edits. |
| apps/sim/lib/workflows/application/replace-workflow-state.ts | Adds full workflow graph replacement with mutability checks, sanitization, and normalized persistence. |
| apps/sim/lib/workflows/persistence/replace-normalized-state.ts | Persists normalized graph changes under transaction and row locking while updating workflow metadata. |
| apps/sim/executor/utils/start-block.ts | Restricts Start-block file inputs to storage keys attributable to the executing workspace. |
| apps/sim/lib/knowledge/application/chunks.ts | Introduces authorized public chunk operations while preserving knowledge-base and document scoping. |
| apps/sim/lib/table/application/rows.ts | Exposes authorized table-row operations through the shared table application layer. |
| apps/sim/lib/workflows/application/download-workflow-run-file.ts | Adds scoped run-output download authorization before resolving and serving stored files. |
| packages/sim-cli/src/generated/v2-api.ts | Expands the generated CLI client to cover the enlarged v2 API surface. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Client[CLI / headless client] --> V2[Public v2 route contracts]
V2 --> Auth[API-key authentication]
Auth --> Principal[Personal or workspace principal]
Principal --> Operation[Semantic operation policy]
Operation --> UseCase[Authorized application use case]
UseCase --> Scope[Canonical resource and workspace checks]
Scope --> Domain[Workflow / Files / Knowledge / Tables / Logs / MCP]
Domain --> State[(Postgres / object storage)]
UseCase --> Audit[Semantic audit and shared effects]
UseCase --> Presenter[v2 response presenter]
Presenter --> Client
Reviews (1): Last reviewed commit: "Merge origin/staging into feat/v2-headle..." | Re-trigger Greptile
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 139e4f6. Configure here.
| edges: sanitized.edges as WorkflowState['edges'], | ||
| variables: input.variables, | ||
| }, | ||
| }) |
There was a problem hiding this comment.
Replace state skips allowlist gates
High Severity
replaceWorkflowState is exposed to workspace API keys and persists a full draft graph after structural validation only. It never runs the EE permission config, block visibility, or credential reachability checks that applyWorkflowOperations requires a human subject for. That leaves PUT …/state as a second graph-write door that can store blocks and credentials an allowlisted member could not add through POST …/operations.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 139e4f6. Configure here.
…e delete - remove the permanent file-delete endpoint; the platform offers no such action in the UI, and its manager wrote outside a transaction with no storage accounting - extract a generated document's text from its compiled artifact rather than its generation source, matching the download path; a `.pdf` source was a 500 and a `.docx` source returned generator JavaScript as clean content - report a run file whose object retention has already swept as 404 rather than 500, on both the inline base64 read and the download stream - report a knowledge tag that loses at a unique index as 409, naming whether the slot or the display name is taken - gate `workflows versions revert` behind a CLI confirm; it overwrites the draft graph and was classified non-destructive
…overage # Conflicts: # apps/docs/openapi-v2-billing.json # apps/docs/openapi-v2-files-audit.json # apps/docs/openapi-v2-knowledge.json # apps/docs/openapi-v2-logs.json # apps/docs/openapi-v2-resources.json # apps/docs/openapi-v2-tables.json # apps/docs/openapi-v2-workflows.json # apps/sim/lib/api/contracts/v2/__tests__/tables.test.ts # apps/sim/lib/api/contracts/v2/openapi/tables.ts # scripts/check-api-validation-contracts.ts
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 36334842 | Triggered | Generic High Entropy Secret | 9ad29f4 | apps/sim/lib/api-key/byok.test.ts | View secret |
| 36334842 | Triggered | Generic High Entropy Secret | 9ad29f4 | apps/sim/lib/api-key/byok.test.ts | View secret |
| 36334842 | Triggered | Generic High Entropy Secret | 9ad29f4 | apps/sim/lib/api-key/byok.test.ts | View secret |
| 36334842 | Triggered | Generic High Entropy Secret | 9ad29f4 | apps/sim/lib/api-key/byok.test.ts | View secret |
| 36334841 | Triggered | Generic High Entropy Secret | 9ad29f4 | apps/sim/lib/api-key/byok.test.ts | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
- `PUT /workflows/{id}/state` now returns the same `lint` report as
`POST /operations`; an agent authoring a graph from scratch needs the
findings at least as much as one editing incrementally
- extract the report into one shared builder so the two writes cannot drift,
and one shared presenter so the wire shape is identical
- skip the credential/tool reference pass when the caller has no human
subject, rather than resolving it against the workspace billing owner:
that would misreport what the workflow can reach and disclose another
person's grants. `lint.notes` says when it was skipped
- add `?dryRun=true` to both graph writes: validates and lints, persists
nothing, records no audit, notifies nobody. A query param, not a body
field, since the body of a PUT is the resource itself
- CLI: a dry run no longer demands `--yes`; requiring confirmation to preview
a change teaches callers to pass `--yes` reflexively
- CLI: name the graph commands for their verbs — `workflows state get`,
`workflows state replace`, `workflows operations apply` — instead of the
derived `state list` / `state update` / `operations create`
- document when to use `rollback` vs `versions/{version}/activate` on both
- add `GET /workflow-mcp-servers/{serverId}` and
`GET /workflow-mcp-servers/{serverId}/tools`. The resource could be
PATCHed and DELETEd but never read, and its tools could be published and
unpublished but never listed — the server list reports tool names only, so
nothing published the `workflowId` that addresses a tool for deletion.
Both mirror `mcp-servers` beside them, and carry that family's
workspace-API-key denial rather than the wider `mcp_servers.read` policy
- rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches
the shipped `files` resource exactly (`move` + `bulk-delete`)
- name the CLI commands for their operations instead of the derived
`... create`: `tables move`, `workflows move`, `tables bulk-delete`, and
`tables rows update-each` for the per-row batch, which sits beside the
existing filter-based `tables rows batch-update`
`POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per
resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided
one word away from the filter form.
…overage # Conflicts: # apps/sim/lib/copilot/tools/server/workflow/edit-workflow/index.ts # apps/sim/tools/generated/tool-metadata.ts # scripts/check-api-validation-contracts.ts
- duplicating a workflow into a locked destination folder answered 500:
`FolderLockedError` is a plain Error carrying `status = 423`, which the v2
error policy does not classify. Converted to OrchestrationError('locked')
at the application boundary, matching the bulk-move path
- restore workflow promised a 413 for an oversized folder tree that its
response list never published; the cap is real, so the status now is too
- move workflows and apply variables documented 409/423 they cannot emit:
every per-item lock and conflict is reported in `failed`, not thrown
- bulk download and delete knowledge tag can both 409 and did not say so;
cleanup tag definitions cannot and did say so
- apply workflow operations denies workspace API keys but never documented it
- the dry-run responses are not byte-identical to a committed write:
`needsRedeployment` describes the pre-write state and persistence warnings
cannot appear. Reworded rather than overclaimed
- read file text and get file upload are head-safe, so the "HEAD skips the
effect" sentence did not apply to them
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GET/PUT /{id}/state,POST /{id}/operations, duplicate, restore, variables, move), a block/tool/connector-type/enrichment catalog, run-output file download and text extraction, knowledge chunk and tag writes with archive/restore, table run state and dispatch polling, log analytics and a sortable query, credential rotation, chat-deployment CRUD, deployment lifecycle, workflow-as-MCP, and a gate-exemptGET /v2/meta500s where a well-formed request could fault: unboundedsegmentCount, aRegExpbuilt from a caller-supplied block id, prototype-chain registry lookups, and strict response schemas asserting shapes that schemaless JSONB columns cannot honourLIKEpatterns, and a folder-name filter reading the wrong columnType of Change
Testing
Tested manually.
bun run check:audits32/32,turbo run type-check26/26,bun run lint, block-registry audit against the base, and the full suite: 30,358 app tests and 452 CLI tests passing. The 14 remainingworkflow-rendererfailures reproduce on the base branch and are unrelated to this change. No migrations touched.Checklist