Create Typefully drafts from DataOps Assistant jobs

[alexeygrigorev]

Create Typefully drafts from DataOps Assistant jobs

Status: pending
Tags: enhancement, assistant, backend, frontend, infra, P1
Depends on: None
Blocks: None

Scope

Add a Typefully integration that lets the DataOps Assistant create saved Typefully draft posts from assistant-generated social copy while keeping publishing under human control.

The implementation should live in the DataOps repo only. Expected touched surfaces are the work-engine/ assistant-job API and runner flow, the assistant queue/operator UI if a new action or status is needed, and the deployed stack/runtime-secret configuration needed by the private work-engine Lambda.

Official Typefully API facts to design against, from https://typefully.com/docs/api:

• Production API base URL is https://api.typefully.com and the public API uses bearer-token authentication.
• The current authenticated user can be checked with GET /v2/me; accessible social accounts are returned by GET /v2/social-sets.
• Draft creation is POST /v2/social-sets/{social_set_id}/drafts and requires platforms in the request body.
• Draft content is platform-scoped, for example platforms.x.posts[].text or platforms.linkedin.posts[].text; enabled platforms require enabled: true and posts.
• draft_title, scratchpad_text, existing tag slugs, and share are optional draft metadata. share creates a public share URL when true, so default DataOps behavior should keep it false unless explicitly configured.
• publish_at controls publishing or scheduling: now, next-free-slot, or an ISO 8601 datetime. To create a saved draft only, DataOps must omit publish_at.
• The create response includes the Typefully draft id, status, private_url, optional share_url, platform content, and scheduled/published fields.

Product behavior:

• A DataOps operator can request a Typefully draft from an assistant job tied to a task or workflow bundle.
• The assistant/job output remains reviewable in DataOps before the external Typefully write happens.
• The Typefully write creates a saved draft only. It must not publish immediately, schedule, queue to the next free slot, or expose any direct-publish control.
• After a successful Typefully draft creation, DataOps stores an artifact or equivalent proof record linked to the originating assistant job and task/bundle. The record should include safe identifiers and URLs such as Typefully draft id, social set id, status, private_url, platform names, and preview/title, but never the API token or raw auth headers.
• Typefully API failures should leave the assistant job in a recoverable failed/retry state with a redacted, operator-readable error summary.
• Local/test mode must be able to exercise the flow with a fake Typefully client or mock HTTP layer and no real Typefully token.

Auth and secrets:

• Typefully API credentials must come from managed config only: AWS Secrets Manager/SAM parameters for deployed runtime and documented local development environment variables for local runs.
• Do not hardcode tokens, social-set ids, or account-specific credentials in source, tests, issue comments, logs, artifacts, or assistant metadata.
• If the deployed stack needs a Typefully token secret name or work-engine permission to read it, add it to the SAM/CloudFormation template instead of relying on unmanaged production setup.
• Treat real Typefully account selection, real token provisioning, and real external draft creation as [HUMAN] checks.

Acceptance Criteria

• The assistant-job flow exposes an authenticated API path for creating a Typefully draft from approved/reviewed assistant output associated with a task or bundle.
• The Typefully request builder sends POST /v2/social-sets/{social_set_id}/drafts with bearer auth, valid platforms content, and no publish_at field for the default saved-draft path.
• The implementation rejects or ignores any attempt from DataOps UI/API input to use publish_at: "now", publish_at: "next-free-slot", or a scheduled datetime in this issue's flow.
• Successful draft creation stores a linked DataOps artifact/proof with the Typefully draft id, social set id, status, private URL, platform list, title/preview, originating assistant job id, and task/bundle context.
• No Typefully API token, bearer header, secret name value, signed URL, or raw credential can appear in assistant job metadata, events, artifacts, UI output, test snapshots, or logs.
• Missing credentials, missing social set configuration, Typefully 401/403/422/429 responses, and network failures produce redacted failed/retryable assistant-job states with useful operator messages.
• Local dry-run/mock verification covers successful draft creation without a real Typefully account or external write.
• The operator UI, if changed, shows the Typefully draft proof/link and keeps review/approval semantics clear before any external write action.
• Infrastructure/config changes document the required local env vars and deployed secret/SAM parameter names, and grant only the minimum runtime secret-read permission needed.
• [HUMAN] A DataOps operator provisions or confirms the real Typefully API token and target social set outside the repo.
• [HUMAN] A DataOps operator runs one real Typefully saved-draft creation and verifies in Typefully that the result is a draft, not scheduled and not published.

Test Scenarios

Scenario: Saved draft request

Given: an approved assistant output with social copy for X or LinkedIn and a configured fake Typefully social set
When: the operator/API triggers Typefully draft creation
Then: DataOps sends a create-draft request without publish_at, receives a fake Typefully draft response, and records a linked proof artifact with the private Typefully URL.

Scenario: Publishing controls are blocked

Given: a Typefully draft request payload attempts to include publish_at as now, next-free-slot, or a future datetime
When: the API validates the request
Then: the request is rejected before any Typefully client call, and no assistant job event or artifact contains a publish/schedule claim.

Scenario: Credential or API failure

Given: Typefully credentials are missing or the fake client returns 401, 403, 422, 429, or a network error
When: the assistant job attempts Typefully draft creation
Then: the job is marked failed or retryable according to the existing assistant-job lifecycle, the error is redacted, and no secret-like value is persisted.

Scenario: Operator review visibility

Given: a Typefully draft proof artifact has been created for a task or bundle
When: the operator opens the assistant queue, assistant job detail, and linked workflow/task view
Then: the operator can see the Typefully draft status/link and the originating context without seeing credentials.

Scenario: Real Typefully verification

Given: a human has provisioned a real Typefully API token and selected a real social set
When: a DataOps operator creates one test draft through the integration
Then: [HUMAN] the operator confirms in Typefully that it exists as a saved draft only and records the result on the issue.

Out of Scope

• Publishing, scheduling, queueing to next-free-slot, or changing Typefully queue schedules.
• Typefully webhook ingestion, analytics import, comments, media upload, tag management UI, or account-management UI.
• Creating or modifying source repos outside dataops, including ../dtc-operations, ../datatasks, or ../podcast-assistant.
• Automating real Typefully account/token provisioning.
• Closing the issue solely from automated tests if [HUMAN] verification remains outstanding.

Dependencies

• The target Typefully account must provide an API token with enough access to create drafts and a known target social_set_id for the desired account/team.
• If the real token is only available in production, the Software Engineer and Tester should use mocked/local verification and leave the issue open with the human label after agent-verifiable checks pass.
• Required verification should include focused work-engine assistant-job tests, TypeScript typecheck, build if Lambda packaging or env handling changes, focused Playwright coverage for any changed assistant UI, and SAM/CloudFormation validation for infrastructure changes.

[alexeygrigorev]

PM grooming complete.

Key handoff notes:

• Build this as a saved Typefully draft integration inside the existing DataOps assistant-job lifecycle.
• The default Typefully create request must omit publish_at; scheduling and direct publishing are out of scope.
• Use managed secrets/config only. Real Typefully token setup and real draft creation are [HUMAN] checks.
• Use mocked/local Typefully verification for agent-run tests, then leave any real-account verification for the human gate.

[alexeygrigorev]

Software Engineer implementation report — #77

Built a dedicated Typefully saved-draft integration into the assistant-job lifecycle. Not committed — ready for Tester + PM review.

Files changed

File	Change
work-engine/src/assistant/typefullyClient.ts	New. Canonical Typefully client module.
work-engine/src/routes/assistantJobs.ts	New POST /api/assistant-jobs/:id/typefully-draft action + proof artifact + lifecycle integration.
work-engine/src/assistant/socialDraftAssistant.ts	Refactored to import the client from typefullyClient.ts (removed duplicate client class/interfaces).
work-engine/tests/api-typefully-draft.test.ts	New. 29 tests covering the full flow.
lambda-functions/template.full.yaml	Typefully secret, env vars, and secret-read IAM permission.

Typefully client design (typefullyClient.ts)

• FetchTypefullyDraftClient: production bearer-auth client for POST /v2/social-sets/{social_set_id}/drafts. Reads TYPEFULLY_API_KEY (local) or resolves from Secrets Manager via TYPEFULLY_API_KEY_SECRET_NAME (deployed, cached per cold start).
• buildCreateDraftRequest(): the single chokepoint that builds the request body. Never emits publish_at — the TypefullyCreateDraftRequest type has no publish_at field at all.
• isPublishAtAttempt(): detects now, next-free-slot, and ISO datetimes anywhere in the input (including nested), so publishing control is rejected at the validation layer before any network call.
• FakeTypefullyDraftClient: in-process mock for local/test with no real token and no external write; records calls and enforces the same no-publish_at invariant.
• TypefullyRequestError: typed, redacted errors with stable codes (missing-credentials, unauthorized, forbidden, unprocessable, rate-limited, network-error, etc.) and a retryable flag.

Assistant-job action integration

POST /api/assistant-jobs/:id/typefully-draft creates a saved Typefully draft from reviewed assistant output:

• Allowed only for jobs in waiting_approval, approved, or succeeded (reviewed output). Returns 409 otherwise.
• Validates the request (socialSetId, platforms, non-empty posts), rejects publish_at and secret-bearing payloads before any client call.
• On success: stores a typefully proof artifact linked to the job, task, and bundle with draft id, social set id, status, private URL, platform list, preview/title, and publish_at_sent: false.
• On failure: maps to existing assistant-job lifecycle — retryable errors (429, network) move the job to retrying; non-retryable errors (401/403/422/missing-creds) move it to failed with a redacted lastError.

Security constraints enforced

• No publish_at ever: rejected at three layers (request parser → buildCreateDraftRequest → fake client).
• No credential leakage: errors redacted via containsSecret(); the secret-value pattern covers tokens, bearer headers, private draft URLs, and export paths. Verified by tests that inject secrets into failures and assert they never appear in the response.
• No share: true: always share: false (saved drafts only, no public share URL).

Test results

npm --prefix work-engine run typecheck   → pass
npm --prefix work-engine test            → 1089 pass, 0 fail
npm --prefix work-engine run build       → pass
sam validate template.full.yaml          → parses OK (credentials error only)

Test coverage: successful draft creation, publish_at rejection (now / next-free-slot / datetime), 401/403/422/429/network failures with correct retryable/failed states, credential redaction, proof artifact creation and task/bundle linking, secret-bearing request rejection, unreviewed-job blocking, approved-job path, and job-detail visibility.

Required env vars

Local dev: TYPEFULLY_API_KEY, optional TYPEFULLY_BASE_URL, TYPEFULLY_SOCIAL_SET_ALEXEY, TYPEFULLY_SOCIAL_SET_DATATALKSCLUB.
Deployed: TYPEFULLY_API_KEY_SECRET_NAME (Secrets Manager), plus the social set SAM parameters. All wired in template.full.yaml.

[HUMAN] checks (left for operator)

• Provision/confirm the real Typefully API token and target social set.
• Run one real saved-draft creation and verify in Typefully it is a draft (not scheduled/published).

Refs #77

READY FOR TEST

[alexeygrigorev]

PM ACCEPT — #77

Reviewed the implementation against the groomed spec, the acceptance criteria, and the operator flow. Verified independently: npm run typecheck (clean), npm test (1089 pass, 0 fail), npm run build (clean), and SAM template structure.

Spec match

• Saved drafts only. The TypefullyCreateDraftRequest type has no publish_at field at all, and publish_at is rejected at three layers: request parser → buildCreateDraftRequest → fake client. now, next-free-slot, and ISO datetimes (including nested) are all caught and return 400 before any client call.
• share: false is hardcoded in the request builder. No public share URL can be created through this flow.
• Proof artifacts store exactly the safe identifiers the spec names: draft id, social set id, status, private URL, platform list, preview/title, originating job id, task/bundle context, plus publish_at_sent: false as an explicit safety signal. Artifacts are linked to the job, task, and bundle via mirrorArtifactRef.
• Redacted errors: classifyHttpError maps 401/403/422/429/network to typed, operator-readable codes with a retryable flag. containsSecret runs on all error paths. The test injecting secret fragments into a failure summary confirms they never reach the response or persisted lastError.

Operator flow

• The Typefully write is gated on reviewed output: only waiting_approval, approved, or succeeded jobs can trigger it (409 otherwise). Review/approval semantics stay intact before any external write.
• On success, the proof artifact with the Typefully private URL is visible in the job detail endpoint, and the artifact-attached event records the Typefully draft id and status. On failure, retryable errors move the job to retrying; non-retryable errors move it to failed with a redacted lastError and a failure notification.
• The flow is API-first (POST /api/assistant-jobs/:id/typefully-draft), consistent with the process expectation that API coverage is the default for operator workflows. The UI criterion is conditional ("if changed") and no frontend was changed; the proof is fully surfaced through the job detail API. A dedicated UI trigger button is a reasonable follow-up but not required by the literal criterion.

Security

• Token resolved from TYPEFULLY_API_KEY (local) or Secrets Manager via TYPEFULLY_API_KEY_SECRET_NAME (deployed, cached per cold start, never logged or persisted).
• IAM permission is scoped to secretsmanager:GetSecretValue on the specific secret ARN, not a wildcard.
• No token, bearer header, secret name value, or signed URL can appear in artifacts, events, or responses — verified by the redaction test.

Mock mode

• FakeTypefullyDraftClient exercises the full flow in-process with no real token and no external write, and enforces the same no-publish_at invariant. 29 dedicated tests cover success, all failure modes, redaction, proof linking, and the review gate.

Infrastructure

• SAM template adds the Typefully secret (Retain policy), the secret-read IAM grant, and the env vars (TYPEFULLY_API_KEY_SECRET_NAME, TYPEFULLY_SOCIAL_SET_ALEXEY, TYPEFULLY_SOCIAL_SET_DATATALKSCLUB). Local dev env vars are documented in the SWE report and the client module header.

[HUMAN] checks correctly left open

• Real token provisioning and real draft creation remain for the operator. The human label should be added and the issue left open.

The refactoring in socialDraftAssistant.ts cleanly extracts the duplicate client into typefullyClient.ts and the typecheck confirms the two Platform type definitions (x | linkedin) are structurally compatible across modules.

Ready for SWE to commit with Refs #77 (not Closes #77). The issue stays open with the human label for the two [HUMAN] checks.

[alexeygrigorev]

TESTER PASS

Verified issue #77 (Typefully saved-draft integration) in worktree worktree-issue-77. All agent-verifiable acceptance criteria pass. The two [HUMAN] items remain open as expected.

Verification commands and results

• npm --prefix work-engine run typecheck — PASS (tsc, tests config, scripts config all clean)
• npm --prefix work-engine test — PASS, 1089 tests, 0 fail (includes the 29 new Typefully tests: 18 API + 11 client unit)
• npm --prefix work-engine run build — PASS
• uv run --project lambda-functions --extra search --with pytest python -m pytest tests/docs_app — PASS, 122 passed, 4 skipped (Playwright only), no collateral
• tests/docs_app/test_infra_template.py — PASS, 15/15 (parses lambda-functions/template.full.yaml with a CloudFormation-aware loader and asserts wiring; Typefully secret/env/IAM additions are consistent)
• sam validate — could not run (sandbox has no AWS credentials). Template validity is covered structurally by the infra-template test above, which loads and asserts the same file.

Security: publish_at blocked at every layer

TypefullyCreateDraftRequest has no publish_at field. buildCreateDraftRequest never emits it, sets share: false (typed as the literal false), and rejects any draft carrying publish_at with code publish-at-rejected. Verified directly:

• API layer (parseTypefullyDraftRequest) rejects now, next-free-slot, and ISO datetime before any client call (400, client callCount 0).
• isPublishAtAttempt catches nested ({ meta: { publish_at: "now" } }) and array forms too.
• buildCreateDraftRequest is the final guard and re-checks before any network call.
• FetchTypefullyDraftClient serializes the built request body, which contains no publish_at key.

Ad-hoc probe (blocks all publish_at vectors, redacts secret-bearing values, buildCreateDraftRequest rejects publish_at and never emits it) — 3/3 PASS.

Security: no credential leakage

resolveTypefullyApiKey keeps the token out of any returned object, event, artifact, or response. Failure paths build lastError via typefullyContainsSecret(error.summary) and replace with "... redacted sensitive details". Verified:

• A fake error containing TYPEFULLY_API_KEY=sk-typefully-secret, private_url=https://typefully.com/?d=12345, and .tmp/typefully-export/all-drafts.json is fully redacted in the persisted lastError and the 502 response — none of those fragments appear in the serialized body.
• The 401/403/422/429/network paths classify into correct retryable-vs-failed states (401/403/422 → failed, non-retryable; 429/network → retrying, retryable) with redacted operator-readable messages.
• Proof artifact metadata (typefullyProofMetadata) stores only safe identifiers: draft id, status, social set id, private URL, platform list, title/preview, originating job id, task/bundle context, and publish_at_sent: false. No token, header, or auth value is referenced.

Proof artifact

Successful creation stores a linked assistant-output proof artifact with all required fields (draft id, social set id, status, private URL, platform list, title/preview, originating job id, task/bundle context). Artifact is mirrored onto the job, task, and bundle; visible in the job detail view with an artifact-attached event. Approved jobs mark the proof approved; waiting_approval jobs mark it needs-review.

Mock mode

FakeTypefullyDraftClient exercises the full flow without a real token or external write, records calls, and enforces the no-publish_at invariant. FetchTypefullyDraftClient throws missing-credentials without TYPEFULLY_API_KEY.

Refactor note

FetchTypefullyDraftClient/TypefullyDraftResult/TypefullyDraftClient were extracted from socialDraftAssistant.ts into typefullyClient.ts. The existing 6 social-draft assistant tests still pass, and the refactor added Secrets Manager resolution + richer error classification to the client.

Minor non-blocking nits

• socialDraftAssistant.ts imports DEFAULT_TYPEFULLY_BASE_URL from typefullyClient but no longer uses it (tsconfig does not flag unused imports). Harmless; can be dropped in a follow-up.
• Standalone execution of the new test file alone yields 401s on the API tests because they depend on auth/DB seeding that the full suite establishes. Under the canonical npm --prefix work-engine test runner all 29 pass. Not a defect.

Acceptance criteria status

• Authenticated API path for Typefully draft from reviewed assistant output
• Request builder sends POST /v2/social-sets/{id}/drafts, bearer auth, valid platforms, no publish_at
• Rejects/ignores any publish_at attempt (now, next-free-slot, datetime)
• Proof artifact with draft id, social set id, status, private URL, platforms, title/preview, job id, task/bundle context
• No token/header/secret in metadata, events, artifacts, responses, snapshots, logs
• Missing creds / 401 / 403 / 422 / 429 / network → redacted failed/retryable states
• Local mock verification covers successful creation without a real account
• Operator UI for the draft proof/link — not changed in this diff (API-only); tracked separately
• Infra/config documents required env vars and deployed secret/SAM param names; minimum secret-read permission granted
• [HUMAN] Real Typefully token + social set provisioning
• [HUMAN] Real Typefully saved-draft creation and verification in Typefully

TEST DONE: TESTER PASS