Skip to content

Two-phase setup: fast path to first ASR - #27

Open
nrodd wants to merge 7 commits into
mainfrom
faster-first-asr
Open

Two-phase setup: fast path to first ASR#27
nrodd wants to merge 7 commits into
mainfrom
faster-first-asr

Conversation

@nrodd

@nrodd nrodd commented Aug 10, 2026

Copy link
Copy Markdown
Member

Overview

Reworks the wizard so the user reaches a captured session — and their first agentic session review (ASR) — as fast as possible, and trims the number of prompts along the way.

What changed

Two-phase install

  • Phase 1 (Step 1 of 2) installs the capture snippet + CSP only — precheck → explore → plan → install — then shows the first-run guide so the user can restart their dev server, capture a session, and review it right away.
  • Phase 2 (Step 2 of 2, optional) is the enrichment step: identify users, link analytics, mask PII. For terminal agents it's a second driven run in the same session; for app/manual hand-offs it's a copyable follow-up prompt.
  • The prompt template was split into a shared shell (install-prompt.md) plus per-phase step bodies (install-steps-snippet.md, install-steps-enrich.md); build.ts now exposes buildSnippetPrompt / buildEnrichPrompt.

Fewer, better-placed prompts

  • The pre-handoff prompt review now carries the autonomy consent for terminal agents (removed the separate "run autonomously?" confirm), and is a lighter Yes / "review first" confirm instead of a 3-option menu.
  • The enrichment opt-in stands alone (removed the redundant second prompt review).
  • Session-review-tools (plugin/MCP) consent for terminal agents is asked up front alongside the handoff rather than mid-flow, so the post-install first-run guide isn't interrupted — the setup itself still runs after the install.
  • Opening banner + telemetry notice trimmed to a line each.

Integration auto-detection

  • The analytics picker moved into phase 2 (where it's used). When detection finds tools in the app's package.json (deps + devDeps), the wizard confirms those up front — names highlighted in the brand color — and keeps the full 15-item catalog collapsed behind a simple "add any others?" prompt, expanding to the picker only if the user opts in. When nothing is detected the picker behaves as before, and an empty selection hands detection to the agent.

Telemetry funnel unchanged — one start/complete spanning both phases, each step reported exactly once.

Post-install flow & tooling

Follow-on refinements to the hand-off and the moments right after install:

  • Unauthenticated-tool catch — the demo prompt now tells the agent to stop and report if the Subtext tools return an auth/authorization error, and the first-run guide notes that the agent will prompt for sign-in on first tool use (the MCP server is OAuth-protected, so installing the plugin lists the tools but they stay unauthorized until the user signs in).
  • Enrichment gated on the demo — instead of prompting for phase 2 on top of the "go watch a session review" guide, a minimal "try the demo, press Enter when done" hold keeps the enrichment details out of view until the user is ready. Shared across all three hand-off paths.
  • Open the agent at hand-off — the demo and follow-up steps offer to open the coding agent with the prompt already on the clipboard. GUI apps reuse openAppAtDir; macOS Terminal.app terminal agents open a fresh window running the harness (iTerm/other/non-macOS fall back to the copy-only offer). New shared openAgent.ts; DetectedAgent gains macAppName so Claude Desktop (no CLI) can still be opened.
  • --print-prompt is now a dry run — it prints each install prompt as it's built and walks the flow, but skips every agent launch (terminal driveLaunch, app launch, packaged-plugin install, demo/follow-up auto-open) so nothing spawns the agent. Previously it printed both prompts and exited.

⚠️ Needs attention before merge: Hydrogen revert

This branch is based on 73c3b92 (#24), which is 2 commits behind main. One of those is #26 "Revert Hydrogen route from main"main no longer has the Shopify Hydrogen block in the install prompt, but our split templates still carry it (install-steps-snippet.md + the Hydrogen skip notes in install-steps-enrich.md). Rebasing onto main will effectively reintroduce the reverted Hydrogen content.

Recommend: rebase onto origin/main, then decide whether to keep the Hydrogen route in the new templates or drop it to stay consistent with the revert.

Testing

  • npm run typecheck + npm run build clean.
  • --print-prompt prints each phase's prompt as it's built and walks the flow while skipping every agent launch; phase-1 has no analytics/identity/PII steps, phase-2 does + the integrations section.
  • Manual (--agent manual) flow verified end-to-end in --mock: Step 1 marker, first-run guide, enrich gate, copyable follow-up, clean exit.
  • Telemetry step tables verified per phase (snippet: precheck/explore/plan/install; enrich: identify/link_analytics/mask_pii; MCP complete row only on the snippet phase).
  • detectInstalledIntegrations unit-checked against a fixture (posthog-js, @amplitude/analytics-browser, @sentry/react detected; react/vite ignored; missing dir → none).
  • AppleScript command/escaping for the Terminal.app open verified in isolation (paths with spaces and apostrophes survive the shell→AppleScript round-trip); the live window-open was not spawned in-session.

Not exercised in an automated run (need a real TTY / would spawn a real agent): the interactive confirms, the driven terminal phase-1 → phase-2 path, and the real Terminal.app/GUI agent auto-open. Worth a manual smoke test with --agent claude-code against a throwaway app before merge.

Follow-ups (not in this PR)

  • Under --yes with no --integrations, selectIntegrations still reaches the interactive picker; could use the detected set non-interactively in CI.
  • Monorepo / script-tag (CDN) integration detection is intentionally left to the agent's explore step.
  • --print-prompt now walks the real flow, so it still fires start/complete telemetry (a synthetic no-op run); pair with --mock to suppress, or wire --print-prompt to imply telemetry-off if that's preferred.

🤖 Generated with Claude Code


Note

Medium Risk
Large orchestration changes in run.ts (multi-path wizard, telemetry bookends, optional phase 2) could affect onboarding metrics or edge-case exits; agent launches and clipboard flows are user-facing but not security-critical.

Overview
Two-phase onboarding gets users to a captured session and first agentic session review sooner. Step 1 of 2 only runs precheck → explore → plan → install for the capture snippet and CSP; Step 2 of 2 (optional) covers user identification, analytics linkage, and PII masking. Terminal agents get a second driven run after the first-run demo; app and manual paths get a copyable follow-up prompt.

The install prompt is split into a shared shell plus install-steps-snippet.md / install-steps-enrich.md, with buildSnippetPrompt and buildEnrichPrompt and per-phase telemetry step tables so each funnel step is reported once across both phases (one start/complete for terminal runs).

UX and consent are tightened: prompt review is a Yes / “review first” confirm that also carries terminal autonomy consent; session-review plugin/MCP consent for terminal agents moves before handoff; enrichment is gated behind “try the demo first” instead of stacking on the guide. Analytics integration picking moves to phase 2, with package.json-based pre-selection and a collapsed “add others?” flow when tools are detected.

Hand-offs add shared offerCopyAndOpen (demo + follow-up), optional agent open via openAgent.ts, and DetectedAgent fields macAppName / opensFolder for reliable GUI reopen. --print-prompt becomes a dry run that prints prompts and walks the flow without spawning agents or inflating funnel telemetry.

Reviewed by Cursor Bugbot for commit 26a9808. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread src/run.ts
Comment thread src/prompt/build.ts
nrodd and others added 3 commits August 11, 2026 13:50
Ctrl+C on the plugin-consent confirm was swallowed as "no" (decline
plugin) and let the autonomous install launch anyway. Nothing has run at
that point, so cancel now aborts (CancelledError) like every other
pre-handoff prompt; only an explicit "No" declines the plugin and lets
the install proceed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts:
#	templates/install-prompt.md
Comment thread src/run.ts
Comment thread src/promptReview.ts
…on, dry-run --print-prompt

- Gate the phase-2 enrichment offer on the user finishing the demo instead of
  prompting over it: a minimal "try the demo, press Enter when done" hold keeps
  the enrichment details out of view until they're ready (shared across all
  three hand-off paths).

- Offer to open the coding agent at the demo and follow-up hand-offs, with the
  prompt already on the clipboard. GUI apps reuse openAppAtDir; macOS Terminal.app
  terminal agents open a fresh window running the harness. New shared openAgent.ts
  (canOpenAgent/openAgent/offerCopyAndOpen); DetectedAgent gains macAppName so
  Claude Desktop (no CLI) can still be opened.

- Collapse integration detection: when package.json detection finds tools, confirm
  those up front and keep the 15-item catalog behind a simple "add any others?"
  rather than showing the whole list next to one detected tool. Detected names
  are highlighted in brandPink so they stand out.

- --print-prompt is now a dry run: it prints each install prompt as it's built
  and walks the flow, but skips every agent launch (terminal driveLaunch, app
  launch, packaged-plugin install, demo/follow-up auto-open) so nothing spawns
  the agent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread src/run.ts
Comment thread src/run.ts
Comment thread src/openAgent.ts
- Dry-run telemetry: --print-prompt no longer authorizes telemetry (or shows the
  "telemetry on" notice), so a dry run that never installs can't emit a
  start/complete and inflate the onboarding funnel.

- Demo/follow-up open no longer clobbers the install prompt: when the clipboard
  still holds a needed prompt (app/manual install prompt), fall back to the
  copy-only offer instead of bundling "open the agent" with a clobbering copy.

- App reopen no longer passes a folder to openAppAtDir: the app is already open
  at the project, so we just bring it forward — avoids `open -a Claude <path>`
  for apps that don't open folders (Claude Desktop).

- Terminal enrich uses followUpSelection so a Ctrl+C on the optional integration
  picker maps to an empty selection and the enrich run still proceeds, matching
  the app/manual paths (was silently dropping a phase the user opted into).

- Phase-2 enrich report writes subtext-enrich-report.md instead of overwriting
  phase-1's subtext-setup-report.md; outro points at any subtext-*-report.md.

- Pre-handoff prompt review carries the autonomy hint on the second confirm too
  (the one that actually authorizes the run), not just the first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 350bbe4. Configure here.

Comment thread src/openAgent.ts Outdated
Follow-up to the previous Bugbot fix: dropping the folder arg entirely fixed
Claude Desktop but made the CLI launchers (cursor/code/zed/windsurf) open a
blank window instead of focusing the project. Thread each app's opensFolder
flag through DetectedAgent -> OpenAgentTarget and pass the project dir only for
folder-opening apps; Claude Desktop (opensFolder: false) still just comes to the
foreground.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant