feat(wizard): add --detect-only headless detection mode - #1056
Draft
ablaszkiewicz wants to merge 2 commits into
Draft
feat(wizard): add --detect-only headless detection mode#1056ablaszkiewicz wants to merge 2 commits into
ablaszkiewicz wants to merge 2 commits into
Conversation
…to PostHog
Runs the agentic source-maps detection headlessly (doctor-style: auth, scan,
POST, exit — never enters the TUI or agent runner) and upserts the report to
/api/projects/{id}/wizard/repository_detections/. Built for the cloud wizard
run so a connected repository can be scanned in the background; also works
locally with --api-key. Failures (agent error, timeout) are recorded to the
same endpoint so the app can present them.
Hidden flag, unstable contract — same policy as the headless flag.
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
ablaszkiewicz
added a commit
to PostHog/posthog
that referenced
this pull request
Aug 5, 2026
One kind identifier flows end to end: POST /api/wizard/detection
{project_id, repository, kind} creates a task and starts the dedicated
detect-repository Temporal workflow, which provisions a sandbox, clones
the repository, and runs the wizard detection program the kind selects
(WIZARD_DETECTION_PROGRAMS registry; error-tracking-source-maps runs
upload-source-maps --detect-only). The wizard posts its report to the
new POST-only repository_detections upsert endpoint under the same
kind, using its cloud-run OAuth token (the wizard_session scope its
grant already carries), so no scope or ceiling changes anywhere.
RepositoryDetection is a generic store: one row per (team, repository,
kind), one typed report serializer shared by all kinds, error variant
for failed runs. The workflow is a sibling of process-task reusing its
activities with none of the agent machinery: no agent server, no PR,
no CI loop, sandbox cleanup guaranteed in finally.
Verified end to end locally: cloud runs against posthog/hedgebox and
posthog/error-tracking-examples completed and stored reports; the
error path stores its failure under the same key.
Wizard side: PostHog/wizard#1056 (must release to npm first).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ablaszkiewicz
added a commit
to PostHog/posthog
that referenced
this pull request
Aug 5, 2026
One kind identifier flows end to end: POST /api/wizard/detection
{project_id, repository, kind} creates a task and starts the dedicated
detect-repository Temporal workflow, which provisions a sandbox, clones
the repository, and runs the wizard detection program the kind selects
(WIZARD_DETECTION_PROGRAMS registry; error-tracking-source-maps runs
upload-source-maps --detect-only). The wizard posts its report to the
new POST-only repository_detections upsert endpoint under the same
kind, using its cloud-run OAuth token (the wizard_session scope its
grant already carries), so no scope or ceiling changes anywhere.
RepositoryDetection is a generic store: one row per (team, repository,
kind), one typed report serializer shared by all kinds, error variant
for failed runs. The workflow is a sibling of process-task reusing its
activities with none of the agent machinery: no agent server, no PR,
no CI loop, sandbox cleanup guaranteed in finally.
Verified end to end locally: cloud runs against posthog/hedgebox and
posthog/error-tracking-examples completed and stored reports; the
error path stores its failure under the same key.
Wizard side: PostHog/wizard#1056 (must release to npm first).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
🦔 ReviewHog reviewed this pull requestFound 0 must fix, 2 should fix, 1 consider. Published 3 findings (view the review). |
Contributor
|
ReviewHog Alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏 |
![posthog[bot]](https://avatars.githubusercontent.com/in/1203049?s=60&v=4){kind=small}
- Dispatch runDetectOnly through runCommandHandler so pre-try/catch throws exit cleanly instead of becoming unhandled rejections - Retry the detection POST (3 attempts, 30s per-attempt timeout, capped backoff) on network errors, 429, and 5xx; fail fast on other 4xx — the upsert is idempotent and guards a 5-minute scan - Reconcile the entry-point docstring with the actual exit-code contract: 0 only on a saved successful detection Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 9, 2026
ablaszkiewicz
changed the title
Contributor
|
Any reason to not make it a command on its own? Is the detection data specific to sourcemap uploads ? |
Contributor
Author
|
@hpouillot yes. The detection data is specific to our program. We have a generic |
hpouillot
approved these changes
Aug 11, 2026
| } | ||
|
|
||
| /** Best-effort: record the failure server-side so the app can show it. */ | ||
| async function recordFailure( |
Contributor
There was a problem hiding this comment.
why not capture the exception inside this method instead ?
| }); | ||
| getUI().log.success( | ||
| `Detection saved: ${report.projects.length} project${ | ||
| report.projects.length === 1 ? '' : 's' |
Contributor
There was a problem hiding this comment.
don't we have pluralize ?
Collaborator
|
@ablaszkiewicz @hpouillot you read my mind. I had a
This way when we see there's a gap in analytics, we can even write scouts to fill gaps An idea I had. I would make it its own command entirely ;) and make it that you can do posthog detect --source-map or smthing (maybe should be by product name) Think about it. This is just me nudging you, please prioritize as you see fit |
Contributor
Author
|
@gewenyu99 I agree that sounds good but I don't feel that I am the right person to come up with a new standard. I am lacking knowledge about wizard and cloud runs and how this plays together so I feel like I just wouldn't design this well. My wizard knowledge is very narrow and very focused just on our program. I am happy to be a part of discussion tho and happy to migrate our ET detection to that once some standard is established. |
ablaszkiewicz
marked this pull request as draft
August 11, 2026 19:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a part of a bigger change:
Adds
--detect-onlycommand variant for uploading source maps. This runs only the detection and posts results back to posthog via scoped endpoint,