Route agents to gusto feedback on friction and raw api calls - #167
Route agents to gusto feedback on friction and raw api calls#167azrosen92 wants to merge 3 commits into
gusto feedback on friction and raw api calls#167Conversation
Add a stderr-only nudge that points agents at `gusto feedback` when they reach for the raw REST escape hatch (`gusto api request`) or hit a genuine failure. The nudge in agent mode is a ready-to-run, pre-filled invocation carrying a PII-safe allowlisted context payload; human mode gets one short pointer line. stdout stays the byte-identical JSON envelope. - New `src/lib/feedback-nudge.ts`: trigger classification (escape_hatch / friction), suppression (feedback/config/auth-login commands, confirmation_required guardrail, dry-runs), a per-category 24h throttle persisted atomically in a separate nudge-state file, and a `feedback_nudge` opt-out. All disk I/O is best-effort and never fails the command. - Wire it into the runner just before exit, with an injectable clock. - Add the tri-state `feedback_nudge` config key (ask/always/never). - Fix `--category` discoverability: add commander `.choices()` so invalid values are rejected at parse time and `--help` lists the four categories. - Mark `--dry-run` short-circuits so the nudge stays suppressed on previews. Signed-off-by: Aaron Rosen <aaron.rosen@gusto.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review follow-ups on the feedback nudge: - Fix under-nudging on runner-synthesized usage failures. The nudge now classifies friction off the FINAL emitted exit code + emitted error envelope, not the raw handler result, so `--fields badkey` (unknown_fields) and bare `--fields` on a mutating command (fields_discovery_unsupported) both nudge. The early discovery rejection now funnels through the single nudge-then-exit tail rather than returning early, so every exit computes the nudge in one place. - Make the agent-mode `--context` argument robust: shell-escape the single-quoted JSON (`'` -> `'\''`) so a stray quote can never break a copy-paste. - Note the accepted best-effort throttle race (concurrent invocations in the same window may double-nudge; no lock needed). Signed-off-by: Aaron Rosen <aaron.rosen@gusto.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bare `--fields` discovery hint on a read command is a successful usage helper (non-zero exit, no error envelope) — discovery, not friction — so it should not nudge. Change the friction condition to require an emitted error (`error !== undefined && error.code !== "confirmation_required"`); the non-Success exit is already implied whenever an error was emitted. This keeps unknown_fields and fields_discovery_unsupported nudging (both emit an error) while excluding the discovery-hint success path. Add a test asserting a bare `--fields` discovery hint on a read command lists fields on stderr but emits no nudge. Signed-off-by: Aaron Rosen <aaron.rosen@gusto.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
⚠️ 1 New Security Finding
The latest commit contains 1 new security finding.
Findings Note: 1 finding is displayed as an inline comment.
Not a finding? Ignore it by adding a comment on the line with just the word noboost.
Scanner: boostsecurity - Semgrep
| { now: deps.now ?? Date.now, configPaths }, | ||
| ); | ||
| code = ExitCode.General; | ||
| if (nudge) sinks.stderr.write(nudge); |
There was a problem hiding this comment.
CWE-79: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
Original Rule ID: rules-javascript-xss-generic
Details
The product does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.
The application passes unsanitized user input to dangerous DOM manipulation methods
or HTTP response functions that can lead to Cross-Site Scripting (XSS). This rule
detects data flowing from untrusted sources like
req.query, req.body, location.href,document.querySelector().value, and function arguments into sensitive sinks includinginnerHTML, outerHTML, document.write(), document.writeln(), jQuery methodslike
$.html(), $.append(), and Express response methods like res.send(). Thesesinks render content directly into the DOM or HTTP responses without automatic escaping.
When user-controlled data flows through these paths, attackers can inject malicious
JavaScript that executes in the victim's browser, potentially stealing session cookies,
accessing local storage, hijacking user accounts, or performing unauthorized API calls.
This vulnerability affects both DOM-based XSS (client-side sources like URL fragments),
reflected XSS (server-echoed input), and stored XSS (persisted malicious content).
📘 Learn More
AI Remediation
This is a false positive: the stderr.write sink is a Node.js CLI stderr stream (a terminal/pipe), not an HTML DOM node or HTTP response, so no browser-side script execution can occur. The fields values are the command's own output-data object keys rendered as plain terminal text, which is never interpreted as HTML/JS, so CWE-79 (XSS) does not apply here.
At line 215, do the following changes:
/** Print the gh-style "you must specify fields" hint to stderr, listing what's available. */
function writeFieldsHint(fields: string[], deps: RunnerDeps): void {
const stderr = deps.sinks?.stderr ?? process.stderr;
- stderr.write(`Specify one or more comma-separated fields for \`--fields\`:\n${fieldLines(fields)}\n`);
+ stderr.write(`Specify one or more comma-separated fields for \`--fields\`:\n${fieldLines(fields)}\n`); // noboost
}
export function notImplementedHandler(commandPath: string): CommandHandler {| // tail, so the nudge is computed in one place off the runner-final code + emitted error. Stderr | ||
| // only: stdout stays the byte-identical JSON envelope. Best-effort: any failure computing or | ||
| // writing the nudge is swallowed rather than allowed to affect the command. | ||
| try { |
There was a problem hiding this comment.
Parse-time failures never reach this tail. main() catches CommanderError and emits the standard error envelope directly, so unknown commands/options, missing positional arguments, and invalid .choices() values produce no friction nudge. That leaves part of the advertised "any command failure that emits an error envelope" behavior uncovered. Could the top-level Commander error path call the same nudge logic (while preserving the command suppressions)?
| } else { | ||
| try { | ||
| const result = await handler({ command, globals, sinks }); | ||
| if (result.ok && result.dryRun === true) dryRun = true; |
There was a problem hiding this comment.
The runner only learns that this was a dry-run when a successful result explicitly carries dryRun: true, but most existing dry-run paths do not set that marker. For example, putResourceWithVersion returns a normal { ok: true, data } preview. Combining one of those previews with an invalid --fields selection makes the runner synthesize unknown_fields and emit a nudge even though --dry-run was passed. Errors returned before a preview short-circuit cannot carry this success-only marker either. Could invocation-level dry-run state be passed to the runner, or all applicable result paths be marked consistently?
What
Routes agents to the existing
gusto feedbackcommand at two high-signal moments via a stderr-only nudge (the stdout JSON envelope stays byte-identical):blocked_on,insufficient_scope,not_implemented,internal_error, and--fieldsusage errors).gusto api request, success or failure (reaching for the raw escape hatch is a command-gap signal).Also:
--categorydiscoverability: it now uses commander.choices(), so an invalid value is rejected at parse time and the four valid values (bug,feature_request,general,praise) appear in--help. The nudge depends on this.feedback_nudgeconfig key (tri-state, modeled onskills_auto_install) to opt out, and a per-category 24h throttle stored in a separate state file (atomic write, best-effort — never fails the command).Why
gusto feedbackworks but was discoverable only via--help, so an agent that hit a gap had no idea the channel existed and the signal was lost. These are the two moments most worth capturing.Guardrails
--contextis a strict allowlist — command slug, exit code, error code, request id, CLI version, environment, trigger. It never includes the response body,error.details, or any employee/admin identifier.gusto feedbackitself,gusto config,gusto auth login,confirmation_required(a working guardrail, not friction), and--dry-run.--contextargument is shell-escaped).Testing
Unit tests cover fires/suppression per trigger, byte-identical stdout, the PII allowlist, throttle + opt-out, and
--categoryparse-time rejection.typecheck+lint+format+ DCO checks pass.Related: AINT-805. Touches
runner.tsalongside the command-header PR — whichever merges second needs a trivial rebase.🤖 Generated with Claude Code