Skip to content

Route agents to gusto feedback on friction and raw api calls - #167

Open
azrosen92 wants to merge 3 commits into
mainfrom
aint-805-feedback-nudge
Open

Route agents to gusto feedback on friction and raw api calls#167
azrosen92 wants to merge 3 commits into
mainfrom
aint-805-feedback-nudge

Conversation

@azrosen92

Copy link
Copy Markdown
Contributor

What

Routes agents to the existing gusto feedback command at two high-signal moments via a stderr-only nudge (the stdout JSON envelope stays byte-identical):

  • friction — any command failure that emits an error envelope (API 4xx/5xx, validation/blocked_on, insufficient_scope, not_implemented, internal_error, and --fields usage errors).
  • escape_hatch — every gusto api request, success or failure (reaching for the raw escape hatch is a command-gap signal).

Also:

  • Fixes --category discoverability: it now uses commander .choices(), so an invalid value is rejected at parse time and the four valid values (bug, feature_request, general, praise) appear in --help. The nudge depends on this.
  • Adds a feedback_nudge config key (tri-state, modeled on skills_auto_install) to opt out, and a per-category 24h throttle stored in a separate state file (atomic write, best-effort — never fails the command).

Why

gusto feedback works but was discoverable only via --help, so an agent that hit a gap had no idea the channel existed and the signal was lost. These are the two moments most worth capturing.

Guardrails

  • No PII: the auto-populated --context is a strict allowlist — command slug, exit code, error code, request id, CLI version, environment, trigger. It never includes the response body, error.details, or any employee/admin identifier.
  • Suppressed on: gusto feedback itself, gusto config, gusto auth login, confirmation_required (a working guardrail, not friction), and --dry-run.
  • The emitted agent-mode command is copy-paste-safe (the --context argument is shell-escaped).

Testing

Unit tests cover fires/suppression per trigger, byte-identical stdout, the PII allowlist, throttle + opt-out, and --category parse-time rejection. typecheck + lint + format + DCO checks pass.

Related: AINT-805. Touches runner.ts alongside the command-header PR — whichever merges second needs a trivial rebase.

🤖 Generated with Claude Code

azrosen92 and others added 3 commits August 18, 2026 15:31
Add a stderr-only nudge that points agents at `gusto feedback` when they
reach for the raw REST escape hatch (`gusto api request`) or hit a genuine
failure. The nudge in agent mode is a ready-to-run, pre-filled invocation
carrying a PII-safe allowlisted context payload; human mode gets one short
pointer line. stdout stays the byte-identical JSON envelope.

- New `src/lib/feedback-nudge.ts`: trigger classification (escape_hatch /
  friction), suppression (feedback/config/auth-login commands,
  confirmation_required guardrail, dry-runs), a per-category 24h throttle
  persisted atomically in a separate nudge-state file, and a `feedback_nudge`
  opt-out. All disk I/O is best-effort and never fails the command.
- Wire it into the runner just before exit, with an injectable clock.
- Add the tri-state `feedback_nudge` config key (ask/always/never).
- Fix `--category` discoverability: add commander `.choices()` so invalid
  values are rejected at parse time and `--help` lists the four categories.
- Mark `--dry-run` short-circuits so the nudge stays suppressed on previews.

Signed-off-by: Aaron Rosen <aaron.rosen@gusto.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review follow-ups on the feedback nudge:

- Fix under-nudging on runner-synthesized usage failures. The nudge now
  classifies friction off the FINAL emitted exit code + emitted error
  envelope, not the raw handler result, so `--fields badkey`
  (unknown_fields) and bare `--fields` on a mutating command
  (fields_discovery_unsupported) both nudge. The early discovery
  rejection now funnels through the single nudge-then-exit tail rather
  than returning early, so every exit computes the nudge in one place.
- Make the agent-mode `--context` argument robust: shell-escape the
  single-quoted JSON (`'` -> `'\''`) so a stray quote can never break a
  copy-paste.
- Note the accepted best-effort throttle race (concurrent invocations in
  the same window may double-nudge; no lock needed).

Signed-off-by: Aaron Rosen <aaron.rosen@gusto.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bare `--fields` discovery hint on a read command is a successful usage
helper (non-zero exit, no error envelope) — discovery, not friction — so it
should not nudge. Change the friction condition to require an emitted error
(`error !== undefined && error.code !== "confirmation_required"`); the
non-Success exit is already implied whenever an error was emitted. This keeps
unknown_fields and fields_discovery_unsupported nudging (both emit an error)
while excluding the discovery-hint success path.

Add a test asserting a bare `--fields` discovery hint on a read command lists
fields on stderr but emits no nudge.

Signed-off-by: Aaron Rosen <aaron.rosen@gusto.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@azrosen92
azrosen92 requested review from a team and ashieh as code owners August 18, 2026 21:52

@boostsecurity-io-ai boostsecurity-io-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️  1 New Security Finding

The latest commit contains 1 new security finding.

Findings Note: 1 finding is displayed as an inline comment.

Not a finding? Ignore it by adding a comment on the line with just the word noboost.

Scanner: boostsecurity - Semgrep

Comment thread src/lib/runner.ts
{ now: deps.now ?? Date.now, configPaths },
);
code = ExitCode.General;
if (nudge) sinks.stderr.write(nudge);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CWE-79: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
Original Rule ID: rules-javascript-xss-generic

Details

The product does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.

The application passes unsanitized user input to dangerous DOM manipulation methods
or HTTP response functions that can lead to Cross-Site Scripting (XSS). This rule
detects data flowing from untrusted sources like req.query, req.body, location.href,
document.querySelector().value, and function arguments into sensitive sinks including
innerHTML, outerHTML, document.write(), document.writeln(), jQuery methods
like $.html(), $.append(), and Express response methods like res.send(). These
sinks render content directly into the DOM or HTTP responses without automatic escaping.
When user-controlled data flows through these paths, attackers can inject malicious
JavaScript that executes in the victim's browser, potentially stealing session cookies,
accessing local storage, hijacking user accounts, or performing unauthorized API calls.
This vulnerability affects both DOM-based XSS (client-side sources like URL fragments),
reflected XSS (server-echoed input), and stored XSS (persisted malicious content).
 📘 Learn More

AI Remediation

This is a false positive: the stderr.write sink is a Node.js CLI stderr stream (a terminal/pipe), not an HTML DOM node or HTTP response, so no browser-side script execution can occur. The fields values are the command's own output-data object keys rendered as plain terminal text, which is never interpreted as HTML/JS, so CWE-79 (XSS) does not apply here.

At line 215, do the following changes:

 /** Print the gh-style "you must specify fields" hint to stderr, listing what's available. */
 function writeFieldsHint(fields: string[], deps: RunnerDeps): void {
   const stderr = deps.sinks?.stderr ?? process.stderr;
-  stderr.write(`Specify one or more comma-separated fields for \`--fields\`:\n${fieldLines(fields)}\n`);
+  stderr.write(`Specify one or more comma-separated fields for \`--fields\`:\n${fieldLines(fields)}\n`); // noboost
 }
 
 export function notImplementedHandler(commandPath: string): CommandHandler {

Comment thread src/lib/runner.ts
// tail, so the nudge is computed in one place off the runner-final code + emitted error. Stderr
// only: stdout stays the byte-identical JSON envelope. Best-effort: any failure computing or
// writing the nudge is swallowed rather than allowed to affect the command.
try {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parse-time failures never reach this tail. main() catches CommanderError and emits the standard error envelope directly, so unknown commands/options, missing positional arguments, and invalid .choices() values produce no friction nudge. That leaves part of the advertised "any command failure that emits an error envelope" behavior uncovered. Could the top-level Commander error path call the same nudge logic (while preserving the command suppressions)?

Comment thread src/lib/runner.ts
} else {
try {
const result = await handler({ command, globals, sinks });
if (result.ok && result.dryRun === true) dryRun = true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runner only learns that this was a dry-run when a successful result explicitly carries dryRun: true, but most existing dry-run paths do not set that marker. For example, putResourceWithVersion returns a normal { ok: true, data } preview. Combining one of those previews with an invalid --fields selection makes the runner synthesize unknown_fields and emit a nudge even though --dry-run was passed. Errors returned before a preview short-circuit cannot carry this success-only marker either. Could invocation-level dry-run state be passed to the runner, or all applicable result paths be marked consistently?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants