Overhaul/auth architecture

.env

@@ -1,2 +1,2 @@
-#GEMITERM_CONFIG_DIR=./.gemiterm
+GEMITERM_CONFIG_DIR=./.gemiterm
 GEMITERM_VERBOSE=true

CHANGELOG.md

@@ -1,11 +1,11 @@
-## [2.7.0] - 2026-08-08
+## [2.7.0] - 2026-08-09
 
 ### Added
 
 - **`gemiterm status --verbose` (`-v`).** Prints per-profile cookie counts and the next `__Secure-1PSIDTS` expiry countdown, followed by the absolute path to each profile storage directory — useful for diagnosing cookie expiry without opening the `%APPDATA%\gemiterm` directory by hand. New `formatDuration(ms)` helper in `infrastructure/formatters.ts` renders compact age strings ("4d 6h" / "2h 30m" / "expired").
 - **`status` PROBE column.** `bun run dev status` now validates every profile against Google's API on every invocation — `models()` and `listChats({ limit: 1 })` run in parallel. Three-state column: `✓ live (N≥1)`, `⚠ phantom (models N)`, or `✗ dead: <error>`. Catches the phantom-auth state that was hiding behind local freshness checks. Always-on, no flag needed.
 - **Targeted L2 recovery for phantom-auth sessions.** When phantom-auth is detected (models works, listChats empty), `ensureAuthenticated` triggers a headless browser refresh that updates only PSIDTS-related cookies (`__Secure-1PSIDTS`, `__Secure-3PSIDTS`, `SIDCC`) instead of replacing the full jar. Preserves the original login's PSID + companion cookies while picking up a fresh PSIDTS from the browser session. Falls through to full headed re-auth when targeted L2 cannot recover.
-- **Phase 0 v2 regression net.** 10 tests (0a–0j) across 8 files lock every known auth bug contract at the cheapest seam: cookie-monitor full-jar capture (0a), auth round-trip (0b), time-passing clock injection (0c), continue-chat metadata (0d), profile routing (0e), recovery ladder (0f), L2 cookie corruption (0g), context roundtrip (0i), status PROBE (0j). Designed to go RED on the exact commit that introduced each historical regression. Documented at `docs/phase-0/phase-0-v2-design.md`.
+- **Dormancy-resilient auth gates.** `ensureAuthenticated` no longer throws when probe/freshness checks fail — the session is allowed to reach the Gemini API, which returns a 401 only if the session is genuinely dead. Expired-but-present cookies no longer force a re-login; stale server probes no longer kill the session. The two fatal gates (cookie freshness and server-side probe) are now non-fatal, with `DO NOT THROW` regression tests locking this behavior. This restores v2.4.0's multi-day session tolerance while keeping v2.7.0's server-side validation and recovery ladder.
 
 ### Fixed
 
@@ -27,10 +27,14 @@
 ### Internal
 
 - `createClientServices` extracted from `src/cli/index.ts` into `src/cli/client-services.ts` to expose a testable seam for the `forProfile` wiring.
-- Test suite: **954 pass / 1 skip / 0 fail / 2030 expects** (was 928 / 1945 at 2.6.1). Red-then-green regression tests added for each fix at the cheapest seam.
+- Test suite: **990 pass / 1 skip / 0 fail / 2089 expects** (was 954 / 2030). Red-then-green regression tests added for each fix at the cheapest seam.
+- **CookieJar unification (Candidate A).** Replaced 5 uncoordinated cookie-jar writers with a single `CookieJar` module offering two policies: `replace()` (login capture) and `upsert()` (rotations/refreshes), keyed by `(name, domain, path)`. All jar mutations now flow through one interface. See `src/services/cookie-jar.ts`.
+- **Explicit auth state machine (Candidate C).** `classifySession()` + `getRecoveryAction()` replace 10+ branches of implicit state logic in `ensureAuthenticated` with 5 named states (Fresh/Phantom/Dead/Stale/Declined) and typed recovery actions. Both functions are pure and independently testable. See `src/services/session-state.ts`.
+- **Conversation threading module (Candidate B).** `makeMetadata()`, `extractMetadata()`, `threadOnto()`, and `captureFrom()` consolidate all cid/rid/rcid magic indices into one file. `sendMessage` threading simplified from 20+ lines to 10. See `src/services/conversation-threading.ts`.
+- **Post-call seam consolidation (Candidate E).** `persistRefreshedCookies` now delegates to `cookieJar.upsert()` as the single write path for SDK-refreshed cookies.
 - Phase 0 v2 regression net: 10 tests (0a–0j) across 8 files. Documented at `docs/phase-0/phase-0-v2-design.md`. Bug history ledger at `docs/phantom-bug-synthesis.md`.
 - `tests/services/cookie-jar-repro.test.ts`: deterministic repro harness for the 4-cookie degradation symptom at the `GeminiClientService`/SDK seam.
-- Architecture review v3 at `docs/phase-0/architecture-review-auth-2026-08-08-v3.html` identifies two deepening candidates for post-v2.7.0 work (state machine explicitness, cookie jar unification).
+- Architecture review v5 at `docs/phase-0/architecture-review-auth-2026-08-09-v5.html` identifies deepening candidates (A+B+C+E) — all four now implemented.
 
 ---
 

docs/alternate-plan-simplify.md

@@ -0,0 +1,101 @@
+# Alternate Plan — Simplify Toward v2.4.0 (Remove RotateCookies + Phantom Detection)
+
+**Date:** 2026-08-09
+**Status:** Exploration. Not the active plan for the current branch.
+**Active plan:** Option 2 (explicit state machine + CookieJar unification) on branch `fix/rotate-cookies-401-session-kill`.
+
+---
+
+## Premise
+
+The post-v2.4.0 auth architecture added three layers — L1 RotateCookies, phantom-auth detection, and targeted L2 recovery — to fix a symptom (`listChats` returns empty) whose **definitive root cause** was the `CookieMonitor` capture trim bug, fixed in commit `6bc51f6` (the cookie-jar-integrity fix).
+
+The symptom and the fix:
+- **Symptom:** `listChats` returned 0 chats after ~2h idle, despite `models()` probe passing and cookies appearing locally valid.
+- **Apparent cause:** Server-side session degradation (PSIDTS rotation, companion cookie expiry) invisible to local freshness checks.
+- **Real root cause:** `CookieMonitor.poll` filtered the browser jar to `REQUIRED_COOKIES` (PSID/PSIDTS only) before passing to persistence. Every capture path saved only 4 cookies. `listChats` requires companion cookies (SID, HSID, SSID, APISID, SAPISID, etc.), which were missing.
+- **Real fix:** `6bc51f6` — separate gating predicate from payload. Keep REQUIRED_COOKIES as the login gate; pass the **full** browser jar as the payload.
+
+The RotateCookies + phantom detection + targeted L2 layers were built on the false premise that the jar was complete and the session was degrading. They were each individually correct but collectively unnecessary once the jar is captured intact.
+
+## Evidence
+
+1. **v2.4.0 worked with 12-day-old sessions** (DHBGAMING2 Linux, sessions from July 29, 12 days idle, still lists 14 conversations). v2.4.0 had no RotateCookies, no probe, no phantom detection. Its `ensureAuthenticated` was 34 lines, sync: check `hasValidCookies()` → return cookies. Done.
+2. **The RotateCookies 401 false-positive** (latest ledger entry in `docs/phantom-bug-synthesis.md`): `accounts.google.com/RotateCookies` returning 401 does NOT mean the Gemini API session is dead. v2.7.0 killed sessions after ~5h idle; v2.4.0 didn't kill them at all.
+3. **Every phantom-auth fix addressed detection/rotation, not the data.** The jar flowing through all the layers was already degraded by the capture trim. The probe, rotation, and phantom detection were operating on 4 cookies and couldn't fix what they couldn't see.
+
+## The Plan
+
+### Remove
+
+| Layer | File | Why |
+|-------|------|-----|
+| L1 RotateCookies from hot path | `profile-auth-manager.ts:114-140` | `accounts.google.com` endpoint has different session validation than Gemini API. False-positive 401 kills valid sessions. With full-jar capture, SDK self-rotation (`persistRefreshedCookies`) is sufficient. |
+| `detectPhantomAuth` | `profile-auth-manager.ts:190-199` | Phantom-auth was the capture bug. With full jars, `listChats` shouldn't return empty on valid sessions. |
+| Targeted L2 | `auth-service.ts:310-328` | Phantom-auth recovery not needed when jars are complete. |
+| `RotateCookiesResult` type + `rotateCookies` dep | `profile-auth-manager.ts` deps interface | No longer called from ensureAuthenticated. |
+| `rotateCookies` adapter on `AuthService` | `auth-service.ts:216-233` | If RotateCookies is removed from hot path. |
+
+### Keep
+
+| Layer | Why |
+|-------|-----|
+| `models()` probe | Cheap, definitive live/dead signal. One round-trip to Google. |
+| `silentRefresh` (full mode) | Headless browser re-auth when session is genuinely dead. |
+| `persistRefreshedCookies` | SDK self-rotation must be persisted between CLI runs. |
+| Full-jar capture (`6bc51f6`) | The definitive root cause fix. Never regress. |
+
+### Simplified `ensureAuthenticated`
+
+The result would look like:
+
+```
+ensureAuthenticated(name):
+  1. Check hasValidCookies → no → throw (or try autoExtendSession → silentRefresh → throw if fail)
+  2. Probe server with models() → stale → silentRefresh → throw if fail
+  3. Return cookies
+```
+
+~20 lines. Sync where possible. No rotation. No phantom detection. No targeted L2. The RotateCookies endpoint could still be called by a `gemiterm watch` background process, but it would not be in the critical path of every CLI command.
+
+## Risks
+
+1. **Server-side PSIDTS rotation** — without L1 RotateCookies, `__Secure-1PSIDTS` will only rotate via SDK self-rotation (which requires an API call). If the user goes days without using gemiterm, PSIDTS may expire server-side while PSID is still valid. The `models()` probe would still catch this and trigger `silentRefresh`. This is the same behavior as v2.4.0, which worked in practice.
+2. **Companion cookie expiry** — SID/HSID/SSID/etc. are session-scoped. If they expire server-side, `listChats` will return empty even with full jars. v2.4.0 didn't handle this either. The `models()` probe + `silentRefresh` ladder would surface it as a dead session → re-auth.
+3. **No proactive rotation** — without L1 RotateCookies, there's no mechanism to keep PSIDTS warm between CLI invocations. A `gemiterm watch` background process could fill this gap for automation users.
+
+## Migration Path
+
+1. Branch `simplify/remove-roteta-phantom` off main@v2.7.0.
+2. Remove `rotateCookies` call from `ensureAuthenticated`.
+3. Remove `detectPhantomAuth`.
+4. Remove targeted L2 from `silentRefresh`.
+5. Simplify `RotateCookiesResult` type or remove the `sessionInvalid` field.
+6. Remove `rotateCookies` dep from `ProfileAuthManagerDeps`.
+7. Update tests — the 10-test Phase 0 v2 regression net must stay GREEN.
+8. Live-verify with a fresh login, wait ~2h, run `gemiterm list` — should still work.
+
+## Comparison with Active Plan (Option 2)
+
+| Aspect | Option 1 (Simplify) | Option 2 (Explicit state machine) |
+|--------|---------------------|----------------------------------|
+| Lines of code | ~100 removed | ~200 added (new modules) |
+| Complexity | Decreased | Same, reorganized |
+| Bug surface | Smaller | Same, typed |
+| Defense-in-depth | Probe + silentRefresh only | All current layers, explicit |
+| Risk | PSIDTS may expire between uses; `models()` probe catches it | RotateCookies 401 false-positives (already fixed); transition bugs (already surfaced) |
+| Test changes | Remove tests for removed paths | Add tests for new modules |
+| Migration effort | ~1 day | ~3-5 days |
+
+## Decision
+
+This is the **alternate plan**, documented for future consideration. The active plan (Option 2: explicit state machine + CookieJar unification) is being implemented on `fix/rotate-cookies-401-session-kill`. If Option 2 proves too complex or introduces new regressions, this plan is the fallback.
+
+## Related
+
+- `docs/phantom-bug-synthesis.md` — write-once bug ledger
+- `docs/phase-0/phase-0-v2-design.md` — regression net design
+- Commit `6bc51f6` — the cookie-jar-integrity fix (definitive root cause)
+- Commit `c4870de` — RotateCookies 401 session-kill fix
+- OpenSpec change `cookie-jar-integrity` — the capture fix
+- `C:\Users\diego\AppData\Local\Temp\architecture-review-auth-2026-08-08-v3.html` — v3 architecture review

docs/phantom-bug-synthesis.md

@@ -496,3 +496,39 @@ Committed `b5dc3de`. Test baseline unchanged (954 pass / 1 skip / 0 fail). Typec
 - §"2026-08-08 — dhb-worker session expired after ~2 hours" — the source data point for the ~1h15m floor.
 - §"2026-08-08 — profile-routing lambda drops profile argument" — the lambda fix being re-verified after the next idle cycle.
 
+## 2026-08-09 — RotateCookies 401 pre-emptively kills sessions that the Gemini API still accepts
+
+**Discovered by:** Diego, cross-version comparison. v2.4.0 on Linux (DHBGAMING2, sessions from July 29, 12 days old) still lists chats fine. v2.7.0 on Windows kills sessions after ~5h idle with `AuthenticationError("Session for profile 'dhb-worker' is no longer valid (server rejected RotateCookies)")`.
+
+**Symptom:**
+- `gemiterm list` on v2.7.0 after ~5h idle: first call targeted-L2 recovers but returns "No conversations found"; second call gets `AuthenticationError` because RotateCookies returns 401.
+- `gemiterm list` on v2.4.0 after 12 days idle: returns 14 conversations, no errors.
+- The core PSID cookie expires Sep 2027 on both machines. It is still valid. Google's Gemini API accepts it. RotateCookies rejects it.
+
+**Root cause:** `profile-auth-manager.ts:121-129` treats RotateCookies 401 as definitive proof of Gemini API session death:
+
+```typescript
+if (rotation.sessionInvalid) {
+  throw new AuthenticationError(
+    `Session for profile '${name}' is no longer valid (server rejected RotateCookies). Run 'gemiterm login'...`,
+  );
+}
+```
+
+In v2.4.0, `ensureAuthenticated` had none of this — it checked `hasValidCookies()` (7-day local freshness) and returned cookies immediately. No RotateCookies call, no probe, no phantom detection. The gemini-web-sdk used the cookies directly, and Google's Gemini API accepted them.
+
+The design flaw: **RotateCookies is an `accounts.google.com` endpoint, not a Gemini API endpoint.** Its session validation behavior differs from the Gemini API endpoints (`models`, `listChats`, `readChat`). A 401 from RotateCookies means Google Accounts won't rotate the PSIDTS token — it does NOT mean the Gemini API will reject the PSID cookie. These are separate services with separate session policies.
+
+The second call in the test session got a 401 because the targeted L2 refresh on the first call partially updated the jar (PSIDTS-family cookies refreshed) while companion cookies (SID/HSID/SSID/etc.) from the expired session remained — creating an inconsistent cookie envelope that RotateCookies rejected. But the Gemini API may have still accepted that envelope for `listChats`/`models`.
+
+**Fix:** Remove the `sessionInvalid` throw. Treat RotateCookies 401/403 the same as "declined" — rotation simply didn't happen, carry on. Run phantom detection (as we do for "declined" already) to attempt targeted L2 recovery. Only throw if targeted L2 also fails. This defers session-validity judgment to the actual Gemini API endpoints rather than a secondary Google Accounts endpoint.
+
+The change is in `profile-auth-manager.ts:121-129` — replace the existing `sessionInvalid` throw block with a fallthrough that mirrors the `rotation.attempted` path (phantom detection → targeted L2 → throw on failure).
+
+**Verified:** TBD after implementation. Test baseline expected unchanged (954/1/0).
+
+**Related ledger entries:**
+- §"2026-08-06 — The recovery-ladder recurrence" (Gap B: `sessionInvalid` surface path) — the original design that added the `sessionInvalid` flag. This entry argues the 401 throw was the wrong fix for Gap B.
+- §"The 3-release arc" — traces how RotateCookies detection was added across v2.6.0–v2.6.2.
+- §"2026-08-06 — Session 3" (L2 removal) — the earlier removal of the L2 cookie-corruption path; this is the companion fix for RotateCookies 401.
+

openspec/changes/archive/2026-08-09-fix-rotate-cookies-401-session-kill/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-08-09

openspec/changes/archive/2026-08-09-fix-rotate-cookies-401-session-kill/design.md

@@ -0,0 +1,56 @@
+## Context
+
+v2.7.0's `ensureAuthenticated` at `profile-auth-manager.ts:121-129` throws `AuthenticationError` when `rotateCookies()` returns `sessionInvalid: true` (i.e., RotateCookies POST returned 401/403). This was added in `4dfe13c` (Gap B fix) under the assumption that RotateCookies 401 means the Gemini API session is dead.
+
+Cross-version comparison disproves this: v2.4.0 (which never calls RotateCookies) works with 12-day-old sessions. The RotateCookies endpoint (`accounts.google.com`) has different session validation behavior than the Gemini API endpoints (`models`, `listChats`, `readChat`). A 401 from RotateCookies means the token rotation was rejected — not that the Gemini API will reject the PSID cookie.
+
+The current code path:
+1. Probe (`models()`) succeeds → session marked "valid"
+2. `rotateCookies()` returns 401 → `sessionInvalid: true` → throw `AuthenticationError`
+3. Session is killed before the Gemini API ever gets a chance to respond
+
+## Goals / Non-Goals
+
+**Goals:**
+- Remove the RotateCookies 401 → `AuthenticationError` throw
+- Merge `sessionInvalid` into the existing `rotation.attempted` branch so phantom detection + targeted L2 recovery can fire
+- Fall through to phantom detection when RotateCookies 401/403 occurs
+
+**Non-Goals:**
+- No change to the RotateCookies endpoint behavior or cookie-rotation.ts
+- No change to the probe cache, L1 throttle, or silentRefresh mechanics
+- No change to how targeted L2 merge works
+
+## Decisions
+
+**D1: Merge `sessionInvalid` into `rotation.attempted` branch**
+
+The existing condition chain is:
+```
+if (rotation.rotated) { ... }
+else if (rotation.attempted) { ... phantom detection ... }
+else { /* throttled/skipped */ }
+```
+
+`sessionInvalid` sets `{ rotated: false, attempted: false }`, so it currently falls into the `else` (throttled/skipped) after the throw is removed.
+
+Change to:
+```
+if (rotation.rotated) { ... }
+else if (rotation.attempted || rotation.sessionInvalid) { ... phantom detection ... }
+else { /* throttled/skipped */ }
+```
+
+This gives RotateCookies 401/403 the same recovery path as "declined" (200 with no fresh PSIDTS): detect phantom → attempt targeted L2 → if targeted L2 fails, throw. The phantom detection step verifies whether the session is actually usable (listChats returns results) or truly dead.
+
+**Rationale:** RotateCookies 401 can happen because (a) session is genuinely dead, (b) companion cookies expired while PSID is still valid, or (c) RotateCookies endpoint behavior differs from Gemini API. Cases (b) and (c) should not kill the session. Case (a) will surface through phantom detection failing → `AuthenticationError`.
+
+**Alternative considered:** Log and skip entirely (no phantom detection). Rejected — if the session IS truly dead, we want targeted L2 to attempt recovery before giving up.
+
+## Risks / Trade-offs
+
+- **[Risk] Genuinely dead sessions take longer to surface.** Instead of immediate throw on RotateCookies 401, we run phantom detection (listChats call) + targeted L2 (browser launch). This adds ~5-10 seconds to the error path.
+  - **Mitigation:** Dead sessions are rare; the common case (session still works via Gemini API) now succeeds without any user intervention.
+
+- **[Risk] Targeted L2 on a RotateCookies-401 session may corrupt the jar.** If the browser auto-signs-in with the same cookies (phantom = frontend-valid), targeted L2 will update PSIDTS cookies while companion cookies may still be expired.
+  - **Mitigation:** This is the same risk as the existing "declined" phantom path. The targeted L2 update is scoped to `COOKIE_NAMES_OF_INTEREST` only. If companion cookies are the problem, targeted L2 won't fix it and will throw `AuthenticationError` → user gets re-auth prompt. This is correct behavior — targeted L2 can't manufacture missing companion cookies.