feat(usage): aggregated usage + cost analytics endpoint (per provider/harness/model; admin all-users) - #4
Draft
marko-k0 wants to merge 2 commits into
Draft
feat(usage): aggregated usage + cost analytics endpoint (per provider/harness/model; admin all-users)#4marko-k0 wants to merge 2 commits into
marko-k0 wants to merge 2 commits into
Conversation
marko-k0
pushed a commit
that referenced
this pull request
Jul 24, 2026
…3061) * feat(projects): first-class projects in the web sidebar Wires the web app to the first-class projects entity (omnigent-ai#2765/omnigent-ai#3053), keeping the legacy omni_project label path working via dual-read so no migration is forced. Folders are keyed by name (the union key that merges a first-class project and a like-named label-project into one folder), carrying the first-class id when one exists. Backend - GET /v1/sessions/projects now dual-reads: unions first-class projects (project_store.list — incl. empty, with id) and legacy label-projects (id=None), merged by name and sorted. Response shape list[str] → list[{id, name}]; still owner-scoped. openapi.json regenerated. Frontend - projectsApi.ts: typed /v1/projects CRUD client (list/create/rename/delete). - Hooks: useProjects → ProjectSummary[] ({id, name}); new useCreateProject, useRenameProject; reworked useDeleteProject (archive + unfile every member, then delete the container). Filing/moving files via project_id, resolving the picked name to an id and creating the first-class row on demand for a label-only folder; "" unfiles. Conversation.project_id added. - Sidebar: folders keyed by {id, name}, members matched by project_id OR the legacy label; always-visible Projects section with a "New project" (create-empty) control extracted to NewProjectButton.tsx; Rename dialog; delete threads id; a row's current-project dual-reads project_id→name so a pinned first-class member keeps its project flyout; "Remove from project" unfiles silently (a first-class project persists when emptied); empty folders read "No sessions". - NewChatDialog: composer files new sessions via project_id. Tests - projectsApi unit tests; reworked hook tests (resolve→file, create-on-demand, archive+unfile+delete); sidebar/composer suites updated; server union test; e2e_ui docstrings + fixtures updated for the project_id membership flow. Deferred (kept on the label path via dual-read): the new-session prefill state machine and the Settings archived-only project picker; retiring label reads is gated on the Phase 4 backfill. Co-authored-by: Isaac Signed-off-by: Serena Ruan <serena.rxy@gmail.com> * fix(projects): rename-dialog Enter, checked promote PATCH, typed projects schema Addresses the review on omnigent-ai#3061: - Rename-project dialog: wrap the body in a <form> so Enter submits natively (Radix Dialog doesn't provide one, and the prior manual key handler looked for the confirm button inside the <input> and never fired). - useRenameProject label-only promote: check res.ok on each re-file PATCH and throw on failure, so a 4xx/5xx no longer reports success with members left unfiled. - GET /v1/sessions/projects: return a typed SessionProjectSummary list instead of list[dict] + response_model=None, which produced an empty ("schema": {}) OpenAPI response and broke client generation. openapi.json regenerated. - Drop the stale test comment describing the removed last-session remove-confirm gate. Copilot #2 (recreate missing metadata row) and #4 (...->NotImplementedError in the abstract method) intentionally declined, consistent with prior rounds. Co-authored-by: Isaac Signed-off-by: Serena Ruan <serena.rxy@gmail.com> * fix(projects): keep dual-read membership coherent on move/rename; lift row lookup Addresses the second web-UI review round on omnigent-ai#3061: - moveConversationToProject now clears the legacy omni_project label in the same PATCH as it sets project_id. The sidebar groups a folder by project_id OR the label during the dual-read transition, so a stale label would keep a moved session in its old label-folder (and match two folders at once). project_id is the single source of truth after a move. - useRenameProject reconciles members for BOTH paths (first-class rename and label-only promote): sweep the folder's members via ?project=<oldName>, re-file each onto the target project_id, and clear the legacy label — so a first-class rename no longer strands label-matched members in an oldName folder. - resolveOrCreateProjectId tolerates the create-on-demand race: a concurrent move to the same new name can 409 on the second POST; re-list and use the winner's id instead of failing. - ConversationRow no longer calls useProjects() per row. A list-level id->name map is provided via context (ProjectNamesContext), so row renders are O(1) with no per-row query observer. Test PATCH-body assertions updated for the added labels field. Co-authored-by: Isaac Signed-off-by: Serena Ruan <serena.rxy@gmail.com> * fix(projects): preserve the original error when create-on-demand truly fails resolveOrCreateProjectId caught the create error to tolerate the 409 race (a concurrent move created the same name), but a genuine 500/network failure was indistinguishable and surfaced as a generic "Could not resolve or create" message. Re-list to disambiguate: if the row now exists a racer won — use it; otherwise rethrow the ORIGINAL error so the true cause isn't masked. Addresses a non-blocking note on omnigent-ai#3061. Co-authored-by: Isaac Signed-off-by: Serena Ruan <serena.rxy@gmail.com> * test(e2e-ui): stub /v1/sessions/projects with the {id,name} shape in prefill test The project-prefill e2e test stubbed GET /v1/sessions/projects with the old bare-string body, but this PR changed the endpoint to return SessionProjectSummary objects. The sidebar parsed no folder, so the project header never rendered and header.hover() timed out. Return the dual-read union shape ({id: None, name} for the label-only project the test seeds), matching the endpoint contract and the sibling sidebar tests. Co-authored-by: Isaac --------- Signed-off-by: Serena Ruan <serena.rxy@gmail.com>
marko-k0
force-pushed
the
feat/usage-cost-analytics
branch
from
July 28, 2026 15:56
1b9614e to
8d16b01
Compare
Adds GET /v1/usage/summary — rolls up token usage and USD spend across a caller's conversations (or, for admins, any user's or all users'), broken down by provider, harness, or model over today/7days/30days, with a per-day (per-hour for today) time series. - Sums each conversation's OWN per-node session_usage blob (not the subtree-summed load_session_usage, which would double-count parent + sub-agent). Every conversation kind is included as disjoint spend. - Owner scope via session_permissions LEVEL_OWNER; admin (users.is_admin) may target ?user=<id> or ?user=all, non-admins get 403; single-user mode rejects ?user with 400. - Provider derived from model id (vendor/model prefix; by_model walked so multi-model sessions split correctly). Harness from harness_override else the bound agent's canonicalized spec harness. - New store method list_usage_records(start_epoch, end_epoch, owner_user_id); UsageGroupEntry/UsageBucketEntry/UsageSummaryResponse schemas; router wired in app.py under /v1. Backend only; web UI (user panel + admin dashboard) is a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…shboard)
Adds the "Usage" settings section that renders GET /v1/usage/summary:
KPI tiles (total cost, input/output tokens), a spend-over-time bar chart,
and a per-group table (provider / harness / model) with a share-of-total
bar. Window (Today / 7d / 30d) and grouping map to the query params.
- Admins get a scope selector (own / all users / per-user drill-down);
the control is chrome-gated via the mode-agnostic useIsAdmin probe, and
the server enforces the `user` scope regardless. Per-user list degrades
gracefully to me/all when listUsers is unavailable (header/OIDC modes).
- Cost labelled as an estimate — subscription-auth harnesses aren't billed
per token.
- Dashboard is lazy-loaded so the settings shell stays lean.
New: lib/usageApi.ts (typed client + formatters), hooks/useUsageSummary.ts,
components/usage/{UsageDashboard,UsageChart}.tsx. Wires the section into
SettingsPage + settingsNav.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
marko-k0
force-pushed
the
feat/usage-cost-analytics
branch
from
July 28, 2026 16:35
8d16b01 to
fc23cc0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. Backend-only first cut of a usage & cost analytics API. Lets a developer see their own spend and an admin see spend across all users, broken down by provider, by harness, and by model over a time window.
Endpoint
GET /v1/usage/summary?period={today|7days|30days}&group_by={provider|harness|model}&user={<id>|all}→UsageSummaryResponse:total_input_tokens,total_output_tokens,total_cost_usd)groups[]—{group_key, input_tokens, output_tokens, total_cost_usd}(sorted by cost desc)buckets[]— per-day time series (per-hour fortoday)Requirement mapping
group_by(provider derived from the model id;by_modelwalked so multi-model sessions split correctly).users.is_adminmay pass?user=<id>or?user=all; non-admins get 403; single-user mode rejects?userwith 400.Design notes (for review)
session_usageblob, notload_session_usage(which subtree-sums and would fold a sub-agent's spend into every ancestor). All conversation kinds included as disjoint spend. Verified end-to-end on SQLite (parent + sub-agent + sibling → correct totals).session_permissionsLEVEL_OWNER(matchesget_session_owner_id).harness_overrideelse the bound agent's canonicalized spec harness (trimmed_resolve_harness), cached peragent_idper request.list_usage_records(start_epoch, end_epoch, owner_user_id)(time-window + owner-scoped SELECT, per-node blob parse) — query stays in the store layer.Scope / caveats
total_cost_usd. For subscription-auth harnesses this is a notional figure (subscriptions don't bill per token) — the UI should label it as an estimate.created_at(session_usageis cumulative-at-node, no per-turn timestamps); switch toupdated_atin the store SELECT if last-activity attribution is preferred.users.is_adminflag (like the session-list route), not the file-backedadmin_list. Injectadmin_listif parity with/auth/meis wanted.by_modelbreakdown bucket under"unknown"for provider/model grouping (spend never dropped).Validation
ruff checkon all touched files: pass.mypyon the new route + store interface: 0 errors (pre-existingsqlalchemy_store.pyfindings are outside the added block). Not yet run against the full project test suite — please run CI/lint locally.main; retarget to your build branch (e.g.staging) if that's where this should land.🤖 Generated with Claude Code