feat(web): add MCP and API key usage tracking to analytics#948
Closed
feat(web): add MCP and API key usage tracking to analytics#948
Conversation
Move audit event creation from client-side to service functions for search, repos, file source, and file tree endpoints. Add source metadata to distinguish MCP requests from other API calls. Extend analytics SQL to include new actions and display MCP request and API request counts on the analytics dashboard. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review infoConfiguration used: Organization UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (10)
WalkthroughMoves audit event emission from client to server-side service functions, adds a request Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Server
participant AuditService
participant DB
participant AnalyticsUI
Client->>Server: request (includes X-Sourcebot-Client-Source header)
Server->>AuditService: emit audit event (actor, action, target, metadata.source)
AuditService->>DB: insert audit record
Note right of DB: DB stores record with source and timestamp
Server->>Client: respond with requested data
AnalyticsUI->>Server: query analytics (rows + retention + oldestRecordDate)
Server->>DB: aggregate audits by action + metadata.source
DB-->>Server: aggregated rows (mcp_requests, api_requests, etc.)
Server-->>AnalyticsUI: analytics response (rows, retentionDays, oldestRecordDate)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
packages/web/src/features/search/searchApi.ts (1)
31-40: Consider extracting duplicated audit logic.The audit blocks in
search()andstreamSearch()are identical. This duplication exists across multiple files in this PR. Consider extracting a helper function to reduce repetition and centralize the audit pattern.♻️ Suggested helper extraction (local to this file or shared)
// Could be placed in a shared audit utilities file const auditCodeSearch = async (user: UserWithAccounts, org: { id: number }) => { const source = (await headers()).get('X-Sourcebot-Client-Source') ?? undefined; getAuditService().createAudit({ action: 'user.performed_code_search', actor: { id: user.id, type: 'user' }, target: { id: org.id.toString(), type: 'org' }, orgId: org.id, metadata: { source }, }).catch(() => {}); };Then in both functions:
withOptionalAuthV2(async ({ prisma, user, org }) => { if (user) { - const source = (await headers()).get('X-Sourcebot-Client-Source') ?? undefined; - getAuditService().createAudit({ - action: 'user.performed_code_search', - actor: { id: user.id, type: 'user' }, - target: { id: org.id.toString(), type: 'org' }, - orgId: org.id, - metadata: { source }, - }).catch(() => {}); + await auditCodeSearch(user, org); }Also applies to: 62-71
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/features/search/searchApi.ts` around lines 31 - 40, Extract the duplicated audit logic into a helper (e.g., auditCodeSearch) and replace the repeated blocks in search() and streamSearch() with a single call to that helper; the helper should capture the source via (await headers()).get('X-Sourcebot-Client-Source') ?? undefined and call getAuditService().createAudit({ action: 'user.performed_code_search', actor: { id: user.id, type: 'user' }, target: { id: org.id.toString(), type: 'org' }, orgId: org.id, metadata: { source } }).catch(() => {}), then invoke auditCodeSearch(user, org) from both search() and streamSearch() to remove duplication.packages/web/src/ee/features/analytics/types.ts (1)
9-11: Tighten counter field validation to integer, non-negative values.These are count metrics; enforcing
.int().nonnegative()avoids silently accepting invalid payloads.Suggested schema hardening
- mcp_requests: z.number(), - api_requests: z.number(), - active_users: z.number(), + mcp_requests: z.number().int().nonnegative(), + api_requests: z.number().int().nonnegative(), + active_users: z.number().int().nonnegative(),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/ee/features/analytics/types.ts` around lines 9 - 11, The count fields mcp_requests, api_requests, and active_users in the analytics schema are plain z.number() and should be restricted to integer, non-negative values; update their validators to use .int().nonnegative() on each of those fields (locate the schema definition in types.ts where mcp_requests, api_requests, and active_users are declared) so the schema enforces integer non-negative counters.packages/web/src/ee/features/analytics/analyticsContent.tsx (1)
178-178: Avoid hardcoded skeleton count drift.Line 178 hardcodes
6; consider deriving from a shared chart-count constant to keep loading UI in sync when charts are added/removed.Minimal refactor
+const ANALYTICS_CHART_COUNT = 6 + function LoadingSkeleton() { return ( @@ - {[1, 2, 3, 4, 5, 6].map((chartIndex) => ( + {Array.from({ length: ANALYTICS_CHART_COUNT }, (_, i) => i + 1).map((chartIndex) => (🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/ee/features/analytics/analyticsContent.tsx` at line 178, Replace the hardcoded array [1,2,3,4,5,6] used to render skeletons with a single source-of-truth chart count constant (e.g., CHART_COUNT) exported from the analytics/chart configuration or the component that defines the list of charts; update the mapping to iterate based on that CHART_COUNT (e.g., Array.from({ length: CHART_COUNT })) so the skeleton count automatically stays in sync whenever charts are added/removed (ensure you add or import CHART_COUNT and replace occurrences in analyticsContent.tsx where the hardcoded six is used).packages/web/src/ee/features/analytics/actions.ts (1)
87-87: Use an explicit API-source allowlist forapi_requests.Line 87 currently treats any non-null, non-
mcpsource as API. That can overcount if new source labels are introduced later (e.g., non-API internal clients). Prefer explicit allowed API source values.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/ee/features/analytics/actions.ts` at line 87, The COUNT(*) FILTER that defines api_requests currently treats any non-null/non-'mcp' metadata->>'source' as API; change this to an explicit allowlist check against known API source values (e.g. metadata->>'source' IN ( ... )) so only intended sources count as api_requests, update the SQL expression that produces api_requests accordingly, and if the allowed sources are used elsewhere consider extracting them to a shared constant or config and adjust any tests that rely on the previous behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CHANGELOG.md`:
- Line 11: The changelog entry mixes tenses—change the phrase "Move audit events
from client-side to service functions to capture all API calls (web UI, MCP, and
non-MCP)" to past tense (e.g., "Moved audit events from client-side to service
functions to capture all API calls (web UI, MCP, and non-MCP)") so the entire
bullet reads consistently in past tense with the rest of the entry.
---
Nitpick comments:
In `@packages/web/src/ee/features/analytics/actions.ts`:
- Line 87: The COUNT(*) FILTER that defines api_requests currently treats any
non-null/non-'mcp' metadata->>'source' as API; change this to an explicit
allowlist check against known API source values (e.g. metadata->>'source' IN (
... )) so only intended sources count as api_requests, update the SQL expression
that produces api_requests accordingly, and if the allowed sources are used
elsewhere consider extracting them to a shared constant or config and adjust any
tests that rely on the previous behavior.
In `@packages/web/src/ee/features/analytics/analyticsContent.tsx`:
- Line 178: Replace the hardcoded array [1,2,3,4,5,6] used to render skeletons
with a single source-of-truth chart count constant (e.g., CHART_COUNT) exported
from the analytics/chart configuration or the component that defines the list of
charts; update the mapping to iterate based on that CHART_COUNT (e.g.,
Array.from({ length: CHART_COUNT })) so the skeleton count automatically stays
in sync whenever charts are added/removed (ensure you add or import CHART_COUNT
and replace occurrences in analyticsContent.tsx where the hardcoded six is
used).
In `@packages/web/src/ee/features/analytics/types.ts`:
- Around line 9-11: The count fields mcp_requests, api_requests, and
active_users in the analytics schema are plain z.number() and should be
restricted to integer, non-negative values; update their validators to use
.int().nonnegative() on each of those fields (locate the schema definition in
types.ts where mcp_requests, api_requests, and active_users are declared) so the
schema enforces integer non-negative counters.
In `@packages/web/src/features/search/searchApi.ts`:
- Around line 31-40: Extract the duplicated audit logic into a helper (e.g.,
auditCodeSearch) and replace the repeated blocks in search() and streamSearch()
with a single call to that helper; the helper should capture the source via
(await headers()).get('X-Sourcebot-Client-Source') ?? undefined and call
getAuditService().createAudit({ action: 'user.performed_code_search', actor: {
id: user.id, type: 'user' }, target: { id: org.id.toString(), type: 'org' },
orgId: org.id, metadata: { source } }).catch(() => {}), then invoke
auditCodeSearch(user, org) from both search() and streamSearch() to remove
duplication.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
CHANGELOG.mdpackages/web/src/app/[domain]/components/searchBar/searchBar.tsxpackages/web/src/app/api/(server)/repos/listReposApi.tspackages/web/src/ee/features/analytics/actions.tspackages/web/src/ee/features/analytics/analyticsContent.tsxpackages/web/src/ee/features/analytics/types.tspackages/web/src/ee/features/audit/types.tspackages/web/src/features/git/getFileSourceApi.tspackages/web/src/features/git/getTreeApi.tspackages/web/src/features/search/searchApi.ts
💤 Files with no reviewable changes (1)
- packages/web/src/app/[domain]/components/searchBar/searchBar.tsx
| ## [Unreleased] | ||
|
|
||
| ### Added | ||
| - Added MCP and API key usage tracking to analytics dashboard. Move audit events from client-side to service functions to capture all API calls (web UI, MCP, and non-MCP). Display MCP requests and API requests on separate charts. [#948](https://github.com/sourcebot-dev/sourcebot/pull/948) |
Contributor
There was a problem hiding this comment.
Use consistent past tense in the changelog sentence.
Line 11 says “Move audit events…” while the rest of the entry is in past tense. This reads awkwardly in release notes.
✏️ Proposed wording tweak
-- Added MCP and API key usage tracking to analytics dashboard. Move audit events from client-side to service functions to capture all API calls (web UI, MCP, and non-MCP). Display MCP requests and API requests on separate charts. [`#948`](https://github.com/sourcebot-dev/sourcebot/pull/948)
+- Added MCP and API key usage tracking to the analytics dashboard. Moved audit events from client-side to service functions to capture all API calls (web UI, MCP, and non-MCP). Displayed MCP request and API request counts on separate charts. [`#948`](https://github.com/sourcebot-dev/sourcebot/pull/948)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| - Added MCP and API key usage tracking to analytics dashboard. Move audit events from client-side to service functions to capture all API calls (web UI, MCP, and non-MCP). Display MCP requests and API requests on separate charts. [#948](https://github.com/sourcebot-dev/sourcebot/pull/948) | |
| - Added MCP and API key usage tracking to the analytics dashboard. Moved audit events from client-side to service functions to capture all API calls (web UI, MCP, and non-MCP). Displayed MCP request and API request counts on separate charts. [`#948`](https://github.com/sourcebot-dev/sourcebot/pull/948) |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@CHANGELOG.md` at line 11, The changelog entry mixes tenses—change the phrase
"Move audit events from client-side to service functions to capture all API
calls (web UI, MCP, and non-MCP)" to past tense (e.g., "Moved audit events from
client-side to service functions to capture all API calls (web UI, MCP, and
non-MCP)") so the entire bullet reads consistently in past tense with the rest
of the entry.
- Add SOURCEBOT_EE_AUDIT_RETENTION_DAYS env var (default 180) and AuditLogPruner background job that prunes old audit records daily in batches - Surface retention period and oldest record date in analytics page header - Update audit action types table in docs (remove 4 stale, add 11 missing) - Add audit log storage section to sizing guide with enterprise callout and storage estimates - Update mock data script with mixed-usage user profiles and new audit actions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
msukkari
added a commit
that referenced
this pull request
Feb 27, 2026
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Analytics: MCP & API usage tracking
sourcemetadata to audit events to distinguish MCP requests from other API callsAudit log retention policy
SOURCEBOT_EE_AUDIT_RETENTION_DAYSenvironment variable (default: 180) to configure how long audit logs are keptAuditLogPrunerbackground job that runs daily and prunes audit records older than the configured retention period, deleting in batches of 10K to avoid long-running transactions0to disable pruning and retain logs indefinitelyAnalytics UI: retention info
{ rows, retentionDays, oldestRecordDate }) instead of returning a flat arrayDocumentation updates
chattotargetTypeenum, addsourceto metadata schemaSOURCEBOT_EE_AUDIT_RETENTION_DAYSentryMock data script
inject-audit-data.tswith 5 mixed-usage user profiles (web-only, web+MCP hybrid, MCP-heavy, API-only, web+API hybrid) with randomized weightssourcemetadata for MCP/API usersScreenshots
Retention policy and oldest audit log date displayed in analytics header:
Test plan
source='mcp'and appear in MCP Requests chartSOURCEBOT_EE_AUDIT_RETENTION_DAYS=0disables pruningLinear: SOU-579
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
Documentation