Skip to content

Document flaky-tests API conventions and pagination#57

Open
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/api-conventions
Open

Document flaky-tests API conventions and pagination#57
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/api-conventions

Conversation

@samgutentag
Copy link
Copy Markdown
Member

Summary

  • Adds an "API conventions" section to flaky-tests/reference/api-reference.mdx covering behavior that applies across endpoints
  • Documents page_token pagination with a working curl example and the "first call → omit, subsequent → pass next_page_token" rule
  • Covers rate-limit expectations, webhook-to-API immediacy, the support-only Bundle Upload ID, the auto-analysis monthly cap (Beta), and that the UI's GraphQL endpoint is private
  • No nav change — the conventions sit above the existing endpoint card index on the same page

Why

Sourced from customer feedback mining (cluster api-rate-limits-and-pagination, verdict partial, 7 pairs across 6 customers). The API reference is a card index; conventions like page_token pagination and rate limits are documented in OpenAPI specs but never in narrative form, so customers hit recurring issues (500 on misused page_token, "missing" tests that are actually on page 2, no surfaced answer to "what's the rate-limit ceiling?").

Items flagged for review

  • Auto-analysis monthly cap: cluster brief specifies "100/month default." I deliberately did not commit to a specific number in the docs because I couldn't find it in any other docs2 page or in the OpenAPI spec — confirm the exact value with the agents team and either inline the number or keep the current "default monthly cap" phrasing.
  • GraphQL phrasing: confirm "private GraphQL endpoint... no compatibility guarantee, and may change without notice" is the framing the API team wants, vs. softer language.
  • Bundle Upload ID: confirm there's still an open internal ticket to surface it (so leaving the doc as "support-only today" remains accurate), and whether we want to name trunk-analytics-cli upload specifically as the source or keep it generic.
  • 504/timeout history: deliberately omitted. Phil flagged the get-test-details timeout issue in March and said a fix was rolling out — confirm shipped (otherwise this PR should not be merged without addressing) or leave omitted if the fix is in production and there's nothing actionable for readers.
  • Rate-limit phrasing: "no externally published SLO, but the practical ceiling is high enough for reasonable polling and dashboard use" mirrors Phil's Slack response — confirm this framing is OK to publish vs. a more conservative "contact support for rate-limit guidance."
  • Pagination 500 behavior: I wrote "Passing a value that didn't come from a prior response returns a 500." Confirm this is the actual error mode and that 500 is the documented response code (vs. 400).

Customer signal

…ation, Bundle Upload ID

Expand the Flaky Tests API reference index page with an "API conventions"
section covering behavior that applies across endpoints. Sourced from
recurring customer questions: page_token misuse returning 500s, "missing"
tests that were actually on later pages, rate-limit expectations for
polling automation, and webhook-to-API immediacy.

Also documents three things that weren't surfaced anywhere in narrative
form: Bundle Upload IDs are support-only, auto-analysis has a default
monthly cap during Beta, and the UI's GraphQL endpoint is private.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@samgutentag samgutentag added the needs review PR sourced from customer-feedback-mining; needs human scrutiny for accuracy before merge label May 20, 2026
@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented May 20, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
trunk 🟢 Ready View Preview May 20, 2026, 11:55 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs review PR sourced from customer-feedback-mining; needs human scrutiny for accuracy before merge

Development

Successfully merging this pull request may close these issues.

1 participant