[rid/store] Fail ISA search instead of silently truncating the result list - #1640
Open
hdimer wants to merge 1 commit into
Open
[rid/store] Fail ISA search instead of silently truncating the result list#1640hdimer wants to merge 1 commit into
hdimer wants to merge 1 commit into
Conversation
… list SearchISAs capped its result list to dssmodels.MaxResultLimit and returned it with no error, so a client asking about a dense area got a non-exhaustive answer and no way to know it. Per the decision recorded in interuss#1120, return BadRequest (400) when more than MaxResultLimit areas match, so the client can narrow its query instead. The two backends also disagreed on the boundary: memstore appended before checking and returned MaxResultLimit+1 items, while sqlstore's LIMIT returned MaxResultLimit. Both now return up to MaxResultLimit and error beyond that. The RID and SCD subscription, operational intent and constraint searches truncate the same way; those are left for follow-up PRs.
|
|
hdimer
marked this pull request as ready for review
August 17, 2026 16:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the RID identification service area case of #1120.
SearchISAscapped its result list todssmodels.MaxResultLimitand returned it witherr == nil, so a client querying a dense area received a non-exhaustive list with no indication that anything had been dropped. Both backends now returndsserr.BadRequestonce more thanMaxResultLimitareas match, which the v1 and v2 ISA handlers already map to a 400 with the message, so no handler change was needed.This follows the approach @barroco recorded from the 2024-09-17 contributors sync call in #1120 (comment): fail with a 400 and ask clients to scope their request more tightly, rather than paginate.
What changed
pkg/rid/store/memstore/identification_service_area.go— thebreak(and its// This mimics sqlstore behaviour, but it's not very good.TODO) becomes aBadRequest.pkg/rid/store/sqlstore/identification_service_area.go—LIMIT MaxResultLimit + 1, then the sameBadRequestwhen that extra row comes back. Selecting one row past the limit is what lets an over-limit result set be distinguished from one that exactly fills it.MaxResultLimitmatches still return the full list with no error.Incidentally this removes a real divergence between the two backends.
memstoreappended before checkinglen(out) > MaxResultLimit, so it returned 10001 items wheresqlstore'sLIMIT 10000returned 10000; I measured both before changing anything. The old TODO claimed memstore mimicked sqlstore, and it didn't quite. Both now agree.ListExpiredISAsis deliberately left alone: it is the operator-runevictsweep, where processing at mostMaxResultLimitentries per run and picking up the rest next time is the desired behaviour, not a lie to a client.One thing I would like your call on
App.SearchISAshas six call sites and only two are the search endpoint. The other four are subscription create/update (pkg/rid/server/v{1,2}/subscription_handler.go), which call it to populateservice_areasin the response — and the subscription is already committed by then. So in an area with more thanMaxResultLimitISAs,PUT/PATCHsubscription now returns 400 after the write, and a client that retries the same ID gets 409. Previously that branch was unreachable, sinceSearchISAscould only fail there on empty cells.That truncated
service_areaslist is arguably the same bug as #1120 — a client bootstrapping its notification state gets a silently incomplete picture — so failing seems consistent with the recorded decision. But the write-then-400 ordering is not great. Moving theSearchISAscall above the insert/update at those four sites would fix it cleanly (inserting a subscription cannot change which ISAs match; different tables), and I am happy to do that here or in a follow-up. I did not want to touch four handlers uninvited on a PR this size. The release note covers the behaviour either way.Testing
TestStoreSearchISAsResultLimitin the memstore pins both halves of the boundary: exactlyMaxResultLimitmatches succeed with a full list, one more errors withBadRequest. I ran it against the unpatched store first and confirmed it fails on the over-limit assertion for the right reason. It also fails if>is loosened to>=, if the truncated list is returned alongside the error, or if the error is raised without theBadRequestcode (which would surface as a 500 rather than a 400).go test ./pkg/... ./cmds/...green,gofmt -s -l .clean,golangci-lint run(v2.12.2, per the Makefile) reports 0 issues.make test-go-units-crdblocally, so the sqlstore half is not covered by a test I executed — the existingTestStoreSearchISAsin that package will confirm in CI that the query still works with the new limit, but nothing reaches the new branch. I left it that way on purpose rather than adding a crdb test that inserts 10001 rows into the shared CI suite for logic already pinned in memstore. Say the word and I will add one.Follow-ups (not in this PR)
The identical pattern is still in
pkg/rid/store/{memstore,sqlstore}/subscriptions.goandpkg/scd/store/{memstore,sqlstore}/{subscriptions,operational_intents,constraints}.go—memstore/subscriptions.goeven carries the same TODO comment. Worth noting for whoever picks those up: the SCD sites break on>=, so they need the samelimit + 1treatment rather than a copy of the predicate from here. Happy to take them one at a time if this shape is what you want.