Repository navigation
feat(knowledge): add full-catalog selection, applicability checks and bounded reads - #534
amy-why-3459 wants to merge 2 commits into
Conversation
Add full evidence cards, explicit Agent selection, condition checks, source-bound assessments and reference expansion with atomic required resources. Agent-assisted-by: Codex (implementation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Retain fixed inputs, Agent selections, qualified read results and source-quote assessments. Report observed omissions and unverified task benefit. Agent-assisted-by: Codex (implementation and evaluation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base 1b826b5f9666a8b734000e1eb2990e582cdef876 and head 3f6d5d32ff42e5fbab1644988d28f02fd0d499c0 (first review on opened; pr/prior-reviews.md carries no formal review on this head, so there is no prior verdict to weigh either way). I read governing Issue #529, including its five 完成条件.
Only docs are visible in the seeded patch, and the patch is truncated. pr/meta.json has diffComplete: false, and pr/diff.patch stops mid-array inside docs/reviews/catalog-selection-529/queries.json. Visible: docs/catalog-selection.md (new), docs/operations.md (+1 table row), and docs/reviews/catalog-selection-529/{README.md, assessments.json, comparison.json, protocol.json, queries.json (partial)}. I cannot PASS on a truncated diff, and the missing portion is exactly the part that carries #529's的实现 acceptance.
Blockers
-
The
feat(knowledge)surface is not in what I could see. The title anddocs/catalog-selection.mdname a CLI (inferrsi catalog-select),inferrsi.apps.knowledge.select_catalog_evidence,confirm_catalog_evidence,inferrsi.knowledge.catalog_selection.CatalogSelection, andtools/compare_catalog_selection.py. Nosrc/module, no CLI wiring, no test file appears in the visible patch. That leaves #529 完成条件 2 (one real end-to-end chain, "交付不能仅停留在导出目录或提示词") and 条件 5 (端到端测试) unverifiable here. Either republish the complete diff or point me at the exact paths and revisions so the next pass can read them. -
The recorded run is scoped to a different revision than this head.
docs/reviews/catalog-selection-529/README.mdpins the code candidate asc457c85, while the reviewed head is3f6d5d32. The README says it explicitly: after fixing the reference closure only deterministic reads were replayed, reusing the same baseline and the pre-fix selections ("修正引用闭包后只重放确定性读取", "候选决定在代码修正前保留"). Soassessments.jsonandcomparison.jsoncharacterizec457c85, not this head. Nothing in the visible patch mapsc457c85onto3f6d5d3. State that relationship (ancestor? equivalent for these artifacts?) or re-run. Good news on the dependency: the baseline1b826b5is this PR's own base and the README records #516 as merged there, which clears the unmerged-dependency blocker from the #529 triage. -
Delivered packages and context-limit omissions are indistinguishable in the trace. In
comparison.json, entryid: "version-alignment"listspackage:inferrsi.ascend.skills.vllm_ascendincandidate_delivered(expansion path,status: "delivered", 86,668 bytes) and also incandidate_omissionswithreason: "context_limit",required_bytes: 21756,available_bytes: 9828. Entryid: "day-0"does the same forpackage:vllm_omni.common.omni.stage_ownership_and_search(delivered 3,044 bytes; also omitted at 18,293/5,223). Those omission records carry noresource/pathand no scope discriminator, so a reader cannot tell which declared resource was dropped. That collides withdocs/catalog-selection.md, "Reads, expansion and trace": "Required skill procedures/resources, source attribution and license closure are atomic: insufficient space produces an omission, never a partial procedure marked complete", and with #529 条件 4 (终止原因、缺失材料与预算不足均可观察). Add the specificresource/path(or an explicitscope: package|resource) to omission records, and confirm the 86,668-byte delivery did not include a partial required procedure. -
Byte accounting does not say whether the typed reader metadata is inside the budget.
docs/catalog-selection.mdstates "read_bytesis the complete bundle size, not a promise about budgeted context size", and the CLI "counts that exact transport, including its accounting envelope". #529's dependency note records that #516'sread --materialpayloads grew ~45% because themetadata.inferrsiblock is delivered inside content. Ifcontext_bytes/expansion_bytesaccounting excludes that block,required_bytes/available_bytesundercount and the over-budget omissions above would be wrong. Please state one way or the other in the doc; this is a question, not a proven defect.
Non-blocking
- In all 9
assessments.jsonentries,claimandreasonare byte-identical. The doc describes them as distinct fields ("the specificclaim,verdict,reason"). Confirm the interface populates them from separate inputs rather than one variable. README.mdreports 168 passed / 2 skipped and a full-repo attempt stopped at 1,143 passed / 12 failed / 12 skipped with all three representative failures reproduced on the unmodified baseline. That reporting style is right, but I ran nothing — no test file is visible, and CI state is not seeded.
Credited from what I could read: cards are capped per field with explicit truncation flags while the candidate list is never cut (that answers the uncapped-limitations/card-size concern from the #529 triage); over-budget input returns catalog_over_budget with ordinary Find results and no selector call; unqualified, outside-snapshot and duplicate refs fail rather than resolve silently; match/analogue/unknown/conflict is kept separate from source verification level; and benefit is stated as INCONCLUSIVE rather than claimed. The review artifacts also disclose their own degradations per query honestly, including the two queries where the selected Record was never delivered.
Not verified here: the implementation itself, its tests, CI status, and the ten-query run. This review does not approve the change or alter the scope of #529. I will re-read the full head after the author pushes or once the complete diff and the c457c85 → 3f6d5d3 relationship are in the seed.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
初次独立检视 1b826b5f → 3f6d5d32,按 #529 的完整目录和可恢复引用契约检查。批准前需修两项:
catalog_selection.py:222–249:可选 Markdown URL 直接交给urlsplit(),其解析异常会中止整个 snapshot,预算超限时也不能正常回退到 Find。应只把坏引用记为不可解析,继续生成其余候选;加测试证明同集合的有效候选和回退均仍可用。catalog_selection.py:70–83:词法检查把1.4.x识别为精确版本,和实际1.4.2比较得到 conflict(方法用途下 analogue),与文档承诺的非精确版本归 unknown 不符。请区分明确版本与通配/范围,覆盖对照用例。
已静态核对实现、文档和测试定义;未执行 CLI、测试或查询 CI。#529 的设计结论不变。
There was a problem hiding this comment.
Independent third-party tip COMMENT (cursor[bot]; not Approve / not Request changes).
Head 3f6d5d32ff42e5fbab1644988d28f02fd0d499c0 still carries tip DO NOT PASS from Dong1017 (always-on) and NO PASS from zengchuang-hw. I am not clearing those.
Blockers I independently treat as still open before any cursor formal vote:
- Dong: complete
feat(knowledge)surface must be reviewable (seed/diff completeness; #529 end-to-end acceptance), recorded-run revision vs tip head relationship, omission records that cannot distinguish delivered vs context-limit drops, and explicit byte-accounting scope for typed reader metadata. - Zeng:
catalog_selection.pyURL parse failures must not abort the whole snapshot (isolate bad refs; keep Find fallback + tests), and lexical version checks must not treat1.4.x-style ranges as exact versions.
Ready checks are green on this tip, but that does not override the review blockers. Waiting on author push; will re-verify after tip moves.
Technical verify only — collaborator permission none; not write+ / dual / merge-ready.
Governing Issue and delivery slice
Refs #529
The requester authorized implementation, a PR, and a ten-query knowledge retrieval comparison (not ten hardware optimization rounds), with selection performed in the current conversation.
This adds an opt-in complete-catalog retrieval path over the shared Skill/Record/Case corpus. The current Agent selects snapshot-bound qualified references; Knowledge checks conditions, expands explicit references within budgets, and delivers original resources with pinned revisions. Find and the seven primary task routes remain unchanged. Over-budget catalogs fall back explicitly to Find.
catalog-selectand the Apps Python selector seam share snapshot generation, identity validation and read assembly.Changed artifacts and evidence
Usage and contracts · ten-query report · qualified results · source-quote assessments
The comparison pins baseline
1b826b5f9666a8b734000e1eb2990e582cdef876and implementationc457c85; this PR's following documentation commit only records results. Knowledge bytes are unchanged: 80 supporting Skills, 48 Records and 7 Cases. Queries reuse the repository's seven task fixtures plus three labelled Qwen/quality-boundary questions.Find returned 50 entries and delivered 39 owners. The Agent selected 25 entries and the final implementation delivered 19 owners; six selected owners could not fit their required resources. Each query's IDs and omissions are retained. The confirmed full-image quality failure question deliberately selects no extra evidence. Full transport size is 77,086–77,113
o200k_basetokens per query under the declared 128,000 budget and 4,096 reserve.These are observational outputs, not accuracy or speedup claims. The same current conversation was used; the Agent inspected all 135 ID/summary entries, then relevant source excerpts. Actual model usage and decision latency are unavailable. Fixed Agent decisions were retained through a correction to deterministic reference reads; the final replay reused the same baseline and selections. No outcome-driven model reruns were used. Task benefit remains INCONCLUSIVE.
改动前后对比(10 个固定查询)
本次对照是旧版 Find 与显式启用的新模式。合入本 PR 后,未启用新模式的调用方仍沿用原有 Find。知识内容、题面与任务入口保持一致。
上述数量按题累加,同一材料跨题重复计算;“成功递送”表示 reader 实际返回原文字节,不表示模型已完整阅读或采用。返回数量不同不能作为精确率、召回率或效果提升的依据。
逐题全部命中与选择
技能采用稳定短 ID;Record/Case 保留原 ID。未递送表示新模式选中了材料,但受读取预算限制没有返回完整必需资源。最后一列只计实际成功递送。
ascend-model-adaptationvllm_omni.ascend.diffusion.qwen_image_21_eager_t2isglang_omni.common.audio.serving_and_streaming_qualityvllm_omni.common.audio.streaming_serving_measurementascend-video-diffusion-optimizationascend-model-adaptationvllm_omni.common.omni.stage_ownership_and_searchvllm_omni.common.diffusion.production_integration_gatesascend-video-diffusion-optimizationvllm_omni.common.diffusion.measurement_and_profilinginference-benchmark-designvideo-diffusion-optimizationhost-overhead-optimizationvllm_omni.common.diffusion.measurement_and_profilingperformance-ceiling-analysis(未递送)host-overhead-profilingcase.qwen-b300-rejected-directionsascend-graph-modepytorch.cuda.compiler.cudagraph_shapes_and_lifetimesflashinfer.cuda.llm.paged_decode_dispatchascend-aclgraph-capture-replaypytorchpytorch.cuda.compiler.cudagraph_shapes_and_lifetimes(未递送)inference-benchmark-designqwen3_tts_engine.ascend.audio.fixed_customvoice_promptqwen3_tts_engine.ascend.audio.codec_constants_and_stateqwen3_tts_engine.ascend.audio.graph_batching_state_ownershiptorch-npuinference-benchmark-design(未递送)sglang_omni.common.audio.serving_and_streaming_qualityvllm_omni.common.audio.streaming_serving_measurementvllm-ascendvllm-metalvllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuseascend-model-adaptationaudit-afd-p2p-exchangesrepository-evolution-analysisvllm-ascendtorch-npu(未递送)deepinfer.common.llm.zero_overhead_deferred_schedulingdeepinfer.common.llm.bounded_windowed_replayvllm-metalhost-overhead-optimizationdeepinfer.common.llm.speculative_verification_budgetdeepinfer.common.llm.zero_overhead_deferred_schedulingvllm.common.llm.runtime_and_kv_ownershipvllm.common.llm.extension_correctness_gatesrepository-evolution-analysisinference-benchmark-designproject-mayaweight-loading-optimizationstrata.common.llm.engine_validation_contractcann-attribution-overlapinference-benchmark-design(未递送)kernel-writing-guideclaim-contractcase.qwen-b300-rejected-directionsvllm_omni.ascend.diffusion.qwen_image_21_eager_t2icase.qwen-ascend-rejected-directionsvllm_omni.cuda.diffusion.minimax_h3_qk_norm_ropeinference-benchmark-designcase.qwen-b300-rejected-directionsinference-benchmark-design(未递送)vllm_omni.ascend.diffusion.qwen_image_21_eager_t2icase.qwen-ascend-rejected-directionscase.qwen-b300-rejected-directionsascend-graph-modeascend-model-adaptationcase.qwen-ascend-rejected-directionsvllm_omni.ascend.diffusion.qwen_image_21_eager_t2icase.qwen-b300-rejected-directionsinference-benchmark-designcase.drama3-rejected-directionsvllm_omni.ascend.diffusion.qwen_image_21_eager_t2iascend-operator-tuning逐题解释、退化与结论边界
o200k_base预算计数不能替代实际会话用量。完整材料:对比报告 · 逐题机器结果 · 原文确认 · 固定题面。
Checks and dependencies
ruff check src tests tools: passed.git diff --check: passed.Remaining work and Issue closure
Formal independent PR review and CI remain separate. The report explicitly retains large-resource omissions, missing target-native evidence, absent independent scoring and conversation-isolation limits. Goal acceptance/closure remains with the requester and maintainers; this PR uses a non-closing issue reference.
Knowledge authorship declaration: not applicable to the configured protected paths; this change adds retrieval implementation and documentation without editing catalog/read implementations or knowledge data. Agent contributions are identified in signed-off commits.