fix(skill): per-type incremental kinds for the --update runbook (#2033)#2050
Open
HerenderKumar wants to merge 1 commit into
Open
fix(skill): per-type incremental kinds for the --update runbook (#2033)#2050HerenderKumar wants to merge 1 commit into
HerenderKumar wants to merge 1 commit into
Conversation
…hify-Labs#2033) The --update runbook called detect_incremental() with the default kind="semantic", so on an AST-built corpus (code files never receive a semantic_hash) every file reported as changed and a 12-file real delta became a 629-file full-corpus re-extraction. Passing plain kind="ast" instead would skip the semantic catch-up for docs an AST-only rebuild downgraded (Graphify-Labs#2014), so neither existing mode fits a pipeline where code is structural-only and docs are semantic. detect_incremental() gains kind="auto": code files compare against ast_hash, everything else against semantic_hash. The runbook (shared update reference plus the aider/devin monoliths) now detects with it. The runbook's save_manifest() call had the matching stamping defect: the kind="both" default forged a semantic_hash for every corpus file, including docs never semantically extracted, silently cancelling their future catch-up. The merge step now stamps kind="ast" over the corpus, then kind="semantic" over just the changed docs/papers/images/videos the semantic pass actually covered; on a code-only update that set is empty, leaving exactly the kind="ast"-only stamp the issue asks for. skillgen: the monolith round-trip sanctions the new detection lines via _is_incremental_kind_fix_line; artifacts and expected/ regenerated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2033.
Problem
Both calls in the
--updaterunbook fall back to defaults that are wrong for this pipeline (verified on currentv8,edec9ea):detect_incremental()defaults tokind="semantic", which treats a missingsemantic_hashas "changed". Code files never receive semantic extraction (the skill sends only docs/papers/images to subagents), so on an AST-built corpus — e.g. one maintained bygraphify update,--watch, or the git hook, which stampkind="ast"— every code file reports as changed. That's the 629-reported vs 12-real blowup in the issue.save_manifest()defaults tokind="both", forging asemantic_hashfor every corpus file — including docs that were never semantically extracted — so a later semantic pass considers them done and silently never runs.Why not a blanket
kind='ast'The issue suggests passing
kind='ast'at detection. That fixes the code-file blowup but breaks the other direction: a doc downgraded by an AST-only rebuild (kind="ast"stamps clear itssemantic_hashon content change — that's the documented catch-up signal, and the silent-downgrade scenario of #2014) would compare by its freshly-stampedast_hashand report unchanged, so the runbook would never semantically re-extract it. Code and docs need different hash columns.Fix
detect.py:detect_incremental()gainskind="auto"— code files compare againstast_hash, everything else againstsemantic_hash. One-line dispatch inside the existing per-type loop;"semantic"/"ast"behavior untouched.references/update.mdfragment: detection passeskind='auto'; the merge step replaces the single default-bothstamp with two:kind='ast'over the corpus (preserves each untouched file'ssemantic_hash, clears it for changed files), thenkind='semantic'over just the changed docs/papers/images/videos the semantic pass actually covered. On the runbook's code-only branch that set is empty, leaving exactly thekind='ast'-only stamp the issue asks for — one shared code path, no branch duplication.aider/devinmonoliths: samekind='auto'detection fix (their legacy update flow never re-saves the manifest, so there's no stamping call to fix).tools/skillgen/gen.py: the monolith round-trip sanctions the new lines via_is_incremental_kind_fix_line, following the existing predicate convention; all artifacts andexpected/regenerated (--check,--audit-coverage,--schema-singleton,--monolith-roundtrip,--always-on-roundtripall pass).Verification
tests/test_detect.py(code-with-ast_hash-only unchanged; doc missingsemantic_hashre-queued; changed code re-queued; doc with validsemantic_hashunchanged). The full 9-suite batch touchingdetect_incremental/skillgen matches the clean-v8baseline failure-for-failure (remaining failures are pre-existing environment gaps: missingrapidfuzz/tree-sitter).graphify/skills/claude/references/update.md: every embeddedpython -csnippet parses; on a 5-file AST-built corpus (3 code + 2 docs) the old detection reports all 5 files changed, the new one reports 0 code files and only the 2-doc semantic backlog; after the two-stamp save, changed code carries a freshast_hashwith no forgedsemantic_hash, covered docs carry a realsemantic_hash, and a follow-up detection reports zero changes.Residual (intentionally out of scope)
graphify extract's own incremental path keepskind="semantic"per the issue's scoping ("the full-pipeline path should keepkind="semantic"/"both"").