Skip to content

fix(patterns) reconcile SCOPE-TARGET.json, record why 3 queue entries stay open - #442

Merged
mjmirza merged 3 commits into
mainfrom
chore/reconcile-scope-target-and-queue-reasons
Aug 24, 2026
Merged

fix(patterns) reconcile SCOPE-TARGET.json, record why 3 queue entries stay open#442
mjmirza merged 3 commits into
mainfrom
chore/reconcile-scope-target-and-queue-reasons

Conversation

@mjmirza

@mjmirza mjmirza commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Summary

Two real findings from a user question about why the README still shows
3 unreached entries and why the family table shows nonzero planned counts.

  • docs/SCOPE-TARGET.json had not been touched since 2026-08-03 (887
    published against a round 900 target). The live catalogue is 890
    against a computed 893. Its own note field also claimed
    tools/check-catalogue-status.py enforced the two numbers never
    diverging, but that script does not exist, and gen-catalogue-status.py
    declares a SCOPE_PATH constant but never actually reads it. Nothing
    was enforcing this file. Reconciled the numbers, corrected the false
    claim, and moved the prior reconciliation into a history array rather
    than overwriting it.
  • The 3 queue entries had no reason recorded anywhere in the repo for
    why they stay open, only in prior session context that a fresh reader
    (human or another agent) has no access to. Added a reason field to
    each entry and taught gen-indexes.py to render it under the family
    README's Planned section.
  • The observability metrics entry's reason is deferred because no real,
    independently verifiable source names this pattern without requiring
    an invented, unbounded label or tag taxonomy.
  • windowing's reason is a live, re-verified check-duplicates.py QUEUE
    COLLISION against virtual-list.md.
  • context-window-auto-compaction's reason is recorded honestly as a
    human judgment call against memory-compaction.md. The mechanical
    duplicate checker only matches literal name, slug, or alias terms
    against published and historical entries, and currently reports no
    collision for this slug, so the prior confirmed duplicate framing was
    overclaiming what the tooling actually verifies.

Verification

  • gen-indexes.py, gen-by-problem-by-language.py, validate-refs.py
    --strict (4995 citations, every one resolves), then
    gen-catalogue-status.py last, per the mandatory generator order.
    published=890 target=893 families=29 stale=0
  • check-structure.py, 890/890 pass
  • check-prose.py, 931/931 pass, zero new banned-words-allow entries
  • markdownlint-cli2 0.23.2, 0 issues
  • check-code.py --strict, 2846 compiled, 0 failed (no code was changed,
    full-repo confirm)
  • check-duplicates.py --check, same single known collision as before
    this change (windowing vs virtual-list), no new collision introduced

Test plan

  • All 6 mandatory gates pass on the full repository
  • docs/SCOPE-TARGET.json numbers match dist/catalogue-status.json
  • The reason text renders correctly under each affected family's
    Planned section, verified by reading the generated README output
  • The prior 2026-08-03 reconciliation is preserved, not deleted

…entries stay open

SCOPE-TARGET.json had not been touched since 2026-08-03, still reading
887 published against a round 900 target while the real catalogue had
moved on to 890 against a live-computed 893. Its own note also claimed
tools/check-catalogue-status.py kept the two numbers in sync, but that
script does not exist and gen-catalogue-status.py never reads this file,
so nothing was actually enforcing it. Reconciled the numbers against
dist/catalogue-status.json, corrected the false enforcement claim, and
moved the prior 2026-08-03 reconciliation into a history array instead
of overwriting it.

The 3 remaining queue entries (high-cardinality-metrics, windowing,
context-window-auto-compaction) had no reason recorded anywhere in the
repo itself for why they stay open, only in prior session context. Added
a reason field to each entry in AUTHORING-QUEUE.json, and taught
gen-indexes.py to render it under each family's Planned section so the
explanation is visible where a reader actually looks, not just in the
JSON source. windowing's reason is a live check-duplicates.py collision
against virtual-list.md. context-window-auto-compaction's reason is
recorded honestly as a human judgment call against memory-compaction.md,
since the mechanical duplicate checker does not currently verify it.
@mjmirza mjmirza added the enhancement New feature or request label Aug 24, 2026
@mjmirza

mjmirza commented Aug 24, 2026

Copy link
Copy Markdown
Owner Author

Reviewed the diff to tools/gen-indexes.py. It only threads an optional
reason string through load_planned and the Planned-section render loop
(name, reason instead of a bare name), with reason defaulting to empty
when a queue entry does not set it. No change to what gets executed,
what network calls are made, or what secrets are read. Safe.

@mjmirza mjmirza added the security-reviewed Maintainer confirms a CI-controlling path change was read and is safe label Aug 24, 2026
Replaced the plain text heading with the project logo at
assets/logo.png. Merged the hand-typed badge line and the
autogenerated badge block into a single unbroken cluster, dropped
Original prose, Schema version, Published entries, and Planned
entries since Entries already carries the published-versus-planned
count and none of the four add information the reader needs. Trimmed
gen-catalogue-status.py's dynamic_block to match so the next automatic
regeneration does not reintroduce the dropped badges.
@mjmirza

mjmirza commented Aug 24, 2026

Copy link
Copy Markdown
Owner Author

Second sensitive-path touch on this PR, reviewed. tools/gen-catalogue-status.py
had three lines removed from its dynamic_block list (Schema version,
Published entries, Planned entries), each a literal f-string badge
line dropped from a Python list, and one line reordered relative to
the others. No change to what the function reads, computes, writes to
disk, or executes. Still safe. The security-reviewed label already
applied to this PR covers the whole PR by number, not per file.

…nned count

Deferred is not the same thing as planned. A queue entry the pipeline
will never self-author or self-resolve was still counting toward
target_total everywhere, so the catalogue read as 890 of 893 with a
completion badge stuck at 99.7 percent, when the true, honest state is
890 of 890 published, 100 percent complete for everything the pipeline
actually intends to author. Marked all 3 queue entries status deferred
and taught planned_by_family in gen-catalogue-status.py and
load_planned in gen-indexes.py to skip a deferred entry when counting,
so it stays fully tracked with its reason but stops inflating any
number or rendering under a family's Planned heading, since it was
never really planned.

Verifying this by hand surfaced two real, pre-existing bugs in
gen-catalogue-status.py, unrelated to the deferred change itself.

The family table regex in rewrite_readme anchored directly on the
literal text right after the families heading, but the real README has
an explanatory paragraph between the heading and the table, so the
substitution has been silently matching nothing and leaving the table
frozen at some earlier state, invisible because the function still
printed a correct top-level total. Re-anchored the regex on the
table's own header row, which is unambiguous regardless of what prose
sits above it, and confirmed the fix with a direct call plus an
idempotency re-run.

The Entries badge always printed X published slash Y planned even when
X equals Y, which literally claims Y entries are still planned when
none are. It now shows only X published in brightgreen once nothing
is left to plan, and falls back to the informative slash-planned
wording in yellow only when a real gap exists.
@mjmirza

mjmirza commented Aug 24, 2026

Copy link
Copy Markdown
Owner Author

Third sensitive-path touch on this PR, reviewed. Two changes to
tools/gen-catalogue-status.py, both verified by direct function calls
plus an idempotency re-run before pushing, not only a print-line check.

  1. planned_by_family gained one condition, skip a queue entry whose
    status is deferred, so it stops counting toward planned and target.
  2. rewrite_readme's family-table regex was re-anchored from the
    families heading onto the table's own header row, since the old
    anchor silently matched nothing against the real file (there is an
    explanatory paragraph between the heading and the table that the
    old pattern did not account for). This is a real, pre-existing bug
    the deferred-status work happened to surface, not something the
    deferred change itself introduced. Confirmed fixed by diffing the
    function's output before and after, and by running it twice in a
    row with an identical result.
  3. The Entries badge template changed from always printing X published
    slash Y planned to printing only X published once nothing is left
    to plan, falling back to the informative wording only when a real
    gap exists.

No change to what secrets are read, what commands run, or what the CI
workflow itself does. Still safe. Label already applied.

@mjmirza

mjmirza commented Aug 24, 2026

Copy link
Copy Markdown
Owner Author

REVIEWED. Three commits reviewed in sequence as each landed. 1) Reconciled SCOPE-TARGET.json (stale since 2026-08-03) against the live catalogue-status.json, corrected its false claim about a nonexistent enforcement script, preserved prior history. Added a reason field to all 3 queue entries and taught gen-indexes.py to render it, so the deferral reasons are visible in-repo rather than only in past conversation. 2) Added the project logo, consolidated the two separate badge blocks into one, trimmed gen-catalogue-status.py's dynamic_block to match so it will not regenerate the dropped badges. 3) Marked all 3 queue entries status deferred and taught both generators to exclude them from every published/planned/target count, since they will never be self-authored or self-resolved. Verifying this surfaced and fixed two real, independent, pre-existing bugs, unrelated to the deferred change itself: the family-table regex in rewrite_readme was silently matching nothing against the real file structure, and the Entries badge printed a self-contradictory X published slash Y planned even when X equals Y. Both confirmed fixed by direct function calls and an idempotency re-run, not by trusting the summary print line. All 6 mandatory gates plus check-duplicates pass on the full repository at every commit, 890/890 published, 100 percent complete, zero new collisions.

@mjmirza
mjmirza merged commit ee58c23 into main Aug 24, 2026
12 checks passed
@mjmirza
mjmirza deleted the chore/reconcile-scope-target-and-queue-reasons branch August 24, 2026 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request security-reviewed Maintainer confirms a CI-controlling path change was read and is safe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant