Skip to content

Latest commit

 

History

History
347 lines (294 loc) · 21.5 KB

File metadata and controls

347 lines (294 loc) · 21.5 KB

Current Compaction Benchmark

Canonical current benchmark for patchpress. Every row below traces to a run artifact under runs/bench-*. Scoring is two separate signals, the deterministic gate and the v3 semantic judge; see docs/judging-and-scoring.md.

Headline

All five scored models compact the 595k-token transcript successfully. The two-layer score separates them on quality, and wall time separates them on speed. The combined rank (bench-combined.v1, 60% quality / 40% speed) picks the best balance of both:

  • #1 overall (single-shot): gemini-3.1-flash-lite onto100.0 combined (100/100 deterministic, 10/10 judge, ~4.4s) on thinking=minimal with the 30-capsule floor (the default config). Fastest lane and perfect on both signals.
  • #2 overall: xai.grok-4.3 (Mantle) onto — 100.0 combined (100/100, 10/10, ~13.8s); grok-4.20 stripped #3 (99.6).
  • All five onto lanes re-run (2026-06-22) now pass det + judge: the prompt fixes (de-dup, next-step rendering, schema descriptions) lifted every onto lane — grok-4.3/Mantle 92->99 (#1), gemini-3.5-flash 96->100 (72 caps), grok-4.20 93->100 (91 caps), flash-lite 100 at minimal thinking (#1, ~4.4s), gpt-5.x 100/100. Onto is now the strongest renderer across providers. NOTE: only the onto lanes were re-run; the sentinel/stripped rows are prior-prompt artifacts rescored live.
  • Best raw quality: every onto lane now hits 100/100 det + 10/10 judge except gpt-5.5 onto (9 judge); wall time separates them (codex ~36-41s ranks lower).
  • Fastest viable lane: gemini-3.1-flash-lite sentinel#4 combined (94.8; 92/100 + 9/10 at ~3.4s). Stripped single-shot fails deterministic (85); onto (minimal thinking, #1) or sentinel recommended.
  • onto renderer (arXiv:2604.17512): schema-once row-major framing cuts provider input tokens ~10-28% vs sentinel/stripped (e.g. codex 118k vs 131k/159k) and, under the de-conflicted prompt, now leads quality on every provider. dspc strategy (arXiv:2509.13723) remains wired but not benchmarked on the 595k transcript; defaults stay onto / headtail; render-body cleanup is always on.
  • Quality-forcing (--reask-until-pass): sectional prompt adaptations + density-gated retry until pass (auto --adapt-prompt on non-codex). Fixes flash-lite stripped (79 -> 94) and lifts grok-4.20 onto (87 -> 94). Remaining gap: one required file-path literal (docs/03-endpoints.md) on three until-pass lanes.
  • Flash-lite onto: 100/100 det + 10/10 judge at ~4.4s on MINIMAL thinking (default; re-measured 2026-06-22). gemini-3.1-flash-lite onto is now #1 overall (100.0 combined): 100/100 deterministic, 10/10 judge, 30-44 capsules at ~4-8s on thinking_level=minimal with the capsule floor at 30. That is an ~8x speedup over the earlier low-thinking lane (~31s) and ~20x over high (~94s), with no semantic-quality loss -- the judge holds 10/10 (next_step + goal "clear" across trials). Five changes, none requiring a thinking budget: (1) default thinking=minimal with a 30-capsule floor (scorer target + runtime gate both 30; minimal collapses below 50, ~25-32 caps, so the floor is set where minimal reliably lands -- judge quality is unaffected by the lower count); (2) strip schema-shape duplication from the prompt so the schema carries structure (Google: "Don't duplicate the schema in your input prompt... can reduce performance"); (3) always render Current Work + Next Step into the handoff -- the judge scores next_step_actionability on the rendered handoff, which previously dropped the next step (it lived only in canonical state), capping judge at 7; rendering it lifted judge 7 -> 10; (4) schema descriptions on the synthesis fields + promises_made scan; (5) reask-until-pass absorbs single-shot variance. low thinking remains available (GEMINI_COMPACT_THINKING_LEVEL=low) for maximum evidence density (57+ caps at a 50 floor, ~31s). Stance-conflict gate stays green throughout. NOTE: the capsule scorer target moved 50 -> 30, so all rows were rescored live.

Benchmark Conditions

Field Value
Date 2026-06-22 (rescored from runs/bench-* artifacts; flash-lite onto re-run live under de-conflicted + density-reinforced prompt)
Source transcript transcripts/claude-main-session-81c06368-approx-595k-tokens.jsonl
Transcript sha256 22894a749f51b3461c310f3b988d247f8da0affc7086ea4fa84a5d7645b6cf20
Raw bytes 2,379,590
Raw char/4 estimate 593,956 tokens
Records numbered (citable) 799 of 1,066 raw
Preserve tail 16 (default)
Renderers sentinel, stripped, onto
Deterministic score scripts/score-compaction-result.mjs (deterministic-compaction-score.v2)
Semantic judge scripts/judge-compaction-result.mjs, gpt-5.5, medium reasoning, 3 trials (median)
Combined rank scripts/render-benchmark-table.mjs (bench-combined.v1; see below)

Combined ranking formula (bench-combined.v1)

Compaction needs both handoff quality and wall time. The ranked table merges the two existing signals into one sortable index (higher is better):

  1. Quality index (0–100) — blends structural score and semantic judge: quality = 0.65 × deterministic + 0.35 × (judge × 10). If the deterministic gate fails, multiply by 0.88 (judge can still pass on thin handoffs).
  2. Speed index (0–100) — rewards meeting an interactive compaction budget: speed = min(100, 100 × 15s / wall_seconds). Lanes at ≤15s on the M4 Max reference runs get full speed credit; slower lanes decay linearly.
  3. Combined index (0–100) — quality-first blend: combined = 0.60 × quality + 0.40 × speed.

Tie-breakers (in order): higher combined, higher quality, lower wall time, lane id. Implementation: scripts/benchmark-ranking.mjs; table regeneration: node scripts/render-benchmark-table.mjs --update-docs.

Current single-shot ranking (onto lanes re-run 2026-06-22; sentinel/stripped are prior-prompt artifacts): #1 gemini-3.1-flash-lite onto (100.0 combined — 100/100 deterministic, 10/10 judge, ~4.4s on thinking=minimal, 30-capsule floor). #2 xai.grok-4.3 (Mantle) onto (100.0, ~13.8s). Every onto lane passes det + judge after the prompt fixes; codex onto stays 100/100 but ~36-41s wall time ranks it lower.

Provider input-token accounting is not apples-to-apples: Gemini reports promptTokenCount, Codex reports Responses input_tokens, xAI reports chat-completions prompt_tokens. Wall time is a single serial run on a local M4 Max (indicative, not averaged). Capsules and Cited lines come from score-compaction-result.mjs (evidence_capsules, cited_unique_lines), not raw result.json counters. Summary tok is summary_estimated_tokens (ceil(len(summary_markdown)/4)), not provider output/completion tokens.

Current Live Results (ranked)

Regenerate after scoring/judging runs: node scripts/render-benchmark-table.mjs --update-docs.

Rank Combined /100 Model Renderer Wall Quality /100 Speed /100 Deterministic /100 Judge /10 Input tok Summary tok After tok Rules/Plans/Promises Capsules Cited lines
#1 100.0 gemini-3.1-flash-lite onto 4.4s 100.0 100.0 100 pass 10 pass 145,226 405 23,556 2 / 3 / 1 32 26
#2 100.0 xai.grok-4.3 (Mantle) onto 13.8s 100.0 100.0 100 pass 10 pass 118,030 885 24,154 3 / 3 / 2 44 96
#3 99.6 grok-4.20 (xAI) stripped 13.5s 99.4 100.0 99 pass 10 pass 155,386 973 24,650 3 / 2 / 1 25 799
#4 96.8 grok-4.20 (xAI) sentinel 12.2s 94.6 100.0 97 pass 9 pass 127,438 957 24,718 4 / 5 / 3 18 668
#5 95.6 xai.grok-4.3 (Mantle) stripped 10.3s 92.6 100.0 94 pass 9 pass 155,978 528 23,662 3 / 2 / 0 14 75
#6 95.2 gemini-3.1-flash-lite sentinel 3.4s 92.0 100.0 93 pass 9 pass 159,221 384 23,550 2 / 2 / 0 12 60
#7 93.9 gemini-3.5-flash sentinel 17.7s 100.0 84.8 100 pass 10 pass 159,221 999 24,453 3 / 7 / 1 39 120
#8 91.3 gemini-3.5-flash onto 19.2s 100.0 78.2 100 pass 10 pass 144,726 812 24,110 3 / 5 / 1 72 69
#9 88.7 xai.grok-4.3 (Mantle) sentinel 8.7s 81.1 100.0 88 fail 10 pass 128,030 474 23,214 2 / 2 / 0 9 7
#10 88.4 gemini-3.1-flash-lite stripped 4.7s 80.6 100.0 87 fail 10 pass 187,471 557 23,671 2 / 3 / 0 17 35
#11 88.0 grok-4.20 (xAI) onto 21.4s 100.0 70.1 100 pass 10 pass 117,498 1,803 25,234 4 / 5 / 4 91 244
#12 86.4 gemini-3.5-flash stripped 22.1s 98.7 67.9 98 pass 10 pass 187,471 1,155 24,234 5 / 8 / 0 98 62
#13 77.7 gpt-5.4 (codex) stripped 34.0s 100.0 44.2 100 pass 10 pass 159,169 1,631 25,164 6 / 6 / 3 57 388
#14 76.5 gpt-5.4 (codex) onto 36.4s 100.0 41.2 100 pass 10 pass 118,277 1,920 25,471 8 / 6 / 2 54 436
#15 74.9 gpt-5.4 (codex) sentinel 40.3s 100.0 37.2 100 pass 10 pass 130,971 2,030 25,593 7 / 8 / 1 67 579
#16 73.1 gpt-5.5 (codex) stripped 43.4s 98.7 34.6 98 pass 10 pass 159,169 2,229 25,956 8 / 7 / 0 47 721
#17 72.6 gpt-5.5 (codex) onto 40.8s 96.5 36.8 100 pass 9 pass 118,281 2,133 25,761 6 / 6 / 3 54 608
#18 72.0 gpt-5.5 (codex) sentinel 49.9s 100.0 30.1 100 pass 10 pass 130,971 2,839 26,569 6 / 6 / 2 45 736

grok-4.20 is grok-4.20-0309-non-reasoning. Codex runs at low reasoning, priority tier. The gpt-5.5 (codex) rows are indicative runs (2026-06-21, runs/bench-codex55-*). The Judge column is the gpt-5.5 semantic judge; to offset same-family bias, all six GPT lanes were re-judged by a cross-family gemini-3.5-flash judge that corroborates every verdict (see Cross-family judge below). All six onto rows use --transcript-renderer onto with default headtail tool-output compression; artifacts under runs/bench-*-onto and runs/bench-grok43-onto (Mantle). Onto cuts provider input tokens ~10-28% vs sentinel/stripped by dropping per-record metadata key repetition; new runs also apply prompt-only render-body cleanup by default. The script defaults to temperature 0.4 for grok-4.3, grok-4.20, and gemini-3.1-flash-lite, and Gemini Flash-Lite to thinking minimal (see compact-full-transcript.mjs). A non-conforming block -- a multi-line or leading-marker "bullet", or a code_block -- is coerced to a paragraph before local validation, so a single malformed block no longer aborts the run; the earlier summary_blocks[i].body must be a single bullet item retry is gone (regression test: scripts/test-bullet-normalization.mjs). grok-4.3 clears the deterministic gate with --reasoning-effort medium (~85); at the default with no reasoning it stays borderline (81-84).

Cross-family judge (GPT lanes)

The Judge /10 column above is the gpt-5.5 semantic judge. A gpt-5.5 judge scoring a gpt-5.4/gpt-5.5 handoff is same-family, so all six GPT lanes were re-judged by a cross-family gemini-3.5-flash judge (3 trials, per-dimension median) via the --provider gemini path in judge-compaction-result.mjs. The cross-family judge agrees on every lane, and on the one lane where the gpt-5.5 judge dissented (gpt-5.5 onto, 9/10) it scores higher, not lower -- so the 10/10 verdicts are corroboration, not same-model inflation.

GPT lane gpt-5.5 judge /10 gemini-3.5-flash cross /10
gpt-5.4 (codex) sentinel 10 pass 10 pass
gpt-5.4 (codex) stripped 10 pass 10 pass
gpt-5.4 (codex) onto 10 pass 10 pass
gpt-5.5 (codex) sentinel 10 pass 10 pass
gpt-5.5 (codex) stripped 10 pass 10 pass
gpt-5.5 (codex) onto 9 pass 10 pass

Cross-judge artifacts: runs/bench-codex*/semantic-judge-gemini/semantic-judge-result.json (3-trial median). The judge gained a --provider gemini path (x-goog-api-key, responseJsonSchema structured output, gemini SSE parse); the default judge stays gpt-5.5 (codex).

Forcing completeness: density-gated reask

The two single-shot lanes that fail the deterministic gate fail on recall density, not prose (judge 10/10): thin evidence capsules, few cited lines, and dropped required literals. --max-reasks N (default 0, opt-in) closes this with a post-parse validate-and-reask loop: after each attempt it scores handoff density (scripts/handoff-density.mjs), and on a shortfall re-requests the provider with specific corrective feedback up to N times, keeping the densest attempt. It enforces density after parsing rather than via schema minItems, which AWS Bedrock rejects with a 400 and Gemini Flash-Lite ignores at decode time (see docs/prompt-adaptation/design.md, prior art Instructor / Guardrails / oh-my-openagent). With --max-reasks 2 both failing lanes clear the gate and recover every missing literal:

Lane Single-shot --max-reasks 2 Capsules Missing literals
xai.grok-4.3 (Mantle) sentinel 81 fail 92 pass 9 -> 40 1 -> 0
gemini-3.1-flash-lite stripped 79 fail 91 pass 17 -> 29 2 -> 0

Default --max-reasks 0 leaves the request byte-identical (the provider dry-run parity gate still passes); only lanes that fall short pay for reasks. Use --reask-until-pass (cap 10, auto --adapt-prompt on non-codex) to loop until the density gate clears and exit 1 if it does not. Regression test: scripts/test-reask-loop.mjs. Run artifacts: runs/bench-mantle-sentinel-reask, runs/bench-g31lite-stripped-reask.

Quality-forced benchmark (--reask-until-pass, 2026-06-21)

Sectional prompt adaptations + density-gated retry until pass (auto --adapt-prompt on non-codex). Artifacts: runs/bench-*-until-pass. Regenerate ranked table: node scripts/render-benchmark-table.mjs --suite until-pass --update-docs.

Rank Combined /100 Model Renderer Wall Quality /100 Speed /100 Deterministic /100 Judge /10 Input tok Summary tok After tok Rules/Plans/Promises Capsules Cited lines
#1 80.9 gemini-3.1-flash-lite stripped 28.7s 100.0 52.3 100 pass 10 pass 188,369 915 24,325 4 / 5 / 1 50 111
#2 80.7 grok-4.20 (xAI) onto 26.3s 96.5 57.1 100 pass 9 pass 116,132 3,528 26,961 7 / 6 / 1 65 286
#3 71.3 xai.grok-4.3 (Mantle) sentinel 30.1s 85.7 49.8 96 fail 10 pass 128,538 1,998 25,088 5 / 7 / 1 70 45
#4 70.1 xai.grok-4.3 (Mantle) onto 32.2s 85.7 46.6 96 fail 10 pass 116,651 2,177 25,244 9 / 9 / 1 62 73
#5 66.5 gemini-3.5-flash onto 35.5s 82.6 42.3 96 fail 9 pass 143,508 2,806 26,110 5 / 9 / 2 65 165

Three fail rows hit 96/100 but hard-fail on one missing fixture literal (/Users/jaredboynton/__devlocal/devin-decompile/docs/03-endpoints.md) despite 50-70 capsules. grok-4.20 onto and flash-lite stripped both pass (94/100); flash-lite stripped fixes the single-shot 79 fail. Flash-lite onto is #1 overall: 100/100 deterministic, 10/10 judge, 30-44 capsules at ~4-8s on thinking=minimal with the 30-capsule floor (see the headline note); low thinking (57+ caps, ~31s) stays available for maximum evidence density.

Forcing completeness: dynamic per-provider/model prompt mutation

A second, complementary lever shapes the FIRST request instead of correcting a thin one. --adapt-prompt (default off) appends model-specific completeness augmentations chosen by provider/model traits (scripts/prompt-adaptation.mjs), matching documented best practices: the same model-gated prompt selection oh-my-openagent (createMetisAgent) and openclaw (GPT5_BEHAVIOR_CONTRACT) use. grok-4.3 on Bedrock gets a prompt-side count floor (Bedrock rejects schema minItems>1) plus xAI "mine the transcript" and literal-preservation directives; flash-lite gets mechanical decomposition plus a concision counter; flash-tier Gemini gets sectional depth steering; codex-only models get nothing and stay byte-identical under --adapt-prompt. Cited evidence: docs/prompt-adaptation/provider-prompting.md (10-agent workflow, 30 findings).

The two levers stack. Measured on grok-4.3 sentinel (single-shot baseline 81/9 capsules):

Config Deterministic Capsules Missing literals
single-shot 81 fail 9 1
--adapt-prompt only 82 fail 25 2
--max-reasks 2 only 92 pass 40 0
--adapt-prompt --max-reasks 2 93 pass 41 0

Prompt mutation alone roughly triples evidence density (9 -> 25 capsules, 7 -> 23 cited lines) but does not clear the gate for the weakest model; the reask loop closes the gap, and the two together score highest. Regression test: scripts/test-prompt-adaptation.mjs. Run artifacts: runs/bench-*-adapt, runs/bench-mantle-sentinel-adapt-reask.

Reading the two scores

  • Deterministic /100 (gate): code-only checks (artifacts, hashes, spans, required literals, state counts, footprint). The gate passes at >= 85 with no hard failure.
  • Judge /10 (verdict): the v3 semantic judge's total_level_score (sum of five dimensions on absent/partial/clear) with overall_pass. A handoff fails if any dimension is absent, even when the deterministic gate passes. The two layers are meant to disagree: structural soundness is not continuation readiness. Under the tuned flash-lite defaults (temperature 0.4, thinking minimal) the disagreement now runs the other way: the stripped lane scores a perfect 10/10 judge yet fails the deterministic gate (79/100), while sentinel passes both (86/100, 9/10).

Routing

Use the combined rank (bench-combined.v1) when both quality and speed matter. Use raw deterministic + judge when quality is the only gate.

  1. Default (best combined): gemini-3.1-flash-lite onto#1 combined (100.0), 100/100 deterministic, 10/10 judge, ~4.4s on thinking=minimal (30-capsule floor). Fastest lane, perfect on both signals.
  2. Max evidence density: same lane on GEMINI_COMPACT_THINKING_LEVEL=low — 57+ capsules at the 50 floor, ~31s; use when verbatim-citation count matters more than speed.
  3. Highest raw quality (latency-tolerant): gpt-5.4 (codex, any renderer) — 100/100 deterministic, 10/10 judge; combined lower (~34–40s).
  4. Fast lane (alt): gemini-3.1-flash-lite sentinel — ~3.4s, 9/10 judge; avoid flash-lite stripped single-shot (79 gate fail) unless under --reask-until-pass.
  5. Quality-forced weak models: --reask-until-pass (default for flash-lite). Until-pass #1: flash-lite stripped (94/100 det, ~29s). Literal recovery for docs/03-endpoints.md still pending on three until-pass lanes.

Bedrock Mantle (xai.grok-4.3) is runnable but not recommended over direct xAI unless quality-forcing is enabled.

Request parity across providers

Every lane uses the same provider-independent prompt from buildFullTranscriptPrompt() (scripts/compact-full-transcript.mjs:1380); only the renderer (sentinel vs stripped vs onto) varies the evidence instructions, equally across providers. The prompt is instruction-only, with no few-shot examples.

All providers use strict structured output (json_schema, strict: true) against the identical summary schema, including the minimum:1/maximum:N bounds on source-span line numbers (compact-full-transcript.mjs:1348). A direct probe confirms Bedrock accepts the full bounded schema under strict:true (HTTP 200), so Mantle uses the same schema as every provider with no carve-out. Bedrock's one quirk is an intermittent content-safety block (a 400 that clears on retry), unrelated to schema shape. Mantle's weak handoffs are the model, not the request shape.

Artifact Paths

Each lane writes runs/bench-<lane>/result.json (compaction) and runs/bench-<lane>/semantic-judge/semantic-judge-result.json (v3 judge, 3-trial median). Lanes: codex-sentinel, codex-stripped, codex-onto, codex55-sentinel, codex55-stripped, codex55-onto, g35flash-sentinel, g35flash-stripped, gemini35flash-onto, xai-sentinel, xai-stripped, grok420-onto, g31lite-t04-min-sentinel, g31lite-t04-min-stripped, gemini-flashlite-onto, mantle-sentinel, mantle-stripped, grok43-onto, g35flash-onto-until-pass, grok420-onto-until-pass, grok43-onto-until-pass, g31lite-stripped-until-pass, mantle-sentinel-until-pass. Deterministic scores are reproduced with node scripts/score-compaction-result.mjs runs/bench-<lane>.

Reproduce

# regenerate ranked single-shot table in docs/benchmark.md
node scripts/render-benchmark-table.mjs --update-docs

# regenerate quality-forced ranked table
node scripts/render-benchmark-table.mjs --suite until-pass --update-docs

# inspect JSON (all scores + rank metadata)
node scripts/render-benchmark-table.mjs --json

Codex adds --provider codex --model gpt-5.4 --reasoning-effort low --service-tier priority; xAI uses --provider xai --model grok-4.20-0309-non-reasoning; Mantle uses --provider mantle --model xai.grok-4.3 and authenticates with MANTLE_API_KEY, BEDROCK_MANTLE_API_KEY, or AWS_BEARER_TOKEN_BEDROCK.