Repository navigation
Description
Produced by a multi-agent investigation over the 500-fixture oracle corpus. Every number carries its measurement setting (see §2) because four incompatible "clean fixture" counts were already in circulation. Claims marked INFERRED were not read from code.
This issue IS the task decomposition — W1.1 through W4.6 each carry a repo, an effort tier, its dependencies, the fixtures it moves, and an acceptance check. Non-goals, open questions and the companion ADR draft are in the comments.
Related: comfy-cloud-mcp-server#1867 (the parity job's stale comfy-cli pin), comfy-cloud-mcp-server#1869 (PreviewAny over-emission), comfy-cli#893 (widget-type classification), #1070 (promoted subgraph values), #1071 (umask test fragility). Triage of the Comfy Hub conversion audit that started this: BE-24127.
Converter convergence roadmap
Scope: Comfy-Org/comfy-cli (comfy_cli/workflow_to_api.py, comfy_cli/cql/), Comfy-Org/comfy-cloud-mcp-server (src/converter/, which is also the source root of the published @comfyorg/api-to-workflow consumed by ComfyUI_frontend), and the oracle corpus at src/converter/test/fixtures/templates/. Comfy-Org/cql is tracked adjacently — see Non-goals.
1. The problem
One contract — "turn a saved workflow into the API prompt the backend executes" — has four independent implementations (frontend graphToPrompt(), @comfyorg/api-to-workflow, comfy-cli convert_ui_to_api, and cql's save-format slot writer), no shared code or data, and no mechanism that notices when they disagree. The user-visible cost is that the same workflow file executes differently depending on which path submitted it: comfy-cli currently converts 4 corpus fixtures to the wrong input image (a saved upload-button object aborts the whole schema walk, so LoadImage falls back to the combo's first option — comfy_cli/workflow_to_api.py:2389-2391); it drops user values entirely for CurveEditor.curve (issue #893, same line); it synthesizes optional widget values the user never set, so a pre-#1076 comfy run sends thinking=True and codec="none" on nodes the frontend sends nothing for; and slot_overrides on a subgraph instance writes to the wrong widget — 92.image lands in RandomNoise.noise_seed and silently changes the generated video (verified live against the pinned wasm). The reference that would catch all of this is frozen: the 500 goldens were captured 2026-09-11 against a frontend and an object_info nobody recorded, and they are now evaluated against an object_info that a cron rewrites daily — 142 of 500 fixtures (28.4%) fail for no reason but that the catalog moved under a fixed golden, which trains everyone to read real defects as staleness (and did: PreviewAny's three invented keys were waved through as "UI-only" for 52 fixtures while over-emitting on 148 of 148 goldens). The one job that compares two implementations pins comfy-cli to 231f85bf of 2026-07-05 — 139 commits and 114 days off main, and not an ancestor of main, so it cannot even detect that it is stale, and it insulates mcp CI from every comfy-cli fix and regression alike. Operationally: three sets of hand-maintained tolerance tables that nothing cross-checks, 4 tolerances that are provably dead (masking any future regression on those keys), a headline stat in KNOWN_ISSUES.md (249/455) that is stale in both numerator and denominator, and a corpus that already misses 38% of the shipped template index.
2. Measurement discipline (prerequisite for every number below)
Four different "clean fixture" counts circulate in the material behind this roadmap — package 266/500 and 394/500; comfy-cli 136/500 and 205/500 — because each was measured under a different combination of (tolerances on/off, catalog = current/at-capture, _meta in/out, comfy-cli revision pre/post #1076). None is wrong; they are not comparable. No item in this roadmap may be accepted against a number measured under an undeclared setting. Wave 1 therefore ships one measurement definition, and every subsequent acceptance check cites it.
Definition (SKEW/1): converter output vs expected_api.json; node-key sets equal; every input deep-equal after unwrapping {__value__} / {__type__,__value__}; _meta excluded both sides; node ids coerced to strings; catalog named explicitly (current | atCapture); tolerances off; comfy-cli revision recorded. Signature grammar: missing-input:<Class>.<in> · extra-input:<Class>.<in> · value:<Class>.<in> · missing-node:<id>:<Class> · extra-node:<id>:<Class> · class:<id>:<Expected>-><Actual>.
3. Corrections to assumptions this plan was nearly built on
| Assumption | Reality | Consequence for the plan |
|---|---|---|
comfy-cli #1069 makes the top-level widget walk follow saved inputs[].widget order |
It no longer does. Scope was cut after six reviewers objected that a consumed-slot count check cannot catch a permutation and that top-level inputs[] order is not a reliable record of widgets_values order. #1069 now reorders only COMFY_DYNAMICCOMBO_V3 sub-inputs (ByteDance2ReferenceNode, 10 of 392 Hub workflows). |
The "109 of 180 missing-input occurrences close for free" claim is void. SaveAudioMP3.audioUI (35), LoadAudio.audioUI (26), SaveGLB.image (34), Preview3D.image (14) need an explicit emission path. W3.1 is promoted from "free rider" to the largest single comfy-cli work item. |
| The marker reader does not exist yet | #1076 already reads inputs[].widget markers to decide whether an optional widget was absent, and falls back to the default when there are no markers. |
W3.1 extends an existing reader rather than introducing one; the "emit undeclared marker names" decision is a one-line policy change at a site that already parses markers. |
The _boundary_targets marker gate and the Load3DAdvanced layout split are unfiled |
Already open: #1039 (promoted value preserved at socket fan-out targets, fixes #974) is the marker-gate relaxation; #1056 shares the Load3DAdvanced legacy-layout check between edit and convert; #1057 strips the legacy button constants on edit; #1038 adds source_address to comfy workflow slots. |
Those move from "write" to "review and merge" in Wave 1/2. Do not open duplicates. |
| The stale pin means the MCP runs a 114-day-old converter | COMFY_CLI_PIN is documented dead — nothing reads it at runtime (src/backend/cli-comfy-client.ts:27-33); live seams resolve comfy from COMFY_BIN/PATH. |
Harm is confined to (a) CI measuring a stale converter and (b) being the documented anchor for whoever packages the runtime image. Urgent, not an incident. |
Wave 1 — already specified; start now
Nothing here depends on anything unbuilt. Three of six are open PRs awaiting review.
W1.1 — Merge comfy-cli #1072 (neutralize agent-attribution env vars in tests)
Repo comfy-cli · Tier trivial · Fixtures 0 (tooling) · Depends —
Change One autouse conftest.py fixture stripping CLAUDECODE / AI_AGENT / COMFY_USER_AGENT, sibling to the existing _neutralize_forced_color. Five test_code_search.py tests fail only when the suite is run from inside an AI tool.
Why first Every other comfy-cli item in this roadmap is verified by running the suite; until this lands, every contributor and agent sees 5 phantom failures and learns to ignore red.
Acceptance pytest tests/comfy_cli/command/test_code_search.py is 27/27 both with and without those vars exported; env -u run and plain run produce identical results.
W1.2 — Merge comfy-cli #1076 (stop synthesizing absent optional widgets)
Repo comfy-cli · Tier routine · Fixtures closes the largest cli-only cluster: CreateVideo.{bit_depth,codec,color_space} 116 each, SimpleMath+.b 111, ModelSamplingAuraFlow.sampling 97, CFGNorm.pre_cfg 43, SimpleMath+.a 20 · Depends W1.1 (to trust the suite)
Change Optional widget inputs absent from widgets_values are omitted; required inputs and optional non-widget sockets keep their schema default; socketless-required path untouched.
Acceptance The 5 signature families above drop to 0 occurrences under SKEW/1(catalog=atCapture); TestSeedreamDynamicCombo::test_pristine_pro_t2i_template_converts asserts thinking absent with the reasoning inline. Blocking caveat: the /tmp/skew differential was taken at comfy-cli c272a5e, which predates this PR — so its cli-side cluster counts are already stale and must be re-measured on post-merge main before any later item is sized against them.
W1.3 — Merge comfy-cli #1069 (dynamic-combo subs in saved order)
Repo comfy-cli · Tier routine · Fixtures 0 in the template corpus; 10 of 392 Hub workflows (Seedance 2.0 ByteDance2ReferenceNode) · Depends W1.1
Change Expands a selector's chosen option's sub-inputs in the order the save recorded, not the option's current order. Conservative in both directions (unknown sub name → current expansion; no dotted markers → unchanged; repeated name → first position; connection-only subs carried through).
Acceptance The 9 tests in test_workflow_to_api_saved_widget_names.py pass, 3 of which fail on main; on the live snapshot model.auto_upscale is absent and seed is the integer, not "randomize". Record explicitly in the PR that this does NOT change the top-level widget walk, so W3.1 is not written against a capability that does not exist.
W1.4 — mcp companion: ancestry guard + pin bump + KNOWN_DIVERGENCES rewrite
Repo par-mcp · Tier routine · Fixtures 3 of 4 KNOWN_DIVERGENCES entries deleted · Depends W1.2, W1.3 merged to comfy-cli main
Change One PR, three parts. (a) A step before the parity run: git -C .ci/comfy-cli fetch --depth=200 origin main && git merge-base --is-ancestor <pin> origin/main, failing with ci-bump-pin-guard.yml's existing wording — this fires today, the pin is NOT-ANCESTOR. (b) Advance all four 231f85bf literals (.github/workflows/ci.yml:401, src/converter/test/conversionParityCi.test.ts:24, src/backend/cli-comfy-client.ts:39, src/backend/cli-comfy-client.test.ts:408) to the post-#1076/#1069 SHA. (c) Delete the three padding entries from KNOWN_DIVERGENCES (conversionParity.test.ts:84+): template:03_video_wan2_2_14B_i2v_subgraphed, template:02_qwen_Image_edit_subgraphed, and the 127:78.sampling half of template:05_audio_ace_step_1_t2a_song_subgraphed — leaving only 59.audioUI (W3.1) and legacy_group_nodes.json.
Why coupled The suite asserts the divergence set bidirectionally — a resolved signature fails exactly like a new one — so #1076 landing will red this suite the moment the pin moves, and the stale pin is the only reason it has not already. Shipping (b) without (c) reds mcp main.
Acceptance npx vitest run src/converter/test/conversionParity.test.ts green with KNOWN_DIVERGENCES at 2 entries; the ancestry step fails on the old SHA and passes on the new one; cli-comfy-client.ts's comment block updated so the dead-constant caveat still reads true.
W1.5 — Batch the Python parity driver
Repo par-mcp · Tier routine · Fixtures 0 directly; unblocks all 500 in CI · Depends —
Change scripts/skew/run-cli.py: one process, stdin-driven loop, --templates <dir> --catalog <name>=<gz> (repeatable), emits {fixture: {catalog: apiGraph | {error}}}. Keep comfyCliConvert.py for single-fixture debugging (its fd-1→stderr dance at :70-85 is the right tool there).
Why Measured: 1.79 s per subprocess vs 6.8 ms of actual work — 224× overhead. 500 fixtures: ~15 min spawned, 5.2 s batched (0.42 s import + 0.32 s catalog inflate + 3.38 s convert). The 9-fixture corpus at conversionParity.test.ts:51-55 was never a corpus decision; it was the only affordable shape given execFileSync per fixture at :267-274.
Acceptance All 500 fixtures converted in one process in <15 s wall; output byte-identical to 500 individual comfyCliConvert.py invocations on a sampled 25.
W1.6 — Pin the oracle to its capture-epoch catalog
Repo par-mcp · Tier analytical · Fixtures +142 (266 → 408 package-side, SKEW/1) · Depends W1.7 (manifest) for the blob SHA; independent of W1.1-1.5
Change src/converter/test/fixtures/templates/MANIFEST.json carries objectInfoBlobSha: 2f37e189b847e7e9d5fbc08419e9e88d836aa767 (= a2c6d012:data/object_info.json.gz, 2026-09-11, md5 8caf766d…, 3655 classes). New src/converter/test/oracleCatalog.ts#loadEpochObjectInfo() materializes it via git cat-file blob <sha> into .ci/oracle-object-info/<sha>.json.gz (~700 KB, ~0.3 s, cached); on failure it falls back to data/object_info.json.gz and sets EPOCH_PIN_UNAVAILABLE, printed as ::warning:: — never a silent substitution. oracleFixtures.test.ts:84 switches to it. Then: delete isReparentedKey (:55-80, collapses to zero under the pin — the epoch snapshot declares SaveVideo.required = [video, filename_prefix, format, codec], bare); delete the 4 dead IGNORABLE_DIFFS entries (:13-14); keep LoadVideo.video-preview, Load3D.image, PreviewAny.* (they survive the pin, so they are not catalog drift); MIN_CORRECT 420 → 400 (stricter than 420 was, since the escape hatches are gone); MIN_WOULD_EXECUTE stays 498; re-enable the _meta.title test (:331) gated on the pin being available; STRUCTURAL_SKIP (:121-125) shrinks to flux1_dev_uso_reference_image_gen alone.
Acceptance Package clean count 408/500 under SKEW/1(catalog=atCapture); isReparentedKey deleted and the suite still green; all four Meshy/Quiver fixtures diff-free (verified by replay); EPOCH_PIN_UNAVAILABLE path exercised by a test that points the loader at a bogus SHA and asserts the warning is emitted and the fallback used.
W1.7 — Provenance manifest + capture validity gate
Repo par-mcp · Tier routine · Fixtures moves 2 from STRUCTURAL_SKIP to a named capture defect · Depends —
Change MANIFEST.json schema v1: capturedAt, capturedFrom, frontendVersion, frontendCommit, comfyuiVersion, objectInfoBlobSha, objectInfoCommit, templatesIndexCommit, captureParams, fixtureCount, captureErrors, degradedFixtures. Backfill from git (capturedAt = a2c6d012 author date 2026-09-11T17:50:40Z; frontendVersion: null, unrecoverable). export-oracle-fixtures.js reads window.__COMFYUI_FRONTEND_VERSION__ + GET /api/system_stats before the loop and emits a reserved __manifest__ bundle key; split-oracle-bundle.ts:48 routes that key to MANIFEST.json, not a fixture dir. Validity gate: after graphToPrompt() (:230), reject any result containing a node with falsy class_type or an input keyed UNKNOWN — push to _errors. Today template_animate_diff_loops and utility_face_detection_mediapipe carry 5 such nodes, captured from a browser session with packs unregistered, and were absorbed into STRUCTURAL_SKIP as converter known-issues.
Acceptance MANIFEST.json validates against its schema; a synthetic bundle containing a class_type-less node makes the splitter exit non-zero; the two degraded fixtures appear in degradedFixtures and no longer in STRUCTURAL_SKIP.
Wave 2 — closes measured gaps; depends only on Wave 1
W2.1 — Delete the package's PreviewAny registry entry
Repo par-mcp · Tier trivial · Fixtures +47 (394 → 441 under tolerances-on; 148 of 148 PreviewAny nodes over-emitted today) · Depends W1.6 (so the measurement is against a pinned catalog)
Change FRONTEND_EXTRA_WIDGETS.PreviewAny (widgetMapper.ts:40-44) emits three keys graphToPrompt() never emits — textPreviewWidgets.ts:39,:53,:66-67 sets options.serialize = false on both widgets it injects, and one of the three names (preview_markdown) does not exist at all. Replace with slot-consuming non-emitting entries on the PointsEditor shape: [{name:'preview_text',serialize:'none',requireSlot:true},{name:'previewMode',serialize:'none',requireSlot:true}]. Both flags are load-bearing: requireSlot is what stops the else if (!extra.requireSlot) inputs[extra.name] = '' fallback at :1576 from emitting anyway. Then delete the PreviewAny tolerance from both mirrored IGNORABLE_DIFFS (oracleFixtures.test.ts:14-18, scripts/oracle-fixtures/verify-oracle-fixtures.ts:83-90) so the suite measures it, and fix the source comment at widgetMapper.ts:24 — it cites src/extensions/core/previewAny.ts, a file that no longer exists.
Acceptance 0 extra-input:PreviewAny.* signatures over 500 fixtures; clean count +47; grep -rn PreviewAny src/converter/test/ scripts/ returns no tolerance entry. Before merging, run git log -S PreviewAny -- src/converter/widgetMapper.ts and cite in the PR which fixture (if any) motivated the entry, so the deletion cites evidence rather than only the current corpus.
W2.2 — comfy-cli: stop aborting the schema walk on a dict slot
Repo comfy-cli · Tier analytical · Fixtures 4 converted to the wrong input image today; plus CurveEditor.curve (#893) · Depends W1.1, W1.2
Change _collect_widget_inputs (workflow_to_api.py:2389-2391) currently does if any(isinstance(v, dict) for v in widget_values): _absorb_dict_widget_values(...); return out — abandoning the whole walk. Replace with a per-slot rule inside the walk: a dict slot whose options.serialize is False (or type == "button") is a frontend-injected non-serializing widget — consume the slot, emit nothing. Keep _absorb_dict_widget_values (:2432-2449) only for the lora / no-schema paths it was written for, entered only when the dicts are self-describing (type or lora keys) and the class declares no positional widget order. Highest-severity item in the whole roadmap: a saved upload-button object makes LoadImage lose its real filename.
Acceptance A fixture-level test asserting the 4 affected fixtures emit the saved filename, not the combo's first option; CurveEditor.curve round-trips its {__type__:"CURVE",__value__:…} value; no converter output ever contains an input literally named button or upload (see W2.5).
W2.3 — comfy-cli: promoted host-value fallback + merge #1039
Repo comfy-cli · Tier analytical · Fixtures 5 improved, 0 regressed, 11 signatures removed, 0 added (measured over all 500, same catalog) · Depends W1.1
Change Two lines on the existing path, not the proposed spike. (a) _overlay_promoted_host_values (workflow_to_api.py:285-287) does continue when the host value is UNSET; call comfy-cli's own already-correct frontend port instead — promoted.effective_value_for(...) / source_value (cql/promoted.py:815-847, :849-861). (b) Relax _boundary_targets' marker gate (promoted.py:2345-2353): when a target's serialized input entry has no widget marker, still accept it if entry["name"] is a declared input of the target class, or the class is unknown to object_info — the frontend inlines a promoted widget's value into every consumer of the boundary link (markerless INT,FLOAT, wildcard *, COMFY_MATCHTYPE_V3 all receive it in the goldens). This is what open PR #1039 does (fixes #974) — review and merge it rather than reimplementing; (a) is the piece that makes it reachable.
Explicitly do NOT wire flush_proxy_migration into convert_ui_to_api: measured over every proxyWidgets instance in all 500 fixtures, it creates 0-1 inputs per workflow, changes no output, and removes none of the 9 cluster signatures. #1070's blocking spike is testing a hypothesis that is already falsified.
Acceptance 5 named fixtures improve, byte-identical on the other 495, 0 signatures added; 1527 passed / 2 skipped across cql/ + workflow_to_api + command/workflow; a perf guard asserting conversion never constructs a full Graph (monkeypatch Graph.from_object_info to raise) — the eager Graph is 0.53 s vs 0.0029 s of conversion, ~180×; use a lazy catalog view (ctor 1.6 µs; warming all 591 distinct classes across all 500 fixtures costs 0.079 s total).
Add to the parity allowlist (conversionParity.test.ts:51-55, today 3 subgraph fixtures, none of this shape): gsc_starter_2 (nested legacy chain, value on a leaf two levels down), video-wan21_scail (markerless INT,FLOAT socket), video_ltx2_3_ic_lora (COMFY_MATCHTYPE_V3 socket).
W2.4 — comfy-cli: fix the write-side widgets_values_named mirror
Repo comfy-cli · Tier trivial · Fixtures 0 (0 of 500 fixtures carry the field) but a reproduced live defect · Depends W1.1
Change cql/engine.py:5113 gates the named-map mirror on has_group, so a plain set_widget on a non-dynamic node leaves the map stale. Reproduced: comfy-cli writes a file where it converts steps=30 while @comfyorg/api-to-workflow and the frontend both convert the same file to the pre-edit steps=20 — comfy-cli silently publishes a self-contradicting workflow whose edit the other two discard. Replace the gate with a module-level _mirror_named_widget_value(node, name, value) that fires whenever the key exists and is a dict, updating an existing key only (never adding one) — the exact rule of the package's mirrorNamedWidgetValue (widgetMapper.ts:2089-2106). Call it from the two other writers that skip it (engine.py:5270 same-option in-place dynamic-combo selector write, :5546 legacy-primitive widgets_values[0]). Move the shape guard at :4950-4951 off the has_group gate.
Acceptance A test that edits steps on a non-dynamic node carrying widgets_values_named, then asserts comfy-cli's and @comfyorg/api-to-workflow's conversions of the written file agree; grep -n has_group comfy_cli/cql/engine.py shows no remaining gate on a named-map mirror.
W2.5 — Regression guards for the two negative cases both sides drifted into
Repo both · Tier trivial · Fixtures guards, not fixes · Depends W2.1, W2.2
Change Assert over the 500-fixture corpus that (a) no converter output ever contains an input named upload, and (b) PreviewAny emits only source. Both are provable against the goldens and both are exactly where the two registries currently disagree with them.
Acceptance Each guard fails when the corresponding registry entry is reintroduced (prove it by reverting W2.1 locally and watching it red).
W2.6 — Rewrite KNOWN_ISSUES.md from the metric, not by hand
Repo par-mcp · Tier trivial · Depends W1.6, W2.1
Change The headline (:3-4) claims 249/455 (55%) against a 500-fixture corpus whose own test comment records 469/500 while the floor sits at 420. Replace with the SKEW/1 output, stating the catalog it was measured against. Close the TextGenerate / LTXVAudioVideoMask / GeminiImage2Node entries (§3, :49-59) as resolved-by-catalog-refresh — all are live classes (10, 1 and 42 fixtures respectively) whose divergence no longer fires. Correct §4: its stated cause for LoadVideo.video-preview (slot count) is measurably false — object_info declares one widget and all 44 saved nodes have identical two-slot widgets_values; document it as capture non-determinism (21 goldens carry it, 23 with identical inputs do not).
Acceptance Every number in the file is reproducible by one command; a test asserts that any IGNORABLE_DIFFS entry producing no signature on a class present in the corpus is a hard failure (3 lines, and the one self-healing property the current design lacks).
Wave 3 — closes the remaining measured gaps; needs Wave 1-2 infrastructure
W3.1 — comfy-cli: build the trailing frontend-injected-widget emission path
Repo comfy-cli · Tier deep · Fixtures closes 109 of 180 missing-input occurrences from the marker-derivable group alone; measured 136 → 160 on the pre-#1076 baseline · Depends W1.2 (#1076's marker reader), W1.3, W2.2, and no longer on #1069 (see §3)
Change comfy-cli has the consumption half and a partial name model (cql/engine.py:1011-1040 frontend_extra_widget_names, :167 FRONTEND_MARKER_SLOTS, :72-80 _LOAD_3D_BUTTON_SLOTS, :978-1008 load_3d_button_slots, pinned for the catalog by tests/comfy_cli/cql/test_frontend_widget_slots.py) but never emits — _schema_widget_pairs (workflow_to_api.py:1908) stops walking at the last object_info-declared input. Four parts, in order:
- After the
orderedloop, walk theinputs[].widgetmarker names that follow the last declared input and emit each from its slot. Two value rules: a marker with no remaining slot still emits""(SaveAudioMP3.audioUI, 35 goldens); a marker in the non-serializing set{upload}emits nothing while consuming its slot. Policy decision to record: emit undeclared marker names rather than intersecting withobject_info— this is where the 109 occurrences live. - Emit the Load3D button slots instead of skipping them (
:1994-2003currently doeswhile widget_values[vidx] in LOAD_3D_BUTTON_VALUES: vidx += 1); append(name, value)from_LOAD_3D_BUTTON_SLOTSpositionally. Goldens emit them asupload 3d model/upload extra resources/clear. Coordinate with fix(workflow): share Load3DAdvanced legacy-layout check between edit and convert #1056/fix(cql): drop legacy Load3DAdvanced button constants on edit #1057, which already touch this code. LoadAudio.audioUIcomputed value: saved slot is always null; emitted value is/api/view?filename={urlquote(inputs['audio'])}&type=input&subfolder=&(trailing&included), applied post-walk only whenaudioUIwas emitted andaudiois a non-empty string (mirrorswidgetMapper.ts:1588-1595).- A small registry for the entries markers do not name:
ImageBatchMulti."Update inputs"(copy raw slot; golden value isnull),PointsEditor.{editor_display → consume+emit nothing, "Reset canvas" → copy slot},FL_PathAnimator.{"Edit Paths","Path Count"}, plusFL_PathAnimator's order override (paths_datais slot 0 at runtime but last ininputs[].widget, so no saved-order walk fixes it — express as the explicit prefix["paths_data","*"]), plus a variadic rule forCustomCombo: afterchoice, emitindex, thenoption{n}for every remaining slot, n from 1 (count =len(widgets_values) - 2, verified 3..9 across the corpus)._schema_widget_pairsmust start reporting slots consumed (today the count lives in its internalvidx).
Deliberately not added:PreviewAny(the package's entry over-emits on 148/148) andLoadVideo.video-preview(undecidable 21/23 corpus split — that belongs in the harness tolerance list, not converter data).
Acceptance Per-cluster, measured after each sub-part lands, not once at the end:SaveAudioMP3.audioUI35→0,LoadAudio.audioUI26→0,SaveGLB.image34→0,Preview3D.image14→0,ImageBatchMulti."Update inputs"18→0, Load3D buttons→0,CustomCombo.option{n}complete for N=3..9. The per-cluster arithmetic assumes a fixture's divergences are independent, and slot alignment is exactly where that breaks (emitting a previously-missing slot can shift a value that currently matches by accident) — so the gate is the total clean count, re-measured each time.
W3.2 — Fix CustomCombo on the package side (the one real both-sides defect)
Repo par-mcp · Tier analytical · Fixtures the only genuine both-sides defect of the 8 families examined · Depends W1.6
Change ExtraWidgetDef (widgetMapper.ts:33-38) needs a repeat flag; CustomCombo becomes [{name:'index',serialize:'value'},{name:'option',serialize:'value',repeat:true}], replacing the option1..option4 cap at :63-69 which under-emits option5-9. Three call sites read the list by fixed length and all three need the flag: the forward walk's for (const extra of extraWidgets) (:1560-1583), the slot-count/defaults helper's extraCount = registryEntry(...)?.length ?? 0 (:766-772), and the reverse api→workflow emitter (:2428-2450). index may keep reading slot 1 (saved == recomputed in all 10 golden instances) but recomputing it as options.indexOf(choice) matches serializeValue more faithfully.
Acceptance All 10 CustomCombo golden instances exact, N from 3 to 9; the reverse direction round-trips a 9-option node.
W3.3 — Widget-type classification: census + conformance vectors
Repo both · Tier analytical · Fixtures 46-odd inputs neither corpus can adjudicate; COLORCODE is latent-but-real, proven by template_purz_nb2_single_image_sprite_sheet · Depends W1.6, W3.1 (injected widgets are simultaneously a classification fact and an order fact)
Change The allowlist-membership framing is mostly a red herring: of 405 distinct input type strings in the 3853-class snapshot, 388 are handled by neither classifier and 370 of those are genuine link types where default-deny is correct. The real backlog is ~20 types, split three ways, and the classifiers disagree on rules, not membership: comfy-cli has widgetType and defaultInput rules the package lacks (43 inputs misclassified as links, incl. Preview3D.model_file); the package has a comma-split rule and CUSTOM_WIDGET_TYPES carrying COLORCODE, which comfy-cli misses on all 16 of its inputs. Ship two committed artifacts and two test files:
widget_type_census.json:type → {verdict: widget|link|hidden-only, registeredBy, declaredInputs, classes, evidence}. Generated, not authored:useWidgetStore().widgetsis literallynew Map([...customWidgets, ...Object.entries(ComfyWidgets)])andinputIsWidgetiswidgets.has(type)(widgetStore.ts:20-27) — that map is the authoritative set. (Settles one open question already:MARKDOWN/CHART/GALLERIA/RANGE/PAINTER/TEXTAREAare not dead — VERIFIED present atsrc/scripts/widgets.ts:249-261.)classifier-vectors.json: the ordered rule procedure as executable{type, options} → verdictcases plus a declared order (forceInput/defaultInput demotion → widgetType override → registered-type membership → socketless promotion → comma-split union →COMFY_*COMBO*family → wildcard refusal), each case carrying afailsToday: [impl]ledger that must shrink and is never a tolerance.tests/comfy_cli/cql/test_widget_type_census.pyandsrc/converter/test/widgetTypeCensus.test.ts, reading the same JSON. Three assertions: COVERAGE (an unknown type fails with its usage count and three exampleClass.inputsites, so a newobject_infodrops a red test rather than a silent link), AGREEMENT (_is_widget_inputandengine._is_linkboth match the census verdict — this is what catches theNUMBERsplit and theCOLORCODEgap without a fixture), REACHABILITY (every allowlist member either has a declared input in the snapshot or is taggedinjected: truewith the injecting class list — this surfacesIMAGEUPLOAD/AUDIOUPLOAD/PREVIEW_3D, which are injected-by-name and therefore unreachable from_is_widget_inputat all).
Plus the cheap comfy-cli fixes the census pins: accept membership inengine._FRONTEND_REGISTERED_WIDGET_TYPESin_is_widget_inputafter the DOM-widget check, guarded behind the existing forceInput check (one line; picks upCOLORCODE's 16 inputs and the 3 non-socketlessCOLORs); reconcile_IMPLICIT_WIDGET_TYPES(engine.py:36, hasNUMBER) with the converter's inline primitive set (workflow_to_api.py:1838, hasCOMFY_DYNAMICGROUP_V3, lacksNUMBER) into one exported definition — today the engine callsNumber Operation.number_aa widget port and the converter calls it a link, soset-widgetcan address a slot the converter will not write; addCOLOR/COLORCODE/IMAGECOMPAREto_FRONTEND_DOM_WIDGET_TYPES(engine.py:48-59); delete the deadtype_id != "*"conjunct atengine.py:1067("*"is not a member of_IMPLICIT_WIDGET_TYPES, so the guard can never fire); give the required-socketless fallback a type-keyed placeholder table instead ofNone(portgetSocketlessDefault,widgetMapper.ts:433-449) but only when a slot exists and is null —SAM3_Detect.bboxesandSDPoseKeypointExtractor.bboxeshave no slot and the golden omits the key entirely (fix(workflow): do not synthesize an optional widget the save never carried #1076 fixed this for optional widget inputs; required-socketless is a separate branch and still over-emits).
Acceptance The census test is non-vacuous: a poison fixture builds a syntheticobject_infowith one classCensusPoisoncarryingpoison_widget: ["POISON_WIDGET",{}]andpoison_link: ["POISON_LINK",{}], neither in the census, and asserts coverage raises; a second poison case asserts that a census entry claimingPOISON_WIDGETis a widget while_is_widget_inputsays link fails the AGREEMENT check. Without both, a refactor that empties the type set or swallows the exception passes silently. Plus:COLORCODEwidget on all 16 inputs in both implementations;NUMBERidentical in engine and converter.
W3.4 — Read-side widgets_values_named (layered, no walk change)
Repo comfy-cli · Tier routine · Fixtures 0 of 500 (latent); closes one verified read-path divergence · Depends W2.4, W3.1
Change After pairs = _schema_widget_pairs(...) returns (workflow_to_api.py:2400-2406), test the node's map for completeness over exactly those pairs — for every (name, value) in pairs with value is not None, the map must carry a non-None own key name — and on pass, substitute named[name] for each pair's value while keeping the pair list (names, order, count) untouched. Because the walk is untouched this cannot left-shift a slot and needs no relaxation of the exact accounting check. Also: when widgets_values is absent but a usable map is present, drive the existing walk with an empty value list instead of returning {} (today: package {seed:7,steps:33} vs cli {seed:0,steps:20}). Deliberately do not port the package's "a named name is emitted even with no slot" extension — on a post-#15924 save the two representations have identical cardinality, so it is unreachable for frontend-written files, and adding it would let the map decide which widgets exist.
Acceptance The four exclusions asserted as tests, not code: a dict/keyed widgets_values ignores the map; _fallback_widget_names (:2086-2119) ignores it (else canvas-only widgets like control_after_generate reach the prompt under real names); a linked input's value comes from the link; promoted subgraph host values stay positional and never consult a leaked/stale map on the container node (promoted.py:901-931). Mirror the package's leak tests (namedWidgetValues.test.ts:661,:751). Measurement caveat: 0 of 500 fixtures and 0 of 11025 nodes carry this field, so the oracle suite is blind to all of it — this item ships against hand-authored fixtures (W4.4) or it ships untested.
W3.5 — cql slot addressing, closed in TypeScript (adjacent track)
Repo par-mcp · Tier analytical · Fixtures 0 (different contract); closes a live silent-corruption path · Depends —
Change On raw-templates/video_ltx2_i2v.json instance 92, 7 of 7 advertised slot addresses misresolve against the pinned wasm (cql_sha=37ab0b5d): 5 fail loudly on a -1 interior id, and 2 succeed silently — 92.ckpt_name → 11.control_after_generate (a no-op, since that widget is serialize: false, which is the ticket's headline exhibit and is not corruption) and 92.image → 11.noise_seed (an INT override that changes the generated video, which the ticket does not name). Extend preResolveTemplateOverrides (tools-saved-workflows.ts:1744) to pre-resolve declared slot addresses the way input_overrides already are (BE-9294), using the existing SubgraphInstanceIndex (template-overrides.ts:55-74): single remapTargets id → flat {"<instance>:<interior>": {...}}; containerProxiedNames (the -1 set) → write the host instance's widgets_values slot, which is what converter.ts:717 reads and what ADR-SUBGRAPH-PROMOTION-0009 mandates; ambiguous/unroutable → leave in slot_overrides and let the wasm answer. Reuse findProxyWidgetForDefSlot (groupNodeFlattener.ts:614) rather than writing a third resolver.
Do not fix resolveSlot's index in Go as the first move: cql implements the pre-ADR-0009 value model (follow proxyWidgets into the interior), which comfy-cli (promoted.py:17-19) and the mcp converter (converter.ts:710-719) have both abandoned in favour of host-wins — so fixing the index alone yields a correctly-wrong engine. File the Go fix (name-match → IO-link → refuse, never positional; ADR-0009 host semantics for -1) as an independent upstream cleanup, and merge BE-9295 into BE-7351 — they are the loud and silent faces of one assumption (positional correspondence between definition inputs[] and instance proxyWidgets), and BE-7351 already carries the 1124-of-2551-addresses sweep.
Acceptance src/cql-slot-addressing.test.ts pins the 7-row table as the pre-fix baseline against the real wasm and real converter (~15 s), then asserts all 7 land on their declared input post-fix, with 92.image → 11.noise_seed pinned explicitly; the ambiguous branch provably falls through to the wasm unchanged.
Wave 4 — prevents the next gap; depends on Waves 1-3
W4.1 — Generate the widget-layout table from the frontend's own browser tier
Repo ComfyUI_frontend + a new data bundle · Tier deep · Depends W3.1, W3.3
Change graphToPrompt()'s entire widget contract is 11 lines (executionUtil.ts:133-150: iterate node.widgets, skip !widget.name || widget.options.serialize === false, apply serializeValue, wrap arrays). The only unknown for a headless converter is therefore the content and order of node.widgets — a per-class fact, not an algorithm, currently approximated by hand in three places. A job that dumps it already exists: browser_tests/fixtures/customNode/allNodesMountTier.ts:93 reads (node.widgets ?? []) with name/type/options for every node in the manifest in batches of 24, and :223 already serializes widgetNames. Adding options.serialize and serializeValue provenance is ~30 lines in a tier that runs against a real backend with every manifest pack installed. Emit widget-layout.json: per class, an array of {slot, name, type, origin, promptEmit?, serializeValue?} plus an optional variadicTail. slot (index in a fresh node's widgets_values) subsumes order and pre-input injection, so PRE_INPUT_INJECTED_WIDGETS (widgetMapper.ts:83-95) and WIDGET_ORDER_OVERRIDES (:107-118) both disappear, as does comfy-cli's needed prefix list. promptEmit ∈ {value,none} is a verbatim read of options.serialize !== false. serializeValue has exactly three closed forms — const:<json>, template:<str with {urlencode:<input>} holes>, indexOf:<input> — covering the whole corpus; a fourth form is refused by the schema and goes in a named escapeHatches list instead. requireSlot is deliberately derived, not a field: identity needs a slot, a const does not, which is what makes SaveAudioMP3.audioUI=""-with-no-slot and PointsEditor.editor_display-consume-and-emit-nothing fall out of one rule instead of two flags.
Acceptance A class-coverage count that may never shrink between releases; a generated-by header check that rejects hand edits; the three existing package tables deleted with the suite still green; comfy-cli's vendored copy diffed against the published bundle by a test_widget_layout_vendored.py (mirroring src/converter/test/localCopies.test.ts:29-38), vendored rather than fetched so comfy workflow stays offline-safe (the precedent and the requirement are both cql/annotations_source.py:9-14, and pyproject.toml:91 already globs comfy_cli.cql.data for *.json).
W4.2 — Three-axis, two-catalog skew CI with a bidirectional manifest
Repo par-mcp (+ comfy-cli mirror) · Tier deep · Depends W1.5, W1.6, W4.1 is not required
Change Replace ci.yml's conversion-parity with a conformance job running all 500 fixtures in one batched pass across three axes (mcp-vs-golden, cli-vs-golden, mcp-vs-cli) × two catalogs (current, at-capture), gated by one skew-manifest.json checked bidirectionally — an extra signature and a resolved-but-still-listed signature both fail, generalizing KNOWN_DIVERGENCES, which already works that way at 9 fixtures. One row per signature (343 rows at today's measurement, vs 2386 for the per-fixture shape), each carrying axis, exact occurrences, catalogSensitive, verdict ∈ {defect, golden-stale, catalog-drift, undecidable}, owner, evidence, and a sunset required on every verdict except defect. catalogSensitive is machine-derived from the two-catalog run and then asserted against the stored value — a human cannot relabel a converter defect as staleness, which is the error this whole effort exists to prevent. Measured split today: 444 of 758 occurrences (58.6%) and 82 of 191 distinct signatures exist only because the catalog moved; at-capture-only is 0, so that bucket is a clean third alarm (converter now correct only against the new catalog → goldens unrepresentable → open a [recapture] issue, never block) rather than noise.
Delete the tautological tier: computePassingFixtures() (oracleFixtures.test.ts:130-166), the it.each(passingFixtures) block (:347-351), and critically the else { it.skip('no fixtures currently pass exact match') } vacuity branch (:352-354), which goes green when the converter breaks on all 500. Drop the '*' / structuralMarker hatch (:98-101, :362-374) — it existed only because the old grammar could not express node-set deltas, and extra-node:/missing-node:/class: now can. Keep the aggregate floors as a second tier beneath the manifest. Add the test-count floor the premise assumed already existed (grep -rn "PARITY_REQUIRED\|MIN_TESTS" returns nothing): vitest --reporter=json, then assert numTotalTests >= 500 && numPassedTests == numTotalTests && numPendingTests == 0 — today, if COMFY_CLI_PATH is unset or renamed, conversionParity.test.ts:307-313 takes it.skip before the non-empty-corpus assertion at :315-321, vitest exits 0, and test goes green with zero parity assertions executed. Exit codes 0/1/2 never collapsed ("we could not measure" must not read as "they diverged"), mirroring litegraph-parity.yml:66-91.
Acceptance 500 fixtures asserted, not a converter-selected subset; 266 fixtures assert the empty signature set; a _poison_skew fixture whose golden is deliberately wrong in three ways has those three signatures in the manifest and the suite reports them — a differ neutered to return empty sets reddens because the expected signatures go missing; budget ≈2 m 15 s (measured components: python 5.2 s + node 2.7 s + diff ~1 s, 3× runner derate).
W4.3 — parity-main.yml: advance the pin automatically, or freeze it visibly
Repo par-mcp · Tier routine · Depends W4.2
Change Daily 08:00 UTC (after refresh-object-info at 06:00), runs the identical skew:ci against comfy-cli main. Clean → open/force-update PR ci/bump-comfy-cli-pin advancing all four pin literals to the SHA just proven green (ci-bump-pin-guard.yml already gates provenance on that branch prefix). Red → open or update exactly one issue titled [skew] comfy-cli main drift (search-by-title dedup, so repeat days edit one issue rather than filing 114), body carrying the added/removed rows, their catalogSensitive labels, and the git log --oneline <pin>..<main> range so the suspect commit set is pre-narrowed; auto-close on the next clean run. Slack arm copied from router-spec-drift.yml:196-204 including its cancelled() branch (a timeout is the same silent-failure class). Never blocking.
Why this specific shape The bump PR cannot open while the issue is red, so ignoring the alert visibly freezes the pin. That inversion is the whole mechanism: today ignoring parity costs nothing, which is how 139 commits accumulated.
Acceptance Run it against the current main and confirm it files one issue, not 139; run it against the W1.4 SHA and confirm it opens a bump PR; run it twice red and confirm it edits one issue.
Risk to gate on: if parity-main's clean check is ever narrower than the blocking job's, the pin advances on weaker evidence than the gate demands. The two must share one script, differing only in the comfy-cli ref.
W4.4 — Hand-author the two shapes the corpus cannot supply
Repo both · Tier deep · Depends W3.4, W2.3
Change The oracle is blind to two things and will report green on both forever: widgets_values_named (0 of 500 fixtures carry it) and a legacy proxyWidgets instance with a populated host widgets_values (all 377 instances across 500 fixtures carry widgets_values: [], because templates ship at defaults by construction). Author both as language-neutral JSON + expected API, flagged provenance: "hand-authored" so their verdicts are never quoted as oracle-backed. The proxyWidgets fixture must include a null hole, a trailing short array, and a previewExposure/quarantine entry occupying a tuple index, so the tuple-index-vs-slot-index distinction is actually exercised; derive the expected output from the frontend's pickHostValue + applyHostValueToInput (proxyWidgetMigration.ts:194,:257-272,:507) and confirm once in a browser. The named-map set needs three variants (complete / partial / map-only) and ideally one post-#10392-pre-#15924 save, whose sparse positional holes are the only reason the hole exclusion exists.
Acceptance Both converters held to both fixtures; a test asserting that no hand-authored fixture is counted in any oracle-derived clean-count or floor.
W4.5 — Invert the golden capture into ComfyUI_frontend's CI
Repo ComfyUI_frontend + par-mcp · Tier deep · Depends W1.7 (manifest), W4.1 (same browser tier)
Change Today's corpus cannot be regenerated — cloud.comfy.org has moved — so a golden/converter disagreement is permanently unadjudicable, and scripts/oracle-fixtures/AGENTS.md:18-36 documents the live-browser path as needing a real authenticated session with a ~64-tab limit that fails silently partway through a long run. Propose an oracle-goldens job in the frontend repo: iterate the bundled template index, app.loadGraphData() + app.graphToPrompt() per template inside its existing browser-mode harness, upload oracle-goldens-<sha>.tar.zst (~3 MB compressed) plus the manifest with frontendCommit filled in by construction. Consume it from a new refresh-oracle-goldens.yml in par-mcp, a near-clone of refresh-object-info.yml: weekly cron, gh run download in place of the blob fetch at :178-187, reusing verbatim the no-diff clean exit (:262-296), the sha256 digest guard (:605-607), and the CLOUD_CODE_BOT app-token push so the auto-PR fires required CI.
Why it is the right shape, not just convenient It makes a golden reproducible per frontend commit instead of per wall-clock instant — bisectable rather than unadjudicable. The argument that gets the PR merged is that the frontend gains a graphToPrompt() regression guard it does not have; the PreviewAny cluster is exactly the frontend-side change that would have been caught at source. Headless, the 3-second settle becomes a deterministic flush of pending widget promises and the tab limit does not exist: ~2 min for ~730 templates vs 500 × 4.5 s ≈ 37 min of flaky serialized browser time.
Acceptance The gating spike first (see Open Questions Q13): one frontend browser-mode test that loads templates/default.json and asserts graphToPrompt().output deep-equals this corpus's templates-default/expected_api.json under a stubbed object_info. If app cannot boot headless, fall back to Playwright against the repo's own vite preview — still inside the frontend's CI, still no cross-org auth. The consuming cron must ::error:: on "no artifact in the last N runs" rather than exit clean — that exact failure mode is what produced the 139-commit pin rot.
Risk to gate on New goldens captured from a frontend that has itself regressed bake the regression in. The refresh PR body must print the per-fixture before/after delta, and any fixture moving clean→dirty or dirty→clean needs a one-line reason.
W4.6 — Grow the corpus to the current template index
Repo par-mcp · Tier routine · Depends W4.5
Change Against data/templates.json with prefix normalization: 277 of 727 index templates have no fixture (38%), and 50 fixtures name templates no longer in the index (api_openai_dall_e_2_t2i, api_stability_ai_*, api_veo2_i2v, the gsc_*/gsl_* set). The uncovered set is where all the new product is (image_flux3_klein_*, image_qwen_image_2_1_*, api_nano_banana_2_1_*, image_moss_image_*). Capture the missing 277; delete the 50 orphans with the reason recorded rather than skip-listing them (BE-11949 set that precedent). The 13 golden classes absent from the bundled catalog — OpenAIDalle2/3, Stability*, LtxvApi*, Kling*VideoEffectNode, VeoVideoGenerationNode — are withdrawn provider nodes, i.e. product churn that the catalog pin does not rescue; they need recapture or deletion, not a tolerance.
Acceptance coverageGap = uncovered / indexTemplates drops below 0.10; every absolute floor is re-expressed as a rate in the same PR that grows fixtureCount 500 → ~730 (never both an absolute re-baseline and a corpus growth in separate PRs); a staleness metric emitting goldenAgeDays, driftRate, coverageGap with a freshness SLA (recapture at goldenAgeDays > 45 ∨ driftRate > 0.10 ∨ coverageGap > 0.25 — all three breached today) and a hard build failure at goldenAgeDays > 90 ∨ driftRate > 0.35, so the corpus can never silently return to its current state.
Activity
christian-byrne commentedon Oct 28, 2026
5. Explicitly NOT doing
graphToPrompt()into a shared packagegraph.serialize(),getInnerNodes(),resolveInput(),serializeValue()and every extension'sonNodeCreated. A "thin extraction" drags litegraph and the whole frontend. The reusable part is 11 lines all three headless implementations already have right. The frontend's contribution is data and goldens — a larger contribution than its code would have been, because the data is the part nobody can derive.compressWidgetInputSlots(litegraphUtil.ts:254-307)app.graph.serialize()uncompressed (10,666 of 15,493 inputs across the 500 workflows still carry awidgetmarker) and the compressed half was never saved. It affects the reverse direction andextra_pnginfoconsumers, not the forward golden. Revisit after W4.1, with its own vector set.widgets_values_form; the package'snullPeekIsCompaniondouble-walk; #1076's padding semantics), each pinned by its own tests. Lands as a rewrite on two of three sides. Share the rules as a spec; leave the walk alone.linkIdswith budget/memo/cycle machinery keyed todefs_by_ididentity (promoted.py:450-520); the package walksproperties.proxyWidgetswith a depth guard plusfindProxyWidgetForDefSlotbecause it must also serve the reverse direction. Converging means adopting the other's graph model, against a measured ~6-line fix.IGNORABLE_DIFFS/KNOWN_DIVERGENCESPreviewAnyis the proof: shared, comfy-cli would have inherited blindness to a defect it does not have. Share the manifest (owner + verdict + sunset per row); never the tolerances.cqlRunQuery/cqlLoadGraph/cqlGetTemplateSchema/cqlApplySlots/cqlExpandVariations;applySlotsreturns save-format and the TS converter runs after). The 500-fixture oracle cannot score it at all. No source is vendored and no Go toolchain owner is identified. Tracked as its ownvendor/cqlslot-layer epic (BE-7351 ∪ BE-9295, BE-12710, BE-14955/14956, BE-11313/11501, BE-8892). W3.5 closes the user-visible half in TypeScript.resolveSlotindex fix as the first cql movepromoted.py:17-19) and the mcp converter (converter.ts:710-719) all make the host win — and still cannot serve the five-1rows, which have no interior id.slot_overridesin favour ofinput_overridesget_template_schema's manifest is the only discovery surface for these knobs, and the read-only-tool rule forbids denying an input that is real in the saved graph. Removing the writer while advertising the addresses makes the dead end permanent.flush_proxy_migrationintoconvert_ui_to_api(#1070's blocking spike)proxyWidgetsinstance in all 500 fixtures: 0-1 inputs created per workflow, outputs unchanged, all 9 cluster signatures still present. The spike tests a falsified hypothesis.Graphper conversioncomfy runholds no Graph on either path (run/__init__.py:277,:1258) and would newly pay 0.53-0.7 s. Use the lazy catalog view (ctor 1.6 µs; all 591 distinct classes warmed in 0.079 s total).LoadAudio.audioUIis the single corpus entry needing a template, and hand-coding it per language is cheaper than a language.submodule initof a 28 MB tree in every comfy-cli job and par-mcp would keep its in-repo copy anyway (its suite resolvesfixtures/templatesby relative path). An unversioned artifact has no lockfile and no offline CI, reproducingannotations_source.py's own documented no-upstream-pinning limitation where it buys nothing.LoadVideo.video-previeworPreviewAnyto any converter registryvideo-previewis undecidable (21 goldens carry it; 23 with identical inputs do not) and belongs in the harness tolerance list;PreviewAnyover-emits on 148 of 148.class_typeand an input keyedUNKNOWN— faithful frontend behaviour for an unregistered type. The prompt is unexecutable either way. Preserve the value if cheap; do not pursue byte parity.6. Open questions
Each with the experiment that settles it. None of these is resolved below; where a plausible answer exists it is labelled as unverified.
On the widget walk and injected widgets
inputs[]order is not a reliable record ofwidgets_valuesorder. Experiment: enumerate corpus nodes where the savedinputs[].widgetmarker order differs from schema order, and for each check whether the golden's values match the marker order or the schema order. If marker order wins on every instance, the objection is answerable with evidence rather than argument; if any instance contradicts it, W3.1 must emit marker names without reordering.upload? No golden emits it, so the current frontend setsoptions.serialize === false— but older saves persisted it intowidgets_values(a literal{type:"button",options:{serialize:false}}object onLoadImage,nullonLoadAudio). Is the rule "never emit a widget namedupload" or the more general "never emit a widget whoseoptions.serializeis false"? The general rule is preferable but needs a source for that flag in the null-slot case, which only a name list provides. Experiment: check whetheruploadImage.ts/uploadAudio.tshave always setserialize === false, or whether theLoadAudionull slots come from a window when they did not.add_nodebuild aSaveAudioMP3whose array is one slot short of what the canvas expects? comfy-cli'sFRONTEND_MARKER_SLOTSand theengine.py:1956-1958comment classifyaudioUIas a non-serializing marker with no default and no value, which 35 goldens contradict (audioUI=""). Experiment:comfy workflow add-node SaveAudioMP3, compare the emittedwidgets_valueslength against the 35 corpus nodes (all length 2 with the marker present — so the canvas may tolerate the short array, which would make this benign; unverified).audioUIclass set? comfy-cli covers six classes (LoadAudio,SaveAudio,PreviewAudio,SaveAudioMP3,SaveAudioOpus,SaveAudioAdvanced), the package two, and the corpus contains nodes for only two. The other four are unverified in either direction. Experiment: place one node of each of the four on a dev server, save, and read the emitted array.UNKNOWNkey?unknownWidgetName(errorNodeWidgets.ts:12-14) should produceUNKNOWN,UNKNOWN_1… andaddWidgetdoes append, yet all five golden placeholder nodes carry exactly oneUNKNOWNholding the lastwidgets_valuesentry, including 4- and 6-value nodes. The file is byte-identical at the capture commit, so it is not a version difference. Experiment: loadutility_face_detection_mediapipein a frontend dev build with the mediapipe pack unregistered and inspectnode.widgets.map(w => [w.name, w.value])afterconfigure. Matters only if exact parity on unknown types is a goal.widget_Nfallback for unknown node types be kept, aligned to the frontend'sUNKNOWN/UNKNOWN_i, or dropped? It preserves values but produces an unexecutable prompt either way, and the same fallback drives the separateOpenAIDalle2/StabilityStableImageUltraNodeclusters (uninvestigated; likely the same withdrawn-node family). Experiment: replay those clusters against the capture-epoch catalog — if they dissolve, they are drift and the question is moot.On classification
NUMBERa widget in the current frontend?engine.py:36says yes,workflow_to_api.py:1838says no, and no fixture covers either (23 inputs across 17 WAS/ComfyMath classes, none in the 500 templates). Experiment: query the live frontend'swidgetStore.inputIsWidget('NUMBER'), or install WAS Node Suite, place aNumber Operation, save, and read whethernumber_aoccupies a slot. Change neither side until the probe confirms.widgetTyperule actually corrupt output, or is it masked? All 43widgetTypeinputs in the corpus are either connected or default-equal (Preview3D.model_fileis linked in all 14 fixtures; the 32FLOAT,INTcases are linked). Experiment: hand-build a workflow with an unconnectedPreview3D.model_filecarrying a non-default string plus a later widget, run both converters, and check whether the package left-shifts the following slot. Expected to left-shift — which would promote this from a rule gap to a 42-input correctness defect — but unverified.COLOR/COLORCODE/IMAGECOMPAREinputs actually mis-align in comfy-cli? No corpus fixture exercises them; the failure mode would be a left-shift of every later widget. Experiment: build a minimal workflow with a Painter or RMBG node (10 and 16 affected inputs in the snapshot) and diff comfy-cli against the package.ColorToRGBIntcarry twowidgets_valuesslots (["#0a9299","#ffffff"]) for one declared widget, with the golden emitting the second value — the schema default — rather than the user's? Both converters emit slot 0 and both are wrong. The node sits in a subgraph with the input relabeledcolor_tint, which smells like subgraph widget promotion rather than classification. Experiment: place a bareColorToRGBIntoutside any subgraph, set a color, save, and count slots. Separately: did the capture server's pre-rename definition declare a second widget ahead ofcolor? No snapshot we hold contains it, so it needs ComfyUI core git history or a pinned pre-0.22.0 catalog.On
widgets_values_namedComfy.Workflow.NamedValuesRestoreon the frontend restores the stale named value; with it off (the default) it restores the positional one andgraphToPromptthen agrees with comfy-cli. There is no golden that settles it — which is the argument for fixing the writer (W2.4) first and treating any surviving disagreement as unrepresentable rather than choosing a winner.widget.serialize === falsewidget own aninputs[].widgetmarker? If so, any saved-order walk lists a name owning no slot on a post-#15924 save and would reject the order for the wrong reason. Experiment: grep the frontend for aserialize === falsewidget that is also input-backed, or construct one and runserializeFromStoreState.On the promoted-subgraph path
proxyWidgetsinstance with a populatedwidgets_values? All 377 instances across 500 fixtures carry[], and templates ship at defaults by construction. Experiment: run the same census over the 25 public Hub workflows behind UI→API conversion ignores host values on a legacy proxyWidgets subgraph instance (convert path never runs flush_proxy_migration) #1070. If that set is also zero, UI→API conversion ignores host values on a legacy proxyWidgets subgraph instance (convert path never runs flush_proxy_migration) #1070's host-value half is theoretical and W2.3 is the whole of the real-world impact.applyHostValueToInputthe frontend's only writer of a legacy host value, or doesgraphToPromptlater re-read the host's positional array for an instance migrated this session? My reading says the widget store is the single source after the flush (proxyWidgetMigration.ts:398-399), which would mean a value quarantined asprimitiveBypassFailedis silently lost at submit — unverified. Experiment: load a crafted legacy workflow and diffgraphToPrompt()against the pre-migration host array._quarantine_entrypreserveshostValue,promoted.py:1978-1983;host_valuereads it back,:785-797); whether the frontend's submit path honours it is unverified.object_info? Gated and ungated forms are byte-identical on this corpus (all 12 markerless targets are declared), so the corpus cannot decide it. Experiment: a custom node missing fromobject_infoseparates them. Ship the gated form meanwhile.On the corpus, the harness, and the generator
/tmp/skew/threeway.jsonwas measured at comfy-clic272a5e, predating fix(workflow): do not synthesize an optional widget the save never carried #1076, so its padding-cluster rows are already stale.appheadless with a mock/api/object_info? This gates W4.5 entirely. Experiment: in aComfyUI_frontendcheckout, write one browser-mode test loadingtemplates/default.jsonand assertinggraphToPrompt().outputdeep-equals this corpus'stemplates-default/expected_api.jsonunder a stubbed catalog. Pass → the job is mechanical. Fail → fall back to Playwright against the repo's ownvite preview.options.serializeread at mount time the same as at submit time?graphToPromptreads it after whatever the user did; an extension that flips it on interaction would be mis-captured by the generator. No corpus evidence of one exists. Experiment: the export should additionally record whetherserializeValue != null, so a dynamic case is at least visible; then grep the extension set for any write tooptions.serializeoutsideonNodeCreated.scripts/oracle-fixtures/AGENTS.md:17-36documents a real browser against cloud.comfy.org, a ~64-tab limit, and a 3 s settle per template, and CI cannot do it. If re-capture is permanently off the table, the capture-epoch catalog pin is mandatory, not merely preferable, since the drift only grows.Comfy-Org/cql'sresolveSlotshare the package's positional fallback or comfy-cli's link-id resolution? The repo is not checked out here, so the relationship is inferred from issue text. Experiment: readnodegraph/runtemplate.goand check whether it indexesproxyWidgetsby ordinal or looks the entry up from the definition's IO links.Comfy-Org/cqlcommitter with a Go 1.24.6 checkout exist to take the upstream fix? Unreachable from here. The answer decides whether that workstream is schedulable at all — if not, W3.5's TypeScript path is the only lever and should be stated as such rather than left implicit.comfy-cloud-mcp-serveris private andcomfy-cliis public (verified), and fork PRs get no secrets — so the corpus must be published publicly or comfy-cli's gate cannot exist on contributor PRs. The templates are already public via cloud.comfy.org and the goldens are derived from them, so I see no new disclosure, but this needs one line of confirmation from whoever owns the private-repo boundary. It is a hard constraint on W4.2's comfy-cli mirror, not a preference.Dependency summary
Wave 1 is ~1.5 engineer-weeks and recovers 142 fixtures of signal plus three merged PRs. W1.6 alone (
+142) is the single highest-leverage change in the roadmap and needs no new team. W4.5 and W4.6 are the only items requiring another team.christian-byrne commentedon Oct 28, 2026
Companion ADR draft
Proposed for
comfy-cli/docs/adr/, as the repo whose converter it most constrains, and mirrored by reference fromcomfy-cloud-mcp-server. Not yet filed as an ADR PR — posting here first so the decision can be argued before it is recorded.ADR-CONVERSION-0001: Own the Save-to-API Contract as Shared Data and Goldens, Not Shared Code
Date: 2026-10-28
Status
Proposed
Context
Converting a saved workflow to the API prompt the backend executes is
implemented four times: the frontend's
graphToPrompt()(
ComfyUI_frontend/src/utils/executionUtil.ts),@comfyorg/api-to-workflow(sourced from
comfy-cloud-mcp-server/src/converter/), comfy-cli'sconvert_ui_to_api(comfy_cli/workflow_to_api.py), and — for the adjacentsave-format slot-write contract —
Comfy-Org/cql. The frontend is groundtruth because it is the only one with the live LiteGraph runtime.
The whole of
graphToPrompt()'s widget contract is eleven lines: iteratenode.widgets, skip a widget with no name oroptions.serialize === false,apply
serializeValue, wrap arrays. Everything else in that function is linkresolution and DTO flattening. The only fact a headless converter cannot
derive is therefore the content and order of
node.widgetsfor a given class.That fact is a per-class property of the installed frontend extensions, and it
is currently approximated by hand three times — as TypeScript tables in the
package, as
object_info-flag predicates in comfy-cli, and (unread here) inGo. Each approximation has drifted, in opposite directions, undetected: the
package's
PreviewAnyentry emits three keys that no golden carries and citesa source file that no longer exists, while comfy-cli covers six
audioUIclasses to the package's two and emits no injected widget at all.
Drift was undetectable for three structural reasons. The only cross-
implementation job compares two implementations to each other rather than to
the oracle, so it is silent whenever both are wrong — and the two share a
lineage, making correlated error the likely kind. It pins comfy-cli to a SHA
that is 139 commits off main and not an ancestor of it, and asserts only that
the pin matches itself. And the 500 goldens, captured 2026-09-11, are
evaluated against an
object_infothat a daily cron rewrites, so 28% of thecorpus fails for catalog movement alone, teaching reviewers to read defects as
staleness.
Decision
The save-to-API contract is owned as versioned data plus a golden corpus,
generated from the frontend and consumed by every other implementation. It is
not owned as shared converter code.
widget-layout.json(per class, theordered
node.widgetswith each widget's slot index,promptEmitderivedverbatim from
options.serialize !== false, and an optionalserializeValuein one of three closed forms) and
widget-type-census.json(the registeredwidget-type set, which is literally
useWidgetStore().widgets) are emittedby jobs in
ComfyUI_frontendCI. A hand edit to either is rejected.classifier-vectors.jsoncarriesthe ordered classification rule procedure as executable cases with a
per-implementation
failsTodayledger that may only shrink.tolerances.jsoncarries corpus capture artifacts only; its schema requires the
implementation-gap field to be null.
object_infopinned to them, published as a versioned artifact so the public repo can
consume it without a private checkout or a secret.
re-implemented per language (~150 LOC each) rather than shared as code or a
subprocess. The signature grammar, the normalization rules, the verdict
ladder, and a dozen
(ours, golden, expected-signatures)triples everykernel must reproduce byte-exactly are normative.
the promoted-subgraph resolver, not
graphToPrompt()itself.next to its own tests. Tolerance lists are never shared.
job proves the other implementation's main branch green and opens the bump
PR; while it is red the pin cannot advance, so ignoring the signal visibly
freezes the dependency instead of costing nothing.
Alternatives considered
graphToPrompt()into a shared headless package. Rejected: itssemantics live in
graph.serialize(),ExecutableNodeDTO.getInnerNodes(),node.resolveInput(),widget.serializeValue()and every extension'sonNodeCreated. A thin extraction would drag litegraph, every widgetconstructor and every extension — the whole frontend. The genuinely reusable
part is the eleven-line widget loop, which all three headless
implementations already have correctly.
exist (comfy-cli's strict/lenient companion heuristic plus its
widgets_values_formrepresentation; the package'snullPeekIsCompaniondouble-walk; the optional-padding semantics), each pinned by its own tests.
A shared walk lands as a rewrite on two of three sides.
the definition's declared link ids with budget, memo and cycle machinery
keyed to object identity; the package walks
properties.proxyWidgetswith adepth guard because it must also serve the reverse direction. Converging
means one side adopting the other's graph model, against a measured ~6-line
fix that closes the actual gap.
encodes one implementation's blind spot. Had the package's list been shared,
comfy-cli would have inherited a blindness to a defect it does not have.
implementations is N² comparisons with no truth side and is silent when both
are wrong. Oracle-vs-N is N comparisons with one.
an unrelated commit in one repo would redden an unrelated PR in the other,
and the same commit would pass or fail by wall clock. The ancestry-verified
pin plus a scheduled advancing job keeps determinism and bounds staleness;
neither half alone does.
evidence: both existing hand-authored tables are measurably wrong today, in
opposite directions, and neither was checked against the corpus by anything.
n≈3: three closed forms plus a named escape-hatch list.
real authenticated browser against a production surface with a tab limit
that fails silently partway through a long run. The capture-epoch catalog
pin is the substitute, and it is better — it makes the corpus measure
converter logic rather than N days of upstream churn.
Consequences
Positive
disagreeing with the frontend — becomes unrepresentable, because the table
is generated by the frontend.
than by a hand-written predicate, so a reviewer can no longer relabel one as
the other.
automated with a visible cost for ignoring it.
another's blind spot.
graphToPrompt()regression guard it does not have.Negative
outside it get no entry, and a missing entry must mean "no injected
widgets" — today's wrong-for-twelve-classes default. Honest coverage
downgrade for unmanifested packs in exchange for correctness on manifested
ones.
two publish channels on every release.
slot-indexed layout assumes a class's widget set is static at creation.A class whose extension branches on a widget value at creation time cannot
be expressed and stays a named per-language escape hatch.
widgets_values_named,populated legacy
proxyWidgetshost arrays). Those need hand-authoredfixtures whose verdicts must never be quoted as oracle-backed.
Testable invariants
object_infosnapshot appears in the censusin exactly one bucket; an unknown type fails with its usage count and three
example
Class.inputsites.synthetic spec of every census type, and with every
classifier-vectorscase not listed in its own
failsToday.snapshot or is tagged injected with its injecting class list.
upload.PreviewAnyemits exactlysourceon all corpus instances.an extra signature and a resolved-but-still-listed signature both fail.
catalogSensitiveis computed from the two-catalog run and must equal themanifest's stored value.
corpus, is a hard failure.
pending tests — a wholesale skip reddens.
exactly those three signatures; a kernel neutered to return empty sets
fails because the expected signatures go missing.
implementation fails the agreement check.
exactly.
pinned version.
and floor.
Revisit triggers
widget whose
options.serializeorserializeValuedepends on runtimeinteraction rather than class identity, which would break the data-only
premise and is the risk this ADR is least able to detect.
serializeValueform, or the escape-hatch list exceeds roughly five entries; at that point the rejected
expression-language alternative should be reconsidered.
manifest backing the export degrades.
cqladoptsADR-SUBGRAPH-PROMOTION-0009's host-wins value model, at which point the
slot-write contract may be foldable into this one.
catalog pin optional rather than mandatory.
tolerance is added without a cited fixture and a sunset date.
christian-byrne commentedon Oct 29, 2026
Program tracking
This epic now has a project tracking it end to end: Save-to-API Converter Convergence (groups the four existing tickets that each describe one face of the same contract — the positional
widgets_valuescontract itself, the slot-address bug that writes the wrong interior widget, the widget-catalog drift check, and the triage of the October Hub audit).One correction worth recording before the next item is sized
The roadmap behind this epic assumed
widgets_values_namedwas a future design whose read side could wait, on the evidence that 0 of 500 corpus fixtures and 0 of 11,025 nodes carry it. That inference was wrong, and in a direction that changes what to build.Checked against
ComfyUI_frontendmainatcfa7aa59c4:serialiseWidgetValueshas emittedwidgets_values_namedunconditionally since 2026-07-30 (#10392), with no feature flag, andcreateWidgetRestorationStateprefers it over the positional array when loading.@comfyorg/api-to-workflowalready reads it, resolving each widget by name and falling back to the positional slot only on a miss (src/converter/widgetMapper.ts:832, with callers at:878,:1661,:2103).comfy_cli/cql/engine.py:5113,:5167,:5303). The convert path ignores it.The field is latent in the corpus, not in the format. The corpus is committed template files, not fresh saves, so every real save since July carries a name→value map that no fixture does.
Two consequences:
fix(workflow): read a marker list as coverage, not as raw names