Skip to content

Commit 5b247dd

Browse files
PhysShellclaude
PhysShell
and
claude
committed
docs(evidence): archive the #400 dictionary-ref PoC (KILL)
Owner ruling: ACCEPT KILL / ARCHIVE AS SEPARATE RESEARCH PR (STOP-PRODUCTION / KEEP-EVIDENCE). The bounded Roslyn PoC found both real stale-ref defects (Meziantou, MAUI), kept both fixed revisions and the mandatory controls silent, but failed the pre-registered >= 80 % precision gate on two independent samples: v1 9/5/4 (64 %), v2 8/3/3 (73 %). Evidence only: frozen probes v1/v2, controls, sample lists pinned by blob SHA, every warning with its label, target/control outputs and the scripts to re-run them. No third-party source, no production, OwnIR, CI or golden change. A third round is not authorised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 3d1d174 commit 5b247dd

34 files changed

Lines changed: 5077 additions & 0 deletions

‎docs/evidence/dictionary-ref-poc-2026-10/.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
samples/
2+
targets/
3+
build/
4+
probe-*/bin/
5+
probe-*/obj/

‎docs/evidence/dictionary-ref-poc-2026-10/PRECISION-PREREG.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Dictionary-ref PoC — precision gate pre-registration (written before the independent sample was fetched)
2+
3+
- Independent sample: GitHub code search, queries different from the development sample
4+
(`docs/evidence/next-bug-class-2026-10/data/sample-index.tsv`); files whose repo+path appear in the development
5+
index are excluded, and so are meziantou/Meziantou.Framework and dotnet/maui (the targets).
6+
- Each file is analysed alone (single-file compilation, implicit usings on), exactly as the probe runs on the
7+
targets. Probe version = the build that passed controls and both targets; it is frozen before labelling.
8+
- Every warning is labelled by reading the source:
9+
- TP: on some execution the event can mutate the same dictionary while the ref is live (direct mutation, or
10+
code reachable from the event that can reach the dictionary and change it);
11+
- FP: the event provably cannot mutate that dictionary;
12+
- UNCLEAR: cannot decide from the file.
13+
- Precision = TP / (TP + FP); conservative precision = TP / all warnings (UNCLEAR counted as FP). Target >= 80 %
14+
on the conservative figure.
15+
- Fewer than 10 warnings in total -> INCONCLUSIVE (insufficient data), regardless of the ratio.
16+
- No probe change after this point except to fix a crash; any change re-runs controls, targets and the sample.
17+
18+
## Round 1 result (probe sha256 cd49e9a5…, recorded before any change)
19+
20+
18 warnings / 274 sites / 429 files: TP 9, FP 5, UNCLEAR 4 -> precision 64 %, conservative 50 %. **Gate FAILED.**
21+
FP causes: (a) the event is followed by `return`/`throw` in its own block, so it cannot reach the later use (3);
22+
(b) a method-local dictionary that escapes only AFTER the event is treated as reachable (2).
23+
24+
## Round 2 pre-registration (written before the round-2 sample is fetched)
25+
26+
- Exactly two probe changes, both principled and both answering a round-1 FP cause:
27+
(a) an event whose own block, after it, unconditionally leaves (`return`/`throw`) before the block ends does not
28+
reach a use outside that block;
29+
(b) a method-local dictionary is reachable at an event only through an escape (capture, argument, alias, return)
30+
that occurs textually BEFORE the event.
31+
- Both shapes become mandatory negative controls; all round-1 controls and both targets must still pass.
32+
- The UNCLEAR rule is NOT changed (no tuning toward opaque delegates).
33+
- Fresh sample: new code-search queries; files already in the development sample or the round-1 sample, and the
34+
two target repos, are excluded. Same labelling rules, same thresholds (>= 80 % conservative, >= 10 warnings).
35+
- Round 2 is the last round within this PoC budget; its result is final.

‎docs/evidence/dictionary-ref-poc-2026-10/REPORT.md‎

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
# #400 dictionary-ref pilot — bounded Roslyn PoC
2+
3+
**Verdict: `KILL-DICTIONARY-REF-PILOT`.** Owner ruling: `ACCEPT KILL / ARCHIVE AS SEPARATE RESEARCH PR`, outcome
4+
`STOP-PRODUCTION / KEEP-EVIDENCE`.
5+
6+
The pre-registered precision gate (≥ 80 %) failed on both independent samples:
7+
- round 1, probe v1: 64 %;
8+
- round 2, probe v2: 73 %.
9+
10+
The two confirmed real defects were found, and the controls passed. Both are kept as positive research evidence.
11+
They do not authorise a production implementation.
12+
13+
No production code, OwnIR, CI or golden file was changed. No implementation PR exists. A third round is not
14+
authorised (§7).
15+
16+
Context: the research report and the decision `GO-400-DICTIONARY-PILOT` are in
17+
[`../next-bug-class-2026-10/REPORT.md`](../next-bug-class-2026-10/REPORT.md) (PhysShell/Own.NET#412).
18+
19+
## 1. What was tested
20+
21+
A ref from `CollectionsMarshal.GetValueRefOrAddDefault` or `GetValueRefOrNullRef` into dictionary `D` becomes stale
22+
when `D` is structurally mutated while the ref is live. A resize moves the entries, and the ref still points into
23+
the old array. The PoC targets only this mechanism. It does not target the second Meziantou defect (default left
24+
inserted when the factory throws).
25+
26+
The probe is a standalone Roslyn console app, outside Own.NET production code. It uses
27+
Microsoft.CodeAnalysis.CSharp 4.9.2 on net8.0, with the running runtime's assemblies as references.
28+
29+
It warns when the ref local is read or written after a hazard event. A write in `x = f()` counts, because the
30+
location of `x` is evaluated before `f` runs. Matching is by symbol.
31+
32+
| Hazard event | Condition |
33+
|---|---|
34+
| direct | `Add`/`TryAdd`/`Remove`/`Clear`/`EnsureCapacity`/`TrimExcess` on the same `D`, an indexer write, or another `GetValueRefOrAddDefault(D, …)` |
35+
| delegate | a delegate invocation, when code outside the method can reach `D`: `D` is a field, property or parameter, or a local that escaped (v1: anywhere in the method; v2: before the event) |
36+
| callee | a first-party method or constructor in the analysed file (depth ≤ 4) whose body has a direct or delegate event on the same field or property `D` |
37+
38+
Not hazards:
39+
- metadata (BCL) calls;
40+
- constructors and first-party callees without events;
41+
- mutation of a different dictionary;
42+
- an event on a branch exclusive with the use;
43+
- a ref re-bound with `x = ref …`;
44+
- v2 only: an event followed by `return`/`throw` in its own block that does not contain the use.
45+
46+
## 2. Frozen artifacts
47+
48+
| Artifact | Identity |
49+
|---|---|
50+
| Probe v1, round 1 | [`probe-v1/Program.cs`](probe-v1/Program.cs), sha256 `cd49e9a5…` |
51+
| Probe v2, round 2 (final) | [`probe-v2/Program.cs`](probe-v2/Program.cs), sha256 `536762c6…` |
52+
| Controls | [`controls/Controls.cs`](controls/Controls.cs), sha256 `444f2bb2…`. N8 and N9 were added with v2 |
53+
| Pre-registration | [`PRECISION-PREREG.md`](PRECISION-PREREG.md) (verbatim). Round-1 rules were written before the round-1 sample was fetched. The round-1 result and the round-2 rules were written before the round-2 sample was fetched |
54+
| Targets | meziantou/Meziantou.Framework `7addb26c` (buggy) and `cf82931e` (fixed); dotnet/maui `7b051d1b` (buggy) and `2db8b6af` (fixed). One file each, fetched by commit SHA |
55+
| Samples | [`data/r1/index.tsv`](data/r1/index.tsv): 429 files, 183 repos. [`data/r2/index.tsv`](data/r2/index.tsv): 252 files, 189 repos. Pinned by blob SHA. Raw search hits are in `raw.tsv`/`all.tsv`; the derivation is in `scripts/make_index.py` |
56+
57+
Full hashes are in [`SHA256SUMS`](SHA256SUMS). No third-party source is committed.
58+
59+
## 3. Gates
60+
61+
| Gate | v1 | v2 |
62+
|---|---|---|
63+
| Meziantou buggy: `DictionaryExtensions.cs:31` (`GetOrAdd`) and `:56` (`TryUpdate`) | found | found |
64+
| MAUI buggy: `BindableObject.cs:793`, chain `CreateContext → defaultValueCreator(this)` | found | found |
65+
| Meziantou fixed (2 sites) | 0 warnings | 0 warnings |
66+
| MAUI fixed | 0 sites: the fix removes the API from that method, so "clean" is trivial | same |
67+
| Positive controls P1–P4 | 4/4 | 4/4 |
68+
| Mandatory negative controls: N1 no mutation; N2 a different dictionary; N3/N3b/N1 safe factory (constructor, pure first-party, BCL); N4 ref used only before the mutation | silent | silent |
69+
| Further negatives: N5 local dictionary; N6 exclusive branches; N7 fresh ref | silent | silent |
70+
| Round-1 FP shapes: N8 `return`/`throw` after the mutation; N9 a local that escapes later | **N8 flagged** (the shape v2 fixes); N9 silent | silent |
71+
| Precision ≥ 80 % on an independent sample | **FAIL** (64 %) | **FAIL** (73 %) |
72+
| Third-party analyzers (Meziantou.Analyzer, Roslynator, ErrorProne.NET) | NOT_TESTED: package download not authorised. This is not a MISS | — |
73+
| Built-in NetAnalyzers (`AnalysisMode=All`, #412) | no rule fires on these shapes | — |
74+
| OwnIR reuse | not attempted: the relation "a mutation invalidates the borrow" has no OwnIR form today, and `release`/`use` would be a false semantic claim | — |
75+
76+
N9 is silent under v1 too, so this control does not separate v1 from v2. The evidence for v2 change (b) is the two
77+
round-1 false positives F17/F18 themselves.
78+
79+
Outputs: [`data/targets.v1.out`](data/targets.v1.out), [`data/targets.v2.out`](data/targets.v2.out),
80+
[`data/controls.v1.out`](data/controls.v1.out), [`data/controls.v2.out`](data/controls.v2.out).
81+
82+
## 4. Precision, as measured
83+
84+
Labelling rules were fixed before each sample was fetched:
85+
- **TP**: on some execution the event can structurally mutate the same dictionary while the ref is live.
86+
- **FP**: the event provably cannot.
87+
- **UNCLEAR**: this cannot be decided from the file.
88+
89+
Precision = TP / (TP + FP). Conservative precision counts UNCLEAR as FP.
90+
91+
| Round | Probe | Files / repos | Sites | Warnings | TP | FP | UNCLEAR | Precision | Conservative | TP repos |
92+
|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|
93+
| 1 | v1 | 429 / 183 | 274 | 18 | 9 | 5 | 4 | **64 %** | 50 % | 6 |
94+
| 2 | v2 | 252 / 189 | 290 | 14 | 8 | 3 | 3 | **73 %** | 57 % | 7 |
95+
96+
These are the measurements as taken:
97+
- no warning was relabelled after the fact;
98+
- v2 was not re-scored on the round-1 sample;
99+
- round 1 is not discounted because v2 fixed some of its causes.
100+
101+
Every warning, with its blob SHA, lines, event, label and reason, is in [`data/r1/findings.tsv`](data/r1/findings.tsv)
102+
and [`data/r2/findings.tsv`](data/r2/findings.tsv). The labels as written during labelling are in `data/r*/labels.tsv`.
103+
104+
Causes:
105+
- Round-1 FP (5):
106+
- an event followed by `return`/`throw` (3: F11, F14, F15);
107+
- a method-local dictionary that escapes only after the event (2: F17, F18).
108+
- Round-2 FP (3):
109+
- a callee's delegate mutates a different, passed-in dictionary (G1);
110+
- an indexer write to the key just added, which is not structural (G2);
111+
- an event followed by `continue` (G7).
112+
- UNCLEAR (4 + 3): opaque non-factory delegates such as a TTS callback, an account getter, a logger provider, a
113+
terrain query or an AABB extractor.
114+
115+
### Contract violation vs observable defect
116+
117+
[`data/effect.tsv`](data/effect.tsv) splits the 17 TPs. It was written after the owner ruling, from the reasons
118+
recorded at labelling time. No label changed.
119+
120+
| Class | Count | Meaning |
121+
|---|---:|---|
122+
| PROVEN | 0 | none of the sample TPs was reproduced at runtime |
123+
| CONTRACT_ONLY | 3 (F5, G9, G11) | the ref contract is broken, but no observable effect on the current runtime: a read of an unchanged reference slot; `Remove` of another entry, which does not move entries; a lost cache write that only causes a recompute (dotnet/runtime `RegexNodeConverter`) |
124+
| PLAUSIBLE | 14 | an effect needs a factory, loader or callback that reaches the same dictionary. That is possible from the code but not demonstrated. 8 of the 14 are generic `GetOrAdd(factory)` helpers whose defect depends on the caller |
125+
126+
Observable defects proven at runtime exist only for the two targets: #412,
127+
[`../next-bug-class-2026-10/data/c_runtime_dictionary.out`](../next-bug-class-2026-10/data/c_runtime_dictionary.out).
128+
129+
## 5. Limitations
130+
131+
- **Symbol resolution.**
132+
- Each file is compiled alone, with SDK implicit usings and runtime references only.
133+
- Types from the file's own project or packages are often unresolved.
134+
- When the arguments do not bind, a single `CollectionsMarshal` candidate is accepted by name. Without this, the
135+
MAUI target had 0 sites.
136+
- Unresolved dictionaries and receivers can hide sites and events.
137+
- The `sites` counts are counts of what resolved, not of what exists.
138+
- **Callee analysis.**
139+
- Only methods and constructors whose body is in the same file are followed, to depth 4.
140+
- Only events on a field or property `D` are propagated.
141+
- Partial classes, base classes, other files, interface and virtual dispatch are not seen.
142+
- **Unknown calls.**
143+
- Metadata calls are never events, by design (precision-first).
144+
- A BCL or package call that reaches `D` through a callback is missed.
145+
- Delegates are treated by reachability alone; an opaque delegate is either flagged (UNCLEAR) or not, never
146+
analysed.
147+
- **Sample.**
148+
- GitHub code search returns at most 100 hits per query, over default branches, ranked by GitHub. It is not a
149+
random sample of C#.
150+
- The query sets differ between rounds and are biased toward files that name the API.
151+
- Search results drift, so `scripts/search.sh` will not reproduce the lists; the committed `index.tsv` files are
152+
the samples.
153+
- The two target repos and the #412 development sample are excluded; round 2 also excludes round 1.
154+
- **Labelling.**
155+
- One researcher labelled every warning by reading one file.
156+
- There was no second labeller and no blind control.
157+
- Labels are judgements about reachability, not runtime results.
158+
- **Silence is not safety.** A file without a warning is not shown to be free of stale refs. The probe measures
159+
precision only; recall outside the two targets and the controls was not measured.
160+
161+
## 6. Plausible hazards found (not reported upstream)
162+
163+
- uno.extensions `StateStore` (G8);
164+
- CalcpadCE `MathParser.CustomFunction` (G12, G13);
165+
- dotnet/runtime `RegexNodeConverter` (G11, benign).
166+
167+
They are recorded here as labels only. Nothing was reproduced or reported.
168+
169+
## 7. Post-hoc observation — not evidence, no further work authorised
170+
171+
In both samples, warnings of the exact form `ref = factory(…)` were labelled TP in nearly every case: the ref
172+
written with the result of a delegate, as in Meziantou. The FP and UNCLEAR cases came from other event kinds.
173+
174+
This subset was selected after labelling, on the test data. It was not pre-registered and was not measured on an
175+
unseen sample. It is not a precision claim.
176+
177+
Per the owner ruling, a third round, including a test of this hypothesis, is **not authorised**. No implementation
178+
or measurement of it was made.
179+
180+
## 8. Reproduce
181+
182+
Needs the .NET 8 SDK (or later), Python 3 and an authenticated `gh`.
183+
184+
```bash
185+
sha256sum -c SHA256SUMS
186+
bash scripts/fetch.sh # samples/ and targets/ by pinned SHA (gitignored)
187+
bash scripts/verify_blobs.sh # every sample file = its blob SHA (r1 429/429, r2 252/252)
188+
bash scripts/run.sh # builds v1 and v2, regenerates data/, prints the tally
189+
python scripts/make_index.py ../next-bug-class-2026-10/data/sample-index.tsv --check
190+
```
191+
192+
`run.sh` ends with the two tally lines of §4:
193+
194+
```text
195+
r1: sites=274 warnings=18 files=429 repos=183 TP=9 FP=5 UNCLEAR=4 precision=64% conservative=50% TP repos=6
196+
r2: sites=290 warnings=14 files=252 repos=189 TP=8 FP=3 UNCLEAR=3 precision=73% conservative=57% TP repos=7
197+
```

‎docs/evidence/dictionary-ref-poc-2026-10/SHA256SUMS‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
cd49e9a50b8eea108836154d763d0df0b772c5508d7f74c47723a191ea8f8b09 *probe-v1/Program.cs
2+
536762c6a0c7587958ec85c7578b09bef4ed5c71b23cf4b1527c79c20da0e193 *probe-v2/Program.cs
3+
444f2bb2a5910aa03c3626388dd3436d41818e9ee244404d3877ef7ffe498e3b *controls/Controls.cs

‎docs/evidence/dictionary-ref-poc-2026-10/controls/Controls.cs‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
using System;
2+
using System.Collections.Generic;
3+
using System.Runtime.InteropServices;
4+
5+
namespace Controls;
6+
7+
public static class Positive
8+
{
9+
// P1 Meziantou shape: the caller's dictionary, the caller's factory
10+
public static TValue GetOrAdd<TKey, TValue>(Dictionary<TKey, TValue> dict, TKey key, Func<TKey, TValue> factory)
11+
where TKey : notnull
12+
{
13+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(dict, key, out var exists);
14+
if (exists) return v!;
15+
v = factory(key); // EXPECT: write after delegate
16+
return v;
17+
}
18+
19+
// P3 direct mutation of the same dictionary, then a read
20+
public static int Direct(Dictionary<int, int> d)
21+
{
22+
ref var v = ref CollectionsMarshal.GetValueRefOrNullRef(d, 1);
23+
d.Add(2, 2);
24+
return v; // EXPECT: read after direct
25+
}
26+
27+
// P4 indexer write on the same dictionary, then a write through the ref
28+
public static void IndexerWrite(Dictionary<int, int> d)
29+
{
30+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out _);
31+
d[2] = 2;
32+
v = 5; // EXPECT: write after direct
33+
}
34+
}
35+
36+
public sealed class Bindable
37+
{
38+
private readonly Dictionary<int, object?> _props = new();
39+
private readonly Func<Bindable, object>? _creator;
40+
public Bindable(Func<Bindable, object>? creator) => _creator = creator;
41+
42+
// P2 MAUI shape: a first-party callee runs the user creator
43+
public object? GetOrCreate(int id)
44+
{
45+
ref var ctx = ref CollectionsMarshal.GetValueRefOrAddDefault(_props, id, out var exists);
46+
if (!exists)
47+
ctx = Create(); // EXPECT: write after callee -> delegate
48+
return ctx;
49+
}
50+
private object Create() => _creator?.Invoke(this) ?? new object();
51+
52+
// N3b safe first-party factory on the same field dictionary
53+
public object? GetOrCreateSafe(int id)
54+
{
55+
ref var ctx = ref CollectionsMarshal.GetValueRefOrAddDefault(_props, id, out var exists);
56+
if (!exists)
57+
ctx = MakePure(id); // silent
58+
return ctx;
59+
}
60+
private static object MakePure(int id) => id.ToString();
61+
}
62+
63+
public static class Negative
64+
{
65+
// N1 no mutation at all
66+
public static void NoMutation(Dictionary<string, int> d)
67+
{
68+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, "k", out _);
69+
v = Math.Max(v, 3); // silent: BCL pure call, no mutation of d
70+
}
71+
72+
// N2 mutation of a DIFFERENT dictionary
73+
public static int OtherDictionary(Dictionary<int, int> a, Dictionary<int, int> b)
74+
{
75+
ref var v = ref CollectionsMarshal.GetValueRefOrNullRef(a, 1);
76+
b.Add(3, 3);
77+
return v; // silent
78+
}
79+
80+
// N3 safe factory (BCL / constructor)
81+
public static void SafeFactory(Dictionary<int, List<int>> d)
82+
{
83+
ref var list = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out var exists);
84+
if (!exists) list = new List<int>(4);
85+
list!.Add(1); // silent: the Add is on the list value, not on d
86+
}
87+
88+
// N4 ref used only BEFORE the potential mutation
89+
public static void UsedBefore(Dictionary<int, int> d, Func<int> f)
90+
{
91+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out _);
92+
v++;
93+
d.Add(9, f()); // silent: v not used afterwards
94+
}
95+
96+
// N5 delegate invoked, but the dictionary is a local nothing else can see
97+
public static int LocalDictionary(Func<int> f)
98+
{
99+
var d = new Dictionary<int, int>();
100+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out _);
101+
v = f(); // silent: f cannot reach d
102+
return v;
103+
}
104+
105+
// N6 event and use on exclusive branches
106+
public static int Exclusive(Dictionary<int, int> d, bool c, Func<int> f)
107+
{
108+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out _);
109+
if (c) { d.Add(2, f()); return 0; }
110+
else { return v; } // silent
111+
}
112+
113+
// N7 fresh ref after the mutation
114+
public static int Refreshed(Dictionary<int, int> d)
115+
{
116+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out _);
117+
d.Add(2, 2);
118+
v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out _);
119+
return v; // silent
120+
}
121+
122+
// N8 the mutation is followed by return / throw in its own block (round-1 FP shape)
123+
public static int LeavesAfterMutation(Dictionary<int, int> d, bool bad)
124+
{
125+
ref var v = ref CollectionsMarshal.GetValueRefOrAddDefault(d, 1, out var exists);
126+
if (!exists)
127+
{
128+
if (bad) { d.Remove(1); return -1; }
129+
try { v = 1; } catch (Exception) { d.Remove(1); throw; }
130+
}
131+
return v; // silent
132+
}
133+
134+
// N9 a method-local dictionary that escapes only after the delegate ran (round-1 FP shape)
135+
public static Dictionary<int, int> LocalEscapesLater(IEnumerable<int> xs, Func<int, int, int> f)
136+
{
137+
var dict = new Dictionary<int, int>();
138+
foreach (var x in xs)
139+
{
140+
ref var acc = ref CollectionsMarshal.GetValueRefOrAddDefault(dict, x % 3, out _);
141+
acc = f(acc, x); // silent: f cannot see dict yet
142+
}
143+
return dict;
144+
}
145+
}

0 commit comments

Comments
 (0)