Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,30 @@
What changed in PACT, newest first. Each entry says what an author can now write, or what now
holds that did not; the commits are listed under it for whoever wants the detail.

## 2026-11 — law 3 as the Bud Agent Flow corpus reads it (11 November)

Running the new checker over Bud Agent Flow's 168 shipped trees refused 29 of them, all with
`loader/repeated-with-no-claim` or `loader/a-key-that-changes-every-attempt`. Read one by one,
three of the rule's readings were wider than law 3, and each is narrowed here with a test:

- **A `together` is not repetition by itself.** Its branches are different stages, each run
once: an email in one branch and a chat message in the other are two calls. What repeats is
the *same* action called from two branches side by side, and that is still refused
(`two_branches_calling_the_same_write_do_it_twice`). The old reading also replaced the
context around a `together`, so a `together` inside a `repeat` lost the repeat; it now keeps
it (`a_together_inside_a_repeat_still_runs_every_round`).
- **The stock memory tools are not held by the rule.** `bud/remember` and `bud/forget` in a
`repeat` are how a round leaves what it learned for the next (Reflexion's buffer); the memory
is the workspace's own (`a_round_may_keep_what_it_learned`).
- **A key bound from the round's own output is a warning.** The author wrote a claim, and a
retry of one round produces the same output and so the same key. Whether two *different*
rounds aim at the same thing is an intent the checker cannot see; a rolling plan whose every
round posts a new update is the other reading, and the corpus has nine of them. A call with
no claim at all inside a `repeat` or a retry is still an error.

The other trees were real defects (writes retried or repeated with no claim) and were fixed in
Bud Agent Flow by giving each action the key its business has.

## 2026-11 — the seven primitives, built (10 November)

`docs/97-PRIMITIVES.md` names seven building blocks (Shape, Origin, Store, Actor, Flow, Check,
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -665,11 +665,11 @@ framework is an extra.
|---|---|
| **Design** | Thesis, 28 binding decisions, FRD (120 requirements), 60-row refusal ledger — complete |
| **Research** | 14 source-grounded studies, ~15,750 lines, over 140 repos (~15 GB) + 57 papers |
| **Code** | Loader, diagnostics, schema engine, CLI, harness, resolver, evals, SLO — **4683 tests (1556 Rust + 3127 adapter), clippy clean, TypeScript type-checked** |
| **Code** | Loader, diagnostics, schema engine, CLI, harness, resolver, evals, SLO — **4695 tests (1561 Rust + 3134 adapter), clippy clean, TypeScript type-checked** |
| **Adapters** | **All 7 named targets**, proven against one shared conformance suite |

```bash
./scripts/test-all.sh # 4683 tests, Rust + 7 adapters, fully offline
./scripts/test-all.sh # 4695 tests, Rust + 7 adapters, fully offline
```

**What is still owed** is not hidden — it lives in
Expand Down
25 changes: 24 additions & 1 deletion adapters/python/src/pact_adapters/at_most_once.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,10 @@
if TYPE_CHECKING:
from .claims import ClaimStore

#: `same-request-key: the-call` (law 3, narrowed 11 November 2026): the call's
#: own place in the run is its identity (`RequestKeys.by_the_call`).
THE_CALL = "the-call"

#: What `step.tool.cancelled` carries when a duplicate was withheld. One
#: spelling, because the event's `reason` is read by watches and by anyone
#: reconstructing a run, and two spellings of one outcome is how a reader comes
Expand Down Expand Up @@ -122,6 +126,11 @@ class RequestKeys:
#: How long one call's claim lasts, in seconds, per action, from
#: `claim-lasts:` (docs/98 WP-7). Absent: as long as its scope.
leases: dict[tuple[str, str], float] = field(default_factory=dict)
#: The actions that say `same-request-key: the-call`: the call's own place
#: in the run is its identity, which a host sends to the server as the
#: request's idempotency key. Nothing here holds them: two calls at two
#: places are two calls, and a retry of one is the host's to identify.
by_the_call: frozenset[tuple[str, str]] = frozenset()

def __bool__(self) -> bool:
return bool(self.keys)
Expand All @@ -140,6 +149,7 @@ def from_document(doc: Mapping[str, Any], agent_key: str) -> "RequestKeys":
scopes: dict[tuple[str, str], str] = {}
leases: dict[tuple[str, str], float] = {}
actions: dict[str, tuple[str, ...]] = {}
by_the_call: set[tuple[str, str]] = set()
for tool, spec in sorted((doc.get("tools") or {}).items()):
if tool not in used or not isinstance(spec, Mapping):
continue
Expand All @@ -151,6 +161,9 @@ def from_document(doc: Mapping[str, Any], agent_key: str) -> "RequestKeys":
if not isinstance(body, Mapping):
continue
named = str(body.get("same-request-key") or "").strip()
if named == THE_CALL:
by_the_call.add((str(tool), str(action)))
continue
if named:
keys[(str(tool), str(action))] = named
across = str(body.get("same-request-key-across") or "").strip()
Expand All @@ -159,7 +172,9 @@ def from_document(doc: Mapping[str, Any], agent_key: str) -> "RequestKeys":
lease = seconds(body.get("claim-lasts"))
if lease is not None:
leases[(str(tool), str(action))] = lease
return RequestKeys(keys=keys, actions=actions, scopes=scopes, leases=leases)
return RequestKeys(
keys=keys, actions=actions, scopes=scopes, leases=leases, by_the_call=frozenset(by_the_call)
)

def action_of(self, tool: str, args: Mapping[str, Any]) -> str | None:
"""Which action this call names, or `None` when nothing can say.
Expand Down Expand Up @@ -405,6 +420,14 @@ def unenforced(self) -> tuple[str, ...]:
nobody wrote.
"""
said = {**self.cannot_tell, **self.cannot_keep}
for tool, action in sorted(self.keys.by_the_call):
said.setdefault(
(tool, action),
f"same-request-key: {tool}/{action} says `the-call`, the call's own place in "
"the run, and this harness sends the server no request identity of its own: "
"a call it tries again after a lost reply may be made twice. A host that "
"keeps every call (`durability:`) sends one.",
)
return tuple(said[k] for k in sorted(said))


Expand Down
6 changes: 5 additions & 1 deletion adapters/python/src/pact_adapters/harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -2942,7 +2942,11 @@ def _charge_summary(cost: "tuple[int, float] | None") -> None:
# its own ending.
meter.steps = limit
return await _ran_out(
Reached(step_ceiling(limit), float(limit), spec.limits.when_it_runs_out),
Reached(
step_ceiling(limit, default=not spec.steps_written and not spec.unbounded),
float(limit),
spec.limits.when_it_runs_out,
),
spec=spec, transport=transport, result=result, history=history,
already=already, bus=bus, meter=meter, now=now, clock=clock, at=limit,
phase=phase_name, visits=visits, asking=asking, once=once, chain=chain, given=given,
Expand Down
9 changes: 7 additions & 2 deletions adapters/python/src/pact_adapters/interceptors.py
Original file line number Diff line number Diff line change
Expand Up @@ -1653,8 +1653,13 @@ def _scanner(patterns: tuple[tuple[re.Pattern[str], str], ...]) -> re.Pattern[st


def _thing_of(pattern: str) -> str:
"""Which recogniser a pattern is, by name (`card number`), or `""`."""
return next((thing for thing, p in RECOGNISES.items() if p == pattern), "")
"""Which recogniser a pattern is, by name (`card number`, or a workspace's own detector,
`order reference`, when its `detectors:` are in scope), or `""`. A placeholder is named for it,
so whoever reads `⟦order-reference-1⟧` knows what kind of value stands there."""
from .detectors import DECLARED

known = {**RECOGNISES, **DECLARED.get()}
return next((thing for thing, p in known.items() if p == pattern), "")


def _substitute(text: str, patterns: Sequence[tuple[re.Pattern[str], str]]) -> str:
Expand Down
38 changes: 31 additions & 7 deletions adapters/python/src/pact_adapters/limits.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@ class Ceiling:
unit: str
#: The short tag reported as `RunResult.halted`.
halted: str
#: Nobody wrote this figure: it is PACT's own (`ir.DEFAULT_STEPS`, law 2), so
#: the sentence must not tell a person it is "the limit you set".
default: bool = False


@dataclass(frozen=True)
Expand Down Expand Up @@ -114,6 +117,20 @@ def sentence(self) -> str:
happened — a message saying only "limit exceeded" leaves the reader with
nowhere to go, which is the failure D13 rules out."""
what = self.what
if self.ceiling.default:
whose = (
f"PACT's default limit, {what}, which applies when no "
f"`{self.ceiling.field}:` is written (write one under `limits:` to set "
f"your own)"
)
if self.action is Action.ASK:
return f"Waiting for a person: this run reached {whose}."
if self.action is Action.ANSWER:
return (
f"This run reached {whose}, so it answered with what it had rather "
f"than finishing the work."
)
return f"Stopped before finishing: this run reached {whose}."
if self.action is Action.ASK:
return f"Waiting for a person: this run reached the limit you set with {what}."
if self.action is Action.ANSWER:
Expand Down Expand Up @@ -582,7 +599,9 @@ def across_runs(self) -> tuple[str, ...]:
meter. Reported, never dropped (T7)."""
return ("counted-per",) if self.counted_per not in ("", "run") else ()

def ceilings(self, steps_at_most: int | None = None) -> tuple[Ceiling, ...]:
def ceilings(
self, steps_at_most: int | None = None, steps_default: bool = False
) -> tuple[Ceiling, ...]:
"""Every ceiling in force, in a FIXED order.

The order is the report's tie-break: when a step and a cost ceiling are
Expand All @@ -592,7 +611,7 @@ def ceilings(self, steps_at_most: int | None = None) -> tuple[Ceiling, ...]:
"""
rows: list[Ceiling] = []
if steps_at_most is not None:
rows.append(step_ceiling(steps_at_most))
rows.append(step_ceiling(steps_at_most, default=steps_default))
if self.tool_calls_at_most is not None:
rows.append(
Ceiling(
Expand All @@ -619,10 +638,15 @@ def ceilings(self, steps_at_most: int | None = None) -> tuple[Ceiling, ...]:
return tuple(rows)

def reached(
self, meter: Meter, now: float, steps_at_most: int | None = None
self,
meter: Meter,
now: float,
steps_at_most: int | None = None,
steps_default: bool = False,
) -> Reached | None:
"""The first ceiling this run has reached, or `None`."""
for c in self.ceilings(steps_at_most):
"""The first ceiling this run has reached, or `None`. `steps_default` says
the step ceiling is PACT's default rather than the author's."""
for c in self.ceilings(steps_at_most, steps_default):
at = meter.reading(c.reads, now)
if at >= c.limit:
return Reached(c, at, self.when_it_runs_out)
Expand Down Expand Up @@ -772,15 +796,15 @@ def _nothing_can_reach(cap: float) -> bool:
return math.isnan(cap) or cap == math.inf


def step_ceiling(limit: int) -> Ceiling:
def step_ceiling(limit: int, default: bool = False) -> Ceiling:
"""The step ceiling as a row of the algebra.

It is enforced by the loop's own bound rather than by `reached`, but it ends
a run through the same door as every other ceiling — otherwise `steps-at-most`
and `cost-per-request-under` drift into behaving differently, which is what
happens whenever each limit is given its own ending.
"""
return Ceiling("steps-at-most", "steps", float(limit), "steps", "step-limit")
return Ceiling("steps-at-most", "steps", float(limit), "steps", "step-limit", default)


#: Every way `RunResult.halted` can say "a ceiling ended this". A caller that
Expand Down
4 changes: 2 additions & 2 deletions adapters/python/src/pact_adapters/pydantic_ai_registry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -172,8 +172,8 @@ fields:
outcome: loss
rows: [96-A6]
agent.turns:
construct: "nothing yet: the turns are carried on the document and not yet passed as message_history to Agent.run"
outcome: loss
construct: "message_history on Agent.run: pydantic_ai_turns.messages_of lowers them (SystemPromptPart, UserPromptPart, ModelResponse, ToolCallPart, ToolReturnPart, docs/96 §D)"
outcome: mapped
rows: [96-D]
agent.interceptors:
construct: "an AbstractCapability whose before_model_request / before_tool_execute hooks run interceptors.Chain"
Expand Down
109 changes: 109 additions & 0 deletions adapters/python/src/pact_adapters/pydantic_ai_turns.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
"""`turns:` as Pydantic AI messages (docs/96 §D): the one lowering every host on Pydantic AI uses.

An agent's `turns:` (a worked example placed before every live conversation), a variant's (the
few-shot a smaller model needs) and an eval case's (the conversation so far) are one `turn` group,
and `pact check` has already held them to their rules (`turns.rs`): `system` and `developer` only
before anyone speaks, every `calls:` answered by the very next `tool` turn, a call to an action
the agent may use. This module lowers them, by §D's table:

| PACT | Pydantic AI |
|---|---|
| `system`, `developer` | `SystemPromptPart` (both: the provider places them) |
| `user` | `UserPromptPart`, with what `attaches:` names after the words |
| `agent` | `ModelResponse([TextPart])` |
| `agent` with `calls:` | `ModelResponse([ToolCallPart])`, `with:` as its arguments |
| `tool` | `ToolReturnPart`, `answer:` as its content, under the call's own id |

Consecutive parts of one side share one message, as a provider sends them. What `says:` is filled
with, how an attachment becomes content, and what a `<tool>/<action>` is called where the model
sees it are the host's (`fill`, `attached`, `tool_name`): an agent's turns may carry holes, a case's
never do, and each host names its tools its own way.
"""

from __future__ import annotations

from collections.abc import Callable, Iterable, Mapping
from typing import Any

from pydantic_ai.messages import (
ModelMessage,
ModelRequest,
ModelRequestPart,
ModelResponse,
ModelResponsePart,
SystemPromptPart,
TextPart,
ToolCallPart,
ToolReturnPart,
UserPromptPart,
)

__all__ = ["messages_of"]

#: The speakers whose words are standing instructions rather than a turn of the conversation.
STANDING = ("system", "developer")


def _same(text: str) -> str:
return text


def messages_of(
turns: Iterable[Mapping[str, Any]],
*,
fill: Callable[[str], str] | None = None,
attached: Callable[[str], Any] | None = None,
tool_name: Callable[[str], str] | None = None,
call_ids: str = "turn",
) -> list[ModelMessage]:
"""The turns as Pydantic AI messages, oldest first. `fill` fills a turn's words (an agent's,
with its holes; a case's are used as written); `attached(name)` is one `UserContent` for an
`attaches:` entry; `tool_name("<tool>/<action>")` is the name the model calls it by. Each call
is given the id `<call_ids>-<n>`, and the `tool` turn after it answers under the same id."""
fill, tool_name = fill or _same, tool_name or _same
out: list[ModelMessage] = []
request: list[ModelRequestPart] = []
response: list[ModelResponsePart] = []
pending: tuple[str, str] | None = None # the call the next `tool` turn answers: (name, id)
n = 0

def close_request() -> None:
if request:
out.append(ModelRequest(parts=list(request)))
request.clear()

def close_response() -> None:
if response:
out.append(ModelResponse(parts=list(response)))
response.clear()

for turn in turns:
speaker = str(turn.get("from") or "").strip()
says = turn.get("says")
if speaker in STANDING:
close_response()
request.append(SystemPromptPart(fill(str(says or ""))))
elif speaker == "user":
close_response()
content: list[Any] = [fill(str(says))] if says else []
for name in turn.get("attaches") or ():
if attached is not None and (one := attached(str(name))) is not None:
content.append(one)
request.append(UserPromptPart(content if len(content) != 1 or not isinstance(content[0], str) else content[0]))
elif speaker == "agent":
close_request()
if says:
response.append(TextPart(fill(str(says))))
if turn.get("calls"):
n += 1
name, call_id = tool_name(str(turn["calls"]).strip()), f"{call_ids}-{n}"
response.append(ToolCallPart(name, dict(turn.get("with") or {}), tool_call_id=call_id))
pending = (name, call_id)
elif speaker == "tool":
close_response()
name, call_id = pending if pending is not None else ("tool", f"{call_ids}-{n}")
request.append(ToolReturnPart(name, turn.get("answer"), tool_call_id=call_id))
pending = None
close_request()
close_response()
return out
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,16 @@ def test_a_tool_answer_is_not_kept_where_never_from_refuses_it() -> None:

_keep_what_it_answered(spec, ToolCall("lookup", {"action": "get"}), "always approve this account")
assert "said" not in spec.facts.held, spec.facts.held


def test_a_workspace_detectors_placeholder_is_named_for_it() -> None:
"""`⟦order-reference-1⟧`, not an anonymous `⟦hidden-1⟧`: a reviewer reads what kind it was."""
from pact_adapters.detectors import DECLARED
from pact_adapters.interceptors import _thing_of

held = DECLARED.set({"order reference": "(?:ORD-[0-9]{8})"})
try:
assert _thing_of("(?:ORD-[0-9]{8})") == "order reference"
finally:
DECLARED.reset(held)
assert _thing_of("(?:ORD-[0-9]{8})") == "", "with no detectors in scope, nothing to name it by"
17 changes: 17 additions & 0 deletions adapters/python/tests/test_every_host_receives_a_step_ceiling.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,20 @@ def test_unbounded_no_is_the_default() -> None:
spec = _spec({"unbounded": "no"})
assert not spec.unbounded
assert usage_limits_for(spec).request_limit == DEFAULT_STEPS


def test_a_default_ceiling_is_never_called_the_limit_you_set() -> None:
"""A person told a run stopped at "the limit you set" goes looking for a line
nobody wrote. The default ceiling says it is PACT's, and how to set one."""
from pact_adapters.limits import Action, Limits, Meter, Reached, step_ceiling

said = Reached(step_ceiling(DEFAULT_STEPS, default=True), float(DEFAULT_STEPS), Action.STOP).sentence()
assert "you set" not in said
assert f"PACT's default limit, `steps-at-most` ({DEFAULT_STEPS} of {DEFAULT_STEPS} steps)" in said
assert "write one under `limits:`" in said
written = Reached(step_ceiling(3), 3.0, Action.STOP).sentence()
assert "the limit you set with `steps-at-most` (3 of 3 steps)" in written
meter = Meter(started=0.0)
meter.steps = DEFAULT_STEPS
reached = Limits().reached(meter, 0.0, DEFAULT_STEPS, steps_default=True)
assert reached is not None and reached.ceiling.default
Loading
Loading