Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,22 @@
What changed in PACT, newest first. Each entry says what an author can now write, or what now
holds that did not; the commits are listed under it for whoever wants the detail.

## 2026-11 — a case expects only what the specification fixes (12 November)

Bud Agent Flow's second replay: 21 of 24 agreed, and the three that did not were each a question
the cases had answered by accident.

### What a runtime must match

- **A run that stopped because its answer does not fit hands back an empty record.** PACT handed
back the unfitting answer beside the word `stopped`, which a caller could still read fields
off. Why it stopped is in `said` (`workflow_runtime.run_workflow`).
- **A field the run did not set is absent from its answer**, not `null`: `gives reply: text,
optional` and a run that never set it answers `{}`.
- **The order agents are asked in across the items of an `each`** is the runtime's to choose
unless the stage says `at-once-at-most: 1`, as the schema's help has always said. The shared
case that expected an order now runs on a tree that writes it.

## 2026-11 — the shared cases, replayed by a second runtime: three things PACT had wrong (12 November)

Bud Agent Flow replayed `tests/conformance/` and sent the cases that differed before recording
Expand Down
5 changes: 5 additions & 0 deletions adapters/python/src/pact_adapters/workflow_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -745,6 +745,11 @@ async def run_workflow(
if wrong:
result.halted = "stopped"
result.said = f"the answer does not fit `answers-with:` — {wrong}"
# And it is not handed back. A value that fails the workflow's own
# contract, returned beside the word `stopped`, is a value a caller can
# still read fields off — fields the workflow said it does not give.
# Why it stopped is in `said`; what the stages answered is in `steps`.
result.answer = {}
return result
result.halted = "final"
return result
Expand Down
4 changes: 4 additions & 0 deletions tests/conformance/trees/routed-desk/desk.pact
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ workflow clear-inbox
identified-by: [id]
limits:
items-at-most: 50
# One at a time, in order. Left out, how many run at once is the
# runtime's to choose, and the order agents are asked in with it — which
# a case could then not expect.
at-once-at-most: 1
when-it-runs-out: stop-and-say-so
one = call route with message: item.text, channel: item.channel
done-all = answer with replies: every.reply
Expand Down
11 changes: 2 additions & 9 deletions tests/conformance/workflows.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"about": "Workflows: stages in the order they run, what each called agent and each person was given, and the answer. Replayed as conformance_kit.replay_workflow does: `agents` is what each agent answers when called (a list is one answer for each time it is asked, the last repeating); `tools` is what each `<tool>/<action>` hands back; `people` is what a person answers to each question by name (absent: nobody is there to ask, and the run parks). In `expect`: `trace` is the stages that ran, nested ones as `outer/inner`; `given` is each agent called and its inputs, in order; `shown` is each question put to a person and what they were shown; `called` is each tool call. An agent a stage calls is an agent's run, under its own limits, approvals and reactions; here each is answered by the case, so what is compared is the workflow.",
"about": "Workflows: stages in the order they run, what each called agent and each person was given, and the answer. Replayed as conformance_kit.replay_workflow does: `agents` is what each agent answers when called (a list is one answer for each time it is asked, the last repeating); `tools` is what each `<tool>/<action>` hands back; `people` is what a person answers to each question by name (absent: nobody is there to ask, and the run parks). In `expect`: `trace` is the stages that ran, nested ones as `outer/inner`; `given` is each agent called and its inputs, in order; `shown` is each question put to a person and what they were shown; `called` is each tool call. An agent a stage calls is an agent's run, under its own limits, approvals and reactions; here each is answered by the case, so what is compared is the workflow. A case expects only what the specification fixes: where the order things happen in is the runtime's to choose (an `each` with no `at-once-at-most:`), the tree a case runs on says `at-once-at-most: 1`. In `answer`, a field the run did not set is absent, not null; a run that stopped hands back an empty record, and why is in `said`.",
"cases": [
{
"name": "a-written-rule-decides-and-nobody-is-asked",
Expand Down Expand Up @@ -644,14 +644,7 @@
]
},
"expect": {
"answer": {
"ideas": [
{
"idea": "Lumen",
"score": 0.95
}
]
},
"answer": {},
"called": [],
"given": [
{
Expand Down
Loading