refactor phase 4: serialization consolidation — utils.py dissolved, codec on public constructors - #301
Merged
Merged
Conversation
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:15
fd55b6c to
5d0d4e8
Compare
hussainsultan
force-pushed
the
refactor/phase4-serialization
branch
from
August 18, 2026 23:15
495bbba to
8eeb595
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:26
5d0d4e8 to
c4fc4cd
Compare
hussainsultan
force-pushed
the
refactor/phase4-serialization
branch
from
August 18, 2026 23:27
8eeb595 to
9be1b17
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:37
c4fc4cd to
6cb2296
Compare
hussainsultan
force-pushed
the
refactor/phase4-serialization
branch
from
August 18, 2026 23:37
9be1b17 to
048161e
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:43
6cb2296 to
98e8955
Compare
hussainsultan
force-pushed
the
refactor/phase4-serialization
branch
from
August 18, 2026 23:44
048161e to
29bcdc4
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:50
98e8955 to
aa32af3
Compare
hussainsultan
force-pushed
the
refactor/phase4-serialization
branch
from
August 18, 2026 23:50
29bcdc4 to
e790de3
Compare
The grab-bag is gone; each cluster now lives with its concern:
- safe_eval.py: the AST-allowlist sandbox for YAML/agent expression
strings (its own trust boundary, unrelated to serialization's)
- io.py: YAML/URL config loading
- serialization/_trust.py: the callable allowlist for ('fn', module,
qualname) payload arms, enforced on write and read; the public
trust_callable_module/UntrustedCallableError now export from the
serialization package (the error hint names the new path)
- serialization/codec.py: the resolver-tree expression codec (scalar
encode/decode, serialize/deserialize_resolver, the structured expr
and join-predicate API) — the serialization stack's lowest layer
finally lives inside the package
All importers updated; bodies verbatim. Full suite 1703 passed, exit 0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lain ibis deserialize_resolver used object.__new__ + object.__setattr__ + a hand-computed __precomputed_hash__ to bypass validation — an entire bug class (rebuilt resolvers raising AttributeError when hashed) that existed only because of the bypass. The public constructors accept already-built child resolvers on both ibis flavors (verified against xorq's vendored ibis and plain ibis 11), so all seven arms now use them and _finalize_frozen_slotted is deleted. This also fixes the latent flavor bug: plain ibis 11 renamed Item's key slot from 'name' to 'indexer', so serializing or rebuilding _["x"]-style expressions crashed on the HAS_XORQ=False branch — invisible because every serialization suite importorskips xorq. The serializer reads whichever slot exists; the deserializer passes the key positionally. New coverage: test_codec_no_xorq.py runs the round-trip in a subprocess with xorq import-blocked (including the Item case); test_codec_units.py pins scalar-codec symmetry, freeze/thaw, and a 16-shape resolver round-trip battery with hashability checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- SCHEMA_VERSION + SUPPORTED_PAYLOAD_MAJORS live together in context.py: the writer stamps and the reader gates from one place; bumping the format is now a one-file change - helpers.py dissolves: deserialize_structured joins the codec beside structured_to_expr, and extract_simple_column_name is rewritten with real isinstance checks against the flavor classes instead of type-name string comparisons (the third, stringly-typed resolver traversal is gone) Trust roots deliberately stay a process-global registry: trust_callable_module is the documented, tested API and additive process-wide trust is the intended semantic; threading a context through every codec call would be churn without a driving use case. Full suite 1749 passed, exit 0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hussainsultan
force-pushed
the
refactor/phase4-serialization
branch
from
August 19, 2026 00:05
e790de3 to
a0865e5
Compare
hussainsultan
changed the base branch from
refactor/phase3c-preagg-decompose
to
main
August 19, 2026 00:05
hussainsultan
marked this pull request as ready for review
August 19, 2026 00:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 4 of the SCC-driven refactor, stacked on #300. The serialization stack becomes one self-contained package, the hand-built ibis internals are gone, and the codec provably works on plain ibis. Full suite green (1749 passed, exit 0 — up 46 tests from the new codec coverage).
utils.py no longer exists
The seven-cluster grab-bag dissolved into its real homes, bodies verbatim:
safe_eval.py— the AST-allowlist sandbox for YAML/agent expression strings (its own trust boundary, now visibly separate from serialization's)io.py— YAML/URL config loadingserialization/_trust.py— the callable allowlist for("fn", module, qualname)payload arms, enforced on write and read;trust_callable_module/UntrustedCallableErrorexport from the serialization package and the refusal message names the new pathserialization/codec.py— the resolver-tree expression codec (scalar codec,serialize/deserialize_resolver, structured expr + join-predicate API). The layering inversion the audit flagged is fixed: the serialization stack's lowest layer finally lives inside the package.Codec rebuilt on public constructors — and the plain-ibis Item bug is fixed
deserialize_resolverusedobject.__new__+object.__setattr__+ a hand-computed__precomputed_hash__to bypass validation — the source of an entire bug class (rebuilt resolvers raisingAttributeErrorwhen hashed). Verified against both flavors: the public constructors are positionally identical and accept already-built child resolvers, so all seven arms now use them and_finalize_frozen_slottedis deleted.BinaryOperator/UnaryOperatorgain upstream's own operator validation for free.This also fixes the latent flavor bug: plain ibis 11 renamed
Item's key slotname→indexer, so_["x"]-style expressions crashed on theHAS_XORQ=Falsebranch — invisible because every serialization suiteimportorskips xorq. The serializer reads whichever slot exists; the deserializer passes the key positionally.New coverage where the audit found none
test_codec_no_xorq.py— runs the round-trip in a subprocess with xorq import-blocked (including theItemcase), so the plain-ibis branch is exercised even on xorq machinestest_codec_units.py— scalar-codec symmetry (dates, Decimal, timedelta…), freeze/thaw symmetry, and a 16-shape resolver round-trip battery with hashability checks (the precomputed-hash bug class now has direct unit coverage)One format constant; less indirection
SCHEMA_VERSION+SUPPORTED_PAYLOAD_MAJORSlive together incontext.py— writer stamps and reader gates from one place; a format bump is a one-file changehelpers.pydissolved into the codec;extract_simple_column_namerewritten with real isinstance checks instead oftype(x).__name__ == "Attr"string comparisonsDeliberately kept: trust roots stay a process-global registry —
trust_callable_moduleis the documented, tested API and additive process-wide trust is the intended semantic; context-scoping it would thread a parameter through every codec call without a driving use case.Verification
🤖 Generated with Claude Code