Skip to content

refactor phase 2: typed errors, shared primitives, fail-closed pruning (SCC 13 → 10) - #297

Merged
hussainsultan merged 8 commits into
mainfrom
refactor/phase2-primitives-and-errors
Aug 18, 2026
Merged

refactor phase 2: typed errors, shared primitives, fail-closed pruning (SCC 13 → 10)#297
hussainsultan merged 8 commits into
mainfrom
refactor/phase2-primitives-and-errors

Conversation

@hussainsultan

Copy link
Copy Markdown
Collaborator

Phase 2 of the SCC-driven refactor, stacked on #296. Theme: one implementation per primitive, and errors that say what went wrong. Full suite green (1693 passed, pytest exit 0).

Typed exception hierarchy + contextful errors

New bottom-layer errors.py: BSLErrorDefinitionError / QueryError / UnknownFieldError / CompilationError / SerializationError / BackendError, exported from the package root. Each subclass also inherits the stdlib exception it replaces (ValueError/KeyError/RuntimeError), so existing except clauses and tests keep working through the migration.

Every .unwrap() on a returns.Result in yaml.py and query.py now goes through unwrap_or_raise, which names the model field and expression and chains the original exception. The worst error in the audit is gone:

  • before: UnwrapFailedError: (no message)
  • after: DefinitionError: Invalid expression for dimension 'b' ('_.b =='): Invalid Python syntax: ...

Typo suggestions are consolidated too: suggest / format_suggestions / suggest_kinded in errors.py replace the duplicated _typo_suggestion bodies in calc_compiler and measure_scope.

Correctness: join pruning now fails closed

When a measure or group key couldn't be analyzed for column requirements, the extractors silently contributed nothing — and a table whose only references were unanalyzable could be pruned from the join, silently changing results (audit finding). Analysis failures now require all columns (disabling pruning for that query) and log a warning naming the measure/key and cause. The dead parallel implementations in ops.py (TableColumnRequirements + both _extract_requirements_from_*, ~140 lines) are deleted — projection_utils.TableRequirements was the live path all along, so one requirements type remains.

Remaining silent fallbacks on correctness paths now log: the xorq-conversion fallback in _ensure_xorq_table, the grain-cardinality analysis swallow in expr.py (which feeds the join_one→join_many upgrade decision), and the four guard-skipping continues in _reject_shadowed_group_keys.

Shared primitives

  • fieldref.py (bottom layer, unit-tested): split_prefixed / suffix_matches / resolve_suffix replace the verbatim-duplicated _parse_prefixed_field (ops + projection_utils) and four hand-rolled unique-suffix resolvers (ops, calc_compiler ×2, measure_scope).
  • _SourcePassThroughOp mixin: Filter/OrderBy/Limit/Unnest each hand-wrote identical values/schema/get_dimensions/get_measures/get_calculated_measures delegates; one mixin now carries the protocol (~90 lines of boilerplate gone). Project/GroupBy deliberately untouched — they never had the get_* protocol and gaining it silently would change behavior.

SCC: 13 → 10 modules

utils.py needed ops only to isinstance-check _CallableWrapper (now duck-typed on ._fn, with a _is_deferred guard — a Deferred synthesizes attribute access, so a naive getattr fallback built bogus ._fn nodes; caught by the serialization round-trip suite and fixed). graph_utils needed only CalcMeasure (duck-typed on depends_on). Cutting those frees projection_utils too. The bottom layer no longer depends on the compiler; the residual SCC is ops/expr/query/api/format + serialization/*.

Deliberately deferred to phase 3

ReductionKind classifier, the dimension-proxy consolidation (7 near-identical classes), the join-tree visitor, and the remaining ~40 broad handlers deep in the pre-agg compiler — those code regions get restructured wholesale in phase 3's _to_untagged_with_preagg decomposition, so consolidating them now would be churn against next phase's diff.

Verification

  • python3 -m pytest src/boring_semantic_layer: 1693 passed, exit 0
  • SCC ratchet shrunk 13 → 10 in the same commits as the cuts; ruff check / format --check clean
  • New unit tests for fieldref; the one test pinning the old contentless UnwrapFailedError updated to assert the typed, contextful error

🤖 Generated with Claude Code

hussainsultan and others added 8 commits August 18, 2026 19:26
New bottom-layer errors.py: BSLError -> DefinitionError / QueryError /
UnknownFieldError / CompilationError / SerializationError /
BackendError. Each subclass also inherits the stdlib exception it
replaces (ValueError/KeyError/RuntimeError) so existing except clauses
and tests keep working. Exported from the package root.

Every .unwrap() on a returns.Result in yaml.py and query.py now goes
through unwrap_or_raise, which names the model field and expression
and chains the original error — the contentless UnwrapFailedError on
bad YAML/filter expressions is gone:

  before: UnwrapFailedError:
  after:  DefinitionError: Invalid expression for dimension 'b'
          ('_.b =='): Invalid Python syntax: ...

Also consolidates the typo-suggestion logic: suggest/format_suggestions/
suggest_kinded in errors.py replace the duplicated _typo_suggestion
bodies in calc_compiler and measure_scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…resolution

New bottom-layer fieldref.py (split_prefixed / suffix_matches /
resolve_suffix) replaces the verbatim-duplicated _parse_prefixed_field
in ops.py and projection_utils.py and four hand-rolled unique-suffix
resolvers (ops._resolve_short_name, calc_compiler's two measure
resolvers, measure_scope's column error path). Direct unit tests added;
behavior preserved (exact match wins, unique suffix resolves, ambiguity
returns None).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SemanticFilterOp/OrderByOp/LimitOp/UnnestOp each hand-wrote the same
values/schema properties and get_dimensions/get_measures/
get_calculated_measures delegates to source. One mixin now carries the
pass-through protocol; unnest keeps its own values/schema overrides.
Project/GroupBy are untouched (they never defined the get_* protocol,
and gaining it silently would change hasattr-style behavior).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When a measure or group key can't be analyzed for column requirements,
the extractors silently contributed nothing — and a table whose only
references were unanalyzable got PRUNED from the join, silently
changing results. Failures now require all columns (disabling pruning
for that query) and log a warning naming the measure/key and cause.

Also deletes ops.py's dead parallel implementations of the same
extractors (TableColumnRequirements + _extract_requirements_from_keys/
_measures, ~140 lines): the live path has been
projection_utils.extract_requirements_from_* all along — one
requirements type remains.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- _ensure_xorq_table: falling back to plain ibis on conversion failure
  is intended for unsupported backends, but now logs the reason instead
  of swallowing genuine xorq errors invisibly
- _detect_grain_cardinality (expr.py): inconclusive predicate analysis
  still keeps the explicit join_one contract, but the swallowed
  exception is now visible at debug level
- _reject_shadowed_group_keys: the four continues that silently skipped
  the shadowing guard now log which dimension/measure could not be
  analyzed and why

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… SCC

utils.py needed ops only to isinstance-check _CallableWrapper (now
duck-typed via the ._fn attribute); graph_utils needed only CalcMeasure
for a depends_on isinstance (now duck-typed on the attribute). Cutting
those two upward edges also frees projection_utils, whose only cycle
path ran through graph_utils. The bottom layer no longer depends on
the compiler. SCC: 13 -> 10 modules (ops/expr/query/api/format +
serialization).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getattr(fn, '_fn', fn) never falls back on a Deferred — attribute
access on a Deferred synthesizes a new deferred node, so serialization
encoded a bogus trailing ._fn attr. Guard with _is_deferred before
duck-typing the wrapper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test pinned the old contentless UnwrapFailedError; bad filter
strings now raise QueryError naming the expression and cause.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hussainsultan
hussainsultan force-pushed the refactor/phase2-primitives-and-errors branch from cc417f7 to 67b9ee8 Compare August 18, 2026 23:26
@hussainsultan
hussainsultan changed the base branch from refactor/phase1-dead-code-and-scc-rim to main August 18, 2026 23:27
@hussainsultan hussainsultan reopened this Aug 18, 2026
@hussainsultan
hussainsultan marked this pull request as ready for review August 18, 2026 23:35
@hussainsultan
hussainsultan merged commit 1e2d784 into main Aug 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant