refactor phase 3c: decompose the pre-aggregation pipeline into named phases - #300
Merged
Merged
Conversation
hussainsultan
force-pushed
the
refactor/phase3b-ops-split
branch
from
August 18, 2026 23:15
42c8229 to
ee75f26
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:15
fd55b6c to
5d0d4e8
Compare
hussainsultan
force-pushed
the
refactor/phase3b-ops-split
branch
from
August 18, 2026 23:26
ee75f26 to
39183ce
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:27
5d0d4e8 to
c4fc4cd
Compare
hussainsultan
force-pushed
the
refactor/phase3b-ops-split
branch
from
August 18, 2026 23:37
39183ce to
abff27b
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:37
c4fc4cd to
6cb2296
Compare
hussainsultan
force-pushed
the
refactor/phase3b-ops-split
branch
from
August 18, 2026 23:43
abff27b to
d3c17e3
Compare
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:43
6cb2296 to
98e8955
Compare
…aggScope The metadata merge, joined-table construction, wrapper-dimension ownership inference, and joined-table filter application move verbatim into _build_scope, returning an explicit read-only _PreaggScope. The _PreaggAccumulators dataclass is introduced for phase 4's outputs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_resolve_filter_ownership, _split_cross_table_legs, _build_plan, _partition_by_source — verbatim bodies over the explicit scope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 556-line per-source pre-aggregation loop moves verbatim into its own function; its six accumulators travel back as an explicit _PreaggAccumulators instead of loose locals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rchestrator _aggregate_joined_fallback, _combine_preaggregates, _attach_deferred_count_distincts, _apply_calc_phase, and _project_requested complete the decomposition: to_untagged_with_preagg is a ~25-line orchestrator over ten named phase functions, each with an explicit signature over _PreaggScope/_PreaggAccumulators instead of 15 loose accumulators threading through 1,089 lines. Bodies verbatim. Full suite green (1703 passed, exit 0); SQL snapshots byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hussainsultan
force-pushed
the
refactor/phase3c-preagg-decompose
branch
from
August 18, 2026 23:50
98e8955 to
aa32af3
Compare
hussainsultan
marked this pull request as ready for review
August 19, 2026 00:04
hussainsultan
added a commit
that referenced
this pull request
Aug 21, 2026
Main's phase 3c (#300) already decomposed the pre-aggregation pipeline into named phase functions in ops/_compile_preagg.py, superseding this branch's preagg.py staging. Resolve by keeping main's implementation and dropping the branch's preagg.py, its stage tests, and ADR 0002. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3c, stacked on #299 — the riskiest single refactor in the plan, executed with the safety net built for it. The 1,089-line
to_untagged_with_preaggbody is now a ~25-line orchestrator over ten named phase functions, with explicit data flow instead of 15 loose accumulators threading through one scope.Shape
Two explicit dataclasses replace the loose state:
_PreaggScope(read-only after phase 1 — op, join tree info, merged fields, the filtered joined table, filter bookkeeping) and_PreaggAccumulators(what phase 4 produces — pre-agg results, decomposed means, re-agg ops, count identities, deferred count-distincts, totals sources).Method
Every phase body moved verbatim: each function unpacks the scope fields into the original local names at the top, so the diff inside each phase is pure indentation. No logic was rewritten. Extraction went one commit per phase group (scope → 1b–3 → 4 → 5–7), each gated on the SQL-snapshot corpus, the full soundness directory, and the pre-agg stress + BI-trap suites; the final commit ran the full suite.
The follow-up this enables (not in this PR): phase 4's interior — per-table filter pushdown, measure classification, grain computation — can now be decomposed against a 556-line function with a typed contract, and the ~40 remaining broad exception handlers in the pre-agg path can be triaged per phase.
Verification
🤖 Generated with Claude Code