fix(optimizer): incorrect merge with literal alias collision#7847
fix(optimizer): incorrect merge with literal alias collision#7847fivetran-kwoodbeck wants to merge 3 commits into
Conversation
SQLGlot Integration Test Results✅ All tests passedComparing:
Overallmain: 192416 total, 153532 passed (pass rate: 79.8%) sqlglot:optimizer/bug-literal-group-projection: 180222 total, 142385 passed (pass rate: 79.0%) Transitions: Dialect pair changes: 0 previous results not found, 3 current results not found ✅ All tests passed |
georgesittas
left a comment
There was a problem hiding this comment.
Tried optimizing this query as well
-- Input
WITH c AS (SELECT DISTINCT x.b AS b FROM x)
SELECT s.a, SUM(s.b) AS b FROM (SELECT 6 AS a, b FROM c) AS s CROSS JOIN x GROUP BY s.a
-- Optimized under this PR
WITH c AS (...) SELECT 6 AS a, SUM(c.b) AS b FROM c CROSS JOIN x GROUP BY aThe GROUP BY now binds a to x.a from the outer cross join. Seems like a similar bug. Additionally, there appears to be a similar bug with ORDER BY on a non-projected literal alias:
-- input
SELECT SUM(s.b) AS b FROM (SELECT 6 AS a, b FROM x) AS s GROUP BY s.a ORDER BY s.a
-- Optimized
SELECT SUM(x.b) AS b FROM x AS x GROUP BY a ORDER BY 6This is also problematic, but let's address it separately to avoid scope creep.
| literal_aliases = {s.alias_or_name for s in inner_select.selects if s.unalias().is_number} | ||
| if not literal_aliases: | ||
| return False |
There was a problem hiding this comment.
We should be careful not to walk selects / expressions more than needed. This can be wasteful if repeated without care. It's worth looking into refactoring this.
There was a problem hiding this comment.
I'm not sure we can get around this one.
| grouped = { | ||
| c.name | ||
| for c in group.find_all(exp.Column) | ||
| if c.table == inner_name and c.name in literal_aliases | ||
| } |
There was a problem hiding this comment.
Similarly, these find / find_all calls can result in walking many nodes. There are other places where they happen in _mergeable. We should make sure we're not repeating a lot of work.
There may be a better alternative of pre-aggregating everything we need for these checks, but for that we would need to run all the cheap "mergeable" checks first, to avoid taking on the overhead preemptively.
There was a problem hiding this comment.
found a way around the find_all
| if isinstance(source, Scope) and grouped & { | ||
| s.alias_or_name for s in source.expression.selects | ||
| }: |
There was a problem hiding this comment.
What about:
| if isinstance(source, Scope) and grouped & { | |
| s.alias_or_name for s in source.expression.selects | |
| }: | |
| if isinstance(source, Scope) and grouped & set(source.expression.named_selects): |
a5a7cfe to
283107e
Compare
283107e to
4f3280b
Compare
Merging a derived table that projects a numeric literal (
6 AS a) whose alias matches aFROMcolumn, and is used in the outerGROUP BY, rewrites it to a bareGROUP BYa that rebinds to the column instead of the constant. This changes the query result. The fix is that_mergeablenow blocks those merges.Sample query:
Before, this query will return different results:
After: