[fix](nullable) Do not generate NULL-only access path for NOT NULL column made nullable by outer join - #66380
Closed
yujun777 wants to merge 1 commit into
Closed
Conversation
…umn made nullable by outer join IS NULL predicate on a physical NOT NULL column that is only made nullable by an outer join generates a [col, NULL] access path, because the rule only checks the expression nullability. The column has no physical null bitmap, so BE crashes with `Check failed: is_column_nullable(*dst)` in FileColumnIterator::read_by_rowids when reading the null map by rowids. Key changes: - AccessPathExpressionCollector.visitIsNull: skip NULL-only access path when the SlotReference's physical column is NOT NULL (made nullable by outer join); the regular data path is sufficient to evaluate IS NULL after the join - merge the sub-column-path early return into the same guard Unit Test: - PruneNestedColumnTest.testIsNullOnNotNullColumnAfterLeftJoin - regression-test/suites/nereids_rules_p0/column_pruning/left_join_not_null_column.groovy
yujun777
requested review from
924060929,
englefly,
morrySnow and
starocean999
as code owners
August 3, 2026 09:04
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29086 ms |
Contributor
TPC-DS: Total hot run time: 169357 ms |
Contributor
ClickBench: Total hot run time: 23.82 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
Author
|
will fix in https://github.com/apache/doris/tree/meta-path |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After an outer join,
IS NULLon a physicalNOT NULLcolumn (e.g. the right-side dimension column of aLEFT JOIN) crashes the BE:Core stack:
doris_main.cpp:518logsFailed to initialize JNI/ SIGABRT with the sameCheck failedwhen the query runs.Root cause
Introduced by #62304 (commit
85ede0bafba0, "opt Optimize I/O operations for the IS NULL predicate"). That PR makescol IS NULLgenerate acol.NULLaccess path so BE reads only the null map (NULL_MAP_ONLYmode), saving I/O. The premise is that the column is physically nullable (has a null bitmap).The bug: the rule only checks expression nullability (
arg.nullable()). ALEFT JOINmakes the right-sideNOT NULLcolumn's output slot nullable, soarg.nullable() == trueand a[col, NULL]access path is generated even though the physical column has no null bitmap. BE then reads the null map by rowids on a NOT NULL column and hits theis_column_nullable(*dst)check — the backend aborts, failing the user's base-table query.This is unrelated to the IVM branch that first surfaced it: any ordinary base-table
LEFT JOINquery withenable_prune_nested_column = true(default) reproduces the crash.Fix
In
AccessPathExpressionCollector.visitIsNull, skip the NULL-only access path generation when theSlotReference's physical column isNOT NULL(only made nullable by the outer join). The join output stays nullable and IS NULL is evaluated on the regular data path; no access info is sent to BE for the whole-column read (shouldSkipAccessInfodrops the single[col]path).Also merged the existing sub-column-path early return into the same guard.
Verification
PruneNestedColumnTest.testIsNullOnNotNullColumnAfterLeftJoin: asserts no[segment, NULL]access path is generated after LEFT JOIN (50 tests, all pass)left_join_not_null_column.groovy: LEFT JOIN aggregate query returns correct result, BE stays alive;explainasserts nosegment.NULLin the plan