Skip to content

[SPARK-59355][SQL] Simplify LIKE patterns containing escaped wildcards - #58644

Open
david-mollitor-db wants to merge 1 commit into
apache:masterfrom
david-mollitor-db:like-escape-simplification
Open

[SPARK-59355][SQL] Simplify LIKE patterns containing escaped wildcards#58644
david-mollitor-db wants to merge 1 commit into
apache:masterfrom
david-mollitor-db:like-escape-simplification

Conversation

@david-mollitor-db

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

LikeSimplification gives up on a pattern the moment it contains the escape character
(if (pattern.contains(escapeChar)) None), so escaped-literal patterns stay a full per-row
regex even when they are trivially sargable once the escapes are decoded:

col LIKE 'ma\%ca'           ==>  col = 'ma%ca'
col LIKE 'abc\%def%'        ==>  StartsWith(col, 'abc%def')
col LIKE '%mn\%'            ==>  EndsWith(col, 'mn%')
col LIKE 'abbc' ESCAPE 'b'  ==>  col = 'abc'

This PR decodes valid escape sequences and applies the rule's existing shape simplifications
(EqualTo / StartsWith / EndsWith / Contains / StartsWith+EndsWith) to the decoded
literals.

How it works:

  • Fast path (unchanged): a quick pattern.contains(escapeChar) scan — when the escape
    character is absent, the existing five regexes run exactly as before, so the common case pays
    nothing for the decode logic.
  • Guard: otherwise, if the escape character is itself % or _ (a pathological
    ESCAPE '%' / ESCAPE '_'), skip — unchanged behavior. This guard follows the fast-path scan
    on purpose: an ESCAPE '%' pattern that contains no % is still simplifiable via the fast
    path.
  • Decode: otherwise, walk the pattern turning \% / \_ / \\ into literal characters,
    treating unescaped %/_ as wildcards, and classify the decoded shape.

An unescaped _ wildcard and LikeAll/LikeAny shapes beyond what simplifyLike already
covers remain out of scope.

Why are the changes needed?

Escaped-literal patterns become sargable: the resulting EqualTo/StartsWith/etc. push down to
data sources (e.g. Parquet prunes on StringStartsWith) and short-circuit the per-row regex,
instead of always running the regex and pushing nothing.

Does this PR introduce any user-facing change?

No. Query results are identical, and error cases still error; this is a performance improvement.

A valid-escape pattern's decoded literal is exactly the string the LIKE matches, so the rewrite
is behavior-preserving. An escape character followed by anything other than %, _, or itself,
or a trailing escape character, makes LIKE throw at runtime (INVALID_FORMAT.ESC_IN_THE_MIDDLE
/ ESC_AT_THE_END); the rule returns None for these, so the Like is kept and throws exactly
as today. Every shape fully replaces the Like (no residual), so the rewrite is idempotent.

How was this patch tested?

LikeSimplificationSuite: updated the existing escape tests (they previously asserted the
pattern was skipped and now assert the simplified form, including the LikeAll/LikeAny
multi-pattern tests whose escaped entries now fold in), and added tests for valid-escape
simplification across all shapes and for the still-skipped cases (invalid escape, trailing
escape, ESCAPE '%'/ESCAPE '_', and an unescaped _).
build/sbt 'catalyst/testOnly *LikeSimplificationSuite' passes; scalastyle clean.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8

@uros-b uros-b left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon first look, this seems like a well tested and behaviour preserving optimization, thank you @david-mollitor-db! Adding @stevomitric to also review this PR

`LikeSimplification` gives up on a pattern the moment it contains the escape character
(`if (pattern.contains(escapeChar)) None`), so escaped-literal patterns stay a full per-row
regex even when they are trivially sargable once the escapes are decoded:

    col LIKE 'ma\%ca'           ==>  col = 'ma%ca'
    col LIKE 'abc\%def%'        ==>  StartsWith(col, 'abc%def')
    col LIKE '%mn\%'            ==>  EndsWith(col, 'mn%')
    col LIKE 'abbc' ESCAPE 'b'  ==>  col = 'abc'

This decodes valid escape sequences and applies the rule's existing shape simplifications
(EqualTo / StartsWith / EndsWith / Contains / StartsWith+EndsWith) to the decoded literals. The
resulting predicates push down to data sources and short-circuit the per-row regex.

How it works:
- Fast path (unchanged): a quick `pattern.contains(escapeChar)` scan -- when the escape
  character is absent, the existing regexes run exactly as before.
- Guard: otherwise, if the escape character is itself `%` or `_` (a pathological `ESCAPE '%'` /
  `ESCAPE '_'`), skip. This guard follows the fast-path scan on purpose: an `ESCAPE '%'` pattern
  that contains no `%` is still simplifiable via the fast path.
- Decode: otherwise, walk the pattern turning `\%` / `\_` / `\\` into literal characters,
  treating unescaped `%`/`_` as wildcards, and classify the decoded shape.

A valid-escape pattern's decoded literal is exactly the string the LIKE matches, so the rewrite
is behavior-preserving. An escape character followed by anything other than `%`, `_`, or itself,
or a trailing escape character, makes LIKE throw at runtime
(`INVALID_FORMAT.ESC_IN_THE_MIDDLE` / `ESC_AT_THE_END`); the rule returns None for these, so the
`Like` is kept and throws exactly as today. Every shape fully replaces the `Like` (no residual),
so no idempotency tag is needed. An unescaped `_` wildcard and `LikeAll`/`LikeAny` shapes beyond
what `simplifyLike` already covers remain out of scope.

Generated-by: Claude Opus 4.8
@david-mollitor-db
david-mollitor-db force-pushed the like-escape-simplification branch from 007773a to afc65f7 Compare September 9, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants