[3.10] gh-98820: Fix quadratic time in csv.Sniffer for quoted fields (GH-154867) - #155539
Merged
Merged
Conversation
…ields (pythonGH-154867) (cherry picked from commit b30c7fa) (cherry picked from commit 39dac55)
They were added in 3.11. An unrolled loop is unambiguous at every position, so it does not backtrack either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Manual backport of #154867, cherry-picked from the 3.12 backport (#155166), with one 3.10-only change in the second commit.
Possessive quantifiers were added in 3.11, so
(?:(?P=quote){2}|(?!(?P=quote)).)*+does not compile here. The body is matched as an unrolled loop instead; since the quote character is a backreference, the "not a quote" atom needs a lookahead:Every position matches only one of the two alternatives, and each iteration of the outer loop consumes the two quote characters, so it does not backtrack.
Checked against the 3.11 version of the file: no difference in the sniffed dialect on 90000 samples, and the same timings — sniffing
'"abcdefghijklmnopqrstuvwxyz"\n' * nwithdelimiters=',:|\t'takes 0.147, 0.298 and 0.602 s for n = 2500, 5000 and 10000, against 0.142, 0.286 and 0.569 s on 3.11.