Skip to content

fix: transform() corrupts expression defaults into string literals - #882

Closed
harsh-thakkar7 wants to merge 1 commit into
simonw:mainfrom
harsh-thakkar7:fix/preserve-expression-defaults-in-transform
Closed

harsh-thakkar7 wants to merge 1 commit into
simonw:mainfrom
harsh-thakkar7:fix/preserve-expression-defaults-in-transform

Conversation

@harsh-thakkar7

@harsh-thakkar7 harsh-thakkar7 commented Sep 24, 2026 •

Copy link
Copy Markdown

transform() silently corrupts expression defaults into string literals. When a column has an expression default:

CREATE TABLE t (a TEXT, b INTEGER DEFAULT (1+2), c BLOB DEFAULT (X'ff'), d TEXT DEFAULT ('x' || 'y'))

PRAGMA table_info reports the expression without its parentheses (b → '1+2', c → X'ff', d → 'x'||'y'). transform() fed that fragment straight back into the rebuilt CREATE TABLE, where it was re-interpreted as a string literal:

>>> table.transform(rename={"a": "aa"})   # any unrelated change
>>> table.schema                          # before this fix
'b" INTEGER DEFAULT \'1+2\''
'c" BLOB DEFAULT \'X\'\'ff\'\'\''

Every subsequent insert silently stored the text '1+2' in the INTEGER column instead of the computed value 3, and the blob became the string "X'ff'". This is the same class of bug the codebase already fixed for keyword literals (TRUE/FALSE/NULL) in test_transform_preserves_keyword_literal_defaults, but expressions were missed.

Fix

  • New Database.transform_default_fragment() classifies a PRAGMA-returned default and re-emits it safely: quoted string literals, blob literals (X'ff'), keywords (TRUE/FALSE/NULL/CURRENT_*) and numeric literals pass through unchanged; anything else was a parenthesized expression (1+2, 'x'||'y', lower('ab')+1) and is wrapped in parentheses again, which SQLite requires for expression defaults.
  • transform_sql() uses it when carrying existing defaults across the rebuild.
  • quote_default_value() now passes blob literals through unquoted (they were previously re-quoted as strings) and avoids double-wrapping already-parenthesized expressions.

Round-tripped schema:

"b" INTEGER DEFAULT (1+2)
"c" BLOB DEFAULT X'ff'
"d" TEXT DEFAULT ('x' || 'y')

Tests

Added test_transform_preserves_expression_defaults to tests/test_transform.py: builds a table with an arithmetic, blob and concatenation default, runs transform(rename=...), asserts the expressions stay unparenthesized-correct in the schema, then inserts a row and asserts the stored values are (3, b"\xff", "xy") — not the literal strings (this test fails before the change; the pre-fix row stored '1+2').

pytest -q tests/test_transform.py tests/test_default_value.py   # 131 passed
pytest -q                                                       # 1498 passed, 16 skipped

Note: plain numeric defaults like DEFAULT 1 still round-trip as DEFAULT '1' — this is existing, documented-by-test behavior (SQLite's type affinity stores the correct integer), and is intentionally left unchanged.


📚 Documentation preview 📚: https://sqlite-utils--882.org.readthedocs.build/en/882/

PRAGMA reports expression defaults without their parentheses (DEFAULT
(1+2) comes back as '1+2'), and transform() fed that fragment straight
back into CREATE TABLE, where it was re-quoted as a string literal.
Every table rebuilt by transform() then silently stored the text '1+2'
instead of the value 3 (and blob/concat defaults like (X'ff') or
('x'||'y') were corrupted too). Re-emit PRAGMA-derived defaults by
re-wrapping non-literal fragments in parentheses and pass blob literals
through unquoted.
@harsh-thakkar7

Copy link
Copy Markdown
Author

Closing this — I don't think this change is mature enough to land right now. Thanks for the project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant