Skip to content

fix: auto-detect single-column CSV files - #881

Closed
harsh-thakkar7 wants to merge 1 commit into
simonw:mainfrom
harsh-thakkar7:fix/detect-single-column-csv
Closed

harsh-thakkar7 wants to merge 1 commit into
simonw:mainfrom
harsh-thakkar7:fix/detect-single-column-csv

Conversation

@harsh-thakkar7

@harsh-thakkar7 harsh-thakkar7 commented Sep 24, 2026 •

Copy link
Copy Markdown

rows_from_file() fails to auto-detect single-column CSV files. Format detection sniffs the first bytes with csv.Sniffer().sniff(), which raises csv.Error: Could not determine delimiter for any file without a delimiter — so a bare list like this could never be imported:

>>> from sqlite_utils.utils import rows_from_file
>>> import io
>>> rows_from_file(io.BytesIO(b"id\n1\n2\n"))
Traceback (most recent call last):
  ...
csv.Error: Could not determine delimiter

Fix

When sniffing fails to find a delimiter, fall back to the default CSV dialect instead of crashing. csv.DictReader (with no custom dialect) then loads the file as a single column exactly as expected:

>>> rows, format_ = rows_from_file(io.BytesIO(b"id\n1\n2\n"))
>>> list(rows)
[{'id': '1'}, {'id': '2'}]

Multi-column CSV, TSV and JSON auto-detection are unaffected.

Tests

Added to tests/test_utils.py:

  • test_rows_from_file_detects_single_column_csv[id... ] — parameterized over a single-column list of words and of integers, asserting Format.CSV and the one-key rows (both fail before this change).
pytest -q tests/test_utils.py   # 21 passed
pytest -q                       # 1502 passed, 16 skipped

📚 Documentation preview 📚: https://sqlite-utils--881.org.readthedocs.build/en/881/

Format detection passed the first bytes to csv.Sniffer().sniff(), which
raises csv.Error('Could not determine delimiter') for files with no
delimiter at all, so a bare single-column list like 'id\n1\n2' could
never be loaded through rows_from_file. Fall back to the default CSV
dialect when sniffing fails so DictReader treats the file as a single
column.
@harsh-thakkar7

Copy link
Copy Markdown
Author

Closing this — I don't think this change is mature enough to land right now. Thanks for the project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant