Fix DuckDB scans over Vortex being merged or stalling

[joseph-isaacs]

Summary

Two bugs surfaced by running all 99 TPC-DS queries against Vortex through DuckDB in a single connection (the TPC-DS SLT suite is stacked on this PR). Both make DuckDB over Vortex either return wrong results or hang, so they are split out here for separate review.

Changes

Wrong results from DuckDB's common-subplan optimizer (vortex-duckdb). DuckDB 1.5's common-subplan optimizer keys scans on their serialized form. The Vortex table function keeps pushed-down filters, projections and aggregates in its FFI bind data, which has no serializer, so two scans of the same file with different pushed-down filters serialized identically and were merged into one shared CTE. TPC-DS q9 returned the q41-60 bucket's average for four of its five buckets; q28 was affected the same way. Parquet was unaffected because its filters stay in table_filters, which are serialized. The fix sets verify_serialization = false on the table function, which is how DuckDB expects functions with opaque bind data to opt out of plan serialization (and therefore of subplan merging). slt/duckdb/common_subplan_pushdown.slt is a regression test: scalar subqueries over one Vortex file with different pushed-down filters must return different results, and the plan must not contain a __common_subplan CTE.

Stalled scans when the calling thread has a tokio context (vortex-io). ObjectStoreFileSystem::local read files through object_store's LocalFileSystem, whose local reads go through tokio::task::spawn_blocking whenever the calling thread has a tokio runtime context. When the awaiting task is driven by the DuckDB extension's non-tokio CurrentThreadRuntime (the SLT runner drives DuckDB from inside a tokio block_on), a long sequence of scans eventually lost a wakeup: tracing showed the read task calling get_opts and never receiving a response, with every thread parked. The same 49-query sequence completed in about ten seconds with no tokio context on the thread. Local files are now opened with FileReadAt, which reads on the Vortex runtime's own blocking pool, so local reads no longer depend on the ambient tokio runtime. With this change the previously hanging sequence passes under the tokio-driven runner.

Checks run: cargo nextest run -p vortex-io (114 passed), full cargo nextest run -p vortex-sqllogictest (144 passed, TPC-H skipped without data), clippy on vortex-io and vortex-sqllogictest, nightly fmt, clang-format on the C++ change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L57FHoJ15heJVNaqgRq7xB

---

Generated by Claude Code

[codspeed-hq]

Merging this PR will improve performance by 10.39%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 2198 untouched benchmarks
⏩ 206 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	allocate_drop_bytes[0]	575.7 ns	521.6 ns	+10.39%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/duckdb-vortex-scan-fixes (385e8ec) with develop (85b70cf)²}

Footnotes

1. 206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on develop (5c8c397) during the generation of this report, so 85b70cf was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.34%. Comparing base (5c8c397) to head (385e8ec).

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.