PyVortex DuckDB to use native extension #5914
gatesn
started this conversation in
Feature Requests
Replies: 1 comment
-
|
If you're just talking about Vortex datasets registered for replacement scans in DuckDB-Python, then: DuckDB-Python handles all Arrow datasets the same: wrapping them in a Scanner and pushing the expressions down. One way to avoid the expression issues from #5801 is to special case Vortex datasets: adding a separate case for Vortex datasets that uses Vortex expressions... possibly reusing the Vortex extension. The relevant code (that handles Datasets and creates the Scanner): There's other places to inject a special case, such as at registration time: all involving DuckDB-Python changes. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
The PyVortex <-> DuckDB integration should (somehow) use the native DuckDB extension, instead of going via Arrow datasets.
I've opened this as a discussion because I haven't really thought about how to do this!
Beta Was this translation helpful? Give feedback.
All reactions