[SPARK-56296][SQL] Pivot createTableLike to pass full TableInfo including schema, partitioning, constraints, and owner#55101
Open
viirya wants to merge 1 commit intoapache:masterfrom
Open
Conversation
…ding schema, partitioning, constraints, and owner Previously createTableLike(ident, sourceTable, userSpecifiedOverrides) only passed user-specified TBLPROPERTIES to the connector via TableInfo, requiring connectors to (1) call CurrentUserContext.getCurrentUser (a Catalyst internal) to set the owner, and (2) read columns/partitioning/constraints themselves from sourceTable. This change pivots to createTableLike(ident, tableInfo, sourceTable) where tableInfo contains all explicit information for the new table: - columns and partitioning copied from the source - constraints copied from the source - user-specified TBLPROPERTIES, LOCATION, and USING provider (if given) - PROP_OWNER set to the current user Source table properties are intentionally excluded from tableInfo; connectors receive sourceTable to clone any format-specific or custom state they need. This matches the pattern used by REPLACE TABLE in DSv2. Update InMemoryTableCatalog, CatalogSuite, and CreateTableLikeSuite accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
aokolnychyi
reviewed
Apr 1, 2026
| sourceTable.columns() | ||
| } | ||
|
|
||
| // Build TableInfo with all explicit information for the new table: |
Contributor
There was a problem hiding this comment.
I am not sure how useful this comment is as the code is pretty obvious on its own.
I would consider adding two helper methods and adding comments there:
// Derive target columns from source; for V1Table sources apply CharVarcharUtils to preserve
// CHAR/VARCHAR types as declared rather than collapsed to StringType.
private def targetColumns: Array[Column] = sourceTable match {
case v1: V1Table =>
CatalogV2Util.structTypeToV2Columns(CharVarcharUtils.getRawSchema(v1.catalogTable.schema))
case _ =>
sourceTable.columns
}
// Source table properties are intentionally excluded; connectors read sourceTable
// to clone any additional format-specific or custom state they need.
private def targetProperties: Map[String, String] = {
properties ++
provider.map(TableCatalog.PROP_PROVIDER -> _) ++
location.map(uri => TableCatalog.PROP_LOCATION -> CatalogUtils.URIToString(uri)) +
(TableCatalog.PROP_OWNER -> CurrentUserContext.getCurrentUser)
}
Then you can drop this larger comment and call it like this:
val tableInfo = new TableInfo.Builder()
.withColumns(targetColumns)
.withPartitions(sourceTable.partitioning)
.withConstraints(sourceTable.constraints)
.withProperties(targetProperties.asJava)
.build()
targetCatalog.createTableLike(targetIdent, tableInfo, sourceTable)
Up to you, though.
aokolnychyi
reviewed
Apr 1, 2026
| val userSpecifiedOverrides = new TableInfo.Builder() | ||
| val tableInfo = new TableInfo.Builder() | ||
| .withColumns(sourceColumns) | ||
| .withPartitions(sourceTable.partitioning()) |
Contributor
There was a problem hiding this comment.
Minor: Drop () for partitioning and constraints as you simply access getters?
aokolnychyi
approved these changes
Apr 1, 2026
Contributor
aokolnychyi
left a comment
There was a problem hiding this comment.
+1 with one minor style suggestion.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Previously createTableLike(ident, sourceTable, userSpecifiedOverrides) only passed user-specified TBLPROPERTIES to the connector via TableInfo, requiring connectors to call CurrentUserContext.getCurrentUser (a Catalyst internal) to set the owner.
We should not expose Catalyst internal to connectors. But putting owner to the TableInfo means that we will have to rename userProvidedOverrides to something else, meaning it no longer contains only user overrides.
This change pivots to createTableLike(ident, tableInfo, sourceTable) where tableInfo contains all explicit information for the new table:
Source table properties are intentionally excluded from tableInfo; connectors receive sourceTable to clone any format-specific or custom state they need. This matches the pattern used by REPLACE TABLE in DSv2.
Update InMemoryTableCatalog, CatalogSuite, and CreateTableLikeSuite accordingly.
Why are the changes needed?
This pivot is necessary to keep a balance between API consistency and internal exposure.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Unit tests
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 4.6