Skip to content

Spark 4.2: Support CREATE TABLE ... LIKE ... - #17954

Open
rahulsmahadev wants to merge 1 commit into
apache:mainfrom
rahulsmahadev:spark-4.2-create-table-like
Open

Spark 4.2: Support CREATE TABLE ... LIKE ...#17954
rahulsmahadev wants to merge 1 commit into
apache:mainfrom
rahulsmahadev:spark-4.2-create-table-like

Conversation

@rahulsmahadev

Copy link
Copy Markdown
Contributor

Implements TableCatalog.createTableLike (added in Spark 4.2) for Iceberg's Spark catalogs, so CREATE TABLE ... LIKE ... creates an Iceberg table from an existing one.

Spark's CreateTableLikeExec intentionally excludes the source table's properties from the TableInfo it passes, leaving the connector to decide what to clone via the sourceTable argument. This implementation:

  • clones the source Iceberg table's schema, partition spec, sort order, and table properties;
  • lets user-specified TBLPROPERTIES and LOCATION take precedence over the cloned values;
  • does not copy snapshots, data, or metadata history, and creates the new table at the catalog's default location unless LOCATION is given.

SparkSessionCatalog routes createTableLike to the Iceberg catalog or the session catalog by provider, matching the existing create-table routing. Documentation is updated in spark-ddl.md.

Tests cover schema / partition spec / sort-order / property cloning, TBLPROPERTIES override precedence, IF NOT EXISTS, and a missing source, across the Hive, Hadoop, session, and REST catalog configurations.

Implement TableCatalog.createTableLike (added in Spark 4.2) for Iceberg's Spark catalogs so CREATE TABLE ... LIKE ... creates an Iceberg table from an existing one. The new table clones the source table's schema, partition spec, sort order, and user-set table properties; user-specified TBLPROPERTIES and LOCATION take precedence. Snapshots, data, and the source location are not copied.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant