Skip to content

Creating Iceberg table does not use ORDER BY as sort order #1935

Description

@xieandrew

Describe the bug

Creating a table from ClickHouse using the Iceberg table engine and ORDER BY does not show
those columns as part of sort order in ice. The Iceberg table is created in a database
with DataLakeCatalog pointing to ice-rest-catalog.

To Reproduce
Steps to reproduce the behavior:

  1. Ensure an instance of ice-rest-catalog is running

  2. Create ice database

CREATE DATABASE ice
  ENGINE = DataLakeCatalog('http://ice-rest-catalog:5000')
  SETTINGS catalog_type = 'rest',
    auth_header = 'Authorization: Bearer your-auth-token', 
    warehouse = 's3://your-warehouse';
  1. Create Iceberg table with ORDER BY
SET write_full_path_in_iceberg_metadata = 1;

CREATE TABLE ice.`test.sorting`
(
    `id` Int64,
    `name` String
)
ENGINE = Iceberg('s3://your-warehouse/test/sorting')
ORDER BY (id, name)
  1. Check the schema using ice and see the sort order is missing
ice describe -s test.sorting

---
- kind: Table
  metadata:
    id: test.sorting
  data:
    schemaRaw: |-
      table {
        1: id: required long
        2: name: required string
      }
    partitionSpecRaw: "[]"
    sortOrderRaw: "[]"
    location: s3://your-warehouse/test/sorting

Expected behavior
The sortOrder should use the ORDER BY from the ClickHouse CREATE TABLE statement.

Creating the table using ice works as intended:

ice create-table --schema-from-parquet=test-sorting.parquet --sort='[{"column":"id"},{"column":"name"}]' test.sorting
ice describe -s test.sorting
---
- kind: Table
  metadata:
    id: test.sorting
  data:
    schemaRaw: |-
      table {
        1: id: required long
        2: name: required string
      }
    partitionSpecRaw: "[]"
    sortOrderRaw: |-
      [
        identity(1) ASC NULLS LAST
        identity(2) ASC NULLS LAST
      ]
    location: s3://your-warehouse/test/sorting

Screenshots
N/A

Key information
Provide relevant runtime details.

  • 26.3.10.20001.altinityantalya
  • Altinity Cloud environment
  • S3 Storage
  • ice-rest-catalog

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions