Skip to content

PostgreSQL: a whole family of types is read as an opaque Any (PGobject) #2103

Description

@zaleslaw

Module: dataframe-jdbc
Type: enhancement (API/behaviour)
Found while working on: #2087 (driver metadata vs. actual value types)

Problem

For eleven PostgreSQL types the library declares the column as Any and puts the driver's
org.postgresql.util.PGobject in it. Nothing is broken in the strict sense — Any does accept the
value, so the -Pkotlin.dataframe.debug=true assertion stays silent — but the result is unusable
without knowing the driver's internals:

val df = DataFrame.readSqlTable(connection, "events")
df["payload"].type()          // kotlin.Any?
df["payload"][0]              // org.postgresql.util.PGobject
df["payload"].cast<String>()  // ClassCastException

Measured against PostgreSQL 17 (org.postgresql:postgresql), one column per type, a single non-null row:

SQL type declared now value class now proposed
json Any PGobject String
jsonb Any PGobject String
inet Any PGobject String
cidr Any PGobject String
macaddr Any PGobject String
macaddr8 Any PGobject String
tsvector Any PGobject String
tsquery Any PGobject String
bit varying / varbit Any PGobject String
int4range, int8range, numrange, tsrange, tstzrange, daterange, and the *multirange types Any PGobject String
uuid Any java.util.UUID kotlin.uuid.Uuid

For every PGobject case toString() already returns exactly the canonical text form
({"k": 1}, 10.0.0.0/8, [1,2), 101), so the conversion is one call with no information loss.

uuid is the odd one out: the value is already a java.util.UUID, and kotlin.uuid.Uuid is what
the library maps UUID to for H2 and DuckDB — so PostgreSQL is the inconsistent one here.

Does it break the API?

Not the binary API. No public signature changes: this is entirely inside
PostgreSql.getExpectedJdbcType / getValueFromResultSet, both already-overridden open members.
apiCheck / the api/*.api dump stay untouched.

It is a source-visible behaviour change for users of these types. The declared column type
changes, so code that already reads them has to change:

  • df["payload"] goes from DataColumn<Any?> to DataColumn<String?>. Code doing
    .cast<String>() or (value as PGobject).toString() on it stops compiling or starts failing.
  • With the compiler plugin / KSP-generated accessors the property type changes, which is a
    recompile for those users.
  • Anything persisting the schema (@DataSchema interfaces, saved schemas) needs regenerating.

Worth weighing honestly: today's Any is correct but useless, so in practice the only working
user code is toString() on a PGobject — that is exactly the population this change breaks, and
exactly the population it helps. Nobody can be relying on a typed value today, because there
isn't one.

Suggested handling: land it in one release together with a note in the changelog and in
docs/StardustDocs/topics/io/readSqlTypeMapping_PostgreSQL.md. Not urgent enough to rush into a
patch release.

Out of scope / open questions

  • xml is declared java.sql.SQLXML and the value is org.postgresql.jdbc.PgSQLXML — that one is
    consistent and typed, leave it alone.
  • Should json/jsonb go further than String and be parsed into DataFrame's own JSON handling?
    That is a much bigger discussion — this ticket only asks for the text form.
  • interval and the geometric types already map to dedicated PG* classes; unchanged.

Acceptance criteria

  • The types in the table above are read as String (resp. kotlin.uuid.Uuid).
  • PostgresTestBase asserts both the declared type and the literal value for each of them.
  • The mapping table in readSqlTypeMapping_PostgreSQL.md matches, row by row.
  • :dataframe-jdbc:testcontainersTest passes with -Pkotlin.dataframe.debug=true, including
    declared column types accept the values the driver returns.

Notes

The doc page currently claims String for most of this family already — it has said so since the
page was written (cce57cce0, 24.07.2026), while the code has returned PGobject since
3ab6bd0b2 (19.02.2026). The page was corrected to describe the real behaviour as part of the
#2087 work; this ticket is about making the behaviour the better one instead.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

databasesJDBC related issues

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions