Skip to content

[#10472] feat(entity store): support entity store read-replica connections#10471

Open
youngyjd wants to merge 4 commits intoapache:mainfrom
youngyjd:junda/separate-read-write-traffic
Open

[#10472] feat(entity store): support entity store read-replica connections#10471
youngyjd wants to merge 4 commits intoapache:mainfrom
youngyjd:junda/separate-read-write-traffic

Conversation

@youngyjd
Copy link
Copy Markdown
Contributor

@youngyjd youngyjd commented Mar 18, 2026

What changes were proposed in this pull request?

  1. Add optional read-replica support for the relational entity store (JDBCBackend): new config keys for read-only JDBC URL, user, password, and pool sizing (or inherit from primary).
  2. When any read-only config is set, create a second JDBC pool and route standalone metadata reads to it; keep writes and in-transaction reads on the primary pool.
  3. Add startup log when the read pool is enabled and optional metrics for the read datasource; leave behavior unchanged when read-only config is not set.

Why are the changes needed?

To allow offloading read-heavy metadata traffic to a read replica (e.g. MySQL) instead of the primary, improving scalability and reducing load on the primary while keeping writes and transactional reads on the primary.

Fix: #10472

Does this PR introduce any user-facing change?

Yes. New optional configuration options for the relational backend (jdbcReadOnlyUrl, jdbcReadOnlyUser, jdbcReadOnlyPassword, readOnlyMaxConnections, readOnlyMaxWaitMillis) and optional metrics for the read pool. Existing deployments are unchanged unless these options are set.

How was this patch tested?

  1. unit tested
  2. existing integration guarantees backward compatibility

@youngyjd youngyjd changed the title support entity store read-replica connections [#10472] feat(entity store): support entity store read-replica connections Mar 18, 2026
@github-actions
Copy link
Copy Markdown

Code Coverage Report

Overall Project 64.96% +0.21% 🟢
Files changed 92.42% 🟢

Module Coverage
aliyun 1.73% 🔴
api 47.14% 🟢
authorization-common 85.96% 🟢
aws 1.1% 🔴
azure 2.6% 🔴
catalog-common 10.0% 🔴
catalog-fileset 80.02% 🟢
catalog-hive 80.98% 🟢
catalog-jdbc-clickhouse 78.25% 🟢
catalog-jdbc-common 42.98% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.05% 🟢
catalog-jdbc-starrocks 78.27% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 45.07% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 87.15% 🟢
catalog-lakehouse-paimon 77.71% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.83% 🟢
common 49.36% 🟢
core 81.35% +0.28% 🟢
filesystem-hadoop3 76.97% 🟢
flink 38.86% 🔴
flink-runtime 0.0% 🔴
gcp 14.2% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 45.82% 🟢
iceberg-common 50.21% 🟢
iceberg-rest-server 66.24% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 23.78% 🔴
lance-rest-server 57.84% 🟢
lineage 53.02% 🟢
optimizer 82.87% 🟢
optimizer-api 21.95% 🔴
server 85.65% 🟢
server-common 69.72% 🟢
spark 32.79% 🔴
spark-common 39.6% 🔴
trino-connector 31.62% 🔴
Files
Module File Coverage
common ConfigConstants.java 0.0% 🔴
core Configs.java 98.26% 🟢
SqlSessionFactoryHelper.java 92.59% 🟢
MetricsSource.java 86.36% 🟢
SessionUtils.java 85.37% 🟢
SqlSessions.java 84.93% 🟢
RelationDatasourceMetricsSource.java 0.0% 🔴

@youngyjd youngyjd marked this pull request as ready for review March 18, 2026 20:22
@jerryshao jerryshao requested a review from yuqi1129 March 23, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Route entity-store reads to optional read-only JDBC URL (read replica)

1 participant