diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2989e9a1..739429a7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,11 @@ All notable changes to this project will be documented in this file.
- Support objectOverrides using `.spec.objectOverrides`.
See [objectOverrides concepts page](https://docs.stackable.tech/home/nightly/concepts/overrides/#object-overrides) for details ([#885]).
- Enable the [restart-controller](https://docs.stackable.tech/home/nightly/commons-operator/restarter/), so that the Pods are automatically restarted on config changes ([#888]).
+- Added support for `2.6.0` ([#XXX]).
+
+### Removed
+
+- Removed support for `1.27.0` and `2.4.0` ([#XXX]).
### Changed
@@ -25,6 +30,7 @@ All notable changes to this project will be documented in this file.
[#884]: https://github.com/stackabletech/nifi-operator/pull/884
[#885]: https://github.com/stackabletech/nifi-operator/pull/885
[#888]: https://github.com/stackabletech/nifi-operator/pull/888
+[#XXX]: https://github.com/stackabletech/nifi-operator/pull/XXX
## [25.11.0] - 2025-11-07
diff --git a/docs/modules/nifi/pages/usage_guide/index.adoc b/docs/modules/nifi/pages/usage_guide/index.adoc
index adaf1f60..6dd43490 100644
--- a/docs/modules/nifi/pages/usage_guide/index.adoc
+++ b/docs/modules/nifi/pages/usage_guide/index.adoc
@@ -17,12 +17,11 @@ metadata:
name: simple-nifi
spec:
image:
- productVersion: 1.27.0
+ productVersion: 2.7.2
clusterConfig:
- zookeeperConfigMapName: simple-nifi-znode # <1>
- authentication: # <2>
+ authentication: # <1>
- authenticationClass: simple-nifi-admin-user
- extraVolumes: # <3>
+ extraVolumes: # <2>
- name: nifi-client-certs
secret:
secretName: nifi-client-certs
@@ -35,18 +34,17 @@ spec:
roleGroups:
default:
config:
- resources: # <4>
+ resources: # <3>
cpu:
min: "500m"
max: "4"
memory:
- limit: '2Gi'
+ limit: 2Gi
replicas: 3
----
-<1> The xref:usage_guide/clustering.adoc#backend-zookeeper[ZooKeeper instance] to use.
-<2> How users should xref:usage_guide/security.adoc[authenticate] themselves.
-<3> xref:usage_guide/extra-volumes.adoc[Extra volumes] with files that can be referenced in custom workflows.
-<4> xref:usage_guide/resource-configuration.adoc[CPU and memory configuration] can be set per role group.
+<1> How users should xref:usage_guide/security.adoc[authenticate] themselves.
+<2> xref:usage_guide/extra-volumes.adoc[Extra volumes] with files that can be referenced in custom workflows.
+<3> xref:usage_guide/resource-configuration.adoc[CPU and memory configuration] can be set per role group.
Not shown are the common settings for xref:usage_guide/operations/cluster-operations.adoc[starting and stopping the cluster] and xref:usage_guide/operations/pod-placement.adoc[distributing Pods]. Additionally, you can set any NiFi setting using xref:usage_guide/overrides.adoc[overrides]. You can also configure xref:usage_guide/log-aggregation.adoc[log aggregation].
diff --git a/docs/modules/nifi/pages/usage_guide/updating.adoc b/docs/modules/nifi/pages/usage_guide/updating.adoc
index 44209e1a..473c2690 100644
--- a/docs/modules/nifi/pages/usage_guide/updating.adoc
+++ b/docs/modules/nifi/pages/usage_guide/updating.adoc
@@ -2,7 +2,7 @@
:description: Easily update or downgrade Apache NiFi on Kubernetes by changing the CRD version.
Updating (or downgrading for that matter) the deployed version of NiFi is as simple as changing the version stated in the CRD.
-Continuing the example above, to change the deployed version from `1.27.0` to `2.0.0` you'd simply deploy the following CRD.
+Continuing the example above, to change the deployed version from `2.6.0` to `2.7.2` you'd simply deploy the following CRD.
[source,yaml]
----
@@ -12,7 +12,7 @@ metadata:
name: simple-nifi
spec:
image:
- productVersion: 2.0.0 # <1>
+ productVersion: 2.7.2 # <1>
----
<1> Change the NiFi version here
@@ -24,7 +24,7 @@ NiFi clusters cannot be upgraded or downgraded in a rolling fashion due to a lim
When upgrading between NiFi 1 versions or from NiFi 1 to NiFi 2, any change to the NiFi version in the CRD triggers a full cluster restart with brief downtime.
However, the Stackable image version can be updated in a rolling manner, provided the NiFi version remains unchanged.
-For upgrades between NiFi 2 versions, e.g. from `2.0.0` to `2.4.0`, rolling upgrades are supported.
+For upgrades between NiFi 2 versions, e.g. from `2.6.0` to `2.7.2`, rolling upgrades are supported.
====
== NiFi 2.0.0
diff --git a/docs/modules/nifi/pages/usage_guide/writing-to-iceberg-tables.adoc b/docs/modules/nifi/pages/usage_guide/writing-to-iceberg-tables.adoc
index 13048014..a3561b37 100644
--- a/docs/modules/nifi/pages/usage_guide/writing-to-iceberg-tables.adoc
+++ b/docs/modules/nifi/pages/usage_guide/writing-to-iceberg-tables.adoc
@@ -6,9 +6,23 @@
Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the same tables, at the same time.
NiFi supports a `PutIceberg` processor to add rows to an existing Iceberg table https://issues.apache.org/jira/browse/NIFI-10442[starting from version 1.19.0].
-As of NiFi version `2.4.0` only `PutIceberg` is supported, you need to create and compact your tables with other tools such as Trino or Spark (both included in the Stackable Data Platform).
+As of NiFi version `2.7.2` only `PutIceberg` is supported, you need to create and compact your tables with other tools such as Trino or Spark (both included in the Stackable Data Platform).
-== NiFi 2
+== NiFi 2.7 and above
+
+In NiFi `2.7.0` Iceberg support was https://issues.apache.org/jira/browse/NIFI-15062[re-added] after the removal in 2.0.0.
+
+The 2.7 version has the following changes over the 2.0/2.6 version, you need to adopt your setup accordingly:
+
+* HDFS and Kerberos support was dropped
+* Hive metastore support was dropped
+* Iceberg REST catalog support was added
+* It now uses the Iceberg S3 IO instead of the Hadoop S3 client libraries
+* It uses much less dependencies and therefore CVEs
+
+There have been efforts from Stackable to re-add at least Hive metastore support, but that turned out to be complicated.
+
+== NiFi 2.0 - 2.6
In NiFi `2.0.0` Iceberg support https://issues.apache.org/jira/browse/NIFI-13938[has been removed] from upstream NiFi.
diff --git a/docs/modules/nifi/partials/supported-versions.adoc b/docs/modules/nifi/partials/supported-versions.adoc
index ac5437d5..dd0612ec 100644
--- a/docs/modules/nifi/partials/supported-versions.adoc
+++ b/docs/modules/nifi/partials/supported-versions.adoc
@@ -2,9 +2,8 @@
// This is a separate file, since it is used by both the direct NiFi-Operator documentation, and the overarching
// Stackable Platform documentation.
-* 2.6.0 (LTS, Please note that you need to upgrade to at least 1.27.x before upgrading to 2.x.x!)
-* 2.4.0 (Deprecated, Please note that you need to upgrade to at least 1.27.x before upgrading to 2.x.x!)
+* 2.7.2
+* 2.6.0 (LTS)
* 1.28.1 (Deprecated)
-* 1.27.0 (Deprecated)
For details on how to upgrade your NiFi version, refer to xref:nifi:usage_guide/updating.adoc[].
diff --git a/rust/operator-binary/src/config/mod.rs b/rust/operator-binary/src/config/mod.rs
index 24da1102..72104ea0 100644
--- a/rust/operator-binary/src/config/mod.rs
+++ b/rust/operator-binary/src/config/mod.rs
@@ -818,7 +818,7 @@ mod tests {
name: simple-nifi
spec:
image:
- productVersion: 1.27.0
+ productVersion: 2.7.2
clusterConfig:
authentication:
- authenticationClass: nifi-admin-credentials-simple
@@ -867,7 +867,7 @@ mod tests {
name: simple-nifi
spec:
image:
- productVersion: 1.27.0
+ productVersion: 2.7.2
clusterConfig:
authentication:
- authenticationClass: nifi-admin-credentials-simple
diff --git a/rust/operator-binary/src/crd/affinity.rs b/rust/operator-binary/src/crd/affinity.rs
index ac1e570e..23168c72 100644
--- a/rust/operator-binary/src/crd/affinity.rs
+++ b/rust/operator-binary/src/crd/affinity.rs
@@ -43,7 +43,7 @@ mod tests {
name: simple-nifi
spec:
image:
- productVersion: 1.27.0
+ productVersion: 2.7.2
clusterConfig:
authentication:
- authenticationClass: nifi-admin-credentials-simple
diff --git a/tests/templates/kuttl/iceberg/00-patch-ns.yaml.j2 b/tests/templates/kuttl/iceberg-hive/00-patch-ns.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/00-patch-ns.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/00-patch-ns.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/00-rbac.yaml.j2 b/tests/templates/kuttl/iceberg-hive/00-rbac.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/00-rbac.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/00-rbac.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/01-create-s3-connection.yaml b/tests/templates/kuttl/iceberg-hive/01-create-s3-connection.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/01-create-s3-connection.yaml
rename to tests/templates/kuttl/iceberg-hive/01-create-s3-connection.yaml
diff --git a/tests/templates/kuttl/iceberg/01_s3-connection.yaml b/tests/templates/kuttl/iceberg-hive/01_s3-connection.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/01_s3-connection.yaml
rename to tests/templates/kuttl/iceberg-hive/01_s3-connection.yaml
diff --git a/tests/templates/kuttl/iceberg/02-assert.yaml.j2 b/tests/templates/kuttl/iceberg-hive/02-assert.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/02-assert.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/02-assert.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/02-install-krb5-kdc.yaml.j2 b/tests/templates/kuttl/iceberg-hive/02-install-krb5-kdc.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/02-install-krb5-kdc.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/02-install-krb5-kdc.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/03-create-kerberos-secretclass.yaml.j2 b/tests/templates/kuttl/iceberg-hive/03-create-kerberos-secretclass.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/03-create-kerberos-secretclass.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/03-create-kerberos-secretclass.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/03_kerberos-secretclass.yaml.j2 b/tests/templates/kuttl/iceberg-hive/03_kerberos-secretclass.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/03_kerberos-secretclass.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/03_kerberos-secretclass.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/10-assert.yaml.j2 b/tests/templates/kuttl/iceberg-hive/10-assert.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/10-assert.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/10-assert.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/10-install-vector-aggregator-discovery-configmap.yaml.j2 b/tests/templates/kuttl/iceberg-hive/10-install-vector-aggregator-discovery-configmap.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/10-install-vector-aggregator-discovery-configmap.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/10-install-vector-aggregator-discovery-configmap.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/20-assert.yaml b/tests/templates/kuttl/iceberg-hive/20-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/20-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/20-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/20-install-minio.yaml b/tests/templates/kuttl/iceberg-hive/20-install-minio.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/20-install-minio.yaml
rename to tests/templates/kuttl/iceberg-hive/20-install-minio.yaml
diff --git a/tests/templates/kuttl/iceberg/20_minio.yaml b/tests/templates/kuttl/iceberg-hive/20_minio.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/20_minio.yaml
rename to tests/templates/kuttl/iceberg-hive/20_minio.yaml
diff --git a/tests/templates/kuttl/iceberg/21-assert.yaml b/tests/templates/kuttl/iceberg-hive/21-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/21-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/21-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/21-install-minio-jobs.yaml b/tests/templates/kuttl/iceberg-hive/21-install-minio-jobs.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/21-install-minio-jobs.yaml
rename to tests/templates/kuttl/iceberg-hive/21-install-minio-jobs.yaml
diff --git a/tests/templates/kuttl/iceberg/21_minio_jobs.yaml b/tests/templates/kuttl/iceberg-hive/21_minio_jobs.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/21_minio_jobs.yaml
rename to tests/templates/kuttl/iceberg-hive/21_minio_jobs.yaml
diff --git a/tests/templates/kuttl/iceberg/25-assert.yaml b/tests/templates/kuttl/iceberg-hive/25-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/25-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/25-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/25-install-hive-postgres.yaml b/tests/templates/kuttl/iceberg-hive/25-install-hive-postgres.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/25-install-hive-postgres.yaml
rename to tests/templates/kuttl/iceberg-hive/25-install-hive-postgres.yaml
diff --git a/tests/templates/kuttl/iceberg/25_helm-bitnami-postgresql-values.yaml.j2 b/tests/templates/kuttl/iceberg-hive/25_helm-bitnami-postgresql-values.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/25_helm-bitnami-postgresql-values.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/25_helm-bitnami-postgresql-values.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/30-assert.yaml b/tests/templates/kuttl/iceberg-hive/30-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/30-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/30-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/30-install-zookeeper.yaml.j2 b/tests/templates/kuttl/iceberg-hive/30-install-zookeeper.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/30-install-zookeeper.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/30-install-zookeeper.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/31-assert.yaml b/tests/templates/kuttl/iceberg-hive/31-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/31-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/31-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/31-opa.yaml.j2 b/tests/templates/kuttl/iceberg-hive/31-opa.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/31-opa.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/31-opa.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/32-assert.yaml b/tests/templates/kuttl/iceberg-hive/32-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/32-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/32-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/32-install-hdfs.yaml.j2 b/tests/templates/kuttl/iceberg-hive/32-install-hdfs.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/32-install-hdfs.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/32-install-hdfs.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/32_hdfs.yaml.j2 b/tests/templates/kuttl/iceberg-hive/32_hdfs.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/32_hdfs.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/32_hdfs.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/33-assert.yaml b/tests/templates/kuttl/iceberg-hive/33-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/33-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/33-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/33-install-hive.yaml.j2 b/tests/templates/kuttl/iceberg-hive/33-install-hive.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/33-install-hive.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/33-install-hive.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/33_hive.yaml.j2 b/tests/templates/kuttl/iceberg-hive/33_hive.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/33_hive.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/33_hive.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/34-assert.yaml b/tests/templates/kuttl/iceberg-hive/34-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/34-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/34-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/34-install-trino.yaml.j2 b/tests/templates/kuttl/iceberg-hive/34-install-trino.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/34-install-trino.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/34-install-trino.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/34_trino.yaml.j2 b/tests/templates/kuttl/iceberg-hive/34_trino.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/34_trino.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/34_trino.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/40-assert.yaml b/tests/templates/kuttl/iceberg-hive/40-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/40-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/40-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/40-create-iceberg-tables.j2 b/tests/templates/kuttl/iceberg-hive/40-create-iceberg-tables.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/40-create-iceberg-tables.j2
rename to tests/templates/kuttl/iceberg-hive/40-create-iceberg-tables.j2
diff --git a/tests/templates/kuttl/iceberg/50-assert.yaml b/tests/templates/kuttl/iceberg-hive/50-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/50-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/50-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/50-install-nifi.yaml.j2 b/tests/templates/kuttl/iceberg-hive/50-install-nifi.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/50-install-nifi.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/50-install-nifi.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/50_nifi.yaml.j2 b/tests/templates/kuttl/iceberg-hive/50_nifi.yaml.j2
similarity index 93%
rename from tests/templates/kuttl/iceberg/50_nifi.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/50_nifi.yaml.j2
index 11097e5c..00bc2cbc 100644
--- a/tests/templates/kuttl/iceberg/50_nifi.yaml.j2
+++ b/tests/templates/kuttl/iceberg-hive/50_nifi.yaml.j2
@@ -5,12 +5,12 @@ metadata:
name: nifi
spec:
image:
-{% if test_scenario['values']['nifi-iceberg'].find(",") > 0 %}
- custom: "{{ test_scenario['values']['nifi-iceberg'].split(',')[1] }}"
- productVersion: "{{ test_scenario['values']['nifi-iceberg'].split(',')[0] }}"
+{% if test_scenario['values']['nifi-iceberg-hive'].find(",") > 0 %}
+ custom: "{{ test_scenario['values']['nifi-iceberg-hive'].split(',')[1] }}"
+ productVersion: "{{ test_scenario['values']['nifi-iceberg-hive'].split(',')[0] }}"
{% else %}
custom: null
- productVersion: "{{ test_scenario['values']['nifi-iceberg'] }}"
+ productVersion: "{{ test_scenario['values']['nifi-iceberg-hive'] }}"
{% endif %}
pullPolicy: IfNotPresent
clusterConfig:
diff --git a/tests/templates/kuttl/iceberg/60-create-nifi-flow-configmap.yaml.j2 b/tests/templates/kuttl/iceberg-hive/60-create-nifi-flow-configmap.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/60-create-nifi-flow-configmap.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/60-create-nifi-flow-configmap.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/60_nifi-flow-with-kerberos.json b/tests/templates/kuttl/iceberg-hive/60_nifi-flow-with-kerberos.json
similarity index 100%
rename from tests/templates/kuttl/iceberg/60_nifi-flow-with-kerberos.json
rename to tests/templates/kuttl/iceberg-hive/60_nifi-flow-with-kerberos.json
diff --git a/tests/templates/kuttl/iceberg/60_nifi-flow-without-kerberos.json b/tests/templates/kuttl/iceberg-hive/60_nifi-flow-without-kerberos.json
similarity index 100%
rename from tests/templates/kuttl/iceberg/60_nifi-flow-without-kerberos.json
rename to tests/templates/kuttl/iceberg-hive/60_nifi-flow-without-kerberos.json
diff --git a/tests/templates/kuttl/iceberg/61-assert.yaml b/tests/templates/kuttl/iceberg-hive/61-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/61-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/61-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/61-provision-nifi-flow.yaml b/tests/templates/kuttl/iceberg-hive/61-provision-nifi-flow.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/61-provision-nifi-flow.yaml
rename to tests/templates/kuttl/iceberg-hive/61-provision-nifi-flow.yaml
diff --git a/tests/templates/kuttl/iceberg/70-assert.yaml b/tests/templates/kuttl/iceberg-hive/70-assert.yaml
similarity index 100%
rename from tests/templates/kuttl/iceberg/70-assert.yaml
rename to tests/templates/kuttl/iceberg-hive/70-assert.yaml
diff --git a/tests/templates/kuttl/iceberg/70-check-iceberg-tables.yaml.j2 b/tests/templates/kuttl/iceberg-hive/70-check-iceberg-tables.yaml.j2
similarity index 100%
rename from tests/templates/kuttl/iceberg/70-check-iceberg-tables.yaml.j2
rename to tests/templates/kuttl/iceberg-hive/70-check-iceberg-tables.yaml.j2
diff --git a/tests/templates/kuttl/iceberg/README.md b/tests/templates/kuttl/iceberg-hive/README.md
similarity index 100%
rename from tests/templates/kuttl/iceberg/README.md
rename to tests/templates/kuttl/iceberg-hive/README.md
diff --git a/tests/templates/kuttl/iceberg-rest/00-patch-ns.yaml.j2 b/tests/templates/kuttl/iceberg-rest/00-patch-ns.yaml.j2
new file mode 100644
index 00000000..67185acf
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/00-patch-ns.yaml.j2
@@ -0,0 +1,9 @@
+{% if test_scenario['values']['openshift'] == 'true' %}
+# see https://github.com/stackabletech/issues/issues/566
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ - script: kubectl patch namespace $NAMESPACE -p '{"metadata":{"labels":{"pod-security.kubernetes.io/enforce":"privileged"}}}'
+ timeout: 120
+{% endif %}
diff --git a/tests/templates/kuttl/iceberg-rest/00-rbac.yaml.j2 b/tests/templates/kuttl/iceberg-rest/00-rbac.yaml.j2
new file mode 100644
index 00000000..7ee61d23
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/00-rbac.yaml.j2
@@ -0,0 +1,29 @@
+---
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+ name: test-role
+rules:
+{% if test_scenario['values']['openshift'] == "true" %}
+ - apiGroups: ["security.openshift.io"]
+ resources: ["securitycontextconstraints"]
+ resourceNames: ["privileged"]
+ verbs: ["use"]
+{% endif %}
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: test-sa
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+ name: test-rb
+subjects:
+ - kind: ServiceAccount
+ name: test-sa
+roleRef:
+ kind: Role
+ name: test-role
+ apiGroup: rbac.authorization.k8s.io
diff --git a/tests/templates/kuttl/iceberg-rest/01-create-s3-connection.yaml b/tests/templates/kuttl/iceberg-rest/01-create-s3-connection.yaml
new file mode 100644
index 00000000..26f46c12
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/01-create-s3-connection.yaml
@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ - script: envsubst '$NAMESPACE' < 01_s3-connection.yaml | kubectl apply -n $NAMESPACE -f -
diff --git a/tests/templates/kuttl/iceberg-rest/01_s3-connection.yaml b/tests/templates/kuttl/iceberg-rest/01_s3-connection.yaml
new file mode 100644
index 00000000..56c30d36
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/01_s3-connection.yaml
@@ -0,0 +1,36 @@
+---
+apiVersion: s3.stackable.tech/v1alpha1
+kind: S3Connection
+metadata:
+ name: minio
+spec:
+ host: "minio.${NAMESPACE}.svc.cluster.local"
+ port: 9000
+ accessStyle: Path
+ credentials:
+ secretClass: s3-credentials-class
+ tls:
+ verification:
+ server:
+ caCert:
+ secretClass: tls
+---
+apiVersion: secrets.stackable.tech/v1alpha1
+kind: SecretClass
+metadata:
+ name: s3-credentials-class
+spec:
+ backend:
+ k8sSearch:
+ searchNamespace:
+ pod: {}
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: minio-credentials
+ labels:
+ secrets.stackable.tech/class: s3-credentials-class
+stringData:
+ accessKey: admin
+ secretKey: adminadmin
diff --git a/tests/templates/kuttl/iceberg-rest/10-assert.yaml.j2 b/tests/templates/kuttl/iceberg-rest/10-assert.yaml.j2
new file mode 100644
index 00000000..50b1d4c3
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/10-assert.yaml.j2
@@ -0,0 +1,10 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: vector-aggregator-discovery
+{% endif %}
diff --git a/tests/templates/kuttl/iceberg-rest/10-install-vector-aggregator-discovery-configmap.yaml.j2 b/tests/templates/kuttl/iceberg-rest/10-install-vector-aggregator-discovery-configmap.yaml.j2
new file mode 100644
index 00000000..2d6a0df5
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/10-install-vector-aggregator-discovery-configmap.yaml.j2
@@ -0,0 +1,9 @@
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: vector-aggregator-discovery
+data:
+ ADDRESS: {{ lookup('env', 'VECTOR_AGGREGATOR') }}
+{% endif %}
diff --git a/tests/templates/kuttl/iceberg-rest/20-assert.yaml b/tests/templates/kuttl/iceberg-rest/20-assert.yaml
new file mode 100644
index 00000000..477bdd02
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/20-assert.yaml
@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 600
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: minio
+status:
+ readyReplicas: 1
+ replicas: 1
diff --git a/tests/templates/kuttl/iceberg-rest/20-install-minio.yaml b/tests/templates/kuttl/iceberg-rest/20-install-minio.yaml
new file mode 100644
index 00000000..985b51e8
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/20-install-minio.yaml
@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ - script: kubectl -n $NAMESPACE apply -f 20_minio.yaml
diff --git a/tests/templates/kuttl/iceberg-rest/20_minio.yaml b/tests/templates/kuttl/iceberg-rest/20_minio.yaml
new file mode 100644
index 00000000..3b9ffcbe
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/20_minio.yaml
@@ -0,0 +1,579 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: "minio-sa"
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: minio
+ labels:
+ app: minio
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+type: Opaque
+data:
+ rootUser: "YWRtaW4="
+ rootPassword: "YWRtaW5hZG1pbg=="
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: minio
+ labels:
+ app: minio
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+data:
+ initialize: |-
+ #!/bin/sh
+ set -e # Have script exit in the event of a failed command.
+ MC_CONFIG_DIR="/etc/minio/mc/"
+ MC="/usr/bin/mc --insecure --config-dir ${MC_CONFIG_DIR}"
+
+ # connectToMinio
+ # Use a check-sleep-check loop to wait for MinIO service to be available
+ connectToMinio() {
+ SCHEME=$1
+ ATTEMPTS=0
+ LIMIT=29 # Allow 30 attempts
+ set -e # fail if we can't read the keys.
+ ACCESS=$(cat /config/rootUser)
+ SECRET=$(cat /config/rootPassword)
+ set +e # The connections to minio are allowed to fail.
+ echo "Connecting to MinIO server: $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT"
+ MC_COMMAND="${MC} alias set myminio $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT $ACCESS $SECRET"
+ $MC_COMMAND
+ STATUS=$?
+ until [ $STATUS = 0 ]; do
+ ATTEMPTS=$(expr $ATTEMPTS + 1)
+ echo \"Failed attempts: $ATTEMPTS\"
+ if [ $ATTEMPTS -gt $LIMIT ]; then
+ exit 1
+ fi
+ sleep 2 # 1 second intervals between attempts
+ $MC_COMMAND
+ STATUS=$?
+ done
+ set -e # reset `e` as active
+ return 0
+ }
+
+ # checkBucketExists ($bucket)
+ # Check if the bucket exists, by using the exit code of `mc ls`
+ checkBucketExists() {
+ BUCKET=$1
+ CMD=$(${MC} stat myminio/$BUCKET >/dev/null 2>&1)
+ return $?
+ }
+
+ # createBucket ($bucket, $policy, $purge)
+ # Ensure bucket exists, purging if asked to
+ createBucket() {
+ BUCKET=$1
+ POLICY=$2
+ PURGE=$3
+ VERSIONING=$4
+ OBJECTLOCKING=$5
+
+ # Purge the bucket, if set & exists
+ # Since PURGE is user input, check explicitly for `true`
+ if [ $PURGE = true ]; then
+ if checkBucketExists $BUCKET; then
+ echo "Purging bucket '$BUCKET'."
+ set +e # don't exit if this fails
+ ${MC} rm -r --force myminio/$BUCKET
+ set -e # reset `e` as active
+ else
+ echo "Bucket '$BUCKET' does not exist, skipping purge."
+ fi
+ fi
+
+ # Create the bucket if it does not exist and set objectlocking if enabled (NOTE: versioning will be not changed if OBJECTLOCKING is set because it enables versioning to the Buckets created)
+ if ! checkBucketExists $BUCKET; then
+ if [ ! -z $OBJECTLOCKING ]; then
+ if [ $OBJECTLOCKING = true ]; then
+ echo "Creating bucket with OBJECTLOCKING '$BUCKET'"
+ ${MC} mb --with-lock myminio/$BUCKET
+ elif [ $OBJECTLOCKING = false ]; then
+ echo "Creating bucket '$BUCKET'"
+ ${MC} mb myminio/$BUCKET
+ fi
+ elif [ -z $OBJECTLOCKING ]; then
+ echo "Creating bucket '$BUCKET'"
+ ${MC} mb myminio/$BUCKET
+ else
+ echo "Bucket '$BUCKET' already exists."
+ fi
+ fi
+
+ # set versioning for bucket if objectlocking is disabled or not set
+ if [ $OBJECTLOCKING = false ]; then
+ if [ ! -z $VERSIONING ]; then
+ if [ $VERSIONING = true ]; then
+ echo "Enabling versioning for '$BUCKET'"
+ ${MC} version enable myminio/$BUCKET
+ elif [ $VERSIONING = false ]; then
+ echo "Suspending versioning for '$BUCKET'"
+ ${MC} version suspend myminio/$BUCKET
+ fi
+ fi
+ else
+ echo "Bucket '$BUCKET' versioning unchanged."
+ fi
+
+ # At this point, the bucket should exist, skip checking for existence
+ # Set policy on the bucket
+ echo "Setting policy of bucket '$BUCKET' to '$POLICY'."
+ ${MC} anonymous set $POLICY myminio/$BUCKET
+ }
+
+ # Try connecting to MinIO instance
+ scheme=https
+ connectToMinio $scheme
+
+ # Create the buckets
+ createBucket demo "public" false false false
+
+ add-user: |-
+ #!/bin/sh
+ set -e ; # Have script exit in the event of a failed command.
+ MC_CONFIG_DIR="/etc/minio/mc/"
+ MC="/usr/bin/mc --insecure --config-dir ${MC_CONFIG_DIR}"
+
+ # AccessKey and secretkey credentials file are added to prevent shell execution errors caused by special characters.
+ # Special characters for example : ',",<,>,{,}
+ MINIO_ACCESSKEY_SECRETKEY_TMP="/tmp/accessKey_and_secretKey_tmp"
+
+ # connectToMinio
+ # Use a check-sleep-check loop to wait for MinIO service to be available
+ connectToMinio() {
+ SCHEME=$1
+ ATTEMPTS=0 ; LIMIT=29 ; # Allow 30 attempts
+ set -e ; # fail if we can't read the keys.
+ ACCESS=$(cat /config/rootUser) ; SECRET=$(cat /config/rootPassword) ;
+ set +e ; # The connections to minio are allowed to fail.
+ echo "Connecting to MinIO server: $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT" ;
+ MC_COMMAND="${MC} alias set myminio $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT $ACCESS $SECRET" ;
+ $MC_COMMAND ;
+ STATUS=$? ;
+ until [ $STATUS = 0 ]
+ do
+ ATTEMPTS=`expr $ATTEMPTS + 1` ;
+ echo \"Failed attempts: $ATTEMPTS\" ;
+ if [ $ATTEMPTS -gt $LIMIT ]; then
+ exit 1 ;
+ fi ;
+ sleep 2 ; # 1 second intervals between attempts
+ $MC_COMMAND ;
+ STATUS=$? ;
+ done ;
+ set -e ; # reset `e` as active
+ return 0
+ }
+
+ # checkUserExists ()
+ # Check if the user exists, by using the exit code of `mc admin user info`
+ checkUserExists() {
+ CMD=$(${MC} admin user info myminio $(head -1 $MINIO_ACCESSKEY_SECRETKEY_TMP) > /dev/null 2>&1)
+ return $?
+ }
+
+ # createUser ($policy)
+ createUser() {
+ POLICY=$1
+ #check accessKey_and_secretKey_tmp file
+ if [[ ! -f $MINIO_ACCESSKEY_SECRETKEY_TMP ]];then
+ echo "credentials file does not exist"
+ return 1
+ fi
+ if [[ $(cat $MINIO_ACCESSKEY_SECRETKEY_TMP|wc -l) -ne 2 ]];then
+ echo "credentials file is invalid"
+ rm -f $MINIO_ACCESSKEY_SECRETKEY_TMP
+ return 1
+ fi
+ USER=$(head -1 $MINIO_ACCESSKEY_SECRETKEY_TMP)
+ # Create the user if it does not exist
+ if ! checkUserExists ; then
+ echo "Creating user '$USER'"
+ cat $MINIO_ACCESSKEY_SECRETKEY_TMP | ${MC} admin user add myminio
+ else
+ echo "User '$USER' already exists."
+ fi
+ #clean up credentials files.
+ rm -f $MINIO_ACCESSKEY_SECRETKEY_TMP
+
+ # set policy for user
+ if [ ! -z $POLICY -a $POLICY != " " ] ; then
+ echo "Adding policy '$POLICY' for '$USER'"
+ set +e ; # policy already attach errors out, allow it.
+ ${MC} admin policy attach myminio $POLICY --user=$USER
+ set -e
+ else
+ echo "User '$USER' has no policy attached."
+ fi
+ }
+
+ # Try connecting to MinIO instance
+ scheme=https
+ connectToMinio $scheme
+
+ # Create the users
+ echo console > $MINIO_ACCESSKEY_SECRETKEY_TMP
+ echo console123 >> $MINIO_ACCESSKEY_SECRETKEY_TMP
+ createUser consoleAdmin
+
+ add-policy: |-
+ #!/bin/sh
+ set -e ; # Have script exit in the event of a failed command.
+ MC_CONFIG_DIR="/etc/minio/mc/"
+ MC="/usr/bin/mc --insecure --config-dir ${MC_CONFIG_DIR}"
+
+ # connectToMinio
+ # Use a check-sleep-check loop to wait for MinIO service to be available
+ connectToMinio() {
+ SCHEME=$1
+ ATTEMPTS=0 ; LIMIT=29 ; # Allow 30 attempts
+ set -e ; # fail if we can't read the keys.
+ ACCESS=$(cat /config/rootUser) ; SECRET=$(cat /config/rootPassword) ;
+ set +e ; # The connections to minio are allowed to fail.
+ echo "Connecting to MinIO server: $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT" ;
+ MC_COMMAND="${MC} alias set myminio $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT $ACCESS $SECRET" ;
+ $MC_COMMAND ;
+ STATUS=$? ;
+ until [ $STATUS = 0 ]
+ do
+ ATTEMPTS=`expr $ATTEMPTS + 1` ;
+ echo \"Failed attempts: $ATTEMPTS\" ;
+ if [ $ATTEMPTS -gt $LIMIT ]; then
+ exit 1 ;
+ fi ;
+ sleep 2 ; # 1 second intervals between attempts
+ $MC_COMMAND ;
+ STATUS=$? ;
+ done ;
+ set -e ; # reset `e` as active
+ return 0
+ }
+
+ # checkPolicyExists ($policy)
+ # Check if the policy exists, by using the exit code of `mc admin policy info`
+ checkPolicyExists() {
+ POLICY=$1
+ CMD=$(${MC} admin policy info myminio $POLICY > /dev/null 2>&1)
+ return $?
+ }
+
+ # createPolicy($name, $filename)
+ createPolicy () {
+ NAME=$1
+ FILENAME=$2
+
+ # Create the name if it does not exist
+ echo "Checking policy: $NAME (in /config/$FILENAME.json)"
+ if ! checkPolicyExists $NAME ; then
+ echo "Creating policy '$NAME'"
+ else
+ echo "Policy '$NAME' already exists."
+ fi
+ ${MC} admin policy create myminio $NAME /config/$FILENAME.json
+
+ }
+
+ # Try connecting to MinIO instance
+ scheme=https
+ connectToMinio $scheme
+
+ add-svcacct: |-
+ #!/bin/sh
+ set -e ; # Have script exit in the event of a failed command.
+ MC_CONFIG_DIR="/etc/minio/mc/"
+ MC="/usr/bin/mc --insecure --config-dir ${MC_CONFIG_DIR}"
+
+ # AccessKey and secretkey credentials file are added to prevent shell execution errors caused by special characters.
+ # Special characters for example : ',",<,>,{,}
+ MINIO_ACCESSKEY_SECRETKEY_TMP="/tmp/accessKey_and_secretKey_svcacct_tmp"
+
+ # connectToMinio
+ # Use a check-sleep-check loop to wait for MinIO service to be available
+ connectToMinio() {
+ SCHEME=$1
+ ATTEMPTS=0 ; LIMIT=29 ; # Allow 30 attempts
+ set -e ; # fail if we can't read the keys.
+ ACCESS=$(cat /config/rootUser) ; SECRET=$(cat /config/rootPassword) ;
+ set +e ; # The connections to minio are allowed to fail.
+ echo "Connecting to MinIO server: $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT" ;
+ MC_COMMAND="${MC} alias set myminio $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT $ACCESS $SECRET" ;
+ $MC_COMMAND ;
+ STATUS=$? ;
+ until [ $STATUS = 0 ]
+ do
+ ATTEMPTS=`expr $ATTEMPTS + 1` ;
+ echo \"Failed attempts: $ATTEMPTS\" ;
+ if [ $ATTEMPTS -gt $LIMIT ]; then
+ exit 1 ;
+ fi ;
+ sleep 2 ; # 2 second intervals between attempts
+ $MC_COMMAND ;
+ STATUS=$? ;
+ done ;
+ set -e ; # reset `e` as active
+ return 0
+ }
+
+ # checkSvcacctExists ()
+ # Check if the svcacct exists, by using the exit code of `mc admin user svcacct info`
+ checkSvcacctExists() {
+ CMD=$(${MC} admin user svcacct info myminio $(head -1 $MINIO_ACCESSKEY_SECRETKEY_TMP) > /dev/null 2>&1)
+ return $?
+ }
+
+ # createSvcacct ($user)
+ createSvcacct () {
+ USER=$1
+ FILENAME=$2
+ #check accessKey_and_secretKey_tmp file
+ if [[ ! -f $MINIO_ACCESSKEY_SECRETKEY_TMP ]];then
+ echo "credentials file does not exist"
+ return 1
+ fi
+ if [[ $(cat $MINIO_ACCESSKEY_SECRETKEY_TMP|wc -l) -ne 2 ]];then
+ echo "credentials file is invalid"
+ rm -f $MINIO_ACCESSKEY_SECRETKEY_TMP
+ return 1
+ fi
+ SVCACCT=$(head -1 $MINIO_ACCESSKEY_SECRETKEY_TMP)
+ # Create the svcacct if it does not exist
+ if ! checkSvcacctExists ; then
+ echo "Creating svcacct '$SVCACCT'"
+ # Check if policy file is define
+ if [ -z $FILENAME ]; then
+ ${MC} admin user svcacct add --access-key $(head -1 $MINIO_ACCESSKEY_SECRETKEY_TMP) --secret-key $(tail -n1 $MINIO_ACCESSKEY_SECRETKEY_TMP) myminio $USER
+ else
+ ${MC} admin user svcacct add --access-key $(head -1 $MINIO_ACCESSKEY_SECRETKEY_TMP) --secret-key $(tail -n1 $MINIO_ACCESSKEY_SECRETKEY_TMP) --policy /config/$FILENAME.json myminio $USER
+ fi
+ else
+ echo "Svcacct '$SVCACCT' already exists."
+ fi
+ #clean up credentials files.
+ rm -f $MINIO_ACCESSKEY_SECRETKEY_TMP
+ }
+
+ # Try connecting to MinIO instance
+ scheme=https
+ connectToMinio $scheme
+
+ custom-command: |-
+ #!/bin/sh
+ set -e ; # Have script exit in the event of a failed command.
+ MC_CONFIG_DIR="/etc/minio/mc/"
+ MC="/usr/bin/mc --insecure --config-dir ${MC_CONFIG_DIR}"
+
+ # connectToMinio
+ # Use a check-sleep-check loop to wait for MinIO service to be available
+ connectToMinio() {
+ SCHEME=$1
+ ATTEMPTS=0 ; LIMIT=29 ; # Allow 30 attempts
+ set -e ; # fail if we can't read the keys.
+ ACCESS=$(cat /config/rootUser) ; SECRET=$(cat /config/rootPassword) ;
+ set +e ; # The connections to minio are allowed to fail.
+ echo "Connecting to MinIO server: $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT" ;
+ MC_COMMAND="${MC} alias set myminio $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT $ACCESS $SECRET" ;
+ $MC_COMMAND ;
+ STATUS=$? ;
+ until [ $STATUS = 0 ]
+ do
+ ATTEMPTS=`expr $ATTEMPTS + 1` ;
+ echo \"Failed attempts: $ATTEMPTS\" ;
+ if [ $ATTEMPTS -gt $LIMIT ]; then
+ exit 1 ;
+ fi ;
+ sleep 2 ; # 1 second intervals between attempts
+ $MC_COMMAND ;
+ STATUS=$? ;
+ done ;
+ set -e ; # reset `e` as active
+ return 0
+ }
+
+ # runCommand ($@)
+ # Run custom mc command
+ runCommand() {
+ ${MC} "$@"
+ return $?
+ }
+
+ # Try connecting to MinIO instance
+ scheme=https
+ connectToMinio $scheme
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: minio
+ labels:
+ app: minio
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+spec:
+ accessModes:
+ - "ReadWriteOnce"
+ resources:
+ requests:
+ storage: "10Gi"
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: minio-console
+ labels:
+ app: minio
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+spec:
+ type: NodePort
+ externalTrafficPolicy: "Cluster"
+ ports:
+ - name: https
+ port: 9001
+ protocol: TCP
+ targetPort: 9001
+ selector:
+ app: minio
+ release: minio
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: minio
+ labels:
+ app: minio
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+ monitoring: "true"
+spec:
+ type: NodePort
+ externalTrafficPolicy: "Cluster"
+ ports:
+ - name: https
+ port: 9000
+ protocol: TCP
+ targetPort: 9000
+ selector:
+ app: minio
+ release: minio
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: minio
+ labels:
+ app: minio
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+ stackable.tech/vendor: Stackable
+spec:
+ strategy:
+ type: RollingUpdate
+ rollingUpdate:
+ maxSurge: 100%
+ maxUnavailable: 0
+ replicas: 1
+ selector:
+ matchLabels:
+ app: minio
+ release: minio
+ template:
+ metadata:
+ name: minio
+ labels:
+ app: minio
+ release: minio
+ stackable.tech/vendor: Stackable
+ annotations:
+ checksum/secrets: fa63e34a92c817c84057e2d452fa683e66462a57b0529388fb96a57e05f38e57
+ checksum/config: ebea49cc4c1bfbd1b156a58bf770a776ff87fe199f642d31c2816b5515112e72
+ spec:
+ securityContext:
+ fsGroupChangePolicy: OnRootMismatch
+ serviceAccountName: minio-sa
+ containers:
+ - name: minio
+ image: "quay.io/minio/minio:RELEASE.2024-12-18T13-15-44Z"
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/bin/sh"
+ - "-ce"
+ - |
+ # minio requires the TLS key pair to be specially named
+ # mkdir -p /etc/minio/certs
+ cp -v /etc/minio/original_certs/tls.crt /etc/minio/certs/public.crt
+ cp -v /etc/minio/original_certs/tls.key /etc/minio/certs/private.key
+
+ /usr/bin/docker-entrypoint.sh minio server /export -S /etc/minio/certs/ --address :9000 --console-address :9001
+ volumeMounts:
+ - name: minio-user
+ mountPath: "/tmp/credentials"
+ readOnly: true
+ - name: export
+ mountPath: /export
+ - mountPath: /etc/minio/original_certs
+ name: tls
+ - mountPath: /etc/minio/certs
+ name: certs
+ ports:
+ - name: https
+ containerPort: 9000
+ - name: https-console
+ containerPort: 9001
+ env:
+ - name: MINIO_ROOT_USER
+ valueFrom:
+ secretKeyRef:
+ name: minio
+ key: rootUser
+ - name: MINIO_ROOT_PASSWORD
+ valueFrom:
+ secretKeyRef:
+ name: minio
+ key: rootPassword
+ - name: MINIO_PROMETHEUS_AUTH_TYPE
+ value: "public"
+ resources:
+ requests:
+ cpu: 1
+ memory: 2Gi
+ securityContext:
+ readOnlyRootFilesystem: false
+ volumes:
+ - name: export
+ persistentVolumeClaim:
+ claimName: minio
+ - name: minio-user
+ secret:
+ secretName: minio
+ - ephemeral:
+ volumeClaimTemplate:
+ metadata:
+ annotations:
+ secrets.stackable.tech/class: tls
+ secrets.stackable.tech/scope: service=minio
+ spec:
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 1
+ storageClassName: secrets.stackable.tech
+ name: tls
+ - emptyDir:
+ medium: Memory
+ sizeLimit: 5Mi
+ name: certs
diff --git a/tests/templates/kuttl/iceberg-rest/21-assert.yaml b/tests/templates/kuttl/iceberg-rest/21-assert.yaml
new file mode 100644
index 00000000..3895aff4
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/21-assert.yaml
@@ -0,0 +1,11 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 600
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: minio-post-job
+status:
+ succeeded: 1
diff --git a/tests/templates/kuttl/iceberg-rest/21-install-minio-jobs.yaml b/tests/templates/kuttl/iceberg-rest/21-install-minio-jobs.yaml
new file mode 100644
index 00000000..d51dae4b
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/21-install-minio-jobs.yaml
@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ - script: kubectl -n $NAMESPACE apply -f 21_minio_jobs.yaml
diff --git a/tests/templates/kuttl/iceberg-rest/21_minio_jobs.yaml b/tests/templates/kuttl/iceberg-rest/21_minio_jobs.yaml
new file mode 100644
index 00000000..bd8f3ac4
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/21_minio_jobs.yaml
@@ -0,0 +1,116 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: minio-post-job
+ labels:
+ app: minio-post-job
+ chart: minio-5.4.0
+ release: minio
+ heritage: Helm
+ annotations:
+ "helm.sh/hook": post-install,post-upgrade
+ "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
+spec:
+ template:
+ metadata:
+ labels:
+ app: minio-job
+ release: minio
+ stackable.tech/vendor: Stackable
+ spec:
+ restartPolicy: OnFailure
+ volumes:
+ - name: etc-path
+ emptyDir: {}
+ - name: tmp
+ emptyDir: {}
+ - name: minio-configuration
+ projected:
+ sources:
+ - configMap:
+ name: minio
+ - secret:
+ name: minio
+ - ephemeral:
+ volumeClaimTemplate:
+ metadata:
+ annotations:
+ secrets.stackable.tech/class: tls
+ secrets.stackable.tech/scope: service=minio
+ spec:
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 1
+ storageClassName: secrets.stackable.tech
+ name: tls
+ - emptyDir:
+ medium: Memory
+ sizeLimit: 5Mi
+ name: certs
+ serviceAccountName: minio-sa
+ containers:
+ - name: minio-make-bucket
+ image: "quay.io/minio/mc:RELEASE.2024-11-21T17-21-54Z"
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/bin/sh"
+ - "-ce"
+ - |
+ # Copy the CA cert from the "tls" SecretClass
+ # mkdir -p /etc/minio/mc/certs/CAs
+ cp -v /etc/minio/mc/original_certs/ca.crt /etc/minio/mc/certs/CAs/public.crt
+
+ . /config/initialize
+ env:
+ - name: MINIO_ENDPOINT
+ value: minio
+ - name: MINIO_PORT
+ value: "9000"
+ volumeMounts:
+ - name: etc-path
+ mountPath: /etc/minio/mc
+ - name: tmp
+ mountPath: /tmp
+ - name: minio-configuration
+ mountPath: /config
+ - name: tls
+ mountPath: /etc/minio/mc/original_certs
+ - name: certs
+ mountPath: /etc/minio/mc/certs/CAs
+ resources:
+ requests:
+ memory: 128Mi
+ - name: minio-make-user
+ image: "quay.io/minio/mc:RELEASE.2024-11-21T17-21-54Z"
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/bin/sh"
+ - "-ce"
+ - |
+ # Copy the CA cert from the "tls" SecretClass
+ # mkdir -p /etc/minio/mc/certs/CAs
+ cp -v /etc/minio/mc/original_certs/ca.crt /etc/minio/mc/certs/CAs/public.crt
+
+ . /config/add-user
+ env:
+ - name: MINIO_ENDPOINT
+ value: minio
+ - name: MINIO_PORT
+ value: "9000"
+ volumeMounts:
+ - name: etc-path
+ mountPath: /etc/minio/mc
+ - name: tmp
+ mountPath: /tmp
+ - name: minio-configuration
+ mountPath: /config
+ - name: tls
+ mountPath: /etc/minio/mc/original_certs
+ - name: certs
+ mountPath: /etc/minio/mc/certs/CAs
+ resources:
+ requests:
+ memory: 128Mi
diff --git a/tests/templates/kuttl/iceberg-rest/22-assert.yaml b/tests/templates/kuttl/iceberg-rest/22-assert.yaml
new file mode 100644
index 00000000..943a1340
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/22-assert.yaml
@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 300
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: keycloak
+status:
+ readyReplicas: 1
+ replicas: 1
diff --git a/tests/templates/kuttl/iceberg-rest/22-install-keycloak.yaml b/tests/templates/kuttl/iceberg-rest/22-install-keycloak.yaml
new file mode 100644
index 00000000..00139c19
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/22-install-keycloak.yaml
@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ - script: envsubst < 22_keycloak.yaml | kubectl apply -n $NAMESPACE -f -
diff --git a/tests/templates/kuttl/iceberg-rest/22-keycloak-realm-cm.yaml.j2 b/tests/templates/kuttl/iceberg-rest/22-keycloak-realm-cm.yaml.j2
new file mode 100644
index 00000000..233077d8
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/22-keycloak-realm-cm.yaml.j2
@@ -0,0 +1,2840 @@
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: hive-metastore-client-credentials
+stringData:
+ clientId: hive-metastore
+ clientSecret: hive-metastore-secret
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: nifi-client-credentials
+stringData:
+ clientId: nifi
+ clientSecret: nifi-secret
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: trino-client-credentials
+stringData:
+ clientId: trino
+ clientSecret: trino-secret
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: keycloak-test-realm
+data:
+ # Exported in the NiFi WebUI.
+ # The passwords (search for **********) have been replaced with the actual values.
+ realm.json: |
+ {
+ "id": "6c19a6b3-a4f6-4021-8d7e-a23039edeafb",
+ "realm": "test",
+ "notBefore": 0,
+ "defaultSignatureAlgorithm": "RS256",
+ "revokeRefreshToken": false,
+ "refreshTokenMaxReuse": 0,
+ "accessTokenLifespan": 300,
+ "accessTokenLifespanForImplicitFlow": 900,
+ "ssoSessionIdleTimeout": 1800,
+ "ssoSessionMaxLifespan": 36000,
+ "ssoSessionIdleTimeoutRememberMe": 0,
+ "ssoSessionMaxLifespanRememberMe": 0,
+ "offlineSessionIdleTimeout": 2592000,
+ "offlineSessionMaxLifespanEnabled": false,
+ "offlineSessionMaxLifespan": 5184000,
+ "clientSessionIdleTimeout": 0,
+ "clientSessionMaxLifespan": 0,
+ "clientOfflineSessionIdleTimeout": 0,
+ "clientOfflineSessionMaxLifespan": 0,
+ "accessCodeLifespan": 60,
+ "accessCodeLifespanUserAction": 300,
+ "accessCodeLifespanLogin": 1800,
+ "actionTokenGeneratedByAdminLifespan": 43200,
+ "actionTokenGeneratedByUserLifespan": 300,
+ "oauth2DeviceCodeLifespan": 600,
+ "oauth2DevicePollingInterval": 5,
+ "enabled": true,
+ "sslRequired": "external",
+ "registrationAllowed": false,
+ "registrationEmailAsUsername": false,
+ "rememberMe": false,
+ "verifyEmail": false,
+ "loginWithEmailAllowed": true,
+ "duplicateEmailsAllowed": false,
+ "resetPasswordAllowed": false,
+ "editUsernameAllowed": false,
+ "bruteForceProtected": false,
+ "permanentLockout": false,
+ "maxTemporaryLockouts": 0,
+ "bruteForceStrategy": "MULTIPLE",
+ "maxFailureWaitSeconds": 900,
+ "minimumQuickLoginWaitSeconds": 60,
+ "waitIncrementSeconds": 60,
+ "quickLoginCheckMilliSeconds": 1000,
+ "maxDeltaTimeSeconds": 43200,
+ "failureFactor": 30,
+ "roles": {
+ "realm": [
+ {
+ "id": "dd05ac50-3e18-4497-96ab-0c3ad2ef63e7",
+ "name": "offline_access",
+ "description": "${role_offline-access}",
+ "composite": false,
+ "clientRole": false,
+ "containerId": "6c19a6b3-a4f6-4021-8d7e-a23039edeafb",
+ "attributes": {}
+ },
+ {
+ "id": "48850aba-5918-404d-af60-855f27917853",
+ "name": "uma_authorization",
+ "description": "${role_uma_authorization}",
+ "composite": false,
+ "clientRole": false,
+ "containerId": "6c19a6b3-a4f6-4021-8d7e-a23039edeafb",
+ "attributes": {}
+ },
+ {
+ "id": "385d57db-e490-423c-916e-63574955601c",
+ "name": "default-roles-test",
+ "description": "${role_default-roles}",
+ "composite": true,
+ "composites": {
+ "realm": [
+ "offline_access",
+ "uma_authorization"
+ ],
+ "client": {
+ "account": [
+ "view-profile",
+ "manage-account"
+ ]
+ }
+ },
+ "clientRole": false,
+ "containerId": "6c19a6b3-a4f6-4021-8d7e-a23039edeafb",
+ "attributes": {}
+ }
+ ],
+ "client": {
+ "realm-management": [
+ {
+ "id": "cef54936-d186-4db8-903d-ed76d15f923c",
+ "name": "manage-authorization",
+ "description": "${role_manage-authorization}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "60a50527-4141-4596-a8e7-371c51647666",
+ "name": "manage-users",
+ "description": "${role_manage-users}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "99e40d09-39bd-46c8-880e-964747a8bf20",
+ "name": "view-users",
+ "description": "${role_view-users}",
+ "composite": true,
+ "composites": {
+ "client": {
+ "realm-management": [
+ "query-groups",
+ "query-users"
+ ]
+ }
+ },
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "c8202ea1-b17e-4711-b58e-3b19d7028b5d",
+ "name": "realm-admin",
+ "description": "${role_realm-admin}",
+ "composite": true,
+ "composites": {
+ "client": {
+ "realm-management": [
+ "manage-authorization",
+ "manage-users",
+ "view-users",
+ "query-groups",
+ "view-clients",
+ "query-realms",
+ "manage-identity-providers",
+ "view-events",
+ "manage-clients",
+ "view-identity-providers",
+ "manage-events",
+ "impersonation",
+ "manage-realm",
+ "view-authorization",
+ "create-client",
+ "view-realm",
+ "query-clients",
+ "query-users"
+ ]
+ }
+ },
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "05e891a2-0b9f-4c0f-9996-b4b85aeedf79",
+ "name": "query-groups",
+ "description": "${role_query-groups}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "3a8600f0-b4f7-4655-bf47-d7df035539ea",
+ "name": "view-clients",
+ "description": "${role_view-clients}",
+ "composite": true,
+ "composites": {
+ "client": {
+ "realm-management": [
+ "query-clients"
+ ]
+ }
+ },
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "a58eccce-9639-49c8-82bd-ed2483dc04bb",
+ "name": "query-realms",
+ "description": "${role_query-realms}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "f4b6c1a2-1780-4080-9023-5ba327834dc8",
+ "name": "manage-identity-providers",
+ "description": "${role_manage-identity-providers}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "931f178c-5012-4330-ba6a-07d6fdf4355a",
+ "name": "manage-clients",
+ "description": "${role_manage-clients}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "86e6651c-fb78-4023-8000-b281025ec44f",
+ "name": "view-events",
+ "description": "${role_view-events}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "d4c3de49-d680-4a59-9c8d-236d13c9e2c3",
+ "name": "view-identity-providers",
+ "description": "${role_view-identity-providers}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "060f1be9-0a52-47ca-9d76-074f3f5d725f",
+ "name": "manage-events",
+ "description": "${role_manage-events}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "34fea702-27c3-4fb1-ba65-b97cdc908711",
+ "name": "impersonation",
+ "description": "${role_impersonation}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "c53f294f-cfd1-4ee4-82b4-cbcb7c3440d8",
+ "name": "manage-realm",
+ "description": "${role_manage-realm}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "f63b0a2e-b8dd-4dda-8865-5f492f434203",
+ "name": "create-client",
+ "description": "${role_create-client}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "733e615b-ccb5-488e-8f88-0260e67f3b12",
+ "name": "view-authorization",
+ "description": "${role_view-authorization}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "ea7ecc00-9798-4973-bfa3-d39456529708",
+ "name": "query-clients",
+ "description": "${role_query-clients}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "4f5981ca-831a-433f-905a-4c6c035d3979",
+ "name": "view-realm",
+ "description": "${role_view-realm}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ },
+ {
+ "id": "56c4359e-5e15-45dd-87e0-eecbb36c3419",
+ "name": "query-users",
+ "description": "${role_query-users}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "attributes": {}
+ }
+ ],
+ "hive-metastore": [],
+ "trino": [],
+ "security-admin-console": [],
+ "admin-cli": [],
+ "account-console": [],
+ "broker": [
+ {
+ "id": "95553cdb-2448-4a9f-a698-eb7814e68802",
+ "name": "read-token",
+ "description": "${role_read-token}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "8a88011e-b057-4e27-baf9-4357edc9b449",
+ "attributes": {}
+ }
+ ],
+ "nifi": [],
+ "account": [
+ {
+ "id": "3b41885d-39bc-43f8-8212-5264b4022203",
+ "name": "view-applications",
+ "description": "${role_view-applications}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "0110c6a1-aedc-43fa-886c-c0ba372bd12e",
+ "name": "view-consent",
+ "description": "${role_view-consent}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "b6caf49b-e6c5-41d2-8747-df694f9a25cb",
+ "name": "delete-account",
+ "description": "${role_delete-account}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "cf292f8f-ec5b-4303-a20b-d8aa2f502aa0",
+ "name": "view-groups",
+ "description": "${role_view-groups}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "9d3fd76b-589e-4136-a5f7-40d0171f05f5",
+ "name": "manage-account-links",
+ "description": "${role_manage-account-links}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "54f93339-4706-4ed6-9fc3-3052d5fc602e",
+ "name": "view-profile",
+ "description": "${role_view-profile}",
+ "composite": false,
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "4a54da67-ba4b-4488-8922-cfc70bd783fa",
+ "name": "manage-consent",
+ "description": "${role_manage-consent}",
+ "composite": true,
+ "composites": {
+ "client": {
+ "account": [
+ "view-consent"
+ ]
+ }
+ },
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ },
+ {
+ "id": "77a103a2-a1b8-43fa-bbc3-b488772f2827",
+ "name": "manage-account",
+ "description": "${role_manage-account}",
+ "composite": true,
+ "composites": {
+ "client": {
+ "account": [
+ "manage-account-links"
+ ]
+ }
+ },
+ "clientRole": true,
+ "containerId": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "attributes": {}
+ }
+ ]
+ }
+ },
+ "groups": [],
+ "defaultRole": {
+ "id": "385d57db-e490-423c-916e-63574955601c",
+ "name": "default-roles-test",
+ "description": "${role_default-roles}",
+ "composite": true,
+ "clientRole": false,
+ "containerId": "6c19a6b3-a4f6-4021-8d7e-a23039edeafb"
+ },
+ "requiredCredentials": [
+ "password"
+ ],
+ "otpPolicyType": "totp",
+ "otpPolicyAlgorithm": "HmacSHA1",
+ "otpPolicyInitialCounter": 0,
+ "otpPolicyDigits": 6,
+ "otpPolicyLookAheadWindow": 1,
+ "otpPolicyPeriod": 30,
+ "otpPolicyCodeReusable": false,
+ "otpSupportedApplications": [
+ "totpAppFreeOTPName",
+ "totpAppGoogleName",
+ "totpAppMicrosoftAuthenticatorName"
+ ],
+ "localizationTexts": {},
+ "webAuthnPolicyRpEntityName": "keycloak",
+ "webAuthnPolicySignatureAlgorithms": [
+ "ES256",
+ "RS256"
+ ],
+ "webAuthnPolicyRpId": "",
+ "webAuthnPolicyAttestationConveyancePreference": "not specified",
+ "webAuthnPolicyAuthenticatorAttachment": "not specified",
+ "webAuthnPolicyRequireResidentKey": "not specified",
+ "webAuthnPolicyUserVerificationRequirement": "not specified",
+ "webAuthnPolicyCreateTimeout": 0,
+ "webAuthnPolicyAvoidSameAuthenticatorRegister": false,
+ "webAuthnPolicyAcceptableAaguids": [],
+ "webAuthnPolicyExtraOrigins": [],
+ "webAuthnPolicyPasswordlessRpEntityName": "keycloak",
+ "webAuthnPolicyPasswordlessSignatureAlgorithms": [
+ "ES256",
+ "RS256"
+ ],
+ "webAuthnPolicyPasswordlessRpId": "",
+ "webAuthnPolicyPasswordlessAttestationConveyancePreference": "not specified",
+ "webAuthnPolicyPasswordlessAuthenticatorAttachment": "not specified",
+ "webAuthnPolicyPasswordlessRequireResidentKey": "Yes",
+ "webAuthnPolicyPasswordlessUserVerificationRequirement": "required",
+ "webAuthnPolicyPasswordlessCreateTimeout": 0,
+ "webAuthnPolicyPasswordlessAvoidSameAuthenticatorRegister": false,
+ "webAuthnPolicyPasswordlessAcceptableAaguids": [],
+ "webAuthnPolicyPasswordlessExtraOrigins": [],
+ "users": [
+ {
+ "id": "d0512df3-4ff4-4c9f-b396-1e83de53447e",
+ "username": "service-account-nifi",
+ "emailVerified": false,
+ "enabled": true,
+ "createdTimestamp": 1769612108575,
+ "totp": false,
+ "serviceAccountClientId": "nifi",
+ "disableableCredentialTypes": [],
+ "requiredActions": [],
+ "realmRoles": [
+ "default-roles-test"
+ ],
+ "notBefore": 0,
+ "groups": []
+ },
+ {
+ "id": "f3af224a-75a7-4edf-a7b3-6e3a080beffc",
+ "username": "service-account-trino",
+ "emailVerified": false,
+ "enabled": true,
+ "createdTimestamp": 1769610938148,
+ "totp": false,
+ "serviceAccountClientId": "trino",
+ "disableableCredentialTypes": [],
+ "requiredActions": [],
+ "realmRoles": [
+ "default-roles-test"
+ ],
+ "notBefore": 0,
+ "groups": []
+ }
+ ],
+ "scopeMappings": [
+ {
+ "clientScope": "offline_access",
+ "roles": [
+ "offline_access"
+ ]
+ }
+ ],
+ "clientScopeMappings": {
+ "account": [
+ {
+ "client": "account-console",
+ "roles": [
+ "manage-account",
+ "view-groups"
+ ]
+ }
+ ]
+ },
+ "clients": [
+ {
+ "id": "1bba6002-b0fc-46bc-804c-c31606ee190f",
+ "clientId": "account",
+ "name": "${client_account}",
+ "rootUrl": "${authBaseUrl}",
+ "baseUrl": "/realms/test/account/",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "redirectUris": [
+ "/realms/test/account/*"
+ ],
+ "webOrigins": [],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": true,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": false,
+ "publicClient": true,
+ "frontchannelLogout": false,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "false",
+ "post.logout.redirect.uris": "+"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": false,
+ "nodeReRegistrationTimeout": 0,
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "2f8ab6ca-509f-4ab3-acc9-497565b4d05f",
+ "clientId": "account-console",
+ "name": "${client_account-console}",
+ "rootUrl": "${authBaseUrl}",
+ "baseUrl": "/realms/test/account/",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "redirectUris": [
+ "/realms/test/account/*"
+ ],
+ "webOrigins": [],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": true,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": false,
+ "publicClient": true,
+ "frontchannelLogout": false,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "false",
+ "post.logout.redirect.uris": "+",
+ "pkce.code.challenge.method": "S256"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": false,
+ "nodeReRegistrationTimeout": 0,
+ "protocolMappers": [
+ {
+ "id": "c99bbca7-8e54-4452-8ce5-ac7ebf60b072",
+ "name": "audience resolve",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-audience-resolve-mapper",
+ "consentRequired": false,
+ "config": {}
+ }
+ ],
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "f6b141e6-cb19-421e-a1dd-7cc7e9d377b5",
+ "clientId": "admin-cli",
+ "name": "${client_admin-cli}",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "redirectUris": [],
+ "webOrigins": [],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": false,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": true,
+ "serviceAccountsEnabled": false,
+ "publicClient": true,
+ "frontchannelLogout": false,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "false",
+ "client.use.lightweight.access.token.enabled": "true",
+ "post.logout.redirect.uris": "+"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": true,
+ "nodeReRegistrationTimeout": 0,
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "8a88011e-b057-4e27-baf9-4357edc9b449",
+ "clientId": "broker",
+ "name": "${client_broker}",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "redirectUris": [],
+ "webOrigins": [],
+ "notBefore": 0,
+ "bearerOnly": true,
+ "consentRequired": false,
+ "standardFlowEnabled": true,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": false,
+ "publicClient": false,
+ "frontchannelLogout": false,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "true",
+ "post.logout.redirect.uris": "+"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": false,
+ "nodeReRegistrationTimeout": 0,
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "795992e7-b5bc-44c4-8ecf-25e93ded17a8",
+ "clientId": "hive-metastore",
+ "name": "",
+ "description": "",
+ "rootUrl": "",
+ "adminUrl": "",
+ "baseUrl": "",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "secret": "hive-metastore-secret",
+ "redirectUris": [
+ "/*"
+ ],
+ "webOrigins": [
+ "/*"
+ ],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": false,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": false,
+ "publicClient": false,
+ "frontchannelLogout": true,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "false",
+ "oidc.ciba.grant.enabled": "false",
+ "client.secret.creation.time": "1769610831",
+ "backchannel.logout.session.required": "true",
+ "standard.token.exchange.enabled": "false",
+ "post.logout.redirect.uris": "+",
+ "oauth2.device.authorization.grant.enabled": "false",
+ "backchannel.logout.revoke.offline.tokens": "false",
+ "dpop.bound.access.tokens": "false"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": true,
+ "nodeReRegistrationTimeout": -1,
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "b01e4cb5-1446-491c-8689-20ee8a9ee1e1",
+ "clientId": "nifi",
+ "name": "",
+ "description": "",
+ "rootUrl": "",
+ "adminUrl": "",
+ "baseUrl": "",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "secret": "nifi-secret",
+ "redirectUris": [
+ "/*"
+ ],
+ "webOrigins": [
+ "/*"
+ ],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": false,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": true,
+ "publicClient": false,
+ "frontchannelLogout": true,
+ "protocol": "openid-connect",
+ "attributes": {
+ "logout.confirmation.enabled": "false",
+ "id.token.as.detached.signature": "false",
+ "client.secret.creation.time": "1769612108",
+ "client.introspection.response.allow.jwt.claim.enabled": "false",
+ "standard.token.exchange.enabled": "false",
+ "frontchannel.logout.session.required": "true",
+ "oauth2.device.authorization.grant.enabled": "false",
+ "backchannel.logout.revoke.offline.tokens": "false",
+ "use.refresh.tokens": "true",
+ "realm_client": "false",
+ "oidc.ciba.grant.enabled": "false",
+ "client.use.lightweight.access.token.enabled": "false",
+ "backchannel.logout.session.required": "true",
+ "request.object.required": "not required",
+ "client_credentials.use_refresh_token": "false",
+ "access.token.header.type.rfc9068": "true",
+ "tls.client.certificate.bound.access.tokens": "false",
+ "require.pushed.authorization.requests": "false",
+ "acr.loa.map": "{}",
+ "display.on.consent.screen": "false",
+ "token.response.type.bearer.lower-case": "false",
+ "dpop.bound.access.tokens": "false"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": true,
+ "nodeReRegistrationTimeout": -1,
+ "defaultClientScopes": [
+ "web-origins",
+ "service_account",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "catalog",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "50cabde2-ecd2-4229-9c6c-af4e3f66a0ee",
+ "clientId": "realm-management",
+ "name": "${client_realm-management}",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "redirectUris": [],
+ "webOrigins": [],
+ "notBefore": 0,
+ "bearerOnly": true,
+ "consentRequired": false,
+ "standardFlowEnabled": true,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": false,
+ "publicClient": false,
+ "frontchannelLogout": false,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "true",
+ "post.logout.redirect.uris": "+"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": false,
+ "nodeReRegistrationTimeout": 0,
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "2a9e8ab3-8342-461d-83b9-4c3a4cc38c1a",
+ "clientId": "security-admin-console",
+ "name": "${client_security-admin-console}",
+ "rootUrl": "${authAdminUrl}",
+ "baseUrl": "/admin/test/console/",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "redirectUris": [
+ "/admin/test/console/*"
+ ],
+ "webOrigins": [
+ "+"
+ ],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": true,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": false,
+ "publicClient": true,
+ "frontchannelLogout": false,
+ "protocol": "openid-connect",
+ "attributes": {
+ "realm_client": "false",
+ "client.use.lightweight.access.token.enabled": "true",
+ "post.logout.redirect.uris": "+",
+ "pkce.code.challenge.method": "S256"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": true,
+ "nodeReRegistrationTimeout": 0,
+ "protocolMappers": [
+ {
+ "id": "db67b4b9-707a-4e35-89de-3405fa2aa824",
+ "name": "locale",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "locale",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "locale",
+ "jsonType.label": "String"
+ }
+ }
+ ],
+ "defaultClientScopes": [
+ "web-origins",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ },
+ {
+ "id": "8ad43616-ebf1-4939-b3cc-0f3d92040af5",
+ "clientId": "trino",
+ "name": "",
+ "description": "",
+ "rootUrl": "",
+ "adminUrl": "",
+ "baseUrl": "",
+ "surrogateAuthRequired": false,
+ "enabled": true,
+ "alwaysDisplayInConsole": false,
+ "clientAuthenticatorType": "client-secret",
+ "secret": "trino-secret",
+ "redirectUris": [
+ "/*"
+ ],
+ "webOrigins": [
+ "/*"
+ ],
+ "notBefore": 0,
+ "bearerOnly": false,
+ "consentRequired": false,
+ "standardFlowEnabled": false,
+ "implicitFlowEnabled": false,
+ "directAccessGrantsEnabled": false,
+ "serviceAccountsEnabled": true,
+ "publicClient": false,
+ "frontchannelLogout": true,
+ "protocol": "openid-connect",
+ "attributes": {
+ "logout.confirmation.enabled": "false",
+ "id.token.as.detached.signature": "false",
+ "client.secret.creation.time": "1769610938",
+ "client.introspection.response.allow.jwt.claim.enabled": "false",
+ "standard.token.exchange.enabled": "false",
+ "frontchannel.logout.session.required": "true",
+ "post.logout.redirect.uris": "+",
+ "oauth2.device.authorization.grant.enabled": "false",
+ "backchannel.logout.revoke.offline.tokens": "false",
+ "use.refresh.tokens": "true",
+ "realm_client": "false",
+ "oidc.ciba.grant.enabled": "false",
+ "client.use.lightweight.access.token.enabled": "false",
+ "backchannel.logout.session.required": "true",
+ "request.object.required": "not required",
+ "client_credentials.use_refresh_token": "false",
+ "access.token.header.type.rfc9068": "true",
+ "tls.client.certificate.bound.access.tokens": "false",
+ "require.pushed.authorization.requests": "false",
+ "acr.loa.map": "{}",
+ "display.on.consent.screen": "false",
+ "token.response.type.bearer.lower-case": "false",
+ "dpop.bound.access.tokens": "false"
+ },
+ "authenticationFlowBindingOverrides": {},
+ "fullScopeAllowed": true,
+ "nodeReRegistrationTimeout": -1,
+ "defaultClientScopes": [
+ "web-origins",
+ "service_account",
+ "acr",
+ "roles",
+ "profile",
+ "basic",
+ "email"
+ ],
+ "optionalClientScopes": [
+ "address",
+ "phone",
+ "catalog",
+ "organization",
+ "offline_access",
+ "microprofile-jwt"
+ ]
+ }
+ ],
+ "clientScopes": [
+ {
+ "id": "21aec941-0685-4daa-9b51-17a726479a8d",
+ "name": "saml_organization",
+ "description": "Organization Membership",
+ "protocol": "saml",
+ "attributes": {
+ "display.on.consent.screen": "false"
+ },
+ "protocolMappers": [
+ {
+ "id": "76f7da0b-0039-413f-8d79-6d78890fe4c5",
+ "name": "organization",
+ "protocol": "saml",
+ "protocolMapper": "saml-organization-membership-mapper",
+ "consentRequired": false,
+ "config": {}
+ }
+ ]
+ },
+ {
+ "id": "64ae51bb-1816-4dec-9e11-6443f91cc6ac",
+ "name": "organization",
+ "description": "Additional claims about the organization a subject belongs to",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "consent.screen.text": "${organizationScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "07727f34-bf92-45f6-b66b-6b614bab6086",
+ "name": "organization",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-organization-membership-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "multivalued": "true",
+ "userinfo.token.claim": "true",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "organization",
+ "jsonType.label": "String"
+ }
+ }
+ ]
+ },
+ {
+ "id": "a9f86f55-aca0-45e2-ba60-02cde1dcc57a",
+ "name": "basic",
+ "description": "OpenID Connect scope for add all basic claims to the token",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "false",
+ "display.on.consent.screen": "false"
+ },
+ "protocolMappers": [
+ {
+ "id": "03c969ca-2200-4ed7-bc4c-9fc1f6d10cff",
+ "name": "sub",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-sub-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "access.token.claim": "true"
+ }
+ },
+ {
+ "id": "04bea7e2-112e-4a9d-be47-071f0d2d7cac",
+ "name": "auth_time",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usersessionmodel-note-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.session.note": "AUTH_TIME",
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "auth_time",
+ "jsonType.label": "long"
+ }
+ }
+ ]
+ },
+ {
+ "id": "28f7c1e2-6144-4e5d-9b5f-98500b2a7ff6",
+ "name": "profile",
+ "description": "OpenID Connect built-in scope: profile",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "consent.screen.text": "${profileScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "a7c64a12-edc7-4f66-be10-2baed17e9c59",
+ "name": "family name",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "lastName",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "family_name",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "912ef641-ee5d-4f38-aec7-73097853bb0b",
+ "name": "picture",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "picture",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "picture",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "666da21e-a0ca-4119-b7cd-b58391df6a48",
+ "name": "profile",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "profile",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "profile",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "b33df9fa-36e4-431a-ba4d-dc8a7123d9bf",
+ "name": "zoneinfo",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "zoneinfo",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "zoneinfo",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "6a15068b-e9cf-4e20-b8ab-ec4093cc0544",
+ "name": "gender",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "gender",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "gender",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "5ff8ce52-10c7-4e9a-887b-4fb932367823",
+ "name": "full name",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-full-name-mapper",
+ "consentRequired": false,
+ "config": {
+ "id.token.claim": "true",
+ "introspection.token.claim": "true",
+ "access.token.claim": "true",
+ "userinfo.token.claim": "true"
+ }
+ },
+ {
+ "id": "f6790da8-1f19-472e-b7a9-a268a72ae51b",
+ "name": "birthdate",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "birthdate",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "birthdate",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "5def6f68-8407-49de-9698-039c1efb0f0a",
+ "name": "updated at",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "updatedAt",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "updated_at",
+ "jsonType.label": "long"
+ }
+ },
+ {
+ "id": "aba6e7c0-0941-4507-acd0-436fc5d17825",
+ "name": "given name",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "firstName",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "given_name",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "9bed8b2f-7512-48e0-999e-217be750a213",
+ "name": "username",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "username",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "preferred_username",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "5943aad1-15c0-4d11-8876-0ac1d95bde0b",
+ "name": "website",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "website",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "website",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "ef3e87bb-78d6-4831-b40a-b63dcff35441",
+ "name": "locale",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "locale",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "locale",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "48e70580-539a-4953-97b9-33c79732b2cb",
+ "name": "nickname",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "nickname",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "nickname",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "99a4b12d-5559-49d2-942c-15b7cc0b4061",
+ "name": "middle name",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "middleName",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "middle_name",
+ "jsonType.label": "String"
+ }
+ }
+ ]
+ },
+ {
+ "id": "e4eb02f7-a5de-4911-824d-4abd83a57565",
+ "name": "offline_access",
+ "description": "OpenID Connect built-in scope: offline_access",
+ "protocol": "openid-connect",
+ "attributes": {
+ "consent.screen.text": "${offlineAccessScopeConsentText}",
+ "display.on.consent.screen": "true"
+ }
+ },
+ {
+ "id": "d814164b-45cf-4b87-b16c-0218c2110cbd",
+ "name": "roles",
+ "description": "OpenID Connect scope for add user roles to the access token",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "false",
+ "consent.screen.text": "${rolesScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "da37e978-f649-410f-a235-be12e85ad7f8",
+ "name": "realm roles",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-realm-role-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.attribute": "foo",
+ "introspection.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "realm_access.roles",
+ "jsonType.label": "String",
+ "multivalued": "true"
+ }
+ },
+ {
+ "id": "f964127b-bb6b-4e5b-918a-4b3b451fc0a1",
+ "name": "audience resolve",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-audience-resolve-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "access.token.claim": "true"
+ }
+ },
+ {
+ "id": "ec554cfc-1093-45e2-9450-97540f25bb7e",
+ "name": "client roles",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-client-role-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.attribute": "foo",
+ "introspection.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "resource_access.${client_id}.roles",
+ "jsonType.label": "String",
+ "multivalued": "true"
+ }
+ }
+ ]
+ },
+ {
+ "id": "970e4ff7-4186-442d-a4a1-a3dc960e04a4",
+ "name": "catalog",
+ "description": "",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "display.on.consent.screen": "true",
+ "gui.order": "",
+ "consent.screen.text": "",
+ "include.in.openid.provider.metadata": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "0ba584cd-cb4f-4bcb-a9dc-68af81d99f9f",
+ "name": "catalog-audience-mapper",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-audience-mapper",
+ "consentRequired": false,
+ "config": {
+ "included.client.audience": "hive-metastore",
+ "id.token.claim": "false",
+ "lightweight.claim": "false",
+ "introspection.token.claim": "true",
+ "access.token.claim": "true",
+ "userinfo.token.claim": "false"
+ }
+ }
+ ]
+ },
+ {
+ "id": "527de19e-82b3-40eb-b220-d9f84e60457b",
+ "name": "web-origins",
+ "description": "OpenID Connect scope for add allowed web origins to the access token",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "false",
+ "consent.screen.text": "",
+ "display.on.consent.screen": "false"
+ },
+ "protocolMappers": [
+ {
+ "id": "1e6aaeaa-6a50-4c3e-8111-298e9aa431a5",
+ "name": "allowed web origins",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-allowed-origins-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "access.token.claim": "true"
+ }
+ }
+ ]
+ },
+ {
+ "id": "50bbcbf2-ab4d-4bda-a03b-6cb5c2ae8ae3",
+ "name": "email",
+ "description": "OpenID Connect built-in scope: email",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "consent.screen.text": "${emailScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "25735f3d-bf2d-493e-acfe-22312ee837e5",
+ "name": "email verified",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-property-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "emailVerified",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "email_verified",
+ "jsonType.label": "boolean"
+ }
+ },
+ {
+ "id": "5223dde4-5f7c-42bd-a05c-41e1a3f99ada",
+ "name": "email",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "email",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "email",
+ "jsonType.label": "String"
+ }
+ }
+ ]
+ },
+ {
+ "id": "377ac681-3cf2-4645-bde0-a0323893ec89",
+ "name": "service_account",
+ "description": "Specific scope for a client enabled for service accounts",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "false",
+ "display.on.consent.screen": "false"
+ },
+ "protocolMappers": [
+ {
+ "id": "53f1f22e-a201-44de-8633-522394459fd4",
+ "name": "Client Host",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usersessionmodel-note-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.session.note": "clientHost",
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "clientHost",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "e088d0c2-dafd-4745-9cf4-53f16cf1623f",
+ "name": "Client IP Address",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usersessionmodel-note-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.session.note": "clientAddress",
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "clientAddress",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "8d8009e6-9dec-4cff-a853-c7d9ca59f7f8",
+ "name": "Client ID",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usersessionmodel-note-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.session.note": "client_id",
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "client_id",
+ "jsonType.label": "String"
+ }
+ }
+ ]
+ },
+ {
+ "id": "a77f389c-6186-41b0-be1a-8c84a7dd3752",
+ "name": "microprofile-jwt",
+ "description": "Microprofile - JWT built-in scope",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "display.on.consent.screen": "false"
+ },
+ "protocolMappers": [
+ {
+ "id": "4cd44ea2-2d91-4e3b-b98b-033007ca2fad",
+ "name": "upn",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "username",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "upn",
+ "jsonType.label": "String"
+ }
+ },
+ {
+ "id": "fe32d0c8-380f-49ec-b435-056a89496fd0",
+ "name": "groups",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-realm-role-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "multivalued": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "foo",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "groups",
+ "jsonType.label": "String"
+ }
+ }
+ ]
+ },
+ {
+ "id": "765cd3e6-af2a-4d4e-af54-acd8bbcb9741",
+ "name": "role_list",
+ "description": "SAML role list",
+ "protocol": "saml",
+ "attributes": {
+ "consent.screen.text": "${samlRoleListScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "295e9fde-2dc3-4257-a486-9308d7464645",
+ "name": "role list",
+ "protocol": "saml",
+ "protocolMapper": "saml-role-list-mapper",
+ "consentRequired": false,
+ "config": {
+ "single": "false",
+ "attribute.nameformat": "Basic",
+ "attribute.name": "Role"
+ }
+ }
+ ]
+ },
+ {
+ "id": "66d58fdc-e806-4b4b-a9cf-1fc89215ae08",
+ "name": "phone",
+ "description": "OpenID Connect built-in scope: phone",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "consent.screen.text": "${phoneScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "2af429e5-12b6-4e72-8ae0-926b4115d011",
+ "name": "phone number verified",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "phoneNumberVerified",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "phone_number_verified",
+ "jsonType.label": "boolean"
+ }
+ },
+ {
+ "id": "23771560-63b1-4847-ab8a-5e38b5a29fc7",
+ "name": "phone number",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-usermodel-attribute-mapper",
+ "consentRequired": false,
+ "config": {
+ "introspection.token.claim": "true",
+ "userinfo.token.claim": "true",
+ "user.attribute": "phoneNumber",
+ "id.token.claim": "true",
+ "access.token.claim": "true",
+ "claim.name": "phone_number",
+ "jsonType.label": "String"
+ }
+ }
+ ]
+ },
+ {
+ "id": "382a7aa6-4c1f-4a57-aa82-eed93f6b5d95",
+ "name": "acr",
+ "description": "OpenID Connect scope for add acr (authentication context class reference) to the token",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "false",
+ "display.on.consent.screen": "false"
+ },
+ "protocolMappers": [
+ {
+ "id": "dfd7e098-f6f3-4741-9934-8994b0087430",
+ "name": "acr loa level",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-acr-mapper",
+ "consentRequired": false,
+ "config": {
+ "id.token.claim": "true",
+ "introspection.token.claim": "true",
+ "access.token.claim": "true",
+ "userinfo.token.claim": "true"
+ }
+ }
+ ]
+ },
+ {
+ "id": "14345900-dd08-44d8-938d-9d36bf832ad9",
+ "name": "address",
+ "description": "OpenID Connect built-in scope: address",
+ "protocol": "openid-connect",
+ "attributes": {
+ "include.in.token.scope": "true",
+ "consent.screen.text": "${addressScopeConsentText}",
+ "display.on.consent.screen": "true"
+ },
+ "protocolMappers": [
+ {
+ "id": "3080d889-61e5-4e6d-b229-e78a41a0a578",
+ "name": "address",
+ "protocol": "openid-connect",
+ "protocolMapper": "oidc-address-mapper",
+ "consentRequired": false,
+ "config": {
+ "user.attribute.formatted": "formatted",
+ "user.attribute.country": "country",
+ "introspection.token.claim": "true",
+ "user.attribute.postal_code": "postal_code",
+ "userinfo.token.claim": "true",
+ "user.attribute.street": "street",
+ "id.token.claim": "true",
+ "user.attribute.region": "region",
+ "access.token.claim": "true",
+ "user.attribute.locality": "locality"
+ }
+ }
+ ]
+ }
+ ],
+ "defaultDefaultClientScopes": [
+ "role_list",
+ "saml_organization",
+ "profile",
+ "email",
+ "roles",
+ "web-origins",
+ "acr",
+ "basic"
+ ],
+ "defaultOptionalClientScopes": [
+ "offline_access",
+ "address",
+ "phone",
+ "microprofile-jwt",
+ "organization"
+ ],
+ "browserSecurityHeaders": {
+ "contentSecurityPolicyReportOnly": "",
+ "xContentTypeOptions": "nosniff",
+ "referrerPolicy": "no-referrer",
+ "xRobotsTag": "none",
+ "xFrameOptions": "SAMEORIGIN",
+ "contentSecurityPolicy": "frame-src 'self'; frame-ancestors 'self'; object-src 'none';",
+ "strictTransportSecurity": "max-age=31536000; includeSubDomains"
+ },
+ "smtpServer": {},
+ "eventsEnabled": false,
+ "eventsListeners": [
+ "jboss-logging"
+ ],
+ "enabledEventTypes": [],
+ "adminEventsEnabled": false,
+ "adminEventsDetailsEnabled": false,
+ "identityProviders": [],
+ "identityProviderMappers": [],
+ "components": {
+ "org.keycloak.services.clientregistration.policy.ClientRegistrationPolicy": [
+ {
+ "id": "f06ce7ba-ca51-4445-86ee-8d5d52d73ec3",
+ "name": "Allowed Registration Web Origins",
+ "providerId": "registration-web-origins",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {}
+ },
+ {
+ "id": "21ab758d-bbca-4287-9015-adc52dc1d289",
+ "name": "Max Clients Limit",
+ "providerId": "max-clients",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {
+ "max-clients": [
+ "200"
+ ]
+ }
+ },
+ {
+ "id": "c25f8de7-f7f8-460e-89da-d7e6dd4d7284",
+ "name": "Allowed Protocol Mapper Types",
+ "providerId": "allowed-protocol-mappers",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {
+ "allowed-protocol-mapper-types": [
+ "oidc-usermodel-attribute-mapper",
+ "oidc-full-name-mapper",
+ "oidc-sha256-pairwise-sub-mapper",
+ "saml-user-property-mapper",
+ "oidc-address-mapper",
+ "oidc-usermodel-property-mapper",
+ "saml-user-attribute-mapper",
+ "saml-role-list-mapper"
+ ]
+ }
+ },
+ {
+ "id": "73b62e00-2fd7-4cc9-a102-8a091413ea2a",
+ "name": "Allowed Protocol Mapper Types",
+ "providerId": "allowed-protocol-mappers",
+ "subType": "authenticated",
+ "subComponents": {},
+ "config": {
+ "allowed-protocol-mapper-types": [
+ "oidc-address-mapper",
+ "oidc-full-name-mapper",
+ "saml-user-attribute-mapper",
+ "saml-role-list-mapper",
+ "oidc-usermodel-attribute-mapper",
+ "oidc-usermodel-property-mapper",
+ "saml-user-property-mapper",
+ "oidc-sha256-pairwise-sub-mapper"
+ ]
+ }
+ },
+ {
+ "id": "3a798227-3318-442a-b1b2-09316d873f2d",
+ "name": "Consent Required",
+ "providerId": "consent-required",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {}
+ },
+ {
+ "id": "e5aa2318-6c6d-41fd-bdf6-be423ab1bdb9",
+ "name": "Trusted Hosts",
+ "providerId": "trusted-hosts",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {
+ "host-sending-registration-request-must-match": [
+ "true"
+ ],
+ "client-uris-must-match": [
+ "true"
+ ]
+ }
+ },
+ {
+ "id": "24406ed9-f533-4e91-982b-730656a829a5",
+ "name": "Full Scope Disabled",
+ "providerId": "scope",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {}
+ },
+ {
+ "id": "524eda51-5780-4b28-940c-4be3a84206b1",
+ "name": "Allowed Registration Web Origins",
+ "providerId": "registration-web-origins",
+ "subType": "authenticated",
+ "subComponents": {},
+ "config": {}
+ },
+ {
+ "id": "87fa2a1c-ff7b-4c26-aafb-82e6d59c6554",
+ "name": "Allowed Client Scopes",
+ "providerId": "allowed-client-templates",
+ "subType": "anonymous",
+ "subComponents": {},
+ "config": {
+ "allow-default-scopes": [
+ "true"
+ ]
+ }
+ },
+ {
+ "id": "70299fc9-6268-4885-af6d-417c0f3a99d0",
+ "name": "Allowed Client Scopes",
+ "providerId": "allowed-client-templates",
+ "subType": "authenticated",
+ "subComponents": {},
+ "config": {
+ "allow-default-scopes": [
+ "true"
+ ]
+ }
+ }
+ ],
+ "org.keycloak.keys.KeyProvider": [
+ {
+ "id": "ac49141c-4422-4f73-9595-018a43c93052",
+ "name": "rsa-generated",
+ "providerId": "rsa-generated",
+ "subComponents": {},
+ "config": {
+ "priority": [
+ "100"
+ ]
+ }
+ },
+ {
+ "id": "00a7e8f1-c32c-453b-b14a-ee0d2e966efb",
+ "name": "aes-generated",
+ "providerId": "aes-generated",
+ "subComponents": {},
+ "config": {
+ "priority": [
+ "100"
+ ]
+ }
+ },
+ {
+ "id": "1298bc87-1071-4a45-bfa3-375d88065154",
+ "name": "rsa-enc-generated",
+ "providerId": "rsa-enc-generated",
+ "subComponents": {},
+ "config": {
+ "priority": [
+ "100"
+ ],
+ "algorithm": [
+ "RSA-OAEP"
+ ]
+ }
+ },
+ {
+ "id": "f560fe09-0036-4316-ae9b-6013d653444c",
+ "name": "hmac-generated-hs512",
+ "providerId": "hmac-generated",
+ "subComponents": {},
+ "config": {
+ "priority": [
+ "100"
+ ],
+ "algorithm": [
+ "HS512"
+ ]
+ }
+ }
+ ]
+ },
+ "internationalizationEnabled": false,
+ "authenticationFlows": [
+ {
+ "id": "b9779335-7239-4340-a9e2-0e3015db623b",
+ "alias": "Account verification options",
+ "description": "Method with which to verify the existing account",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "idp-email-verification",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "ALTERNATIVE",
+ "priority": 20,
+ "autheticatorFlow": true,
+ "flowAlias": "Verify Existing Account by Re-authentication",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "fc713b73-cb72-44f6-84d9-c4100113f559",
+ "alias": "Browser - Conditional 2FA",
+ "description": "Flow to determine if any 2FA is required for the authentication",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "conditional-user-configured",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorConfig": "browser-conditional-credential",
+ "authenticator": "conditional-credential",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "auth-otp-form",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 30,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "webauthn-authenticator",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 40,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "auth-recovery-authn-code-form",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 50,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "30ebc090-ba8c-4d53-9a0f-5a46d65fedf7",
+ "alias": "Browser - Conditional Organization",
+ "description": "Flow to determine if the organization identity-first login is to be used",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "conditional-user-configured",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "organization",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "12beb5f6-4fb4-4dad-a711-c5f4102f4238",
+ "alias": "Direct Grant - Conditional OTP",
+ "description": "Flow to determine if the OTP is required for the authentication",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "conditional-user-configured",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "direct-grant-validate-otp",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "cb01da9e-4d7f-4ab8-9353-db1388fe4170",
+ "alias": "First Broker Login - Conditional Organization",
+ "description": "Flow to determine if the authenticator that adds organization members is to be used",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "conditional-user-configured",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "idp-add-organization-member",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "027be322-3a64-4d28-8fd0-a30d41c928b4",
+ "alias": "First broker login - Conditional 2FA",
+ "description": "Flow to determine if any 2FA is required for the authentication",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "conditional-user-configured",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorConfig": "first-broker-login-conditional-credential",
+ "authenticator": "conditional-credential",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "auth-otp-form",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 30,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "webauthn-authenticator",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 40,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "auth-recovery-authn-code-form",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 50,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "4f08cfc4-8b14-4f4f-9d51-be3e098fa4c0",
+ "alias": "Handle Existing Account",
+ "description": "Handle what to do if there is existing account with same email/username like authenticated identity provider",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "idp-confirm-link",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": true,
+ "flowAlias": "Account verification options",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "52dd5049-1726-4fae-b111-37ab32e0bfad",
+ "alias": "Organization",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticatorFlow": true,
+ "requirement": "CONDITIONAL",
+ "priority": 10,
+ "autheticatorFlow": true,
+ "flowAlias": "Browser - Conditional Organization",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "a35a8d06-5c60-4506-9308-f6fa80d4f3a1",
+ "alias": "Reset - Conditional OTP",
+ "description": "Flow to determine if the OTP should be reset or not. Set to REQUIRED to force.",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "conditional-user-configured",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "reset-otp",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "14b4593b-bb2a-4f5a-ab17-1ed05cc3d029",
+ "alias": "User creation or linking",
+ "description": "Flow for the existing/non-existing user alternatives",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticatorConfig": "create unique user config",
+ "authenticator": "idp-create-user-if-unique",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "ALTERNATIVE",
+ "priority": 20,
+ "autheticatorFlow": true,
+ "flowAlias": "Handle Existing Account",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "674f861a-882a-49f4-a285-2c27a2dcdd4f",
+ "alias": "Verify Existing Account by Re-authentication",
+ "description": "Reauthentication of existing account",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "idp-username-password-form",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "CONDITIONAL",
+ "priority": 20,
+ "autheticatorFlow": true,
+ "flowAlias": "First broker login - Conditional 2FA",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "c957d817-3e0e-4ec7-8f11-daba64dd95f0",
+ "alias": "browser",
+ "description": "Browser based authentication",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "auth-cookie",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "auth-spnego",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "identity-provider-redirector",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 25,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "ALTERNATIVE",
+ "priority": 26,
+ "autheticatorFlow": true,
+ "flowAlias": "Organization",
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "ALTERNATIVE",
+ "priority": 30,
+ "autheticatorFlow": true,
+ "flowAlias": "forms",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "41767010-b4ba-44d6-8bc0-825ac2dd41a7",
+ "alias": "clients",
+ "description": "Base authentication for clients",
+ "providerId": "client-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "client-secret",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "client-jwt",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "client-secret-jwt",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 30,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "client-x509",
+ "authenticatorFlow": false,
+ "requirement": "ALTERNATIVE",
+ "priority": 40,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "9f3a7396-89d3-42cd-b0eb-3e604d3562c9",
+ "alias": "direct grant",
+ "description": "OpenID Connect Resource Owner Grant",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "direct-grant-validate-username",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "direct-grant-validate-password",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "CONDITIONAL",
+ "priority": 30,
+ "autheticatorFlow": true,
+ "flowAlias": "Direct Grant - Conditional OTP",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "0d3d2f3e-10fb-46bc-bf6a-ba7ec0d99f11",
+ "alias": "docker auth",
+ "description": "Used by Docker clients to authenticate against the IDP",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "docker-http-basic-authenticator",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "85b9ecad-5c89-4cee-a158-c50dd4a7ecbe",
+ "alias": "first broker login",
+ "description": "Actions taken after first broker login with identity provider account, which is not yet linked to any Keycloak account",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticatorConfig": "review profile config",
+ "authenticator": "idp-review-profile",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": true,
+ "flowAlias": "User creation or linking",
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "CONDITIONAL",
+ "priority": 60,
+ "autheticatorFlow": true,
+ "flowAlias": "First Broker Login - Conditional Organization",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "074b1b7e-2b89-4513-b108-4e51ee5989f4",
+ "alias": "forms",
+ "description": "Username, password, otp and other auth forms.",
+ "providerId": "basic-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "auth-username-password-form",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "CONDITIONAL",
+ "priority": 20,
+ "autheticatorFlow": true,
+ "flowAlias": "Browser - Conditional 2FA",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "584bc735-bc68-4e38-af17-0ffaca9f8784",
+ "alias": "registration",
+ "description": "Registration flow",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "registration-page-form",
+ "authenticatorFlow": true,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": true,
+ "flowAlias": "registration form",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "fd6ae7c8-3994-4088-82f1-dc1d473af2e6",
+ "alias": "registration form",
+ "description": "Registration form",
+ "providerId": "form-flow",
+ "topLevel": false,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "registration-user-creation",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "registration-password-action",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 50,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "registration-recaptcha-action",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 60,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "registration-terms-and-conditions",
+ "authenticatorFlow": false,
+ "requirement": "DISABLED",
+ "priority": 70,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "67a357a9-b20b-4b7f-84d8-cf2e37c0a488",
+ "alias": "reset credentials",
+ "description": "Reset credentials for a user if they forgot their password or something",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "reset-credentials-choose-user",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "reset-credential-email",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 20,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticator": "reset-password",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 30,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ },
+ {
+ "authenticatorFlow": true,
+ "requirement": "CONDITIONAL",
+ "priority": 40,
+ "autheticatorFlow": true,
+ "flowAlias": "Reset - Conditional OTP",
+ "userSetupAllowed": false
+ }
+ ]
+ },
+ {
+ "id": "567a89b0-7a14-4f57-b1b1-af5059fbeb29",
+ "alias": "saml ecp",
+ "description": "SAML ECP Profile Authentication Flow",
+ "providerId": "basic-flow",
+ "topLevel": true,
+ "builtIn": true,
+ "authenticationExecutions": [
+ {
+ "authenticator": "http-basic-authenticator",
+ "authenticatorFlow": false,
+ "requirement": "REQUIRED",
+ "priority": 10,
+ "autheticatorFlow": false,
+ "userSetupAllowed": false
+ }
+ ]
+ }
+ ],
+ "authenticatorConfig": [
+ {
+ "id": "9c7fd3ea-006c-44eb-831a-c0085c612fbc",
+ "alias": "browser-conditional-credential",
+ "config": {
+ "credentials": "webauthn-passwordless"
+ }
+ },
+ {
+ "id": "14ad4f06-110b-4629-acb9-9ee599317fef",
+ "alias": "create unique user config",
+ "config": {
+ "require.password.update.after.registration": "false"
+ }
+ },
+ {
+ "id": "39b5172b-38a7-42cc-8201-c8647d56dfb8",
+ "alias": "first-broker-login-conditional-credential",
+ "config": {
+ "credentials": "webauthn-passwordless"
+ }
+ },
+ {
+ "id": "04775fd7-462d-472f-ba74-989cae0a5294",
+ "alias": "review profile config",
+ "config": {
+ "update.profile.on.first.login": "missing"
+ }
+ }
+ ],
+ "requiredActions": [
+ {
+ "alias": "CONFIGURE_TOTP",
+ "name": "Configure OTP",
+ "providerId": "CONFIGURE_TOTP",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 10,
+ "config": {}
+ },
+ {
+ "alias": "TERMS_AND_CONDITIONS",
+ "name": "Terms and Conditions",
+ "providerId": "TERMS_AND_CONDITIONS",
+ "enabled": false,
+ "defaultAction": false,
+ "priority": 20,
+ "config": {}
+ },
+ {
+ "alias": "UPDATE_PASSWORD",
+ "name": "Update Password",
+ "providerId": "UPDATE_PASSWORD",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 30,
+ "config": {}
+ },
+ {
+ "alias": "UPDATE_PROFILE",
+ "name": "Update Profile",
+ "providerId": "UPDATE_PROFILE",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 40,
+ "config": {}
+ },
+ {
+ "alias": "VERIFY_EMAIL",
+ "name": "Verify Email",
+ "providerId": "VERIFY_EMAIL",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 50,
+ "config": {}
+ },
+ {
+ "alias": "delete_account",
+ "name": "Delete Account",
+ "providerId": "delete_account",
+ "enabled": false,
+ "defaultAction": false,
+ "priority": 60,
+ "config": {}
+ },
+ {
+ "alias": "UPDATE_EMAIL",
+ "name": "Update Email",
+ "providerId": "UPDATE_EMAIL",
+ "enabled": false,
+ "defaultAction": false,
+ "priority": 70,
+ "config": {}
+ },
+ {
+ "alias": "webauthn-register",
+ "name": "Webauthn Register",
+ "providerId": "webauthn-register",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 80,
+ "config": {}
+ },
+ {
+ "alias": "webauthn-register-passwordless",
+ "name": "Webauthn Register Passwordless",
+ "providerId": "webauthn-register-passwordless",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 90,
+ "config": {}
+ },
+ {
+ "alias": "VERIFY_PROFILE",
+ "name": "Verify Profile",
+ "providerId": "VERIFY_PROFILE",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 100,
+ "config": {}
+ },
+ {
+ "alias": "delete_credential",
+ "name": "Delete Credential",
+ "providerId": "delete_credential",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 110,
+ "config": {}
+ },
+ {
+ "alias": "idp_link",
+ "name": "Linking Identity Provider",
+ "providerId": "idp_link",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 120,
+ "config": {}
+ },
+ {
+ "alias": "CONFIGURE_RECOVERY_AUTHN_CODES",
+ "name": "Recovery Authentication Codes",
+ "providerId": "CONFIGURE_RECOVERY_AUTHN_CODES",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 130,
+ "config": {}
+ },
+ {
+ "alias": "update_user_locale",
+ "name": "Update User Locale",
+ "providerId": "update_user_locale",
+ "enabled": true,
+ "defaultAction": false,
+ "priority": 1000,
+ "config": {}
+ }
+ ],
+ "browserFlow": "browser",
+ "registrationFlow": "registration",
+ "directGrantFlow": "direct grant",
+ "resetCredentialsFlow": "reset credentials",
+ "clientAuthenticationFlow": "clients",
+ "dockerAuthenticationFlow": "docker auth",
+ "firstBrokerLoginFlow": "first broker login",
+ "attributes": {
+ "cibaBackchannelTokenDeliveryMode": "poll",
+ "cibaExpiresIn": "120",
+ "cibaAuthRequestedUserHint": "login_hint",
+ "oauth2DeviceCodeLifespan": "600",
+ "clientOfflineSessionMaxLifespan": "0",
+ "oauth2DevicePollingInterval": "5",
+ "clientSessionIdleTimeout": "0",
+ "parRequestUriLifespan": "60",
+ "clientSessionMaxLifespan": "0",
+ "clientOfflineSessionIdleTimeout": "0",
+ "cibaInterval": "5",
+ "realmReusableOtpCode": "false"
+ },
+ "keycloakVersion": "26.5.0",
+ "userManagedAccessAllowed": false,
+ "organizationsEnabled": false,
+ "verifiableCredentialsEnabled": false,
+ "adminPermissionsEnabled": false,
+ "clientProfiles": {
+ "profiles": []
+ },
+ "clientPolicies": {
+ "policies": []
+ }
+ }
diff --git a/tests/templates/kuttl/iceberg-rest/22_keycloak.yaml.j2 b/tests/templates/kuttl/iceberg-rest/22_keycloak.yaml.j2
new file mode 100644
index 00000000..62898aaf
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/22_keycloak.yaml.j2
@@ -0,0 +1,118 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: keycloak
+---
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+ name: keycloak
+{% if test_scenario['values']['openshift'] == 'true' %}
+rules:
+- apiGroups: ["security.openshift.io"]
+ resources: ["securitycontextconstraints"]
+ resourceNames: ["privileged"]
+ verbs: ["use"]
+{% endif %}
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+ name: keycloak
+subjects:
+ - kind: ServiceAccount
+ name: keycloak
+roleRef:
+ kind: Role
+ name: keycloak
+ apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: keycloak
+ labels:
+ app: keycloak
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: keycloak
+ template:
+ metadata:
+ labels:
+ app: keycloak
+ spec:
+ serviceAccountName: keycloak
+ containers:
+ - name: keycloak
+ image: quay.io/keycloak/keycloak:26.5.0
+ args:
+ - start-dev
+ - --import-realm
+ - --https-certificate-file=/tls/tls.crt
+ - --https-certificate-key-file=/tls/tls.key
+ env:
+ - name: KEYCLOAK_ADMIN
+ value: admin
+ - name: KEYCLOAK_ADMIN_PASSWORD
+ value: admin
+ - name: JAVA_OPTS_KC_HEAP
+ value: -Xmx500M
+ ports:
+ - name: https
+ containerPort: 8443
+ readinessProbe:
+ httpGet:
+ scheme: HTTPS
+ path: /realms/master
+ port: https
+ resources:
+ limits:
+ cpu: 1
+ memory: 1024Mi
+ requests:
+ cpu: 500m
+ memory: 1024Mi
+ volumeMounts:
+ - name: data
+ mountPath: /opt/keycloak/data/
+ - name: tls
+ mountPath: /tls/
+ - name: realm-volume
+ mountPath: /opt/keycloak/data/import
+ volumes:
+ - name: data
+ emptyDir: {}
+ - name: tls
+ ephemeral:
+ volumeClaimTemplate:
+ metadata:
+ annotations:
+ secrets.stackable.tech/class: tls
+ secrets.stackable.tech/scope: service=keycloak
+ spec:
+ storageClassName: secrets.stackable.tech
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: "1"
+ - name: realm-volume
+ configMap:
+ name: keycloak-test-realm
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: keycloak
+ labels:
+ app: keycloak
+spec:
+ ports:
+ - name: https
+ port: 8443
+ targetPort: 8443
+ selector:
+ app: keycloak
diff --git a/tests/templates/kuttl/iceberg-rest/23-assert.yaml b/tests/templates/kuttl/iceberg-rest/23-assert.yaml
new file mode 100644
index 00000000..1ac12423
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/23-assert.yaml
@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 600
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: postgresql
+status:
+ readyReplicas: 1
+ replicas: 1
diff --git a/tests/templates/kuttl/iceberg-rest/23-install-hive-postgres.yaml b/tests/templates/kuttl/iceberg-rest/23-install-hive-postgres.yaml
new file mode 100644
index 00000000..2b1930ed
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/23-install-hive-postgres.yaml
@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+timeout: 300
+commands:
+ - script: >-
+ helm upgrade postgresql
+ --install
+ --version=12.5.6
+ --namespace $NAMESPACE
+ -f 23_helm-bitnami-postgresql-values.yaml
+ --repo https://charts.bitnami.com/bitnami postgresql
diff --git a/tests/templates/kuttl/iceberg-rest/23_helm-bitnami-postgresql-values.yaml.j2 b/tests/templates/kuttl/iceberg-rest/23_helm-bitnami-postgresql-values.yaml.j2
new file mode 100644
index 00000000..40dfcd33
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/23_helm-bitnami-postgresql-values.yaml.j2
@@ -0,0 +1,42 @@
+---
+global:
+ security:
+ allowInsecureImages: true # needed starting with Chart version 16.3.0 if modifying images
+
+image:
+ repository: bitnamilegacy/postgresql
+
+volumePermissions:
+ enabled: false
+ image:
+ repository: bitnamilegacy/os-shell
+ securityContext:
+ runAsUser: auto
+
+metrics:
+ image:
+ repository: bitnamilegacy/postgres-exporter
+
+primary:
+ extendedConfiguration: |
+ password_encryption=md5
+ podSecurityContext:
+{% if test_scenario['values']['openshift'] == 'true' %}
+ enabled: false
+{% else %}
+ enabled: true
+{% endif %}
+ containerSecurityContext:
+ enabled: false
+ resources:
+ requests:
+ memory: "512Mi"
+ cpu: "512m"
+ limits:
+ memory: "512Mi"
+ cpu: "1"
+
+auth:
+ username: hive
+ password: hive
+ database: hive
diff --git a/tests/templates/kuttl/iceberg-rest/30-assert.yaml b/tests/templates/kuttl/iceberg-rest/30-assert.yaml
new file mode 100644
index 00000000..50c27fd9
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/30-assert.yaml
@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 900
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: hive-metastore-default
+status:
+ readyReplicas: 1
+ replicas: 1
diff --git a/tests/templates/kuttl/iceberg-rest/30-install-hive.yaml.j2 b/tests/templates/kuttl/iceberg-rest/30-install-hive.yaml.j2
new file mode 100644
index 00000000..05ceefcc
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/30-install-hive.yaml.j2
@@ -0,0 +1,6 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ # We need to replace $NAMESPACE (by KUTTL)
+ - script: envsubst '$NAMESPACE' < 30_hive.yaml | kubectl apply -n $NAMESPACE -f -
diff --git a/tests/templates/kuttl/iceberg-rest/30_hive.yaml.j2 b/tests/templates/kuttl/iceberg-rest/30_hive.yaml.j2
new file mode 100644
index 00000000..8ec64b29
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/30_hive.yaml.j2
@@ -0,0 +1,55 @@
+---
+apiVersion: hive.stackable.tech/v1alpha1
+kind: HiveCluster
+metadata:
+ name: hive
+spec:
+ image:
+{% if test_scenario['values']['hive-l'].find(",") > 0 %}
+ custom: "{{ test_scenario['values']['hive-l'].split(',')[1] }}"
+ productVersion: "{{ test_scenario['values']['hive-l'].split(',')[0] }}"
+{% else %}
+ productVersion: "{{ test_scenario['values']['hive-l'] }}"
+{% endif %}
+ pullPolicy: IfNotPresent
+ clusterConfig:
+ database:
+ connString: jdbc:postgresql://postgresql:5432/hive
+ credentialsSecret: postgres-credentials
+ dbType: postgres
+ s3:
+ reference: minio
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+ vectorAggregatorConfigMapName: vector-aggregator-discovery
+{% endif %}
+ metastore:
+ config:
+ logging:
+ enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
+ configOverrides:
+ hive-site.xml:
+ # See https://hive.apache.org/docs/latest/admin/oauth2/keycloak/
+ # for Hive metastore and Keycloak setup.
+ metastore.catalog.servlet.port: "9001"
+ metastore.catalog.servlet.auth: oauth2
+ metastore.catalog.servlet.auth.oauth2.issuer: https://keycloak.$NAMESPACE.svc.cluster.local:8443/realms/test
+ metastore.catalog.servlet.auth.oauth2.audience: hive-metastore
+
+ hive.metastore.warehouse.dir: s3a://demo/lakehouse
+ hive.metastore.warehouse.external.dir: s3a://demo/lakehouse
+ roleGroups:
+ default:
+ replicas: 1
+# curl http://localhost:9001/iceberg/v1/config
+# curl -X POST http://localhost:9001/iceberg/v1/namespaces/default/tables -H "Content-Type: application/json" -d '{"name": "test", "location":"s3a://trino/default/test", "schema": {"type": "struct", "fields": [{"id": 1, "name": "id", "type": "long", "required": true}]}, "write-disposition": "create"}'
+# curl -X DELETE http://localhost:9001/iceberg/v1/namespaces/default/tables/test -H "Content-Type: application/json"
+# curl http://localhost:9001/iceberg/v1/namespaces/default/tables/test
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: postgres-credentials
+type: Opaque
+stringData:
+ username: hive
+ password: hive
diff --git a/tests/templates/kuttl/iceberg-rest/31-assert.yaml b/tests/templates/kuttl/iceberg-rest/31-assert.yaml
new file mode 100644
index 00000000..b9f1bbe1
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/31-assert.yaml
@@ -0,0 +1,6 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 720
+commands:
+ - script: kubectl -n $NAMESPACE wait --for=condition=available=true trinoclusters.trino.stackable.tech/trino --timeout 301s
diff --git a/tests/templates/kuttl/iceberg-rest/31-install-trino.yaml.j2 b/tests/templates/kuttl/iceberg-rest/31-install-trino.yaml.j2
new file mode 100644
index 00000000..0568ad24
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/31-install-trino.yaml.j2
@@ -0,0 +1,6 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ # We need to replace $NAMESPACE (by KUTTL)
+ - script: envsubst '$NAMESPACE' < 31_trino.yaml | kubectl apply -n $NAMESPACE -f -
diff --git a/tests/templates/kuttl/iceberg-rest/31_trino.yaml.j2 b/tests/templates/kuttl/iceberg-rest/31_trino.yaml.j2
new file mode 100644
index 00000000..f5198ec4
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/31_trino.yaml.j2
@@ -0,0 +1,56 @@
+---
+apiVersion: trino.stackable.tech/v1alpha1
+kind: TrinoCluster
+metadata:
+ name: trino
+spec:
+ image:
+{% if test_scenario['values']['trino-l'].find(",") > 0 %}
+ custom: "{{ test_scenario['values']['trino-l'].split(',')[1] }}"
+ productVersion: "{{ test_scenario['values']['trino-l'].split(',')[0] }}"
+{% else %}
+ productVersion: "{{ test_scenario['values']['trino-l'] }}"
+{% endif %}
+ pullPolicy: IfNotPresent
+ clusterConfig:
+ catalogLabelSelector:
+ matchLabels:
+ trino: trino
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+ vectorAggregatorConfigMapName: vector-aggregator-discovery
+{% endif %}
+ coordinators:
+ config:
+ logging:
+ enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
+ roleGroups:
+ default:
+ replicas: 1
+ workers:
+ config:
+ gracefulShutdownTimeout: 120s # Let the test run faster, but allow for slow queries
+ logging:
+ enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
+ roleGroups:
+ default:
+ replicas: 1
+---
+apiVersion: trino.stackable.tech/v1alpha1
+kind: TrinoCatalog
+metadata:
+ name: lakehouse
+ labels:
+ trino: trino
+spec:
+ connector:
+ iceberg:
+ s3:
+ reference: minio
+ configOverrides:
+ iceberg.catalog.type: rest
+ # We are using the headless service, as the hive-metastore service is missing port 9001
+ iceberg.rest-catalog.uri: http://hive-metastore-default-headless:9001/iceberg
+ iceberg.rest-catalog.security: OAUTH2
+ iceberg.rest-catalog.oauth2.server-uri: https://keycloak.$NAMESPACE.svc.cluster.local:8443/realms/test/protocol/openid-connect/token
+ # TODO Mount as env var from Secret trino-client-credentials
+ iceberg.rest-catalog.oauth2.credential: trino:trino-secret
diff --git a/tests/templates/kuttl/iceberg-rest/40-assert.yaml b/tests/templates/kuttl/iceberg-rest/40-assert.yaml
new file mode 100644
index 00000000..485373c8
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/40-assert.yaml
@@ -0,0 +1,7 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: create-iceberg-tables
+status:
+ succeeded: 1
diff --git a/tests/templates/kuttl/iceberg-rest/40-create-iceberg-tables.j2 b/tests/templates/kuttl/iceberg-rest/40-create-iceberg-tables.j2
new file mode 100644
index 00000000..4fa76773
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/40-create-iceberg-tables.j2
@@ -0,0 +1,22 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: create-iceberg-tables
+spec:
+ template:
+ spec:
+ containers:
+ - name: create-iceberg-tables
+ image: "oci.stackable.tech/sdp/trino-cli:{{ test_scenario['values']['trino-l'] }}-stackable0.0.0-dev"
+ command:
+ - bash
+ - -euo
+ - pipefail
+ - -c
+ - |
+ cat << 'EOF' | ./trino-cli --server https://trino-coordinator:8443 --insecure --user admin
+ CREATE SCHEMA IF NOT EXISTS lakehouse.test;
+ CREATE TABLE IF NOT EXISTS lakehouse.test.greetings (hello varchar);
+ EOF
+ restartPolicy: OnFailure
diff --git a/tests/templates/kuttl/iceberg-rest/50-assert.yaml b/tests/templates/kuttl/iceberg-rest/50-assert.yaml
new file mode 100644
index 00000000..35aae31d
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/50-assert.yaml
@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 1200
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: nifi-node-default
+status:
+ readyReplicas: 1
+ replicas: 1
diff --git a/tests/templates/kuttl/iceberg-rest/50-install-nifi.yaml.j2 b/tests/templates/kuttl/iceberg-rest/50-install-nifi.yaml.j2
new file mode 100644
index 00000000..006c39e3
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/50-install-nifi.yaml.j2
@@ -0,0 +1,6 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ # We need to replace $NAMESPACE (by KUTTL)
+ - script: envsubst '$NAMESPACE' < 50_nifi.yaml | kubectl apply -n $NAMESPACE -f -
diff --git a/tests/templates/kuttl/iceberg-rest/50_nifi.yaml.j2 b/tests/templates/kuttl/iceberg-rest/50_nifi.yaml.j2
new file mode 100644
index 00000000..eca9c899
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/50_nifi.yaml.j2
@@ -0,0 +1,65 @@
+---
+apiVersion: nifi.stackable.tech/v1alpha1
+kind: NifiCluster
+metadata:
+ name: nifi
+spec:
+ image:
+{% if test_scenario['values']['nifi-iceberg-rest'].find(",") > 0 %}
+ custom: "{{ test_scenario['values']['nifi-iceberg-rest'].split(',')[1] }}"
+ productVersion: "{{ test_scenario['values']['nifi-iceberg-rest'].split(',')[0] }}"
+{% else %}
+ productVersion: "{{ test_scenario['values']['nifi-iceberg-rest'] }}"
+{% endif %}
+ pullPolicy: IfNotPresent
+ clusterConfig:
+ authentication:
+ - authenticationClass: nifi-users
+ sensitiveProperties:
+ keySecret: nifi-sensitive-property-key
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+ vectorAggregatorConfigMapName: vector-aggregator-discovery
+{% endif %}
+ nodes:
+ config:
+ logging:
+ enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
+ roleConfig:
+ listenerClass: external-unstable
+ configOverrides:
+ nifi.properties:
+ # Quicker startup, and we only have a single node
+ nifi.cluster.flow.election.max.wait.time: 5 secs
+ jvmArgumentOverrides:
+ add:
+ # Needed for NiFi to trust the minio cert
+ - -Djavax.net.ssl.trustStore=/stackable/keystore/truststore.p12
+ - -Djavax.net.ssl.trustStorePassword=secret
+ - -Djavax.net.ssl.trustStoreType=PKCS12
+ roleGroups:
+ default:
+ replicas: 1
+---
+apiVersion: authentication.stackable.tech/v1alpha1
+kind: AuthenticationClass
+metadata:
+ name: nifi-users
+spec:
+ provider:
+ static:
+ userCredentialsSecret:
+ name: nifi-users
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: nifi-users
+stringData:
+ admin: adminadmin
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: nifi-sensitive-property-key
+stringData:
+ nifiSensitivePropsKey: mYsUp3rS3cr3tk3y
diff --git a/tests/templates/kuttl/iceberg-rest/60-create-nifi-flow-configmap.yaml.j2 b/tests/templates/kuttl/iceberg-rest/60-create-nifi-flow-configmap.yaml.j2
new file mode 100644
index 00000000..fe92ec83
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/60-create-nifi-flow-configmap.yaml.j2
@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+ - script: cat 60_nifi-flow.json | envsubst '$NAMESPACE' | kubectl -n $NAMESPACE create configmap nifi-flow --from-file=nifi-flow.json=/dev/stdin
diff --git a/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json b/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json
new file mode 100644
index 00000000..383e63fe
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json
@@ -0,0 +1,726 @@
+{
+ "flowContents": {
+ "identifier": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "instanceIdentifier": "ff8d3e24-019b-1000-67a0-8eee9b405b68",
+ "name": "NiFi Flow",
+ "comments": "",
+ "position": {
+ "x": 0.0,
+ "y": 0.0
+ },
+ "processGroups": [],
+ "remoteProcessGroups": [],
+ "processors": [
+ {
+ "identifier": "b1668992-d593-3bbe-b1ae-6987c9e4dc41",
+ "instanceIdentifier": "ff97785d-019b-1000-ffff-ffffdde19d67",
+ "name": "GenerateFlowFile",
+ "comments": "",
+ "position": {
+ "x": -120.0,
+ "y": -336.0
+ },
+ "type": "org.apache.nifi.processors.standard.GenerateFlowFile",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-standard-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ },
+ "properties": {
+ "File Size": "0B",
+ "Batch Size": "1",
+ "Unique FlowFiles": "false",
+ "Mime Type": null,
+ "Custom Text": "{\"hello\": \"world from NiFi :)\"}",
+ "Character Set": "UTF-8",
+ "Data Format": "Text"
+ },
+ "propertyDescriptors": {
+ "File Size": {
+ "name": "File Size",
+ "displayName": "File Size",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Batch Size": {
+ "name": "Batch Size",
+ "displayName": "Batch Size",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Unique FlowFiles": {
+ "name": "Unique FlowFiles",
+ "displayName": "Unique FlowFiles",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Mime Type": {
+ "name": "Mime Type",
+ "displayName": "Mime Type",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Custom Text": {
+ "name": "Custom Text",
+ "displayName": "Custom Text",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Character Set": {
+ "name": "Character Set",
+ "displayName": "Character Set",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Data Format": {
+ "name": "Data Format",
+ "displayName": "Data Format",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ }
+ },
+ "style": {},
+ "schedulingPeriod": "1 min",
+ "schedulingStrategy": "TIMER_DRIVEN",
+ "executionNode": "ALL",
+ "penaltyDuration": "30 sec",
+ "yieldDuration": "1 sec",
+ "bulletinLevel": "WARN",
+ "runDurationMillis": 0,
+ "concurrentlySchedulableTaskCount": 1,
+ "autoTerminatedRelationships": [],
+ "scheduledState": "ENABLED",
+ "retryCount": 10,
+ "retriedRelationships": [],
+ "backoffMechanism": "PENALIZE_FLOWFILE",
+ "maxBackoffPeriod": "10 mins",
+ "componentType": "PROCESSOR",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "ee4389f6-388b-39f9-aba4-f1d85bdf7c29",
+ "instanceIdentifier": "ff94896d-019b-1000-0000-00007a222339",
+ "name": "PutIcebergRecord",
+ "comments": "",
+ "position": {
+ "x": -128.0,
+ "y": -80.0
+ },
+ "type": "org.apache.nifi.processors.iceberg.PutIcebergRecord",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-processors-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ },
+ "properties": {
+ "Table Name": "greetings",
+ "Iceberg Catalog": "059cb4fe-6c2f-3d9b-9b8e-e30d9d3ffb4f",
+ "Record Reader": "27175bc0-818e-3e08-a5db-b961636e2d54",
+ "Iceberg Writer": "50489d60-6dea-3d4a-91c5-c754a2f021be",
+ "Namespace": "test"
+ },
+ "propertyDescriptors": {
+ "Table Name": {
+ "name": "Table Name",
+ "displayName": "Table Name",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Iceberg Catalog": {
+ "name": "Iceberg Catalog",
+ "displayName": "Iceberg Catalog",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Record Reader": {
+ "name": "Record Reader",
+ "displayName": "Record Reader",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Iceberg Writer": {
+ "name": "Iceberg Writer",
+ "displayName": "Iceberg Writer",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Namespace": {
+ "name": "Namespace",
+ "displayName": "Namespace",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ }
+ },
+ "style": {},
+ "schedulingPeriod": "0 sec",
+ "schedulingStrategy": "TIMER_DRIVEN",
+ "executionNode": "ALL",
+ "penaltyDuration": "30 sec",
+ "yieldDuration": "1 sec",
+ "bulletinLevel": "WARN",
+ "runDurationMillis": 0,
+ "concurrentlySchedulableTaskCount": 1,
+ "autoTerminatedRelationships": [],
+ "scheduledState": "ENABLED",
+ "retryCount": 10,
+ "retriedRelationships": [],
+ "backoffMechanism": "PENALIZE_FLOWFILE",
+ "maxBackoffPeriod": "10 mins",
+ "componentType": "PROCESSOR",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ }
+ ],
+ "inputPorts": [],
+ "outputPorts": [],
+ "connections": [
+ {
+ "identifier": "6a382847-605e-38d1-8362-b54d6ba62ffe",
+ "instanceIdentifier": "ff983997-019b-1000-ffff-ffffb43d9d5e",
+ "name": "",
+ "source": {
+ "id": "ee4389f6-388b-39f9-aba4-f1d85bdf7c29",
+ "type": "PROCESSOR",
+ "groupId": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "name": "PutIcebergRecord",
+ "comments": "",
+ "instanceIdentifier": "ff94896d-019b-1000-0000-00007a222339"
+ },
+ "destination": {
+ "id": "27e89d06-61dc-3a79-97e9-fdea255d724d",
+ "type": "FUNNEL",
+ "groupId": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "name": "Funnel",
+ "comments": "",
+ "instanceIdentifier": "ff982cf2-019b-1000-ffff-ffffa45d6088"
+ },
+ "labelIndex": 0,
+ "zIndex": 3,
+ "selectedRelationships": [
+ "failure"
+ ],
+ "backPressureObjectThreshold": 10000,
+ "backPressureDataSizeThreshold": "1 GB",
+ "flowFileExpiration": "0 sec",
+ "prioritizers": [],
+ "bends": [],
+ "loadBalanceStrategy": "DO_NOT_LOAD_BALANCE",
+ "partitioningAttribute": "",
+ "loadBalanceCompression": "DO_NOT_COMPRESS",
+ "componentType": "CONNECTION",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "9ebbb5ad-0c41-3ab2-b443-313c58895b2b",
+ "instanceIdentifier": "ff97882f-019b-1000-0000-000037cea26c",
+ "name": "",
+ "source": {
+ "id": "b1668992-d593-3bbe-b1ae-6987c9e4dc41",
+ "type": "PROCESSOR",
+ "groupId": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "name": "GenerateFlowFile",
+ "comments": "",
+ "instanceIdentifier": "ff97785d-019b-1000-ffff-ffffdde19d67"
+ },
+ "destination": {
+ "id": "ee4389f6-388b-39f9-aba4-f1d85bdf7c29",
+ "type": "PROCESSOR",
+ "groupId": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "name": "PutIcebergRecord",
+ "comments": "",
+ "instanceIdentifier": "ff94896d-019b-1000-0000-00007a222339"
+ },
+ "labelIndex": 0,
+ "zIndex": 1,
+ "selectedRelationships": [
+ "success"
+ ],
+ "backPressureObjectThreshold": 10000,
+ "backPressureDataSizeThreshold": "1 GB",
+ "flowFileExpiration": "0 sec",
+ "prioritizers": [],
+ "bends": [],
+ "loadBalanceStrategy": "DO_NOT_LOAD_BALANCE",
+ "partitioningAttribute": "",
+ "loadBalanceCompression": "DO_NOT_COMPRESS",
+ "componentType": "CONNECTION",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "86ca407c-f0de-31bf-8b9b-bedbcc4cc34a",
+ "instanceIdentifier": "ff9815d0-019b-1000-0000-00007f41d818",
+ "name": "",
+ "source": {
+ "id": "ee4389f6-388b-39f9-aba4-f1d85bdf7c29",
+ "type": "PROCESSOR",
+ "groupId": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "name": "PutIcebergRecord",
+ "comments": "",
+ "instanceIdentifier": "ff94896d-019b-1000-0000-00007a222339"
+ },
+ "destination": {
+ "id": "9f73c89f-a352-358c-8b7e-ed33379accb5",
+ "type": "FUNNEL",
+ "groupId": "2e5eba24-12b8-376f-831e-92b58b146e97",
+ "name": "Funnel",
+ "comments": "",
+ "instanceIdentifier": "ff98099b-019b-1000-0000-000039048911"
+ },
+ "labelIndex": 0,
+ "zIndex": 2,
+ "selectedRelationships": [
+ "success"
+ ],
+ "backPressureObjectThreshold": 10000,
+ "backPressureDataSizeThreshold": "1 GB",
+ "flowFileExpiration": "0 sec",
+ "prioritizers": [],
+ "bends": [],
+ "loadBalanceStrategy": "DO_NOT_LOAD_BALANCE",
+ "partitioningAttribute": "",
+ "loadBalanceCompression": "DO_NOT_COMPRESS",
+ "componentType": "CONNECTION",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ }
+ ],
+ "labels": [],
+ "funnels": [
+ {
+ "identifier": "9f73c89f-a352-358c-8b7e-ed33379accb5",
+ "instanceIdentifier": "ff98099b-019b-1000-0000-000039048911",
+ "position": {
+ "x": -215.0,
+ "y": 269.5
+ },
+ "componentType": "FUNNEL",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "27e89d06-61dc-3a79-97e9-fdea255d724d",
+ "instanceIdentifier": "ff982cf2-019b-1000-ffff-ffffa45d6088",
+ "position": {
+ "x": 179.0,
+ "y": 307.5
+ },
+ "componentType": "FUNNEL",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ }
+ ],
+ "controllerServices": [
+ {
+ "identifier": "9f657262-7b83-3a51-9b99-0f37dcd21430",
+ "instanceIdentifier": "ff94fb03-019b-1000-0000-00000a4be464",
+ "name": "S3IcebergFileIOProvider",
+ "comments": "",
+ "type": "org.apache.nifi.services.iceberg.aws.S3IcebergFileIOProvider",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-aws-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ },
+ "properties": {
+ "Authentication Strategy": "BASIC_CREDENTIALS",
+ "Access Key ID": "admin",
+ "Secret Access Key": "adminadmin",
+ "Endpoint Override URL": "https://minio.${NAMESPACE}.svc.cluster.local:9000",
+ "Use Path Style Access": "true",
+ "Client Region": "us-east1"
+ },
+ "propertyDescriptors": {
+ "Authentication Strategy": {
+ "name": "Authentication Strategy",
+ "displayName": "Authentication Strategy",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Session Token": {
+ "name": "Session Token",
+ "displayName": "Session Token",
+ "identifiesControllerService": false,
+ "sensitive": true,
+ "dynamic": false
+ },
+ "Endpoint": {
+ "name": "Endpoint",
+ "displayName": "Endpoint",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Path style access": {
+ "name": "Path style access",
+ "displayName": "Path style access",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Access Key ID": {
+ "name": "Access Key ID",
+ "displayName": "Access Key ID",
+ "identifiesControllerService": false,
+ "sensitive": true,
+ "dynamic": false
+ },
+ "Client Region": {
+ "name": "Client Region",
+ "displayName": "Client Region",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Secret Access Key": {
+ "name": "Secret Access Key",
+ "displayName": "Secret Access Key",
+ "identifiesControllerService": false,
+ "sensitive": true,
+ "dynamic": false
+ }
+ },
+ "controllerServiceApis": [
+ {
+ "type": "org.apache.nifi.services.iceberg.IcebergFileIOProvider",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-service-api-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ }
+ }
+ ],
+ "scheduledState": "DISABLED",
+ "bulletinLevel": "WARN",
+ "componentType": "CONTROLLER_SERVICE",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "27175bc0-818e-3e08-a5db-b961636e2d54",
+ "instanceIdentifier": "ff94bc63-019b-1000-ffff-ffffe8b8471f",
+ "name": "JsonTreeReader",
+ "comments": "",
+ "type": "org.apache.nifi.json.JsonTreeReader",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-record-serialization-services-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ },
+ "properties": {
+ "Schema Branch": null,
+ "Schema Reference Reader": null,
+ "Schema Text": "${avro.schema}",
+ "Starting Field Name": null,
+ "Max String Length": "20 MB",
+ "Schema Inference Cache": null,
+ "Starting Field Strategy": "ROOT_NODE",
+ "Schema Registry": null,
+ "Schema Access Strategy": "infer-schema",
+ "Schema Name": "${schema.name}",
+ "Timestamp Format": null,
+ "Date Format": null,
+ "Schema Application Strategy": "SELECTED_PART",
+ "Schema Version": null,
+ "Time Format": null,
+ "Allow Comments": "false"
+ },
+ "propertyDescriptors": {
+ "Schema Branch": {
+ "name": "Schema Branch",
+ "displayName": "Schema Branch",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Reference Reader": {
+ "name": "Schema Reference Reader",
+ "displayName": "Schema Reference Reader",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Text": {
+ "name": "Schema Text",
+ "displayName": "Schema Text",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Starting Field Name": {
+ "name": "Starting Field Name",
+ "displayName": "Starting Field Name",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Max String Length": {
+ "name": "Max String Length",
+ "displayName": "Max String Length",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Inference Cache": {
+ "name": "Schema Inference Cache",
+ "displayName": "Schema Inference Cache",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Starting Field Strategy": {
+ "name": "Starting Field Strategy",
+ "displayName": "Starting Field Strategy",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Registry": {
+ "name": "Schema Registry",
+ "displayName": "Schema Registry",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Access Strategy": {
+ "name": "Schema Access Strategy",
+ "displayName": "Schema Access Strategy",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Name": {
+ "name": "Schema Name",
+ "displayName": "Schema Name",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Timestamp Format": {
+ "name": "Timestamp Format",
+ "displayName": "Timestamp Format",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Date Format": {
+ "name": "Date Format",
+ "displayName": "Date Format",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Application Strategy": {
+ "name": "Schema Application Strategy",
+ "displayName": "Schema Application Strategy",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Schema Version": {
+ "name": "Schema Version",
+ "displayName": "Schema Version",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Time Format": {
+ "name": "Time Format",
+ "displayName": "Time Format",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Allow Comments": {
+ "name": "Allow Comments",
+ "displayName": "Allow Comments",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ }
+ },
+ "controllerServiceApis": [
+ {
+ "type": "org.apache.nifi.serialization.RecordReaderFactory",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-standard-services-api-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ }
+ }
+ ],
+ "scheduledState": "DISABLED",
+ "bulletinLevel": "WARN",
+ "componentType": "CONTROLLER_SERVICE",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "50489d60-6dea-3d4a-91c5-c754a2f021be",
+ "instanceIdentifier": "ff94aa25-019b-1000-ffff-ffffcdb3cb75",
+ "name": "ParquetIcebergWriter",
+ "comments": "",
+ "type": "org.apache.nifi.services.iceberg.parquet.ParquetIcebergWriter",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-parquet-writer-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ },
+ "properties": {},
+ "propertyDescriptors": {},
+ "controllerServiceApis": [
+ {
+ "type": "org.apache.nifi.services.iceberg.IcebergWriter",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-service-api-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ }
+ }
+ ],
+ "scheduledState": "DISABLED",
+ "bulletinLevel": "WARN",
+ "componentType": "CONTROLLER_SERVICE",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ },
+ {
+ "identifier": "059cb4fe-6c2f-3d9b-9b8e-e30d9d3ffb4f",
+ "instanceIdentifier": "ff94a010-019b-1000-ffff-ffffd6746d87",
+ "name": "RESTIcebergCatalog",
+ "comments": "",
+ "type": "org.apache.nifi.services.iceberg.catalog.RESTIcebergCatalog",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-rest-catalog-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ },
+ "properties": {
+ "Access Delegation Strategy": "disabled",
+ "File IO Provider": "9f657262-7b83-3a51-9b99-0f37dcd21430",
+ "Authentication Strategy": "OAUTH2",
+ "Access Token Scopes": "catalog",
+ "Catalog URI": "http://hive-metastore-default-headless:9001/iceberg",
+ "Authorization Server URI": "https://keycloak.${NAMESPACE}.svc.cluster.local:8443/realms/test/protocol/openid-connect/token",
+ "Warehouse Location": "s3a://demo/lakehouse",
+ "Authorization Grant Type": "CLIENT_CREDENTIALS",
+ "Client ID": "nifi",
+ "Client Secret": "nifi-secret"
+ },
+ "propertyDescriptors": {
+ "Access Delegation Strategy": {
+ "name": "Access Delegation Strategy",
+ "displayName": "Access Delegation Strategy",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "File IO Provider": {
+ "name": "File IO Provider",
+ "displayName": "File IO Provider",
+ "identifiesControllerService": true,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Authentication Strategy": {
+ "name": "Authentication Strategy",
+ "displayName": "Authentication Strategy",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Client ID": {
+ "name": "Client ID",
+ "displayName": "Client ID",
+ "identifiesControllerService": false,
+ "sensitive": true,
+ "dynamic": false
+ },
+ "Access Token Scopes": {
+ "name": "Access Token Scopes",
+ "displayName": "Access Token Scopes",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Bearer Token": {
+ "name": "Bearer Token",
+ "displayName": "Bearer Token",
+ "identifiesControllerService": false,
+ "sensitive": true,
+ "dynamic": false
+ },
+ "Catalog URI": {
+ "name": "Catalog URI",
+ "displayName": "Catalog URI",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Authorization Server URI": {
+ "name": "Authorization Server URI",
+ "displayName": "Authorization Server URI",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Warehouse Location": {
+ "name": "Warehouse Location",
+ "displayName": "Warehouse Location",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ },
+ "Client Secret": {
+ "name": "Client Secret",
+ "displayName": "Client Secret",
+ "identifiesControllerService": false,
+ "sensitive": true,
+ "dynamic": false
+ },
+ "Authorization Grant Type": {
+ "name": "Authorization Grant Type",
+ "displayName": "Authorization Grant Type",
+ "identifiesControllerService": false,
+ "sensitive": false,
+ "dynamic": false
+ }
+ },
+ "controllerServiceApis": [
+ {
+ "type": "org.apache.nifi.services.iceberg.IcebergCatalog",
+ "bundle": {
+ "group": "org.apache.nifi",
+ "artifact": "nifi-iceberg-service-api-nar",
+ "version": "2.7.2-stackable0.0.0-dev"
+ }
+ }
+ ],
+ "scheduledState": "DISABLED",
+ "bulletinLevel": "WARN",
+ "componentType": "CONTROLLER_SERVICE",
+ "groupIdentifier": "2e5eba24-12b8-376f-831e-92b58b146e97"
+ }
+ ],
+ "defaultFlowFileExpiration": "0 sec",
+ "defaultBackPressureObjectThreshold": 10000,
+ "defaultBackPressureDataSizeThreshold": "1 GB",
+ "scheduledState": "ENABLED",
+ "executionEngine": "INHERITED",
+ "maxConcurrentTasks": 1,
+ "statelessFlowTimeout": "1 min",
+ "flowFileConcurrency": "UNBOUNDED",
+ "flowFileOutboundPolicy": "STREAM_WHEN_AVAILABLE",
+ "componentType": "PROCESS_GROUP"
+ },
+ "externalControllerServices": {},
+ "parameterContexts": {},
+ "flowEncodingVersion": "1.0",
+ "parameterProviders": {},
+ "latest": false
+}
diff --git a/tests/templates/kuttl/iceberg-rest/61-assert.yaml b/tests/templates/kuttl/iceberg-rest/61-assert.yaml
new file mode 100644
index 00000000..813504b1
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/61-assert.yaml
@@ -0,0 +1,11 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 600
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: provision-nifi-flow
+status:
+ succeeded: 1
diff --git a/tests/templates/kuttl/iceberg-rest/61-provision-nifi-flow.yaml b/tests/templates/kuttl/iceberg-rest/61-provision-nifi-flow.yaml
new file mode 100644
index 00000000..ab6bdf23
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/61-provision-nifi-flow.yaml
@@ -0,0 +1,114 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: provision-nifi-flow
+spec:
+ template:
+ spec:
+ containers:
+ - name: provision-nifi-flow
+ image: oci.stackable.tech/sdp/testing-tools/nifi:0.3.0-stackable0.0.0-dev
+ command:
+ - bash
+ - -euo
+ - pipefail
+ - -c
+ - python -u /tmp/script/script.py
+ volumeMounts:
+ - name: script
+ mountPath: /tmp/script
+ - name: nifi-flow
+ mountPath: /tmp/nifi-flow
+ - name: nifi-users
+ mountPath: /nifi-users
+ env:
+ - name: NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ volumes:
+ - name: script
+ configMap:
+ name: provision-nifi-flow-script
+ - name: nifi-flow
+ configMap:
+ name: nifi-flow
+ - name: nifi-users
+ secret:
+ secretName: nifi-users
+ restartPolicy: OnFailure
+---
+# Taken from https://github.com/stackabletech/demos/blob/1744be00054eec2827b2d9ef0a90645843cb0075/demos/nifi-kafka-druid-earthquake-data/create-nifi-ingestion-job.yaml#L52
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: provision-nifi-flow-script
+data:
+ script.py: |
+ from nipyapi.canvas import get_root_pg_id, schedule_process_group, list_all_controllers, schedule_controller
+ from nipyapi.security import service_login
+ import nipyapi
+ import os
+ import requests
+ import urllib3
+
+ # As of 2022-08-29 we cant use "https://nifi:8443" here because The request contained an invalid host header [nifi:8443] in the request [/nifi-api]. Check for request manipulation or third-party intercept.
+ ENDPOINT = f"https://nifi-node-default-0.nifi-node-default-headless.{os.environ['NAMESPACE']}.svc.cluster.local:8443" # For local testing / developing replace it, afterwards change back to f"https://nifi-node-default-0.nifi-node-default-headless.{os.environ['NAMESPACE']}.svc.cluster.local:8443"
+ USERNAME = "admin"
+ PASSWORD = open("/nifi-users/admin").read()
+
+ urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+
+ nipyapi.config.nifi_config.host = f"{ENDPOINT}/nifi-api"
+ nipyapi.config.nifi_config.verify_ssl = False
+
+ print(f"Logging in as {USERNAME}")
+ service_login(username=USERNAME, password=PASSWORD)
+ print("Logged in")
+
+ filename = "/tmp/nifi-flow/nifi-flow.json"
+
+ pg_id = get_root_pg_id()
+ print(f"Got root process group id: {pg_id}")
+
+ if not nipyapi.config.nifi_config.api_client:
+ nipyapi.config.nifi_config.api_client = ApiClient()
+
+ header_params = {}
+ header_params['Accept'] = nipyapi.config.nifi_config.api_client.select_header_accept(['application/json'])
+ header_params['Content-Type'] = nipyapi.config.nifi_config.api_client.select_header_content_type(['multipart/form-data'])
+
+ print("Uploading process group")
+ nipyapi.config.nifi_config.api_client.call_api('/process-groups/{pg_id}/process-groups/upload', 'POST',
+ path_params={'pg_id': pg_id},
+ header_params=header_params,
+ _return_http_data_only=True,
+ post_params=[
+ ('id', pg_id),
+ ('groupName', 'Iceberg Test'),
+ ('positionX', 100),
+ ('positionY', 10),
+ ('clientId', nipyapi.nifi.FlowApi().generate_client_id()),
+ ],
+ files={
+ 'file': filename
+ },
+ auth_settings=['tokenAuth'])
+ print("Process group uploaded")
+
+ # As they are started in the wrong order :D we need to retry
+ max_retries = 5
+ for _ in range(max_retries):
+ controllers = list_all_controllers(pg_id)
+ print(f"Found {len(controllers)} controllers")
+ for controller in controllers:
+ if controller.component.state != "ENABLED":
+ try:
+ print(f"Scheduling controller {controller.component.name}")
+ schedule_controller(controller, scheduled=True)
+ print(f"Scheduled controller: {controller.component.name}")
+ except Exception as e:
+ print(f"Failed to schedule controller {controller.component.name}: {e}")
+
+ schedule_process_group(pg_id, scheduled=True)
diff --git a/tests/templates/kuttl/iceberg-rest/70-assert.yaml b/tests/templates/kuttl/iceberg-rest/70-assert.yaml
new file mode 100644
index 00000000..4a7bd272
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/70-assert.yaml
@@ -0,0 +1,11 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 600
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: check-iceberg-tables
+status:
+ succeeded: 1
diff --git a/tests/templates/kuttl/iceberg-rest/70-check-iceberg-tables.yaml.j2 b/tests/templates/kuttl/iceberg-rest/70-check-iceberg-tables.yaml.j2
new file mode 100644
index 00000000..e985505b
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/70-check-iceberg-tables.yaml.j2
@@ -0,0 +1,36 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: check-iceberg-tables
+spec:
+ template:
+ spec:
+ containers:
+ - name: check-iceberg-tables
+ image: "oci.stackable.tech/sdp/trino-cli:{{ test_scenario['values']['trino-l'] }}-stackable0.0.0-dev"
+ command:
+ - bash
+ - -euo
+ - pipefail
+ - -c
+ - |
+ for SCHEMA in lakehouse.test; do
+ COUNT=$(cat << EOF | ./trino-cli --server https://trino-coordinator:8443 --insecure --user admin
+ SELECT COUNT(*) FROM $SCHEMA.greetings WHERE hello = 'world from NiFi :)';
+ EOF
+ )
+
+ COUNT="${COUNT%\"}" # Remove trailing quote if any
+ COUNT="${COUNT#\"}" # Remove leading quote if any
+ echo "Count is $COUNT"
+
+ # Check if it's a number greater than 0
+ if ! [[ "$COUNT" =~ ^[0-9]+$ ]] || [ "$COUNT" -le 0 ]; then
+ echo "Invalid or zero count: $COUNT"
+ exit 1
+ fi
+
+ echo "Count $COUNT was valid"
+ done
+ restartPolicy: OnFailure
diff --git a/tests/templates/kuttl/iceberg-rest/README.md b/tests/templates/kuttl/iceberg-rest/README.md
new file mode 100644
index 00000000..1bdee97b
--- /dev/null
+++ b/tests/templates/kuttl/iceberg-rest/README.md
@@ -0,0 +1,25 @@
+The file `60_nifi-flow.json` was exported from the NiFi UI.
+
+*However*, we need to update some stuff, such as adding S3 credentials and templating the namespace of MinIO.
+
+TIP: I used `JSON: Sort Document` in VScode to somewhat have consistent formatting, which makes reading and diffs easier.
+
+Notable the following diff has been made (may not be up to date!):
+
+```diff
+diff --git a/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json b/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json
+index eb64241..6ead26e 100644
+--- a/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json
++++ b/tests/templates/kuttl/iceberg-rest/60_nifi-flow.json
+@@ -331,7 +331,9 @@
+ },
+ "properties": {
+ "Authentication Strategy": "BASIC_CREDENTIALS",
+- "Endpoint Override URL": "https://minio.kuttl-test-dear-bug.svc.cluster.local:9000",
++ "Access Key ID": "admin",
++ "Secret Access Key": "adminadmin",
++ "Endpoint Override URL": "https://minio.${NAMESPACE}.svc.cluster.local:9000",
+ "Use Path Style Access": "true",
+ "Client Region": "us-east1"
+ },
+```
diff --git a/tests/templates/kuttl/iceberg/interactive-nifi.yaml b/tests/templates/kuttl/iceberg/interactive-nifi.yaml
deleted file mode 100644
index 81e42630..00000000
--- a/tests/templates/kuttl/iceberg/interactive-nifi.yaml
+++ /dev/null
@@ -1,89 +0,0 @@
-apiVersion: nifi.stackable.tech/v1alpha1
-kind: NifiCluster
-metadata:
- name: nifi-interactive
-spec:
- image:
- productVersion: 2.2.0
- clusterConfig:
- authentication:
- - authenticationClass: nifi-users
- sensitiveProperties:
- keySecret: nifi-interactive-sensitive-property-key
- autoGenerate: true
- zookeeperConfigMapName: nifi-interactive-znode
- extraVolumes:
- - name: nifi-processors
- persistentVolumeClaim:
- claimName: nifi-interactive-processors
- - name: hdfs-config
- configMap:
- name: hdfs
- - name: nifi-interactive-hive-site
- configMap:
- name: nifi-interactive-hive-site
- - name: kerberos
- ephemeral:
- volumeClaimTemplate:
- metadata:
- annotations:
- secrets.stackable.tech/class: kerberos-kuttl-test-united-pheasant
- secrets.stackable.tech/kerberos.service.names: nifi
- secrets.stackable.tech/scope: service=nifi
- spec:
- accessModes:
- - ReadWriteOnce
- resources:
- requests:
- storage: "1"
- storageClassName: secrets.stackable.tech
- nodes:
- roleConfig:
- listenerClass: external-unstable
- configOverrides:
- nifi.properties:
- nifi.nar.library.directory.myCustomLibs: /stackable/userdata/nifi-processors/
- nifi.kerberos.krb5.file: /stackable/userdata/kerberos/krb5.conf
- # Quicker startup
- nifi.cluster.flow.election.max.wait.time: 3 secs
- envOverrides:
- KERBEROS_REALM: PROD.MYCORP
- roleGroups:
- default:
- replicas: 1
----
-apiVersion: zookeeper.stackable.tech/v1alpha1
-kind: ZookeeperZnode
-metadata:
- name: nifi-interactive-znode
-spec:
- clusterRef:
- name: zookeeper
----
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
- name: nifi-interactive-processors
-spec:
- accessModes:
- - ReadWriteOnce
- resources:
- requests:
- storage: 10Gi
----
-apiVersion: v1
-kind: ConfigMap
-metadata:
- name: nifi-interactive-hive-site
-data:
- hive-site.xml: |
-
-
- hive.metastore.kerberos.principal
- hive/hive.kuttl-test-united-pheasant.svc.cluster.local@MY.CORP
-
-
- hive.metastore.sasl.enabled
- true
-
-
diff --git a/tests/templates/kuttl/oidc-opa/19_keycloak.yaml.j2 b/tests/templates/kuttl/oidc-opa/19_keycloak.yaml.j2
index 9f43dcbb..dc168c9d 100644
--- a/tests/templates/kuttl/oidc-opa/19_keycloak.yaml.j2
+++ b/tests/templates/kuttl/oidc-opa/19_keycloak.yaml.j2
@@ -61,7 +61,7 @@ spec:
serviceAccountName: keycloak
containers:
- name: keycloak
- image: quay.io/keycloak/keycloak:25.0.0
+ image: quay.io/keycloak/keycloak:26.5.0
args:
- start-dev
- --import-realm
diff --git a/tests/templates/kuttl/oidc-opa/25-opa-rego.yaml b/tests/templates/kuttl/oidc-opa/25-opa-rego.yaml
index 0f82301e..16d60f5d 100644
--- a/tests/templates/kuttl/oidc-opa/25-opa-rego.yaml
+++ b/tests/templates/kuttl/oidc-opa/25-opa-rego.yaml
@@ -12,7 +12,7 @@ data:
nifi_node_proxy := "CN=generated certificate for pod"
nifi_reporting_task_user := "admin"
- # Setting "resourceNotFound" to true results in the parent resource beingevaluated for authorization,
+ # Setting "resourceNotFound" to true results in the parent resource being evaluated for authorization,
# e.g. the parent of a processor is the processor-group.
# If a resource is matched by a rego rule that is not the default the parent resource will be ignored.
default allow := {
@@ -54,7 +54,7 @@ data:
group in ["nifi-process-group-b"]
}
- # Allow read-only access to group "nifi-user" to proccess group C
+ # Allow read-only access to group "nifi-user" to process group C
allow := {
"allowed": true
} if {
diff --git a/tests/test-definition.yaml b/tests/test-definition.yaml
index a8b330e5..dcc5dc37 100644
--- a/tests/test-definition.yaml
+++ b/tests/test-definition.yaml
@@ -2,37 +2,40 @@
dimensions:
- name: nifi
values:
- - 1.27.0
- 1.28.1
- - 2.4.0
- 2.6.0
+ - 2.7.2,oci.stackable.tech/sandbox/sbernauer/nifi:2.7.2-no-hive-with-endpoint-stackable0.0.0-dev-amd64
# Alternatively, if you want to use a custom image, append a comma and the full image name to the product version
# as in the example below.
# - 2.6.0,oci.stackable.tech/sandbox/nifi:2.6.0-stackable0.0.0-dev
- name: nifi_old
values:
- - 1.27.0
+ - 1.28.1
- name: nifi_new
values:
- - 2.6.0 # oci.stackable.tech/sandbox/nifi:2.6.0-stackable0.0.0-dev
+ - 2.7.2
- name: nifi-latest
values:
- - 2.6.0 # oci.stackable.tech/sandbox/nifi:2.6.0-stackable0.0.0-dev
- - name: nifi-iceberg
+ - 2.7.2,oci.stackable.tech/sandbox/sbernauer/nifi:2.7.2-no-hive-with-endpoint-stackable0.0.0-dev-amd64
+ - name: nifi-iceberg-hive
# Not all NiFi versions support Iceberg with the same functionality!
# E.g. our own implementation started with NiFi 2.2.0
values:
- - 2.6.0 # oci.stackable.tech/sandbox/nifi:2.6.0-stackable0.0.0-dev
- # TODO: Can be removed once NiFi 1.x.x is removed - only for seperate smoke tests
+ - 2.6.0
+ - name: nifi-iceberg-rest
+ values:
+ # - 2.7.2
+ # - 2.7.2,oci.stackable.tech/sandbox/sbernauer/nifi:2.7.2-no-hive-stackable0.0.0-dev-amd64
+ - 2.7.2,oci.stackable.tech/sandbox/sbernauer/nifi:2.7.2-no-hive-with-endpoint-stackable0.0.0-dev-amd64
+ # TODO: Can be removed once NiFi 1.x.x is removed - only for separate smoke tests
- name: nifi-v1
values:
- - 1.27.0
- 1.28.1
- # TODO: Can be removed once NiFi 1.x.x is removed - only for seperate smoke tests
+ # TODO: Can be removed once NiFi 1.x.x is removed - only for separate smoke tests
- name: nifi-v2
values:
- - 2.4.0
- 2.6.0
+ - 2.7.2,oci.stackable.tech/sandbox/sbernauer/nifi:2.7.2-no-hive-with-endpoint-stackable0.0.0-dev-amd64
# Use Kubernetes manager if set to false
- name: use-zookeeper-manager
values:
@@ -137,9 +140,9 @@ tests:
- oidc-use-tls
- opa-l
- openshift
- - name: iceberg
+ - name: iceberg-hive
dimensions:
- - nifi-iceberg
+ - nifi-iceberg-hive
- opa-l
- zookeeper-latest
- hdfs-l
@@ -149,6 +152,12 @@ tests:
- iceberg-use-kerberos
- kerberos-realm
- openshift
+ - name: iceberg-rest
+ dimensions:
+ - nifi-iceberg-rest
+ - hive-l
+ - trino-l
+ - openshift
- name: custom-components-git-sync
dimensions:
- nifi