chore: Add support for 2.7.0, drop 1.27.0 and 2.4.0

CHANGELOG.md

@@ -9,6 +9,11 @@ All notable changes to this project will be documented in this file.
 - Support objectOverrides using `.spec.objectOverrides`.
   See [objectOverrides concepts page](https://docs.stackable.tech/home/nightly/concepts/overrides/#object-overrides) for details ([#885]).
 - Enable the [restart-controller](https://docs.stackable.tech/home/nightly/commons-operator/restarter/), so that the Pods are automatically restarted on config changes ([#888]).
+- Added support for `2.6.0` ([#XXX]).
+
+### Removed
+
+- Removed support for `1.27.0` and `2.4.0` ([#XXX]).
 
 ### Changed
 
@@ -25,6 +30,7 @@ All notable changes to this project will be documented in this file.
 [#884]: https://github.com/stackabletech/nifi-operator/pull/884
 [#885]: https://github.com/stackabletech/nifi-operator/pull/885
 [#888]: https://github.com/stackabletech/nifi-operator/pull/888
+[#XXX]: https://github.com/stackabletech/nifi-operator/pull/XXX
 
 ## [25.11.0] - 2025-11-07
 

docs/modules/nifi/pages/usage_guide/index.adoc

@@ -17,12 +17,11 @@ metadata:
   name: simple-nifi
 spec:
   image:
-    productVersion: 1.27.0
+    productVersion: 2.7.2
   clusterConfig:
-    zookeeperConfigMapName: simple-nifi-znode  # <1>
-    authentication:  # <2>
+    authentication:  # <1>
       - authenticationClass: simple-nifi-admin-user
-    extraVolumes:  # <3>
+    extraVolumes:  # <2>
       - name: nifi-client-certs
         secret:
           secretName: nifi-client-certs
@@ -35,18 +34,17 @@ spec:
   roleGroups:
     default:
       config:
-        resources:  # <4>
+        resources:  # <3>
           cpu:
             min: "500m"
             max: "4"
           memory:
-            limit: '2Gi'
+            limit: 2Gi
         replicas: 3
 ----
 
-<1> The xref:usage_guide/clustering.adoc#backend-zookeeper[ZooKeeper instance] to use.
-<2> How users should xref:usage_guide/security.adoc[authenticate] themselves.
-<3> xref:usage_guide/extra-volumes.adoc[Extra volumes] with files that can be referenced in custom workflows.
-<4> xref:usage_guide/resource-configuration.adoc[CPU and memory configuration] can be set per role group.
+<1> How users should xref:usage_guide/security.adoc[authenticate] themselves.
+<2> xref:usage_guide/extra-volumes.adoc[Extra volumes] with files that can be referenced in custom workflows.
+<3> xref:usage_guide/resource-configuration.adoc[CPU and memory configuration] can be set per role group.
 
 Not shown are the common settings for xref:usage_guide/operations/cluster-operations.adoc[starting and stopping the cluster] and xref:usage_guide/operations/pod-placement.adoc[distributing Pods]. Additionally, you can set any NiFi setting using xref:usage_guide/overrides.adoc[overrides]. You can also configure xref:usage_guide/log-aggregation.adoc[log aggregation].

docs/modules/nifi/pages/usage_guide/updating.adoc

@@ -2,7 +2,7 @@
 :description: Easily update or downgrade Apache NiFi on Kubernetes by changing the CRD version.
 
 Updating (or downgrading for that matter) the deployed version of NiFi is as simple as changing the version stated in the CRD.
-Continuing the example above, to change the deployed version from `1.27.0` to `2.0.0` you'd simply deploy the following CRD.
+Continuing the example above, to change the deployed version from `2.6.0` to `2.7.2` you'd simply deploy the following CRD.
 
 [source,yaml]
 ----
@@ -12,7 +12,7 @@ metadata:
   name: simple-nifi
 spec:
   image:
-    productVersion: 2.0.0 # <1>
+    productVersion: 2.7.2 # <1>
 ----
 
 <1> Change the NiFi version here
@@ -24,7 +24,7 @@ NiFi clusters cannot be upgraded or downgraded in a rolling fashion due to a lim
 When upgrading between NiFi 1 versions or from NiFi 1 to NiFi 2, any change to the NiFi version in the CRD triggers a full cluster restart with brief downtime.
 However, the Stackable image version can be updated in a rolling manner, provided the NiFi version remains unchanged.
 
-For upgrades between NiFi 2 versions, e.g. from `2.0.0` to `2.4.0`, rolling upgrades are supported.
+For upgrades between NiFi 2 versions, e.g. from `2.6.0` to `2.7.2`, rolling upgrades are supported.
 ====
 
 == NiFi 2.0.0

docs/modules/nifi/pages/usage_guide/writing-to-iceberg-tables.adoc

@@ -6,9 +6,23 @@
 Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the same tables, at the same time.
 
 NiFi supports a `PutIceberg` processor to add rows to an existing Iceberg table https://issues.apache.org/jira/browse/NIFI-10442[starting from version 1.19.0].
-As of NiFi version `2.4.0` only `PutIceberg` is supported, you need to create and compact your tables with other tools such as Trino or Spark (both included in the Stackable Data Platform).
+As of NiFi version `2.7.2` only `PutIceberg` is supported, you need to create and compact your tables with other tools such as Trino or Spark (both included in the Stackable Data Platform).
 
-== NiFi 2
+== NiFi 2.7 and above
+
+In NiFi `2.7.0` Iceberg support was https://issues.apache.org/jira/browse/NIFI-15062[re-added] after the removal in 2.0.0.
+
+The 2.7 version has the following changes over the 2.0/2.6 version, you need to adopt your setup accordingly:
+
+* HDFS and Kerberos support was dropped
+* Hive metastore support was dropped
+* Iceberg REST catalog support was added
+* It now uses the Iceberg S3 IO instead of the Hadoop S3 client libraries
+* It uses much less dependencies and therefore CVEs
+
+There have been efforts from Stackable to re-add at least Hive metastore support, but that turned out to be complicated.
+
+== NiFi 2.0 - 2.6
 
 In NiFi `2.0.0` Iceberg support https://issues.apache.org/jira/browse/NIFI-13938[has been removed] from upstream NiFi.
 

docs/modules/nifi/partials/supported-versions.adoc

@@ -2,9 +2,8 @@
 // This is a separate file, since it is used by both the direct NiFi-Operator documentation, and the overarching
 // Stackable Platform documentation.
 
-* 2.6.0 (LTS, Please note that you need to upgrade to at least 1.27.x before upgrading to 2.x.x!)
-* 2.4.0 (Deprecated, Please note that you need to upgrade to at least 1.27.x before upgrading to 2.x.x!)
+* 2.7.2
+* 2.6.0 (LTS)
 * 1.28.1 (Deprecated)
-* 1.27.0 (Deprecated)
 
 For details on how to upgrade your NiFi version, refer to xref:nifi:usage_guide/updating.adoc[].

rust/operator-binary/src/config/mod.rs

@@ -818,7 +818,7 @@ mod tests {
           name: simple-nifi
         spec:
           image:
-            productVersion: 1.27.0
+            productVersion: 2.7.2
           clusterConfig:
             authentication:
               - authenticationClass: nifi-admin-credentials-simple
@@ -867,7 +867,7 @@ mod tests {
           name: simple-nifi
         spec:
           image:
-            productVersion: 1.27.0
+            productVersion: 2.7.2
           clusterConfig:
             authentication:
               - authenticationClass: nifi-admin-credentials-simple

rust/operator-binary/src/crd/affinity.rs

@@ -43,7 +43,7 @@ mod tests {
           name: simple-nifi
         spec:
           image:
-            productVersion: 1.27.0
+            productVersion: 2.7.2
           clusterConfig:
             authentication:
               - authenticationClass: nifi-admin-credentials-simple

tests/templates/kuttl/iceberg/50_nifi.yaml.j2 → tests/templates/kuttl/iceberg-hive/50_nifi.yaml.j2

@@ -5,12 +5,12 @@ metadata:
   name: nifi
 spec:
   image:
-{% if test_scenario['values']['nifi-iceberg'].find(",") > 0 %}
-    custom: "{{ test_scenario['values']['nifi-iceberg'].split(',')[1] }}"
-    productVersion: "{{ test_scenario['values']['nifi-iceberg'].split(',')[0] }}"
+{% if test_scenario['values']['nifi-iceberg-hive'].find(",") > 0 %}
+    custom: "{{ test_scenario['values']['nifi-iceberg-hive'].split(',')[1] }}"
+    productVersion: "{{ test_scenario['values']['nifi-iceberg-hive'].split(',')[0] }}"
 {% else %}
     custom: null
-    productVersion: "{{ test_scenario['values']['nifi-iceberg'] }}"
+    productVersion: "{{ test_scenario['values']['nifi-iceberg-hive'] }}"
 {% endif %}
     pullPolicy: IfNotPresent
   clusterConfig:

tests/templates/kuttl/iceberg-rest/00-patch-ns.yaml.j2

@@ -0,0 +1,9 @@
+{% if test_scenario['values']['openshift'] == 'true' %}
+# see https://github.com/stackabletech/issues/issues/566
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+  - script: kubectl patch namespace $NAMESPACE -p '{"metadata":{"labels":{"pod-security.kubernetes.io/enforce":"privileged"}}}'
+    timeout: 120
+{% endif %}

tests/templates/kuttl/iceberg-rest/00-rbac.yaml.j2

@@ -0,0 +1,29 @@
+---
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: test-role
+rules:
+{% if test_scenario['values']['openshift'] == "true" %}
+  - apiGroups: ["security.openshift.io"]
+    resources: ["securitycontextconstraints"]
+    resourceNames: ["privileged"]
+    verbs: ["use"]
+{% endif %}
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: test-sa
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: test-rb
+subjects:
+  - kind: ServiceAccount
+    name: test-sa
+roleRef:
+  kind: Role
+  name: test-role
+  apiGroup: rbac.authorization.k8s.io

tests/templates/kuttl/iceberg-rest/01-create-s3-connection.yaml

@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+  - script: envsubst '$NAMESPACE' < 01_s3-connection.yaml | kubectl apply -n $NAMESPACE -f -

tests/templates/kuttl/iceberg-rest/01_s3-connection.yaml

@@ -0,0 +1,36 @@
+---
+apiVersion: s3.stackable.tech/v1alpha1
+kind: S3Connection
+metadata:
+  name: minio
+spec:
+  host: "minio.${NAMESPACE}.svc.cluster.local"
+  port: 9000
+  accessStyle: Path
+  credentials:
+    secretClass: s3-credentials-class
+  tls:
+    verification:
+      server:
+        caCert:
+          secretClass: tls
+---
+apiVersion: secrets.stackable.tech/v1alpha1
+kind: SecretClass
+metadata:
+  name: s3-credentials-class
+spec:
+  backend:
+    k8sSearch:
+      searchNamespace:
+        pod: {}
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: minio-credentials
+  labels:
+    secrets.stackable.tech/class: s3-credentials-class
+stringData:
+  accessKey: admin
+  secretKey: adminadmin

tests/templates/kuttl/iceberg-rest/10-assert.yaml.j2

@@ -0,0 +1,10 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: vector-aggregator-discovery
+{% endif %}

tests/templates/kuttl/iceberg-rest/10-install-vector-aggregator-discovery-configmap.yaml.j2

@@ -0,0 +1,9 @@
+{% if lookup('env', 'VECTOR_AGGREGATOR') %}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: vector-aggregator-discovery
+data:
+  ADDRESS: {{ lookup('env', 'VECTOR_AGGREGATOR') }}
+{% endif %}

tests/templates/kuttl/iceberg-rest/20-assert.yaml

@@ -0,0 +1,12 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestAssert
+timeout: 600
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: minio
+status:
+  readyReplicas: 1
+  replicas: 1

tests/templates/kuttl/iceberg-rest/20-install-minio.yaml

@@ -0,0 +1,5 @@
+---
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+  - script: kubectl -n $NAMESPACE apply -f 20_minio.yaml