We are thrilled to announce the release of Stackable Data Platform 25.3. It offers new features, improved security and higher performance to help organizations operate and optimize their data platform efficiently and securely.
With this new release, the Stackable Data Platform now supports running on Kubernetes 1.32, 1.31 and 1.30, as well as RedHat Openshift in version 4.17, 4.16, 4.15 and 4.14.
In addition to many new platform features and enhancements, new features come from product releases such as Trino 470 and Superset 4.1.1.
We would also like to highlight the new accompanying Stackablectl demo jupyterhub-keycloak, which shows the integration of the Stackable Data Platform with Jupyterhub.
New Platform Features
- General:
- Operators deployed by Helm now include an extra pod annotation to trigger rollouts when ConfigMap contents change.
- Java product operators now support setting JVM arguments.
- Authorization:
- Apache Airflow and Superset now support Open Policy Agent for authorization.
- Security:
- Additional trust roots can be specified in an autoTls SecretClass.
- Custom key lengths are now supported for self-signed certificates.
- Observability:
- Products now log container environment information on startup and at intervals.
Platform Improvements
- Vulnerabilities:
- Fixed 41 CVEs, including 6 critical and 16 high-severity issues.
- OCI Registry:
- The OCI registry at
oci.stackable.techis now stable and will be the default as of this release.
- The OCI registry at
- DNS Lookup Performance:
- Experimental support for improved DNS performance by specifying a fully qualified domain name.
- Authorization:
- OPA Rego rules no longer require the
future.keywordsimport.
- OPA Rego rules no longer require the
- Bug Fixes:
- Resolved issues with pods stuck in “Unknown” state and performance degradation in
jmx_exporter.
- Resolved issues with pods stuck in “Unknown” state and performance degradation in
New Product Versions
The following new product versions are now supported (get the list of all supported product versions here):
| Product | New version/s | What’s new ? |
|---|---|---|
| Airflow | 2.10.4 | Version was already included in 24.11, for relevant new features please check the release notes for that release. |
| Druid | 31.0.1, 30.0.1 (LTS) | Official info: https://druid.apache.org/docs/31.0.0/release-info/release-notes/ highlights: – Window functions are GA #17087 – Concurrent append and replace GA – Delta Lake support for complex types #16884 – Delta lake support for snapshot versions #17004 – Projections (experimental) #17214 – Storage improvements #16863 #16849 #16958 – Java 8 support is now deprecated |
| Apache Hadoop | 3.4.1 (LTS) | Bug fix release only. |
| HBase | 2.6.1 (LTS) | Version was already included in 24.11, for relevant new features please check the release notes for that release. |
| Hive | 4.0.0 (LTS) 4.0.1 (experimental) | New version 4.0, see official release notes. There are numerous new features and improvements, but most of these are concerned with the Hive query engine. Since Stackable only uses the metastore, most of these are irrelevant here. Mostly for Stackable users the benefit is that the CVE attack surface has been decreased a lot. |
| Kafka | 3.7.2 (LTS) 3.9.0 | Highlights include: – Implement disable & re-enablement for Tiered Storage KAFKA-15132 – Add remote log metadata flag to the dump log tool KAFKA-16228 |
| NiFi | 1.28.1 2.2.0 (experimental) | Add configurable groups claim mapping for OIDC to NiFi Registry NIFI-13016 |
| Spark | 3.5.5 (LTS) | Bug fix release only. |
| Superset | 4.1.1 | Highlights include: – feat(alert/report): Added optional CC and BCC fields for email notifications #29088 – add support to NOT LIKE operator #2984 – Apache Superset “Factory Reset” CLI command #27221 – Support a dynamic minimum interval for alerts and reports #29241 – feat(trino): Add functionality to upload data #29164 – Data Zoom scrolls using the mouse #28629 – Adding ALL the countries to the Country Map plugin #28265 – feat(dashboard): Add metadata bar to the header #27857 |
| Trino | 470 | Breaking Changes – ⚠️ Breaking change: Shorten the name for the Kafka event listener to kafka. (#23308) – ⚠️ Breaking change: Deactivate legacy file system support for all catalogs. You must activate the desired file system support with fs.native-azure.enabled,fs.native-gcs.enabled, fs.native-s3.enabled, or fs.hadoop.enabled in each catalog. Use the migration guides for Azure Storage, Google Cloud Storage, and S3 to assist if you have not switched from legacy support. (#23343) – ⚠️ Breaking change: Require JDK 23 to run Trino, including updated JVM config. (#21316) – Iceberg: ⚠️ Breaking change: Remove the deprecated schema and table arguments from the table_changes table function. Use schema_name and table_name instead. (#24324) – Iceberg: ⚠️ Breaking change: Use the iceberg.rest-catalog.warehouse configuration property instead of iceberg.rest-catalog.parent-namespace with Unity catalogs. (#24269) – Delta: ⚠️ Breaking change: Reduce coordinator memory usage for the Delta table metadata cache and enable configuration delta.metadata.cache-max-retained-size to control memory usage. Remove the configuration property delta.metadata.cache-size and increase the default for delta.metadata.cache-ttl to 30m. (#24432) – Iceberg: ⚠️ Breaking change: Rename the partitions column in the $manifests metadata table to partition_summaries. (#24103) – JDBC: ⚠️ Breaking change: Raise minimum runtime requirement to Java 11. (#23639) Features and Improvements – Improve performance for LZ4, Snappy and ZSTD compression and decompression used for exchange spooling with fault-tolerant execution. (#22532) – Publish a JDBC driver JAR without bundled, third-party dependencies. (#22098) – Add support for tracing with OpenTelemetry in JDBC. (#23458) – Improve performance for queries with a redundant DISTINCT clause. (#23087) – Iceberg: Add support for WASB to Azure Storage file system support. (#23511) – Delta: Add support for WASBS to Azure Storage file system support. (#23548) – Iceberg: Add support for WASBS to Azure Storage file system support. (#23548) – WebUI: Add buttons on the query list to access query details. (#22831) |
| Zookeeper | 3.9.3 (LTS) | Bug fix release only. |
stackablectl
- New Demo:
- A new demo called jupyterhub-keycloak is available, offering a secure multi-user data science environment on Kubernetes with Single Sign-on Jupyter notebooks integrated with Stackable Spark and S3 storage.
- Versioning:
- Demos and stacks are now versioned. stackablectl by default installs the latest stable demo and/or stack.
- Helm Chart Source:
- Added a new argument
--chart-sourceto pull operator charts from either an OCI registry or an index.yaml-based repository.
- Added a new argument
- Internal PKI Support:
- stackablectl now supports environments with internal PKI using
rustls-native-certs.
- stackablectl now supports environments with internal PKI using
- Improved Tracing and Logging:
- Enhanced tracing and log output for better diagnostics and monitoring.
More Info
Further details on our release and how to upgrade can be found in the release notes as well as in the change logs of the individual operators:
Airflow, Druid, HBase, HDFS, Hive, Kafka, NiFi, OpenPolicyAgent, Spark, Superset, Trino, ZooKeeper