We’re excited to introduce Stackable Data Platform 25.7. It is a feature-rich release, helping you design modern, open and enterprise-ready data architectures faster than ever.
This version brings early experimental support for Airflow 3.0.1 with a new React UI, Spark Connect for interactive data science workflows directly from notebooks, re-added Iceberg support and new OPA authorization to NiFi 2.4.0, and Trino 476 with built-in AI functions to be used with an LLM of your choice e.g. Ollama, OpenAI, or Anthropic. Combined with OCI-first delivery on oci.stackable.tech, expanded support for Kubernetes 1.30 – 1.33 and OpenShift 4.16 – 4.18, and over 230 CVE fixes, SDP 25.7 makes your data platform smarter, more secure and easier to run.
New Platform Features
- General enhancements
- Products built from source now carry version suffixes (e.g.
3.9.3-stackable25.7.0). - Breaking: Listener operator integrated across all products for consistent external access and metrics.
- Automatic cluster domain detection, removing manual DNS setup.
- Products built from source now carry version suffixes (e.g.
- Security
- New TrustStore resource to fetch trust root certificates via SecretClass.
- Format-specific annotations for customizing secret file names.
- Observability
- All operators now emit OpenTelemetry logs & traces.
- Unified CLI args, environment variables & Helm values to configure exporters.
- Authorization
- Experimental support to fetch user groups from Microsoft Entra in OPA backend.
- Operations
- Operator Helm charts now include RBAC patch permissions for events.k8s.io/events, enabling event aggregation like “error happened 10 times over 5 minutes.”
Platform Improvements
- Security & vulnerabilities
- 230 CVEs fixed across Stackable product images, including 3 critical and 97 high-severity issues addressed.
- System diagnostics & noise reduction
- Internal system diagnostics now logged every 30 minutes instead of every minute.
- Apache Spark
- Support for Spark Connect (experimental).
- NiFi
- Support for rolling upgrades between NiFi 2 versions.
- Restored Iceberg support in NiFi 2.4.0.
- Authorization support using the Open Policy Agent (OPA).
- ZooKeeper
- Native Prometheus metrics support.
- Metrics endpoint added via built-in provider; backports from 3.9.4 into supported 3.9.3 image.
New Product Versions
The following new product versions are now supported (get the list of all supported product versions here):
| Product | New version/s | What’s new ? |
|---|---|---|
| Airflow | 2.10.5 3.0.1 (experimental) | 3.0.1 (Experimental) – Service-Oriented Architecture: A new Task Execution API and airflow api-server enable task execution in remote environments with improved isolation and flexibility (AIP-72). – Edge Executor: A new executor that supports distributed, event-driven, and edge-compute workflows (AIP-69), now generally available. – Stable Authoring Interface: DAG authors should now use the new airflow.sdk namespace to import core DAG constructs like @dag, @task, and DAG. – Scheduler-Managed Backfills: Backfills are now scheduled and tracked like regular DAG runs, with native UI and API support (AIP-78). – DAG Versioning: Airflow now tracks structural changes to DAGs over time, enabling inspection of historical DAG definitions via the UI and API (AIP-66). – Asset-Based Scheduling: The dataset model has been renamed and redesigned as assets, with a new @asset decorator and cleaner event-driven DAG definition (AIP-74, AIP-75). – Support for ML and AI Workflows: DAGs can now run with logical_date=None, enabling use cases such as model inference, hyperparameter tuning, and non-interval workflows (AIP-83). – Removal of Legacy Features: SLAs, SubDAGs, DAG and Xcom pickling, and several internal context variables have been removed. Use the upgrade tools to detect deprecated usage. – Split CLI and API Changes: The CLI has been split into airflow and airflowctl (AIP-81), and REST API now defaults to logical_date=None when triggering a new DAG run. – Modern React UI: A complete UI overhaul built on React and FastAPI includes version-aware views, backfill management, and improved DAG and task introspection (AIP-38, AIP-84). – Migration Tooling: Use ruff and airflow config update to validate DAGs and configurations. Upgrade requires Airflow 2.7 or later and Python 3.9–3.12. |
| Druid | 33.0.0 | – Increase segment load speed #17834 – Scheduled batch ingestion (experimental) #17353 – Improved S3 upload #17674 – Realtime query processing for multi-value strings #1738 |
| HBase | 2.6.2 | Bug fix release only. |
| Apache Hadoop | 3.4.1 | No version change. |
| Hive | 4.0.0 (LTS) 4.0.1 (experimental) | No version change. |
| Kafka | 3.7.2 (LTS) 3.9.0 4.0.0(experimental) | 4.0.0 (experimental) – KRaft now default mode – KIP-848: The Next Generation of the Consumer Rebalance Protocol – KIP-932: Queues for Kafka (Early Access) – KIP-966: Eligible Leader Replicas (Preview) – KIP-1106: Add duration based offset reset option for consumer clients – KIP-1102: Enable clients to rebootstrap based on timeout or error code – KIP-653: Upgrade log4j to log4j2 |
| NiFi | 2.4.0 | – Bitbucket Registry Client – Flow Action Reporter Interface to Framework API for extensible tracking for Flow Configuration History – New Box processors – Support for AWS MSK (Kafka) IAM authentication – OAuth 2.0 Controller Service to support JWT Bearer Flow – Astral uv for dynamic download of Python dependencies – Kafka refactor with significant improvements and bug fixes – Registry client API call frequency changed from 1 minute to 30 minutes |
| Open Policy Agent | 1.4.2 | – OpenTelemetry: HTTP Support and Expanded Batch Span Configuration. – Also lots of Perf improvements, New Buffer Option for Decision Logs |
| Spark | 3.5.6 | Bug fix release only. |
| Superset | 4.1.2 | Bug fix release only. |
| Trino | 476 | In general: – ⚠️ Breaking change: Require JDK 24 to run Trino. (#23498) – Add AI functions for textual tasks on data using OpenAI, Anthropic, or other LLMs using Ollama as backend. (#24963) – Add support for comparing values of geometry type. (#25225) – Fix failure for queries involving casts with row types. (#25864) – Fix query failures when dynamic catalog names contain mixed case letters. (#25701) – Disallow dropping the system catalog. (#24745) – Add support for exporting OpenTelemetry traces using the HTTP protocol with the tracing.exporter.protocol configuration property set to http/protobuf. (#25573) – Add support for array literals. (#25301) – Reduce the amount of memory required for DISTINCT and GROUP BY operations. (#25127) – Color the server console output for improved readability. (#25090) Hive Connector: – Add Local file system support. (#25006) – Allow selecting the AWS signing protocol to use when authenticating S3 requests. (#25820) – Add support for user-assigned managed identity authentication for AzureFS. (#23447) – Improve metadata reading performance by optimizing directory listing cache behavior. (#25749) – Allow cross-region data retrieval when using the S3 native filesystem. (#25200) – Add support for all storage classes when using the S3 native filesystem for writes. (#25435) – Ensure Hive metastore locks are released if a failure occurs during lock acquisition. (#25380) Iceberg Connector: – Add support for user-assigned managed identity authentication for AzureFS. (#23447) – Add signer type support to the native S3 filesystem. (#25820) Improve query planning performance when reading from materialized views. (#24734) – Add the $partition hidden column. (#24301) – Add Local file system support. (#25006) – Add support for S3 Tables. (#24815) |
| Zookeeper | 3.9.3 (LTS) | No version change. |
stackablectl
- Info: In earlier versions, the stackablectl CLI tool releases were synchronized with platform updates, although the tool itself was not linked to any specific version of SDP. Consequently, the decision was made to version stackablectl independently. From this release onward, the release notes for stackablectl will be available on a dedicated page.
More Info
Further details on our release and how to upgrade can be found in the release notes as well as in the change logs of the individual operators:
Airflow, Druid, HBase, HDFS, Hive, Kafka, NiFi, OpenPolicyAgent, Spark, Superset, Trino, ZooKeeper