Stackable Data Platform (SDP) Release 23.1 is now publicly available!
Highlights
The focus on this release is on logging, product image selection and LDAP authentication support.
Further details on our release and how to upgrade can be found in our release notes as well as in the change logs of the individual operators (e.g. for Airflow).
Logging
Capturing, persisting and consolidating log output across multiple products is complex. To make this easier we have implemented a logging architecture that consolidates product log data to a searchable “sink”, OpenSearch. This is already available for Apache HBase, Apache Hadoop and Apache Zookeeper and will be rolled out across all our products in future releases.
Product Image selection
This is a breaking change across all operators but adds significant flexibility when selecting product images as these no longer have to come from our public repository. Private repositories can be referenced using your own pull policies and secrets.
LDAP
LDAP authentication has been extended to cover multiple products and we have extended our documentation support for this accordingly.
New product-specific features
We have also added new product-specific features such as:
- support for Trino connectors (Black Hole and Google Sheets)
- support for a configurable Druid segment cache
- support for multiple storage directories for HDFS
- an implementation of Spark History Server
Learning Stackable
To illustrate and explain these changes we have added the following:
- a stackablectl demo showing logging with Vector and OpenSearch
- a stackablectl stack for OpenLDAP that is used by an LDAP tutorial
- concept pages for authentication, logging and product image selection