Introduction
We would like to give you an update on our efforts to port the Stackable Data Platform (SDP) to the ARM64 architecture.
At Stackable, we strongly believe that ARM64 has a bright future as support is growing rapidly, primarily due to the power consumption, competitive pricing and economic advantages of the ARM64 processor architecture.
We have been working hard to port the Stackable Data Platform to ARM64. Initially, we released manually built ARM64 images. With the 23.11.0 release we reached a point where we could test and develop on a bare-metal ARM cluster. So far, we’ve been using ARM-based instances on Azure, and going forward, we’re working with HPE, who has graciously provided us with resources to build and test Stackable Data Platform on multiple architectures using their brand new and powerful Ampere-based hardware.
We anticipate that the next Stackable release will include multi-architecture images for all operators and all products except Apache Superset and Apache Airflow. Unfortunately, these currently require workarounds that we don’t want to include in our products until a robust solution exists.
The rise of ARM64 processors
In recent years, ARM64 technologies have become increasingly accessible to the general public. The movement started with ARM-based processors gaining widespread adoption in smartphones. Then ARM64 entered personal computers with Apple’s M1 chip. Today, many cloud service providers including IONOS, AWS, Hetzner, Google and Azure have added ARM64 hardware to their portfolios.
ARM64 vs X86_64
As more ARM64 options become available in the cloud, the question is whether ARM64-based systems can offer an advantage over their x86 twin.
In addition to generally competitive pricing, ARM64 installations typically consume less power for the same workload compared to it’s Intel x86 sibling and often outperform the X86_64 architecture for pure CPU based computational jobs.
In summary, depending on the workload, we think that ARM64 can be a valid, cost-effective option as a runtime foundation.
The road of porting SDP to ARM64
The Stackable Data Platform (SPD) was originally designed for the X86_64 architecture. This means that our infrastructure, docker files and dependency management were designed and optimized for X86_64 only.
However, in 2022, some new software engineers joined the company and chose Apple’s M1 (M1-Pro) chip as their development environment. The challenge was therefore to provide at least basic support for this architecture. This led to the start of a series of changes:
- A major challenge along the way was to host, select, and extend dependencies so that 3rd party software could be provided for both architectures.
- Besides moving basic dependencies like
ubi8-minimalto a manifest list, we had to rework some architecture selection mechanisms in our Dockerfiles as well as make some major adjustments to our build process as not all standard github actions were and are available for ARM64. - We also had to rewrite some of our Rust code as porting to other architectures was not type safe.
- Another big challenge was choosing the runner on which to build ARM64. We evaluated QEMU builds, as well as BuildJet and a custom runner solution discussed later in this post.
We envision that the end result will be an automated workflow that supports both platforms out of the box.
Current state of our solution
As of today, we have done all the groundwork to have a working solution on ARM64. With our release 23.11.0 we have made ARM64 images available to the public. We have also tested these images to support our popular technology demos. However, we have decided to release this functionality as an experimental feature.
Moving forward, we are currently evaluating several options for our build process. One requirement is that we need ARM64 runners that follow the same workflow as the X86_64 runners and finally bundle them into a manifest list. We also want to take advantage of our powerful Ampere hardware to build a long-lived test cluster based on the ARM64 architecture.
Status summary:
| Vendor | Current State | Evaluation |
| Github runner | Tested / Non Productive | Not enough software support |
| BuildJet | Tested / Prod. Candidate | Easy setup, low maintenance |
| Custom Runner (self-hosted, powered by Ampere) | Tested / Non Productive | Most flexible, high maintenance, risks on custom runner github |
So far we have had the most success with a custom runner and alternatively with BuildJet’s ARM64 runner. Both allowed us to build multi-architecture images in our github CI pipeline.
To achieve a high quality multi-architecture build, we use docker’s manifest list feature to create one tag, e.g. ubi8-rust-builder:latest, which is a manifest list that includes ubi8-rust-builder:latest-aarch64 and ubi8-rust-builder:latest-x86_64. This means that if one executes docker pull ubi8-rust-builder:latest, docker will automatically choose the correct image architecture. In the same way, helm or kubectl will choose which image architecture to pull. This abstracts away the added complexity for the user.
Early access
As mentioned above, we have the 23.11.0 release working on ARM64. You can check out our documentation if you want a more detailed look.
To make a long story short, you can use the --set image.repository=docker.stackable.tech/stackable-experimental/ feature of helm to select an ARM64 image from any SDP operator while following our installation guide. Deployment of products with ARM64 architecture becomes possible by using our Image Selection Mechanism.
These two steps will give you access to experimental SDP builds for ARM64.
Outlook
Stackable is fully committed to porting our data platform to ARM64. Thus, we are in the process of releasing 24.03 partly via our automated pipeline into the stackable-experimental domain of our public repo. Our plan is to create a comprehensive testing structure to guarantee the same quality you are used to.
We also anticipate merging stackable-experimental into the stackable organization with release 24.07. At that point, multi-architecture will no longer be experimental, but a fully qualified first-class citizen of SDP, accessible in the same way you are already familiar with.