A few weeks ago we already announced what we are working on at Stackable: a distribution for open source Big Data software for on-premise and the cloud. In this blog post, we would now like to tell more about our origin story and provide insights on why we decided to go this way.
2008 – How we found our niche
For me, it all started in 2008 when I discovered Apache HBase. For those who don’t know what HBase is, it’s a “database” for Big Data use cases, modeled after the Google BigTable paper. I immediately felt at home with HBase, the community, and the entire open source movement.
From that starting point, I started consulting around Big Data projects, became a committer for HBase, Apache Hive and other projects. In 2016, Sönke Liebau, Lars George and I co-founded OpenCore, a company with this very focus: Big Data consulting.
We quickly realized: With our offering, we fill a niche that is in high demand. We focused on very short, but focused deployments: Trainings, deployments, firefighting missions, on-site architecture help, but were also always ready to support our customers in other ways, flexibly and with a hands-on mentality. The goal has always been to empower our customers to help themselves. On the one hand, this has allowed us to serve many customers around the world as a very small team; on the other hand, it has also allowed us to cooperate with numerous other consulting firms, as our business models complement each other perfectly.
2019 – Cloudera & Hortonworks: How we became a community
That brings us to 2019, when software vendors Cloudera and Hortonworks announced their successful merger and, just a short time later, a big change for customers: All previously free Cloudera services were to be available only behind a paywall or subscription in the future. In addition, a price increase for already paid services and a new cloud strategy were to follow. Although the announced changes did not come into effect until spring 2021, we have received numerous inquiries from Cloudera customers since the announcement who wanted to find out about alternatives and exchange information. It was therefore quickly clear to us: there is a need for action here! Already in December 2019, the first of numerous meet-ups with around 30 German companies in the industry took place. The goal: exchange, knowledge transfer, and building an open source community that continues to this day.
2020 – How the community evolved into Stackable
This repeated exchange with “like-minded people” was and is valuable for everyone and has essentially produced four challenges or requirements that are common to all:
- Flexibility: especially against the backdrop of ever faster evolving technologies, there is a strong desire for less rigid solutions. Particularly when it comes to updates, many of the companies want fluid optimizations that can be easily integrated into ongoing operations, rather than individual large-scale updates, some of which may take several months.
- Security: As the number of clusters increases, so does the complexity of securing them. For many users, therefore, a holistic, integrated and transparent security concept is critical, from authentication, authorization and auditing to governance and much more.
- Infrastructure as a Code: Simplicity and flexibility are also playing an increasingly important role when it comes to infrastructure. Accordingly, there is a strong desire to describe the infrastructure in versionable code rather than via a graphical interface.
- Price: Of course – also against the background of the Cloudera developments – the topic of price or license model plays a major role. Modularity and scalability are becoming increasingly important and are currently still available from only a few providers.
2021 – How we now want to take off with Stackable
All these points formed the starting point for the development of the Stackable platform, in which we focus, among other things, on Infrastructure as a Code, on modularity and flexibility as well as easy integration into existing systems and, above all, on the open source principle, which has already proven itself within the community. With the founding of Stackable in September 2020, Sönke and I have laid the foundation and would like to continue to expand it from now on – with the support of the very community to which we owe the idea and the impetus for Stackable. Our thanks go out to all of you!
The community will continue to grow in the future. You want to be part of it? Then sign up now for our newsletter and don’t miss any news about open source, cloud technologies or our next blog posts. There The community will continue to grow in the future. Would you like to be part of it? Then sign up now for our newsletter and don’t miss any news about open source, cloud technologies or our next blog posts. There you will learn more about our next steps, the distribution of the Stackable platform and how you can test it. Be curious!