How does a data lakehouse work?

A data lakehouse is a unified data architecture that combines the low-cost, flexible storage of a data lake with the structure, query performance, and reliability of a data warehouse. It stores raw and structured data in open file formats on object storage, then applies a metadata and transaction layer on top to enable SQL analytics, machine learning, and streaming workloads from a single platform. The sections below break down how each piece of that architecture fits together, and where Stackable comes into the picture at the end.

What separates a data lakehouse from a data warehouse and data lake?

A data lakehouse sits between a data lake and a data warehouse by design. A data lake stores raw data cheaply but lacks the transactional guarantees and schema enforcement that analytical workloads need. A data warehouse offers strong query performance and data consistency but is expensive to scale and struggles with unstructured data. The lakehouse merges both: open storage with warehouse-grade reliability.

The practical difference comes down to three things. First, a data lake stores everything but makes it hard to trust. Schema drift, duplicate records, and no transaction support mean query results can be inconsistent. Second, a traditional data warehouse enforces structure and delivers fast queries, but ingesting raw or semi-structured data requires significant transformation work upfront, and storage costs scale quickly.

A lakehouse resolves this by introducing a transactional metadata layer, such as Apache Iceberg or Delta Lake, directly on top of open object storage. This layer adds ACID transactions, schema evolution, and time travel without requiring data to be copied into a proprietary store. The result is a single system that data engineers, analysts, and data scientists can all work with, using tools they already know.

What are the core components of a data lakehouse architecture?

A data lakehouse architecture has four core components: open file storage, a table format layer, a query and compute engine, and a governance and catalog layer. Together, these allow structured and unstructured data to coexist in one system while supporting diverse workloads with consistent data quality.

Open file storage

The foundation is object storage, typically using open formats like Apache Parquet or ORC. Data lands here in its raw or semi-processed form, and the storage layer is decoupled from compute, meaning you can scale each independently. This is what keeps lakehouse storage costs significantly lower than traditional warehouses.

Table format and transaction layer

Sitting on top of the file storage is a table format such as Apache Iceberg or Apache Hive™ Metastore. This layer tracks which files belong to which table version, manages schema changes, and enforces transactional semantics. Without it, a data lake is just a collection of files with no reliable way to update or delete records safely.

Query and compute engine

Engines like Trino or Apache Spark™ read from the table format layer and execute queries across the stored data. Because these engines are decoupled from storage, multiple engines can query the same data simultaneously without conflict, and you can swap or add engines without migrating data.

Catalog and governance layer

A metadata catalog registers tables, schemas, and lineage information. This is where access control policies, data classification, and audit logging are enforced. Without a catalog, a lakehouse becomes very difficult to govern at scale.

How does a data lakehouse handle ACID transactions?

A data lakehouse handles ACID transactions through its table format layer, not the storage layer itself. Formats like Apache Iceberg implement atomicity, consistency, isolation, and durability by managing a transaction log that tracks every change to a table as an immutable snapshot. Writes either complete fully or do not appear at all, which prevents partial updates from corrupting query results.

When a writer commits a change, it creates a new snapshot in the transaction log pointing to the updated set of files. Readers see a consistent view of the table based on the snapshot at the time their query starts, even if a concurrent write is in progress. This is called snapshot isolation, and it allows multiple readers and writers to operate on the same table without locking each other out.

This mechanism also enables time travel. Because every snapshot is retained for a configurable period, you can query the state of a table at any point in the past. That’s useful for auditing, debugging pipelines, and recovering from accidental deletes, none of which is straightforward in a traditional data lake.

What types of workloads can a data lakehouse support?

A data lakehouse can support SQL analytics, machine learning, batch processing, and streaming ingestion from a single storage layer. This multi-workload capability is one of its defining advantages over architectures that require separate systems for different use cases.

SQL analytics: Query engines like Trino execute SQL directly against Iceberg or Hive tables on object storage, supporting business intelligence and ad hoc analysis without moving data into a separate warehouse.
Machine learning: Data scientists can read training datasets directly from the same storage using Apache Spark™ or Python-based frameworks, avoiding the need to export data to a separate feature store for every experiment.
Batch ETL and data engineering: Spark and similar engines process large-scale transformations and write results back to the lakehouse in structured table formats, maintaining full ACID guarantees throughout.
Streaming ingestion: Apache Kafka® can feed real-time event data into the lakehouse, with streaming engines writing micro-batches to Iceberg tables. This allows near-real-time analytics without a separate streaming database.

The key enabler here is the decoupling of storage from compute. Because the table format layer is engine-agnostic, you are not forced to route all workloads through a single engine. Each tool reads the same data, governed by the same metadata layer.

How does data governance work in a lakehouse?

Data governance in a lakehouse is enforced through a combination of the metadata catalog, access control policies applied at the table format layer, and audit logging. The catalog is the central registry where tables, schemas, ownership, and classification labels are managed. Governance policies reference this catalog to determine who can read or write which tables and under what conditions.

Because the lakehouse uses open table formats, governance can be applied consistently regardless of which compute engine is running the query. A Trino query and a Spark job reading the same Iceberg table both pass through the same policy enforcement point, which is not always the case in architectures where different engines maintain separate permission systems.

Column-level security and row-level filtering are supported by mature lakehouse governance implementations. This lets teams expose a single table to multiple consumers with different access rights, rather than maintaining separate copies of the same dataset for each team. That matters especially for organizations operating under strict data residency or privacy requirements.

For organizations building toward a data mesh architecture, the lakehouse governance model maps well to federated ownership: each domain team manages its own tables and enforces its own policies, while a central catalog maintains discoverability and interoperability across domains.

What open-source tools are used to build a data lakehouse?

A production-grade open-source data lakehouse typically combines Apache Iceberg as the table format, Trino or Apache Spark™ as the query engine, Apache Kafka® for streaming ingestion, and a metadata catalog such as Apache Hive™ Metastore or an Iceberg-compatible REST catalog. Each of these tools is independently maintained, widely adopted, and vendor-neutral.

Apache Iceberg: The most widely adopted open table format for lakehouse architectures. It provides ACID transactions, schema evolution, partition pruning, and time travel on top of any object storage.
Trino: A distributed SQL query engine optimized for interactive analytics across large datasets. It supports Iceberg natively and can federate queries across multiple data sources simultaneously.
Apache Spark™: The standard engine for large-scale batch processing and machine learning workloads. Spark’s Iceberg connector allows it to read and write transactional tables with full ACID support.
Apache Kafka®: The standard for high-throughput event streaming. Kafka feeds real-time data into the lakehouse, which streaming engines then commit to Iceberg tables in near-real time.
Apache Hive™ Metastore or Iceberg REST Catalog: Provide the central metadata registry that query engines use to discover tables and enforce schema.

Running these tools together requires careful coordination of versions, configurations, and network policies. Kubernetes-native deployment helps significantly here, as it provides a consistent operational model across all components. You can explore working lakehouse demos that show how these tools connect in practice.

When should an organization choose a lakehouse over a traditional warehouse?

An organization should choose a data lakehouse over a traditional warehouse when it needs to support multiple workload types from a single storage layer, when data volumes make proprietary warehouse storage costs prohibitive, or when it requires full control over its data without dependence on a specific vendor’s format or pricing model.

A traditional warehouse remains a reasonable choice when the workloads are exclusively SQL analytics, the data is well-structured and relatively modest in volume, and the team does not have the operational capacity to manage a distributed open-source stack. Warehouses trade flexibility for simplicity, and that trade-off is worth it for some organizations.

The lakehouse becomes the stronger choice when:

Data scientists and data engineers need to share the same datasets without copying data between systems
Streaming and batch workloads need to coexist on the same data
Data sovereignty requirements mean the organization cannot store data in a managed cloud service
Long-term storage costs need to be controlled by using open formats on commodity object storage
The organization wants to avoid format lock-in and retain the ability to switch query engines without migrating data

For organizations in regulated industries, the lakehouse model also supports stricter audit and lineage requirements, since every change to every table is recorded in the transaction log and queryable through time travel.

How Stackable helps you build a data lakehouse

The Stackable Data Platform (SDP) provides a modular, Kubernetes-native foundation for deploying and operating the open-source tools that make up a production data lakehouse. Rather than assembling and configuring each component manually, the SDP gives you operators for Apache Kafka®, Trino, Apache Spark™, and Apache Hive™ Metastore that are designed to work together out of the box, with consistent configuration, monitoring, and lifecycle management across all components.

Concretely, the SDP supports lakehouse deployments by:

Providing Kubernetes operators for each core lakehouse component, handling installation, configuration, upgrades, and failure recovery automatically
Supporting on-premises, cloud, edge, and hybrid deployments with the same configuration model, so your lakehouse runs where your data needs to be
Keeping all data and metadata within your own infrastructure, with no dependency on a managed cloud service
Offering a self-serve experience through infrastructure-as-code, so platform teams can provision lakehouse environments reproducibly and audit every change
Including policy-based access control and governance tooling aligned with federated data mesh architectures

If you want to see how a complete lakehouse stack comes together on the SDP, the demo environment is a good starting point. To discuss your specific architecture requirements, get in touch with the team directly.