Apache Iceberg vs. Delta Lake vs. Apache Hudi: what's the difference?

Apache Iceberg, Delta Lake, and Apache Hudi are the three dominant open table formats for data lakehouses, and they solve the same core problem in meaningfully different ways. All three add ACID transactions, schema evolution, and time travel to object storage, but they differ in architecture, ecosystem support, and the specific trade-offs they make. The choice between them depends on your query engine, your streaming requirements, and how much you value format neutrality. We look at where Stackable fits into this picture at the end.

How do Apache Iceberg, Delta Lake, and Apache Hudi actually work?

All three formats work by adding a metadata layer on top of files stored in object storage, S3, HDFS, whatever you’re using, turning a directory of Parquet or Avro files into a versioned, queryable table. That metadata layer is what makes transactions, snapshots, and schema changes possible without rewriting all your underlying data.

Apache Iceberg uses a tree of metadata files: a catalog pointer references a metadata JSON file, which references manifest lists, which in turn reference manifest files that describe the actual data files. This design is what makes Iceberg so engine-agnostic. No single compute engine owns the table, so Trino, Apache Spark™, Flink, and others can all read and write the same Iceberg table without any coordination between them.

Delta Lake stores its transaction log as a sequence of JSON files in a _delta_log directory alongside the data. Each commit appends a new log entry, and periodic checkpoints compact the log for faster reads. Delta Lake was originally built within the Databricks ecosystem, and that heritage shows in the transaction log format: it’s simple and effective, but historically more tightly coupled to Spark as the primary write engine.

Apache Hudi takes a different approach entirely. It organizes data into two storage types: Copy-on-Write (CoW) tables, which rewrite Parquet files on every update, and Merge-on-Read (MoR) tables, which append delta log files and merge them at read time. Hudi also maintains its own index structures to enable fast record-level upserts, and that’s where it has historically had a real edge over the other two.

What are the key differences between Iceberg, Delta Lake, and Hudi?

The differences really come down to three things: governance model, write semantics, and ecosystem coupling. Iceberg prioritizes format neutrality and open governance. Delta Lake prioritizes tight integration with Spark-based workflows. Hudi prioritizes low-latency record-level mutations and incremental processing.

Governance: Apache Iceberg and Apache Hudi are both Apache Software Foundation projects with fully open specifications. Delta Lake is governed by the Linux Foundation, with the specification open but historically driven by one primary commercial contributor.
Write model: Iceberg uses optimistic concurrency control with snapshot isolation. Delta Lake uses optimistic concurrency with a serializable log. Hudi uses its own timeline-based concurrency model with index lookups for upserts.
Record-level updates: Hudi was purpose-built for upserts and deletes at the record level, making it the strongest fit for CDC (change data capture) pipelines. Iceberg and Delta Lake both support row-level deletes and updates, but they arrived at this capability later.
Metadata scalability: Iceberg’s manifest-based metadata scales well to tables with billions of partitions, which is one reason large-scale analytical workloads often favor it.

Which engines and tools are compatible with each format?

Engine compatibility varies quite a bit across the three formats, and in practice this is often what makes the decision for you.

Apache Iceberg has the broadest multi-engine support, including Trino, Apache Spark™, Apache Flink, Presto, Dremio, StarRocks, and many others. Its open specification and catalog abstraction, which supports REST, Hive, Glue, and Nessie catalogs, make it straightforward for any engine to implement a connector.

Delta Lake has first-class support in Apache Spark™ and is increasingly supported in other engines through the Delta Kernel library and connectors for Trino, Flink, and others. That said, the maturity and feature parity of those connectors can lag behind the Spark implementation.

Apache Hudi supports Apache Spark™ as its primary write engine, with read support in Trino, Presto, and Hive. Flink write support exists and has improved considerably, but Hudi’s connector ecosystem is narrower than Iceberg’s.

If your platform uses multiple query engines, which is common in modern data platform architectures, Iceberg’s engine-neutral design reduces integration friction considerably.

When should you use Apache Iceberg over Delta Lake or Hudi?

Iceberg is the right choice when format neutrality, multi-engine access, and long-term openness matter to you. It’s the strongest option when you don’t want any single vendor or engine to own your table format, when your platform runs more than one query engine, or when you’re building for scale with very large partition counts.

Iceberg also fits well in environments where data governance and catalog interoperability are important. The REST catalog standard and support for Project Nessie, a Git-like catalog with branching semantics, make Iceberg a natural fit for data mesh architectures where different teams manage their own data products independently.

If your workload is primarily analytical, batch-oriented, or involves large-scale joins and aggregations rather than continuous record-level updates, Iceberg’s Copy-on-Write model is efficient and well-supported across the widest range of tools.

When does Delta Lake or Apache Hudi make more sense?

Delta Lake makes the most sense when your team is already deeply invested in an Apache Spark™-centric workflow and wants tight, well-tested integration between compute and storage. Many teams find that Delta Lake’s straightforward log structure and mature Spark integration reduce operational overhead when Spark is the primary engine. It’s also a practical choice when you’re working within a managed Spark environment and want a format that’s well documented and widely understood.

Apache Hudi makes the most sense for streaming ingestion pipelines with high-frequency upserts and deletes, particularly when you’re running CDC workloads from transactional databases. Hudi’s MoR storage type lets you ingest data with low write amplification and merge at read time, a real advantage when records are updated frequently. If your pipeline needs to process incremental changes efficiently and maintain a near-real-time view of mutable data, Hudi’s timeline model and indexing capabilities are built exactly for that.

How do the three formats handle schema evolution and time travel?

All three formats support schema evolution and time travel, but they differ in how they implement these features and how safely they handle breaking changes.

Schema evolution

Apache Iceberg has the most comprehensive schema evolution support. It tracks column IDs independently of column names, which means renaming or reordering columns doesn’t break existing queries or require rewriting data files. Iceberg supports adding, renaming, reordering, widening, and dropping columns safely.

Delta Lake supports adding and renaming columns, with schema enforcement enabled by default. Schema evolution requires explicit configuration but is generally safe for additive changes. Apache Hudi supports schema evolution with some constraints depending on the storage type and the Avro schema compatibility settings in use.

Time travel

All three formats let you query historical snapshots of a table. Iceberg uses snapshot IDs and timestamps. Delta Lake uses version numbers and timestamps via its transaction log. Hudi uses its timeline, which records commits, compactions, and clean operations. The practical difference is mostly in how clean the experience is: Iceberg’s snapshot-based model makes it straightforward to query any prior state by ID or timestamp without ambiguity. How long historical snapshots are retained depends on your configuration in all three cases.

Which open table format is best for a Kubernetes-native data platform?

For a Kubernetes-native data platform, Apache Iceberg is generally the most practical choice. Its engine-agnostic design fits naturally with the modular, multi-tool architectures that Kubernetes enables, where different workloads use different compute engines and no single stack owns the entire pipeline.

On Kubernetes, you’re likely running multiple tools side by side: Trino for interactive queries, Apache Spark™ for batch processing, Apache Flink for streaming. Iceberg’s open specification means all of these can read and write the same tables without format-specific connectors that vary in maturity. The REST catalog standard also integrates cleanly with Kubernetes-native service discovery and access control patterns.

Delta Lake is a reasonable choice if Spark is your dominant engine, and Hudi remains strong for CDC-heavy streaming pipelines. But if you’re building a platform that needs to stay flexible as your tooling evolves, Iceberg’s format neutrality and broad ecosystem support make it the lowest-risk foundation. You can explore working demos of these kinds of architectures to see how they behave in practice.

How Stackable helps with open table format deployments

The Stackable Data Platform (SDP) is a modular, Kubernetes-native data platform that makes it straightforward to deploy and operate the tools that work with open table formats in production. Rather than locking you into a proprietary lakehouse stack, the SDP gives you the components to build your own, with full control over where your data lives and which engines access it.

Specifically, the SDP provides:

Kubernetes-native operators for Apache Spark™, Trino, Apache Flink, and other engines that read and write Apache Iceberg, Delta Lake, and Apache Hudi tables.
Infrastructure-as-code provisioning that lets you define your entire lakehouse stack, including catalogs, compute engines, and storage configuration, in version-controlled manifests.
No vendor lock-in: the SDP is 100% open source and cloud-agnostic, so your table format choice is not constrained by a managed service’s supported formats.
Data sovereignty by design: deploy on-premises, in any cloud, or in a hybrid environment, with your data staying where you decide it should stay.

If you’re evaluating which table format fits your architecture and want to understand how a Kubernetes-native platform can support that decision, get in touch with the Stackable team to talk through your specific setup.

Apache Kafka® is a registered trademark of The Apache Software Foundation. Apache Spark™ is a trademark of The Apache Software Foundation. Apache Iceberg, Apache Flink, Apache Hudi, Apache Hive™, and Apache Hadoop® are trademarks or registered trademarks of The Apache Software Foundation in the United States and/or other countries. Trino is a trademark of the Trino Software Foundation. All other trademarks are the property of their respective owners. Use of these names does not imply endorsement by The Apache Software Foundation or any other organization.

Apache Iceberg vs. Delta Lake vs. Apache Hudi: what’s the difference?