What are the performance benchmarks you should set before migrating a data platform?

Before you migrate a data platform, you should establish performance benchmarks across query latency, throughput, job completion times, resource utilization, and pipeline ingestion rates. These benchmarks form the baseline that tells you whether your new platform is actually an improvement or just a lateral move with a different set of problems. The sections below work through how to set those benchmarks, what thresholds to aim for, and where tools like the Stackable Data Platform (SDP) fit into the picture.

Which performance metrics matter most before a data platform migration?

The metrics that matter most before a data platform migration are query latency, job throughput, pipeline ingestion rates, resource utilization (CPU, memory, disk I/O), and time-to-result for your most critical workloads. These five categories cover the full spectrum of what a data platform actually does, and they give you a concrete comparison point once migration is complete.

Start with the workloads that are most visible to your users or most consequential to your business. If your analysts run interactive queries against a data warehouse, latency is your primary metric. If you run batch ETL jobs overnight, job completion time and throughput matter more. If you operate event streaming pipelines with Apache Kafka®, focus on consumer lag and end-to-end message latency.

Resource utilization tends to get underweighted at this stage, but it is critical for cost comparison. A new platform might deliver identical query performance while consuming significantly more compute. Without a utilization baseline, you will not catch that until the invoice arrives.

Query latency: p50, p95, and p99 response times for representative queries
Throughput: queries per second or jobs per hour under normal and peak load
Ingestion rate: records per second for streaming and batch pipelines
Resource utilization: CPU, memory, and disk I/O per workload type
Job completion time: wall-clock duration for scheduled batch jobs

How do you establish a reliable performance baseline before migrating?

To establish a reliable performance baseline before migrating, capture metrics from your current platform under realistic load conditions over a sustained period, typically two to four weeks. A single day of measurements is not enough to account for weekly batch cycles, month-end reporting spikes, or irregular data volumes. The baseline must reflect actual usage patterns, not ideal conditions.

Start by identifying your top 20 to 30 most-executed queries or job types. These are your benchmark workloads. Run them repeatedly under controlled conditions and record the full distribution of response times, not just the average. Averages hide tail latency, and tail latency is where user experience and SLA violations occur.

Document the infrastructure configuration alongside the performance data. Node counts, memory allocations, storage type, and network topology all affect results. A benchmark without its infrastructure context is just a number with no meaning in six months when someone tries to compare it to the new environment.

If your current platform lacks built-in observability, instrument it before you start. Prometheus with a Grafana dashboard is a practical choice for most environments. Collect data at the job level, not just the cluster level, so you can map specific workloads to specific resource consumption patterns.

What are acceptable performance thresholds for a successful migration?

Acceptable performance thresholds for a successful migration are workload-specific, but a widely used starting point is that the new platform should match or improve on the baseline at p95 latency, with no regression beyond 10 to 15% on any critical workload. That 10 to 15% tolerance accounts for environmental differences and early tuning gaps, but it should narrow to zero or better within the first operational cycle.

Defining “acceptable” requires input from the teams that own each workload. An analytics team might accept a 20% increase in batch job duration if interactive query latency improves significantly. A streaming team running near-real-time fraud detection has no tolerance for increased consumer lag whatsoever. These thresholds need to be written down and agreed upon before migration starts, not negotiated after the fact when someone is under pressure to declare success.

For resource utilization, the target is generally cost-neutral or better at equivalent performance. If the new platform requires significantly more infrastructure to match the existing one, that is a signal to investigate configuration or architecture before proceeding.

How does a Kubernetes-native platform change performance benchmark expectations?

A Kubernetes-native data platform changes benchmark expectations because resource allocation is dynamic rather than static. In a traditional deployment, nodes are sized and assigned to specific services. In a Kubernetes environment, pods compete for resources within defined limits and requests, which means performance characteristics vary based on cluster load, scheduling decisions, and resource quotas. Your benchmarks need to account for this variability.

The practical implication is that you should benchmark under realistic cluster load, not on an empty cluster. An isolated benchmark on a freshly provisioned Kubernetes cluster will produce results that do not reflect production behavior, where other workloads are co-located and the scheduler is making trade-offs in real time.

Kubernetes also enables horizontal scaling patterns that are difficult to replicate in traditional environments. A query engine like Trino can scale worker pods in response to query volume, which changes the shape of your latency curve at high concurrency. This is a genuine improvement, but it means your benchmark methodology needs to test across a range of concurrency levels, not just a single load scenario.

Finally, storage and networking behavior in Kubernetes can differ from bare-metal or VM deployments, particularly around persistent volume performance and pod-to-pod communication. Include storage I/O benchmarks explicitly when migrating workloads that are disk-bound.

What tools can you use to run pre-migration performance benchmarks?

For pre-migration performance benchmarks, the most practical tools are the TPC-DS or TPC-H benchmark suites for SQL workloads, Apache JMeter or k6 for load testing APIs and pipelines, and Prometheus with Grafana for infrastructure-level metric collection. The right combination depends on your workload mix.

SQL and analytical workloads

TPC-DS is the industry standard for data warehouse benchmarking. It covers a realistic mix of query types and scales to large data volumes. If you are migrating a Trino or Apache Druid™ deployment, running TPC-DS against both the old and new environments gives you a direct, reproducible comparison. Many teams also maintain a library of their own representative queries alongside TPC-DS, which captures workload-specific patterns that the standard suite does not.

Streaming and pipeline workloads

For Apache Kafka® pipelines, the built-in kafka-producer-perf-test and kafka-consumer-perf-test scripts provide a straightforward way to measure throughput and latency. For more complex pipeline benchmarks involving Apache Spark™ or batch ingestion, custom scripts that replay production-representative data volumes are more reliable than synthetic tests. The goal is to reproduce the actual data shape and access patterns, not just the volume.

Infrastructure observability

Prometheus is the standard for metric collection in Kubernetes environments. Pair it with Grafana for visualization and set up dashboards that track the five core metrics identified earlier. Export baseline snapshots before migration begins so you have a fixed reference point that cannot be overwritten by new data.

Should you benchmark in production or in a staging environment?

You should benchmark primarily in a staging environment that mirrors production as closely as possible, with production monitoring as a secondary validation layer. Benchmarking directly in production introduces risk and often produces inconsistent results because real user traffic creates noise that is difficult to separate from benchmark workloads.

The staging environment needs to be a genuine mirror, not a scaled-down approximation. This means matching node types, storage configuration, network topology, and data volumes as closely as budget and logistics allow. A staging environment running on different hardware or with a fraction of the data will produce numbers that do not transfer reliably to production.

That said, staging benchmarks alone are not sufficient. Production has workload patterns, data distributions, and concurrency levels that are genuinely difficult to replicate. Use production monitoring to validate staging benchmark results once migration is complete, and treat the first two to four weeks in production as a live benchmarking phase with close observation and rollback capability if thresholds are breached.

One practical approach is to run shadow benchmarks during migration: route a copy of production traffic to the new platform in parallel, compare results in real time, and use the delta to validate your staging findings before cutting over completely. This is more operationally complex but significantly reduces the risk of post-migration surprises.

How Stackable helps with data platform migration benchmarking

The SDP is built on Kubernetes and uses a declarative, infrastructure-as-code model, which makes it straightforward to create reproducible environments for pre-migration benchmarking. Because every component is defined in version-controlled configuration, you can spin up an identical staging environment, run your benchmark suite, and tear it down without manual configuration drift between runs.

Reproducible environments: Declarative configuration via Kubernetes operators means your staging benchmark environment is defined in code, not tribal knowledge. You can reproduce it exactly for retesting.
Integrated observability: The SDP includes Prometheus-compatible metrics for all managed components, including the Stackable Operator for Apache Kafka®, Trino, Apache Druid™, and Apache Spark™, so you get consistent metric collection across your full stack without additional instrumentation.
Modular workload testing: Because the SDP is modular, you can benchmark individual components in isolation before testing them as an integrated stack, which makes it easier to locate performance regressions.
Cloud-agnostic deployment: You can run benchmark environments on-premises or in any cloud, which means your staging environment can match your production infrastructure regardless of where it lives.
Data sovereignty preserved: Benchmark workloads run on your infrastructure, under your control. No data leaves your environment during the testing process.

If you are planning a migration and want to talk through how to set up a reproducible benchmark environment on the SDP, get in touch with the Stackable team to discuss your specific workload requirements.