Stackable Docs Hub

Stackable

Stackable

How does Trino let you query data across multiple sources without moving it?

Isometric hexagonal cubes in crimson and steel-blue plus formation with floating database, cloud, server, and padlock icons on white background.

Trino lets you query data across multiple sources without moving it. It works as a distributed SQL query engine that pushes computation down to where the data already lives. Instead of copying datasets into a central store, Trino connects directly to each source, translates your SQL into source-specific operations, and assembles the results in memory. At the end, you’ll find a section on how Stackable helps you run Trino in a production-grade, Kubernetes-native environment.

How does Trino connect to different data sources simultaneously?

Trino connects to different data sources through a connector architecture. Each connector is a plugin that translates between Trino’s internal query model and the APIs, protocols, or storage formats of a specific system. When you configure a connector, Trino treats that system as a catalog, making its tables queryable through standard SQL alongside every other configured catalog.

When a query references multiple catalogs, Trino’s coordinator identifies which connectors are involved and opens parallel connections to each. The connectors handle authentication, schema discovery, and predicate pushdown independently. From the query writer’s perspective, the sources look like ordinary schemas in a single SQL namespace, even if they’re running on completely different infrastructure.

This design means adding a new data source is a configuration change, not an engineering project. You define the connector type, provide connection credentials, and the source becomes queryable immediately.

What is federated querying and how does it differ from ETL?

Federated querying means running a single SQL query that reads from multiple, physically separate data systems and returns a unified result, without consolidating the data in one place first. The query engine coordinates reads across sources at query time rather than at ingestion time.

ETL takes the opposite approach. Data is physically moved from source systems into a central destination, usually a data warehouse, before any analysis happens. This produces a stable, pre-integrated dataset, but it introduces pipeline latency, storage duplication, and ongoing maintenance work every time a source schema changes.

Federated querying trades some of that stability for freshness and flexibility. Because no data is copied, results reflect the current state of each source system. There’s no pipeline to break when an upstream schema evolves, and there’s no secondary copy of sensitive data to govern separately. The trade-off is that query performance depends on the responsiveness of the underlying sources, whereas a warehouse query hits pre-loaded, often pre-aggregated data.

For teams working across many source systems with heterogeneous ownership, federated data solutions often reduce the coordination overhead that centralized ETL pipelines require.

How does Trino execute a query that spans multiple data systems?

When Trino receives a cross-source query, the coordinator parses the SQL, builds a logical query plan, and splits that plan into stages that can execute in parallel across the cluster. Each stage is broken into tasks, and each task is assigned to a worker node. Workers assigned to a given source use that source’s connector to read only the data their task requires.

Trino applies predicate pushdown wherever the connector supports it. If your query filters on a date column, Trino instructs the connector to apply that filter at the source, so only matching rows travel across the network. For selective queries, this cuts data transfer significantly.

After each source has returned its portion of data, Trino performs joins, aggregations, and sorting in memory across the worker nodes using a pipeline execution model. Results stream back to the coordinator and then to the client as they become available, rather than waiting for the entire dataset to materialize. This keeps memory usage manageable even for large result sets.

What data sources can Trino query out of the box?

Trino ships with connectors for a wide range of storage systems and databases. Out of the box, you can query object stores like Amazon S3, Google Cloud Storage, and Azure Blob Storage using file formats such as Parquet, ORC, and Avro. Trino also includes connectors for relational databases including PostgreSQL, MySQL, and Microsoft SQL Server, as well as connectors for Apache Kafka® for streaming data, Apache Hive™ metastore-compatible catalogs, and the Apache Iceberg table format.

Beyond these bundled connectors, the Trino connector API is open, so the community and vendors have built additional connectors for systems like Elasticsearch, Cassandra, MongoDB, and various cloud-native data services. If a connector doesn’t exist for a niche system, the API provides a defined interface to build one.

In practice, most enterprise data estates span relational databases, object storage, and streaming systems. Trino can query all of these without custom integration work for the most common sources.

Why doesn’t Trino store or cache the data it queries?

Trino is designed as a stateless query engine. It has no persistent storage layer because its purpose is to query data where it already resides, not to become another place where data lives. This keeps the architecture simple: Trino nodes can be added or removed without migrating stored data, and there’s no secondary copy of your data to keep synchronized or secured separately.

The absence of persistent storage also means Trino does not own your data. Governance, access control, and retention policies stay with the source systems, which is often exactly what data sovereignty requirements demand. You’re not delegating custody of sensitive records to the query layer.

Trino does use in-memory buffers during query execution for intermediate results, joins, and aggregations, but this memory is released when the query completes. Some deployments add an optional result cache at the coordinator level for repeated identical queries, but this is a performance optimization, not a data store, and it’s not enabled by default.

How does Trino handle performance when querying large distributed datasets?

Trino addresses performance across large distributed datasets through massively parallel execution, predicate pushdown, and cost-based query optimization. The coordinator analyzes the query plan and estimates the cost of different execution strategies before dispatching work, choosing join orderings and partition strategies that minimize data movement between workers.

For queries over large object storage datasets, partition pruning is the most impactful optimization. If your data is partitioned by date and your query filters on a date range, Trino reads only the relevant partitions rather than scanning the entire dataset. Combined with columnar file formats like Parquet or ORC, which allow Trino to skip columns and row groups that don’t satisfy filter conditions, this can reduce the volume of data read by orders of magnitude.

For cross-source joins, Trino uses broadcast joins for small tables and distributed hash joins for large ones. The cost-based optimizer uses table statistics, where available, to choose between these strategies automatically. You can also influence execution with session properties and query hints when you have specific knowledge about your data distribution that the optimizer cannot infer.

Performance ultimately depends on the source systems as well. A connector can only push down what the underlying system supports. For latency-sensitive workloads, pre-aggregated or materialized views in the source systems remain a valid complement to federated querying rather than a replacement for it.

When should you use Trino instead of moving data to a warehouse?

Trino is the better choice when you need to query data that is too fresh, too sensitive, or too distributed to centralize practically. If your analysis requires joining data that is updated continuously, copying it to a warehouse introduces lag that may make the results stale by the time the pipeline completes. Trino reads directly from the source, so results reflect the current state.

Trino also suits situations where data cannot or should not be copied. Regulatory constraints, data residency requirements, or contractual agreements may prohibit replicating certain datasets. Federated querying lets you analyze that data in place without creating a secondary copy that triggers additional compliance obligations.

A centralized warehouse remains the stronger choice for high-frequency analytical queries over stable, historical datasets where pre-aggregation and indexing provide significant performance advantages. Many production environments use both: a warehouse for curated, high-traffic reporting workloads and Trino for exploratory queries, operational data access, and cross-domain analysis that spans systems the warehouse doesn’t ingest.

The decision comes down to whether the cost of moving and maintaining data is justified by the query performance gains. When the data is too diverse, too sensitive, or too dynamic to centralize, Trino’s distributed query approach fits naturally.

How Stackable helps you run Trino

The Stackable Data Platform (SDP) includes a Kubernetes-native operator for Trino that handles the full lifecycle of a Trino deployment, from initial provisioning to configuration management, scaling, and version upgrades. Rather than maintaining custom Helm charts or manual cluster configurations, you describe your desired Trino setup declaratively and the operator reconciles the running state to match.

  • Connector configuration as code: Catalog definitions are managed through Kubernetes custom resources, making connector setup reproducible and version-controlled alongside the rest of your infrastructure.
  • Integration with the broader SDP stack: Trino in the SDP works alongside Apache Kafka®, Apache Spark™, and Open Policy Agent out of the box, so you can build a complete data lakehouse or federated query layer without assembling integrations manually.
  • Security by default: The operator configures TLS between Trino components and integrates with your existing identity provider for authentication, so you’re not starting from an open configuration and hardening it afterward.
  • Cloud-agnostic deployment: Because the SDP runs on Kubernetes, your Trino cluster runs on-premises, in any cloud, or in a hybrid environment without changes to the operator configuration.

If you want to see how Trino fits into a complete open-source data platform, talk to the Stackable team about your setup and requirements.

Related Articles

Comments are closed.