Stackable Operator for Apache NiFi
Apache NiFi™ on the
Stackable Data Platform
Run NiFi natively on Kubernetes
Move data streams visually: import, transform, export
Getting data from where it is to where it needs to be – enriched, filtered, routed, with nothing lost on the way – is its own discipline. Apache NiFi makes those data flows visual and traceable, and the Stackable Operator brings Kubernetes-native automation to deploying, scaling and managing NiFi clusters. Role-based node groups, integrated security and compliance checks, with you in control of the infrastructure. Run it on-prem, air-gapped or in a sovereign cloud – your data stays yours.
Why Apache NiFi in the Stackable Data Platform (SDP)?
Apache NiFi is a reliable open-source data integration tool for automating the movement and transformation of data across systems. Its intuitive, web-based interface lets users design complex flows visually, while a scalable architecture keeps performance steady across different environments. In the data platform, NiFi is the central hub for ingestion and distribution, connecting directly with Kafka, Spark and Trino.
Flow-based programming, visually
Design data pipelines on a visual canvas, simplifying complex workflows and reducing the room for error.
Runtime flow changes
Adjust flows on the fly through the web interface, stopping and starting individual processors as needed – without restarting the cluster.
Guaranteed delivery & provenance
Configurable quality-of-service, guaranteed delivery and back-pressure handling, plus full data-provenance tracking for complete visibility into lineage and transformations.
Extensible, wide protocol support
A broad range of protocols and formats, with custom processors for requirements that don’t fit off the shelf.
Enhanced security & access control
TLS encryption, user authentication and fine-grained access control, with LDAP and OIDC integration.
Comprehensive monitoring
Built-in flow health and performance metrics, with Prometheus integration for advanced metrics collection.
What does the Stackable Operator add for Apache NiFi?
NiFi as your ingestion hub
Pre-configured connectors for Kafka, S3/MinIO, databases route data across the platform without custom code. NiFi feeds the rest of your stack.
Land data straight into Iceberg lakehouses
Direct Apache Iceberg integration, including schema evolution and time-travel queries. Flows write straight into open lakehouse table formats.
Scale the UI and workers separately
Scale NiFi’s UI and worker nodes independently by role, matching each to its load.
What do all Stackable operators have in common?
One operator framework (Rust)
Every operator follows the same CRD pattern with Roles, RoleGroups and ConfigOverrides. Learn one operator, and you find your way around the next one instantly.
Infrastructure-as-Code
Every data app is a YAML CRD that lives in Git – reviewable, lintable and CI/CD-ready. The same definition works on Dev, Test and Prod.
Lifecycle management (Day-2 operations)
Deployment, restarts, certificate rotation and rolling upgrades are handled automatically – including pod restarts when configs or secrets change. And because you need to see what’s running, monitoring and logging come built in: Prometheus, Vector and OpenTelemetry feed ready-made Grafana dashboards – one observability setup for the whole platform.
Security by default
TLS, Kerberos for HDFS, OIDC login and OPA-based fine-grained authorization, all configurable per CRD. Daily vulnerability scans, SBOMs and cosign-signed images keep the whole platform patched – so you don’t track a dozen upstream projects yourself.
Kubernetes-native and modular
Runs on-prem, in any cloud or on a laptop, with no vendor lock-in – and explicit air-gapped support. Install only the operators you need, and add more later without touching the rest.
OpenShift-certified
All operators are Red Hat–certified and install directly from the Red Hat Certified Operators Catalog (OperatorHub) – compatible with Security Context Constraints and RBAC. Certified operation via a Stackable subscription.
Use Cases for NiFi
Real-time data ingestion
Ingest from diverse sources – databases, APIs, IoT devices, messaging systems. With Stackable, clusters scale easily as data volumes grow, for continuous, reliable ingestion.
Data transformation & data routing
Enrich, filter and route data to the right destination – S3/MinIO for long-term storage, Kafka for streaming, OpenSearch for real-time search. Stackable keeps these pipelines stable and easy to manage.
Edge & IoT data flows
Ingest, process and route data from IoT and edge devices across manufacturing, energy and telecommunications. The operator supports secure connectivity to distributed devices and direct integration with engines like Spark or Trino.
FAQ - Frequently Asked Questions about NiFi and Stackable
You can deploy the operator via stackablectl, which automates installation, configuration and dependency management, or via Helm if you prefer traditional Kubernetes tooling. After installation, create a NifiCluster CRD to define your cluster declaratively. This standardizes deployments, reduces manual errors, and lets you version-control your infrastructure like code. On OpenShift, the operator can be installed via the Red Hat Certified Operators Catalog.
The supported Apache NiFi versions – covering both 1.x and 2.x lines – are listed on the Stackable documentation’s supported-versions page, updated each release. Pick the version that fits your existing workflows by setting the image in your NifiCluster, or use a custom image for experimental setups.
Click here for the current list.
Scaling is handled declaratively by adjusting the replica count per role-group in the NifiCluster CRD. The operator automatically provisions new pods, updates configurations and ensures that services remain consistent during scale-up or scale-down events. This accommodates higher data volumes or more concurrent flows without downtime for the rest of the cluster (except for NiFi 1.x upgrades, which may require a restart).
NiFi 2.x supports rolling upgrades, with nodes updated sequentially without stopping the entire cluster – the operator handles pod restarts, configuration updates and flow-state migration, minimizing disruption to running workloads. NiFi 1.x requires a cluster restart for updates, as leader election and internal state management necessitate full cluster cycling.
Security is layered: SBOM verification and signed container images guarantee trustworthy, auditable deployments. TLS encryption secures communication between nodes and clients. Integration with OIDC/LDAP lets organizations enforce enterprise authentication standards, and optional OPA policies enable fine-grained, policy-based access control. Together, this makes NiFi deployments secure and suitable for regulated environments.
Yes. Stackable provides SBOM (Software Bill of Materials) verification, signed container images and support for air-gapped deployments, ensuring that clusters can operate securely without external internet access. This is crucial for highly regulated industries or environments with strict network isolation policies.
The operator has been tested on major managed and self-hosted Kubernetes platforms: EKS, AKS, GKE, OpenShift, IONOS and K3s. This flexibility lets you deploy NiFi reliably across cloud providers or on-premise infrastructure, while still benefiting from declarative management and operator automation.
Connections are configured through NiFi processors in the NiFi web UI. The operator simplifies deploying clusters with preconfigured connectors for Kafka, S3/MinIO, databases and other systems. This makes it easy to integrate NiFi into an existing data ecosystem without writing custom code – while keeping central control over your flows.
Absolutely. The operator allows you to override almost all NiFi configuration settings via the CRD, including JVM options, cluster properties, processor defaults and security settings. This makes it possible to tailor deployments to specific performance, compliance or operational requirements.
Yes. NiFi exposes Prometheus metrics for all cluster components. These can be consumed by existing monitoring and alerting systems to track processor throughput, queue sizes, JVM performance and overall cluster health. Stackable also documents best practices for integrating metrics with Grafana dashboards for real-time observability.
Yes. You can define multiple NifiCluster CRDs in the same Kubernetes namespace or across namespaces. Each cluster runs independently, allowing isolation between environments (e.g. dev, staging, production) or multi-tenant architectures while using the same operator.
Resources - Learn how to use the Stackable Operator for Apache NiFi
Getting Started Guide
A step-by-step introduction to deploying and running Apache NiFi with the Stackable Operator. Perfect for users new to NiFi on Kubernetes – covers installation, basic configuration and first data flows.
Technical Documentation
Comprehensive technical documentation detailing CRD fields, configuration options, security setup and advanced use cases. Essential for production deployments and fine-tuning NiFi clusters.
GitHub Repository
The official open-source repository containing the NiFi Operator source code, Helm charts and examples. Ideal for developers and operators who want to explore configurations, contribute or automate deployments.
Demo
The nifi-kafka-druid-water-level-data demo shows a NiFi workflow that fetches water-level measurements from stations across Germany in real time via a public API and feeds them into Kafka. You can watch the ingestion pipeline live in the NiFi UI. Deployable with a single stackablectl command.
Subscribe to our Newsletter
With the Stackable newsletter, you’ll always stay up to date on the latest from Stackable!
Newsletter
Subscribe to the newsletter
With the Stackable newsletter you’ll always be up to date when it comes to updates around Stackable!