Stackable and JupyterHub
Stackable and JupyterHub
A multi-user data science environment: JupyterHub with Keycloak single sign-on, spinning up Apache Spark jobs straight from a notebook against data in HDFS.
HOW TO START
With a Kubernetes cluster, run a single command:
Key Features
What this demo shows
Multi-user notebooks
Per-user JupyterLab servers spun up on demand by JupyterHub.
Notebook-native Spark
Launch distributed Spark jobs from inside a notebook.
Reproducible images
Pinned, curated container images for every user session.
Single sign-on
Keycloak-backed OIDC login, with no shared credentials.
HDFS-backed storage
Read and write large datasets on a real Hadoop file system.
Isolated resources
Per-user CPU and memory limits keep the cluster fair.
Demo Components
The stack under the hood
JupyterHub
Notebook hub
Provisions isolated single-user JupyterLab servers and manages their lifecycle.
Keycloak
Identity
OIDC identity provider handling authentication and user federation.
Apache Spark
Compute
Distributed processing launched from notebooks via the Stackable Spark operator.
Apache HDFS
Storage
Distributed file system holding the demo datasets.
Demo Workflow
From login to distributed Spark job
Installation
One stackablectl command installs JupyterHub, Keycloak, Spark and HDFS.
Sign in via Keycloak
Log in with SSO; JupyterHub provisions your personal notebook server.
Launch a notebook
Open JupyterLab and pick a curated, reproducible image.
Submit a Spark job
Kick off a distributed Spark job straight from a notebook cell.
Read the results
Load the job output back from HDFS and explore it in the notebook.
Ready to run it yourself?
Give your data scientists governed, self-service notebooks backed by Spark and HDFS.