Retrieval Augmented Generation with OpenSearch
Retrieval Augmented Generation with OpenSearch
This demo showcases a RAG pipeline that grounds large language model responses in a specific knowledge base using semantic search, entirely on Kubernetes (no GPU needed).
HOW TO START
With a Kubernetes cluster, run a single command:
Key Features
What this demo shows
Hybrid OpenSearch search
Combines k-NN vector similarity with BM25 keyword matching for retrieval accuracy that beats either approach alone.
768-dim embeddings
Uses nomic-embed-text for open-source, high-performance dense retrieval, pre-loaded from the Stackable docs.
Transparent retrieval
See exactly which documentation chunks informed each answer, including relevance scores and source URLs.
Local LLM inference
Runs Llama 3.1 8B entirely on-cluster via Ollama, with no external API calls and no data leaving your environment.
Interactive notebook
A pre-configured JupyterLab notebook walks through every step of the RAG pipeline end-to-end.
Extensible by design
Swap in your own document corpus, try different embedding or generation models, or tune the hybrid search weights.
Demo Components
The stack under the hood
OpenSearch
Search & vectors
Distributed search engine using the k-NN plugin for HNSW nearest-neighbor search plus BM25 rescoring in a single hybrid query. Stores ~4,200 documentation chunks with 768-dim embeddings.
OpenSearch Dashboards
Inspection UI
Web UI for inspecting indexed documents, browsing vector embeddings, and testing raw queries via the Dev Tools console.
Ollama
LLM runtime
Local LLM runtime loading two models on startup: nomic-embed-text:v1.5 (~274 MB) for embeddings and llama3.1:8b (~4.7 GB) for answer generation.
JupyterLab
Notebook server
Pre-configured notebook server with a complete RAG pipeline notebook downloaded via init container, covering setup, retrieval, context formatting and streamed generation.
Demo Workflow
From install to first answer
Installation
Deploy the demo on your existing Kubernetes cluster with a single stackablectl command. Operators and all data products are bootstrapped automatically.
Embeddings loaded automatically
A Kubernetes Job downloads pre-generated embeddings (~89 MB, ~4,200 chunks) from GitHub and indexes them into OpenSearch with k-NN vector mappings.
Access JupyterLab
Port-forward and open the notebook at localhost:8888. It verifies connectivity, shows document counts, and guides you step by step.
Explore RAG queries
Ask questions about the Stackable documentation. Observe product detection, retrieved chunks with scores, and the streamed LLM answer, all within the notebook.
Inspect in Dashboards
Browse the rag-documents index in OpenSearch Dashboards to see vectors and document metadata.
Ready to run it yourself?
Spin up the full RAG pipeline on your own Kubernetes cluster, or talk to us about your use case.