Self-Hosted AI RAG Agent System using n8n, Qdrant & Ollama

## 📌 Overview

An end-to-end, privacy-focused Retrieval-Augmented Generation (RAG) agent that runs 100% locally on system hardware. The system ingests external documents into a local vector database and enables an AI agent to perform contextual document search—with zero cloud costs and zero data leaving the local machine.

 

---

 

## 🛠️ Tech Stack & Architecture

- **Workflow Orchestration:** n8n (Docker containerized)

- **Vector Database:** Qdrant Vector Store

- **Local LLM Engine:** Ollama (`llama3.2` for logic, `nomic-embed-text` for vector embeddings)

- **Container Infrastructure:** Docker Compose (with GPU acceleration)

- **Exposing Endpoints:** ngrok tunneling

 

---

 

## ⚙️ Key Technical Features & Workflow Breakdown

 

1. **Document Ingestion & Indexing Pipeline:**

  - Ingests text documents via default data loaders.

  - Splits content into uniform chunks using a `Token Splitter`.

  - Generates vector embeddings via local Ollama embedding models (`nomic-embed-text`).

  - Stores and indexes embeddings inside `Qdrant` vector memory.

 

2. **AI Agent Query & Retrieval Pipeline:**

  - Connects an interactive `AI Agent` powered by `llama3.2`.

  - Integrates `Qdrant Vector Store` directly as a tool available to the agent.

  - Dynamically searches indexed knowledge to answer complex technical queries accurately based on context.

 

---

 

## 🟢 Results & Key Takeaways

- **100% Local Processing:** Full data privacy achieved without sending data to third-party LLM APIs.

- **Cost Efficiency:** Zero API overhead or recurring subscription costs.

- **Modular Design:** Scalable pipeline adaptable to multiple document types or larger local LLM models.

24 Jul 2026

Keywords
n8n automation
automation
automation specialist
n8n workflows
make
zapier
worflows
AI Usage

To build this local RAG system, AI was integrated directly into both the core system architecture and the developer workflow: AI System Execution: Used Ollama (nomic-embed-text) to locally process and convert text documents into 1024-dimensional vector embeddings for indexing in Qdrant. Deployed Ollama (llama3.2) as the central reasoning engine for the n8n AI Agent, enabling offline semantic retrieval and contextual question answering. Manual Engineering & Human Decisions: Designed the dual-pipeline architecture (Ingestion vs. Retrieval) manually in n8n. Configured Docker Compose network rules, volume persistence (n8n_data), and GPU acceleration (gpu-nvidia). Set up ngrok tunneling for secure webhook access and managed node credential routing (http://ollama:11434 and http://qdrant:6333).

AI Tool Stack
n8n ollama Qdrant