LocalRAG
LocalRAG answers questions about your own documents. You give it PDF, text and Markdown files, ask a question in your browser, and get an answer with the document name and page it came from. Everything runs on your own computer: no cloud service and no API keys.
What RAG means
RAG (retrieval-augmented generation) means that before a language model answers, the system first searches your documents for the passages that best match the question and gives only those passages to the model. The model then writes its answer from that text, which keeps the answer tied to your documents and lets the system show where each part came from.
Who it is for
- People with a pile of manuals, notes, papers or policies who want to ask questions instead of searching by hand.
- Teams that cannot send documents to an online service.
- Developers who want a small, local question-answering API to build on.
What it does
| Feature | What you get |
|---|---|
| Documents in | Upload PDF, TXT and Markdown files in the browser, by API or from the command line. The same file twice is stored once. |
| Cited answers out | Each answer streams in word by word, followed by chips naming the document and page it used. |
| Honest "I don't know" | If nothing in your documents matches the question well enough, LocalRAG says so instead of guessing. |
| Local models | Embeddings with BAAI/bge-small-en-v1.5, answers from a model you run in Ollama, llama3.2 by default. |
| Web interface and API | A single-page web UI and a small HTTP API with streaming responses. |
How it works
When you add a document, LocalRAG pulls out the text page by page, cuts it into chunks, turns each chunk into an embedding (a list of numbers that captures its meaning) and stores them in a local vector database, ChromaDB.
When you ask a question, LocalRAG turns the question into an embedding too, finds the chunks closest to it, and asks the Ollama model to answer using only those chunks. If no chunk is close enough, the model is not asked at all.
Platforms
LocalRAG runs on Linux, on x86-64 and arm64. You run it with Docker Compose, which also starts Ollama for you, or install it with pip next to an Ollama you already have. See Install.
Privacy
After two one-time downloads, the embedding model and the Ollama model, LocalRAG makes no network calls. Your documents, their embeddings and your questions stay on your machine.
Licence and source
LocalRAG is open source under the MIT licence. The source code and the issue tracker are on GitHub.