Configuration

LocalRAG is configured with environment variables, read once when the server starts. To change a setting, set the variable and restart LocalRAG.

Environment variables

Variable	Default	What it sets
LOCALRAG_HOST	127.0.0.1	The network address the server listens on. 127.0.0.1 means only this computer.
LOCALRAG_PORT	8090	The port the server listens on.
LOCALRAG_DATA_DIR	./data	The folder for the vector store and the document list.
LOCALRAG_OLLAMA_URL	http://localhost:11434	The address of the Ollama server.
LOCALRAG_MODEL	llama3.2	The Ollama model that writes answers.
LOCALRAG_EMBED_MODEL	BAAI/bge-small-en-v1.5	The sentence-transformers model that makes embeddings.
LOCALRAG_TOP_K	5	How many passages to retrieve for each question, if the request does not say.
LOCALRAG_SCORE_THRESHOLD	0.30	The lowest similarity score a passage needs to be used.
LOCALRAG_CHUNK_TOKENS	500	The largest chunk, in tokens.
LOCALRAG_CHUNK_OVERLAP	50	About how many tokens neighbouring chunks on the same page share.

A value that is not a number where a number is expected stops the server at start-up with a message such as invalid integer value for LOCALRAG_PORT.

Set a variable with pip

Put the variable in front of the command for one run:

LOCALRAG_PORT=9000 LOCALRAG_MODEL=mistral localrag serve

Or export it for the whole terminal session:

export LOCALRAG_DATA_DIR="$HOME/localrag-data"
localrag serve

Set a variable with Docker

Add it under environment: for the localrag service in docker-compose.yml, then run docker compose up -d again:

  localrag:
    environment:
      - LOCALRAG_OLLAMA_URL=http://ollama:11434
      - LOCALRAG_MODEL=mistral
      - LOCALRAG_SCORE_THRESHOLD=0.25

The Docker image already sets LOCALRAG_HOST=0.0.0.0 and LOCALRAG_DATA_DIR=/data, and the Compose file points LOCALRAG_OLLAMA_URL at the ollama container.

Tuning retrieval

If you see	Try
Too many "I don't know" answers to questions your documents do answer	Lower LOCALRAG_SCORE_THRESHOLD a little, for example to 0.25.
Answers that drift into unrelated passages	Raise LOCALRAG_SCORE_THRESHOLD, or lower LOCALRAG_TOP_K.
Answers that miss details spread over many places	Raise LOCALRAG_TOP_K.

LOCALRAG_CHUNK_TOKENS and LOCALRAG_CHUNK_OVERLAP only affect documents added after the change. To apply new values to existing documents, delete them and add them again.

Command-line commands

Command	What it does
localrag serve	Starts the web server and API with the settings above.
localrag ingest PATH	Adds a file, or every .pdf, .txt and .md file in a folder and its subfolders. Prints a line per file and a summary, and exits with status 1 if nothing was added.

Where data is stored

Install	Location
pip	The folder in LOCALRAG_DATA_DIR, by default data in the folder where you start LocalRAG
Docker	The localrag-data volume, mounted at /data in the container

Inside the data folder, docs.json lists the documents and chroma/ holds the vector store. In Docker, the downloaded embedding model is kept there too, under hf/. Ollama models are kept by Ollama, in the ollama volume under Docker.

To start again with no documents, stop LocalRAG and delete the data folder, or run docker compose down -v (which also deletes the Ollama models).

Next steps

• Choose models
• Use the API
• Troubleshooting and FAQ