Ollama Chat

A local RAG-powered chat app — Flutter frontend + FastAPI backend with local LLM inference via Ollama and a hybrid retrieval pipeline over your own HTML documents.

Architecture

┌─────────────────────────────────────────────┐
│  Docker Compose                              │
│                                             │
│  frontend (nginx :3000) ──▶ backend (:8000) │
│                                    │        │
└────────────────────────────────────┼────────┘
                                     │ host.docker.internal:11434
                           Ollama (native, Metal GPU)

• Frontend — Flutter web, served by nginx, proxies API calls to the backend
• Backend — FastAPI with SSE streaming, ChromaDB vector store, BM25 hybrid search
• Ollama — runs natively on the host (GPU acceleration); containers reach it via host.docker.internal

Quick Start

Prerequisites: Docker Desktop, Ollama

git clone <repo> && cd ollama-chat
make setup   # copies .env, pulls Ollama models (~5GB, one-time)
make run     # builds and starts both containers
open http://localhost:3000

The app comes with sample HTML documents in backend/data/sample/ so it works out of the box. To use your own content, drop HTML files into that directory (or point DATA_PATH elsewhere) and restart the backend to re-index.

Make Commands

Command	Description
make setup	First-time setup: copy .env, pull Ollama models
make run	Start backend + frontend via Docker Compose
make stop	Stop all containers
make logs	Tail container logs
make build	Rebuild images without cache
make dev	Run backend natively with hot reload
make dev-web	Run Flutter web in Chrome (hot reload, connects to local backend)
make test	Run all tests — backend unit (90) + frontend (67)
make test-backend	Backend unit tests only — fully offline, ~1s
make test-frontend	Flutter widget tests only — fully offline, ~3s
make e2e	End-to-end tests in visible macOS window against the live backend
make lock	Regenerate backend/requirements.lock via pip-compile
make clean	Remove containers and local images

Development Workflow

Backend changes — skip Docker, use hot reload:

make dev
# Edit any .py file → instant reload

Flutter web changes — hot reload in Chrome:

make dev-web
# Connects to localhost:8000 directly

Flutter iOS:

cd frontend && flutter run -d "iPhone 17 Pro Max" --no-enable-impeller

Testing

make test           # all offline tests (backend unit + Flutter widgets, ~5s)
make test-backend   # 90 backend unit tests: RAG pipeline, API, config, rate limiter
                    # +12 integration tests (auto-skipped unless OLLAMA_URL is set)
make test-frontend  # 67 Flutter widget tests: model, services, ChatScreen flows

To run the integration tests against a live Ollama instance:

OLLAMA_URL=http://localhost:11434 make test-backend

End-to-end tests drive the real app in a desktop window against the live backend:

make dev &          # start backend
make e2e            # opens desktop window, runs 7 full-stack scenarios

Configuration

Copy .env.example to .env (done automatically by make setup) and adjust as needed. Key variables:

Variable	Default	Description
OLLAMA_URL	http://host.docker.internal:11434	Ollama endpoint (Docker)
MODEL	llama3.1:8b	Chat model
EMBED_MODEL	nomic-embed-text	Embedding model
DATA_PATH	./data/sample	Directory of HTML files to index
CORS_ORIGINS	http://localhost:3000,...	Comma-separated allowed origins

See backend/README.md for the full variable reference, including all RAG tuning parameters.

Using Your Own Data

1. Drop HTML files into backend/data/sample/ (or set DATA_PATH to any directory)
2. Delete backend/chroma_db/ to clear the existing index
3. Restart the backend — it re-indexes on startup

The RAG pipeline extracts text from <body>, strips navigation/footer boilerplate, chunks content into ~800-character segments, and builds a hybrid ChromaDB + BM25 index.

Docs

• backend/README.md — API reference, RAG pipeline, backend setup
• frontend/README.md — Flutter setup, builds, configuration
• docs/rag-hyperparameter-tuning.md — RAG parameter guide, profiles, and diagnostics
• docs/adr/ — Architecture Decision Records (ChromaDB, hybrid retrieval, Flutter)