chatui

A ChatGPT-like web UI on top of vllm-router: login, conversation history, streaming responses, model picker, a checklists product, and tool calling - built-in tools plus MCP servers an admin configures, both off until allowed per user. Phone-call-specific tooling drawing on fw/ochat/reporter is still a later addition.

Accounts are admin-provisioned only - no public signup. Every chatui user gets their own vllm-router user, provisioned automatically at creation time via vllm-router's POST /admin/users, so vllm-router's /usage tracking works per real person for free.

Backend (backend/)

cd backend
uv venv --python 3.13 --seed
. activate_env.inc
uv sync
cp .env.example .env

Fill in .env:

• VLLM_ROUTER_URL - where vllm-router is running (default http://localhost:8001).
• VLLM_ROUTER_ADMIN_KEY - a vllm-router admin API key (see vllm-router/add_user.py).
• CHATUI_JWT_SECRET - random secret for signing session cookies, e.g. python3 -c "import secrets; print(secrets.token_urlsafe(32))".
• CHATUI_TIMEZONE (optional) - the timezone reported to the model in every system message, e.g. Europe/Kyiv. Unset falls back to the server's own.

Bootstrap the first admin:

python3 add_user.py alice --role admin

Prints a generated login password - save it, it's shown only once. Admins can add more users afterward via POST /admin/users (same shape as vllm-router's).

In production the backend runs as a systemd service (backend/chatui-backend.service, installed to /etc/systemd/system/), so it survives reboots and crashes:

sudo cp backend/chatui-backend.service /etc/systemd/system/   # once
sudo systemctl daemon-reload
sudo systemctl enable --now chatui-backend

systemctl status chatui-backend
sudo systemctl restart chatui-backend

It still logs to backend/logs/backend.log, and is ordered after vllm-router.service (which it calls) so a boot brings the two up in the right order.

For local debugging without systemd: bash serve.sh fg (foreground) or bash serve.sh (background, port 8002) - but not while the service is running, since port 8002 is already bound.

Frontend (frontend/)

cd frontend
npm install
npm run dev     # dev server with hot reload

Serves on port 3000, proxying /api/* to the backend (port 8002) via Next.js rewrites - no CORS setup needed. Both dev and start bind 127.0.0.1: nginx is what publishes this, at https://chatui.kibermova.com and at the http://100.97.153.111:3000 tailnet URL it always answered at. See deploy/README.md for that half.

In production it runs the next build output under systemd (frontend/chatui-frontend.service):

sudo cp frontend/chatui-frontend.service /etc/systemd/system/   # once
sudo systemctl daemon-reload
sudo systemctl enable --now chatui-frontend

After changing frontend code, rebuild and restart - next start serves whatever is in .next/, it does not pick up source changes:

cd frontend && npm run build && sudo systemctl restart chatui-frontend

Logs go to frontend/logs/frontend.log. The unit hardcodes the nvm node path (~/.nvm/versions/node/v24.18.0/bin/node), so a node upgrade means editing ExecStart/PATH in the unit - otherwise it fails with status=203/EXEC.

Architecture

• backend/db.py - SQLite (chatui.db, gitignored): users, conversations, messages, attachments, the checklist_session* tables, user_limits, user_prompts, user_prompt_settings, plus mcp_servers, user_tools and message_tool_calls.
• backend/vllm_client.py - talks to vllm-router: list models, stream chat completions, provision new vllm-router users.
• backend/auth.py - chatui's own browser session auth (JWT in an httpOnly cookie) - a separate concern from vllm-router's Bearer-token API auth. Also what a user may change about themselves: their password (the current one has to be given) and their personal details (real name, email - both optional, both only ever seen by them and an admin with DB access).
• backend/permissions.py - per-user limits for capabilities that are off by default: custom_prompts (how many of their own system prompts a user may keep, 0 up to 20), tools and mcp (yes/no), and one per product (product_chats, product_checklists). Stored as rows, one per user+limit, so a later capability is a new name here and no schema change.
• backend/prompt_templates.py - the built-in system prompts offered in Settings; product content, so code and not DB rows. time_context() puts the current date and time at the top of every system message, so the model never has to guess (or extrapolate) what 'now' is. A user's own prompts live in user_prompts, and user_prompt_settings records which one is active.
• backend/tools.py - the built-in tools a model may call: current_time (for another timezone or a to-the-second reading), calculate, and web_search / fetch_url.
• backend/websearch.py - the search and page-reading behind those two: which engine to ask, how HTML becomes text, and which addresses are refused.
• backend/mcp_client.py - a small MCP client over the Streamable HTTP transport. Admins add servers by URL; a server's token is named, not stored - put MCP_FOO_KEY=... in .env and record MCP_FOO_KEY against the server.
• backend/agent.py - the model -> tool -> model loop, bounded by MAX_ROUNDS. A user gets a tool only if an admin allowed the capability (tools / mcp) and the user switched that tool on in Settings -> Tools.
• backend/app.py - FastAPI routes: auth, conversations/messages (streaming), /api/profile[/password], /api/models, /api/system-prompt, /api/tools, /api/checklists/*, /admin/users[/{id}/limits], /admin/mcp-servers[/{id}[/check]].
• frontend/app/ - Next.js App Router: /login, /chats (chat, sidebar + streaming message view + model picker), /checklists, /help.

The admin user list shows each account's real name and email under its login name, so an admin can tell who is who without opening the DB.

Products, and who may open them

/chats and /checklists are separate products in one app, and each is a per-user switch an admin flips in Settings -> Administration. Only the chat is on by default - an account exists to chat - so a new product stays out of sight until someone is given it, and the sidebar's "Products" section only appears once the user actually has another product to open.

The switch is not cosmetic: every one of that product's routes is refused for a user without it (403, "not enabled for your account"), so a typed-in URL gets an explanation rather than an app whose every request fails. Admins hold every product implicitly, as with every other limit.

/help is the exception that proves the shape: it uses the same shell and sidebar as a product but is not in permissions.PRODUCTS - an account that can log in can read the manual, so there is nothing to switch on. It is reached from the user menu, in a tab of its own, and its admin section renders only for an admin.

Searching the web

With web_search and fetch_url switched on, a model can look something up instead of answering from memory: it searches, opens the most promising result, and cites the URL. Both are ordinary tools - an admin has to have granted the user tools, and the user has to switch them on in Settings -> Tools - and both mean the question, or part of it, leaves this server.

The engine is chosen in .env:

CHATUI_SEARCH_PROVIDER=duckduckgo   # or: searxng, with CHATUI_SEARXNG_URL
CHATUI_SEARCH_RESULTS=5

duckduckgo needs no key, which is its whole appeal: it reads the HTML page meant for browsers, falling back to the /lite/ one. Expect it to be intermittent. A server that searches several times in a row is handed a "select all the ducks" challenge instead of results, and it sticks: a burst of test searches from this box kept both endpoints challenged for more than ten minutes. Mojeek serves this address a captcha as well, so it is the datacenter IP being judged, not the engine - which means no keyless scraper will be dependable here. Good enough for the occasional question, not for a busy chat.

The way out is a provider of this deployment's own: searxng (self-hosted, real JSON API, CHATUI_SEARXNG_URL) is written and waiting for an instance, and a keyed API - Brave, Tavily - would be a small provider function in websearch.py's PROVIDERS next to it.

fetch_url only reaches the public internet. Every address - and every redirect it follows - is resolved first and refused unless it is a globally routable one, so a page cannot talk a model into fetching 127.0.0.1:8001, a Tailscale address or anything else on this network.

Settings -> Account is a user's own page: their username, role and join date (read-only - an admin provisions the account), the real name and email they can fill in, and a password change. The name, once set, is what the sidebar shows beside the avatar instead of the login name. Changing the password does not sign anyone out: the session JWT carries no password state, so there is nothing to invalidate.

A reply streams in token by token, but the first token takes a few seconds (the router queue plus the model's prefill), so the wait is shown instead of looking like a hang: pulsing dots with a running seconds counter while nothing has arrived yet, then a blinking caret at the end of the text for as long as it keeps arriving.

If a reply fails part-way - most often because vllm-router has no backend for the model the chat needs - the reason is written into the reply itself (⚠️ The reply failed: …) and stored with it. The response is already streaming by then, so there is no HTTP status left to fail with, and a reply that simply stops looks exactly like a slow one.

The model picker lists whatever vllm-router reports as up, plus its synthetic auto entry ("Auto (router picks)"), which is first and therefore the default for a new chat. With auto the router resolves a concrete model per request from what the request contains - see vllm-router's README. A model that can read images is marked with a 🖼 in the picker.

Messages can carry up to 5 attachments. A text file is inlined into the prompt as before; a PNG/JPEG/WebP/GIF (up to 6 MB) is sent to the model as a real image part, so a chat pinned to a text-only model automatically falls back to auto:vision for that request - the router then picks whichever vision-capable model is up. Images are stored with the message and served back to the browser from GET /api/attachments/{id}, so reloading a chat doesn't drag megabytes of base64 through the conversation JSON.