fix(deploy): complete the LLM env wiring for the demo stack#66
Open
librowski wants to merge 3 commits into
Open
fix(deploy): complete the LLM env wiring for the demo stack#66librowski wants to merge 3 commits into
librowski wants to merge 3 commits into
Conversation
The adapt endpoint runs its LLM call in the backend, not the worker, and answered 501 adapt_disabled on every request because only the worker received the key. Executions kept working, which masked it.
librowski
requested review from
lukasz-jazwa,
piotrblaszczyk and
szymon-t-sc
as code owners
July 17, 2026 09:52
The worker reads TAVILY_API_KEY to enable the AI Agent's web search tool, but the deploy compose never passed it, so web search was silently disabled on deployments: agents with the toggle on ran without the tool. Optional - an empty value keeps it off.
Backend and worker code defaulted to google/gemini-2.5-flash-lite while the deploy stack defaults to mistralai/mistral-small-3.2-24b-instruct, so an unset AI_MODEL meant different models depending on where the process ran. One default everywhere now - the deployed demo already runs this model, so production behavior does not change.
librowski
changed the title
piotrblaszczyk
approved these changes
Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Three gaps in how LLM configuration reaches the demo services, all silent in the UI:
Backend never received
OPENROUTER_API_KEY. The adapt endpoint (/api/visualize/adapt) makes its LLM call in the backend process, but the compose passed the key only to the worker. Every adapt call answered501 adapt_disabled; the frontend swallows adapt failures and falls back to raw text, so "Adapt with AI" never worked on deployments while executions (worker-side LLM calls) kept working. The backend now gets the sameOPENROUTER_API_KEY/AI_MODELpair as the worker - no new secret needed on the host.TAVILY_API_KEYwas never wired into the deploy. The worker reads it to enable the AI Agent's web search tool; without it, agents with web search toggled on silently run without the tool. Now passed through the compose (optional, empty keeps it off) and documented in.env.example.Two different model defaults. Backend and worker code defaulted to
google/gemini-2.5-flash-litewhile the deploy stack defaults tomistralai/mistral-small-3.2-24b-instruct- an unsetAI_MODELmeant different models depending on where the process ran. Unified on the demo model everywhere (cheap, EU-hosted, solid tool calling); the deployed demo already runs it, so production behavior does not change.Deploy note
The deploy workflow only pulls images and recreates containers - it does not sync the compose file. After merging: update the compose on the host with this version, optionally add
TAVILY_API_KEYto the host.envto enable web search, thendocker compose up -d --force-recreate backend worker.