Repository navigation
docs: auto-sync cli with merged PRs (through 2026-10-31) #15139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
64354d2
458d312
4a819a1
59d20e8
ea36966
4705d17
6092102
7b3df62
37b7a2d
c980f44
ae198fc
c298228
1173b37
61c99f7
6e62dfe
d9a8c45
90ddc55
df9b24c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -41,36 +41,43 @@ The underlying models behind each Auto Model tier are updated server-side as bet | |||||
|
|
||||||
| You get lean costs on routine work and stronger models when the work demands it — with no manual switching. | ||||||
|
|
||||||
| ### Custom Efficient pools | ||||||
| ### Platform pools | ||||||
|
|
||||||
| You can constrain `kilo-auto/efficient` to the exact models you trust by configuring an **Efficient model pool** on the **Auto routing** card: | ||||||
| Kilo selects the model pool for `kilo-auto/efficient` and `kilo-auto/frontier`; they are not configurable per owner. You can still choose your account's platform routing mode, and organization model policy continues to filter which models its members can route to. | ||||||
|
|
||||||
| - **Personal** — your [profile page](https://app.kilo.ai/profile) | ||||||
| - **Organization** — your organization's **Providers & Models** page. Owners and billing managers can edit; members see a read-only view. | ||||||
| To route across the exact models you trust — including client models the gateway cannot reach — create a [custom auto router](#custom-auto-routers). | ||||||
|
|
||||||
| A pool holds 1–10 exact model and thinking-variant pairs. Variants stay distinct, so the same model with different thinking variants (for example `max` and `xhigh`) counts as separate entries. Leave the pool empty to inherit: an organization without a pool uses each member's personal pool, and a member without a personal pool uses the platform pool. | ||||||
| ### Routing modes | ||||||
|
|
||||||
| New entries are benchmarked on demand before they can serve traffic. Each entry shows a status: | ||||||
| Account owners choose a routing mode in the Kilo web app. The mode decides how each request is matched to a candidate from the pool, and it applies to the platform Efficient and Frontier tiers and to each custom auto router. | ||||||
|
|
||||||
| | Status | Meaning | | ||||||
| | Mode | Behavior | | ||||||
| |---|---| | ||||||
| | Benchmarking | The pair is being measured and is not used for routing yet | | ||||||
| | Ready | Proven accurate enough and eligible for routing | | ||||||
| | Failed | Benchmarking failed — retry the entry | | ||||||
| | Unavailable | The model or variant is no longer in your catalog — remove the entry | | ||||||
| | **Best accuracy** | Routes to the most accurate candidate that passes the router's filters. | | ||||||
| | **Cost per accuracy** | Balances cost against accuracy, preferring a cheaper candidate but tolerating small cost differences so it does not switch away from a solid incumbent for a marginal saving. | | ||||||
| | **Cheapest** | Routes to the lowest-cost candidate that still meets the router's accuracy threshold. A cost tie goes to the higher-accuracy candidate, and when no candidate meets the threshold it routes to the most accurate candidate instead. | | ||||||
|
|
||||||
| Routing decides only among ready entries. If no pool entry can serve a request, the request falls back to a fixed, cost-effective baseline model, so quality never drops below that baseline. | ||||||
|
|
||||||
| {% callout type="note" %} | ||||||
| You can benchmark up to 10 new or retried pairs per owner per rolling 24 hours. Entries that are already ready or benchmarking don't count against this limit. | ||||||
| {% /callout %} | ||||||
| Cheapest reuses the same deny, capability, and active filters as the other modes, along with the low-confidence override that falls back to the most accurate model. Once Cheapest selects a model, that model must keep meeting the router's strict accuracy threshold to stay selected; the small-cost-difference tolerance applies only to Cost per accuracy. Cheapest keeps the model variant it picked. | ||||||
|
|
||||||
| {% callout type="warning" title="Data handling for Auto Free" %} | ||||||
| Auto Free may route your requests to providers that log prompts and outputs and use them to improve their services. Do not submit personal or confidential data when using Auto Free. In particular, it may route to NVIDIA's free endpoints. | ||||||
|
|
||||||
| For NVIDIA free endpoints (Super/Ultra/etc): Trial use only - do not submit personal or confidential data. Your use is logged for security purposes and to improve NVIDIA products and services. The logged session data for improvement purposes is not linked to your identity or any persistent identifier. For more information about our data processing practices, see our [Privacy Policy](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/). By interacting with this endpoint, you consent to our collection, recording, and use of such information and the [NVIDIA API Trial Terms of Service](https://assets.ngc.nvidia.com/products/api-catalog/legal/NVIDIA%20API%20Trial%20Terms%20of%20Service.pdf). | ||||||
| {% /callout %} | ||||||
|
|
||||||
| ### Custom auto routers | ||||||
|
|
||||||
| A custom auto router is your own routing pool under a `kilo-auto/custom/<slug>` model id. Create the router in the Kilo web editor, then register the models this machine can run so the CLI can serve it locally instead of always forwarding to the gateway: | ||||||
|
|
||||||
| - `kilo auto router list` — list your custom routers, their entries, and each entry's eval status | ||||||
| - `kilo auto router add <router> <model>` — register a model configured in this CLI as a router entry | ||||||
| - `kilo auto router remove <router> <model>` — remove a client model entry | ||||||
| - `kilo auto eval <router>` — run the router's missing client evals locally in Docker | ||||||
|
|
||||||
| When you select a custom router, Kilo ranks the entries this machine can serve, asks the routing service for a decision, and runs the chosen model through its local provider. See the [CLI command reference](/docs/code-with-ai/platforms/cli-reference#kilo-auto) for every command and option. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [SUGGESTION]: Dispatch does not always run on the local provider
Suggested change
|
||||||
|
|
||||||
| An entry can link to a platform model so it reuses that model's platform benchmark instead of running a local client eval. Pass `--measured-as <model>` to `kilo auto router add`, accept the link the CLI suggests when it finds a matching platform model, or set it in the web editor, which suggests a link for each entry and lets you pick another model or clear it. The link applies to each variant whose name matches a measured platform variant; other variants still need a client eval, and linked variants are skipped by `kilo auto eval`. A linked entry serves the newest platform measurement for that model and is flagged stale when the measurement is older than the current benchmark engine or the model has left the catalog. Once stored, a link stays until you clear it in the web editor. | ||||||
|
|
||||||
| ## Benefits | ||||||
|
|
||||||
| ### Cost Optimization | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -39,6 +39,8 @@ kilo --help | |||||
|
|
||||||
| After installation, run `kilo` and use the `/connect` command to add your first provider credentials. This is the interactive way to configure API keys for model providers. | ||||||
|
|
||||||
| The dialog lists each provider with a short description — Kilo Gateway is marked **Recommended**. When the Kilo Gateway model list cannot load, its row shows a red `!` with the reason, such as credentials rejected, not signed in, no connection, or a provider error, so you can fix the connection before starting a session. | ||||||
|
|
||||||
| ## Opening Links | ||||||
|
|
||||||
| Links in agent responses use terminal hyperlink metadata when the terminal supports it. Kilo also handles links inside the TUI, so mouse capture does not make supported HTTP(S) links inactive. Move the pointer over a link to see the hover underline, then click it to open the default browser. | ||||||
|
|
@@ -122,9 +124,12 @@ The `kilo console` command and its browser interface are deprecated and will be | |||||
|
|
||||||
| #### Provider Commands | ||||||
|
|
||||||
| | Command | Description | | ||||||
| |---|---| | ||||||
| | `/connect` | Connect/add a provider - entry point for new users to add API credentials | | ||||||
| | Command | Aliases | Description | | ||||||
| |---|---|---| | ||||||
| | `/connect` | - | Connect/add a provider - entry point for new users to add API credentials | | ||||||
| | `/usage` | `/plans`, `/quota` | Show plan and quota usage for your connected providers | | ||||||
|
|
||||||
| `/usage` opens a **Plans & usage** dialog with one card per detected source: Kilo coding plan quotas, ChatGPT (Codex) rate limits, and MiniMax Token Plan quotas. Each card shows the plan, the usage windows with their percentage used, any reset time, and a **Manage** link when the provider offers one. Press `ctrl+r` to refresh. | ||||||
|
|
||||||
| #### System Commands | ||||||
|
|
||||||
|
|
@@ -262,6 +267,26 @@ The `diff_open` and `session_move` TUI keybindings run the same actions and are | |||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ## Home Screen and Session Sidebar | ||||||
|
|
||||||
| ### Home screen news | ||||||
|
|
||||||
| When you are signed in to Kilo Gateway, the CLI home screen shows the most recent unread Kilo news item, with a `(N new)` count when more than one item is unread. Select the banner to open the **News** dialog, which lists every unread item and its action links. Opening the dialog marks those items as read, and they stay hidden after a restart. Items that are not intended for the CLI are not shown. | ||||||
|
|
||||||
| The command palette has **Hide news** and **Show news** on the home screen, and the choice persists. Kilo fetches news only while you are signed in, and refetches after you sign in, sign out, or switch teams. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [SUGGESTION]: News is not refetched after sign-in, sign-out, or a team switch in the current CLI
Suggested change
|
||||||
|
|
||||||
| ### Session sidebar | ||||||
|
|
||||||
| The session sidebar shows Kilo panels alongside the built-in ones: | ||||||
|
|
||||||
| - **Branch PR** — the pull request for the current branch, when `gh` is installed and the branch has one. | ||||||
| - **Token Usage** — input, output, reasoning, cache read, and cache write tokens, cache hit rate, generation speed, and cost for the whole session, including subagents. | ||||||
| - **Models** — one row per provider and model, with its step count and cost. Expand a row for its token breakdown. | ||||||
| - **Terminal Bench 2.0** — the completion score and cost per attempt for the current Kilo model. | ||||||
| - **Footer** — your Kilo balance, Kilo Pass usage (personal accounts), and the Kilo version. | ||||||
|
|
||||||
| Select a panel heading to collapse or expand it. | ||||||
|
|
||||||
| ## Permissions | ||||||
|
|
||||||
| Kilo Code uses the permission config to decide whether a given action should run automatically, prompt you, or be blocked. | ||||||
|
|
@@ -707,6 +732,21 @@ The CLI supports overriding config values with environment variables. The suppor | |||||
| - For `kilocode` provider: `KILOCODE_<FIELD_NAME>` (e.g., `KILOCODE_MODEL` → `kilocodeModel`) | ||||||
| - For other providers: `KILO_<FIELD_NAME>` (e.g., `KILO_API_KEY` → `apiKey`) | ||||||
|
|
||||||
| ## Self-Hosted Backends | ||||||
|
|
||||||
| By default the CLI sends Kilo API traffic (sign-in, profile, defaults) to `https://api.kilo.ai` and [AI gateway](/docs/gateway) traffic (chat, model listing, embeddings, autocomplete, transcription) to `https://ai-gateway.kilo.ai/api/v1`. Point either at a different server with these environment variables — for example, to route gateway traffic through a self-hosted gateway for compliance, or a local backend during development. | ||||||
|
|
||||||
| | Variable | Default | Controls | | ||||||
| |---|---|---| | ||||||
| | `KILO_API_URL` | `https://api.kilo.ai` | Kilo API traffic. When set on its own (without `KILO_AI_GATEWAY_URL`), gateway traffic also goes here, under `/api/gateway/v1`. | | ||||||
| | `KILO_AI_GATEWAY_URL` | `https://ai-gateway.kilo.ai/api/v1` | AI gateway traffic only. | | ||||||
|
|
||||||
| Set `KILO_AI_GATEWAY_URL` to send only gateway traffic to another server while leaving sign-in and profile requests on the default: | ||||||
|
|
||||||
| ```bash | ||||||
| KILO_AI_GATEWAY_URL=https://gateway.example.com/api/v1 kilo | ||||||
| ``` | ||||||
|
|
||||||
| ## Using the CLI in an Organization | ||||||
|
|
||||||
| If you belong to a Kilo organization (Team or Enterprise), you can route CLI requests through that organization. The process differs slightly between interactive and non-interactive usage. | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[SUGGESTION]: Cheapest is not in the CLI's custom-router mode schema
This paragraph says the routing mode applies to each custom auto router, but the CLI validates a router's mode with
AutoRoutingModeSchema = z.enum(["cost_per_accuracy", "best_accuracy"])(packages/kilo-gateway/src/api/custom-router.ts:161) andkilo auto router listprints that value. If the web app can save a custom router as Cheapest, the CLI will fail to parse the router summary until the enum is extended; otherwise, scope this sentence and the Cheapest row to the platform tiers.