Skip to content

[EE] Key limits, and user and team budgets - #230

Closed
ganisback wants to merge 3 commits into
mainfrom
feat/ee-key-governance
Closed

ganisback wants to merge 3 commits into
mainfrom
feat/ee-key-governance

Conversation

@ganisback

Copy link
Copy Markdown
Collaborator

Summary

E2 of the EE plan. Split the same way as E1, following LiteLLM's open-source proxy: limits on a key are free, and budgets that span people are Enterprise.

Free (Apache-2.0, internal/keygov) — per API key:

  • Allowed models (Qwen/* prefix matching); /v1/models lists only those
  • Expiry date
  • RPM / TPM over a sliding minute
  • Token and/or cost budget, per day, per month or in total

Enterprise (ee/budget, feature.lite.budgets):

  • Budgets for a user (across all their keys and the web UI) and for a team (all members together)
  • Warning at 80% of any budget: X-CSGLite-Budget-Warning: key=85%, team=92% on responses, spend.warning in list responses, amber marker in the UI

How it works

  • Every metered request is recorded in ~/.csghub-lite/spend.db against its key, its owner and the owner's team (per local day and currency). Deleting and recreating a key does not reset a user's or team's budget.
  • Requests past a limit are refused before they run: 401 api_key_expired, 403 model_not_allowed, 429 budget_exceeded, 429 rate_limited with Retry-After. OpenAI/Anthropic routes put the code in error.type, Ollama routes in code.
  • Anyone can restrict a key they create; afterwards only an administrator changes limits (PUT /api/api-keys/{id}). Members can only rename.
  • If the license lapses, budgets already set keep being enforced (spending stays capped) but cannot be changed.

API

  • PUT /api/api-keys/{id}: rename a key or replace its limits
  • POST /api/api-keys: now accepts limits
  • PUT /api/team/users/{id}/budget, PUT /api/team/teams/{id}/budget: Enterprise
  • Key, user and team responses include limits/budget, spend and status

OpenAPI is updated and the static copy is in sync.

UI

  • AI Gateway → API Keys: a 「限制(可选)」 section when creating a key, a 「限制」 button per key (administrators), and chips for models, expiry, RPM/TPM and budget usage
  • My API Keys: the same chips, read-only
  • Teams: a budget column and a 「预算」 action for users and teams; without a license the dialog is read-only with an Enterprise note

Docs

  • New docs/guides/key-governance.md
  • Updated team-management.md, ee-license-design.md and ee/README.md

Test plan

  • go test ./... passes
  • New tests:
    • internal/keygov: limits, ledger, limiter
    • ee/budget
    • internal/server/key_governance_test.go: allowlist, rate limit, expiry, key budget, admin-only limit changes, unlicensed 403, user/team warning and exhaustion, spend attribution, model list filtering, body peeking for JSON and multipart
  • npm test passes (100 tests, 5 new)
  • tsc passes
  • Checked on the :11435 preview with headless Chrome: create dialog with limits, edit dialog, key chips, Teams budget column and the unlicensed budget dialog

🤖 Generated with Claude Code

ganisback and others added 3 commits October 22, 2026 17:46
An API key can now be limited to a list of models, given an expiry date,
rate-limited by requests and tokens per minute, and capped by a token or
cost budget per day, per month or in total. As in LiteLLM's open-source
proxy these limits are free; they live in internal/keygov with a spend
ledger (spend.db) that counts every metered request against its key, its
owner and the owner's team.

The Enterprise part, gated by feature.lite.budgets, is budgets for users
and teams, counted across every key and the web UI, and the warning at
80% of any budget (X-CSGLite-Budget-Warning, and a flag in list
responses). Only that lives in ee/budget.

Inference requests past a limit are refused before they run: 401
api_key_expired, 403 model_not_allowed, 429 budget_exceeded or 429
rate_limited with Retry-After. /v1/models lists only allowed models.
Anyone may restrict a key they create; afterwards only an administrator
changes limits, so a member cannot lift what was set for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A member, like an internal user in LiteLLM, uses the instance rather than
running it, so the dashboard, apps, models and datasets are for
administrators and auditors. A member who opens one of those pages is
taken to chat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The member page had an always-visible name field; it now opens the same
dialog the AI gateway uses, shared as CreateKeyDialog, which also lets a
member add optional limits to their own key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ganisback

Copy link
Copy Markdown
Collaborator Author

Closing: E1 (accounts, #226) was reverted on main in 1c19721 because the design followed LiteLLM too closely. The branch feat/ee-key-governance is kept for reference.

@ganisback ganisback closed this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant