Feat/provider capabilities rework - #80
Open
Zeptiny wants to merge 15 commits into
Open
Conversation
…ns, and connection document v2 U1 of the provider capabilities rework: typed facet metadata (thinking policy, tier mechanism, pricing dimensions incl. non-fiat currency, cache, quota, discovery) as optional hooks on the trusted driver interface; catalog facet declarations gated by the trust policy list; connection document v2 (discoveredModels, per-model pricing overrides, tier selections) with fill-absent v1->v2 migration on read; new openai-responses protocol member.
U2: createNativeLanguageModel branches to .responses() for the openai-responses protocol (native OpenAI, OpenCode Go per-model table, generic compatible endpoints); trust pins allow it for those drivers. Fixes dead provider options (openai maxReasoningTokens, anthropic reasoningEffort -> effort). Verified end-to-end against a mock SSE fetch: /responses route, reasoning effort mapping, paired tool items, and usage normalization (reasoning_tokens, cached_tokens).
…and dynamic refresh U4: PricingResolver composes rates provider-API -> user -> catalog per rate field with per-field provenance; request-driven PricingRefresher keeps latest-known dynamic rates without blocking requests, falling back with stale provenance. FrozenPricingSnapshot gains cacheWriteByTtl, perRequest, contextTiers, currencyUnit; cost.ts handles non-fiat units, context tiers, and per-request fees; attempts record cost_rung (schema v4) and costRungStale evidence. Cost evidence extraction moves behind the driver pricing facet (Neuralwatt extractor, header allowlist); Lilac's inline multiplier becomes its dynamic pricing facet.
U3: drivers declare per-model thinking policies (exposure, replay rule, declared request knobs); stream adapter captures Anthropic signature/redacted and Responses itemId/encrypted-content artifacts into Message.thinking_payload; history replays per policy (mandatory-in-tool-loop / recommended / impossible) with strip-on-switch; reasoning parts carry providerOptions through toModelMessages; opaque thinking renders as a token-count indicator; subagent runs resolve and persist artifacts the same way.
…l listing U5: discovery facets for Neuralwatt (inline pricing) and Lilac via a shared models-endpoint helper; discovery runs on connection create, first ready credential, and manual fetch (no polling); discovered models merge into connection.discoveredModels with provider provenance and fill-absent reasoning seeds; unified model_list rows carry catalog/provider/user source badges with uniform affordances; effective metadata resolves user > live > catalog per field; PricingRefresher invalidate wired into connection delete/disconnect/key rotation.
… Go, and compatible endpoints U5 follow-up: native OpenAI discovers /v1/models ids with openai-responses protocol stamped from the catalog protocol table; OpenCode Go discovers ids-only with per-model protocol mapping; generic OpenAI-compatible discovers ids from the validated connection endpoint.
… placement U6 (tier/cache half): per-model tier selections on the connection with per-session override (draft-tier, mirroring draft-reasoning); variant drivers (Neuralwatt -flex/-fast/-short) group under the base model row with a tier selector and map the selection to the variant id at request time with precondition assertions; parameter drivers send serviceTier opt-in. Served tier is captured into attempt evidence and billed at the served variant's rates. Anthropic places cache_control breakpoints on the stable tools+system prefix and the advancing conversation tail with TTL as the only user knob; OpenAI sends a session-scoped promptCacheKey; generic connections send no markers. Stable-prefix order guarded so dynamic context lands after the static prefix for main and subagents.
…ering U6 (quota half): typed read-only quota facet (balances, subscription, allowances in provider-native units) validated into the status layer with TTL/single-flight refresh, surfaced over providers:quota_refresh; Neuralwatt maps USD credit + kWh energy + key allowance, Lilac maps documented per-model supply-state/discount into allowances (balances stay empty — no documented endpoint). Quota is informational only and never gates sends. Analytics renders kWh and USD buckets separately without merging and shows per-attempt energy evidence (charged kWh, consumed, multiplier) plus a quota overview panel.
…, dead code (review fixes) Addresses P0 #1, P1 #3, P2 #7/#10/#12/#19 from code review: - cap thinking replay artifact sizes at capture, hydration, IPC schema, and replay (64 KiB blob/displayText, 256 itemId) - pricing refresher backs off after failure (once per cadence) and invalidate() races in-flight refreshes via generation guard - Anthropic tail cache breakpoint walks back from reasoning-only tails - remove dead legacy Neuralwatt quota path; repoint tests at canonical neuralwatt-quota.ts - replace draft-tier/draft-reasoning copies with shared generic createDraftOverrideStore factory
…g, tier tests (review fixes) Addresses P2 #4/#6/#8/#11/#13/#14/#15/#16 and P1 #2 from code review: - user pricing overrides key off the base model id on variant-tier requests; live-discovered inline pricing feeds the ladder as the top provider-api rung with discoveredAt provenance - request-parameter tier attempts refuse to silently bill base rates when no tier-aware rates are frozen - discovery delist prunes orphaned modelIds/tierSelections/reasoningConfig and orphaned enabled ids warn instead of hard-throwing on validate/update - session tierOverride persistence + v2->v3 DB migration + set/get_service_tier IPC now covered; zod-rejection tests for discover_models/quota_refresh - real-facet tier wiring tests (resolveTierContext, session override vs connection selection, variant vs parameter mechanism, extractServedTier) - numeric reasoning budgets flagged as unsupported for OpenAI in the editor; session tier selector filters variant tiers to present models
…eview fixes) Addresses P2 #18/#20: extract the duplicated thinking/tier/cache provider-options merge block from send.ts and subagent-runner.ts into assembleFacetProviderOptions (facets/turn-options.ts), preserving merge order; rename mergeThinkingProviderOptions to mergeProviderOptions and update all call sites.
…C (review fixes) Addresses P2 #5/#17: pricingOverrides now round-trips through create/update payload schemas and ProviderConnectionView, is wired into the update handler and modelOptions rows keyed by base model id, and is editable per model in the unified listing (input/output/cache/reasoning rates + per-request fee, save/clear). Unified-model-listing, discovery, tier/cache TTL view, and quota IPC move from ipc/providers.ts into a new ipc/provider-models.ts; channel names, zod schemas, and message types preserved; ipc/index.ts registers both modules.
…viders and connections
|
Important Review skippedToo many files! This PR contains 125 files, which is 25 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (125)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.