diff --git a/docs/cloud/guides/integrations/index.mdx b/docs/cloud/guides/integrations/index.mdx index 91b45c2..7cc92e9 100644 --- a/docs/cloud/guides/integrations/index.mdx +++ b/docs/cloud/guides/integrations/index.mdx @@ -28,6 +28,7 @@ Use the OpenAI client guide when you are configuring code directly. Use the inte - [Continue](/cloud/guides/integrations/continue) - [Cline, Roo Code, and Kilo Code](/cloud/guides/integrations/cline-roo-kilo) - [Aider and Zed](/cloud/guides/integrations/aider-zed) +- [OpenClaw](/cloud/guides/integrations/openclaw) - [OpenCode and Goose](/cloud/guides/opencode-goose) ### Self-hosted and team apps diff --git a/docs/cloud/guides/integrations/model-discovery.mdx b/docs/cloud/guides/integrations/model-discovery.mdx index a017d96..b09070b 100644 --- a/docs/cloud/guides/integrations/model-discovery.mdx +++ b/docs/cloud/guides/integrations/model-discovery.mdx @@ -42,13 +42,18 @@ The response uses OpenAI-style model objects plus NEAR AI Cloud metadata. Import | `id` | The model ID to use with the gateway. For GLM 5.2, use `z-ai/glm-5.2`. | | `name` | Human-readable display name. | | `context_length` | Maximum input context length reported for the model. | -| `max_output_length` | Maximum output length reported by NEAR AI Cloud. | +| `max_output_length` | OpenRouter-compatible advisory hint curated in the NEAR AI Cloud catalog. It is not enforced: the gateway forwards `max_tokens` unchanged and does not clamp it. For a NEAR-hosted model, the binding limit is the model context window (`max_model_len` at its direct completions endpoint), shared between prompt and completion. | | `top_provider.context_length` | Provider context length used by clients that read OpenRouter-style metadata. | -| `top_provider.max_completion_tokens` | Provider maximum completion tokens used by clients that read OpenRouter-style metadata. | +| `top_provider.max_completion_tokens` | Mirrors `max_output_length` for clients that read OpenRouter-style metadata and carries the same advisory, non-enforced caveat. | +| `is_ready` | OpenRouter catalog publication flag, exposed verbatim. `false` keeps the model hidden on OpenRouter's side; `true` enables auto-staging there. It does not indicate NEAR AI Cloud availability, may be absent, and should be ignored when deciding whether a model is usable. If a model is listed in `/v1/models`, it is available. | | `supported_features` | Feature flags such as `tools`, `structured_outputs`, `reasoning`, and `json_mode`. | | `supported_sampling_parameters` | Supported request parameters such as `temperature`, `top_p`, `max_tokens`, and `stop`. | | `input_modalities` and `output_modalities` | Supported input and output types. | +### Fields that do not mean what their names suggest + +As checked on 2026-08-18, the gateway reports `max_output_length: 8192` for `deepseek-ai/DeepSeek-V4-Flash`, while `https://dsv4-flash.completions.near.ai/v1/models` reports `max_model_len: 1048576`; the gateway field is advisory metadata rather than an enforced output cap. Every `openai/*`, `anthropic/*`, and `google/*` entry reports `is_ready: false` while remaining fully serviceable, because `is_ready` describes OpenRouter publication rather than NEAR AI Cloud availability. + As checked on 2026-06-23, the live GLM 5.2 gateway record reports `top_provider.context_length: 500000`, `top_provider.max_completion_tokens: 131072`, and `max_output_length: 131072`. ## Discover direct completions endpoints diff --git a/docs/cloud/guides/integrations/openclaw.mdx b/docs/cloud/guides/integrations/openclaw.mdx new file mode 100644 index 0000000..8c74ffe --- /dev/null +++ b/docs/cloud/guides/integrations/openclaw.mdx @@ -0,0 +1,154 @@ +--- +id: openclaw +title: OpenClaw +sidebar_label: OpenClaw +slug: /cloud/guides/integrations/openclaw +description: "Configure OpenClaw to use NEAR AI Cloud as an OpenAI-compatible provider." +--- + +# OpenClaw + +## Overview + +OpenClaw talks to NEAR AI Cloud as an OpenAI-compatible provider. NEAR AI also runs OpenClaw as a hosted worker, so this provider configuration is the same one NEAR uses. + +## Prerequisites + +- A NEAR AI Cloud API key. +- OpenClaw installed. This configuration was verified with OpenClaw 2026.7.12. + +## Base URL + +Use the NEAR AI Cloud gateway base URL: + +```text +https://cloud-api.near.ai/v1 +``` + +Do not append `/chat/completions` to the base URL. + +## Model ID + +Add each model to the provider's `models` array using its gateway model ID. This guide uses: + +```text +z-ai/glm-5.2 +``` + +See [Model Discovery](/cloud/guides/integrations/model-discovery) for the current model list. + +## Configure + +Add the NEAR AI Cloud provider to `~/.openclaw/openclaw.json` and wire it as the default model: + +```json +{ + "models": { + "mode": "merge", + "providers": { + "nearai": { + "baseUrl": "https://cloud-api.near.ai/v1", + "apiKey": "${NEARAI_API_KEY}", + "auth": "api-key", + "api": "openai-completions", + "models": [{ "id": "z-ai/glm-5.2", "name": "GLM 5.2" }] + } + } + }, + "agents": { + "defaults": { + "model": { + "primary": "nearai/z-ai/glm-5.2" + } + } + } +} +``` + +OpenClaw applies conservative defaults to an unrecognized provider. NEAR AI Cloud supports all three compatibility flags, so you can enable them explicitly: + +```json +{ + "models": { + "mode": "merge", + "providers": { + "nearai": { + "baseUrl": "https://cloud-api.near.ai/v1", + "apiKey": "${NEARAI_API_KEY}", + "auth": "api-key", + "api": "openai-completions", + "models": [{ "id": "z-ai/glm-5.2", "name": "GLM 5.2" }], + "compat": { + "supportsUsageInStreaming": true, + "supportsStrictMode": true, + "supportsDeveloperRole": true + } + } + } + }, + "agents": { + "defaults": { + "model": { + "primary": "nearai/z-ai/glm-5.2" + } + } + } +} +``` + +| Flag | Why it is safe to enable | +| --- | --- | +| `supportsUsageInStreaming` | NEAR AI Cloud handles `stream_options: {"include_usage": true}` by setting intermediate chunks to `usage: null` and appending one final usage chunk. | +| `supportsDeveloperRole` | NEAR AI Cloud accepts `developer` alongside `system`, `user`, `assistant`, and `tool` roles. | +| `supportsStrictMode` | NEAR AI Cloud accepts and forwards strict JSON-schema response formats and strict tool definitions. Proxied third-party passthrough providers may ignore strict mode; strict schema enforcement is guaranteed only on NEAR-hosted models whose `supported_features` includes `structured_outputs`. | + +## Refresh models + +OpenClaw reads the provider's `models` array from `openclaw.json`; it does not auto-fetch `/v1/models`. Add new models to the array by hand. Use [Model Discovery](/cloud/guides/integrations/model-discovery) for the current list. + +## Quick test + +First, verify the gateway independently of OpenClaw: + +```bash +curl https://cloud-api.near.ai/v1/chat/completions \ + -H "Authorization: Bearer $NEARAI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "z-ai/glm-5.2", + "messages": [ + { + "role": "user", + "content": "Reply with one sentence confirming the model is reachable." + } + ], + "max_tokens": 64 + }' +``` + +Then start an OpenClaw chat and confirm its default model is `nearai/z-ai/glm-5.2`. + +## Troubleshooting + +| Symptom | Fix | +| --- | --- | +| Model not listed | Add the model ID from [Model Discovery](/cloud/guides/integrations/model-discovery) to the provider's `models` array by hand. | +| `401` | Set `NEARAI_API_KEY` to a NEAR AI Cloud API key and keep `apiKey` as `${NEARAI_API_KEY}`. | +| Base URL includes `/chat/completions` | Set `baseUrl` to `https://cloud-api.near.ai/v1`. Use `/chat/completions` only in a full request URL. | +| Streaming usage is missing | Set `supportsUsageInStreaming` to `true` in the `compat` block. | +| A `developer`-role message is rejected by a non-NEAR provider in the same config | NEAR AI Cloud accepts the `developer` role. Check the compatibility behavior of the non-NEAR provider that handled the message. | + +## Related guides + +- [OpenAI Compatibility](/cloud/guides/openai-compatibility) +- [Model Discovery](/cloud/guides/integrations/model-discovery) +- [Chat Verification](/cloud/verification/chat) +- [Available Models](/cloud/models) + +## Sources Checked + +Sources checked on 2026-08-18: + +- [`GET https://cloud-api.near.ai/v1/models`](https://cloud-api.near.ai/v1/models) +- `openclaw-nearai-worker/worker/openclaw.json.template` +- OpenClaw 2026.7.12 diff --git a/docs/cloud/guides/openai-compatibility.mdx b/docs/cloud/guides/openai-compatibility.mdx index 105daf5..d9445ed 100644 --- a/docs/cloud/guides/openai-compatibility.mdx +++ b/docs/cloud/guides/openai-compatibility.mdx @@ -193,6 +193,52 @@ NEAR AI Cloud also provides non-OpenAI extensions: - **Privacy redaction** (`/v1/privacy/redact`) - Gateway-only endpoint for redacting personally identifiable information - **Attestation & Signatures** (`/v1/attestation/report`, `/v1/signature/{chat_id}`) - See [Verification](/cloud/verification) +## Compatibility Matrix + +| OpenAI request feature | NEAR AI Cloud support | `/v1/models` advertisement | +| --- | --- | --- | +| Tool / function calling (`tools`) | Supported on tool-capable models. | `supported_features: tools` | +| `tool_choice` | Supported on tool-capable models. | `supported_features: tools` | +| Structured outputs (`response_format: {"type": "json_schema"}`), including `strict: true` | Accepted and forwarded. Strict schema enforcement is guaranteed only on NEAR-hosted models that advertise structured outputs; proxied third-party passthrough providers may ignore strict mode. | `supported_features: structured_outputs` | +| JSON mode (`response_format: {"type": "json_object"}`) | Supported on models that advertise JSON mode. | `supported_features: json_mode` | +| Reasoning | Supported on models that advertise reasoning. | `supported_features: reasoning` | +| `developer` role messages | Accepted alongside `system`, `user`, `assistant`, and `tool` roles. | No dedicated flag. | +| `stream_options.include_usage` | Supported. Intermediate chunks carry `usage: null`, followed by one final usage chunk. | No dedicated flag. | +| Prompt caching | Supported on TEE-hosted models; a cache miss may return `prompt_tokens_details: null`. | No dedicated flag. | +| `max_tokens` | Forwarded unchanged to the provider; the gateway does not clamp it to `max_output_length`. | `supported_sampling_parameters: max_tokens` | + +Client authors can discover per-model capability from `supported_features` and `supported_sampling_parameters` in [`GET /v1/models`](https://cloud-api.near.ai/v1/models). Clients that cannot consume those fields can safely enable usage-in-streaming, the `developer` role, and strict mode for NEAR AI Cloud, subject to the strict-mode caveat for proxied third-party passthrough providers above. + +### Known deviations from the OpenAI schema + +On a cache miss, TEE-hosted models return `"prompt_tokens_details": null` rather than OpenAI's `{"cached_tokens": 0}` object. The inference engine emits `null`, and the gateway returns the provider's exact response bytes so the model-TEE signature verifies byte-for-byte. Normalizing the field at the gateway would break [Chat Verification](/cloud/verification/chat). Treat a missing details object as zero cached tokens: + + + + +```python +cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0) +``` + + + + +```javascript +const cached = usage.prompt_tokens_details?.cached_tokens ?? 0; +``` + + + + +See [Prompt Caching](/cloud/guides/prompt-caching#cache-misses-and-null-details) for the cache-miss behavior. + +The `max_output_length` field in `/v1/models` is advisory catalog metadata, not an enforced cap. The `is_ready` field is an OpenRouter catalog publication flag and must not be read as NEAR AI Cloud model availability. See the [Model Discovery field table](/cloud/guides/integrations/model-discovery#discover-gateway-models) for both fields. + ### Files API Example You can use the Files API to upload documents for use with supported models. diff --git a/docs/cloud/guides/prompt-caching.mdx b/docs/cloud/guides/prompt-caching.mdx index bdac16f..bf27d9c 100644 --- a/docs/cloud/guides/prompt-caching.mdx +++ b/docs/cloud/guides/prompt-caching.mdx @@ -6,6 +6,9 @@ slug: /cloud/guides/prompt-caching description: "How NEAR AI Cloud caches prompt prefixes to reduce latency and cost" --- +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + # Prompt Caching NEAR AI Cloud automatically caches prompt prefixes on TEE-hosted models. When consecutive requests share a common prefix — a long system prompt, conversation history, few-shot examples, or a large document — the cached portion is reused instead of being recomputed, which lowers both latency and cost. @@ -51,7 +54,37 @@ The number of input tokens served from cache is reported in the `usage` object o } ``` -On the first request with a given prefix, `cached_tokens` is `0` (or `prompt_tokens_details` is `null`). Repeat the request — or send another request sharing the same prefix — and `cached_tokens` reflects the reused portion. +### Cache misses and null details + +On a cache miss, TEE-hosted models return `"prompt_tokens_details": null`, not `{"cached_tokens": 0}` as OpenAI does. The inference engine emits `null`, and the gateway returns the provider's exact response bytes so the model-TEE signature verifies byte-for-byte. Normalizing the field at the gateway would break [Chat Verification](/cloud/verification/chat). + +Treat a missing details object as zero cached tokens so client code never indexes into `null`: + + + + +```python +cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0) +``` + + + + +```javascript +const cached = usage.prompt_tokens_details?.cached_tokens ?? 0; +``` + + + + +See [Known deviations from the OpenAI schema](/cloud/guides/openai-compatibility#known-deviations-from-the-openai-schema) for the corresponding OpenAI Compatibility guidance. + +Repeat the request, or send another request with the same prefix, and `cached_tokens` will reflect the reused portion. ## Getting the Most Out of the Cache diff --git a/docs/cloud/verification/chat-verification.md b/docs/cloud/verification/chat-verification.md index 98244c4..f206d29 100644 --- a/docs/cloud/verification/chat-verification.md +++ b/docs/cloud/verification/chat-verification.md @@ -226,7 +226,7 @@ curl -X GET 'https://qwen35-122b.completions.near.ai/v1/signature/afa7975eaf844b ``` :::note -A model can be served by multiple TEE nodes behind the same domain. The signature is cached on the node that served your chat completion, so a lookup may transiently return `Chat id not found or expired` if it lands on a different node — simply retry until you hit the right one. +For multi-replica models, use [Deterministic Signature Retrieval](#deterministic-signature-retrieval) instead of blind retries. ::: ***Example Response:*** @@ -260,6 +260,86 @@ This exactly matches the model we requested and the values we calculated in the --- +## Deterministic Signature Retrieval + +The direct signature endpoint can return `404 Chat id not found or expired` because the signature is cached in memory inside the model TEE process that served the completion. The model domain is an L4 SNI load balancer with least-connections selection and no session affinity, so the lookup can land on a replica that never saw the completion. `dsv4-flash.completions.near.ai` had 5 healthy backends on 2026-08-18: blind retrying is a coin flip rather than a deterministic strategy, and a retry succeeds roughly one time in five. + +### Option 1 — Gateway (recommended) + +Send the completion through `https://cloud-api.near.ai/v1`, then fetch its signature from: + +```text +GET https://cloud-api.near.ai/v1/signature/{chat_id} +``` + +The gateway pins the chat ID to the backend that served it and stores the signature durably at completion time, so the lookup is deterministic and needs no retries. The stored signature remains `signature_kind: provider_tee` (signed by the model TEE) unless the gateway rewrote the stream; see [Signature Kinds](#signature-kinds). + +### Option 2 — Pin the replica + +Address one backend directly with `https://{slug}-i{N}.completions.near.ai`, which routes to backend `N % healthy_count`. Send both the completion and signature lookup to the same hostname for a single deterministic lookup against that model TEE: + +```bash +CHAT_ID=$(curl -fsS https://dsv4-flash-i0.completions.near.ai/v1/chat/completions \ + -H "Authorization: Bearer $NEARAI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "deepseek-ai/DeepSeek-V4-Flash", + "messages": [{"role": "user", "content": "Reply with one sentence."}] + }' | jq -r '.id') + +curl -fsS "https://dsv4-flash-i0.completions.near.ai/v1/signature/${CHAT_ID}" \ + -H "Authorization: Bearer $NEARAI_API_KEY" +``` + +### Option 3 — Bounded sweep + +If you already hold a `chat_id` returned through the load-balanced domain, read the public backend count: + +```bash +curl 'https://completions.near.ai/backends/count?domain=dsv4-flash.completions.near.ai' +``` + +The response has this shape: + +```json +{ + "domain": "dsv4-flash.completions.near.ai", + "requested_domain": "dsv4-flash.completions.near.ai", + "healthy": 5, + "total": 5 +} +``` + +Iterate from `0` through `healthy - 1`; exactly one replica holds the signature. This loop stops on the first HTTP 200: + +```bash +HEALTHY=$(curl -fsS \ + 'https://completions.near.ai/backends/count?domain=dsv4-flash.completions.near.ai' \ + | jq -r '.healthy') + +for ((i = 0; i < HEALTHY; i++)); do + if curl -fsS \ + "https://dsv4-flash-i${i}.completions.near.ai/v1/signature/${CHAT_ID}" \ + -H "Authorization: Bearer $NEARAI_API_KEY" \ + -o signature.json; then + cat signature.json + break + fi +done +``` + +:::caution +The `-i{N}` index is positional. Its index-to-backend binding is stable only while the healthy backend count is stable, and it can shift when backends are added, removed, or flap health. Re-read `/backends/count` rather than caching an index across a long-running session, and prefer Option 1 for unattended or at-scale verification. +::: + +| Option | Where the completion must be sent | Determinism | When to use | +| --- | --- | --- | --- | +| Gateway (recommended) | `https://cloud-api.near.ai/v1` | Deterministic lookup with no retries. | Unattended or at-scale verification. | +| Pin the replica | The same `https://{slug}-i{N}.completions.near.ai` hostname used for lookup. | Deterministic while the index-to-backend binding remains stable. | Direct model-TEE requests when you control both calls. | +| Bounded sweep | The load-balanced `https://{slug}.completions.near.ai` domain. | Bounded search across `0..healthy-1`. | A `chat_id` already returned through the load-balanced domain. | + +--- + ## Signature Kinds Signatures fetched from the gateway (`GET https://cloud-api.near.ai/v1/signature/{chat_id}`) carry a `signature_kind` field that tells you which key signed and what the `text` payload contains: diff --git a/sidebars.js b/sidebars.js index 056eb3d..beb3468 100644 --- a/sidebars.js +++ b/sidebars.js @@ -101,6 +101,7 @@ const sidebars = { "cloud/guides/integrations/continue", "cloud/guides/integrations/cline-roo-kilo", "cloud/guides/integrations/aider-zed", + "cloud/guides/integrations/openclaw", "cloud/guides/opencode-goose", ], },