Skip to content

API.md

Latest commit

 

History

History
1309 lines (1034 loc) · 76.7 KB

API.md

File metadata and controls

1309 lines (1034 loc) · 76.7 KB
Raw

API Reference

This document lists the local HTTP API exposed by ima2 serve.

Base URL:

http://localhost:3333

Local and LAN access

The configured bind determines access mode. Loopback keeps single-user no-token use; a non-loopback bind requires IMA2_LAN_TOKEN, including requests arriving from a loopback proxy. LAN protects both /api and /generated before file, Range or conditional-response processing. UI/static assets remain public.

Endpoint Contract
GET /api/auth/lan/session No-store {mode, authenticated, expiresAt}; no token/cookie value returned
POST /api/auth/lan/session Same-origin Origin, JSON {}, valid x-ima2-token; 204 and an opaque HttpOnly/SameSite=Strict cookie in LAN mode
DELETE /api/auth/lan/session Same-origin Origin and presented cookie; revoke it and return 204

Sessions are in memory, origin-bound and expire after an absolute eight hours. Restart/rotation invalidates them. Settings → General → Studio session signs out; sign-out/expiry closes admitted streams but does not cancel accepted generation. Reauthentication does not automatically submit a waiting generation again. The browser removes a supplied ?token=... before loading App and never stores the token in local/sessionStorage; entering it in the sign-in form avoids putting it in initial URL/access logs. API clients can use x-ima2-token; legacy query tokens remain accepted. Header → query → cookie precedence applies. Malformed duplicates are rejected; an explicit invalid credential cannot fall back to a cookie.

Serving Host/Origin must match the actual listener or exact configured server.publicOrigins / IMA2_PUBLIC_ORIGINS JSON array (maximum 16). No wildcards, userinfo, paths, query or fragments. This is a serving-origin list, not CORS permission: cross-origin and same-site sibling-origin requests remain rejected. Forwarded headers do not establish trust. The case-compatible exact GET MCP OAuth callback retains its existing state/PKCE check; admin stop still needs its nonce.

For Vite development, use http://127.0.0.1:5173 and explicitly include that origin on the backend (IMA2_PUBLIC_ORIGINS='["http://127.0.0.1:5173"]'). The dev proxy preserves Host. If opening localhost instead, list that exact origin too; local mode canonicalizes it to 127.0.0.1, while LAN mode preserves the entered origin.

LAN media uses private, no-store, max-age=0, no ETag/Last-Modified, and same-origin resource policy. JSON sidecars and outside-root links are not served; SVG retains its restrictive CSP. Previously cached/downloaded copies cannot be revoked: purge operator-controlled proxy caches during upgrade. HTTP LAN is for trusted networks only; use TLS/VPN on untrusted networks. HTTPS sessions use Secure cookies. A malicious process on the operator host is outside this boundary.

Wrong/missing LAN credentials return 401 LAN_TOKEN_REQUIRED; invalid Host/Origin returns 403. Bootstrap is limited to ten failed attempts per socket peer per minute and shares the app API request budget. Respect Retry-After on 429; a full 256-session store returns 503 instead of evicting users. Tokens are bounded to 4096 UTF-8 bytes. Access errors contain fixed messages, never supplied secrets.

Provider Policy

Image generation supports OAuth, API-key, Grok, and Gemini (agy and gemini-api) providers.

  • provider: "oauth" calls ChatGPT's Codex backend from the server: a GPT-6 model plans the prompt and gpt-image-2 renders it.
  • provider: "api" uses the OpenAI Responses API with the hosted image_generation tool.
  • provider: "grok" calls https://api.x.ai directly with the xAI OAuth session in ~/.progrok/auth.json. Classic, Node and multimode image generation honor webSearchEnabled: false (Node also honors searchMode: "off"): this skips /v1/responses search, not the configured planner's forced generate_image call. Planner default is grok-4.3; grok-4.6 and grok-4.5 remain selectable overrides. References select /v1/images/edits instead of /v1/images/generations. Agent's omitted-option search default is unchanged.
  • provider: "agy" spawns the Antigravity CLI (agy -p) to generate images via Google Gemini's default_api:generate_image tool. Model is nano-banana-2. Output is fixed at 1024×1024 JPEG. Max 3 reference images (i2i). No web search, quality, size, or mask controls. Multimode returns a single image. Video is unsupported (AGY_VIDEO_UNSUPPORTED).
  • provider: "grok-api" uses a direct xAI API key with the same image search/planner/image semantics. Missing or blank image credentials are refused with GROK_API_KEY_MISSING before admission, never changed to the OAuth lane. Video remains a separate path. Configure the key in Settings or XAI_API_KEY.
  • provider: "gemini-api" calls the Google Generative Language API directly (or Vertex AI with a service account JSON). Supports models nano-banana-2 (Gemini 3.1 Flash Image) and nano-banana-pro (Gemini 3 Pro Image). Supports variable aspect ratios (1:1 through 21:9) and four resolution tiers (512px, 1K, 2K, 4K) on both auth paths — the direct API path sends generation_config.response_format.image (snake_case) while the Vertex AI endpoint (aiplatform.googleapis.com) sends generationConfig.imageConfig (camelCase). With size: "auto" the image config is omitted entirely and the model decides ratio/size. Auth: GEMINI_API_KEY env var, web UI key management (/api/keys/gemini), or a Vertex AI service account JSON (VERTEX_SERVICE_ACCOUNT_JSON or /api/keys/vertex). When both Vertex credentials and an API key are configured, Vertex takes priority. The chosen auth mode (apikey or vertex) persists to ~/.ima2/config.json as geminiAuthMode and is restored on server startup. Per-model cost: nano-banana-2 (Flash): 512=$0.001, 1K=$0.003, 2K=$0.004, 4K=$0.006; nano-banana-pro: 1K=$0.007, 2K=$0.007, 4K=$0.013. No web search or mask controls.
  • provider: "nai" calls NovelAI text-to-image generation with one of nai-diffusion-5-full, nai-diffusion-5-curated, nai-diffusion-4-5-full, or nai-diffusion-4-5-curated. It accepts the provider-native request fields documented below and returns a ZIP that ima2 decodes to PNG. References, edits, and masks are explicitly refused.
  • API-key generation covers classic generate, edit, mask-guided edit, multimode, and node generation.
  • If provider: "api" is requested without an API key, routes fail before upstream with 401 and API_KEY_REQUIRED.
  • Grok generation maps size to xAI aspect_ratio and resolution; it does not send an OpenAI-style size field upstream. Grok edit uses xAI /v1/images/edits; Grok mask edit remains unsupported and returns GROK_MASK_UNSUPPORTED.
  • Mask edits are mask/selection guided edits. The model may redraw outside the mask, so the edit route restores source pixels wherever the mask is opaque before saving (features.preserveOutsideMask, env IMA2_PRESERVE_OUTSIDE_MASK, default on; the sidecar records maskOutsidePreserved: true). The output keeps the source dimensions. When the provider returns a different aspect ratio, the result is stretched to the source size before compositing, so the edited area may look stretched or squeezed. Transparent source pixels in the kept area stay transparent. Composites run one at a time and refuse sources above 4096 × 4096 pixels. If compositing fails or the source is above that bound, the provider result is saved unchanged and maskOutsidePreserved is omitted.

Grok video generation uses POST /api/video/generate (SSE). See the Video Generation section below for the full endpoint specification.

Grok image artifact retrieval

Returned image URLs require HTTPS and a conservative numeric-address policy. Private/link-local/shared/loopback/documentation/transition destinations are rejected by default; every DNS answer and redirect hop is validated, and the connection uses only those validated addresses. No default re-resolution, connection pooling, API credential, cookie or referrer forwarding occurs.

The exact server-configured proxy origin is a local exception only when it is the initial artifact origin. Leaving it permanently removes that exception for the chain: public→private-proxy and proxy→public→private-proxy cannot regain trust. At most five redirects and50MiB are accepted; the single30s default deadline covers DNS, retries, redirects and streamed bytes. An omitted/false Content-Length does not bypass the streamed cap. HTTP GET may retry under the existing policy; billable image POST is not automatically retried.

The downloader reports GROK_IMAGE_DOWNLOAD_FAILED/502 for policy/body failures, GROK_IMAGE_TIMEOUT/504 for deadline, and GENERATION_CANCELED/499 for caller abort; existing surface error normalization still applies. Errors do not expose raw returned URL/query credentials. This restricted policy is not an exhaustive public-address/routability guarantee and does not change video or MCP URL policy.

Agy artifact reads and cleanup

Reported and fallback artifact paths must be absolute regular files within the canonical .gemini, .cache, or system temporary roots. Intentionally relocated root directories remain supported; missing roots never broaden the boundary. Leaf symlinks, paths escaping those roots, unusable file identities and detected path/file replacement are rejected. The same checks apply to parsed result paths and fallback discovery; filename matching alone is not acceptance.

The reader enforces an inclusive 50MiB limit before and during descriptor reads, using bounded blocks even when individual reads are short. A successful receipt authorizes cleanup only for that same checked file; replacements and unrelated siblings are left alone. Direct reader/operation cancellation returns GENERATION_CANCELED/499 after pending file I/O and descriptor close settle; existing node normalization can expose INVALID_REQUEST/499 instead. Policy-rejected artifacts are not deleted. The reader and Agy operation add no retries, but caller policies remain unchanged: reference-free node requests can retry once after retryable artifact errors, starting another generation attempt.

AGY_PATH_REJECTED/502 covers the path/identity policy; AGY_ARTIFACT_TOO_LARGE/502 covers overflow, and AGY_ARTIFACT_NOT_FOUND/502 covers missing reported files. Existing generation normalization may expose code: UNKNOWN while retaining the original rawCode and errorClass: INTERNAL_STATE_ERROR. Windows uses explicit path/identity checks without POSIX no-follow open flags. This is not an atomic filesystem sandbox against another same-user process, a hardlink-provenance guarantee, or a process-wide memory ceiling.

Health And Status

Core provider surface metadata

GET /api/capabilities includes providerSurfaces[coreProviderId][surface]. GET /api/models includes the same surfaces on each core lane; MCP lanes retain their existing model capabilities. Surfaces are generate, edit, multimode, node, and video.

Each record contains supported, references, mask, streaming, and catalogAccess (static or runtime). These are application capability facts, not credential/liveness or entitlement checks. references means the surface can accept image input; a Comfy workflow still needs the appropriate binding. streaming describes partial-image delivery on Node/Multimode, not lifecycle SSE.

NovelAI's generate/node/multimode entries are supported without references; edit/video are unsupported. Its models explicitly declare generation support, not fictitious edit support. Comfy is runtime-catalog backed: empty static model IDs do not disable its generate/edit/video surfaces, while node/multimode remain unsupported. Local capabilities expose the structural map without inventing a runtime lanes availability result.

Method Path Notes
GET /api/health Server health, version, paths, provider policy; includes grok: { auth: "oauth" | "none" } and the runtime identity bootId, launcher (foreground | background | service | desktop) and root, mirrored in ~/.ima2/server.json
POST /api/admin/stop Clean shutdown (local admin only): requires the boot-generated X-Ima2-Admin-Nonce from ~/.ima2/server.json; any request with an Origin header is refused (browser drive-by protection). Responds 202, prints IMA2_STOP_INTENT <bootId> on the server's stdout (the desktop app reads it to tell a requested stop from a crash), then runs the server's shutdown routine directly (a self-SIGTERM would skip it on Windows)
GET /api/providers Provider availability and runtime ports
GET /api/oauth/status GPT OAuth status and visible models, plus auth / grokAuth login verdicts: { provider, loggedIn, health: healthy|warning|reauth_required|not_logged_in, reason?, source?, email?, plan?, accountId (masked), expiresAt?, refreshable, action?, note? }. A proxy that exited because no session file exists reports auth_required.
GET /api/grok/status xAI OAuth session state and visible xAI image models: ready, no_image_model, error, or offline with reason: "login_required" when no session is stored
GET /api/billing Billing/status probe, including API key source when configured
GET /api/quota Provider quota: returns { codex, grok, nai }. Codex windows are 5h and 7d. NovelAI returns a v5-battery remaining-charge window when available (resetsAt is the ISO ETA for +1%, not full recharge), with Anlas balances in nai.anlasFixed / nai.anlasPurchased; a missing meter has no window. NovelAI account email is always null. Eligible Grok Build xAI OIDC/external auth returns a weekly percentage/reset window from GET /v1/billing?format=credits. If unavailable, the legacy endpoint may return a monthly window plus billing: { usedUsd, limitUsd }.

Account Switching

Method Path Notes
POST /api/auth/switch Start an OAuth login. Body: { "provider": "grok" | "codex", "flow"?: "browser" | "device" }. codex defaults to the browser flow (callback on localhost:1455, so the browser must be on the server machine); device works from any machine. Grok always uses the device code. Returns { sessionId, flow, userCode, verificationUrl, expiresIn }; userCode is empty for the browser flow. A completed codex login makes GPT OAuth pick up the new session.
GET /api/auth/switch/:sessionId Poll switch-account session status. Returns { status } where status is pending, complete, error, or expired.
DELETE /api/auth/switch/:sessionId Cancel a pending login and release its callback port.
POST /api/oauth/restart Make GPT OAuth pick up the current session file (it re-reads the file on every request; this clears the cached model roster) (used by ima2 login / ima2 gpt login / ima2 gpt logout). Returns { restarted, reason? }.

The Switch Account flow opens a browser verification URL. Once the user completes the device-code step, the server saves the new credentials (Grok: ~/.progrok/auth.json; Codex: via codex login --device-auth) and the session transitions to complete. This endpoint is surfaced as a Switch Account button in the Settings QuotaCard for Grok and Codex providers.

Storage

Method Path Notes
GET /api/storage/status Summarized gallery storage status for support UI
POST /api/storage/open-generated-dir Ask the server process to open the generated image folder

GET /api/storage/status returns a support-safe summary, not raw legacy path arrays by default.

{
  "ok": true,
  "data": {
    "generatedDirLabel": "~/.ima2/generated",
    "generatedCount": 0,
    "legacyCandidatesScanned": 18,
    "legacySourcesFound": 0,
    "legacyFilesFound": 0,
    "state": "not_found",
    "messageKind": "apology",
    "recoveryDocsPath": "docs/RECOVER_OLD_IMAGES.md",
    "doctorCommand": "ima2 doctor",
    "overrides": {
      "generatedDir": false,
      "configDir": false
    }
  }
}

Storage state values:

State Meaning
ok Current gallery has files or no recovery notice is needed
recoverable Legacy folders/files are still present and may be recoverable
not_found Current gallery is empty and no legacy folder was found
unknown Storage status inspection failed or was incomplete

POST /api/storage/open-generated-dir opens the generated image folder on the machine running ima2 serve. If the browser is connected to a remote server, VM, container, WSL instance, or another computer on the network, this action targets that server machine, not necessarily the browser device.

In-Flight Jobs

Method Path Notes
GET /api/inflight Active jobs only by default
GET /api/inflight?includeTerminal=1 Includes recent terminal jobs for debugging
DELETE /api/inflight/:requestId Cancel or forget an active job
GET /api/events Persistent SSE multiplex channel for all async generation progress (see below)

In-flight logs and responses use requestId for correlation. Logs should not include raw prompts, reference data URLs, generated base64, tokens, cookies, auth headers, or raw upstream bodies.

Events (SSE Multiplexing)

GET /api/events (SSE Multiplexing)

Single persistent Server-Sent Events channel that carries progress for all async generation jobs. The browser UI opens one EventSource here instead of holding a per-request SSE connection for each job, avoiding browser per-origin connection limits.

Query Notes
lastEventId Optional. Reconnect cursor; also accepted via the Last-Event-ID request header

Response: text/event-stream (persistent). Each frame uses standard SSE fields id, event, and data (JSON).

Connection limits: When active listeners reach 512, the server returns 503 with SSE_CAPACITY before opening the stream.

Heartbeat: Every 15 seconds the server writes a comment frame:

: ping

Replay: On reconnect, the server replays events from an in-memory ring buffer (size 2000) for IDs newer than lastEventId. Large image payloads (>1000 characters) are omitted from replay with _imageOmitted: true in the data payload. If the requested ID is older than the oldest buffered event, the server emits a replay-gap event before live fan-out:

Event Data Description
replay-gap { lastEventId, oldestAvailableId } Client should reconcile inflight state (for example via GET /api/inflight)

Job routing: Every data payload includes jobId (same value as the job's requestId). Event bodies also carry requestId where applicable. Clients filter events by matching data.jobId or data.requestId to the job they started.

Event types (fan-out to all connected clients):

Event Emitted by Description
phase node, multimode, video Lifecycle phase change
partial node, multimode Progressive preview image (base64 data URL)
image multimode Final saved GenerateItem for one sequence image
done node, multimode, video Terminal success payload (route-specific shape)
error all generation routes Terminal failure
submitted video Job submitted to xAI
progress video Progress fraction 0.0–1.0
planning video Video planner running

Example SSE frame:

id: 42
event: phase
data: {"requestId":"req_abc","jobId":"req_abc","phase":"streaming"}

Async generation mode

POST /api/node/generate, POST /api/generate/multimode, and POST /api/video/generate support an async POST mode for clients that already hold GET /api/events:

{
  "async": true,
  "requestId": "req_xxx",
  "...": "other route fields"
}
Outcome HTTP Body
Accepted 202 { "requestId": "req_xxx" }
Duplicate active requestId 409 REQUEST_ID_IN_USE
More than the configured concurrent active job limit 429 TOO_MANY_JOBS with Retry-After: 5; default limit is 24 via IMA2_MAX_PARALLEL

Progress events are published on GET /api/events. The POST response returns immediately; clients must not expect SSE on the POST connection when async: true.

CLI and legacy clients omit async and keep the original behavior: per-request SSE on the same POST response (Accept: text/event-stream where applicable). The server dual-emits in that mode — it writes SSE to the POST response and also publishes the same events on GET /api/events.

Generation

Sprite Atlas

Sprite atlas imports require both a sprite-gen-compatible manifest and a PNG atlas. Unknown manifest fields are preserved during read/write round trips.

Method Path Notes
POST /api/sprite-atlas/import JSON { manifest, atlasBase64, runId?, name? }; validates explicit rects and creates a sprite run plus representative image asset.
GET /api/sprite-atlas/:runId Returns manifest, optional curation, and atlas URL.
PUT /api/sprite-atlas/:runId/curation Stores sprite-gen curation v1 atomically without changing source frames.
POST /api/sprite-atlas/:runId/unpack Extracts frames using manifest rects.
POST /api/sprite-atlas/:runId/bake Applies curation and rebuilds atlas, manifest, and report.
POST /api/sprite-atlas/:runId/export/contact-sheet Body { state, columns? }; creates a PNG contact sheet.
POST /api/sprite-atlas/:runId/export/gif Body { state, fps?, loop? }; creates and decode-validates a transparent GIF through ffmpeg.

Import without a manifest returns SPRITE_MANIFEST_REQUIRED. GIF export returns FFMPEG_UNAVAILABLE with HTTP 503 when ffmpeg is unavailable.

POST /api/generate

Text-to-image and reference-guided root generation.

{
  "prompt": "a shiba in space",
  "quality": "medium",
  "size": "1024x1024",
  "format": "png",
  "moderation": "low",
  "provider": "oauth",
  "model": "gpt-5.4",
  "references": [],
  "requestId": "optional-client-id",
  "storyboard": false
}

Supported quality values: low, medium, high.

For provider: "api", generation, edit, multimode, and node generation also accept optional imageToolModel: gpt-image-2.5-sunburst or gpt-image-2.5-flare. It selects tools[].model for the Responses image-generation tool; model still selects the outer GPT reasoning model. Omit the field (or send null/empty) for upstream selection and the existing defaults. Other providers ignore it. Unknown API values return INVALID_IMAGE_TOOL_MODEL (400, or the existing synchronous multimode SSE error). Only an explicit API 2.5 selection enables quality xhigh and max; elsewhere those values normalize to medium with QUALITY_DEFAULTED. Sidecars, history, generation results and existing XMP embedding paths retain the selected imageToolModel. Edit continues to preserve upstream bytes and uses sidecar metadata. GET /api/capabilities advertises the API-only tool model/quality allowlist separately from the outer models in /api/models.

Supported moderation values: auto, low.

When storyboard is true, the server prepends storyboard keyframe instructions so image generations maintain character and scene continuity for multi-shot video production.

GPT OAuth default: gpt-6-luna (also gpt-6-sol and gpt-6-astra; legacy OAuth ids map to their GPT-6 tier). API-key default: gpt-5.6-luna; gpt-5.5 and the other API models remain available when callers select them.

When provider is "nai", classic, multimode, and node generation accept the same 13 provider-native fields: negativePrompt, sampler, noiseSchedule, steps, scale, cfgRescale, seed, ucPresetId, qualityPresetId, autoSmea, decrisper, varietyPlus, and straightAlpha. Missing values stay sparse and resolve from config.naiProvider; operator defaults defaultAutoSmea and defaultDecrisper are false unless overridden by config or IMA2_NAI_DEFAULT_AUTO_SMEA / IMA2_NAI_DEFAULT_DECRISPER. Quality preset and enabled alpha are V5-only. The four exact image models are nai-diffusion-5-full, nai-diffusion-5-curated, nai-diffusion-4-5-full, and nai-diffusion-4-5-curated. The lane is text-to-image only: NAI_REF_UNSUPPORTED, NAI_EDIT_UNSUPPORTED, and NAI_MASK_UNSUPPORTED fail closed rather than discarding input.

When provider is "grok", supported models include grok-imagine-image-2.0, grok-imagine-image and grok-imagine-image-quality. The server uses grok-4.3 as the configured search/planner default (IMA2_GROK_PLANNER_MODEL) and times search and planner steps separately from the image call (IMA2_GROK_PLANNER_TIMEOUT_MS). For n > 1, search and planning run once and the planned prompt is reused for the image requests. Search-disabled classic generations report zero search calls; otherwise the shared plan reports one logical search call.

If references are present on a Grok classic request, ima2 still performs the configured planning phase, with search only when enabled. The planner receives the reference images as multimodal image_url inputs, and its forced generate_image.prompt argument is instructed to be English-only except for exact visible text requested by the user. The final image call then uses xAI /v1/images/edits with the same reference images instead of /v1/images/generations. This keeps image-to-image/reference context alive through the pipeline. The server permits up to three source images for these Grok classic requests; more than three references return GROK_REF_TOO_MANY.

Grok size mapping:

Requested size xAI aspect_ratio xAI resolution
1024x1024 1:1 1k
1536x1024 3:2 1k
1024x1536 2:3 1k
1360x1024 4:3 1k
1024x1360 3:4 1k
1824x1024 16:9 1k
1024x1824 9:16 1k
2048x2048 1:1 2k
2048x1152 16:9 2k
1152x2048 9:16 2k
3840x2160 16:9 2k
2160x3840 9:16 2k
auto auto omitted

Custom sizes are reduced to the closest xAI-supported aspect ratio and use 2k when the requested longest edge or pixel budget is closer to a 2K image.

POST /api/edit

Image edit / image-to-image generation.

The request includes a prompt and image payload. provider: "api" sends the prompt and image through the shared Responses image adapter. Optional masks are forwarded as mask guidance; after the provider returns, pixels where the mask is opaque are restored from the source image (see IMA2_PRESERVE_OUTSIDE_MASK).

With provider: "grok", edit requests are sent to xAI /v1/images/edits with the stored OAuth session. Masked Grok edits are rejected before upstream with GROK_MASK_UNSUPPORTED.

Grok multimode plans each attempted image in order and performs search only when enabled. Sparse successes retain their original attempt index: a failed first attempt cannot make the final sweep persist the second image twice. Identical bytes at different successful indices remain separate outputs. The internal originalIndexes array is not added to request/SSE/history/sidecar schemas.

POST /api/node/generate

Node-mode generation and child edits.

Body fields:

{
  "parentNodeId": "optional-server-node-id",
  "extraParentNodeIds": ["optional-reference-node-id"],
  "prompt": "continue this image",
  "quality": "medium",
  "size": "1024x1024",
  "format": "png",
  "moderation": "low",
  "model": "grok-imagine-image",
  "references": [],
  "externalSrc": "optional-history-url",
  "sessionId": "session-id",
  "clientNodeId": "client-node-id",
  "requestId": "request-id",
  "provider": "grok"
}

When parentNodeId is present, the server loads the stored parent node image and uses the edit path. Node-local references are allowed on both root and child/edit nodes; for child/edit nodes the parent image is sent first, then references, then the text prompt.

extraParentNodeIds adds stored node images as ordered references before explicit references. Duplicate IDs and the base parent ID are removed; a missing requested extra image fails the request. Malformed IDs, per-reference size overflow, and the combined extra-plus-explicit reference cap are rejected before provider work. The existing configured reference limits apply, followed by provider-specific admission counting the base image too. parent-only does not load extra parents; existing provider-specific handling of explicit user references is unchanged. Graph connections preserve their saved order: the first image connection supplies the base and the remaining image connections supply references.

Grok Node Mode uses the configured planner and Images API, with search suppressed by searchMode: "off" or webSearchEnabled: false. A parent node image, externalSrc, or extra references are passed to the planner and then to /v1/images/edits; otherwise the final call uses /v1/images/generations. The server caps total input images at three, counting parent/current image plus references, and returns GROK_REF_TOO_MANY before upstream when exceeded. Existing quality-model resolution is unchanged.

The route can stream Server-Sent Events when the client sends Accept: text/event-stream. Possible events include phase, partial, done, and error. Alternatively, send { "async": true, "requestId": "req_xxx" } in the body to receive 202 { requestId } immediately and follow progress on GET /api/events (see Events section).

Grok Node SSE responses do not include Responses API partial image events because the xAI Images API call is synchronous JSON. They still emit phase and done/error events so the Node UI can use the same in-flight lifecycle.

POST /api/generate/multimode (SSE)

Multi-image sequence generation. SSE-only on the POST response unless async mode is used.

{
  "prompt": "a story in four panels",
  "maxImages": 4,
  "quality": "medium",
  "size": "1024x1024",
  "format": "png",
  "moderation": "low",
  "model": "gpt-5.4",
  "provider": "oauth",
  "references": [],
  "requestId": "optional-client-id",
  "async": false
}

Send Accept: text/event-stream for per-request SSE on the POST connection. Or set "async": true with a client requestId to get 202 { requestId } and receive events on GET /api/events.

SSE events:

Event Data Description
phase { requestId, phase, sequenceId?, maxImages? } Lifecycle phase
partial { requestId, image, index } Progressive preview
image full GenerateItem One saved sequence image
done route-specific summary; may include status: "partial" after timeout if at least one image was saved Sequence complete
error { requestId, error, code?, status? } Generation failed

GET /api/node/:nodeId

Fetch stored node metadata and asset URL.

Reference Images

Reference uploads are capped at 5 items. The frontend compresses large JPEG/PNG files before sending them. HEIC/HEIF files are rejected with a user-facing conversion hint.

Server-side validation may return these reference codes:

Code Meaning
REF_NOT_ARRAY references was not an array
REF_TOO_MANY More than the configured reference count
REF_NOT_STRING A reference item was not a string
REF_EMPTY A reference item was empty
REF_TOO_LARGE A reference exceeded the configured base64 size
REF_NOT_BASE64 A reference was not valid base64
GROK_REF_TOO_MANY Grok classic generation received more than three reference images
GROK_MASK_UNSUPPORTED Grok edit was requested with a mask; xAI mask edit is not wired in this release

Video Generation

POST /api/video/generate (SSE)

Generate a video via the Grok video provider. Returns Server-Sent Events on the POST connection, or accepts async mode ({ "async": true, "requestId": "req_xxx" }) for 202 { requestId } with progress on GET /api/events (see Events section).

{
  "prompt": "a cat playing piano",
  "provider": "grok",
  "model": "grok-imagine-video",
  "duration": 5,
  "resolution": "480p",
  "aspectRatio": "auto",
  "sourceImage": "<base64>",
  "referenceImages": ["<base64>", "<base64>"],
  "referenceFilenames": ["existing-file.png"],
  "continueFromVideo": "1780226256355_50252101.mp4",
  "continuityLineage": { "lineageId": "optional-client-hint", "entries": [] },
  "sessionId": "optional",
  "requestId": "optional-client-id"
}

Models: grok-imagine-video-1.5 (default), grok-imagine-video. The legacy grok-imagine-video-1.5-preview string is accepted as a compatibility alias and normalized before the upstream request.

Mode is auto-detected from reference inputs:

Inputs Mode Duration cap
No images text-to-video 1–15s
1 image (sourceImage or sourceFilename) image-to-video 1–15s
2–14 images (referenceImages / referenceFilenames) reference-to-video 1–15s on grok-imagine-video-1.5, 1–10s on grok-imagine-video

1080p is accepted for grok-imagine-video-1.5 prompt-only text-to-video and image-to-video with one image/frame source, including continueFromVideo after the server extracts the parent video's last frame. Prompt-only 1.5 text-to-video uses the internal white-canvas image-to-video shim before the upstream request. 1.5 does not add Ref2V, V2V edit, or extension support.

Parameters:

Field Type Default Notes
prompt string — Required
provider string "grok" "grok" or "grok-api"
model string grok-imagine-video-1.5 Video model
duration integer 5 1–15 seconds (clamped to 10 for reference-to-video)
resolution string "480p" 480p, 720p, or 1080p (1080p uses 1.5 T2V canvas shim or I2V)
aspectRatio string "auto" 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3, auto
sourceImage string — Base64 image for image-to-video
sourceFilename string — Existing generated file for image-to-video
referenceImages string[] — Base64 images for reference-to-video
referenceFilenames string[] — Existing generated files for reference-to-video
continueFromVideo string — Generated .mp4 parent; server extracts its last frame and rebuilds lineage from sidecar
continuityLineage object — Optional client hint; used only when continueFromVideo is absent
plannerModel string grok-4.3 Grok video planner model override; grok-4.6 and grok-4.5 are selectable (also via settings UI or IMA2_GROK_PLANNER_MODEL)
storyboard boolean false Enable storyboard mode — maintains character/scene continuity across sequential clips

Blank prompts return PROMPT_REQUIRED with a guidance string. The active prompt should describe visual flow, motion flow, sound/music/no-music, dialogue/no-dialogue, ending frame, and duration pacing. The video planner uses the selected duration as the full clip runtime and expands short requests into a production-level sequence with opening composition, connected motion/emotion change, and a stable ending frame suitable for continuation. For multi-character scenes, the planner identifies speakers by visual appearance (clothing, physique, position, props) rather than names, and attributes each dialogue line accordingly.

When continueFromVideo is present, the server treats the generated .mp4 sidecar as authoritative. Client continuityLineage cannot override it. The saved child sidecar includes videoContinuity, a branch-local max-4 stack using keep-start-plus-latest-3 retention.

videoContinuity shape:

{
  "lineageId": "lineage:parent",
  "parentFilename": "parent.mp4",
  "sourceFrame": "last",
  "maxEntries": 4,
  "retention": "keep-start-plus-latest-3",
  "entries": [
    {
      "id": "clip:parent.mp4",
      "ordinal": 1,
      "role": "start",
      "filename": "parent.mp4",
      "userPrompt": "original user prompt",
      "revisedPrompt": "planner prompt actually sent to Grok video",
      "createdAt": 1780300000000
    }
  ]
}

Entry role is start, ancestor, parent, or current. The first clip is kept as the start anchor; later generations keep only the latest three entries. lineageId uses the generated video basename without the .mp4 extension. This metadata is stored in the generated .mp4.json sidecar and returned in history rows and video done events; /generated/*.json remains private.

Grok prompt surfaces used by video APIs:

Surface Model Responsibility
Video planner grok-4.5 (override via plannerModel) Converts user prompt, search context, refs, and optional continuity lineage into the final English video prompt. It must structure core subject, action/motion, camera/composition, environment/style, dialogue/audio, ending-frame handoff, and constraints. Multi-character dialogue uses appearance-based speaker identification.
Video generation xAI video model Receives the planner prompt plus sourceImage or referenceImages when present.
Video analysis grok-4.5 Reads first/last frame images from /api/video/analyze and returns recreation/continuation guidance.

SSE events:

Event Data Description
planning { requestId } Preparing video generation
submitted { requestId, xaiVideoRequestId, requestedModel, effectiveModel, modelFallback } Submitted to xAI
progress { requestId, progress, stalled } Progress 0.0–1.0
done { requestId, filename, url, mediaType, revisedPrompt, elapsed, usage, requestedModel, effectiveModel, modelFallback, video, videoContinuity } Video ready
error { error, code, status, requestId, guidance? } Generation failed

Video error codes:

Code Meaning
VIDEO_PROVIDER_UNSUPPORTED Provider is not "grok" or "grok-api"
PROMPT_REQUIRED Empty or missing prompt
INVALID_GROK_VIDEO_MODEL Model not in valid set
INVALID_VIDEO_RESOLUTION Resolution is not 480p/720p/1080p, or 1080p was requested outside grok-imagine-video-1.5 prompt-only T2V / I2V
INVALID_VIDEO_ASPECT_RATIO Aspect ratio not in valid set
INVALID_VIDEO_DURATION Duration not 1–15 integer
GROK_VIDEO_REF_TOO_MANY More than 7 reference images
GROK_VIDEO_FAILED Upstream xAI video generation failed
GROK_VIDEO_FRAME_FAILED Server could not extract the parent video's last frame
GROK_VIDEO_DOWNLOAD_FAILED Download-stage HTTP, size, MIME, empty-body, MP4-prefix or read failure (502)
GROK_VIDEO_TIMEOUT The download's own non-resetting timeout expired (504)
GENERATION_CANCELED Caller cancellation observed while downloading, including custom reasons (499)

Completed Grok video artifacts are consumed incrementally with an inclusive 100 MiB byte cap. Declared size and actual streamed bytes are checked before retaining an overflowing chunk; invalid/empty/aborted bodies do not reach caller persistence. Accepted MIME is MP4 or octet-stream (missing MIME defaults to MP4), and the existing minimum-length/ftyp prefix check is retained. This is not full MP4 decoding or a100MiB process-memory ceiling: fetch buffers, chunk overhead and downstream copies are separate. Safe GET retry ends at response headers; body failure never starts a new billed generation.

The video destination policy is unchanged: initial HTTPS or the existing literal HTTP-loopback allowlist, with fetch's default redirect behavior. It does not gain the image downloader's DNS/private-address/per-hop pinning policy. Do not treat HTTPS alone or these body bounds as an SSRF/security guarantee. Download error status appears in the existing JSON or SSE/terminal envelope for each caller.

POST /api/video/edit

Edit an existing video via Grok V2V. This is a blocking JSON endpoint that starts the xAI edit job, polls it, downloads the final MP4, and saves it as a generated video artifact.

{
  "prompt": "make it sunset",
  "videoUrl": "https://vidgen.x.ai/.../clip.mp4",
  "model": "grok-imagine-video"
}

videoUrl may be an HTTPS video URL, xAI file_id, data:video/* URL, or generated .mp4 filename. Generated-file inputs are restricted to real .mp4 files under the generated directory.

POST /api/video/extend

Extend a video from its last frame (last-frame→I2V orchestration). This is an async job endpoint: it returns HTTP 202 immediately and streams lifecycle events (queued → extracting-frame → planning → submitted/progress → persisting → done or error) over GET /api/events. The server extracts the parent video's last frame, injects it as the image-to-video source, and records durable lineage on the child artifact.

{
  "sourceVideoId": "1780226256355_50252101.mp4",
  "requestId": "vext_optional",
  "prompt": "camera pulls back (optional — inherits parent prompt when empty)",
  "provider": "grok",
  "model": "grok-imagine-video",
  "duration": 6
}

Immediate response:

{ "ok": true, "requestId": "vext_...", "sourceVideoId": "1780226256355_50252101.mp4", "workflow": "last-frame-i2v" }

The terminal done payload carries video.operation: "extend", video.sourceFrame: "last", and videoLineage (id, parentId, rootId, seriesId, sequenceIndex). Duplicate requestId returns 409. Frame-extraction failures map to VIDEO_FRAME_EXTRACT_UNAVAILABLE (503), VIDEO_FRAME_EXTRACT_TIMEOUT (504, retryable), or VIDEO_FRAME_EXTRACT_FAILED (500).

POST /api/video/extend/native

Legacy provider-native extension (blocking JSON). Starts the xAI extension job, polls it, downloads the combined output MP4, and saves it as a generated video artifact. Prefer /api/video/extend for new integrations.

{
  "prompt": "camera pulls back",
  "videoUrl": "1780226256355_50252101.mp4",
  "duration": 6,
  "model": "grok-imagine-video"
}

duration must be an integer from 2 to 10 seconds. Edit and native extension support grok-imagine-video only; grok-imagine-video-1.5 and its preview alias are not accepted for these endpoints.

GET /api/video/frame

Extract a PNG frame from a generated .mp4 file.

Query Notes
file Required generated .mp4 filename or generated-dir absolute path
position last (default) or non-negative seconds

POST /api/video/analyze

Analyze first and last frames from a generated .mp4 using the configured planner model (grok-4.5 by default). This does not upload the video as temporal video; it extracts two PNG frames and asks the vision model to infer likely motion.

{
  "videoUrl": "1780226256355_50252101.mp4"
}

Remote URLs and data: inputs are intentionally rejected to avoid server-side URL fetching through ffmpeg.

Generation Request Log

Method Path Notes
GET /api/generation-requests Returns { items: GenerationRequestLogEntry[] } — the last 200 generation attempts (prompt, requested/succeeded flags, error). Surfaced in the web UI dev panel (GenerationRequestLogPanel); no CLI wrapper (#95).

History

Method Path Notes
GET /api/history List generated assets
GET /api/history?groupBy=session Group assets by session title
DELETE /api/history/:filename Tombstone a generated asset
POST /api/history/:filename/restore Restore a recently deleted asset

History rows can include node metadata such as sessionId, nodeId, clientNodeId, requestId, and refsCount.

Assets Library

Persistent library catalog over generated files (phase 050). Records reference files inside generated/; deleting an asset never deletes the file.

Method Path Notes
GET /api/assets List/search assets (kind, folderId, tag, q, cursor, limit)
GET /api/assets/:id Fetch one asset by ID; returns 404 ASSET_NOT_FOUND when absent
POST /api/assets Promote/create an asset (filePath, kind, name?, folderId?, tags?, metadata?)
POST /api/assets/promote-element Promote a gallery result to an element asset (result.path or filePath, elementKind, name?, notes?, folderId?, tags?)
POST /api/assets/derived Save a derived asset (raw image/png body; query source, kind=keyed-png, projectId?, name?, meta? JSON) — writes <src>-keyed-<ts>.png + sidecar with derivedFrom and registers an asset record kind=vector-svg sends NO body: the server reads the validated source from generated storage, traces it via lib/vectorizeImage.ts, and writes <src>-vector-<ts>.svg (query preset?, colorPrecision?, filterSpeckle?, cornerThreshold?; a non-raster source is refused with DERIVED_SOURCE_NOT_RASTER). Traced SVGs are served with a restrictive CSP and nosniff.
POST /api/video/keying Derive an alpha WebM from a generated green-screen mp4 (source, keyParams{tolerance,softness,keyColor?}, projectId?, name?) — responds 202 {requestId, filePath}, publishes keying-start/progress/done/error on the event bus, writes sidecar with derivedFrom and registers a video asset
PATCH /api/assets/:id Update name/folder/notes/tags/metadata
POST /api/assets/:id/test-sheet Run an element test sheet; currently returns 501 TEST_SHEET_NOT_IMPLEMENTED after validating the element asset
DELETE /api/assets/:id Delete the catalog row only (file untouched)
DELETE /api/assets/all Delete all asset records (files untouched)
GET /api/assets/folders List folders (flat; tree assembled client-side)
POST /api/assets/folders Create folder (name, parentId?)
PATCH /api/assets/folders/:id Rename/move folder (cycle-safe)
DELETE /api/assets/folders/:id Delete an empty folder
GET /api/assets/tags Distinct tags

kind is one of image | video | element | preset | template. filePath is required for image/video, must stay inside generated/, and is stored relative to it. Cursor pagination orders by created_at DESC, id DESC; errors use the standard envelope with codes such as INVALID_ASSET_KIND, INVALID_FILENAME, INVALID_PARENT, FOLDER_CYCLE, FOLDER_NOT_EMPTY.

Sessions And Graphs

Method Path Notes
GET /api/sessions List graph sessions
POST /api/sessions Create a session
GET /api/sessions/:id Load a session and graph
PATCH /api/sessions/:id Rename a session
DELETE /api/sessions/:id Delete a session
PUT /api/sessions/:id/graph Save graph snapshot

PUT /api/sessions/:id/graph requires an If-Match header containing the current graph version.

Version mismatch returns GRAPH_VERSION_CONFLICT and the current version. This only means the client saved against a stale graph version; it is not proof that another browser tab changed the graph.

Node Templates

Node graph templates (higgsfield 120). Seed templates ship with the app and are read-only; user templates are created from the canvas.

Method Path Notes
GET /api/node-templates List template summaries (seed + user)
POST /api/node-templates Create a user template (201 { template })
POST /api/node-templates/:id/instantiate Return a graph copy with fresh node IDs (never auto-runs)
PATCH /api/node-templates/:id Rename a user template (seed → 403)
DELETE /api/node-templates/:id Delete a user template (seed → 403)
GET /api/node-templates/:id/export Download a portable template file (seed or user) as attachment; filename="<slug>.ima2-template.json"
POST /api/node-templates/import Create a user template from a portable file (201 { template }); never instantiates or runs

Portable template files

A file is { kind: "ima2.node-template", version: 1, exportedAt, sourceId?, name, description, tags, graph }. sourceId is informational; imports always get a new id, and a taken name becomes Name (2), Name (3)… within 80 characters.

Both directions rebuild the graph from allowlists. Nodes keep only id, type, position, width, height and data; edges keep id, source, target, handles and label. data keeps prompt, provider, model, size, kind, node type, reasoning effort, variation, style, element name, video settings and unresolved media placeholders, with status reset to idle. Runtime ids, generated media URLs, filenames, secrets and any other key are dropped. Element nodes come back with missing: true because their asset ids only exist on the exporting machine.

Import limits: 2 MB of request bytes (its own parser, ahead of the global body limit), the smaller of 300 nodes/1200 edges and the server's graph limits, 16 levels of nesting, names of 1-80 characters, descriptions up to 2000 characters, 20 tags of up to 40 characters. Structural problems are rejected rather than repaired. Errors use fixed messages and never echo file content:

Code Status Meaning
TEMPLATE_FILE_INVALID 400 Body is not JSON of the expected shape (including parser and charset errors)
TEMPLATE_FILE_KIND 400 Not an ima2.node-template file
TEMPLATE_FILE_VERSION 400 version is not exactly 1
TEMPLATE_FILE_TOO_LARGE 413 Over 2 MB, or too many nodes or edges
INVALID_TEMPLATE_NAME 400 Empty or over 80 characters
INVALID_TEMPLATE_GRAPH 400 No nodes, bad ids or positions, duplicate ids, dangling edges, cycles, forbidden keys (__proto__, constructor, prototype) or excess nesting

Graph save requests may include observability headers:

X-Ima2-Graph-Save-Id
X-Ima2-Graph-Save-Reason
X-Ima2-Tab-Id

Style Sheets

Method Path Notes
GET /api/sessions/:id/style-sheet Load session style sheet
PUT /api/sessions/:id/style-sheet Save style sheet
PATCH /api/sessions/:id/style-sheet/enabled Toggle style sheet usage
POST /api/sessions/:id/style-sheet/extract Extract style fields from prompt/reference

Style-sheet extraction can require an API key/openai client. Image generation also supports provider: "api" through the shared Responses API image adapter when an API key is configured.

Prompt Library

Backed by routes/prompts.ts and SQLite prompt tables in lib/db.ts.

Method Path Notes
GET /api/prompts List prompts (folderId, q, favoritesOnly, pagination)
POST /api/prompts Create prompt
GET /api/prompts/:id Fetch one prompt
PATCH /api/prompts/:id Update prompt fields
DELETE /api/prompts/:id Delete prompt
POST /api/prompts/:id/favorite Toggle favorite
POST /api/prompts/import Legacy bulk import (JSON body)
GET /api/prompts/export Export prompt library JSON
GET /api/prompts/folders List folders
POST /api/prompts/folders Create folder
PATCH /api/prompts/folders/:id Rename folder
DELETE /api/prompts/folders/:id Delete folder

Prompt Import

Preview/commit import flow for local files, GitHub folders, curated sources, and discovery review. Implemented in routes/promptImport.ts.

Method Path Notes
GET /api/prompts/import/curated-sources List curated source registry entries
GET /api/prompts/import/discovery List discovery review queue
POST /api/prompts/import/discovery-search Search GitHub for prompt-pack candidates
POST /api/prompts/import/discovery-review Approve/reject discovery candidate
POST /api/prompts/import/curated-search Search indexed curated sources
POST /api/prompts/import/curated-refresh Refresh curated index cache
POST /api/prompts/import/folder-files List files in a GitHub folder
POST /api/prompts/import/folder-preview Preview selected GitHub folder files
POST /api/prompts/import/preview Preview local/GitHub import candidates
POST /api/prompts/import/commit Commit selected candidates into the prompt library

Card News (dev-gated)

Registered only when config.features.cardNews is true (routes/cardNews.ts). Web UI requires VITE_IMA2_CARD_NEWS=1 or VITE_IMA2_DEV=1; CLI uses ima2 cardnews ….

Method Path Notes
GET /api/cardnews/image-templates List image templates
GET /api/cardnews/image-templates/:templateId/preview Template preview image
GET /api/cardnews/role-templates Built-in role templates
GET /api/cardnews/sets List card-news sets
GET /api/cardnews/sets/:setId Fetch one set
GET /api/cardnews/sets/:setId/manifest Set manifest JSON
POST /api/cardnews/draft Create planner draft
POST /api/cardnews/generate Start card generation job
POST /api/cardnews/jobs Create job record
GET /api/cardnews/jobs/:jobId Poll job status
POST /api/cardnews/jobs/:jobId/retry Retry failed job
POST /api/cardnews/cards/:cardId/regenerate Regenerate one card
POST /api/cardnews/export Export completed set assets

GitHub Star

The studio's one-time star prompt stars the repository with the user's own gh login; ima2 never stores a GitHub token. These routes answer only to this computer (loopback peers); LAN clients get 403 LOCAL_ONLY. The prompt state file is shared with the CLI prompt (<configDir>/state/star-prompt.json).

Method Path Result
GET /api/github/star { state: "starred" | "not-starred" | "unauthenticated" | null, prompted, repo, url } — unauthenticated when gh is missing or logged out; null once prompted (gh is no longer asked)
POST /api/github/star Stars via gh api -X PUT /user/starred/lidge-ai/ima2-gen and marks prompted; 409 GH_UNAUTHENTICATED, 502 GH_FAILED
POST /api/github/star/dismiss Marks prompted without starring

Updates

npm installs check the registry for a newer ima2 while the server runs (at most once every 20 hours, hourly retries with backoff; preview installs follow the preview tag). State lives under the config dir: version.json (check results), update-dismissed.json and update-seen/<version> markers. Servers launched by the desktop app report surface: "desktop", never check npm and never claim notices; the app updates itself. IMA2_DISABLE_UPDATE_CHECK=1 stops the automatic check only.

Method Path Result
GET /api/update/badge { surface, enabled, currentVersion, latestVersion, available, dismissed, stale, checkedAt, tag, command, releaseUrl, noticePending } from the cache, no network. The web UI receives the same badge as an update frame when it opens GET /api/events (npm servers only, no id), then re-reads this route every 10 minutes and claims the notice only when noticePending
POST /api/update/check Forced registry check, then the badge; 503 UPDATE_CHECK_FAILED with { message, badge } on failure
POST /api/update/dismiss Body { version }; hides that version's badge; 400 INVALID_VERSION
POST /api/update/notice { updatedTo: string | null } — the one-time "updated to vX" claim (null on first run, downgrade or repeat)

Common Error Codes

Code Meaning
API_KEY_REQUIRED provider: "api" was requested without a configured API key
APIKEY_DISABLED Legacy/deprecated hard-block code from older builds
INVALID_IMAGE_MODEL Model name is unknown or unsupported
IMAGE_MODEL_UNSUPPORTED Model exists but cannot use image generation
INVALID_REQUEST Upstream request parameters are invalid; raw provider details may be included as upstreamCode, upstreamType, and upstreamParam
INVALID_MODERATION Moderation value is not auto or low
SAFETY_REFUSAL Upstream safety refusal
MODERATION_REFUSED Content generation refused by moderation
AUTH_CHATGPT_EXPIRED Codex/ChatGPT OAuth session expired
AUTH_API_KEY_INVALID API key is invalid, revoked, out of quota, or wrong org
NETWORK_FAILED Network, proxy, VPN, or firewall failure
OAUTH_UNAVAILABLE GPT OAuth is not available
OPEN_GENERATED_DIR_FAILED The server could not open the generated image folder
GRAPH_VERSION_REQUIRED Missing graph If-Match header
GRAPH_VERSION_CONFLICT Stale graph version
GRAPH_TOO_LARGE Graph exceeds node/edge limits
NODE_NOT_FOUND Node metadata was not found
INVALID_GROK_IMAGE_MODEL A Grok request used a model outside grok-imagine-image or grok-imagine-image-quality
GROK_RATE_LIMITED xAI returned a rate-limit response
GROK_AUTH_FAILED The xAI request could not be authenticated
GROK_SEARCH_TIMEOUT / GROK_PLANNER_TIMEOUT / GROK_IMAGE_TIMEOUT The Grok search, planner, or image API step exceeded its timeout budget
AGY_GENERATION_FAILED Gemini (agy) image generation failed
AGY_TIMEOUT Agy CLI process exceeded its 360-second timeout
AGY_PROCESS_ERROR Agy CLI binary failed to start or crashed
AGY_QUOTA_EXHAUSTED Gemini API quota exhausted (rate limit)
AGY_PARSE_FAILED Could not parse artifact path from agy output
AGY_ARTIFACT_NOT_FOUND Agy reported an artifact path that does not exist
AGY_PATH_REJECTED Agy artifact path, file type, or identity failed the allowed-root policy
AGY_ARTIFACT_TOO_LARGE Agy artifact exceeded the inclusive 50MiB read limit
AGY_VIDEO_UNSUPPORTED Video generation is not supported by the Gemini (agy) provider
AGY_MASK_UNSUPPORTED Mask-based editing is not supported by the Gemini (agy) provider
AGY_REF_TOO_MANY Too many reference images for agy (max 3)
GEMINI_API_KEY_MISSING Gemini API key or Vertex AI credentials not configured
GEMINI_API_RATE_LIMITED Gemini API rate limited (429)
GEMINI_API_BAD_REQUEST Gemini API bad request (400/403)
GEMINI_API_SAFETY_BLOCKED Gemini API generation blocked by safety filter
GEMINI_API_NO_IMAGE Gemini API returned no image in response
VIDEO_PROVIDER_UNSUPPORTED Video generation requires provider "grok" or "grok-api"
SSE_CAPACITY More than 512 concurrent GET /api/events listeners
REQUEST_ID_IN_USE Async POST used a requestId that already has an active job
TOO_MANY_JOBS More than the configured concurrent active generation job limit (Retry-After: 5; default 24)

Key Management

API key management endpoints for configuring provider credentials at runtime through the web UI or HTTP API.

Endpoint Method Description
/api/keys/status GET Returns configured/valid/maskedKey status for all providers (openai, xai, gemini, atlascloud, minimax, nai, vertex) plus geminiAuthMode ("apikey" or "vertex")
/api/keys/:provider PUT Save an API key. Body: { "apiKey": "..." }. Validates key format and upstream before saving to config.json. Provider: openai, xai, gemini, atlascloud, minimax, or nai. minimax and nai have no fixed key prefix, so only the upstream validation call gates them.
/api/keys/:provider DELETE Remove a config-sourced API key. Env-sourced keys cannot be removed (ENV_KEY_IMMUTABLE).
/api/keys/vertex PUT Save a Vertex AI service account JSON. Body: { "serviceAccountJson": "..." }. Validates JSON structure (type: "service_account", project_id required).
/api/keys/vertex DELETE Remove a config-sourced Vertex AI service account.
/api/keys/gemini-auth-mode PUT Persist the Gemini auth mode chosen in the settings dropdown. Body: { "mode": "apikey" | "vertex" }. Saved to config.json and hot-updated.

Keys saved via PUT are stored in config.json and hot-updated in the runtime context (no server restart required). Keys loaded from environment variables (OPENAI_API_KEY, XAI_API_KEY, GEMINI_API_KEY, VERTEX_SERVICE_ACCOUNT_JSON) take precedence and are immutable through the API.

Thumbnail Backfill

Endpoint Method Description
/api/history/backfill-thumbnails POST Generate missing .thumb.jpg thumbnails for all images and videos in the generated directory. Returns { ok, total, created, skipped, failed }. Also available offline via ima2 backfill-thumbs.

Thumbnails are also generated automatically on server startup for any media files that lack them.

Agent Mode

Agent Mode is a conversational image workspace (web UI only — no CLI). All routes are under /api/agent/* and are backed by routes/agent.ts + lib/agent*.ts.

Method Path Notes
GET /api/agent/tools Slash-command and tool metadata
GET /api/agent/sessions List sessions (?limit=)
POST /api/agent/sessions Create session (title, currentImage, webSearchEnabled) → 201
GET /api/agent/sessions/:sessionId Fetch one session
PATCH /api/agent/sessions/:sessionId Update title, webSearchEnabled, generationSettings, currentImage, locks
DELETE /api/agent/sessions/:sessionId Delete session
POST /api/agent/sessions/:sessionId/compact Session compaction
GET /api/agent/sessions/:sessionId/manifest XML manifest export
POST /api/agent/sessions/:sessionId/turns Synchronous turn (prompt, provider, quality, size, model, …)
GET /api/agent/sessions/:sessionId/errors Recent errors (?limit=, default 10)
GET /api/agent/sessions/:sessionId/queue Per-session queue items
POST /api/agent/sessions/:sessionId/queue Enqueue async turn / slash command → 202
GET /api/agent/queue Global queue listing
POST /api/agent/queue/:itemId/cancel Cancel queued item
POST /api/agent/queue/:itemId/retry Retry failed item

Prompt Builder

Method Path Body Response
POST /api/prompt-builder/chat { messages, backend?, model?, context? } { provider, backend, requestedBackend, model, message, usage }
GET /api/prompt-builder/config none { backend, model, options: { backends, models, autoOrder }, locked }
PUT /api/prompt-builder/config { backend, model? } Same config payload after atomic persistence

The persisted keys are promptBuilder.backend and promptBuilder.model; the corresponding environment locks are IMA2_PROMPT_BUILDER_BACKEND and IMA2_PROMPT_BUILDER_MODEL. backend=auto tries oauth -> grok -> api -> grok-api and selects the first ready lane. An explicit backend stays pinned and never falls back. Chat responses separate requestedBackend from the answering backend, which the UI renders as via <backend>.

Invalid backend/model pairs return typed 400 errors. An unavailable explicit lane returns its typed 401/503 error, no ready Auto lane returns 503 PROMPT_BUILDER_NO_BACKEND_READY, and environment-locked config writes return 409 PROMPT_BUILDER_CONFIG_ENV_LOCKED. Auto selection happens before sending upstream; an accepted or failed upstream request is not retried on another backend.

Endpoint → CLI Mapping

Most server routes under /api/* have a CLI wrapper. The exception is Agent Mode (/api/agent/*), which is server + web-UI-only and has no ima2 subcommand. The prompt builder HTTP route (POST /api/prompt-builder/chat) is wrapped by ima2 prompt build. Use this table to find the command that calls a given endpoint. (See README.md "Client" section for full flag lists.)

Endpoint CLI
POST /api/generate ima2 gen
POST /api/edit ima2 edit
POST /api/generate/multimode (SSE) ima2 multimode
POST /api/video/generate (SSE) ima2 video
POST /api/video/generate with continueFromVideo ima2 video continue
POST /api/video/edit ima2 video edit
POST /api/video/extend ima2 video extend
GET /api/video/frame ima2 video frame
POST /api/video/analyze ima2 video analyze
POST /api/node/generate (SSE) / GET /api/node/:id ima2 node generate / ima2 node show
GET /api/history ima2 ls
DELETE /api/history/:name / …/permanent ima2 history rm [--permanent]
POST /api/history/:filename/restore ima2 history restore --trash-id
POST /api/history/favorite ima2 history favorite
POST /api/history/import-local ima2 history import
POST /api/metadata/read ima2 metadata / ima2 show --metadata
GET/POST/PUT/DELETE /api/sessions[/…] ima2 session ls/show/create/rm/rename
GET /api/sessions/:id, PUT /api/sessions/:id/graph ima2 session graph load/save
GET/PUT /api/sessions/:id/style-sheet[/…] ima2 session style-sheet …
GET/PUT/DELETE /api/annotations/:name ima2 annotate get/set/rm
POST /api/canvas-versions / PUT /api/canvas-versions/:name ima2 canvas-versions save/update
GET/POST/PUT/DELETE /api/prompts[/…] ima2 prompt …
GET/POST/PATCH/DELETE /api/prompts/folders[/…] ima2 prompt folder …
…/api/prompts/import/… ima2 prompt import sources/refresh/curated/discovery/folder
…/api/cardnews/… (gated on features.cardNews) ima2 cardnews …
POST /api/comfy/export-image ima2 comfy export
GET /api/comfy/workflows ima2 comfy workflow ls
POST /api/comfy/workflows ima2 comfy workflow add
DELETE /api/comfy/workflows/:id ima2 comfy workflow rm
POST /api/comfy/inspect ima2 comfy workflow inspect
POST /api/comfy/probe Web UI only (origin reachability in the workflow form)
GET /api/inflight / DELETE /api/inflight/:id ima2 inflight ls (alias ps) / ima2 inflight rm (alias cancel)
GET /api/events (SSE multiplex) Web UI only (persistent EventSource; no CLI wrapper)
GET /api/storage/status / POST /api/storage/open-generated-dir ima2 storage status / ima2 storage open
GET /api/billing / GET /api/providers / GET /api/oauth/status / GET /api/grok/status ima2 billing / ima2 providers / ima2 oauth status / ima2 grok status
GET /api/quota Web UI only (Grok and NovelAI quota in Settings)
POST /api/auth/switch / GET /api/auth/switch/:sessionId Web UI (Settings > QuotaCard > Switch Account) and ima2 grok login
POST /api/oauth/restart ima2 login, ima2 gpt login, ima2 gpt logout
GET /api/health ima2 ping
GET /api/update/badge / POST /api/update/check ima2 update --check (the CLI checks the registry itself and shares the cache)
POST /api/update/dismiss / POST /api/update/notice Web UI only (update popover and the one-time toast)
GET /api/capabilities ima2 capabilities
GET /api/config/grok-planner — (Grok planner model query)
PATCH /api/config/grok-planner — (Grok planner model update)
GET /api/agy/status — (Antigravity CLI install status)
POST /api/history/backfill-thumbnails ima2 backfill-thumbs
GET /api/keys/status, PUT/DELETE /api/keys/:provider, PUT/DELETE /api/keys/vertex Web UI only (Settings > API Keys)
GET/POST/PATCH/DELETE /api/agent/* (sessions, turns, queue) — (Agent Mode; web UI only, no CLI)
POST /api/prompt-builder/chat ima2 prompt build
GET/PUT /api/prompt-builder/config Web UI only (Settings > Providers > Prompt Builder backend / Builder model)

Notes:

  • ima2 history favorite and ima2 annotate … send X-Ima2-Browser-Id: cli-<sha1prefix> derived from the config dir, so CLI activity does not collide with browser sessions.
  • ima2 session graph save performs a GET-then-PUT with If-Match: "<version>" to guard against GRAPH_VERSION_CONFLICT.
  • Graph CLI commands accept current flat session fields and legacy nested graph responses. First save uses version 0; load exports { version, nodes, edges }.
  • ima2 history import and ima2 canvas-versions save/update send raw bytes with Content-Type: image/<png|jpeg|webp>; the SSE endpoints (multimode, node generate, video) use Accept: text/event-stream. The web UI instead uses GET /api/events plus async: true on POST routes.
  • ima2 cardnews … checks runtimeConfig.features.cardNews before calling the gated endpoints; when disabled the CLI exits 2 with a clear message instead of producing a 404.

CLI Discovery

The server writes an advertisement file at:

~/.ima2/server.json

CLI commands such as ima2 ping, ima2 gen, and ima2 ls use this file unless --server or IMA2_SERVER is provided.

Current shape:

{
  "port": 3334,
  "url": "http://localhost:3334",
  "pid": 12345,
  "startedAt": 1777180000000,
  "version": "1.0.0",
  "backend": {
    "configuredPort": 3333,
    "actualPort": 3334,
    "url": "http://localhost:3334"
  },
  "oauth": {
    "configuredPort": 10531,
    "actualPort": 10532,
    "url": "http://127.0.0.1:10532",
    "status": "ready"
  }
}

Top-level port and url are kept for older CLI clients. New code should prefer backend.url.

Sprite Recipe Routes

GET /api/sprite-recipes

List all sprite recipes. Returns { recipes: SpriteRecipeRecord[] }.

POST /api/sprite-recipes

Create a new sprite recipe. Body: SpriteRecipeDefinition. Returns 201 { recipe }.

GET /api/sprite-recipes/:id

Get a single recipe. Returns { recipe } or 404 { error }.

PATCH /api/sprite-recipes/:id

Update recipe fields. Returns { recipe }.

DELETE /api/sprite-recipes/:id

Delete a recipe. Returns { ok: true }.

POST /api/sprite-recipes/:id/anchor/approve

Approve an idle candidate as the identity anchor. Body: { assetId }. Returns { recipe }.

POST /api/sprite-recipes/:id/anchor/generate

Generate an idle anchor candidate. Async: returns 202 { requestId }, progress via /api/events.

POST /api/sprite-recipes/:id/generate

Generate sprite rows for approved recipes. Body: { states?, async, requestId }. Async: 202 { requestId }.

MCP Provider Connections

Remote subscription MCP providers (Runway, Higgsfield) connect through a compiled registry — arbitrary endpoints are rejected. All responses are secret-free: tokens live only in versioned ${configDir}/mcp/<provider>.json records (0600), bound to the provider endpoint and live callback origin.

After the server has selected and published its actual port, it automatically restores each enabled provider with a completed same-binding token bundle. This path does not open a browser. Missing, corrupt, pending-only, disabled, or binding-mismatched records do not send a Bearer request and are not silently deleted. A mismatch is reported as auth_required; start Connect again to authorize the new endpoint/origin. OAuth state and PKCE are memory-only, so a browser flow interrupted by restart must be restarted.

GET /api/mcp/providers

List registry providers with per-provider connection status.

POST /api/mcp/temp-references

Stage local reference sources (data URLs) as a temporary gallery batch so MCP generation can upload them by filename. Returns { ok, batchId, files[] }.

DELETE /api/mcp/temp-references/:batchId

Delete a staged temp-reference batch after the MCP job finishes.

GET /api/models

Canonical lane catalog for CLI/agent routing. Returns { ok, lanes: { [lane]: { status, reason?, defaults: { image?, video? }, models: { image[], video[] } } } } for the ten core lanes (oauth|api|grok|grok-api|agy|gemini-api|atlascloud|minimax|nai|comfy) plus MCP lanes (runway|higgsfield). Status is one of ready|locked|disconnected|key-missing with precedence locked > key-missing|disconnected > ready. MCP static snapshot models are always listed; dynamic (models_explore) models appear only while connected. Consumed by ima2 models, ima2 defaults set image|video, and the CLI model resolver.

GET /api/mcp/providers/:id/status

Connection status: disconnected | connecting | auth_required | connected | offline | error. The optional detail is a stable secret-free diagnostic code. connected means the current generation/transport is usable; offline means a terminal transport failure was observed and at most one reconnect is scheduled; error is an unrecovered failure.

POST /api/mcp/providers/:id/connect

Start or resume a connection. Returns 202 { status: { state: "auth_required", authorizationUrl } } when the user must approve OAuth in a browser; 202 while connecting; 200 once connected. Terminal responses preserve state: 409 disconnected, 503 offline, or 502 error. ok is true only for connected.

GET /api/mcp/oauth/callback

OAuth redirect target (?state=&code=). Exempt from the LAN token guard; protected by the single-use OAuth state + PKCE. Invalid state → 400 with no token exchange. Completion HTML is returned only after the manager reaches connected; otherwise the callback returns the state's mapped 202/409/503/502 response and a failure page.

POST /api/mcp/providers/:id/refresh

Close and re-establish the session reusing stored tokens (refresh-token path). It uses the same state-to-HTTP mapping as Connect and cannot overwrite a newer disconnect or connection generation.

DELETE /api/mcp/providers/:id/connection

Clear local tokens and close the session. The response note explicitly says this is local-only; it does not revoke the provider-side grant. The tombstone prevents older connect, callback, restore, or refresh work from recreating the credential.

Transport recovery never replays a host callTool request. In particular, mutating or billed media operations are not automatically retried by the connection manager.

POST /api/mcp/generate

Generate media through a connected MCP provider. Body: { provider: "runway", kind: "image"|"video", prompt, model?, ratio?, startFrameUrl?, requestId? }. Async: returns 202 { requestId }; progress (submitted, provider-queued, provider-running, downloading) and terminal done/error arrive on /api/events. The route is the single persistence owner: results are committed to the generated library (file + strict sidecar + thumbnail) before done is emitted. Catalog-only providers (e.g. Higgsfield on a free plan) return 409 MCP_EXECUTION_LOCKED. startFrameFilename accepts an existing generated-library image: it is uploaded to the provider and used as the image-to-video start frame, recording parent: { filename, mediaType, role: "start-frame" } lineage in the sidecar.

POST /api/mcp/media-action

Run a media workflow action. Body: { action: "stitch"|"upscale-video"|"upscale-image"|"edit-video"|"extend"|"reframe", files: [generated filenames], prompt?, provider? }. The workflow router decides per-tool: native (provider tool present live with a matching schema), fallback (stitch → local ffmpeg concat; extend → last-frame I2V), or unavailable (409 MEDIA_ACTION_UNAVAILABLE, e.g. reframe while the provider is catalog-only). Async: 202 { requestId, mode, plan }; results commit through the same single persistence owner with parent/inputs lineage.

POST /api/mcp/tasks/:taskId/recover

POST /api/mcp/multishot

Generate a multishot (multi-scene) video through Runway MCP. Body: { prompt?: string, shots?: string[] (3-5), duration?: 5|10|15, resolution?: "720p"|"1080p", aspectRatio?, sound?: boolean, firstSceneFilename?, requestId? }. prompt maps to auto mode (storyPrompt); shots[] maps to custom mode. One of prompt or shots is required (400 INVALID_MULTISHOT otherwise). Async: 202 { requestId, provider }; lifecycle events on /api/events. Results commit with workflow: "video.multishot" and mcpParameters.

Re-download a remote-succeeded MCP task into the generated library. Body: { provider?: "runway", kind?: "video"|"image", requestId? }. Use after a generation's download/commit step failed transiently — provider assets stay fetchable for ~24-48h. Re-polls get_task, requires SUCCEEDED with an output URL (error SSE event with MCP_TASK_NOT_SUCCEEDED otherwise), then runs the same download (with retry + IPv4 fallback) → single-persistence commit path as a normal generation. Async: 202 { requestId, taskId }; done carries recovered: true. requestId dedupes only while the job is still running — a retry with the same id returns 409 REQUEST_ID_IN_USE, and once the job settles a new recover may reuse it (same scope as /api/mcp/generate). Catalog-only providers (e.g. Higgsfield on a free plan) return 409 MCP_EXECUTION_LOCKED, same as /api/mcp/generate.

Contract Discovery

Machine-readable tool contracts for AI agents (ima2 tools CLI backs onto these).

GET /api/contracts

Full catalog summary: { ok, data: { tools: [{ id, namespace, availability, executable, description }] }, catalogVersion, schemaVersion, cliVersion, requestId, generatedAt }. Availability is promoted from live connection state: callable requires a connected session plus post-connect ingest evidence; bundled snapshots alone stay documented.

GET /api/contracts/:id

Full contract for one tool, including the execution binding block: bound tools carry { binding, endpoint, inputContract } — the normalized schema ima2 tools call accepts (the raw upstream inputSchema is reference material only).