feat(api): restrict stateful APIs for migration

CLAUDE.md

@@ -103,14 +103,14 @@ cargo test --lib --bins
 # Run ALL e2e tests (requires PostgreSQL running)
 cargo test --test e2e_test
 
-# Run a single e2e test file
-cargo test --test e2e_conversations
+# Run the stateless Responses e2e module
+cargo test -p api --test e2e_all responses_stateless
 
 # Run vLLM integration tests (requires vLLM server)
 cargo test --test integration_tests
 
-# Run a specific test by name
-cargo test test_create_conversation
+# Run a specific Responses test by name
+cargo test -p api --test e2e_all responses_stateless
 ```
 
 ### Database Setup for Tests
@@ -172,8 +172,8 @@ Organization (Tenant Root)
 - Storage: Hashed session token in database, returned as HTTP-only cookie
 
 **2. API Key-Based (AI Inference Operations)**
-- Used for: Chat completions, conversations, responses, attestation
-- Endpoints: `/v1/chat/completions`, `/v1/responses/*`, `/v1/conversations/*`
+- Used for: Chat completions, Responses, and attestation
+- Endpoints: `/v1/chat/completions`, `POST /v1/responses`, `/v1/signature/{chat_id}`, `/v1/attestation/*`
 - Format: `Authorization: Bearer sk-live-xxx` or `Authorization: Bearer sk-test-xxx`
 - Storage: SHA-256 hashed, workspace-scoped
 - Tracking: Last used timestamp, optional expiration
@@ -191,19 +191,18 @@ POST /v1/completions
 - Supports streaming (SSE) and non-streaming
 - Standard OpenAI format with `[DONE]` terminator
 
-**B. Response API (Platform-specific)**
+**B. Responses API (single-turn, stateless)**
 ```
 POST /v1/responses
 ```
-- Links to conversation history for context
-- Rich metadata and event types
+- `store: false` is the only supported mode; an omitted `store` is treated as `false` and `store: true` is rejected
+- Clients send all needed prior context in each request. Conversations, `previous_response_id`, background responses, and response-history endpoints are retired or unsupported.
+- Raw request/response content, response items, and conversation history are not persisted.
+- **Existing completed-response attestation is preserved best-effort**: when its signature write succeeds, `GET /v1/signature/resp_*` can retrieve the response ID and signatures over SHA-256 request/response digests. The signature material contains no raw request or response content. A disconnected stream has no completed `resp_*` attestation record or legacy disconnect fallback.
 - Event types: `response.created`, `response.output_text.delta`, `response.completed`, `response.failed`
-- **Tool use**: Supports external function calls, code_interpreter, computer tools (client-executed),
-  plus server-executed web_search, file_search, and MCP tools
-- **Function call flow**: LLM requests a function, response pauses with status `incomplete`,
-  client executes and resumes via `previous_response_id` + `FunctionCallOutput` input.
-  Resumption verifies workspace ownership on `previous_response_id` and validates
-  each `call_id` maps to exactly one stored FunctionCall
+- Only custom `type: "function"` tools are supported. They are client-managed: Cloud returns a `function_call` but never executes it; the client sends the original call and its matching `function_call_output` in a later fresh `store: false` request with its own history.
+- Server-executed Responses tools—including `web_search`, `web_context_search`, `file_search`, `code_interpreter`, `computer`, and remote `mcp`—plus file input and image-generation/editing output models are rejected. The independent root `POST /mcp` endpoint continues to expose web search and is not part of Responses execution.
+- During Stage I, existing Conversation/File data remains available through authenticated, workspace-scoped migration routes: `POST /v1/conversations/batch`, `GET /v1/conversations/{conversation_id}`, `GET /v1/conversations/{conversation_id}/items`, `DELETE /v1/conversations/{conversation_id}`, `GET /v1/files`, `GET /v1/files/{file_id}`, `GET /v1/files/{file_id}/content`, and `DELETE /v1/files/{file_id}`. They return `Cache-Control: no-store`; creation, upload, all other mutations, and unsupported legacy paths return `410 Gone`.
 
 **Streaming Flow**:
 ```
@@ -217,7 +216,7 @@ Client → CompletionService → Provider Pool (round-robin)
 - Discovery Server polled every 5 minutes (configurable)
 - `GET /models` returns available models and their vLLM endpoints
 - Provider Pool updated dynamically (no hardcoded models)
-- Load balancing: round-robin for new requests, sticky routing for conversations
+- Load balancing: round-robin for new requests
 
 ### Database Layer (Patroni High-Availability)
 - PostgreSQL 16 with deadpool connection pooling
@@ -226,6 +225,10 @@ Client → CompletionService → Provider Pool (round-robin)
 - **Migrations**: SQL-based using Refinery, run on startup
 - Located at: `crates/database/src/migrations/sql/`
 
+Historical Conversation, Response, ResponseItem, and File schemas/repositories remain for now,
+but the retired public APIs must not be wired back to them. Data deletion and schema removal are
+separate work.
+
 ### External Billing / Credits Service
 
 Cloud API itself has no direct Stripe integration. The billing/credits service is the
@@ -256,14 +259,14 @@ Located in `crates/services/src/`:
 - `workspace` - Workspace CRUD, settings
 - `user` - User profiles, session management
 - `completions` - AI completion orchestration
-- `conversations` - Conversation lifecycle
-- `responses` - Response streaming with token tracking, tool orchestration (function calls, web search, file search, MCP)
+- `conversations` - Legacy state module with temporary Stage I migration routes; only per-conversation deletion remains enabled
+- `responses` - Request-scoped, stateless response orchestration with client-managed custom function calls
 - `attestation` - TEE attestation reports, chat signatures
 - `models` - Model catalog and pricing
 - `usage` - Token tracking, limit enforcement, billing
 - `inference_provider_pool` - Model discovery, load balancing
 - `mcp` - Model Context Protocol client management
-- `files` - File storage (AWS S3)
+- `files` - Legacy state module with temporary Stage I migration routes; only per-file deletion remains enabled
 - `metrics` - OpenTelemetry metrics
 - `admin` - Admin operations, analytics
 - `common` - Shared utilities
@@ -276,13 +279,13 @@ Located in `crates/api/src/routes/`:
 - `workspaces.rs` - Workspace & API key management
 - `users.rs` - User profile, invitations, sessions
 - `completions.rs` - Chat & text completions
-- `conversations.rs` - Conversation management
-- `responses.rs` - AI response streaming
+- `conversations.rs` - Temporary authenticated, no-store Stage I migration routes; only per-conversation deletion remains enabled
+- `responses.rs` - Stateless AI response streaming with client-managed custom function calls
 - `models.rs` - Model catalog
 - `usage.rs` - Usage tracking, billing
 - `attestation.rs` - TEE verification, signatures
 - `admin.rs` - Admin endpoints
-- `files.rs` - File upload/download
+- `files.rs` - Temporary authenticated, no-store Stage I migration routes; only per-file deletion remains enabled
 - `health.rs` - Health checks
 - `api.rs` - API versioning
 
@@ -355,7 +358,6 @@ Comprehensive C4 diagrams and flows: `docs/architecture/c4-diagrams.md`
 - No hardcoded model configuration
 - Automatic scaling as inference servers added/removed
 - Graceful handling of provider failures
-- Conversation consistency via sticky routing
 
 ## API Documentation
 

README.md

@@ -1,6 +1,6 @@
 # NEAR AI Cloud API
 
-A Rust-based cloud API for AI model inference, conversation management, and organization administration. Part of the NEAR AI platform alongside the Chat API.
+A Rust-based cloud API for AI model inference and organization administration. Part of the NEAR AI platform alongside the Chat API.
 
 ## Quick Start
 
@@ -171,6 +171,30 @@ Once all checks pass, you're ready to commit!
 
 ## API Documentation
 
+### Temporary migration APIs
+
+During Stage I of the confidential-data migration, cloud-api keeps a small,
+authenticated, workspace-scoped surface so existing migration/export tooling
+can retrieve data it already identifies and existing account deletion can
+complete. This is not a new export API and does not add a Conversation-list
+endpoint.
+
+- Conversations: `POST /v1/conversations/batch`,
+  `GET /v1/conversations/{conversation_id}`, and
+  `GET /v1/conversations/{conversation_id}/items`, plus
+  `DELETE /v1/conversations/{conversation_id}`.
+- Files: `GET /v1/files`, `GET /v1/files/{file_id}`, and
+  `GET /v1/files/{file_id}/content`, plus `DELETE /v1/files/{file_id}`.
+
+Each temporary route requires an API key, is scoped to that key's workspace,
+and returns `Cache-Control: no-store`. Creation, upload, and every Conversation
+or File mutation other than the existing per-resource `DELETE` routes return
+`410 Gone`. The temporary routes will be removed after data migration.
+
+`POST /v1/responses` remains stateless (`store: false`) and does not accept a
+Conversation reference, response history, or File input. Clients must send any
+inference history needed for a request themselves.
+
 Interactive API documentation is available when running the server:
 
 - **Scalar UI**: `http://localhost:3000/docs` - Modern, beautiful API documentation with interactive playground