🏘️ feat: Add Hosted App Control Plane

docs/lambda-microvm/README.md

@@ -200,6 +200,108 @@ again after expiry. Static assets and request-shaped handlers should remain on
 cheaper stateless delivery paths; this target is only the resident-server
 adapter.
 
+#### Hosted-app control plane and preview gateway
+
+The service-side control plane is stateful-profile only. It snapshots the
+source runtime session under its existing lock, records an exact checkpoint and
+AWS idempotency intent in the fenced Redis registry, then launches/restores the
+dedicated app-host VM on an isolated BullMQ queue. API pods enqueue lifecycle
+work and proxy preview bytes; only worker pods need Lambda MicroVM IAM.
+Start requests share the authenticated execution rate limiter. Before enabling
+this feature broadly for untrusted multi-tenant traffic, add a plan-aware cap on
+active hosted-app leases per owner; the initial feature-flagged slice relies on
+the deployment's Lambda MicroVM quota as its hard fleet ceiling.
+
+Authenticated API contract:
+
+```http
+POST /v1/hosted-apps
+Content-Type: application/json
+
+{
+  "runtime_session_hint": "conversation-123",
+  "app_id": "my-app",
+  "revision": "rev-1",
+  "language": "node",
+  "version": ">=22",
+  "entrypoint": "server.js",
+  "cwd": ".",
+  "args": [],
+  "env": {}
+}
+```
+
+`GET /v1/hosted-apps/:app_id?runtime_session_hint=...` returns status and a
+fresh five-minute `preview_url`; `DELETE` on the same resource terminates the
+lease. A revision is immutable. Retrying the identical spec reasserts the
+resident process; changing code or launch settings requires a new revision and
+captures a new exact checkpoint. An ambiguous provider launch is replayed only
+with its persisted token and can never be overwritten by a newer revision.
+
+Preview traffic uses a wildcard **unprivileged origin**, not a path below the
+CodeAPI or LibreChat origin. Configure wildcard DNS and TLS such that
+`*.apps.example.net` reaches the stateful CodeAPI API service, then set the bare
+origin `https://apps.example.net`. The short-lived URL capability is exchanged
+for an HttpOnly, Secure, host-only cookie and redirected to `/`; every app gets
+its own `happ-<digest>.apps.example.net` origin, so absolute asset paths work
+without exposing privileged-origin cookies to AI-generated JavaScript. Use a
+dedicated registrable domain in production—do not set broad parent-domain
+cookies that also match the app domain.
+
+Set these on both API and worker pods:
+
+| Env | Default | Meaning |
+|---|---|---|
+| `CODEAPI_HOSTED_APPS_ENABLED` | `false` | Enables the stateful-only lifecycle API, isolated preview gateway, and worker. |
+| `CODEAPI_HOSTED_APP_CREDENTIAL_KEY` | — | Base64 of 32 random bytes; AES-GCM encrypts the AWS preview credential stored in Redis. |
+
+Set these only on API pods (the signing key must differ from the credential
+key):
+
+| Env | Default | Meaning |
+|---|---|---|
+| `CODEAPI_HOSTED_APP_PREVIEW_SIGNING_KEY` | — | Different base64 32-byte key for owner-bound preview URL/cookie capabilities. |
+| `CODEAPI_HOSTED_APP_PREVIEW_ORIGIN` | — | Bare HTTPS origin for wildcard app hosts, for example `https://apps.example.net`. |
+
+Set these on worker pods in addition to the ordinary stateful/checkpoint
+configuration:
+
+| Env | Default | Meaning |
+|---|---|---|
+| `LAMBDA_MICROVM_APP_IMAGE_ARN` | — | Dedicated `lambda-microvm-app-host` image ARN. |
+| `LAMBDA_MICROVM_APP_IMAGE_VERSION` | — | Required pinned image version. |
+| `LAMBDA_MICROVM_APP_MAX_DURATION_SECONDS` | `28800` | App-VM hard lifetime. The control plane relaunches an immutable revision after expiry. |
+| `LAMBDA_MICROVM_APP_IDLE_SECONDS` | `300` | Seconds idle before AWS suspends the VM. |
+| `LAMBDA_MICROVM_APP_SUSPEND_SECONDS` | `900` | Seconds suspended before AWS terminates the VM. Suspended VMs still consume quota. |
+
+The pinned app-host image contract fixes the root-owned control/checkpoint
+listener at port 8080, the resident app at port 3000, and resident readiness at
+30 seconds. `RunMicrovm` cannot override the image environment; changing this
+contract requires publishing a matching image and control-plane revision.
+
+Generate the two keys independently:
+
+```bash
+openssl rand -base64 32 # CODEAPI_HOSTED_APP_CREDENTIAL_KEY
+openssl rand -base64 32 # CODEAPI_HOSTED_APP_PREVIEW_SIGNING_KEY
+```
+
+The preview proxy strips CodeAPI authorization, cookies, forwarded identity,
+caller-provided AWS headers, app `Set-Cookie`, app-controlled caching,
+cross-origin policy, and external redirects. Gateway responses are private and
+non-storable so an older revision cannot survive through the browser cache. It
+supports streamed HTTP/SSE and same-origin redirects. WebSockets
+are not part of this first resident adapter. A gateway-owned CSP constrains
+fetches and subresources to the app origin and disables workers/service workers,
+so one app revision cannot leave a persistent worker controlling a later
+revision. Top-level app JavaScript can still navigate the owner's browser to an
+external origin; treat this experimental viewer as owner-trusted. Before broad
+untrusted enablement, serve app content from a separate origin inside a sandboxed
+gateway wrapper. The request `env` map is persisted
+with the immutable launch spec in the registry; it is configuration, not a
+secret store. Add a dedicated secret-reference flow before passing application
+secrets to hosted code.
+
 ### 3. Generate the split execution-manifest keys
 
 The worker signs each execution manifest; the runner only receives the public
@@ -654,3 +756,32 @@ terraform -chdir=docs/lambda-microvm/terraform destroy
 MicroVM images are billed as stored snapshots; running VMs bill while RUNNING and
 suspended VMs bill at a reduced rate, so terminate stray VMs before deleting the
 image.
+
+## Hosted-app retention and rollout
+
+Hosted revisions retain a separate immutable checkpoint under the source session's
+`hosted/` object prefix before releasing the source lease. Rolling workspace
+checkpoint pruning never deletes these snapshots. Keep this prefix out of short
+bucket lifecycle expiry policies; an operator may reclaim retained objects only
+after retiring all hosted revisions that reference them. Automatic reclamation of
+unreferenced hosted snapshots is not yet implemented.
+
+Hosted apps require the `lambda-microvm` backend and an HTTPS preview origin in
+every environment, including development. Preview authentication uses Secure
+host-only cookies. Worker shutdown allows the full hosted-operation budget plus
+launch cleanup and a 30-second reserve; configure the orchestrator's termination
+grace period to cover that same budget (16 minutes with default timeouts).
+
+Roll the service binary to every hosted-app worker before enabling hosted apps
+on API pods. For upgrades from experimental builds, disable hosted-app admission
+on API pods and drain the hosted queue before replacing workers; re-enable it
+after the workers are ready. Older workers do not recognize the status job and
+must not consume jobs submitted by the new API. The feature defaults to disabled.
+
+Running status is a worker-reconciled observation of the resident process, not
+just a cached VM lease. It is rate-limited like start, waits at most 15 seconds
+for the worker result, and returns unavailable rather than a stale running
+claim if reconciliation fails. A crashed process does not erase its VM identity:
+stop can still terminate the VM and start can reassert the same revision.
+Preview refresh waits at most two seconds (or half the remaining credential/VM
+lifetime when shorter), then rereads and reauthorizes any still-valid credential.

helm/codeapi/README.md

@@ -118,6 +118,15 @@ hardening variables documented in `docs/lambda-microvm/README.md`. This chart
 still renders its bundled sandbox-runner, though a Lambda worker does not call
 it; a platform-specific stateful deployment may omit that component.
 
+Resident hosted apps are an opt-in capability of that stateful deployment.
+Configure `CODEAPI_HOSTED_APPS_ENABLED`, the preview origin, and both hosted-app
+keys through `api.extraEnv`; configure only the feature flag and credential key
+on `workerSandbox.extraEnv`, along with the dedicated app image settings.
+Wildcard DNS and TLS for the preview origin must route to the API service
+separately from the normal CodeAPI host. See “Hosted-app control plane and preview gateway” in
+`docs/lambda-microvm/README.md`; do not serve previews beneath the privileged
+LibreChat/CodeAPI origin.
+
 For an existing affinity/strict deployment from before execution profiles,
 first roll the new binary to API and worker pods with
 `CODEAPI_EXECUTION_PROFILE` still unset. The inferred stateful compatibility

service/.env.example

@@ -9,6 +9,14 @@ REDIS_HOST=localhost
 REDIS_PORT=6379
 REDIS_PASSWORD=mysecretpassword
 
+# Stateful Lambda resident hosted apps (optional; see docs/lambda-microvm/README.md)
+# CODEAPI_HOSTED_APPS_ENABLED=true
+# LAMBDA_MICROVM_APP_IMAGE_ARN=arn:aws:lambda:REGION:ACCOUNT:microvm-image:codeapi-app-host
+# LAMBDA_MICROVM_APP_IMAGE_VERSION=1
+# CODEAPI_HOSTED_APP_PREVIEW_ORIGIN=https://apps.example.net
+# CODEAPI_HOSTED_APP_CREDENTIAL_KEY=<openssl rand -base64 32>
+# CODEAPI_HOSTED_APP_PREVIEW_SIGNING_KEY=<different openssl rand -base64 32>
+
 # -----------------------------------------------------------------------------
 # Stripe: https://shipfa.st/docs/features/payments
 # -----------------------------------------------------------------------------
@@ -20,4 +28,4 @@ STRIPE_WEBHOOK_SECRET=
 # Mailgun: https://shipfa.st/docs/features/emails
 # -----------------------------------------------------------------------------
 # EMAIL_SERVER=smtp://postmaster@[mail.yourdomain.com]:[copied_password]@smtp.mailgun.org:587 (without the brackets)
-EMAIL_SERVER=
+EMAIL_SERVER=

service/openapi.yml

@@ -227,7 +227,155 @@ components:
           type: string
           enum: [default, stateful]
 
+    HostedAppStartRequest:
+      type: object
+      required:
+        - runtime_session_hint
+        - app_id
+        - revision
+        - language
+        - version
+        - entrypoint
+      properties:
+        runtime_session_hint:
+          type: string
+          maxLength: 128
+          pattern: '^[A-Za-z0-9._:-]+$'
+        adapter:
+          type: string
+          enum: [resident]
+          default: resident
+        app_id:
+          type: string
+          maxLength: 64
+          pattern: '^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$'
+        revision:
+          type: string
+          maxLength: 128
+          pattern: '^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$'
+        language:
+          type: string
+        version:
+          type: string
+        entrypoint:
+          type: string
+          description: Canonical path relative to the restored stateful workspace.
+        cwd:
+          type: string
+          default: .
+        args:
+          type: array
+          maxItems: 64
+          items:
+            type: string
+        env:
+          type: object
+          additionalProperties:
+            type: string
+
+    HostedAppStatus:
+      type: object
+      required: [app_id, revision, state, preview_id, updated_at]
+      properties:
+        app_id:
+          type: string
+        revision:
+          type: string
+        state:
+          type: string
+          enum: [starting, running, stopping, stopped, failed]
+        preview_id:
+          type: string
+          description: Opaque hosted-app lease identity.
+        preview_url:
+          type: string
+          format: uri
+          description: Five-minute owner capability exchange URL on the isolated app origin.
+        hard_deadline_at:
+          type: integer
+          format: int64
+        updated_at:
+          type: integer
+          format: int64
+        error:
+          type: string
+
 paths:
+  /hosted-apps:
+    post:
+      summary: Start or reassert a resident hosted app
+      description: >-
+        Stateful-profile only. Captures an exact workspace checkpoint and runs
+        the immutable revision in a dedicated Lambda MicroVM app-host image.
+      operationId: startHostedApp
+      parameters:
+        - $ref: '#/components/parameters/ExpectedExecutionProfile'
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/HostedAppStartRequest'
+      responses:
+        '200':
+          description: Hosted app is running
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HostedAppStatus'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '409':
+          $ref: '#/components/responses/Conflict'
+        '429':
+          description: Hosted app start rate limit exceeded
+        '503':
+          description: Lifecycle or provider operation unavailable
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Error'
+
+  /hosted-apps/{app_id}:
+    parameters:
+      - $ref: '#/components/parameters/ExpectedExecutionProfile'
+      - name: app_id
+        in: path
+        required: true
+        schema:
+          type: string
+      - name: runtime_session_hint
+        in: query
+        required: true
+        schema:
+          type: string
+          maxLength: 128
+          pattern: '^[A-Za-z0-9._:-]+$'
+    get:
+      summary: Get hosted app status and a fresh preview URL
+      operationId: getHostedApp
+      responses:
+        '200':
+          description: Hosted app status
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HostedAppStatus'
+        '404':
+          description: Hosted app not found
+    delete:
+      summary: Terminate a hosted app lease
+      operationId: stopHostedApp
+      responses:
+        '200':
+          description: Hosted app stopped
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HostedAppStatus'
+        '404':
+          description: Hosted app not found
+
   /exec:
     post:
       summary: Execute code

service/src/api-server.ts

@@ -27,6 +27,8 @@ import { executionProfileMiddleware } from './middleware/execution-profile';
 import { traceHttpRequest } from './telemetry';
 import { env } from './config';
 import logger from './logger';
+import hostedAppRouter from './hosted-app/router';
+import { hostedAppPreviewGateway } from './hosted-app/preview-gateway';
 
 const { LOCAL_MODE: isLocalMode } = env;
 
@@ -36,6 +38,7 @@ app.set('trust proxy', 1);
 app.use(traceHttpRequest('codeapi.api.request'));
 app.use(httpMetricsMiddleware);
 app.use(executionProfileMiddleware);
+app.use(hostedAppPreviewGateway);
 
 const v1 = Router();
 
@@ -57,6 +60,7 @@ v1.use('/bridge', bridgeRouter);
 v1.use(isLocalMode ? localAuth : apiKeyAuth);
 
 v1.use(workspaceToolsRouter);
+v1.use('/hosted-apps', hostedAppRouter);
 v1.use(serviceRouter);
 v1.use(programmaticRouter);