From a86389cd3f0700f583feae0b6a3638d31f057f55 Mon Sep 17 00:00:00 2001 From: Dan Barr <6922515+danbarr@users.noreply.github.com> Date: Tue, 16 Jun 2026 11:35:18 -0400 Subject: [PATCH 1/3] Document sessionAffinity for scaled MCPServers Co-Authored-By: Claude Opus 4.8 (1M context) --- .../guides-k8s/redis-session-storage.mdx | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/docs/toolhive/guides-k8s/redis-session-storage.mdx b/docs/toolhive/guides-k8s/redis-session-storage.mdx index 390d815f..aec698c7 100644 --- a/docs/toolhive/guides-k8s/redis-session-storage.mdx +++ b/docs/toolhive/guides-k8s/redis-session-storage.mdx @@ -839,6 +839,59 @@ spec: # highlight-end ``` +### Session affinity and shared session storage + +Session affinity and Redis session storage solve related but distinct problems +for a scaled `MCPServer`, and they work best together. + +The `MCPServer` spec exposes a `sessionAffinity` field that controls how +Kubernetes routes repeated client connections to the proxy `Service`: + +```yaml title="mcp-server-with-affinity.yaml" +apiVersion: toolhive.stacklok.dev/v1beta1 +kind: MCPServer +metadata: + name: my-server + namespace: toolhive-system +spec: + image: ghcr.io/example/my-mcp-server:latest + replicas: 2 + # highlight-next-line + sessionAffinity: ClientIP # default; set to None for free load balancing +``` + +The field accepts two values: + +- **`ClientIP`** (default) - routes connections from the same client IP to the + same pod. Because MCP transports (SSE and streamable HTTP) are stateful, this + keeps a client pinned to the replica that holds its in-memory session. +- **`None`** - lets the `Service` load-balance each connection freely across + replicas. + +Affinity only influences routing. It does not move session state between pods. +Redis-backed shared session storage solves that: when a client lands on a +different replica, whether because of `sessionAffinity: None`, a pod restart, or +pod replacement, the new pod rebuilds the session from Redis instead of failing. +Use the two together for resilient scaling - `ClientIP` reduces cross-pod hops +during normal operation, while Redis lets sessions survive when a hop or restart +happens anyway. + +:::warning[ClientIP affinity is unreliable behind NAT or shared egress IPs] + +`ClientIP` affinity relies on the client source IP reaching kube-proxy. When +clients sit behind a NAT gateway, corporate proxy, or cloud load balancer +(common in EKS, GKE, and AKS), all traffic appears to originate from the same +IP, routing every client to one pod and negating the benefit of multiple +replicas. Configure Redis session storage so any pod can serve any client, and +consider `sessionAffinity: None` so the `Service` load-balances evenly. + +::: + +This is the `MCPServer` equivalent of the affinity behavior documented for vMCP. +For the same field on `VirtualMCPServer`, including guidance on stateful +backends, see +[When horizontal scaling is challenging](../guides-vmcp/scaling-and-performance.mdx#when-horizontal-scaling-is-challenging). + ### Configure VirtualMCPServer session storage The `sessionStorage` field is identical for `VirtualMCPServer`: From 62e0261fd8aa36478faf72021ba8e1740a3e7219 Mon Sep 17 00:00:00 2001 From: Dan Barr <6922515+danbarr@users.noreply.github.com> Date: Tue, 16 Jun 2026 12:23:49 -0400 Subject: [PATCH 2/3] Apply Copilot review feedback Co-Authored-By: Claude Opus 4.8 (1M context) --- .../guides-k8s/redis-session-storage.mdx | 23 ++++++++++--------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/toolhive/guides-k8s/redis-session-storage.mdx b/docs/toolhive/guides-k8s/redis-session-storage.mdx index aec698c7..d8172bf3 100644 --- a/docs/toolhive/guides-k8s/redis-session-storage.mdx +++ b/docs/toolhive/guides-k8s/redis-session-storage.mdx @@ -871,25 +871,26 @@ The field accepts two values: Affinity only influences routing. It does not move session state between pods. Redis-backed shared session storage solves that: when a client lands on a different replica, whether because of `sessionAffinity: None`, a pod restart, or -pod replacement, the new pod rebuilds the session from Redis instead of failing. -Use the two together for resilient scaling - `ClientIP` reduces cross-pod hops -during normal operation, while Redis lets sessions survive when a hop or restart -happens anyway. +pod replacement, the new pod restores the session from Redis so the client can +resume without reinitializing. Use the two together for resilient scaling. +`ClientIP` reduces cross-pod hops during normal operation, and Redis lets +sessions survive when a hop or restart happens anyway. :::warning[ClientIP affinity is unreliable behind NAT or shared egress IPs] `ClientIP` affinity relies on the client source IP reaching kube-proxy. When clients sit behind a NAT gateway, corporate proxy, or cloud load balancer -(common in EKS, GKE, and AKS), all traffic appears to originate from the same -IP, routing every client to one pod and negating the benefit of multiple -replicas. Configure Redis session storage so any pod can serve any client, and -consider `sessionAffinity: None` so the `Service` load-balances evenly. +(common in EKS, GKE, and AKS), much of the traffic can appear to originate from +the same IP, routing many clients to one pod and negating the benefit of +multiple replicas. Configure Redis session storage so any pod can serve any +client, and consider `sessionAffinity: None` so the `Service` load-balances +evenly. ::: -This is the `MCPServer` equivalent of the affinity behavior documented for vMCP. -For the same field on `VirtualMCPServer`, including guidance on stateful -backends, see +This is the `MCPServer` equivalent of the affinity behavior documented for +Virtual MCP Server (vMCP). For the same field on `VirtualMCPServer`, including +guidance on stateful backends, see [When horizontal scaling is challenging](../guides-vmcp/scaling-and-performance.mdx#when-horizontal-scaling-is-challenging). ### Configure VirtualMCPServer session storage From 79292b0896a91e4762089661b3dffdeb61d85adf Mon Sep 17 00:00:00 2001 From: Dan Barr <6922515+danbarr@users.noreply.github.com> Date: Thu, 18 Jun 2026 10:38:55 -0400 Subject: [PATCH 3/3] Note MCPRemoteProxy as a sessionAffinity peer Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/toolhive/guides-k8s/redis-session-storage.mdx | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/toolhive/guides-k8s/redis-session-storage.mdx b/docs/toolhive/guides-k8s/redis-session-storage.mdx index d8172bf3..10f07587 100644 --- a/docs/toolhive/guides-k8s/redis-session-storage.mdx +++ b/docs/toolhive/guides-k8s/redis-session-storage.mdx @@ -888,10 +888,12 @@ evenly. ::: -This is the `MCPServer` equivalent of the affinity behavior documented for -Virtual MCP Server (vMCP). For the same field on `VirtualMCPServer`, including -guidance on stateful backends, see -[When horizontal scaling is challenging](../guides-vmcp/scaling-and-performance.mdx#when-horizontal-scaling-is-challenging). +`MCPRemoteProxy` and `VirtualMCPServer` expose the same `sessionAffinity` field +with identical behavior. For VirtualMCPServer, including guidance on stateful +backends, see +[When horizontal scaling is challenging](../guides-vmcp/scaling-and-performance.mdx#when-horizontal-scaling-is-challenging); +for MCPRemoteProxy, see +[Run multiple replicas for high availability](./remote-mcp-proxy.mdx#run-multiple-replicas-for-high-availability). ### Configure VirtualMCPServer session storage