Skip to content

Commit 70acb38

Browse files
yroblaclaude
yrobla
and
claude
authored
Add horizontal scaling Redis session storage guide (#709)
* Add horizontal scaling Redis session storage guide The existing redis-session-storage page only covered the embedded auth server's Redis setup (Sentinel + ACL). Users scaling MCPServer or VirtualMCPServer horizontally were sent to this page but the auth model is completely different — sessionStorage.passwordRef uses a simple password with no ACL username. Changes: - Restructure redis-session-storage.mdx into two named sections: "Embedded auth server session storage" (existing content) and "Horizontal scaling session storage" (new) - New section covers: standalone Redis Deployment/Service/Secret manifests, full MCPServer and VirtualMCPServer sessionStorage examples, verification steps (SessionStorageWarning condition, cross-pod session test), and a "Sharing a Redis instance" guide using keyPrefix to multiplex use cases - Update run-mcp-k8s.mdx and scaling-and-performance.mdx to link directly to the new #horizontal-scaling-session-storage anchor Closes #707 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * changes from review * Explain proxyrunner session-aware backend pod routing (#710) Users scaling backendReplicas > 1 had no explanation of why Redis is needed or how the proxy runner routes sessions to backend pods. - Add a "Session routing for backend replicas" subsection explaining that the proxy runner uses Redis to store session-to-pod mappings, what happens when a backend pod restarts, and why client-IP affinity alone is unreliable behind NAT - Absorb the standalone SessionStorageWarning note into the new subsection so the backendReplicas implication is explicit - Update Redis link to point directly to the horizontal scaling anchor Closes #708 Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent ea3af30 commit 70acb38

3 files changed

Lines changed: 395 additions & 36 deletions

File tree

‎docs/toolhive/guides-k8s/redis-session-storage.mdx‎

Lines changed: 351 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,43 @@
11
---
2-
title: Redis Sentinel session storage
2+
title: Redis session storage
33
description:
4-
How to deploy Redis Sentinel and configure persistent session storage for the
5-
ToolHive embedded authorization server and horizontal scaling.
4+
Deploy Redis Sentinel for the ToolHive embedded auth server, or a standalone
5+
Redis instance for MCPServer and VirtualMCPServer horizontal scaling.
66
---
77

8-
Deploy Redis Sentinel and configure it as the session storage backend for the
9-
ToolHive [embedded authorization server](../concepts/embedded-auth-server.mdx).
10-
By default, sessions are stored in memory, which means upstream tokens are lost
8+
ToolHive uses Redis for several purposes. This page covers two that require
9+
different configuration:
10+
11+
- **Embedded authorization server sessions** - stores upstream tokens so users
12+
don't need to re-authenticate after pod restarts. Uses Redis Sentinel with
13+
ACL-based authentication and a fixed `thv:auth:*` key pattern. See
14+
[Embedded auth server session storage](#embedded-auth-server-session-storage).
15+
16+
- **MCPServer and VirtualMCPServer horizontal scaling** - shares MCP session
17+
state across pod replicas so any pod can handle any request. Uses a standalone
18+
Redis instance with a simple password. Session data is not persisted to disk.
19+
If the Redis pod restarts, active sessions are lost and clients must
20+
reconnect. See
21+
[Horizontal scaling session storage](#horizontal-scaling-session-storage).
22+
23+
Redis is also required for [rate limiting](./rate-limiting.mdx), which stores
24+
token bucket counters independently of session data.
25+
26+
You can reuse the same Redis instance for all three purposes by using different
27+
`keyPrefix` values or different databases - see
28+
[Sharing a Redis instance](#sharing-a-redis-instance) for details.
29+
30+
---
31+
32+
## Embedded auth server session storage
33+
34+
Configure Redis Sentinel as the session storage backend for the ToolHive
35+
[embedded authorization server](../concepts/embedded-auth-server.mdx). By
36+
default, sessions are stored in memory, which means upstream tokens are lost
1137
when pods restart and users must re-authenticate. Redis Sentinel provides
1238
persistent storage with automatic master discovery, ACL-based access control,
1339
and optional failover when replicas are configured.
1440

15-
Redis is also required as the backend for [rate limiting](./rate-limiting.mdx),
16-
which stores token bucket counters in Redis independently of session data. It is
17-
also required for horizontal scaling when running multiple
18-
[MCPServer](./run-mcp-k8s.mdx#horizontal-scaling) or
19-
[VirtualMCPServer](../guides-vmcp/scaling-and-performance.mdx#session-storage-for-multi-replica-deployments)
20-
replicas, so that sessions are shared across pods.
21-
2241
:::info[Prerequisites]
2342

2443
Before you begin, ensure you have:
@@ -604,6 +623,321 @@ session storage is working correctly.
604623

605624
</details>
606625

626+
---
627+
628+
## Horizontal scaling session storage
629+
630+
When you run multiple replicas of an `MCPServer` proxy runner or a
631+
`VirtualMCPServer`, MCP sessions must be shared across pods so that any replica
632+
can handle any client request. ToolHive stores this session state in Redis using
633+
a simple password. No ACL user configuration or Sentinel is required.
634+
635+
### Deploy a standalone Redis instance
636+
637+
A single Redis pod with a password is sufficient for sharing session state
638+
across replicas during normal operation. The manifests below create Redis in the
639+
`toolhive-system` namespace alongside your ToolHive workloads.
640+
641+
:::note[Session durability]
642+
643+
This deployment uses ephemeral storage with no PVC. If the Redis pod is
644+
rescheduled to another node, all active sessions are lost and MCP clients must
645+
reconnect. Sessions may also be lost on pod restart depending on Redis
646+
persistence settings. For production deployments where session continuity is
647+
required, replace the `Deployment` with a `StatefulSet` and add a
648+
`volumeClaimTemplates` entry to persist Redis data to a PVC.
649+
650+
:::
651+
652+
:::tip[Generate a strong password]
653+
654+
```bash
655+
openssl rand -base64 32
656+
```
657+
658+
:::
659+
660+
```yaml title="redis-scaling.yaml"
661+
# --- Redis password Secret
662+
apiVersion: v1
663+
kind: Secret
664+
metadata:
665+
name: redis-password
666+
namespace: toolhive-system
667+
type: Opaque
668+
stringData:
669+
# highlight-next-line
670+
password: YOUR_REDIS_PASSWORD
671+
---
672+
# --- Redis Service
673+
apiVersion: v1
674+
kind: Service
675+
metadata:
676+
name: redis
677+
namespace: toolhive-system
678+
spec:
679+
selector:
680+
app: redis
681+
ports:
682+
- name: redis
683+
port: 6379
684+
targetPort: 6379
685+
---
686+
# --- Redis Deployment
687+
apiVersion: apps/v1
688+
kind: Deployment
689+
metadata:
690+
name: redis
691+
namespace: toolhive-system
692+
spec:
693+
replicas: 1
694+
selector:
695+
matchLabels:
696+
app: redis
697+
template:
698+
metadata:
699+
labels:
700+
app: redis
701+
spec:
702+
containers:
703+
- name: redis
704+
image: redis:7-alpine
705+
args:
706+
- redis-server
707+
- --requirepass
708+
- $(REDIS_PASSWORD)
709+
env:
710+
- name: REDIS_PASSWORD
711+
valueFrom:
712+
secretKeyRef:
713+
name: redis-password
714+
key: password
715+
ports:
716+
- containerPort: 6379
717+
readinessProbe:
718+
exec:
719+
command: ['sh', '-c', 'redis-cli -a "$REDIS_PASSWORD" PING']
720+
initialDelaySeconds: 5
721+
periodSeconds: 5
722+
resources:
723+
requests:
724+
cpu: 100m
725+
memory: 128Mi
726+
limits:
727+
cpu: 500m
728+
memory: 256Mi
729+
```
730+
731+
Apply the manifests:
732+
733+
```bash
734+
kubectl apply -f redis-scaling.yaml
735+
kubectl wait --for=condition=available deployment/redis \
736+
--namespace toolhive-system --timeout=120s
737+
```
738+
739+
### Configure MCPServer session storage
740+
741+
Reference the Redis Service and Secret in your `MCPServer` spec:
742+
743+
```yaml title="mcp-server-with-redis.yaml"
744+
apiVersion: toolhive.stacklok.dev/v1alpha1
745+
kind: MCPServer
746+
metadata:
747+
name: my-server
748+
namespace: toolhive-system
749+
spec:
750+
image: ghcr.io/example/my-mcp-server:latest
751+
# highlight-start
752+
replicas: 2
753+
sessionStorage:
754+
provider: redis
755+
address: redis.toolhive-system.svc.cluster.local:6379
756+
db: 0
757+
keyPrefix: mcp-sessions
758+
passwordRef:
759+
name: redis-password
760+
key: password
761+
# highlight-end
762+
```
763+
764+
### Configure VirtualMCPServer session storage
765+
766+
The `sessionStorage` field is identical for `VirtualMCPServer`:
767+
768+
```yaml title="vmcp-with-redis.yaml"
769+
apiVersion: toolhive.stacklok.dev/v1alpha1
770+
kind: VirtualMCPServer
771+
metadata:
772+
name: my-vmcp
773+
namespace: toolhive-system
774+
spec:
775+
# highlight-start
776+
replicas: 2
777+
sessionStorage:
778+
provider: redis
779+
address: redis.toolhive-system.svc.cluster.local:6379
780+
db: 0
781+
keyPrefix: vmcp-sessions
782+
passwordRef:
783+
name: redis-password
784+
key: password
785+
# highlight-end
786+
config:
787+
groupRef: my-group
788+
incomingAuth:
789+
type: anonymous
790+
```
791+
792+
### Verify session storage is working
793+
794+
After applying your configuration, check that ToolHive has connected to Redis
795+
successfully.
796+
797+
**Check the `SessionStorageWarning` condition:**
798+
799+
```bash
800+
kubectl describe mcpserver my-server -n toolhive-system
801+
```
802+
803+
When Redis is properly configured, the `SessionStorageWarning` condition is set
804+
to `False`:
805+
806+
```
807+
Conditions:
808+
Type: Ready
809+
Status: True
810+
...
811+
Type: SessionStorageWarning
812+
Status: False
813+
Reason: SessionStorageConfigured
814+
```
815+
816+
If `SessionStorageWarning` is `True`, Redis is not configured or the
817+
configuration is invalid. Check the proxy runner pod logs:
818+
819+
```bash
820+
kubectl logs -n toolhive-system \
821+
-l app.kubernetes.io/name=my-server \
822+
| grep -i "redis\|session"
823+
```
824+
825+
**Test cross-pod session reconstruction:**
826+
827+
Scale down to one replica and connect an MCP client to establish a session. Then
828+
scale back up and delete the original pod. Deleting the pod terminates the TCP
829+
connection, so your client will need to reconnect. If Redis session storage is
830+
working, the session state is preserved and the client can resume making
831+
requests without reinitializing.
832+
833+
:::note
834+
835+
If the Service has `sessionAffinity: ClientIP` configured, the load balancer may
836+
route your reconnect back to the same pod. Delete the original pod first to
837+
force traffic to the new replica.
838+
839+
:::
840+
841+
```bash
842+
# Start with 1 replica
843+
kubectl scale deployment vmcp-my-vmcp -n toolhive-system --replicas=1
844+
845+
# Connect your MCP client and establish a session, then scale up:
846+
kubectl scale deployment vmcp-my-vmcp -n toolhive-system --replicas=2
847+
848+
# Delete the original pod to force routing to the new replica
849+
kubectl delete pod -n toolhive-system \
850+
-l app.kubernetes.io/name=my-vmcp --field-selector='status.podIP=<ORIGINAL_POD_IP>'
851+
852+
# Reconnect your MCP client — it should resume the session without reinitializing
853+
```
854+
855+
---
856+
857+
## Sharing a Redis instance
858+
859+
You can reuse the same Redis instance for embedded auth server sessions,
860+
MCPServer scaling, and VirtualMCPServer scaling by using different `keyPrefix`
861+
values per use case. If you share an instance, use the Redis Sentinel
862+
StatefulSet from the [embedded auth server section](#deploy-redis-sentinel),
863+
which has persistent storage. The standalone `Deployment` from the scaling
864+
section is not suitable as a shared instance because it has no persistent
865+
storage.
866+
867+
The embedded auth server uses `thv:auth:*` by default; set distinct prefixes for
868+
your scaling workloads:
869+
870+
| Use case | Suggested `keyPrefix` |
871+
| ------------------------ | ----------------------------------- |
872+
| Embedded auth server | `thv:auth` (fixed, set by ToolHive) |
873+
| MCPServer scaling | `mcp-sessions` |
874+
| VirtualMCPServer scaling | `vmcp-sessions` |
875+
876+
Alternatively, use separate `db` values (Redis databases 0-15) to provide hard
877+
namespace isolation without requiring separate Redis instances.
878+
879+
#### ACL configuration for a shared instance
880+
881+
The two use cases authenticate differently and require separate ACL entries:
882+
883+
- The **embedded auth server** connects as the `toolhive-auth` ACL user,
884+
restricted to the `~thv:auth:*` key pattern.
885+
- The **scaling use case** (`SessionStorageConfig`) only supports a
886+
`passwordRef` with no username field, so it always authenticates as the
887+
**default** Redis user.
888+
889+
To satisfy both on one instance, enable the default user with a password and
890+
restrict it to the scaling key patterns. Add this line to the `users.acl` entry
891+
in the `redis-acl` Secret alongside the existing `toolhive-auth` entry:
892+
893+
```
894+
user default on >YOUR_SCALING_PASSWORD ~mcp-sessions:* ~vmcp-sessions:* &* +@all -@dangerous
895+
```
896+
897+
Replace `YOUR_SCALING_PASSWORD` with the password you put in the
898+
`redis-password` Secret, and adjust the key patterns to match your `keyPrefix`
899+
values.
900+
901+
:::warning
902+
903+
`SessionStorageConfig` does not support Sentinel. It requires a direct Redis
904+
address and cannot follow a Sentinel-managed master if failover promotes a new
905+
master. If your Sentinel cluster has replicas and failover enabled, hardcoding a
906+
pod DNS (`redis-0.redis...`) will break session storage if that pod is no longer
907+
the master.
908+
909+
For this reason, the recommended approach is to run a **separate standalone
910+
Redis instance** (the `Deployment` from the
911+
[scaling section](#deploy-a-standalone-redis-instance)) for scaling workloads,
912+
rather than sharing the Sentinel instance. If you do share the instance, disable
913+
Redis replication so there is only ever one master and Sentinel cannot trigger
914+
failover.
915+
916+
If you share the Sentinel instance with replication disabled, point
917+
`sessionStorage.address` at the master pod directly:
918+
919+
```yaml
920+
sessionStorage:
921+
provider: redis
922+
address: redis-0.redis.redis.svc.cluster.local:6379
923+
passwordRef:
924+
name: redis-password
925+
key: password
926+
```
927+
928+
:::
929+
930+
:::warning
931+
932+
Restricting the default user to specific key patterns (as shown above) prevents
933+
scaling workloads from accidentally reading or writing auth session keys. If you
934+
omit the key restriction, the default user has full access to the entire
935+
keyspace, including `thv:auth:*` tokens.
936+
937+
:::
938+
939+
---
940+
607941
## Next steps
608942

609943
- [Configure token exchange](./token-exchange-k8s.mdx) to let MCP servers
@@ -614,5 +948,8 @@ session storage is working correctly.
614948
## Related information
615949

616950
- [Set up embedded authorization server authentication](./auth-k8s.mdx#set-up-embedded-authorization-server-authentication)
951+
- [Horizontal scaling for MCPServer](./run-mcp-k8s.mdx#horizontal-scaling)
952+
- [Horizontal scaling for VirtualMCPServer](../guides-vmcp/scaling-and-performance.mdx#session-storage-for-multi-replica-deployments)
617953
- [Backend authentication](../concepts/backend-auth.mdx)
618-
- [Kubernetes CRD reference](../reference/crd-spec.md#apiv1alpha1authserverstorageconfig)
954+
- [Kubernetes CRD reference — SessionStorageConfig](../reference/crd-spec.md#apiv1alpha1sessionstorageconfig)
955+
- [Kubernetes CRD reference — auth server storage](../reference/crd-spec.md#apiv1alpha1authserverstorageconfig)

0 commit comments

Comments
 (0)