Running AI Agents on Kubernetes: A Map of the Open Source Projects
Agent Sandbox, Agent Substrate, AX and kagent each fill a different gap Kubernetes leaves for AI agents. What each one solves, how they fit together, and which to build on today.
Kubernetes runs most cloud native software, so it is where many teams want to run AI agents too. The fit is not obvious. An agent is a single stateful process that runs code a model just wrote, works hard for a minute, then waits hours for a model, a tool or a person. And a team may want thousands of them at once.
Four open source projects now fill most of that gap:
- Agent Sandbox gives each agent an isolated, stateful pod.
- Agent Substrate packs many mostly idle agents onto a few machines.
- Agent Executor (AX) turns an agent task into a declarative object you apply to a cluster.
- kagent defines agents, their models and their tools as Kubernetes resources.
Around them, a second ring of projects governs what agents can reach. This post explains what each project solves, how they fit together and which to build on today.
Why AI agents need new Kubernetes projects
Kubernetes was built around three shapes of workload: stateless replicas (Deployments), stable sets of stateful pods (StatefulSets) and run-to-completion work (Jobs). An AI agent fits none of them well.
- It is a singleton with state. One session has its own identity, files and memory.
- It runs untrusted code. Code a model generated needs a stronger boundary than a regular container, such as gVisor, Kata Containers or a microVM.
- It is mostly idle. A pod held for the whole session wastes most of its machine while the agent waits.
- It comes in large numbers. The Kubernetes control plane is tuned for thousands of long-lived services, not millions of short scheduling decisions.
Kubernetes is adapting. The CNCF’s AI Conformance program for Kubernetes 1.37 recommends strong workload sandboxing, calling it crucial for AI agents that may execute untrusted code. Here is which project fills which gap.
| What an agent needs | Kubernetes alone | Filled by |
|---|---|---|
| One isolated, stateful pod with a stable name and disk | Partly: a StatefulSet of one, a Service and a PVC, wired by hand | Agent Sandbox |
| A strong boundary around untrusted code | Yes: a RuntimeClass backed by gVisor or Kata | Used by Agent Sandbox and Agent Substrate |
| A new session in under a second | No: scheduling and image pulls take seconds | Agent Sandbox warm pools |
| Pause an idle agent and resume it with its memory intact | No: a pod is either running or gone | Agent Substrate |
| Many mostly idle agents on a few machines | No: one pod per session | Agent Substrate |
| An agent’s prompt, model and tools as an API object | No | kagent, AX |
| Rules for which models, tools and agents an agent may call | Partly: NetworkPolicy sees addresses and ports, not tools | agentgateway, ToolHive, kube-agentic-networking |
The open source stack at a glance
The core projects form three layers. kagent and AX define what runs; Agent Sandbox and Agent Substrate are two ways to run isolated agent compute; Kubernetes and a sandbox runtime are the floor. Gateways and MCP operators work alongside all three.
Agent Sandbox: one isolated, stateful pod per agent
Agent Sandbox is a Kubernetes SIG Apps project. It reached v1.0 in August 2026 with a v1beta1 API, and Google has offered it as a generally available GKE feature since May 2026.
It adds a Sandbox custom resource: one short manifest gets you a single long-running pod with a stable identity and persistent storage, isolated by gVisor or Kata Containers, which you can run, suspend or delete on a schedule. Without it, you wire a StatefulSet of one, a Service and a PVC by hand, with no suspend, no expiry and no simple API an agent can call. A SandboxWarmPool keeps pre-started sandboxes ready and a SandboxClaim takes one, so an agent gets a sandbox without waiting on scheduling or an image pull.
The tradeoff is in what survives a pause. Suspending deletes the pod and keeps the disk, so on resume the process starts over and its memory is gone. On GKE, Pod Snapshots can save memory too.
Agent Substrate: many idle agents on a few machines
Agent Substrate is a Google-founded project, pre-1.0, that passed its CNCF Sandbox vote in October 2026. It starts from a different premise: most agent sessions are idle most of the time.
Substrate maps many actors (stateful sessions such as agents) onto a small pool of ready workers (pods). When an actor goes idle, Substrate snapshots its memory and filesystem to object storage and frees the worker. When a request arrives, it restores the actor onto any free worker, and the process continues where it stopped. The project claims resumes in under 400 milliseconds and 10x to 30x oversubscription; its demo runs about 250 stateful actors on 8 pods.
The design choice that matters is where scheduling happens. Kubernetes still provisions and autoscales the worker pods, but Substrate runs its own control plane beside Kubernetes and decides in milliseconds which actor lives on which worker, so the hot path never touches the Kubernetes API server. Every actor runs in gVisor or a microVM, gets its own identity and starts with outbound traffic denied. Its APIs may still change significantly.
Agent Executor (AX): agent tasks as declarative objects
AX is Google’s open source orchestrator for agent workloads, at v0.3 and in heavy development. You declare a Task with a goal, a Workspace that brings Git repositories, MCP servers and skill packages, and a Model, then run ax apply. AX runs each task as a sandboxed actor on Agent Substrate, which it requires, so you can suspend an idle agent, resume it or shell into it. Substrate runs actors; AX adds the agent-shaped layer that says what to work on, with which tools and which model.
AX changed shape in September 2026. Google’s launch post describes a runtime built on a durable event log; the current project is the orchestrator described here, with more breaking changes expected.
kagent: agents, models and tools as Kubernetes resources
kagent is a CNCF Sandbox project, created at Solo.io. An Agent resource brings together a system prompt, a model and a set of tools, and tools come from MCP servers registered as RemoteMCPServer resources shared across agents. A Deployment does not know what a prompt or a tool is; kagent makes them first-class objects you manage with kubectl and GitOps. It ships MCP tools for Kubernetes, Istio, Helm, Argo, Prometheus, Grafana and Cilium, so its sweet spot is agents that operate the cluster itself.
kagent is also changing runtimes. The stable 0.10 line runs agents as ordinary Deployments, with a SandboxAgent resource to run one on Agent Substrate. kagent 1.0, in alpha, runs every agent on Substrate and adds a Harness resource to choose the runtime: kagent’s own Go or Python ADK, Codex, Claude or a custom one.
How the projects fit together
There are two ways to run isolated agents. Agent Sandbox is a classic controller on the Kubernetes scheduler: one pod per agent, stable today. Agent Substrate brings its own scheduler: many agents per pod, still early. Choose Agent Sandbox when one sandbox serves one user or task for hours, and Substrate when you have far more sessions than you want pods.
The definition layers are converging on Substrate. AX requires it and kagent 1.0 moves onto it. That is the bigger shift in this space. In effect, Google is saying that per-pod scheduling tops out for agent workloads, and a runtime that steps around the Kubernetes API server is becoming the shared execution layer for agent frameworks.
“Pause and resume” means different things. Both compute layers can stop an idle agent and bring it back, but they keep different state.
| Agent Sandbox | Agent Substrate | |
|---|---|---|
| What survives a pause | The agent’s identity and disk | Its memory, filesystem and running processes |
| Where it is kept | A persistent volume; the pod is deleted | A snapshot in object storage; the worker is reused |
| What resume does | Schedules a new pod and restarts the process | Restores the snapshot on any free worker and continues |
Around the core: traffic, tools and identity
An agent also calls models, MCP tool servers and other agents, and each call needs authentication and rules. These projects cover that ring:
- agentgateway, a Linux Foundation project started at Solo.io, proxies agent traffic to models, MCP servers and other agents (A2A) with authentication, access rules and rate limits, and has its own Gateway API controller.
- ToolHive, from Stacklok, runs each MCP server in its own container with access policy on every request, and has a Kubernetes operator. It is pre-1.0.
- kube-agentic-networking, a SIG Network subproject at v0.2, is defining standard Kubernetes APIs for which agent may call which tool.
- Dapr Agents, part of the CNCF-graduated Dapr project, is a Python framework for agents that survive crashes, built on Dapr workflows. It reached 1.0 in March 2026.
- OpenKruise Agents and OpenSandbox are alternative sandbox layers with warm pools and pause and resume.
The four core projects side by side
| Agent Sandbox | Agent Substrate | AX | kagent | |
|---|---|---|---|---|
| Question it answers | How do I give one agent an isolated, stateful pod? | How do I run far more agents than pods? | How do I declare an agent task and run it at scale? | How do I define agents, models and tools as Kubernetes resources? |
| Core objects | Sandbox, SandboxWarmPool, SandboxClaim | Actor, Worker, WorkerPool | Task, Workspace, Model | Agent, RemoteMCPServer, Harness (1.0) |
| Who runs it | Kubernetes SIG Apps | Google-founded, CNCF Sandbox | CNCF Sandbox, created at Solo.io | |
| Maturity | v1.0, API v1beta1, GA on GKE | v0.6, pre-1.0 | v0.3, heavy development | v0.10 stable, 1.0 in alpha |
What to build on today
- For isolated agent compute you depend on now, use Agent Sandbox. It has a stable API, an upstream home in Kubernetes and a GA offering on GKE.
- For agents as Kubernetes resources, start with kagent, especially for cluster operations, and watch its move to Substrate in 1.0.
- Prototype on Agent Substrate and AX, but don’t depend on them yet. Both say plainly that their APIs will change.
- Put a gateway in front of models and tools early. It is easier to add before agents multiply than after.
Where Signadot fits
Signadot doesn’t run or host agents; we work alongside these projects. We build ephemeral environments for testing a code change against the rest of a running system: only the changed services are deployed, wired into a shared baseline in the cluster through request and message routing. Both kinds of tool get called sandboxes; the Kubernetes sandbox explainer separates the two. A growing share of the changes tested this way now come from agents.
An agent with a shell in one of these runtimes can already use the Signadot CLI, or the MCP server that ships with it, to create an environment for its change, run tests in it and read the results. Each project also has a natural place for a closer integration:
- An Agent Sandbox
SandboxTemplatecan include the CLI in the sandbox image, so every claimed sandbox starts with it. - An AX
Workspacealready brings MCP servers and skill packages to each task. - kagent connects agents to tools through
RemoteMCPServerresources. - On Agent Substrate, which denies outbound traffic by default, an egress policy can allow Signadot’s API, and its egress gateway could inject the API key so it never sits inside the sandbox.
We are exploring these integrations and would like to hear from teams running agents on these projects.
Frequently asked questions
How do you run AI agents on Kubernetes?
Combine an isolation layer with a way to define the agent. Agent Sandbox, a Kubernetes SIG Apps project, gives each agent an isolated, stateful pod using gVisor or Kata Containers, and is the stable choice today. Agent Substrate packs many mostly idle agents onto a few pods. kagent and Google's Agent Executor (AX) let you declare agents and agent tasks as Kubernetes-style resources. Gateways such as agentgateway control which models and MCP tools agents can reach.
What is kubernetes-sigs/agent-sandbox?
Agent Sandbox is a Kubernetes SIG Apps project that adds a Sandbox custom resource for isolated, stateful, singleton workloads such as AI agents. Each sandbox is one pod with a stable identity and persistent storage, isolated with gVisor or Kata Containers, and warm pools let an agent claim a ready sandbox in under a second. It reached v1.0 in August 2026 with a v1beta1 API, and Google offers it as a generally available GKE feature.
What is the difference between Agent Sandbox and Agent Substrate?
Agent Sandbox runs one pod per agent through the normal Kubernetes scheduler, and suspending an agent deletes its pod but keeps its disk. Agent Substrate runs its own control plane beside Kubernetes and multiplexes many agents onto a small pool of pods, snapshotting an idle agent's memory and filesystem and restoring it on any free pod. Agent Sandbox is stable; Agent Substrate is pre-1.0 and joined the CNCF Sandbox in October 2026.
What is the difference between kagent and Agent Sandbox?
They solve different problems. kagent defines an agent, its model and its MCP tools as Kubernetes resources and runs it. Agent Sandbox provides isolated compute for an agent's code and has no notion of a prompt, a model or a tool. kagent 1.0, now in alpha, runs its agents on Agent Substrate.
What is Google's Agent Executor (AX)?
AX is an open source, declarative orchestrator for agent workloads from Google. You describe a Task with a goal, a Workspace that brings Git repositories, MCP servers and skills, and a Model, then apply them to a cluster. AX runs each task as a sandboxed actor on Agent Substrate, with suspend, resume and shell access. It is in heavy development, and its APIs will change.
Which project should I use to run AI agents on Kubernetes today?
For isolated agent compute you can depend on now, use Agent Sandbox: it has a stable API, an upstream home in Kubernetes SIG Apps and a generally available offering on GKE. To define agents, models and tools as Kubernetes resources, start with kagent. Agent Substrate and AX are promising but early, and both say their APIs will change, so prototype on them before depending on them.
