Coding Agents Run Their Own Code Now. How Much of the System Does That Reach?
Greptile, Cursor, Codex and Devin now execute the code an agent writes. In a microservices system, a private runtime with mocks still misses the bugs that live between services.
Agent tools no longer stop at the diff. Greptile’s TREX spins up a sandbox to run the code under review and attaches logs, screenshots and API traces to its comments. Cursor’s cloud agents clone the repository into an isolated VM where they can build, test and interact with the changed software. OpenAI’s Codex runs each task in its own cloud environment preloaded with the repository, and Devin works with its own shell, VS Code editor and browser.
Different products, one decision: before a person reviews an agent’s change, the agent should run it and look at what happened. That is the right call. The question for teams on microservices is how much of the system that run includes, which is the core of verifying AI-generated code on Kubernetes.
Why runtime verification became table stakes
Review is where agent output piles up. At Stripe, more than 1,300 pull requests merged each week are produced entirely by its internal coding agents and reviewed by people. Few teams have the senior reviewers to read that volume line by line.
Static analysis has a ceiling. It can read intent from source but cannot observe execution. A race that needs a real request, a regression that appears only after a page renders, a timeout that depends on load: none of it shows up in the text of a diff.
An agent that runs its own code, reads the failure and fixes it before handoff takes that load off reviewers. The open question is fidelity: how real does the runtime have to be before a passing run means the change works?
Four levels of runtime verification
Verification for agent changes falls into four levels, and each one proves something different.
| Level | What runs | What a pass proves | What it misses |
|---|---|---|---|
| 0. Read the diff | Nothing. Static analysis or an AI reviewer | The code looks plausible | Everything behavioral |
| 1. Unit tests with mocks | The changed function, dependencies stubbed | The logic matches the author’s assumptions | Whether the assumptions are right |
| 2. A private runtime per agent | The changed service, local dependencies, mocks for the rest | The service starts and behaves as expected alone | Behavior at the boundaries with real services |
| 3. The change inside the real system | The changed service among live versions of everything it calls | The change works where it will run | Production-only concerns such as real user load |
Most agent tools have moved from level 1 to level 2, and that move matters. For a self-contained application, level 2 is close to complete.
For a cloud-native application built from dozens of services, level 2 covers the smaller part of the risk. The part it misses is how the change behaves next to everything else.
What a private runtime cannot see
A mock hands back whatever response its author predicted. That makes it good at confirming a belief and useless at exposing a mistaken one, and in a private runtime most of those beliefs came from the agent.
The costly bugs in distributed systems sit at the seams between services, where one side’s expectation meets the other side’s behavior:
- A producer and consumer that disagree about a contract
- A payload whose encoding differs from what the receiving service parses
- A retry or timeout setting that causes trouble a couple of services downstream
- A schema migration that breaks a query in a service the agent never opened
None of these fail a unit test or a mocked call. They surface in integration and end-to-end tests that mean something only when the change runs against real services, real data and real traffic. Performance regressions, resource contention and runtime security issues follow the same rule.
Why cloning the system does not get you to level 3
The obvious fix is a full copy of the system for every agent. The numbers rule it out. Recreating dozens of stateful services, each with its data and configuration, for every agent on every iteration is too slow and too expensive at the pace agents produce changes.
A copy also goes stale. It is a snapshot of the system at creation time, not the system your change will meet when it merges.
How level 3 works in practice
Level 3 comes from sharing rather than copying: every agent verifies against one production-like system, with each change isolated from the others.
A shared Kubernetes cluster runs the stable version of every service, kept current by the normal deployment pipeline. To verify a change, the agent deploys only the service it modified. Requests tagged with that change’s routing key reach the modified service and then continue through the shared stable services. Untagged traffic and every other agent’s traffic never touch the change.
The change now executes alongside the real services, datasets and mesh policies it will meet after merge. You can observe integration behavior because nothing in the path is simulated.
It also matches agent workloads. A change adds one or two services to a system that is already running, not a private replica of all of it, so dozens of agents can verify in parallel at low cost and stay out of each other’s way. The lightweight ephemeral environment is short-lived and runs on infrastructure the team already operates.
The agent’s loop does not change: write, run, inspect the failure, retry, with all of it finished before a PR exists. The difference is what counts as a pass. A pass means the change worked against the real system its assumptions describe, not that it agreed with mocks written to match those assumptions.
Where does your verification stop?
Runtime verification is becoming a standard part of how agent changes ship, and the teams furthest along treat the agent’s job as write, prove, debug, repeat. For many changes, a private runtime is enough proof.
For a change that has to work alongside everything around it, the proof needs level 3. Signadot provides it with lightweight ephemeral environments on your Kubernetes cluster, which agents create and drive through the CLI or MCP server. Signadot for agentic development covers the full workflow. The teams that get the most from background agents will be the ones whose verification reaches the whole system, not only the service in front of the agent.
