Code Generation Isn't the Bottleneck Anymore: What Platform Leaders Told Us Over Dinner

Last week, we co-hosted a dinner with Weave Intelligence in San Francisco for about a dozen engineering leaders. Around the table were heads of DevOps and platform engineering, a platform and quality engineering lead, a director of security engineering, a developer experience leader, directors of AI enablement, a chief AI officer, and a couple of CTOs. They came from fintech, cloud communications, enterprise finance software, collaboration and media software, AI-native automation, retail and AI infrastructure.

A room that varied should have produced a spread of opinions and pain points. What surprised me was the consensus. Nearly every thread came back to the same observation: generating code is no longer the hard part. The hard part is everything that happens after an agent opens a pull request, and for most teams in the room, that part is getting slower.

Below are the themes that came up most, with my take on each.

Code got cheap, and shipping didn’t get faster

Some of our customers have seen pull requests (PRs) per developer go from roughly 10 a month to more than 50 since rolling out coding agents. Production deployments have not kept pace. Several leaders described something more uncomfortable than a plateau. Their delivery cycle has slowed, and production incidents have gone up since agents arrived.

The reasons were consistent. Code review does not scale to this volume, because people cannot read five times as much code. When the path to production is painful, developers take shortcuts, and some push straight to production. One platform and quality engineering leader named it as the main challenge for the year: PRs piling up since the coding agents rolled out, with verification as the bottleneck.

My take is that most teams are measuring the wrong thing. The question isn’t how fast an agent produces a unit of code. It’s how many verified changes the organization ships each week.

A delivery pipeline only moves as fast as its slowest stage. Agents sped up the first stage, writing code, while review, CI and staging kept the capacity they had when people wrote every change. The extra output doesn’t ship. It waits at the slowest stage as open pull requests.

Agents made code cheap. Verified code is still expensive, and verified code is the only kind that ships.

Software runs on loops, and ours close too late

The point that resonated most was that software development is fundamentally about loops, and the earlier a loop closes, the cheaper every mistake is. Write, run, observe, fix. When one developer works alone, that loop is tight. When 50 agent PRs a month land in a shared staging environment without being verified first, the loop stretches across days and several teams, and staging becomes the place where everyone’s changes collide.

So shifting left matters more with agents, not less. The difference is that shifting left now has to include the agent itself. An agent should be able to stand up the services its change touches, run the relevant tests and evals against real dependencies, read the errors and try again, all before a person reviews anything.

That’s the problem we work on at Signadot. Every unit of change an agent produces gets its own lightweight ephemeral environment inside the existing Kubernetes cluster, so the agent can verify end to end and keep iterating until the change works. Kaspar made the same point at the table and said Signadot is a product he sees used daily for exactly this.

One story from the evening stayed with me. A platform leader had built an internal preview environment tool, and only frontend teams adopted it. Frontend previews are the straightforward case. Backend services with databases, queues and downstream APIs are where homegrown tools usually stop, and that’s where agents need the most help.

Everyone wants to build now

Every leader at the table described the same pressure from outside engineering. HR, finance and legal teams want to build their own tools, and agents make that possible. One cloud platform leader has a formal directive to let people across departments ship their own internal tools. Another is building a platform so non-technical teammates can ship product changes, the shift we described in your next pull request will come from a product manager.

The concern wasn’t intent. It was operational maturity. Citizen developers typically have no observability, no logs, no metrics and no working model of version control. They don’t think in systems, and an agent writing the code doesn’t transfer twenty years of process knowledge into the software.

The common response was an internal, hosted agent platform with curated connectors, a shared repository of skills, service accounts tied to individual people for authentication, and guardrails built in. I think that’s right, and I’d add one thing. If the number of people who can produce changes goes up by an order of magnitude, the paved road has to include verification, not only deployment. A change from a finance analyst’s agent should get the same isolated environment and the same checks as one from a staff engineer.

The paved road used to end at deploy. With agents and citizen developers on it, it has to start at verification.

From coding agents to software factories

The conversation then zoomed out. One leader described knowledge work as roughly 20 to 25 discrete workflows a year. Some of that is strategic thinking or new creation, but much of it is deterministic process, which is essentially business process automation with smarter steps. One organization has a mandate to cover half of its vertical workflows with agents by the end of Q1.

The most concrete pattern was a triage agent that routes incoming requests by complexity. A trivial or frontend-only change gets an ephemeral environment, gets built and ships to production on its own. A critical change gets built in a PR environment and routed to a person for approval.

One attendee cited a figure of about 3% of organizations already at full autonomy, and expected that share to grow quickly. One company said 17% of its internal identities are now non-human service accounts. Teams heading this way are building software factories, and the checks that tell an agent its change works have to scale with them.

What makes the triage pattern work is a cheap, fast, isolated environment for every change. Without one, the trivial path has nowhere safe to prove itself, and the critical path hands a reviewer a diff with no evidence attached. The risk tier decides who signs off. The environment produces the evidence they sign off on.

Accountability, and the illusion of control

Identity was the least solved problem in the room. When an agent acts on your behalf, who is accountable? The model most people liked treats the agent as a stripped-down version of the person who authorized it, with permissions scoped down from theirs. Another treats agents like contractors, with a named manager who owns the outcome.

Google and Microsoft have early answers, but nobody felt they were robust enough for full autonomy. Some teams are running supervisors outside the agent’s own context and runtime to enforce permission boundaries.

Security drew the sharpest comments. Several leaders said security is still the last consideration rather than the first. A security engineering leader described AI-driven changes piling onto security testing the same way they pile onto code review.

Supply chain risk grows as developers and agents pull in packages nobody has vetted, and prompt injection and rogue agent behavior remain largely unsolved. New monitoring tools are arriving, but a good demo is not production readiness. One leader called the current state an illusion of security: running AI pen tests while S3 buckets upstream sit unencrypted.

The design principle that came out of this is the one I’d underline. Stop designing systems that assume a person is in control. Assume zero trust and no human oversight by default. For test environments, that means agents work in isolation, with scoped credentials, no copies of production data and the network policy the cluster already enforces.

Assume no one is watching the agent, and design its environment on that basis.

What I’m taking away

Put together, here’s where I’ve landed. Agents moved the constraint from writing code to verifying it, and the teams that pull ahead will measure throughput of verified changes rather than speed of generation. That means closing the loop before the PR, giving every change from every author its own environment, routing by risk with real evidence, and assuming no one is watching. The verification layer should also be model-agnostic, because the agent writing the code will keep changing.

Thanks to Kaspar and to everyone who joined us for a candid evening. If you’re working through the same problems, I’d like to compare notes. You can read how Signadot gives every agent and every PR its own ephemeral environment in our docs. Or book a call to talk it through.