What Ephemeral Environments Cost, and How Teams Keep the Bill Under Control
What drives the cost of ephemeral environments, a full copy per pull request versus only what changed, the hidden costs, the controls that work, and what Brex, Arkestro and Yellow.ai report.
Ephemeral environments cost whatever they run, for as long as they run. Copy the whole application for every pull request and the bill grows with the number of services times the number of open pull requests. Duplicate only what each change touched and plug it into a shared environment, and the bill grows with the changed services alone, which at any real concurrency is an order of magnitude less.
This article is the cost part of the complete guide to ephemeral environments, which covers what they are, how they work on Kubernetes and the tools that build them. The question here is narrower: what drives the bill, where the costs hide, and what teams do to keep an environment per pull request affordable as coding agents multiply the number of changes. For the per-pull-request case and the platforms that offer it, see the complete guide to preview environments.
What drives the cost of ephemeral environments
Five things set the bill, and only the first three show up on a cloud invoice:
- What each environment runs. Every service, database and queue in an environment reserves CPU and memory, and the node count follows those requests.
- How many exist at once, and for how long. An environment for a pull request that waits two days for review runs for two days, whether or not anything tests it.
- Data. A database restored or seeded per environment costs storage and minutes of wall-clock time on every pull request.
- Waiting. Minutes of spin-up per environment, multiplied by every developer, CI job and agent that needs one.
- Operations. Engineering time to keep the environment automation, the data seeding and the shared services working.
The model you choose sets the first item. Lifetimes and cleanup set the second. Everything else follows from those two.
A full copy per change vs duplicating only what changed
Take a system of 40 services where a typical change touches two. A full copy per change, in its own cluster or namespace, runs 40 service instances per environment. Duplicating only what changed runs the 40 stable versions once and adds two instances per change. The table counts running instances, which is what drives node count and the cloud bill.
| Concurrent environments | Full copy per change | Only what changed |
|---|---|---|
| 1 | 40 instances | 42 instances |
| 10 | 400 instances | 60 instances |
| 30 | 1,200 instances | 100 instances |
| 100 | 4,000 instances | 240 instances |
| 300 | 12,000 instances | 640 instances |
The same arithmetic holds at any size. A team with 20 services and 10 feature branches open at once runs 200 service replicas with full copies, and about 40 when each branch duplicates the two services it changed. In dollars, a 50-service system that fits on a single 32-vCPU, 128 GiB node such as an m6a.8xlarge costs about $1.38 an hour at AWS on-demand prices, roughly $12,000 a year per always-on copy, so 50 copies cost more than half a million dollars a year before storage and data transfer.
Three things follow from the shape:
- Duplicating only what changed has a fixed cost, the shared stable versions, which is why it is not cheaper at one environment. The gap widens with every environment after the first.
- A full copy bills for concurrency, not for the size of the change. A one-line fix and a schema migration cost the same 40 instances, and 100 open pull requests pay for 100 copies of the system.
- Most of a full copy is waste. Brex ran more than 800 services so a developer could work on one or two of them, and on the margin 99.8 percent of a duplicated environment’s infrastructure cost went to services nobody was testing.
The models that build ephemeral environments on Kubernetes sit along the same line:
| Model | What each environment runs | Spin-up | Cost as environments multiply |
|---|---|---|---|
| Cluster per environment | Every service, every datastore and a control plane | Tens of minutes, under a minute for a virtual cluster | Highest. Grows with the whole stack plus cluster overhead |
| Namespace per environment | Every service and datastore, on shared nodes | Minutes, growing with the application | High. Grows with the whole stack |
| Only what changed, on a shared cluster | The changed services, plus a database branch or topic when the change needs one | Seconds | Low. Grows with the changed services, on top of one shared set |
Isolation models for ephemeral environments compares the models on isolation and fidelity as well as cost.
The costs that never reach the invoice
The compute bill is the visible part. Four other costs are usually larger.
- Idle time. A full copy runs from the moment a pull request opens until it merges, and most of that time nothing is testing it. Reviews take hours or days, and the environment bills for all of them.
- Waiting. Brex’s namespace-based previews took 30 to 60 minutes to deploy, and moving to duplicating only what changed brought that under five minutes. Multiply the difference by every pull request and every rerun. A shared staging environment has the same cost in another form, a queue of changes waiting for their turn.
- Overprovisioning. Test environments often copy production’s resource requests, sized for production traffic, into every copy. Environments that serve a handful of test requests need a fraction of that, and copied requests are paid for whether or not they are used.
- Building and running the system. Teams that build environments per pull request in-house spend years of platform time on it. Bitso spent a few years on its own full-copy preview environments and found them expensive in both time and cost, and Arkestro put the implementation cost it avoided by not building in-house at $120,000.
How teams keep ephemeral environment costs under control
Three controls matter more than the rest: duplicate only what each change needs, put a time to live (TTL) on every environment, and size test environments for test load rather than production load. The rest are refinements.
- Duplicate only what changed. The largest single lever, because it changes what the bill scales with. Each change duplicates the services it touched, and a database or queue only when it writes destructively, while request routing and message routing connect those copies to the shared environment.
- Put a TTL on every environment, and delete on merge or close. Most of an environment’s life is spent waiting for review. Yellow.ai’s platform team called Signadot cost effective because of the TTL on sandboxes and because only the application under test is deployed.
- Sleep what you cannot delete. Full-copy environments that must stay up for a long review can scale to zero after a period of inactivity and wake on the next request.
- Right-size requests. Set CPU and memory requests for test traffic, run one replica of each service, and keep autoscalers at their minimum in environments that nobody loads.
- Use cheaper capacity. Run environment workloads on a node pool of spot or preemptible instances, since an environment that loses a node can be recreated.
- Branch or share data instead of copying it. A database branch from Neon or Xata, or the shared database with each environment’s test data kept apart, avoids a full restore per pull request.
- Cap and measure. Label every environment with its pull request and team, attribute cost with a tool such as OpenCost, set a concurrency limit per team, and review environment count, cost and spin-up time monthly.
A short checklist covers the same ground:
- Each change deploys only the components it changed.
- The stable services and infrastructure are shared, not copied.
- Environment creation and cleanup are automated, from CI and from the CLI.
- Existing CI pipelines and test frameworks run against the environment unchanged.
- Cost per environment is a number someone reviews.
Cost at scale: hundreds of environments and coding agents
Coding agents change the concurrency number, and concurrency is what full copies bill for. A 50-engineer team that opened 100 to 150 pull requests a day can face closer to 1,000 once agents write much of the code, and an agent wants an environment for every attempt in its build, test and fix loop, not one per feature. At Bitso, pull requests per developer rose 60 percent in a year as agents took on more of the work.
As an illustration, 1,000 environments a day that each live two hours average about 80 at once, and peaks run several times higher. At 250 concurrent environments of the 40-service system above, a full copy per change runs 10,000 service instances and duplicating only what changed runs 540. Teams that operate in several regions or clusters multiply the full-copy figure again for each one, while duplicating only what changed adds one shared set per cluster. At that scale the model is the cost decision, and TTLs and right-sizing tune the result.
What pricing models ephemeral environment platforms use
Platforms for ephemeral environments charge in one of three ways: per seat, by usage such as environments created or environment hours, or a platform fee while the environments run in your own cloud account and the compute lands on your cloud bill. Whichever model a vendor uses, the compute the environments consume is paid for somewhere, so the number of instances each environment runs matters more than the line item it appears on. Signadot’s Business plan is usage-based, metered on sandboxes created per month, test invocations and concurrent local connections, with the environments themselves running on your own cluster, as the pricing page details.
What teams report
- Brex replaced namespace-based previews across more than 800 microservices and cut infrastructure cost by 99 percent, about $2 million a year, while previews went from 30 to 60 minutes to under five.
- Arkestro reported more than $1 million a year in developer bandwidth, from a 12.5 percent increase across a 40-person engineering team, five hours saved per developer per week, and $120,000 in implementation cost avoided.
- Yellow.ai tried a virtual cluster per developer, found it cost quite a lot and was hard to keep in sync, and moved to one shared cluster where each change deploys only the application under test and every environment carries a TTL.
Getting started
The path to lower-cost ephemeral environments on a cluster you already run has four steps:
- Install the Signadot operator in the cluster you already run, such as staging, next to the stable versions of your services.
- Create an ephemeral environment per pull request from CI, as the PR sandboxes guide shows, or per task from a coding agent through the CLI or the MCP server. Each one duplicates only what the change needs and carries a TTL.
- Run your existing tests against it: Playwright, Cypress or Selenium for the UI, Postman or RestAssured for APIs, or in-cluster with Jobs. Smart Tests compare the changed version of a service against the stable one to flag regressions.
- Post the result back to the pull request and let the TTL remove the environment.
Related reading
- Ephemeral environments: the complete Kubernetes guide
- Isolation models for ephemeral environments: full copy vs shared cluster
- Smart ephemeral environments: share more, copy less
- How Brex scales developer testing across hundreds of engineers
- Cut the cost of Kubernetes preview environments
- The complete guide to staging environments
