Customer Story How Bitso cut change failure rate 83% while scaling delivery with coding agents

Local Development on Kubernetes: The Complete Guide

How should you do local development when your services run on Kubernetes? Compare the four workable approaches, local clusters, sync tools, connecting your laptop to a shared cluster, and cloud development environments, against the three dimensions that decide the choice: cost, quality, and speed.

Local development on Kubernetes means iterating on a service that will ultimately run in a cluster, with a feedback loop fast enough to keep you in flow and an environment faithful enough that passing locally predicts passing in production. Every way of setting this up is a position on three dimensions: speed, how quickly a change can be tried; quality, how much a passing test actually proves; and cost, what the environment consumes in infrastructure and upkeep.

There are four workable approaches. You can run a scaled-down copy of the system on your machine with a local Kubernetes cluster, automate the rebuild and redeploy loop with a sync tool, run only the service you are changing and connect it to a shared remote cluster, or move the development workspace off the laptop entirely into a cloud development environment. Each approach settles the three dimensions differently, and this guide takes them in order before comparing them directly.

1 · Local clusterlaptopCode: laptopDependencies: laptop2 · Sync toolswatch · build · syncCode: synced to clusterDependencies: cluster3 · Shared clusterlaptopCode: one serviceDependencies: shared cluster4 · Cloud workspaceslaptopCloud workspaceCode: cloud workspaceDependencies: remote
The four approaches: where the code runs and where its dependencies run.
1 · Local clusterCode: laptopDependencies: laptop2 · Sync toolsCode: synced to clusterDependencies: cluster3 · Shared clusterCode: one serviceDependencies: shared cluster4 · Cloud workspacesCode: cloud workspaceDependencies: remoteCloud
The four approaches: where the code runs and where its dependencies run.

Why localhost stops working

A microservices system outgrows a laptop in predictable stages. Two or three services and a database run fine under Docker Compose. Somewhere between ten and twenty services, the combined footprint of the system, its datastores, its message queues, and its sidecars exceeds what a development machine can hold. Past that point nobody runs the whole system locally, whatever the onboarding wiki claims.

The standard response is to mock the dependencies you cannot run. Mocks keep the loop fast, but they encode yesterday’s understanding of a contract. As other teams ship changes, the mocks drift from the behavior of the real services, and the drift surfaces late, in staging or in an incident, where it is most expensive to fix. The local versus remote environments tradeoff has been argued for years, and mock drift is the recurring complaint that drives teams away from pure-local setups.

Dependency behaviorreal service, changes each releaseyour mock, frozen at day onerelease 1release 2release 3timealigned on day onedriftthe gap widens with every releaseand surfaces in staging or an incident
Time runs left to right, one tick per release of the dependency. The green line is what the real service does, the dashed line is what your mock still does, and the shaded area is drift your local tests cannot see.
real servicemock, frozen at day oneDependency behaviorrelease 1release 2timedriftthe gap surfaces in staging
Time runs left to right, one tick per release of the dependency. The green line is what the real service does, the dashed line is what your mock still does, and the shaded area is drift your local tests cannot see.

Configuration drifts too. In-cluster DNS, service discovery, ConfigMaps, secrets, resource limits, and mesh sidecars have no faithful equivalent in a plain process-on-a-laptop setup. A service that behaves on localhost can fail in the cluster for reasons unrelated to the code change, which trains developers to distrust local results.

The outcome is a familiar failure mode. The inner loop stays on the laptop and becomes less trustworthy, integration testing moves to a shared staging environment and becomes more contended, and the time between writing a change and trusting it grows. Each approach below is an attempt to shorten that distance from a different direction.

Option 1: Run a local Kubernetes cluster on your machine

The most direct answer is to run a real cluster locally. Several mature tools do this well, and they differ mainly in how they trade startup time and resource use against similarity to a production cluster.

  • minikube runs a full single-node cluster, optionally multi-node, inside a virtual machine or containers, with an addon system covering ingress, dashboards, and load balancer emulation.
  • kind runs Kubernetes nodes as Docker containers and was built for testing Kubernetes itself, which makes it fast to create and destroy and a common choice in continuous integration (CI).
  • k3s is a certified lightweight distribution packaged as a single binary, aimed at edge and resource-constrained machines. k3d wraps it in Docker containers for the same convenience kind offers.
  • Docker Desktop bundles a one-checkbox cluster, provisioned as kind nodes since early 2025, so images built locally are immediately visible to it.

The differences between these distributions are covered in more depth in the guide to lightweight Kubernetes environments.

Some teams sidestep the cluster entirely and keep local development on Docker Compose, defining the service set as plain containers. This is the fastest option available and the furthest from production. Compose now ships a bridge that generates Kubernetes manifests from Compose files, which narrows the format gap but not the behavioral one, since none of the cluster machinery listed above exists in a Compose stack.

A local cluster is the right tool in specific situations:

  • a small number of services that one machine can hold comfortably
  • platform components like admission controllers, operators, or Helm charts, where the cluster itself is the subject of the work
  • offline development
  • CI pipelines that need a disposable cluster per run

It breaks down on system size and on state. The resource ceiling is absolute, and no local distribution changes the arithmetic of running fifty services on sixteen gigabytes of memory. Local datastores start empty or rely on seed scripts that age poorly. And the local cluster’s manifests, versions, and configuration drift from production unless someone actively maintains parity, a cost that lands on every individual developer.

Option 2: Automate the inner loop with sync tools

The second approach keeps your code running inside a cluster, local or remote, and automates the loop that gets changes there. These tools watch your source tree and rebuild, redeploy, or sync files into running containers on every change.

Editorsource treechangeSync toolSkaffold · TiltDevSpacebuild · syncCLUSTER, LOCAL OR REMOTEyour applogs and status back to the developer
Sync tools automate the watch, build, deploy cycle so every save lands in the cluster.
Editorsource treefile changeSync toolSkaffold · Tilt · DevSpacebuild or live-syncCLUSTERyour applogs and status
Sync tools automate the watch, build, deploy cycle so every save lands in the cluster.
  • Skaffold, from Google, pipelines the watch, build, tag, deploy cycle. It remains actively developed, shipping regular releases and underpinning Google Cloud Deploy.
  • Tilt defines the development environment as code in a Starlark Tiltfile and adds live-update rules that sync files into running containers to skip image rebuilds, plus a web interface showing service health and logs. Docker acquired Tilt in 2022 and continues to maintain it, though feature velocity has been low since.
  • DevSpace, a Cloud Native Computing Foundation (CNCF) sandbox project, deploys your application and then swaps a development container into the cluster with bidirectional file sync and port forwarding, so changes hot-reload in place.
  • Okteto builds a development platform around the same loop, provisioning a namespace-isolated copy of the application per developer on your own cluster and syncing local files into a development container inside it. It has repositioned in recent years around provisioning those environments for coding agents as well as humans.

Sync tools solve a real problem, which is that the unassisted build, push, redeploy loop on Kubernetes is far too slow for iterative work. Applied to a local cluster they make Option 1 more livable. Applied to a remote cluster they remove the laptop resource ceiling, since your code runs in the cluster rather than on your machine.

What they do not answer is where everything else runs, and, for remote clusters, who else is using them. A sync tool deploys your in-progress code into whatever cluster it points at, so that cluster must provide isolation. Either each developer or coding agent gets a dedicated cluster, or each gets a dedicated namespace holding a copy of the system’s services. Point two developers at the same namespace and they collide: one engineer’s half-finished deployment starts answering the other’s requests. Okteto is the clearest expression of the isolated model, one namespaced environment per developer, and it works. But it is isolation by duplication: every developer needs their own copy of the dependencies, which sets up the cost problem the comparison below returns to.

Option 3: Develop locally against a shared Kubernetes cluster

The third approach inverts the problem. Instead of bringing the system to your machine, you run only the service you are changing, as a normal local process under your debugger, and connect it into a shared cluster where every dependency is real and current. Tools in this category differ in how they wire the connection.

YOUR LAPTOPcheckout (modified)under your debuggertunnelSHARED CLUSTER · DEPENDENCIES AT CURRENT VERSIONSfrontendorderspaymentsother requestscheckoutstablereal datacalls checkoutto your laptop
Your requests are redirected down the tunnel to the modified checkout on your laptop. Everyone else's traffic keeps using the stable checkout in the cluster.
SHARED CLUSTERfrontendorderspaymentscheckoutstableother requeststunnelYOUR LAPTOPcheckout (modified)
Your requests are redirected down the tunnel to the modified checkout on your laptop. Everyone else's traffic keeps using the stable checkout in the cluster.
  • Telepresence, a CNCF sandbox project, creates a virtual network interface connecting your workstation to the cluster and intercepts traffic destined for an in-cluster deployment through a sidecar agent, redirecting it to your local process either globally or per request header. The open source project remains actively maintained. Its commercial story changed substantially, with Ambassador Labs folding the commercial edition into its Blackbird platform in 2025 before being acquired by Gravitee, a shift that pushed many teams to reevaluate the Telepresence-style toolchain. Operationally, the daemon requires elevated privileges for network configuration, and virtual private network (VPN) software on corporate machines is a recurring source of conflict with its network interface.
  • mirrord, from MetalBear, takes a process-level approach instead of a network-level one. It injects a layer into your local process that hooks system calls, so the process reads the target pod’s environment variables and files and receives mirrored or stolen traffic through an in-cluster agent, without a VPN-style interface or root access. Its enterprise edition adds header-based traffic filtering so multiple engineers can share one target.
  • Gefyra takes a related approach for containerized local workloads, bridging local containers into the cluster network.
  • Signadot connects the workstation to the cluster through an authenticated tunnel and applies the same run-one-service-locally pattern, with the difference that the connection is expressed as a sandbox, a declarative unit the platform manages, rather than an ad hoc intercept. How that model handles many concurrent developers is covered in the sharing section below, and the practical differences from Telepresence and mirrord are broken out in the Telepresence and mirrord comparison pages.

The strengths of this category are exactly what Options 1 and 2 lack. The feedback loop is a local process restart, seconds rather than a rebuild cycle. Dependencies are the real services at their current versions, with real data and real configuration, so mock drift disappears for everything you did not change. Laptop resource use is one service instead of fifty.

The costs are organizational more than technical. There must be a shared cluster that reflects production, and someone must own keeping it healthy, typically a platform team. And once more than a handful of developers connect to the same cluster, their changes can collide, one developer’s modified service answering another developer’s requests. Solving that collision problem is what separates the tools in this category at team scale, and it is where this guide ends up in the final section.

Option 4: Cloud development environments (CDEs)

The fourth approach relocates the development workspace itself. Cloud development environments, often shortened to CDEs and also called remote development environments, give each engineer, or each task, a cloud machine holding the source tree, toolchain, and editor backend, accessed from a thin client or the browser.

  • GitHub Codespaces provisions per-branch cloud machines defined by devcontainer configuration. It is not Kubernetes-aware by itself, so teams either run kind or minikube inside the workspace or point it at a remote cluster, inheriting the tradeoffs of whichever they choose.
  • Coder is a self-hosted platform that provisions workspaces on your own infrastructure from Terraform templates, aimed at organizations that want development environments behind their own network boundary, and increasingly at running fleets of coding agents inside them.
  • Ona, formerly Gitpod, provisions ephemeral cloud development environments defined as code, with prebuilt workspaces that start in seconds, and has expanded toward orchestrating fleets of coding agents in those same environments.

CDEs fit teams that value fast onboarding, standardized toolchains, and code never residing on laptops, which matters in regulated environments. They also fit the emerging pattern of running many coding agents, since agents need environments provisioned programmatically and laptops are not where that happens.

What a CDE does not do is answer the testing question. It moves the development environment into the cloud, and nothing else: the services your code depends on still have to run somewhere, and that somewhere is still one of the first three options, a local cluster inside the workspace, a sync tool pointed at an isolated namespace, or a connection to a shared cluster. Choosing a CDE changes where you develop, not how you validate, so the test environment decision remains open and just as consequential as before.

Choosing an approach: cost, quality, and speed

Strip away the tooling details and every Kubernetes development environment is a position on three dimensions. Speed is the pace of the inner loop: the time from saving a change to seeing it behave, including any waiting on rebuilds, provisioning, or other people. Quality is the fidelity of the signal: how well a pass in the development environment predicts a pass in production. Cost is what the environment consumes in infrastructure spend and platform upkeep, multiplied by every person and coding agent that needs one.

No traditional setup delivers all three. Each of the standard positions sacrifices a different dimension to get the other two.

Speedfast feedbackCostcheap to runQualityhigh fidelityLocal environmentsquality lost to mocksDuplicated environmentscost explodes per copyShared stagingspeed lost to queueingMulti-tenantshared clusterspeed, quality, and cost
Traditional setups pick two of the three dimensions and sacrifice the third. A multi-tenant shared cluster is the position that holds all three.
Speedfast feedbackCostcheap to runQualityhigh fidelityLocal environmentsquality lost to mocksDuplicated environmentscost explodes per copyShared stagingspeed lost to queueingMulti-tenantshared clusterspeed, quality, and cost
Traditional setups pick two of the three dimensions and sacrifice the third. A multi-tenant shared cluster is the position that holds all three.

Local environments choose speed and cost. Everything from Docker Compose to a kind cluster on your machine (Option 1) is free to run and fast to iterate in. Quality is what gets sacrificed: past a handful of services, most dependencies must be mocked or seeded, and the mocks drift from reality as the real system ships. The faster the organization ships, the faster that drift accumulates. Event-driven and data-heavy systems feel it earliest, because brokers like Kafka and realistic datastores are exactly what mocks and seed scripts reproduce worst.

Duplicated environments choose speed and quality. Give every developer a private copy of the system, a namespace each in the sync tool model (Option 2) or a full cluster each, and iteration is fast against real dependencies with nobody contending. Cost is what gets sacrificed, and it scales the wrong way: linearly with the number of environments. That was survivable when one engineer meant one environment. In the agentic era it is not. Each engineer now runs multiple coding agents producing changes in parallel, every agent needs its own environment to validate in, and duplication multiplied by agents produces a number no budget approves.

Shared staging chooses cost and quality. One live, continuously deployed copy of the system is the cheapest high-fidelity environment an organization can operate, which is why nearly every organization has one. Speed is what gets sacrificed: with no isolation between changes, developers queue to validate their work, contend for the environment, and a single broken deploy blocks everyone behind it. Connecting laptops to a shared cluster (Option 3) inherits exactly this failure mode once more than a handful of developers connect at the same time, unless something keeps their changes from colliding.

Cloud development environments (Option 4) do not occupy a corner of this triangle at all. They relocate the workspace and then inherit whichever position they are paired with, so adopting a CDE still means choosing among the three positions above, or the fourth one below.

The fourth position is the center of the triangle: share one live environment and make it multi-tenant, so every change is isolated inside it and sharing does not mean colliding. That keeps the shared model’s cost and quality while removing the queue that made it slow. The final section covers how it works. First, the summary:

ApproachCostQualitySpeedBreaks down when
Local environment (local cluster or Compose)Low. Laptop onlyLow. Mocked dependencies drift from realityHigh. Everything at handThe system outgrows the laptop and mocks outnumber real services
Environment per developer (sync tools, namespaces)High. One system copy per developer or agentHigh. Real dependencies in every copyHigh once provisioned. Copies drift between syncsCopy count multiplies with the team, then again with coding agents
Shared staging without isolationLow. One environment for everyoneHigh. Real and continuously deployedLow. Queueing and contentionEvery concurrent change fights for the same environment
Multi-tenant shared clusterLow. One environment plus one service instance per changeHigh. Real dependencies at current versionsHigh. Local process restart, no queueServices do not propagate trace headers
Cloud development environmentPer-seat workspace pricingSet by the paired approachSet by the paired approachThe test environment question is still open

Getting all three: a multi-tenant shared cluster

Every path through the trilemma points at the same destination. A shared cluster is already the cheapest environment an organization runs and the highest-fidelity one. The only thing wrong with it is contention, and contention is fixable.

The fix is to make the shared environment multi-tenant, isolating each change inside it rather than duplicating infrastructure around it. For the synchronous traffic between services, isolation works at the request level. The cluster runs one stable version of every service, continuously deployed. A developer working on a change runs only the modified service, typically as a local process under a debugger, registered as an additional version alongside the stable one. Requests carry a routing key in a header, propagated across service hops through standard context propagation such as OpenTelemetry baggage or B3 headers, and the routing layer sends requests tagged with your key to your modified service while all other traffic flows through the stable versions.

SHARED CLUSTER · ONE STABLE VERSION OF EVERY SERVICEAB · stableB · modifiedCkeyno keyrouting key propagated in headers across hops
Request-level isolation: one live system, with each change reachable only by requests that carry its key.
keyno keySHARED CLUSTERAB · stableB · modifiedCrouting key propagated in headers
Request-level isolation: one live system, with each change reachable only by requests that carry its key.

Measured against the three dimensions, this is the center of the triangle rather than another compromise:

  • Cost stays low. The whole team, and its coding agents, share one cluster. Each concurrent change adds one service instance, not one system copy, so the marginal cost of the fiftieth simultaneous change is a single process.
  • Quality stays high. Every dependency is the real service at its current version, with real data and real configuration. Nothing is mocked, so nothing drifts.
  • Speed stays high. The inner loop is a local process restart, seconds rather than a rebuild or a redeploy, and there is no queue, because requests isolated by key cannot collide with anyone else’s changes.

Two requirements follow from the mechanism. Services must propagate trace context headers, which most teams already have through distributed tracing instrumentation. And something in the data path must make routing decisions per request, either an existing service mesh or dedicated sidecars.

How Signadot implements the pattern

Signadot packages this model as its core primitive. A sandbox is a lightweight environment defined declaratively, holding the changed version of one or more services while everything else resolves to the stable versions in the shared cluster. For local development specifically, the CLI connects your workstation to the cluster over an authenticated tunnel, cluster service names resolve from the laptop as if the process were running inside the cluster, and the sandbox maps an in-cluster workload to the local process, pulling down its environment variables and files so the local service starts with in-cluster configuration. Requests carrying the sandbox’s routing key reach the laptop, and all other traffic never sees it.

Control planeauth and SSOYOUR LAPTOPmodified serviceSignadot CLItunnelKUBERNETES CLUSTERSignadot Operatorsvc Asvc Bsvc Csidecarroutingsandboxmaps svc B to the local process
The sandbox maps one in-cluster workload to the process on your laptop. Only requests with its routing key are sent down the tunnel.
Control planeauth and SSOKUBERNETES CLUSTERSignadot Operatorsidecar routingsvc Asvc Bsvc Csandboxmaps svc B to the local processtunnelYOUR LAPTOPmodified serviceSignadot CLI
The sandbox maps one in-cluster workload to the process on your laptop. Only requests with its routing key are sent down the tunnel.

Routing runs through a built-in sidecar or through an existing Istio or Linkerd mesh. Route groups combine multiple sandboxes under a single routing key, so a change spanning several services, some local and some in-cluster, tests as one unit. Access flows through a central control plane with single sign-on rather than through per-developer cluster credentials, which keeps the shared cluster workable for teams that must keep credentials and data centrally controlled.

Request routing is only the mechanism for synchronous traffic, not the whole model. For message queues like Kafka, isolation moves into the messages themselves: routing keys carried in message headers decide whether the stable consumer or a sandboxed one processes each message. For databases, sandboxes take different approaches depending on the engine and the isolation a change needs, from an ephemeral schema or instance per sandbox to branches on copy-on-write databases like Neon. And because these mechanisms sit on an extensible plugin framework, any system in the stack, including internal ones no vendor ships support for, can be made multi-tenant in the same shared environment rather than duplicated alongside it.

The same primitives serve the local development workflow and the pull request validation workflow described in the ephemeral environments guide, so the environment a developer debugs in and the environment CI validates in are the same kind of thing.

The honest tradeoffs run in the same directions as the rest of Option 3. There is an operator to run and a platform team decision to make, header propagation is a prerequisite, and isolating shared state like databases and message queues requires explicit configuration per resource rather than coming free with the routing. Teams whose systems are small enough to hold on a laptop do not need any of it yet.

Develop locally against your real cluster

Signadot connects your laptop to a shared Kubernetes cluster so you can run one service under your debugger while every dependency stays real, with sandbox isolation keeping your changes invisible to the rest of the team. The free tier is open to every developer.

Start free

Frequently asked questions

What is the best way to develop locally against a shared Kubernetes cluster?

Run only the service you are changing on your machine and connect it to the cluster with a tool that routes requests by header, so your changes are isolated from other developers using the same cluster. Telepresence, mirrord, and Signadot all implement variants of this, differing in connection mechanics, concurrency handling, and how much platform tooling surrounds the core workflow.

Can I run Kubernetes locally on my laptop?

Yes. minikube, kind, k3s, and Docker Desktop all run real clusters on a development machine, and they are the right choice for small systems and for platform-level work like operators and Helm charts. The limit is system size, since a laptop cannot hold a large microservices deployment with its datastores, so most teams outgrow purely local clusters as the service count climbs into the tens.

Do I need a service mesh to share a development cluster?

No. Header-based routing needs something making per-request decisions in the data path, and an existing Istio or Linkerd mesh can serve that role, but platforms in this space also ship their own lightweight sidecar routing for clusters without a mesh. The harder requirement is context propagation, since services must forward tracing headers for routing keys to survive multi-hop calls.

Should every developer get their own namespace instead?

Per-developer namespaces work at small scale and are the natural first architecture to try. Their cost grows linearly with team size, each copy takes minutes to provision and drifts from the live system between syncs, and datastores and third-party dependencies usually end up shared anyway. Teams typically move to request-routed sharing of one environment when the namespace fleet's cost or staleness becomes visible.

Is Docker Compose good enough for Kubernetes development?

For a small number of services it is often the fastest loop available, and that speed is worth real fidelity costs. What Compose cannot reproduce is cluster behavior, including service discovery, ConfigMaps and secrets, resource limits, ingress, and sidecars, so services pass locally and fail in the cluster for environment reasons. Teams usually pair it with a real cluster stage or replace it once those failures become frequent.

Stay in the loop

Get the latest updates from Signadot

Validate code as fast as agents write it.