# Resources Concepts Guide ## Overview A sandbox forks only the services you changed and shares everything else with the [baseline environment](https://www.signadot.com/docs/concepts/baseline-environment.md), including the data layer. Request-level isolation keeps sandboxes fast and cheap, and for most changes the shared databases and brokers are fine: tests partition their data by tenant or user ID and clean up after themselves. Some changes cannot share stateful infrastructure safely: * A schema migration would break every other service and sandbox that reads the same tables. * A test needs a known seed dataset, or deletes and truncates data. * A sandboxed consumer on a shared topic would take messages that belong to the baseline. * The code under test needs its own cloud resource, such as a bucket, a table, or a VM. For these cases, a sandbox can declare **resources**. A resource is anything whose lifecycle is bound to a sandbox. It is created when the sandbox is created, and destroyed when the sandbox is deleted, including when the sandbox's [TTL](https://www.signadot.com/docs/reference/sandboxes/spec.md#specttl) expires. A resource does not have to be a physical piece of infrastructure. It can be: * A Postgres database on a shared RDS instance, or a whole MariaDB server in the cluster * A copy-on-write branch of a Neon or Xata database * A Kafka topic, an SQS queue, or a RabbitMQ vhost * An S3 bucket, a DynamoDB table, or an IAM role * A VM, a Lambda function, or a test tenant in a third-party SaaS * A one-off script that seeds data or registers a webhook Resources are created and deleted by **[resource plugins](https://www.signadot.com/docs/reference/resource-plugins.md)**. A plugin is a short YAML spec plus the scripts that create and delete one kind of resource. Usually a platform team writes and installs a plugin once. After that, developers, CI pipelines, and coding agents request resources from it by name in their sandbox specs. To go straight to building one, see [Writing Your First Plugin](#writing-your-first-plugin). ## How Resources Work A resource plugin defines two workflows. The `create` workflow runs when a sandbox that references the plugin is created. The `delete` workflow runs when that sandbox is deleted. Each workflow is made up of steps. Each step is a script (bash, Python, or anything else with a `#!` line) that runs in a short-lived *runner* pod in your cluster, on an image you choose. If the image contains `terraform`, `aws`, `psql`, `helm`, `neonctl`, or your own tooling, the step can use it. A few properties make resources a good fit for ephemeral environments: * **Ordering.** The sandbox's workloads start only after the `create` workflow has finished. A forked service never comes up before the database it depends on. * **Outputs are wired in by name.** Each step can publish outputs such as a host, a connection string, or a topic name. The sandbox injects these into forked workloads as environment variables, and into [local workloads](https://www.signadot.com/docs/reference/sandboxes/spec.md#speclocal_) as environment variables or files. No intermediate Kubernetes Secret is needed. * **Sandbox context is available.** Every step receives `SIGNADOT_SANDBOX_NAME`, `SIGNADOT_SANDBOX_ROUTING_KEY` (the sandbox's unique [routing key](https://www.signadot.com/docs/concepts/sandbox.md#request-routing)), and `SIGNADOT_RESOURCE_NAME`. Use them to give each resource a unique name that traces back to its sandbox. * **Plugins are parameterized.** A sandbox passes `params` to the plugin, such as a database name, a parent branch, a region, or a dataset to seed. One plugin can serve many teams. * **Plugins are versioned.** Plugins are published with [semver](https://www.signadot.com/docs/reference/resource-plugins/overview.md#versioning), and each sandbox pins the version it was created with, so you can change a plugin without affecting sandboxes that are already running. Here is a sandbox that requests a database branch and points a forked service at it: ```yaml name: users-migration spec: cluster: my-cluster resources: - name: usersDb plugin: neon-branch # an installed resource plugin params: parent-branch: main forks: - forkOf: kind: Deployment namespace: default name: users-service customizations: env: - name: DATABASE_URL valueFrom: resource: name: usersDb outputKey: createbranch.connection-string # . ``` See [Sandbox Resources](https://www.signadot.com/docs/reference/sandboxes/resources.md) for the full usage reference, and the [Resource Plugins Overview](https://www.signadot.com/docs/reference/resource-plugins/overview.md) for how plugins are built. ## What Resource Plugins Are Used For The plugin model is deliberately general. If a script can create something and delete it again, a plugin can tie it to a sandbox. The sections below cover the most common patterns. ### Temporary Databases for Full Data Isolation Most sandboxes share the baseline database. When a change needs data isolation, a plugin can create a database that belongs to one sandbox. Common reasons: * **Testing database migrations.** A pull request adds, renames, or drops a column, changes an index, or backfills data. Applied to a shared database, that migration would break the baseline and every other sandbox. Applied to a temporary database, it can run, be inspected, be rolled back and run again, all inside the sandbox. * **Deterministic test data.** Integration and end-to-end tests that expect a specific seed dataset can start from a known state every time. * **Destructive tests.** Tests that truncate tables, bulk-delete records, or exhaust connection pools stay inside the sandbox. There are two common ways to build the plugin: * **A dedicated server per sandbox.** The plugin installs a small database server in the cluster, for example with a Helm chart, and deletes it with the sandbox. The [`mariadb`](https://github.com/signadot/plugins/tree/main/mariadb) example works this way. * **A temporary database on an existing server.** The plugin creates a database or schema on a shared instance such as Amazon RDS, seeds it, and drops it afterwards. This is lighter and faster than a new server. The [`postgres-vault`](https://github.com/signadot/plugins/tree/main/postgres-vault) example works this way, and fetches its admin credentials from HashiCorp Vault. The migration itself can run in one of two places. The plugin can apply it, taking the image or Git ref under test as a parameter. Or the forked service can apply it on startup, as many frameworks do by default. Either way, any service that has to see the new schema must also be in the sandbox and wired to the same resource output. Services left in the baseline keep using the shared database. To get started, follow the [MariaDB resource plugin tutorial](https://www.signadot.com/docs/tutorials/deploying-mariadb-resource-plugin.md). For a comparison of data isolation approaches, see [Using Sandboxes with Databases](https://www.signadot.com/docs/guides/set-up-data-isolation.md). ### Database Branches (Neon, Xata, and Others) Databases with copy-on-write branching give you the isolation of a temporary database without the seeding. A branch starts with all of the parent's schema and data, but writes to it never reach the parent. Branches take seconds to create no matter how large the database is, and use storage only for the blocks that change. A branching plugin is usually small. The `create` step calls the provider's API or CLI to branch from a parent (often `main`, or a sanitized copy of production), names the branch after the sandbox's routing key, and outputs the connection string. The `delete` step deletes the branch. Branching works well for: * **Migrations on realistic data.** Problems such as long table locks, slow backfills, constraint violations on existing rows, and data-dependent bugs appear on a branch of real data. A small seed dataset usually hides them. * **A database for every pull request.** Each preview environment gets its own data, and you don't have to maintain seed scripts. * **AI agents and parallel work.** Many sandboxes can each write freely to their own branch at the same time. Step-by-step guides are available for [Neon](https://www.signadot.com/docs/integrations/databases/neon.md) and [Xata](https://www.signadot.com/docs/integrations/databases/xata.md). The same pattern works for any database that has an API for branching or cloning. ### Temporary Cloud Resources Services often depend on cloud resources that hold state: object storage, NoSQL tables, caches, secrets, and the IAM policies around them. When the code under test changes how it uses one of these, or when tests write to it, a plugin can create a copy that belongs to the sandbox. Examples: * An **S3 or GCS bucket** for testing upload, export, or retention features without writing into the shared bucket * A **DynamoDB table** or a **Redis/ElastiCache** keyspace with its own data * A **secret, IAM role, or service account** scoped to the sandbox's resources only * Kubernetes objects the sandbox needs, such as **ConfigMaps, Secrets, or Jobs**, which are garbage-collected with the sandbox Plugins usually use the tools you already have, such as Terraform, Pulumi, or a cloud CLI. The [`terraform`](https://github.com/signadot/plugins/tree/main/terraform) example creates a temporary S3 bucket and outputs its name for the forked service to use. The [`k8s-apply`](https://github.com/signadot/plugins/tree/main/k8s-apply) example creates Kubernetes objects that are owned by the sandbox, so Kubernetes deletes them automatically. tip Every step runs in its own pod, so nothing on disk survives from `create` to `delete`. Pass small values, such as a bucket name or resource ID, from a `create` output to a `delete` input. Keep larger state, such as Terraform state, in a remote backend. Tag cloud resources with the sandbox's routing key so you can find anything left behind if a deletion fails. ### Message Queue Topics and Queues Asynchronous flows are hard to isolate. When a sandboxed consumer subscribes to a shared topic or queue, it competes with the baseline consumer. It can take messages that belong to the baseline, and the baseline can take the messages the sandbox is waiting for. Signadot supports two approaches: 1. **Message-level routing.** Producers and consumers share the broker. Consumers read the routing key on each message and skip messages that aren't theirs. This is the recommended default. It requires no provisioning and scales with the number of teams, but it does need small changes to consumer code. 2. **Topic- or queue-level isolation with a resource plugin.** A plugin creates a topic, queue, subscription, or vhost for the sandbox and outputs its name. The sandbox's producers and consumers are configured to use it. The `delete` workflow removes it. With topic-level isolation, you can test a whole async flow end to end, including new message schemas, retry and dead-letter behavior, and poison messages, without touching the shared broker's traffic. Consumer code doesn't change. The topic name comes from configuration, which the plugin output overrides. The cost is that the baseline producers keep publishing to the shared topic. Every producer whose messages the flow needs must be co-forked into the sandbox (running the same image, configured with the sandbox's topic), or the sandbox topic stays empty. The [Kafka guide](https://www.signadot.com/docs/guides/set-up-message-queue-isolation/kafka.md#choosing-between-them) compares both approaches in detail. The [`amazon-sqs`](https://github.com/signadot/plugins/tree/main/amazon-sqs) example creates a temporary SQS queue for each sandbox. For broker-specific guidance on Kafka, RabbitMQ, SQS, Google Pub/Sub, and NATS, see [Message Queue Isolation](https://www.signadot.com/docs/guides/set-up-message-queue-isolation.md). ### Infrastructure Outside Kubernetes A plugin's scripts can call any API the runner can reach, so resources are not limited to Kubernetes. A plugin can start a VM, deploy a Lambda function, run an ECS task, or create infrastructure in another cloud account, and have all of it deleted when the sandbox is. Because the sandbox's TTL also triggers deletion, nothing is left running after the sandbox is gone. What a plugin can do today depends on how the workload outside Kubernetes is used: * **As a dependency of the sandbox.** The sandboxed services call the external workload directly at an address the plugin outputs. Examples include a legacy service on a VM that the forked service needs, a stub of a third-party API, or a GPU host for an inference service. This works today in the same way as a database or a bucket. * **As the sandboxed version of a baseline service.** A request carrying the sandbox's routing key should reach the external workload instead of the baseline service. This takes more work. Signadot's request routing currently covers workloads that Signadot runs itself: forks in the cluster and local workloads connected through the CLI. For an external workload, something in front of the baseline service (for example, your API gateway or a proxy) has to read the routing key and send the request to the external workload. You also have to keep track of which routing key maps to which external address yourself. Roadmap We are working on first-class routing for workloads that Signadot doesn't run itself, such as VMs, ECS tasks, Lambda functions, and forks in other clusters. The goal is that a resource plugin can create such a workload and register it as a routing destination for its sandbox. Signadot would then route keyed traffic to it like any other fork, and remove the route automatically when the sandbox is deleted. If this matters for your use case, [let us know](https://github.com/signadot/community/issues/new). ### Custom Lifecycle Logic Some plugins create no infrastructure at all. A plugin is a reliable hook for anything that should happen when a sandbox is created or deleted, for example: * Loading a seed dataset into a shared database under a tenant ID unique to the sandbox, and deleting that tenant afterwards * Creating test users, API keys, or accounts in a third-party sandbox environment (a payment provider's test mode, for example) * Registering the sandbox's preview URL as a webhook or OAuth callback, and removing it afterwards * Issuing short-lived credentials from Vault or a cloud IAM service The [`example-python`](https://github.com/signadot/plugins/tree/main/example-python) plugin is a starting point for running arbitrary Python code. ## Choosing an Isolation Approach Resources add isolation, but they also add work: the sandbox waits for the `create` workflow, and every resource costs something to run. Use the lightest approach that keeps the test correct: | Approach | Isolation | Sandbox start-up | Good for | | --------------------------------------------------- | ----------------------- | ---------------------------------------- | -------------------------------------------------------------------- | | Shared baseline data, partitioned by tenant or user | Logical | No extra time | Most feature work and tests (the default) | | Message-level routing on a shared broker | Logical, per message | No extra time | Async flows where consumers can check routing keys | | Temporary database or schema (plugin) | Physical | Seconds to minutes, depending on seeding | Migrations, seed data, destructive tests | | Database branch (plugin) | Physical, copy-on-write | Seconds | Migrations on realistic data, a database per pull request | | Per-sandbox topic or queue (plugin) | Physical | Seconds | Async flows where consumer code can't change | | Cloud or non-Kubernetes resources (plugin) | Physical | Depends on the provider | Buckets, tables, VMs, and other infrastructure the change depends on | You can combine approaches in one sandbox. For example, a sandbox can use a database branch for the service whose schema changes and keep everything else on shared, partitioned data. ## Writing Your First Plugin A plugin is one YAML file: a runner image, a `create` workflow, and a `delete` workflow. The quickest way to write one is to start from the example closest to what you need. Publish a test version, and try it out with a sandbox that declares only the resource. ### Step 1: Start From an Example | What you need | Start from | | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A database on a server you already run | The example below, or [`postgres-vault`](https://github.com/signadot/plugins/tree/main/postgres-vault) | | A database server inside the cluster | [`mariadb`](https://github.com/signadot/plugins/tree/main/mariadb) and its [tutorial](https://www.signadot.com/docs/tutorials/deploying-mariadb-resource-plugin.md) | | A database branch | The plugin in the [Neon](https://www.signadot.com/docs/integrations/databases/neon.md) or [Xata](https://www.signadot.com/docs/integrations/databases/xata.md) guide | | Cloud resources managed with Terraform | [`terraform`](https://github.com/signadot/plugins/tree/main/terraform) | | Kubernetes objects | [`k8s-apply`](https://github.com/signadot/plugins/tree/main/k8s-apply) | | A queue | [`amazon-sqs`](https://github.com/signadot/plugins/tree/main/amazon-sqs) | | Custom code | [`example-python`](https://github.com/signadot/plugins/tree/main/example-python) | ### Step 2: Write the Spec This complete plugin creates a temporary Postgres database for each sandbox on a Postgres server you already run. It copies the database from a template that you have seeded with test data, and drops it when the sandbox is deleted: ```yaml name: pg-temp-db@0.1.0 spec: description: Temporary Postgres database per sandbox, copied from a template runner: image: postgres:16-alpine # any image with the tools your scripts use namespace: signadot podTemplateOverlay: | spec: containers: - name: main env: - name: ADMIN_URL # admin connection string for the shared server valueFrom: secretKeyRef: name: pg-admin key: url create: - name: createdb inputs: - name: template # a param set by the sandbox valueFromSandbox: true as: env: TEMPLATE_DB script: | #!/bin/sh set -e DB="sbx_${SIGNADOT_SANDBOX_ROUTING_KEY}_${SIGNADOT_RESOURCE_NAME}" psql "$ADMIN_URL" -c "CREATE DATABASE \"$DB\" TEMPLATE \"$TEMPLATE_DB\"" echo -n "$DB" > /tmp/dbname outputs: - name: dbname # sandboxes read this as createdb.dbname valueFromPath: /tmp/dbname delete: - name: dropdb inputs: - name: dbname # handed over from the create step valueFromStep: name: createdb output: dbname as: env: DB script: | #!/bin/sh set -e psql "$ADMIN_URL" -c "DROP DATABASE IF EXISTS \"$DB\" WITH (FORCE)" ``` What each part does: * **`runner`** picks an image that has `psql`. It uses a `podTemplateOverlay` to inject the admin connection string from a Kubernetes Secret, so credentials never appear in the spec. * **`createdb`** reads the `template` param from the sandbox (`valueFromSandbox`). It names the new database after the sandbox's routing key and the resource name, and publishes that name as an output by writing it to a file. Postgres copies template databases quickly, but nothing else can be connected to the template while it is being copied. * **`dropdb`** receives the name from `createdb` through `valueFromStep`, so `delete` needs no state of its own. `IF EXISTS` makes it safe to run even if the database is already gone. ### Step 3: Publish and Test It Create the Secret the runner reads, publish the plugin, and create a sandbox that declares only the resource. That is enough to run the `create` workflow: ```yaml # test-sandbox.yaml name: pg-temp-db-test spec: cluster: "@{cluster}" resources: - name: appdb plugin: pg-temp-db@0.1.0 params: template: app_seed ``` ```bash kubectl -n signadot create secret generic pg-admin \ --from-literal=url='postgres://:@:5432/postgres' signadot resourceplugin apply -f plugin.yaml signadot sandbox apply -f test-sandbox.yaml --set cluster= signadot logs --sandbox pg-temp-db-test --resource appdb ``` Check that the database exists, then run `signadot sandbox delete pg-temp-db-test` and check that the `delete` workflow dropped it. Published versions are immutable, so bump the version (`pg-temp-db@0.1.1`) each time you change the plugin. [Testing and Debugging](https://www.signadot.com/docs/reference/resource-plugins/testing-and-debugging.md) explains how to find out why a step failed. ### Step 4: Use It From a Sandbox Point a forked service at the new database through the plugin's output. The fork already connects to the shared server, so it only needs the new database name: ```yaml customizations: env: - name: DB_NAME valueFrom: resource: name: appdb outputKey: createdb.dbname # . ``` Roadmap We are also working on an [agent skill](https://www.signadot.com/docs/integrations/coding-agents/agent-skills.md) for writing resource plugins. You would describe the resource you need, such as "a Postgres database per sandbox, seeded from our fixtures." Your coding agent would then draft the plugin, publish a test version, run it in a sandbox, and use the step logs to fix any failures. If you'd like to try it early, [let us know](https://github.com/signadot/community/issues/new). ### Design Tips * **Make `create` fast.** Sandboxed workloads don't start until it finishes. Prefer branching, cloning, templates, and pre-built images to long data loads. * **Name resources predictably.** Build names from the routing key and `SIGNADOT_RESOURCE_NAME`. This keeps two resources in one sandbox from colliding, and lets you trace anything left behind to its sandbox. * **Pass state from `create` to `delete` through outputs.** Each step runs in its own pod. Hand IDs and names to `delete` with `valueFromStep`, and keep larger state, such as Terraform state, in a remote backend. * **Make `delete` forgiving.** Failed steps are not retried, so a `delete` that errors leaves the resource behind. It should succeed even if the resource is already gone or was only partly created. * **Keep secrets out of the spec.** Mount credentials from in-cluster Secrets using the [`podTemplateOverlay`](https://www.signadot.com/docs/reference/resource-plugins/spec.md#specrunnerpodtemplateoverlay), and grant only the permissions the plugin needs. * **Set TTLs on sandboxes** that use expensive resources, so they are cleaned up even when nobody remembers to delete them. ## Next Steps * [Resource Plugins Overview](https://www.signadot.com/docs/reference/resource-plugins/overview.md): how plugins, runners, steps, inputs, and outputs fit together * [Example plugins](https://www.signadot.com/docs/reference/resource-plugins/examples.md): MariaDB, Postgres with Vault, Terraform (S3), Amazon SQS, Kubernetes objects, and Python * [Sandbox Resources](https://www.signadot.com/docs/reference/sandboxes/resources.md): requesting resources from a sandbox and using their outputs * [Deploying a MariaDB Resource Plugin](https://www.signadot.com/docs/tutorials/deploying-mariadb-resource-plugin.md): an end-to-end tutorial * [Neon](https://www.signadot.com/docs/integrations/databases/neon.md) and [Xata](https://www.signadot.com/docs/integrations/databases/xata.md): database branching integrations * [Message Queue Isolation](https://www.signadot.com/docs/guides/set-up-message-queue-isolation.md): topic-level and message-level isolation for asynchronous flows