Specification
Overview
This feature is in beta. Please leave feedback via our community GitHub or Slack.
- This feature requires Signadot Operator v0.19+.
- You must enable Managed Runner Group on your cluster via the Signadot Dashboard (under Settings) and have at least 1 runner pod.
Signadot tests are API integration test specifications designed to help you maintain robust and reliable tests. These tests are automatically executed on both sandbox and baseline environments to ensure stability and prevent flakiness. They currently support contract testing via traffic capture & diff as described below.
Test Specification
Tests are specified in Starlark and run using the starlet interpreter with additional custom modules. Starlark is a language similar to Python3, but smaller in scope and more deterministic.
Starlet Libraries
The standard starlet libraries are available in the Test interpreter. Detailed documentation of the libraries can be found here.
Support for Checks
The smart_test
module provides support for Checks
which are included in Test results.
Traffic Capture and Relevancy Diffs
The traffic capture docs
explains how one can easily turn on traffic capture for specific requests using
an extended http
module. Doing so will
automatically enable getting relevancy diffs in the Test results.
Triggers
Triggers define which sandboxes will trigger tests and how the tests are executed. A single test can have many triggers.
Triggers are specified via the Signadot Dashboard using the "Triggers" Tab after you create a Smart Test. Triggers currently allow matching sandboxes based on the baseline workload being sandboxed. This means that if you designate a specific cluster and workload as a trigger, the test will automatically run on any sandbox that includes that workload whenever it is created or updated.

In the above example, whenever you create or update a sandbox that contains
the workload location
in the namespace hotrod-istio
, the test will be
automatically triggered on that sandbox.
Timeouts
Tests have a timeout of 5 minutes.
Known Limitations
Currently, Smart Tests are limited to handling RESTful APIs with JSON payloads. This means that other types of traffic, such as gRPC, SOAP, or non-JSON data formats, are not supported at this time. Future updates may expand the scope of supported protocols and data formats.