Skip to main content

Specification

Overview

info

This feature is in beta. Please leave feedback via our community GitHub or Slack.

  • This feature requires Signadot Operator v0.19+.
  • You must enable Managed Runner Group on your cluster via the Signadot Dashboard (under Settings) and have at least 1 runner pod.

Signadot tests are API integration test specifications designed to help you maintain robust and reliable tests. These tests are automatically executed on both sandbox and baseline environments to ensure stability and prevent flakiness. They currently support contract testing via traffic capture & diff as described below.

Test Specification

Tests are specified in Starlark and run using the starlet interpreter with additional custom modules. Starlark is a language similar to Python3, but smaller in scope and more deterministic.

Starlet Libraries

The standard starlet libraries are available in the Test interpreter. Detailed documentation of the libraries can be found here.

Support for Checks

The smart_test module provides support for Checks which are included in Test results.

Traffic Capture and Relevancy Diffs

The traffic capture docs explains how one can easily turn on traffic capture for specific requests using an extended http module. Doing so will automatically enable getting relevancy diffs in the Test results.

Triggers

Triggers define which sandboxes will trigger tests and how the tests are executed. A single test can have many triggers.

Triggers are specified via the Signadot Dashboard using the "Triggers" Tab after you create a Smart Test. Triggers currently allow matching sandboxes based on the baseline workload being sandboxed. This means that if you designate a specific cluster and workload as a trigger, the test will automatically run on any sandbox that includes that workload whenever it is created or updated.


In the above example, whenever you create or update a sandbox that contains the workload location in the namespace hotrod-istio, the test will be automatically triggered on that sandbox.

Timeouts

Tests have a timeout of 5 minutes.

Known Limitations

Currently, Smart Tests are limited to handling RESTful APIs with JSON payloads. This means that other types of traffic, such as gRPC, SOAP, or non-JSON data formats, are not supported at this time. Future updates may expand the scope of supported protocols and data formats.