Find every regression in your pull requests
Deep validation on every code change
Catch what assertion-based tests miss
“We basically stopped creating full preview environments and replaced our custom solution with Signadot. The strategy using routing keys is much lighter, and we are able to provide an isolated environment, even with isolated databases, per PR quite fast.”
Behavioral validation on every PR, without the test debt.
Bitso gives 250+ engineers an isolated environment per PR on Signadot, the same environments Smart Tests validate against on every change.
A risk score you can read at a glance
Give every PR a risk score before review
Most teams have their first Smart Tests running in under an hour, on the cluster and staging environment they already operate.
Frequently asked questions
Is this a contract testing tool like Pact?
No. Smart Tests is not a direct alternative to consumer-driven contract testing tools like Pact. Smart Tests is a provider-focused validation tool that gives you a "risk score" for your PR by analyzing its runtime behavior, rather than coordinating contracts between consumer/provider teams.
How does the AI system in Smart Tests work?
The AI system performs two main functions: First, it analyzes API responses from both versions to identify structural, data type, and content differences. The models determine which differences actually impact compatibility rather than flagging every change.
Second, it reduces test flakiness by learning from historical test runs, distinguishing between expected variations (timestamps, random IDs) and genuine breaking changes. The system adapts to your specific APIs over time, improving its accuracy with continued use.
What kinds of contract breakages can Smart Tests detect?
Smart Tests detects:
- Schema changes: Removed fields, changed data types, altered object structures
- Behavioral changes: Different response codes, new error conditions, altered validation rules
Because Smart Tests observes runtime behavior rather than just validating static schemas, it can identify subtle issues that traditional approaches miss.
How can Smart Tests be integrated with my CI/CD pipeline?
Integration requires two components:
1) The Signadot CLI integrated into your pipeline to create sandboxes for each PR
2) Smart Test triggers configured to automatically run tests against those sandboxes
How long does it take to set up Smart Tests?
Most teams have their first Smart Tests running in under an hour:
1) Install the Signadot operator on your Kubernetes cluster (10-15 minutes)
2) Write your first test in Starlark to call your API endpoints (5-10 minutes)
3) Configure triggers to run the test on your sandboxes (5 minutes)
4) Integrate sandbox creation with your CI/CD pipeline (15-30 minutes)
Since Smart Tests runs against your existing staging/pre-production environment, you avoid the typically time-consuming process of setting up separate test environments or managing test data.
What is the maintenance overhead of Smart Tests when APIs change?
Smart Tests minimizes maintenance when APIs evolve:
- When you make compatible API changes (adding fields, extending functionality in backward-compatible ways), Smart Tests automatically recognizes these as non-breaking and doesn't require test updates.
- For intentional breaking changes, you typically only need to update the test code that calls the changed endpoints - not assertions or expectations, since those are handled by the AI comparison system.
