Experiments

The Experiments feature (also known as Champion–Challenger) allows you to safely test a new workflow strategy by comparing a live workflow (the "Champion") with a published workflow (the "Challenger").

• The Champion produces the actual decision returned in the API response.
• The Challenger runs in shadow mode, producing results for comparison only.

This enables you to monitor both workflows side by side, analyze their performance, and decide whether to promote the Challenger to become the new live workflow.

How it works

• Both Champion and Challenger run on 100% of the same live traffic.
• Only the Champion’s decision is returned via API.
• The Challenger’s result is logged but does not affect outcomes, cases, or webhooks.
• Metrics from both workflows are recorded for head-to-head comparison.

Before you begin

• Single Challenger limit: Only one Challenger can run at a time. To test a new one, stop the current experiment first.
• Immutable Champion: You can't edit the live workflow while an experiment is running. To update it, stop the experiment first.
• Synchronous workflows only: Experiments for workflows including asynchronous steps (for example, OTP, DocV, Wait steps) or asynchronous data partners are now unavailable.

1. Start an experiment

1. Navigate to the Experiments section in the dashboard.
2. Click Create New Experiment.
3. Enter the following:
   • Experiment Name
   • Challenger Workflow (choose from your published workflows)
4. Click Run Experiment.

📘

Note:

You can only run one experiment at a time.

2. Monitor status and results

On the Experiments page, under the Active tab, you’ll see:

• Experiment status → Running or ended, with a timer.

• Outcome metrics for Champion vs. Challenger:

• Accept / Reject / Resubmit / Cancel / Review counts

  • API Calls Executed
  • Evaluations Processed

Available actions:

• Declare Winner → Promote the Challenger to Live.
• Stop → End the experiment and move results to the Ended tab.

3. Declare a winner

After stopping an experiment, you can optionally declare the Challenger as the winner.

• If promoted, the Challenger becomes the new Live (Champion) workflow.
• Past experiments are retained in the Ended tab for future reference.

Metrics tracked

Empty state scenarios

• If no traffic is processed during the experiment, an empty state message appears.
• Ended experiments with no data are still listed but without results.