Skip to main content

​
A/B Testing

A/B Testing lets you test a clear hypothesis against control and variant prompt buckets. Use it when you want to know whether a messaging, content, or positioning change affects AI visibility.
Experiment lifecycle from draft to prompt buckets, scheduled run, completed result, and decision modal
A/B Testing is available on Professional and Enterprise plans.

​
When to run an experiment

Run an experiment when all of these are true:
  • You have a clear hypothesis
  • The prompts reflect one intent area
  • Control and variant prompts are comparable
  • You can wait for the run to complete and measure the result
  • You are willing to record a decision after the run

​
Create an experiment

1

Open A/B Testing

Go to Tracking -> A/B Testing and select New Experiment.
2

Name the experiment

Use a name that captures the content or positioning change.
3

Write the hypothesis

Example: “Use-case-focused prompts will improve visibility versus feature-focused prompts.”
4

Select the model

Choose one available provider/model for the experiment.
5

Assign prompts

Add prompts to control and variant buckets. The app enforces minimum prompt requirements before a started run.
6

Save or start

Save a draft while building. Start when the buckets are ready.

​
Experiment status

The experiment exists but has not been scheduled. You can edit details and prompt assignments.

​
Good experiment design

One hypothesis

Test one change at a time. Mixing product copy, pricing, and page structure makes results hard to explain.

Comparable buckets

Control and variant prompts should have the same intent and difficulty.

Enough prompts

Use enough prompts to reduce noise, but keep the experiment focused.

Decision discipline

Record a decision after completion so the experiment history remains useful.
Experiments measure AI response behavior for the configured prompts and model. They do not prove that every AI assistant will change in the same way.