Metrics

Step 3 of the experiment builder is Metrics & Goals. This is where you pick which goals you're measuring for this experiment. The metrics you choose decide how A vs B works out which variation is winning.

What metrics do

A metric is a tracked event that represents a goal. It could be a visitor clicking a button, visiting a page, completing a purchase, or firing a custom event from your code. Each time a visitor triggers it, that counts as a conversion for their variation (one specific version being tested, like Control or Variant A).

A vs B compares the conversion rate of each variation and runs statistical analysis to work out whether any difference is real or just random noise.

Selecting metrics

The Metrics step lists every metric already created for this project. Click Select on a metric, choose how to measure it (for example "unique conversions per visitor"), then click Attach metric. It now appears in an ordered list above the picker.

You can attach as many metrics as you want, but every experiment needs at least one.

Roles: primary, secondary, and guardrail

Every attached metric plays one of three roles, set from a dropdown under its row:

• Primary. This is the metric an experiment is actually judged on, and there's always exactly one. The first metric you attach becomes primary automatically. When A vs B shows you a result like "Variant 1 is winning with 95% confidence," it's talking about the primary metric.
• Secondary. Tracked for context and read alongside the primary. It never decides the result on its own. Every metric after the first defaults to secondary.
• Guardrail. A metric you're not trying to improve but don't want to make worse, like page load time or refund rate. A breach blocks a Ship verdict, even if your primary metric is winning. You can set a safety margin per guardrail (how much worse than control is still acceptable), or leave it blank to use your project's default.

Changing the primary metric

You can't demote the current primary directly. Instead, promote a different metric to Primary from its role dropdown, which automatically demotes the old one to Secondary. Dragging a metric by its handle (the six-dot icon) straight to the top slot does the same thing.

Dragging metrics around anywhere else in the list is just for looking, though. Only a move into the very top slot is saved. A vs B doesn't yet store a custom order for your secondary and guardrail metrics. It shows them sorted by role, then by when you attached them.

Creating a new metric

If the metric you need doesn't exist yet, click Create a new metric at the bottom of the picker. This takes you to the project Metrics page, where you can define a metric by type:

• Click: fires when a visitor clicks an element matching a CSS selector.
• Pageview: fires when a visitor views a specific URL.
• Custom: fires when your code calls avsb.track.event('your_metric_key'). Turn on "has a value" and a custom metric can also carry a number with the event, for example a revenue amount: avsb.track.event('purchase_completed', { revenue: 49.99 }).

There's no separate "Revenue" metric type. Revenue tracking is just a Custom metric with a value attached, measured as a sum or an average once you attach it to an experiment.

For more detail on each metric type, see the Metrics section of the documentation.

Removing a metric

Click the X next to any attached metric to detach it from this experiment. This doesn't delete the metric itself, just its attachment here, and you can always add it again from the picker below.

Saving and continuing

Once you've attached at least one metric, click Next to move on to Step 4 (Analysis), where you choose or confirm the stats engine. Review & Publish, where scheduling lives, is Step 5. Click Back at any point to return to the Variations step.