Managing Running Experiments

Once an experiment is live, you may need to pause it, change something about it, or end it once you have enough data to decide. This page covers every management action for a running experiment.

Pausing an experiment

Pausing stops an experiment from running without discarding it or its data. While paused:

• The experiment stops evaluating new visitors. Nobody new gets sorted into a variation (one specific version being tested, like Control or Variant A).
• Visitors who were already assigned to a variation stop seeing it. They see the original page instead, because the snippet (A vs B's tracking script) skips a paused experiment.
• Everything collected so far stays on the results page.

To pause: open the experiment and click Pause. On the results page you'll also see Schedule end (only if you haven't already scheduled one) and Edit Variation next to it. In the builder header, Pause appears alone.

To resume: a paused experiment shows Resume and Stop buttons in the same place. Click Resume and the experiment goes back to Running and starts sorting visitors again. Resuming keeps the original launch: the Launched time and the person who launched it stay the same, and results still count every visitor from the first launch on. Every one of these controls needs the Publish to production permission; members without it don't see them.

You don't have to open the experiment to do it. A paused row also offers Resume in its actions menu on the experiments list and on the dashboard, and it behaves exactly as the button inside the experiment does, including asking about unpublished changes first.

Turning a variation off

Separately from pausing the whole experiment, you can turn off a single variation while everything else keeps running. This shows up as a "Turn a variation off" card on the results page, but only while the experiment is Running or Paused.

Turning a variation off stops serving it right away: visitors already on it see the control instead, usually within a minute. The control itself can never be turned off, so there's always something to show.

• Turning a variation off needs confirmation. Click Turn off, then click Confirm turn off (or Cancel to back out). This is a two-step action on purpose, since it's the one that changes what live visitors see.
• Turning a variation back on is one click. Click Turn on and it's live again immediately, no confirmation needed.

This action needs the Publish to production permission, the same as pausing or stopping.

Stopping an experiment

Stopping ends the experiment for good. Once stopped, it can't be restarted the way it was running before. When stopped:

• All variation code stops running immediately: everyone sees the original page.
• The experiment's status becomes Completed, which is what every screen calls it (see Experiment Statuses).
• Everything collected stays saved and viewable on the results page.

Stop is on the builder, the results page, and the row menu of a running or paused experiment, both on the experiments list and in the dashboard's recent experiments. It always asks you to confirm first, because a stopped experiment can't run again.

The Frequentist engine reports results as a p-value against a fixed threshold. If your experiment uses it, Stop is checked before it runs. Stopping before your planned sample size is reached asks you to confirm and log a reason, because stopping early skews the statistics. A vs B logs an EARLY_STOPPED entry to the audit trail when this happens. Archiving is not checked this way, because archiving doesn't end an experiment.

Concluding a feature-flag A/B test rule early stamps the same reason and timestamp fields on the rule. It doesn't write a matching audit-log entry today, though, so this entry is experiment-only for now.

Stop an experiment once you've reached a clear conclusion. Either a variation has won with statistical significance (a result unlikely enough to be chance), or you've decided the experiment isn't worth continuing.

Editing a live experiment

You can edit some parts of a running experiment without stopping it. Editing live is riskier than editing a draft, though, because changes apply immediately to every visitor and can throw off your results.

Safe edits

These have little effect on your results:

• Renaming the experiment or a variation.
• Adding secondary metrics (not the primary one).
• Narrowing your targeting to fewer pages.

Risky edits: use with caution

These can invalidate your results if you make them mid-run:

• Changing variation CSS or JS. Visitors already bucketed into a variation start seeing the new code, so your data ends up mixing two different versions of the same variant.
• Changing traffic splits. Changing how much traffic each variation gets can cause a Sample Ratio Mismatch. That's a red flag that fires when the real split doesn't match what you configured, and it invalidates the analysis.
• Changing the primary metric. If you switch the metric an experiment is judged on mid-run, your historical data was collected against a different goal.

Metric changes are different. Attaching, detaching, or editing a metric on the Metrics step saves immediately and isn't covered by either Publish Changes or Discard. Those changes go live right away, even while other edits on the experiment are still pending.

If a republish runs into experiment code that no longer compiles, visitors keep seeing the last successfully published version. Whoever launched or last edited the experiment gets a notification so they can fix it.

Pausing, stopping, resuming, and Publish Changes can also come back with a notice saying the change was saved but publishing did not finish. The status change did happen, so the experiment really is paused (or stopped, or running). What has not happened yet is the update reaching visitors, and A vs B keeps retrying that on its own. Until it succeeds, visitors see the last version that published cleanly.

Unsaved edits when you pause, stop, resume, or schedule

Pause, Stop, Resume, and Schedule all reload the experiment from its saved version once they run, so anything you typed and have not saved yet would go with it. Click one of them with unsaved edits on screen and A vs B asks first, with three choices:

• Save edits first. Your edits are saved, and then the action runs. On a running or paused experiment they are saved as pending changes, so they stay unpublished until you publish them.
• Continue and lose them. The action runs straight away and the unsaved edits are dropped. This cannot be undone.
• Cancel. Nothing happens and your edits stay on screen.

On a paused experiment that also has unpublished changes, Resume asks this question first, then asks what should happen to the unpublished changes.

Pending changes while paused

Pausing never publishes your unpublished edits, and it never throws them away. The banner stays visible while the experiment is paused, and both of its buttons keep working:

• Publish Changes while paused saves your edits as the new live version and records the publish in the history. Visitors don't see anything change yet, because a paused experiment is skipped entirely. The published version goes live the moment the experiment resumes.
• Discard while paused puts the draft back to the last published version, the same as it does while running.

If you click Resume while edits are still unpublished, A vs B asks what should happen to them before anything goes live. You pick one:

• Publish changes and resume. Your unpublished edits become the live version, and visitors see them as soon as the test is running again.
• Discard changes and resume. Your unpublished edits are thrown away, and the test resumes exactly as it was running before you paused it. This cannot be undone.
• Cancel. Nothing changes and the test stays paused.

While delivery to your site is paused, a test cannot resume. So neither choice runs: a dialog explains the pause, and your unpublished edits stay as they are. Publishing changes to a running test is refused the same way, and the dialog says that nothing was published. See Delivery to your site is paused.

Publish history & restore

Every successful publish on a web experiment writes an entry to its history: the first launch and every later "Publish changes" from the builder. Pausing and stopping don't create entries, because they don't change the experiment's configuration, only its status. Nothing in the history ever gets trimmed.

To open it, click the History tab in the experiment builder's left navigation. It lists every publish, newest first, with who did it and a relative timestamp. Each entry also gets a one-line summary of what changed: variation code, traffic split, targeting, metrics, or audiences (a named, reusable group of visitors defined by targeting rules).

• Use the search box to filter by summary text or author name. The list updates as you type.
• Click Load more at the bottom to see older entries.
• Click any entry to see its details on the right: author, timestamp, and a field-by-field difference against the experiment's current published state. Only the parts that changed are highlighted.

Restoring a previous version

Inside the detail panel, click Restore this version to copy that snapshot into the experiment's current draft. Restoring never publishes automatically. It sets the "pending changes" flag and takes you to the Variations step, so you can review and republish from the bottom bar. On a running experiment, live visitors keep seeing the current version until you publish the restored draft.

If your current draft already has unsaved changes, restoring asks you to confirm before overwriting them.

Restore is unavailable in two cases:

• The experiment is Completed. A stopped experiment can't be republished, so restoring would lead nowhere. The button is disabled with a tooltip explaining why.
• The snapshot references a metric or audience that's since been removed from the project. The detail panel lists exactly which ones are missing, so you know what to recreate, or you can pick a different entry to restore instead.

Variations are the one exception. If the snapshot references a variation you've since deleted, restoring recreates it using its original ID. Any results already tied to that variation stay linked to it.

What restore changes: variation names and code, traffic split, targeting rules, metric selection, audience configuration, scheduling, and triggers. What it leaves alone: the experiment's name, description, and the results you've already collected.

Permissions

Anyone in the organization can browse the history list and open any entry. The Restore this version button is hidden for anyone without the Edit code variations permission, so viewers and collaborators see history as read-only. Publishing a restored draft still needs the publish permission. That means someone who can edit but not publish can stage a restore for a teammate to ship.

Archiving experiments

Archive an experiment to file it out of your main list once you're done looking at it. It leaves the main list and the dashboard's recent experiments, and appears under the Archived filter tab.

Archiving changes nothing about the experiment itself. It keeps its real status, so a completed test still says Completed and a paused one still says Paused, shown in grey with an Archived tag beside it. Its results, dates and history are all kept.

Restore puts it back exactly where it was: a completed test comes back Completed with its results, a paused one comes back Paused with Start waiting for you. Nothing ever comes back Running on its own.

A Running or Scheduled experiment can't be archived as it is, because archiving must never be the thing that changes what your visitors see. Choosing Archive on one tells you so and offers Pause, Stop, or cancelling the schedule in the same place. Then archive it.

Archive and Restore are both in the three-dot row menu on the experiments list, and on the experiment page itself. Opening an archived experiment's own setup pages says the same thing: the builder header shows the Archived tag beside its status, Publish, Schedule and Resume are switched off with the reason beside them, and Restore is right there in the header. See Experiment Statuses for the full picture.

Running a finished test again

A finished experiment can't go back to Running. Duplicate is the way to run the same test again: it makes a fresh draft with the same variations and their code, the same audiences, metrics, settings and statistics engine, names it after the original with "(Copy)" on the end, and opens it ready to edit. The original is untouched, and the copy starts with no results, no history and no schedule of its own.

Duplicate works on an archived experiment too, which is the usual case: file the finished test away, and duplicate it when you want to run it again.

One thing worth knowing: the copy is made from what's saved right now, not from what's live. If the experiment you're copying holds changes you saved but never published, the copy carries those saved changes, since they're the current version of it.