Experiment Error Log

The Experiment Error Log surfaces JavaScript errors and warnings that the A vs B snippet captures while your experiment runs. It breaks them down per variation (one version being tested: the control, or a challenger). It appears on the Results page once at least one error or warning has been captured, and stays hidden on clean experiments, so it never adds noise to a healthy run. Every count in the panel is distinct visitors: the header's "N visitors with errors" and a row's "N visitors affected" both mean how many different people hit at least one problem, not how many times problems fired.

On the card itself, the severity tabs filter the list to All, Errors, or Warnings, the variation dropdown narrows it to one variation, and CSV and JSON download the filtered list as a file. Each row shows the severity, error type, affected variation, message, how many times it happened, and when it was last seen.

What the error log shows

The log tracks two severity levels:

Error: a JavaScript exception or a failed variation apply that likely broke the variation for the affected visitor.
Warning: a non-fatal issue, such as a CSS selector that could not be matched. The variation may still have run, but not as intended.

Within each severity level there are four error types:

JS Exception: an unhandled JavaScript error thrown inside variation code or project JavaScript.
Apply Failed: the snippet attempted to apply a variation (DOM modification or JS execution) and the operation threw before completing.
Selector Miss: a CSS selector targeted by a visual variation could not be found in the page DOM at application time.
Integration: an event could not be forwarded to a connected analytics tool. Common reasons are that the tool was never loaded on the page, its own code threw, or it did not appear before the snippet stopped waiting for it.

What the header counts

The panel header's "N visitors with errors · M with warnings" counts the people behind the rows in the list below it: the same time window and variation filter, across every version of your code. So the header always agrees with the rows you can see. The severity tabs only filter the list; the header keeps showing both counts.

How the error rate is calculated

Under the header, each variation shows its error rate on its current code, since that code went live. This is the number alerting and auto-pause watch, so an older problem that a later deploy fixed doesn't count against it. The rate is computed as:

Plain text

errorVisitors ÷ exposedVisitors × 100

Plain text1 line

errorVisitors is the count of distinct visitors who had at least one error captured on the current shipped code. This is keyed to the code hash published at the last deploy. exposedVisitors is the count of distinct visitors exposed to that variation (meaning: shown it) since that same deploy.

When a variation has fewer than 50 exposed visitors on the current code, the rate denominator is too small to be meaningful. In that case the UI shows the raw affected visitor count instead of a percentage, and displays a dash (—) where the rate would appear. This avoids misleading 100% error rates on very new deploys.

Reading and expanding an error group

Errors are grouped by fingerprint: a client-computed hash of the error type, message, and stack frame locations. Each group row shows:

Severity badge (Error or Warning).
Error type (JS Exception, Apply Failed, Selector Miss, or Integration).
Which variation triggered it.
Truncated error message.
Total occurrence count and distinct affected visitor count.
When it was last seen, in your own time zone, which is named next to the time (charts and CSV exports use UTC).

Click a group row to expand it. The expanded panel shows:

Sample stack trace: a representative stack captured with the first (or most recent) occurrence. Long stacks can be expanded with "Show more."
By browser: occurrence counts broken down by browser user-agent family.
By page: occurrence counts broken down by the page path where the error happened (query strings are stripped; see Privacy below).
First seen and last seen timestamps.

Filtering by severity

The toolbar at the top of the log provides three tabs:

All: shows every captured group regardless of severity.
Errors: limits the list to error-severity groups only.
Warnings: limits the list to warning-severity groups only.

You can also filter by a specific variation using the dropdown next to the tabs. The view refetches automatically a short time after you change either filter.

The log section starts expanded when there are any error-severity groups, and collapsed when there are only warnings. This means critical errors are always visible above the fold on the Results page without requiring a click.

Downloading the error log

The toolbar includes CSV and JSON download buttons. Both respect the current severity and variation filters, so you can export exactly the subset you are looking at. The download is a direct file from the server: your browser will prompt you to save it.

When a variation is shipping with errors on its current code, a banner appears at the very top of the Results page. You see it the moment you open the experiment. It names the affected variation(s) and their error rate (or affected-visitor count under the small-sample floor), with two actions:

View error log: jumps straight to the log section lower on the page.
Dismiss: acknowledges the banner for the current code. It stays dismissed until a new, distinct problem appears on that same code. Then it re-appears, so a fresh issue is never silently hidden. Re-publishing the variation always clears it, since a code change starts a fresh measurement window.

The banner is driven by errors only: warnings notify (below) but never raise the banner. Experiments with active errors also show a red Errors badge next to their status on the experiment list.

Alerts & notifications

A background check runs on all running experiments whenever new errors come in, and within fifteen minutes at the latest. The first time a variation starts erroring on its current code, A vs B sends an alert. After that, you get at most one reminder per hour while the problem carries on. Alerts fan out through two channels:

In-app bell + email to the person who launched the experiment (falling back to whoever scheduled it, then an org admin), so the alert always has an owner.
Your org's integrations: Slack, Microsoft Teams, Jira, and outgoing webhooks (automatic messages A vs B sends to your own server) all receive the event if configured. Code-error alerts are a safety event, the same class as SRM (a red flag for a broken traffic split) and a guardrail (a metric you do not want to get worse) breach. They bypass your org's notification keyword gate, so a real breakage is never filtered out by a keyword rule.

Errors always alert. Warnings (selector misses) only escalate to a notification once they affect a meaningful share of traffic: a rate of 5% or more, or 25 or more affected visitors. Below that they are still recorded in the log, just not pushed.

Auto-pause on errors

Auto-pause on errors is on by default. A vs B automatically pauses an experiment when any variation's error rate on the current code crosses your threshold (default 25%). A broken test takes itself offline before it costs you conversions. The alert tells you it was auto-paused. You can change the threshold, or turn auto-pause off, in an experiment's Analysis settings.

Unlike the statistical-analysis settings, which lock once an experiment launches, the auto-pause toggle and threshold stay editable for running experiments. Their whole job is to protect a live test. A threshold only trips on a measured rate, so a variation under the small-sample floor (fewer than 50 exposed visitors) is never auto-paused on thin data.

Turn a single variation off

Sometimes only one variation misbehaves and you want the rest of the test to keep running. On a running or paused experiment, open the Results page and find the Turn a variation off panel next to the error log. Every variation has a switch: one click and a quick confirmation turns it off, and one click turns it back on. The same switch is built into the visual editor, at the bottom of the sidebar. That way, you can turn a variation off without leaving the page you are editing.

The Turn a variation off panel on the Results page, listing each variation with a live or off status badge and a Turn off button next to each one. — Turning a variation off stops serving it right away. Visitors already on it see the control instead, usually within a minute.

Turning a variation off stops serving it right away. Anyone already shown that variation sees the control instead, usually within a minute. Your other variations keep running and collecting data, so one bad arm does not cost you the whole test. Once you fix the problem, the same switch turns the variation back on.

The control cannot be turned off. It is what a turned-off variation falls back to, so there is always a safe experience to show.

Turning a variation off takes effect on the running experiment right away. It skips the usual publish step entirely, which is why it reaches visitors within about a minute instead of waiting for a redeploy.

Errors in the preview panel

Errors captured while a visitor (including you) uses a preview link also flow into the error log. This makes the log useful for QA before launch: open a preview link, interact with your variation, then check the Results page to see whether any JS exceptions were triggered.

What we capture and privacy

The snippet captures the error message, stack trace, error type, and the page path (with query strings stripped before transmission). It never captures DOM content, form values, cookies, localStorage, or any personally identifiable information. Stack frames reference line and column numbers in your bundled JavaScript, not user data.

The page path recorded in the "By page" breakdown is the location.pathname at the moment the error was captured, with the query string (?…) removed. If your app encodes user identifiers in path segments (e.g. /users/12345), those will appear in the breakdown. Use path-based identifiers with care, or configure your server-side routing to keep sensitive identifiers out of URL paths.