Why counts differ
Your A vs B numbers and the numbers in your analytics tool will not match exactly. That is expected, and it is true of any two tools measuring the same website.
A gap of roughly 5 to 10 percent is normal and is not worth investigating. What is worth investigating is a gap that is much larger than that, one that keeps growing, or one that appears in only one variation.
Why a gap exists at all
We count people, most analytics tools count events
This is usually the biggest single reason, and it is not an error on either side.
Your A vs B results count unique visitors per variation: how many people saw the variation, and how many of them converted. Most analytics tools count events: how many times something happened. One person who converts three times is one converting visitor to us, and three events to them.
Before comparing any two numbers, check that both are answering the same question.
Ad blockers and privacy extensions
Blockers work from lists of names and domains. A list that blocks your analytics tool may not block us, and the other way round. Whichever tool is blocked more often reports fewer events, for the same real traffic.
Consent choices
If a visitor declines analytics, we record nothing for them: no experiment view, no conversion, and nothing forwarded to any tool. Your analytics tool has its own consent setup, which may be stricter, looser, or simply applied at a different moment in the page.
Browser tracking protection
Browsers limit how long storage survives, and they do it differently per tool. The same person can be a returning visitor in one tool and a brand new one in another, which moves visitor counts without changing a single real visit.
The page closing mid send
Some events happen right as someone leaves. We flush what we have when the page is hidden, using the browser's background send, but a send that starts as the tab closes can still be lost. Every tool on your page has this problem, and each one loses a slightly different set of events.
Single page apps and reloads
Moving between screens on a single page app, or reloading a page, re-runs the experiment checks, because the new screen may need a different variation. We still send an experiment view only once per browser session for each variation a visitor sees, so these re-runs add no views. A visitor who switches variation, for example after a forced variation during QA, produces one view per variation.
Deliveries that arrive more than once
We would rather send an event twice than lose it, so a delivery that looks like it failed can be sent again. Each event keeps the same ID across those attempts, so the same real event can still be counted once. See the dedupe advice below.
Bot, preview, and internal traffic
We keep known bots, your preview sessions, and traffic you have tagged as internal out of your results, and we never forward preview or internal traffic to your analytics tools. Your analytics tool has its own bot filtering, and its rules are not our rules.
Which events line up one to one
Every event we forward carries a unique event ID. What that ID lets you do depends on the event.
| Event | Lines up one to one | What the ID gives you |
|---|---|---|
| Experiment views (from the snippet) | Yes | The forwarded event carries the same ID as the matching row in your A vs B data |
| Experiment views (from the developer SDK) | Yes | Same again: one ID, used for our own record and for every tool it is sent to |
| Conversions | No | Sent once per live experiment, so one action can produce several events, each with its own ID |
| Purchases | No | Same as conversions: one order, one forwarded event per live experiment |
| Recommendation views and clicks | Yes | Recorded once against the experiment they were shown in, and forwarded under that same ID |
| Test events | Never in reports | Clearly labelled, and never counted in your A vs B results |
Experiment views reconcile exactly
An experiment view is recorded once and forwarded with the same ID it was recorded under. Send it to two tools and both rows carry that same ID, so a row in one tool, a row in the other, and the row in your A vs B data are all provably the same event.
Recommendation views and clicks work the same way. They belong to the one experiment they were shown inside, so they are not copied per experiment. A recommendation shown outside any experiment is not forwarded to your analytics tools at all, because an event with no experiment on it would be worse than no event.
Conversions and purchases legitimately multiply
If a visitor is in three live experiments at once and they buy something, your analytics tool receives three purchase events, one per experiment, each with its own ID.
That is deliberate. It is what lets the same order be measured against each experiment separately. It is also the number one cause of a panicked "our revenue tripled" message.
Forwarded purchase and conversion events are per experiment copies of one real action. Summing them across experiments multiplies your revenue by the number of live experiments. To count orders, use the experiment and variation fields to look at one experiment at a time, or use your own ecommerce tracking.
Test events are never in your results
The test event from the Send a test event link goes through the real path so you can see the wiring working, and it is labelled as a test in every payload. It never creates an experiment view and never affects your A vs B results.
Practical advice
Count distinct event IDs, not rows. In any tool where you can, count the unique event ID rather than the raw number of events. That removes the duplicates a retried delivery can create, and it is the closest you can get to "how many real events happened".
Compare like with like. Match visitors to visitors and events to events. If your tool can only give you events, compare its per variation split against our per variation split rather than the absolute totals.
Compare shapes, not totals. If A vs B says variation B converts 12 percent better and your analytics tool says 11 percent better, the two systems agree. Chasing the last few percent of absolute counts costs time and changes no decision.
Trust your results page for the decision. The results page counts visitors, filters out bot, preview, and internal traffic, and runs the statistics. Your analytics tool is the right place to explore the experiment alongside everything else you measure. It is not the place to call the winner.
GA4 has one extra trap that makes conversion reports look almost empty, and it is not a counting difference. See analyzing in GA4 before you build a report there.
When a gap is worth investigating
Something is genuinely wrong if:
- One tool shows experiment views and the other shows none at all.
- The gap is far larger than 10 percent and does not settle.
- The split between variations is very different in the two tools. A vs B splits traffic evenly by design, so a heavy imbalance in one tool and not the other points at a measurement problem rather than an experiment problem.
- Conversions arrive but experiment views do not, or the other way round.
Start with Send a test event on the Integrations tab, watch for it in the tool's live or debug view, and check the tool's library is loaded on the pages your experiment runs on.