Recommendation experiments release (June 2026)

In June 2026, A vs B added a way to A/B test recommendation sources directly. A new Recommendation test experiment type served a different source per variation (one specific version being tested, control or one of the challengers). Each variation showed its source at the same spot on your page. The source could be an engine recipe, an uploaded feed, or a holdout (a group that saw no recommendations at all). A holdout let you measure whether recommendations helped in the first place. The whole comparison was scored on revenue per visitor.

E-commerce projects using the recommendation engine or feed datasets. If you ever wondered whether viewed-together beats bestsellers, or whether recommendations help at all, this experiment type answered that question.

A vs B no longer has a separate Recommendation test type. The Create Experiment flow now offers only Visual, Custom code, and Split URL. The rest of this page describes what shipped in June 2026, for the record. Jump to What replaced it for how recommendation testing works today.

A new experiment type

The New Experiment drawer gained Recommendation test. Instead of writing variation code, you gave each variation a source. You could use up to 4 variations, including control. Then you set up one shared placement: a CSS selector, an insert position, and a layout (carousel, grid, or headless). You also chose a seed-product source (a URL pattern, a data attribute, or a JS hook) and a maximum item count. A vs B attached revenue per visitor automatically, as the primary metric (the one metric the experiment was actually judged on). Four launch-readiness checks caught sources that could not serve, before launch.

Read more about the API this experiment type used: the Recommendations API.

The drop-in ribbon, with serve-time fallback resolution

The snippet rendered the assigned source as a fixed-height skeleton first, so the page never jumped. Then it showed a carousel or grid of product cards. Recipe fallback chains, set up since the engine release, were walked at serve time. When the primary source had no row for the seed product, the chain served the first step that could. A Headless layout handed the resolved items to your own rendering code through avsb.recs.get().

Coverage and position-bias analysis, as they shipped

The results page added two panels for this experiment type:

A coverage panel, per variation: impressions, served impressions, coverage-full percentage, and click-through rate. It warned you when coverage differed by more than 10 percentage points between variations. It also had a toggle. The toggle recomputed every metric using only visits with full coverage.
A click-position panel: clicks by rendered position per variation, plus a breakdown of which fallback step actually served.

Winner promotion, as it shipped

On a paused or completed Recommendation test, a Promote winner button made the winning source the production default. It activated a pinned feed version as the live version, or it re-enabled a disabled recipe. Promotions were audit-logged, and the experiment carried a "Winner promoted" badge.

What replaced it

On 12 July 2026, A vs B retired the dedicated experiment type and both results panels above. One simpler mechanism replaced them. Any experiment, of any type, is now measured the same way. It starts as soon as its variation code calls the recs API. There is no separate experiment type to choose.

What is different today:

No separate experiment type to pick. Build an ordinary Visual, Custom code, or Split URL experiment. Call avsb.recs.get() or avsb.recs.trackClick() from your variation code. A vs B detects the recommendation events on its own.
One results table, not two panels. The results page shows a Recommendation performance table: variation, recipe, impressions, served impressions, clicks, and click-through rate. It also shows visitors who added a product to cart after clicking. There is no coverage-percentage column, no coverage-divergence warning, and no full-coverage toggle. There is also no click-position or fallback breakdown. The snippet still fires rec:impression and rec:click events with a documented attribute vocabulary. Those events include a click's rendered position and which fallback step served. No dashboard page shows that data today.
No promotion flow. The Promote winner button, and the flow behind it, no longer exist. Today, you turn a feed version or a recipe back on by hand.

Released June 2026. The experiment type described above was retired 12 July 2026.