A/B testing a deal

Applies to: TreStack Bundles, every plan
Updated: 11 October 2026

Setting up a test

There are two ways to start. Setup A/B test, in a deal's menu in the deal list or above the preview in the editor, gives you a second variant to change by hand. Smart A/B test builds the second variant around one change you pick from suggestions and starts the test at once; see Smart A/B tests.

With Setup A/B test, variant A is your deal and variant B starts as a copy of it. Where A still has the default texts and colors, B gets the title "BUY MORE - SAVE MORE", the bar titles "Buy 1", "Buy 2" and "Buy 3", and a blue background and border, so the two look different from the start. "Add variant" adds C and D, up to four in all. The variant buttons above the preview switch which variant you are editing, and a variant other than A can be deleted while the test has more than two.

Shared across variants, edited on A only: the deal's name, its products and visibility (selected and excluded products and collections), the market and excluded markets, the currency, Exclude B2B customers, Apply discount only via bundle widget, the active dates, and the shop-wide custom CSS. Everything else can differ, including bars, discounts, gifts, upsells, design, texts, badges and the timer.

If a variant differs from A only in the collections of "Volume discount with other products", the editor warns you: checkout gives every variant the same discount there, so that test changes nothing at checkout.

The split. Settings opens "A/B test settings". "Even allocation" splits traffic equally across all variants. "Custom allocation" sets the "Traffic sent into variants" other than A, from 5 to 95 percent in steps of 5 (with buttons for 10, 25, 50 and 95 percent); A gets the rest, and the other variants share the custom part equally. For example, a custom 50 percent with three variants gives A 50 percent, B 25 percent and C 25 percent. "Save settings" keeps it. As the modal says, changing the split moves some visitors to another variant.

Starting. The test starts when the deal is published with its variants: Publish, or Save on a deal that is already live. Until then it waits. A deal has one open test at a time, and there is no limit on tests across the store.

What shoppers see and pay

When a shopper's browser first reaches a deal with a running test, the widget gives that browser a random id, bndl_sid, kept in the browser's local storage. The id decides the variant, so the same browser keeps seeing the same variant on later visits, and one id serves every test in your store. Another device or browser, a private window, or cleared site data can land on a different variant. The id never leaves the browser and the app sets no cookies.

Each variant has its own prices: a shopper pays the price of the variant they saw, at checkout too. Two shoppers can see different prices for the same deal while a test runs.

Reading the results and the winner rule

A deal with a running test shows an A/B badge in the deal list. Expand it for a row per variant: how long the test has run (for example "3d 4h"), the columns you chose for the list, counted from the day the test started, a preview of the variant, and Set winner.

Variants are compared on conversion rate: orders with an item added through the deal, divided by visitors, counted as in Deal analytics. The best variant is tested against the second best with a two-sided test of two proportions. A "Significant winner" badge shows only when all of these hold:

• every variant has at least 200 visitors and at least 10 orders,
• the test has run at least 72 hours,
• the difference is significant at 95 percent with two variants. With three or four variants the bar is stricter: the 5 percent margin is split, to 2.5 percent with three variants and about 1.7 percent with four.

Hover the badge for how much higher the winner's conversion rate is. Until the badge shows, treat the numbers as a running count, not an answer. The test never stops on its own and never picks a winner for you.

Ending a test

• Set winner, on a variant's row: "Choose the winning variant" sends 100 percent of traffic to that variant and archives the rest. Confirm to apply.
• Stop A/B test, in the deal's menu: "End A/B test" lists every variant with its numbers. Choose any one to keep, significant or not.

Either way, the chosen variant becomes the deal, with A's shared settings, and a live deal updates on the storefront. The test moves to the A/B test archive, a view of the deal list, with its period, its numbers per variant and a "Winner" badge on the one you kept. A test that never started is dropped rather than archived. Removing a deal also closes its open test.

Start over, in the archive, runs the same variants again as a new test on the same deal. The deal must still exist and have no open test. Its settings go back to the archived variant A, keeping its current shared settings such as products and dates, and on a live deal the test starts at once.

Running a test, in four steps.

Settings reference

If this does not work

The failures most likely with a test, with the check that resolves each one.