An A/B test gives you one link to put in your ads. Every visitor who clicks it lands on one of the products you picked, chosen at random, so each product gets an equal share of the traffic. The statistics then show which product sells best, in all your markets together and in each market, so you can stop guessing and keep the winner. You can test different products, or two versions of the same product, for example the same item with a different price or different photos.
Creating a test
- Open Products from the left menu.
- Click the A/B tests button, then Add new A/B test.
- Fill in the fields below, then click Create.
- Name: only visible to you.
- SEO slug: the last part of your ad link. Use letters (Latin or Arabic), numbers and dashes only, for example
summer-offer. As you type, capital letters are turned into lowercase and spaces into dashes. Each test in your store needs its own SEO slug. The full link is shown under the field as you type. - Product: search for a product, select it, then click the plus button. Pick between 2 and 5 products. The same product can only be added once.
To test two versions of the same product, duplicate it first, change the price or photos of the copy, then pick both. See managing products.
Using the ad link
Your ad link has the form https://example.com/go/your-link. Open the test from the list and copy it from the Ad link field with the copy button. You can add tracking tags such as utm_source to the end of it. They are passed on to the product page, so your pixels and campaign reports keep working.
- Each visitor keeps seeing the same product for 30 days, even if they click the ad again. If that product is removed from the test or unpublished, they get another product of the test.
- A visitor is counted once per test, even when they come back later, and in the market of their first visit.
- Bots, blocked visitors and your own visits from the builder preview are not counted.
- An unpublished product is left out of the split until you publish it again.
Reading the statistics
Open a test to see its Statistics tab. The All markets tab adds up every market. When visitors come from more than one market, a tab with each market's flag shows that market alone. For each product you see:
- Visitors: the people who clicked the link and were sent to this product.
- Orders: orders from those visitors that contain this product, counted in the market of the order. Test orders, unfinished checkouts and duplicate orders are not counted.
- Conversion rate: orders divided by visitors. This is the number that decides the winner.
- Price (market tabs only): the product's price today in that market, so you can see if the price explains the result.
- Revenue and Average order value: converted to your store currency, so products sold in different markets can be compared.
The product with the best conversion rate is marked Leading, and the sentence above the table tells you how confident the result is. It becomes the Winner when the confidence reaches 95%, every product has at least 100 visitors, and the leader has at least 5 orders. Each market tab is judged the same way on its own numbers. Before that, the gap could still be luck, so keep your ads running.
- Not enough data yet: the products need more visitors and orders first.
- The products are too close to call: no product is clearly ahead yet.
- Each market has a different winner: one product wins in one market and another product in another market. Open each market tab to see its result. In the list, the test shows Different winner per market with the market codes under it.
In the list of tests, the Leading product column shows the leader or the winner and its confidence. Hover over the number in the Products column to see each product with its visitors, orders and conversion rate.
Adding or removing products
While a test is running, you can change its products in the Settings tab, then click Save. A test always keeps between 2 and 5 products.
- Adding a product: it starts from zero visitors and zero orders, while the other products keep their results. It cannot be the Winner before it reaches 100 visitors too.
- Removing a product: once you save, its visitors and orders in this test are deleted for good. Adding it back later starts it from zero again.
The products of an ended test cannot be changed. Create a new test to try other products.
Ending a test
Click End test and confirm. From then on, every visitor of the link is sent to the product with the best conversion rate in all markets together, so your ads keep working without any change. If nobody ordered yet, visitors are sent to the first product. The statistics stop counting when the test ends, and an ended test cannot be started again.
To send visitors to a different product after the test ended, for example the winner of your main market, open the Settings tab, pick it in Product shown to every visitor, and click Save.
Changing or deleting a test
- In the Settings tab you can change the name and the SEO slug at any time. Remember, if you already shared the link or used it in your ads, changing the SEO slug will make visitors see a page not found.
- To delete tests, tick them in the list and choose Delete A/B tests. Deleting a test removes it and its statistics for good, and its link then shows a page not found. Deleting a product also removes it from your tests, together with its statistics.
To build tracking tags for a single product link instead, see How do I build a UTM campaign URL for a product?