Skip to main content

Setting up Smart Tests in Alia

Let Alia run a series of A/B tests on a campaign, keep each winner, and move on to the next test.

Written by Rob D

A Smart Test is a schedule of A/B tests that Alia runs one after another on a campaign. You choose and approve the tests. Alia runs each one, ends it when one variant is clearly better, keeps the winner, and moves on to the next. Smart Tests are available to every store.

Create a Smart Test

  1. In Alia, go to Campaigns and open the campaign's A/B test section.

  2. Select Create Smart Test.

  3. In Build a Smart Test, choose Optimization essentials, the recommended pack, or a single test under All tests.

  4. Select Next.

Review each test

Optimization essentials walks you through each test in turn:

  1. Smart Trigger

  2. Mystery copy

  3. First step variants

  4. Countdown timer

  5. Remove mobile images

  6. Remove the "No Thanks" button

  7. Move the X button (mobile only)

  8. X button delay

  9. Delay time

For each test, review the variants Alia proposes. You can edit them, accept them, or skip the test. Delay tests find the best delay for you, so you don't enter delays yourself.

Start the test

  1. Review the schedule. Drag tests to change the order they run in.

  2. Select Start test.

To throw away a Smart Test before it starts, open the menu next to Start test and select Discard test.

Tests you can run

Test

What it tests

Trigger delay

The best time to trigger the popup

Smart Trigger

Triggering again when the visitor is likely to engage

Header and button copy

Alternative header and button copy

Remove mobile images

Removing unnecessary images on mobile

Remove "No thanks"

Removing the "No thanks" button

Custom content

Your own variations of the popup content

X button position

Moving the close button to the opposite side

X button delay

Delaying the close button

Different templates

The best visual template for your popup

First step variants

Different first step designs, such as a game or a question

Countdown timer

Adding a countdown timer to create urgency

Some tests aren't available on every popup, for example when it uses Postscript on-site opt-in or Adaptive timing. The builder greys these out and says why.

Follow your results

Select Smart Test info on the campaign to see progress. From there you can Pick winner, move to the Next test, or End Smart Test.

  • Alia judges variants on signups and orders, and shows each one's % prob. to win.

  • A variant that's clearly losing can be hidden early.

  • If two variants stay too close to call for long enough, the test ends as a tie and moves on.

Change the popup during a Smart Test

Each test in a Smart Test is built from your campaign's popup, called the control. While the Smart Test runs, changes to the control don't reach visitors until you publish them, and publishing them restarts the current test.

  1. Open the control popup in the popup editor.

  2. Make your changes.

  3. Select Update and restart test, which takes the place of Publish.

  4. In Restart A/B test, select Publish and restart A/B test.

Restarting the current test can also change the variants of the tests that come after it. To compare a big change, like fullscreen vs. not fullscreen, end the Smart Test first and run that comparison as its own test.

💡 Tip: Lia, Alia's built-in AI assistant, can end a Smart Test or start the next A/B test for you, after you approve. She can also change the control popup, but she can't publish it. Publish her changes with Update and restart test. See Getting started with Lia.

Frequently asked questions

Do Smart Tests cost extra?

No. Smart Tests are available to every store.

How does a Smart Test pick a winner?

It waits until each variant has had enough views, then ends the test when one variant is very likely to be best on signups and orders. You can also Pick winner yourself at any time.

Why is a test greyed out?

It doesn't work with this popup. For example, Optimization essentials isn't available when the popup uses Postscript on-site opt-in, and the delay test isn't available when the popup uses Adaptive timing.

Can I change my popup while a Smart Test is running?

Yes. Edit the control popup in the popup editor, then select Update and restart test. Your changes go live, and the current test starts over with the updated popup. See Change the popup during a Smart Test in this article.

Still have questions?

Start a chat from inside the Alia app, or use the chat in our Help Center.

Did this answer your question?