Skip to main content

How to A/B test a popup without mobile images

Test whether your popup gets more signups on mobile when its images are hidden.

Written by Rob D

Images can crowd a small screen. This test compares your current popup with a copy that hides its images on mobile, so you can see which one gets more signups.

Set up the test

  1. On the Campaigns page, open the campaign you want to test.

  2. Select Add test variant. Alia copies your live popup into Drafts.

  3. Open the new draft in the popup editor and switch to Mobile at the top of the left sidebar.

  4. Select the image you want to hide, either on the canvas or in the left sidebar.

  5. In the right sidebar under Styles, leave the dropdown on All screen sizes. Expand Visibility and set Display to Hidden.

  6. Change the dropdown to Desktop only and set Display to Visible.

šŸ’” Tip: Lia, Alia’s built-in AI assistant, can hide elements on one device for you. Select the image, then ask Lia in the popup editor, for example ā€œHide this image on mobile only.ā€ Check the before and after in Review changes, then select Apply edits. See Getting started with Lia.

  1. Repeat for each step that has an image, then adjust spacing so your button stays easy to see.

  2. Rename the draft so it's easy to spot in the results, such as "No mobile images". If it shows Not ready, select Mark as ready.

  3. Back on the campaign, select Start new A/B test. Enter a Title, check the Control and Traffic weight, then select Start test.

šŸ’” Note: Deleting an image from the step's content removes it on desktop too. Use Visibility so desktop visitors still see it.

Starting a test puts every popup in the campaign into it, including all drafts. Archive any draft you don't want in the test first.

Let a Smart Test run it for you

The Optimization essentials Smart Test includes a Remove mobile images test, along with other common tests. Alia builds the variant, runs the test, and picks the winner. To start one, select Create Smart Test in the campaign's A/B test section. See Setting up Smart Tests in Alia.

Read the results

Select View test results on the campaign. Compare the email and phone submit rates and the attributed metrics. Use Show filters to look at mobile visitors only, since desktop visitors see the same popup in both variants.

Wait for check marks on the metrics you care about, and give the test at least a full week before picking a winner.

šŸ’” Tip: Lia, Alia's built-in AI assistant, can build the variant for you. Select the arrow next to Add test variant and choose Build variant with Lia. See Getting started with Lia.

Frequently asked questions

Will hiding images on mobile change my desktop popup?

Not if you use Desktop only visibility, or delete the image from the mobile template. In the element tree, an eye icon marks elements that are visible on desktop only.

Can I add this variant to a test that's already running?

No. Pick a winner first, then start a new test that includes the new variant.

Should I only filter results by mobile?

Yes, for this test. Desktop visitors see the same images in both variants, so their numbers dilute the difference.

Still have questions?

Start a chat from inside the Alia app, or use the chat in our Help Center.

Did this answer your question?