AB Test Planning Guide

Before you start, it helps to plan your test properly. Here are the key steps:

1. Template test vs Theme test

  • Template test: Use this if you’re testing elements within a single page template, like product page layout, or upsell block placement.
  • Theme test: Use this if you want to test elements that appear site-wide, like announcement bars, footers, or cart changes.

2. Pick what you want to test

  • Choose one page type (e.g. product page, homepage, collection page, cart).
  • Decide on one clear change (e.g. new layout, moving a section, adding an app block).

Keep it simple - one page and one change gives the cleanest results.


3. Define your goal

  • Decide what success means for your test:
    • Higher add-to-cart rate
    • Higher conversion rate
    • Higher average order value (AOV)

This keeps your results actionable.


Hypothesis template:

If we [change X] on the [page], then [metric] will improve because [reason].


4. Run the test right

  • Run one test at a time on a given page/type.
  • Let it run long enough to stabilize (at least a full business cycle, e.g., 1–2 weeks) or until you have solid traffic/conversions.
  • Avoid overlapping changes or running multiple A/B tools simultaneously.
  • The app will show you when your test reaches statistical significance.

Read more about interpreting test results.


5. Choose the winner

  • When the test is complete, pick the winning version and publish it with one click.


💡 Still need help?

If you have any questions or run into issues, our support team is here for you. Just reach out through the in-app chat and we’ll be happy to help.

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.

Still need help? Contact Us Contact Us