AB Test Planning Guide
Before you start, it helps to plan your test properly. Here are the key steps:
1. Template test vs Theme test
- Template test: Use this if you’re testing elements within a single page template, like product page layout, or upsell block placement.
- Theme test: Use this if you want to test elements that appear site-wide, like announcement bars, footers, or cart changes.
2. Pick what you want to test
- Choose one page type (e.g. product page, homepage, collection page, cart).
- Decide on one clear change (e.g. new layout, moving a section, adding an app block).
Keep it simple - one page and one change gives the cleanest results.
3. Define your goal
- Decide what success means for your test:
- Higher add-to-cart rate
- Higher conversion rate
- Higher average order value (AOV)
This keeps your results actionable.
Hypothesis template:
If we [change X] on the [page], then [metric] will improve because [reason].
4. Run the test right
- Run one test at a time on a given page/type.
- Let it run long enough to stabilize (at least a full business cycle, e.g., 1–2 weeks) or until you have solid traffic/conversions.
- Avoid overlapping changes or running multiple A/B tools simultaneously.
- The app will show you when your test reaches statistical significance.
Read more about interpreting test results.
5. Choose the winner
- When the test is complete, pick the winning version and publish it with one click.
💡 Still need help?
If you have any questions or run into issues, our support team is here for you. Just reach out through the in-app chat and we’ll be happy to help.