Testing rules

Last updated: November 1, 2026

How we run every A/B test and pick a winner. Every detail, for anyone who wants to check our math. The short version is on How we test.

What decides the winner

• Sales and the money they bring, from the deals, booking, store or call tracking app you connect. Not clicks.
• If you don’t connect an app, the leads your website gets decide.
• If one version gets more orders and the other brings in more money, money decides.
• With few sales, the step closest to a sale decides, like a booked appointment or a qualified lead. We still check that the winner doesn’t bring in less money.

When a version wins

• At 95% confidence that it brings in more.
• We check results every day with sequential testing, so daily checks don’t create false winners.
• If the new version is worse at 99% confidence, we turn it off early.
• No clear winner by the end of the test: your current page stays.

How long a test runs

• At least two full weeks, so every day of the week counts twice. Eight weeks at most.
• Your dashboard forecasts the length from your last 7 days of visitors. If a clear answer is unlikely within eight weeks, we tell you.

How visitors are split

• 50/50, at random. Each visitor sees the same version on every visit.
• We check the split every day. If it drifts from 50/50 more than chance allows (sample ratio mismatch), we pause the test and find the cause.

How search engines see the test

• Your page keeps its address, with no redirects. Your canonical tag stays as it is.
• When the test ends, everyone sees one version: the winner or your current page.

Who isn’t counted

• Bots. They see a version like everyone else, so search engines see the same pages as visitors.
• Visitors where the law asks for consent first (the EU and EEA, the UK, Switzerland, Quebec) and browsers with Global Privacy Control. They see your current page.

How a sale is matched to a version

• Forms: we note which version the visitor saw. We find the sale in your app by a hash of their email or phone.
• Calls: through your call tracking app, like CallRail.
• Online orders: from Stripe, Shopify or WooCommerce, matched to the visit.
• A sale counts for the test if it happens within 30 days of the visit. Your report keeps counting after that.

Rules that don’t change

• What we change and the longest the test can run are set before it starts. You can change what picks the winner at any time, and we count the whole test by it from day one.
• One test per page at a time.
• Nothing goes live until you say yes.

What you see

Your dashboard shows:

• Visitors, sales and money for each version, how sure we are, and about how many days are left.
• The steps before a sale, like call clicks and started forms, for each version. You see which steps the new version wins or loses, at the same 95% confidence. Steps explain the result. Sales decide the winner.
• A simple funnel from visit to sale, so you see where visitors drop off.

You can turn the new version off, or back on, at any time. If you stop before the end, we can’t tell which version sells more.

See what your dashboard looks like.

Questions

Email hello@simplesplit.io.