A/B test significance calculator
Enter the visitors and conversions for two versions to see whether the difference is statistically significant. You get the conversion rates, the relative lift, the p-value and a confidence interval, with every step of the maths shown, plus a sample size planner.
How to use the A/B Test Significance Calculator
- 1Enter the visitors and conversions for version A (the original) and version B (the change).
- 2Choose the confidence level, 95% for most tests, and a two-sided test unless you have a reason not to.
- 3Read the verdict, the lift and the p-value. Open Show the maths to see how it was worked out.
- 4Before your next test, use the sample size planner to decide how many visitors you need.
Frequently asked questions
What does statistically significant mean in an A/B test?
That a difference this large would be unlikely if the two versions really performed the same. At 95% confidence, the p-value must be below 0.05. It doesn't tell you the difference is big or important, only that it's probably not chance.
Which statistical test does this calculator use?
A two-proportion z-test with a pooled standard error, the standard test for comparing two conversion rates. The confidence interval of the difference uses the unpooled standard error.
Can I stop my A/B test as soon as it becomes significant?
It's better not to. Checking results again and again and stopping at the first significant moment raises the chance of a false winner. Decide the sample size first and run the test until you reach it.
How many visitors does an A/B test need?
It depends on your current conversion rate and the smallest lift you care about. Small lifts on low conversion rates need thousands of visitors per version. The planner under the calculator works it out.
Related free tools
All toolsThen schedule it with Join Postly
Plan, publish and track your posts on every network from one calendar. Free plan, no credit card.