A/B Test Calculator
Plan how many visitors you need before launching a test — and check whether the test you already ran is statistically significant.
Inputs
Your current control conversion rate.
Smallest lift you care about. 10–20% is typical for landing pages.
For a duration estimate. Combined across both variants.
24,190visitors per variant
Total of 48,380 visitors across both groups. At 5,000 daily visitors that is roughly 10 days.
Two-tailed test for two proportions. Lower MDE = more visitors needed (relationship is quadratic). Run for full week-cycles to avoid weekday bias.
Control (Variant A)
Variant B (Challenger)
99.2%
Statistically significant winner! Safe to roll out Variant B to 100%.
Control CR
3.20%
Variant CR
4.10%
Uplift
+28.1%
Try these next
Related tools you'll probably want
Want to optimize your conversion funnels?
I configure GA4 custom event tracking, micro-conversions, and Looker Studio dashboards to measure test results accurately.
How to use the Statistical Significance A/B Test Calculator
Making business decisions on under-powered A/B tests can damage revenue. This calculator uses two-tailed statistical hypothesis testing to compute confidence intervals, p-values, and statistical power to verify whether a conversion rate lift is genuine or random noise.
Step by step
- Enter total visitors and conversions for Variation A (Control).
- Enter total visitors and conversions for Variation B (Challenger).
- Set your required statistical confidence level (standard is 95%).
- Check whether the test reached statistical significance and review the required additional sample size if needed.
How to read your result
Statistically Significant (p < 0.05)
95%+ confidence that the observed difference is real and not due to random sampling chance. Safe to declare a winner.
Inconclusive / More Data Needed
Observed lift does not meet the 95% confidence threshold. Continue running the test to reach the required sample size.
Common mistakes
- Calling tests too early during the first few days before capturing weekday/weekend cyclical variance.
- Testing multiple overlapping changes simultaneously without multivariate test architecture.
- Stopping tests immediately after a brief positive spike caused by novelty effect.
Questions
What sample size do I need for an A/B test?
Required sample size depends on baseline conversion rate and Minimum Detectable Effect (MDE). Smaller uplifts require significantly larger sample sizes to prove statistically.
How long should an A/B test run?
A test should run for at least two full business cycles (typically 2 to 4 weeks) to account for day-of-week seasonality, even if statistical significance is reached early.
Want to optimize your conversion funnels?
I configure GA4 custom event tracking, micro-conversions, and Looker Studio dashboards to measure test results accurately.