A/B Test Significance Calculator
Two-proportion z-test with p-value, confidence interval, and relative lift. 90 / 95 / 99% levels.
published
- [FREE]
- [NO_SIGNUP]
- [NO_UPLOAD]
An A/B test significance calculator runs a two-proportion z-test on conversion data from two variants and tells you whether the difference is statistically significant.
Privacy
Pure JS math. No upload.
Frequently asked questions
Which test is used?
Two-proportion z-test, two-tailed. Standard for comparing two binary outcome rates (converted / not converted) when each visitor is counted once.
What does "significant at 95%" mean?
P-value < 0.05. Roughly: if there were no true difference between A and B, you would see a result this extreme less than 5% of the time by chance alone. Not a 95% chance the result is real — that's a Bayesian framing this frequentist test doesn't directly provide.
What sample size do I need?
Depends on your minimum detectable effect (MDE). Rough rule: for a 5% baseline rate and 20% relative lift, you need ~1500 visitors per variant for 80% power at 95% confidence. Use a power calculator before running the test, not after.
Limitations?
This test assumes binary outcomes, independent visitors, and stationary populations. Not appropriate for: repeated measures, multi-armed bandits, sequential testing (peeking), or revenue/continuous metrics (use a t-test).