How long an A/B test should run: sample and traffic → whole weeks, the end date with Russian public holidays, the traffic budget. If the duration is unrealistic — four ways to shorten it, recalculated, and the reverse question: which effect you will see in the time you can wait.
Your numbers stay in the browser and are never sent anywhere
Fix the fields marked in red and the result appears at once.
3 weeks — feasible.
Do not stop it before the planned date, even if it is “already significant”.
02
How to shorten the test
01
Look for a bigger effect
+20% → 21,109 per group, 2 weeks; +30% → 1 week
02
More traffic
To fit into 4 weeks you need 2,621 visitors a day — 0.5 times more
03
Relax the threshold: α = 0.10
≈ 28,904 per group → 2 weeks. Up to 10% false wins
04
CUPED on a pre-test metric
At a correlation ρ = 0.5 the sample × 0.75 → 2 weeks
02
How long you can wait — which effect you will see
A whole number of weeks, 1 to 104
In 4 weeks the test sees effects from +11 % (2.00 → 2.22 %)
A realistic threshold for a noticeable change.
DurationEffect fromConversion B
1 wk+22 %2.44 %
2 wk+15 %2.31 %
3 wk+12 %2.25 %
4 wk+11 %2.22 %
6 wk+9 %2.17 %
8 wk+8 %2.15 %
12 wk+6 %2.12 %
03
Examples
04
Frequently asked questions
Until the sample calculated in advance is reached, rounded up to whole weeks and no shorter than one week. The duration is all test visitors divided by daily traffic. Example: 2% conversion and an expected +15% uplift need 36,693 visitors per group, 73,386 in total. At 5,000 visitors a day that is 14.7 days, so 3 whole weeks rather than 15 days.
People behave differently on weekdays and weekends: on Monday they browse more, at the weekend they buy more, or the other way round. A 10-day test has three Mondays and two Sundays, so the result leans towards the days it has more of. Whole weeks balance the days, which is why the calculator always rounds the duration up to a whole number of weeks.
Not unless the plan included interim looks. “Already significant” on day five is a common trap: checking every day raises the chance of seeing significance by accident, and 20 checks raise the false-win rate from 5% to 24%. If you must look along the way, plan a sequential analysis in advance, with stricter thresholds at each look.
Shorten it in one of four ways — the calculator recalculates the weeks for each. At 2% conversion and 1,000 visitors a week, a test for +15% takes 74 weeks. Looking for +30% takes 20 weeks, relaxing the threshold to α = 0.10 takes 58, CUPED 56, and fitting into 4 weeks needs 18 times the traffic.
They change behaviour and the audience: during holidays different people come with different goals. The calculator marks Russian public holidays in the test period using the official work calendar. A two-week test starting on 21 December, for example, covers the New Year holidays. Such tests are not stopped, but check that both groups met the holidays equally and draw no conclusions from the holiday days alone.
The main cost is traffic: multiply the number of test visitors by the price of one visitor. A button converting at 2% with a +15% uplift needs 73,386 visitors; at 12 roubles a visitor that is about 880,000 roubles. Enter the price in the calculator and the budget appears with the duration. If it is too high, look for a bigger effect: the sample falls with the square of the effect.