Onboarding A/B testing
Test guide variations and see which one drives more activation. Split traffic across variants, let Wakeline assign users automatically, and read a clear winner.
Test any variation
Pit two or more versions of a guide against each other.
Fair, sticky splits
Each user is assigned one variant automatically and consistently across visits.
Know the winner
Compare conversion and uplift with statistical confidence.
Live in three steps.
Pick your variants
Choose the guides to test and a goal.
Start the test
Wakeline splits traffic by weight.
Read the results
See conversion, uplift, and confidence.
Settling the three-step-versus-five-step argument.
Half the team says the tour is too long; the other half says it is thorough. An experiment replaces the argument with a number.
Create the variants
Duplicate the five-step tour and cut the copy to three steps. Variant A (control): the original. Variant B: the short one. Change only length — a test that varies five things teaches nothing.
Split the traffic
50/50 weights; Wakeline assigns users automatically and stickily — nobody sees both versions. Goal: completion, not starts (starts only measure reach, which the weights already control).
Wait for confidence
The results table shows started, goal, conversion, and uplift versus control per variant. "Confidence" reads not yet until the two-proportion z-test clears 95% — resist calling it early.
Ship the winner
B converts 22% better at 95%+ confidence. Ship B, retire A, and log the finding: your users prefer short. That heuristic compounds into every future guide.
What to expect: A decision backed by statistics instead of seniority. Pause works at any time; editing a variant mid-test does not — changed content mid-experiment muddies what you measured.
When not to use a/b testing.
No pattern fits every job. Here’s where we’d point you at something else — sometimes ours, sometimes not.
Low-traffic guides
Significance needs volume. A guide seeing 40 users a week will take months to reach 95% — for those, ship your best judgment and watch the funnel instead.
Testing five changes at once
If variants differ in copy, length, timing, and design, a win is unattributable. One variable per test; run the next test after.
Peeking and calling early
Day-two "trends" reverse routinely. The confidence column exists precisely so you do not ship on noise — not yet means not yet.
Questions a test cannot answer
A/B tells you which variant converts better, not whether the guide should exist. "Should we interrupt users here at all" is a judgment call informed by dismissal rates — make it before optimizing the interruption.
How are users assigned to variants?
Automatically and randomly per the weights you set, and stickily — a user keeps their variant across sessions, so nobody experiences both and contaminates the result.
What goals can I measure?
Guide started or guide completed, per variant, with conversion (goal ÷ started) and relative uplift versus your control. Completion is usually the honest goal — the weights already control reach.
When is a result trustworthy?
When the confidence column shows 95%+ — a two-proportion z-test against control. "Not yet" means keep collecting; the display is deliberately binary to keep day-two enthusiasm from shipping noise.
Can I stop or change a running test?
Pause stops serving the experiment instantly. Avoid editing variant content mid-test — the results would mix two different experiences under one label. Finish, then iterate in the next test.
What should my first experiment be?
Your highest-traffic guide, one variable, completion as the goal. Length (three steps versus five) and first-step copy are the classic high-yield tests — both regularly move double digits.
See it on your own app.
Ship your first guide in minutes. Free to start — no credit card.