How to A/B Test Your Email Sequences (With Real Numbers)

The Most Common A/B Testing Mistake

Teams obsess over subject line tests. "Version A: 'Quick question for you.' Version B: 'Your account is ready.' 4,200 sends, Version A wins by 3 points." And then they move on feeling like they've done the science.

The problem: subject line tests optimize for opens. Opens don't pay the bills. If Version A gets more people into the email but those people convert at a lower rate than Version B's audience, you've made things worse while feeling good about the test. You need to test the things that affect the outcome you actually care about — which, for most SaaS products, is activation or paid conversion.

What to Actually Test

The call to action in your highest-leverage email. For most onboarding sequences, that's email 2 or 3 — the one where you're asking someone to do the thing that makes them a real user. Test the action itself: is it a demo video, a step-by-step guide, a direct link into the product? That choice drives clicks and downstream behavior far more than whether the subject line starts with a question.

Send timing on key emails. Day 1 vs day 2. Immediately after signup vs 4 hours later. These seem small, but for fast-moving trial periods, timing on the first two emails can shift activation rates by 15-20%. Test it properly before assuming the default timing is right.

Email length on the close email. Short and direct (three sentences, one link) vs longer and explanatory (objection handling, social proof, full rationale). The right answer varies by product and audience — high-consideration purchases generally benefit from more copy, low-friction signups generally don't. Test yours. Don't assume.

Full sequence paths, not just individual emails. This is where A/B testing in a sequence tool like Driplane becomes more powerful than testing individual emails. You can split your audience at a branch point and send path A through one sequence of emails and path B through a different one. Measure conversion at the end of the entire path, not just the email you're testing.

How to Run a Clean Test

The basics: equal split, single variable, sufficient sample size, measure the right metric.

Single variable. Change one thing at a time. If you change the subject line and the CTA and the email length in the same test, you don't know what moved the needle. It sounds obvious but teams violate this constantly because they want to "optimize everything at once."

Sample size before you call it. For open rate tests, you need at least 500 sends per variant before the numbers mean much. For conversion tests (which have much lower base rates), you need 1,000+. Running a test on 200 sends and calling Version B the winner is how you make confident decisions based on noise. If you don't have the traffic yet, run the test longer rather than deciding early.

Statistical significance.) A 3-point difference in open rates on 200 sends is not a result — it's a coin flip. You want 95% confidence before acting on a test. Most email tools will calculate this for you. Use it.

What a 20% Lift Actually Looks Like

Say your trial-to-paid conversion rate is 5% and you run 200 trial users per month through the sequence. That's 10 conversions per month. A 20% lift — which is very achievable from a single good A/B test on the right email — takes you to 12 conversions per month. That's 24 extra conversions per year. At $47 per conversion, that's $1,128 from a test that took a few hours to set up.

The math compounds. If you run three well-designed tests per quarter over a year, and each one moves conversion by 15-20%, you don't end up with a 60-80% improvement — sequential tests on an improving baseline give you a compound effect. That's why teams who take A/B testing seriously end up with conversion rates 2-3x higher than teams who don't, even starting from the same product.

A/B Testing in Driplane

Driplane has a built-in A/B split step that uses a deterministic hash to split contacts between two paths — you set the ratio (50/50, 70/30, whatever you want) and each contact is consistently assigned to the same path across sessions. You can split at any point in the sequence and measure downstream outcomes per path in the analytics dashboard. It's not a complex tool, but it's enough to run clean tests without needing to build the split logic yourself.