A/B Testing Landing Pages: What Actually Moves Conversion Rates
Most A/B tests fail not because the concept is wrong, but because of how they're run.
Testing button color while leaving a weak headline untouched is the most common version of this mistake — technically an experiment, practically noise. The teams that get real lift from testing share one habit: they test the things that actually drive the decision, not the things that are easiest to change.
Start with a hypothesis, not a hunch
Before touching anything, write down why you think a change will improve conversions.
"Let's try a different CTA" is not a hypothesis.
"Users drop off because the CTA is vague — changing it from 'Get Started' to 'Start Your Free Trial' removes ambiguity about commitment at the decision point" is one. This forces you to think about mechanism, not just outcome. It also tells you what data you need to call the test correctly, and what you've learned if the test loses.
What's worth testing first
Research from CXL Institute consistently shows that messaging and offer changes produce larger lifts than visual changes. Start here:
Headline — The single highest-leverage element on most landing pages. Nielsen Norman Group research shows users decide whether to stay on a page within 10–20 seconds. The headline is doing most of that work.
CTA copy — Specific and outcome-focused beats generic. "Book a 20-minute walkthrough" outperforms "Contact Us" in most tests because it sets expectations and reduces the perceived commitment.
Form length — Hubspot's research on form fields found that reducing form fields from 4 to 3 increases conversions by 50% on average. Test cutting fields before anything else.
Hero section — Image vs. video vs. product screenshot. Does showing the product working earlier change signups?
Social proof placement — Moving testimonials above the fold changes the frame before the visitor reads anything else. The same proof at different positions produces measurably different results.
Lower-impact tests not worth your traffic budget early:
- Button color (unless contrast is broken)
- Font choices
- Footer layout
- Icon styles
How to run a test that's reliable
1. One variable at a time. Non-negotiable for clean results. If you change headline and CTA simultaneously, you won't know which one produced the movement.
2. Calculate sample size before you start. Evan Miller's sample size calculator is free and takes two minutes. Most teams stop at 200 sessions per variant and call it. For a baseline conversion rate of 5% and a minimum detectable effect of 20%, you need roughly 2,500 visitors per variant. Know that number before you start, not after.
3. Run for at least two weeks regardless of when you hit significance. Traffic behaves differently on different days of the week and different weeks of the month. Four days of data can show 95% confidence and still be wrong because it only captured weekend traffic.
4. Measure the conversion event, not proxies. Not time on page. Not scroll depth. Your actual business goal — signup, demo request, purchase.
5. Document every test. What you changed, the hypothesis, the result, the confidence level, what you learned. This becomes your most valuable growth asset over time — a record of what your specific audience responds to. Most teams skip this and re-run the same tests years later.
Reading results honestly
95% confidence means there's a 5% chance you're looking at noise. Run enough tests and you'll get false positives. This is why you retest meaningful wins before making them permanent.
A losing test is still useful. If a more specific CTA didn't outperform the generic one, that tells you something real about how your audience processes the decision — maybe they're not confused about commitment, maybe they're confused about the offer itself. Write it down.
| Element tested | Control | Variant | Lift | Lesson |
|---|---|---|---|---|
| Headline | "The smarter way to manage projects" | "Cut weekly status meetings in half" | +18% | Specific outcome > vague benefit |
| CTA | "Get Started" | "Try it free — no card required" | +11% | Commitment reduction works |
| Form | 5 fields | Email only | +34% | Friction is real |
If you're running paid traffic to pages and the conversion rate isn't moving — testing is usually the right answer, but only after the page's core offer and message are solid.
Fixing the offer and then testing is faster than testing your way to a good offer.
We look at both →
