A/B Testing Landing Pages

When to Use

Use this skill when planning, running, or analyzing A/B tests on landing pages.

Core Rules

1. Test One Variable at a Time

Each A/B test should change exactly one element: the headline, the CTA copy, the hero image, or the pricing display. Testing multiple changes simultaneously makes it impossible to know what drove the result.

2. Start With High-Impact Elements

Test in this order: headline, primary CTA, hero image, pricing display, social proof placement. These elements have the highest potential impact. Testing button border radius or footer color wastes statistical power.

3. Run Tests Until Statistical Significance

A test is not complete when you feel confident — it's complete when you reach 95% statistical significance with at least 200 conversions per variant. Stopping early because one variant looks better is the most common A/B testing mistake.

4. Segment Results by Traffic Source

A headline that wins for paid traffic may lose for organic traffic. Always segment results by traffic source, device type, and new vs. returning visitors before drawing conclusions about a winning variant.

5. Document Every Test

Maintain a test log with hypothesis, variant description, run dates, sample size, result, and what you learned. Teams that document tests compound learning over time. Teams that don't repeat the same mistakes.

Quick Reference

Common Mistakes to Avoid

• Stopping tests after seeing a promising early result — early leads often reverse
• Testing on insufficient traffic (under 1,000 visitors per variant)
• Not accounting for day-of-week effects — run tests for full 7-day cycles

Test Your Product with Racoonn

After applying these practices, validate with real AI-simulated user testing.

Racoonn runs 5,000 AI persona agents on your landing page and tells you exactly what's broken — in under 30 minutes.

→ API coming soon — Join the waitlist for early access: racoonn.me