← Back to glossary
Performance & Measurement
4 min read
A/B testing — also called split testing — is a controlled experiment that pits two or more versions of an ad, page or setting against each other to see which performs better. Traffic is split at random so every variant meets a comparable audience, and one success metric decides the winner. It is the difference between believing a change worked and knowing it did — exactly the kind of call you want made on evidence, not instinct.
Form a clear hypothesis. Start with one specific, measurable prediction — a shorter headline will lift click-through — not a vague hope that something improves.
Change one variable at a time. Hold everything else steady so any difference in results traces back to the single element you changed.
Split traffic at random. Divide the audience into statistically equivalent groups, so the outcome reflects the creative and not the crowd.
Read for significance. Wait until enough data has built up to trust the gap is real — usually a 95% confidence level — before calling a winner.
Roll the winner out. Promote the stronger variant to all traffic, retire the weaker one, and feed the lesson into your next test.
Ad copy. Headlines, descriptions, calls to action and the core value proposition are the fastest, cheapest things to test.
Creative format. Static versus video, single image versus carousel — format often shifts performance more than wording does.
Landing pages. Layout, form length, social proof and imagery all decide whether a click becomes a conversion.
Audiences. Broad against narrow targeting, or one segment against another, to learn where the message lands.
Bidding and budget. Compare strategies like target-CPA against maximise-conversions to see which spends more efficiently.
Small gains compound. A steady cadence of modest wins adds up to outsized performance across a quarter.
It de-risks change. Validating on a slice of traffic before full rollout limits the damage of a bad idea.
It builds real knowledge. Every test leaves you with evidence about your audience that outlives the campaign.
Significance is easy to misread. Reliable results need thousands of impressions per variant over a week or two; calling it early is how false winners slip through.
Too many tests at once. Dozens of concurrent experiments inflate false positives unless you tighten the thresholds.
Velocity beats perfection. A consistent testing rhythm improves results faster than occasional, ultra-precise tests.
Platforms push back. Automated delivery algorithms can blur clean test design, so account for their behaviour.
Winners go unused. The most common failure is not a bad test — it is never shipping the variant that won.
Keep exploring
Browse all 61 advertising terms
→
OpenAds connects your assistant to every major ad platform, so you can plan, launch and optimise campaigns in plain language — approving the moves that matter.
Explore OpenAds