Choose the decision the test must inform
Within a performance marketing plan, an A/B test should answer a narrow question that affects the next spending decision. Two ads appearing side by side in a dashboard do not automatically form a controlled experiment. Delivery can give one more spend, and ads running on different dates can face different competition. Define the choice and the outcome before launching.
For example, compare a video explaining a product’s ease of use with one explaining its durability. Keep the price, destination, period and purchase event comparable while changing the message. Changing the video, discount, country and landing page together produces a comparison of packages; it cannot isolate a single creative feature.
Tie the question to a practical decision: which message should the next production brief develop, or which market merits another test? A minor colour variation may have little value when unavailable stock is the actual sales constraint. Prioritise tests by their ability to change a decision rather than by curiosity alone.
Verify measurement and campaign readiness
A conversion measurement review should precede the test. Duplicate purchase records or a broken destination on one variant prevent a fair reading of the advertising idea. Use consistent event definitions, currency, reporting periods and attribution settings. Connect the campaign objective to the customer action you need; inexpensive clicks do not automatically mean profitable orders.
Document how audience eligibility, product availability, service geography and placement constraints apply to both options. Broad audiences are not inherently wrong, and narrow audiences are not inherently qualified. Existing customers and retargeting exposure can change the interpretation. If the question concerns acquisition, explain how a new customer is identified in the report.
Technical checks
Destination links, mobile pages, price and stock, event triggering, and deduplication where browser and server events describe the same action.
Commercial checks
Consistency between the ad and destination offer, actual delivery terms, returns exposure and capacity to fulfil the response.
Experiment record
Variant identifiers, the changed factor, fixed conditions, start period, primary metric and reasons that would require stopping.
FROM READING TO A NEXT STEP
Clarify your test plan
Prepare a scope covering the objective, current measurement and decision the test must inform.
Make the creative difference understandable
An ad creative brief makes the test variable visible to the whole team. When comparing messages, keep product presentation, the offer and destination reasonably comparable. Record any automatic visual or text enhancements that are enabled: the material shown to a user can differ from the approved production file.
Deliberately weakening one option does not produce useful learning. Both should be readable, accurate and suitable for publication. A different opening scene, demonstration or response to an objection can test a customer reason. Two barely distinguishable versions of the same idea may not create a useful contrast for the decision.
Read spend distribution as well as results. An ad receiving very little delivery cannot be declared a failed message simply because it produces no order. Automated allocation within one ad set can support operational exploration, but report it with different limits from a controlled A/B test. Duplicating an ad set alone does not establish comparable exposure.
Agree on duration, budget and interventions
Marketing decision management includes when a test will be reviewed and when it must stop. No single number of days or budget suits every account. Expected outcome frequency, sales delay, weekday patterns and the difference you want to distinguish all matter. When evidence will be scarce, narrow the question instead of adding more variants.
Do not call a winner from ordinary daily variation. Equally, do not keep spending through a broken checkout, incorrect price or breached loss limit just because the planned end date has not arrived. Record an intervention for technical protection separately from a performance decision, and explain changed conditions in the final interpretation.
Before launch
Write the decision, primary metric, loss limit, period and owner. Check the experiment options actually available in the account.
Launch
Complete technical checks and start the variants under the planned conditions. Avoid introducing another discount or audience change during the comparison.
Monitor
Review errors, spending and customer feedback. Log necessary interventions with their timing and reason.
Close
Assess the planned evidence, delayed outcomes and comparison limits. Record the decision and the next question.
Interpret a winner alongside business outcomes
The guide to CAC, ROAS and MER explains why a test should not end with click-through rate alone. For an order objective, acquisition cost and revenue may matter, while returns, margins and the share of new customers change the commercial decision. For lead generation, read sales acceptance alongside cheap form submissions.
The observed difference, confidence in that difference and its practical importance are separate considerations. Sparse outcomes can leave wide uncertainty; “not enough evidence yet” is a valid conclusion. A dashboard’s winner label cannot resolve an unmeasured margin difference or lead quality problem. The report should make its evidence and inference explicit.
The finding applies to the audience, period and implementation tested. A message leading in one experiment need not lead in every country or at a larger budget. Scaling creates another delivery context. Monitor the account’s overall business result after implementation to see whether the decision remains useful.
Carry the learning into the next brief
A usability review can investigate destination problems that an advertising test does not explain. A persuasive message can still lead to a confusing form, missing product information or a failing checkout. Keep a creative finding separate from a website problem in the test log.
Save the hypothesis, files, settings, evidence window, decision and limitations for each experiment. Rising frequency, negative comments or falling conversion are signals to investigate in context, rather than proof of an automatic account penalty. Reduce unnecessary duplication where it creates fragmentation, without promising that consolidation always lowers costs.
The next brief should say more than “B was better”. Record the customer reason explored, the element that changed and the feature worth preserving in the next production round. This turns test spending into a reusable decision record rather than a temporary dashboard comparison.
BEFORE YOU DECIDE
Frequently asked questions
Is adding two creatives to one ad set an A/B test?
It can help explore alternatives, but delivery may be uneven. For a controlled comparison, define the available experiment tool, variables and limits separately.
How many days should a test run?
There is no universal duration. Plan around outcome frequency, sales delay and period effects. Do not declare a winner from a small sample simply because the calendar period has ended.
Must a test change only one variable?
Keep other conditions comparable when isolating a factor. If several elements change, interpret the comparison as a package and avoid attributing the difference to one feature.
Can the budget change during a test?
An unplanned change can affect comparability. Log necessary technical or spending interventions. Do not assume that a fixed percentage applies to every learning reset.
Should a winning ad be scaled immediately?
Check commercial suitability and uncertainty first. Delivery can change at a larger budget, so monitor implementation. A test finding does not guarantee the same performance when scaled.
LET’S DEFINE THE SCOPE
Clarify your test plan
Prepare a scope covering the objective, current measurement and decision the test must inform.
Discuss the scope