Attributed Is Not Incremental: Design the Holdout Before Reading ROAS

Attributed conversions are outcomes assigned to an ad interaction under a platform or analytics rule. Incremental conversions are the difference between what occurred with an intervention and a credible estimate of what would have occurred without it. Loyal buyers, branded demand, seasonality, and other channels can all appear inside attributed ROAS.

Write one decision first: maintain, expand, reduce, or stop a defined intervention for an eligible population. Name the spend, campaign, audience, market, offer, and period. “Does paid social work?” is too broad to test and too vague to act on.

Before results are visible, record the hypothesis, primary metric, guardrails, unit of assignment, eligibility, exclusions, sample-size or feasibility assessment, duration, conversion lag, analysis method, and decision threshold. This limits the temptation to choose whichever segment or window looks most persuasive afterward.

Google describes lift as treatment outcomes minus control outcomes and offers user-based and geography-based studies subject to eligibility. Platform tooling can make assignment and measurement practical, but the team still owns the business outcome and the assumptions around spillover, interference, and data quality.

A study that ends before delayed conversions arrive measures a different outcome from the one the business cares about. Google’s guidance notes that duration should reflect conversion lag and that many lift studies run longer than fourteen days. Promotions, inventory changes, outages, pricing, and other campaigns should be logged before and during the test.

Do not peek each morning and stop at the first attractive result. Follow the planned duration and analysis unless a safety, budget, or data-quality condition requires stopping. Record that reason. Frequent unplanned checking raises the chance of mistaking noise for evidence.

Report the estimated lift with uncertainty, treatment and control outcomes, cost, population, period, exclusions, and deviations. An estimate that spans commercially negative and positive effects is inconclusive for that decision. It should not be polished into either “the ads did nothing” or “the test proved success.”

Translate the range into unit economics and capacity. Then decide whether to repeat with more power, change the intervention, narrow the question, or stop. Store the study beside creative, audience, landing-page, and offer context so the next team does not treat a local result as a universal law.