Some sales credited to ads would have happened anyway. A loyal customer clicks a retargeting ad on the way to buying. Attribution credits the ad; incrementality asks what would have happened without it.
What Incrementality Means
Incremental sales are the sales your ads caused that wouldn't have happened otherwise. Incrementality testing estimates them by comparing a group exposed to ads with a similar group that wasn't.
What this means for you: if retargeting shows a 10x ROAS but a test reveals most of those buyers would have purchased anyway, its true incremental return is far lower.
Types of Incrementality Tests
| Method | How It Works | Best For |
|---|---|---|
| Holdout test | Exclude a random audience segment from ads | Retargeting and email |
| Geo test | Turn ads on or off in matched regions | Channels without user-level control |
| Conversion lift study | Platform-run randomized test | Meta, Google and TikTok campaigns |
| Spend on/off test | Pause a channel and watch total revenue | Quick directional checks |
How to Run a Simple Holdout Test
- Pick one question, such as "does retargeting drive incremental sales?"
- Split an audience randomly into test and holdout groups.
- Run ads only to the test group for a set period.
- Compare conversion rates between groups using your real sales data.
- Calculate lift: (test rate − holdout rate) ÷ holdout rate.
Example (illustrative): test group converts at 3.0%, holdout at 2.4%. Lift = 25%. Only that extra 0.6 points is incremental.
Incrementality vs Attribution
They complement each other. Attribution runs continuously and guides daily optimization at the ad level. Incrementality tests are periodic and answer bigger questions, like how much retargeting or a new channel truly adds.
Common Pitfalls
- Test periods too short to reach significance.
- Groups that aren't truly comparable.
- Changing creative or budgets mid-test.
- Measuring with platform-reported conversions instead of real sales.
Incrementality Testing FAQs
What is incrementality in marketing?
The share of conversions caused by marketing that wouldn't have happened without it.
Do you need a big budget?
Larger budgets reach reliable results faster; small tests can still be directional.
Does attribution measure incrementality?
No. Attribution assigns credit; experiments test causation.
Which channels to test first?
Retargeting and branded search, which often look strongest in attribution.
Make Better Daily Decisions Between Tests
Book a Hyros demo to see accurate ad-level revenue across your platforms.
Book a Hyros Demo