If you have ever cut a channel that attributed badly and watched total pipeline fall anyway, or scaled a channel that attributed brilliantly and got nothing extra, you have already seen the gap between attribution and incrementality, and paid for it.

This guide covers what each method measures, how to run an incrementality test without a data science team, the branded-search trap that catches most B2B budgets, and how to use attribution, incrementality and marketing mix modelling together. The evidence for why attribution misleads in the first place is covered separately in why marketing attribution misleads: what the experiments show.

Attribution vs Incrementality vs Mix Modelling: What Each One Measures

Three measurement methods sit behind most B2B budget decisions, and each method answers a different question. Confusing them is where most bad reallocations start.

The gap between the attribution number and the incrementality number is largest for the channels that reach buyers just before they purchase. A buyer who searches your brand name, clicks the ad and buys hands that ad full credit under last-click attribution. But they searched for you by name. Most of that conversion was already yours, and the ad charged you for it. Retargeting has the same flaw: it shows ads to people who had already visited your site and were likely to return without the ad.

The Branded Search Trap

The best-known incrementality result in marketing comes from eBay. In 2015 Thomas Blake, Chris Nosko and Steven Tadelis published a large-scale field experiment in Econometrica in which eBay switched off paid search advertising on its own brand terms in a set of US markets. Sales in those markets did not fall. Almost all of the traffic that had been attributed to brand-term ads arrived anyway through eBay's free organic listing, the unpaid search result that sits directly below the ads on the results page.

The same experiment found non-brand search ads did produce incremental sales, but mostly among new and infrequent buyers, and at a return on investment that was negative for eBay overall. Under attribution, both campaigns had looked like the best-performing spend in the company.

eBay is not your business, and a smaller brand with weak organic rankings or an aggressive competitor bidding on its name may find branded search more incremental than eBay did. That is the point: the question has to be tested rather than assumed, and it is usually the cheapest incrementality test available, because the channel is small and the switch-off is easy to run.

How to Run an Incrementality Test

An incrementality test is a controlled comparison. You give one group the marketing and withhold it from a comparable group, then measure the difference in the outcome you care about. The design costs more thought than money. Three designs cover most B2B situations.

1. Geographic holdout

Split your markets into two groups with similar history: similar pipeline, similar seasonality, similar size. Run the channel in one group and pause it in the other for the test period. Compare pipeline or revenue between the two groups against how they compared before the test. This is the workhorse design for paid media and works whenever you can target by region.

2. Audience split

Randomly hold out a share of the target audience, typically 10 to 20 per cent, and show them nothing (or a public-service placeholder) while the rest see the campaign. Most advertising platforms support this natively. It works well for account-based programmes, where you can hold out a matched set of target accounts and compare engagement and pipeline between the two groups.

3. Clean pause

Switch a channel off entirely for four to six weeks and watch what happens to total conversions, not just the conversions attributed to that channel. This is the crudest design because there is no control group, so seasonality and other campaigns can confound it. It is also the easiest to run, and for a channel like branded search it is often enough to settle the question.

Four rules that make an incrementality test result trustworthy

Decide the metric before you start. Qualified pipeline is usually the right outcome for B2B. Clicks and form fills are attribution metrics in disguise.

Run the test long enough to cover a full buying cycle. If your average deal takes three months, a two-week test measures noise. Four to eight weeks is a practical minimum for most B2B channels, and longer for anything that works through brand.

Expect the result to be a range, such as "between 2 and 9 per cent more pipeline", not a single precise figure. Randall Lewis and Justin Rao's analysis of 25 large field experiments found that the effect of advertising is usually small compared with the normal week-to-week variation in sales, so even a large test can only narrow the answer to a range. A test that says "between zero and a modest lift" is still more useful than a dashboard that claims "300 per cent ROI" with false confidence.

Change nothing else during the test. A new campaign, a price change or a product launch during the test period makes the result unreadable. Put the test dates in the marketing calendar, tell sales and product, and refuse other launches in that window.

How Attribution and Incrementality Work Together

Attribution and incrementality are layered, not competing. Attribution runs every day and tells you whether the machinery is working: which creative is winning, where journeys break, whether a bid change moved anything. Incrementality tests run a few times a year and tell you whether a major channel deserves its budget at all. Marketing mix modelling, where you have two or three years of spend and sales history, answers the allocation question across all channels at once. It also has a practical advantage: it uses aggregate data rather than tracking individual users, so it keeps working as cookie restrictions and privacy rules remove more of the user-level data that attribution depends on.

The practical rhythm for a B2B team is one incrementality test per year on the biggest attributed line item, attribution for the weekly operating decisions, and a brand tracker so demand creation has evidence of its own. The budget call is then made with judgement across all three, and documented, so it can be defended in the room and revisited next year. How to size and present that budget is covered in how to set a B2B marketing budget.

The One Test to Run This Year

Pick the channel that looks best on attribution and design an incrementality test for it. Either result is a win. If the channel survives, it is a real performer and deserves more budget. If it does not, it was charging you for demand that already existed, and that money is now available for brand and demand creation work, which attribution under-reports because its effect arrives long before the tracked conversion.

Attribution and Incrementality FAQs

What is the difference between attribution and incrementality?

Attribution assigns credit for a conversion to the marketing touchpoints that preceded it. Incrementality measures how many conversions the marketing caused, by comparing a group of buyers who saw the marketing with a comparable group who did not. Attribution describes the journey; incrementality tests the effect.

Is branded search incremental?

Branded search ads usually produce far fewer extra sales than attribution reports credit them with. eBay's 2015 field experiment found brand-term ads produced almost no incremental sales because buyers searching the brand name clicked the organic listing instead. Brands with weak organic rankings or competitors bidding on their name may see more lift. Pause your branded search ads for four to six weeks and measure what happens to total sales, rather than assuming the answer.

How much budget do you need for incrementality testing?

Less money than most teams assume, because a test withholds spend rather than adding it. A geographic holdout or a four-to-six-week pause on one channel costs planning time, not extra media budget. The real cost is any pipeline the holdout group fails to produce while the marketing is switched off, and the size of that shortfall is the result the test exists to measure.

How long should an incrementality test run?

Long enough to cover a buying cycle for the outcome you are measuring. For most B2B channels that means four to eight weeks at minimum, and longer for brand-led activity whose effect arrives months later. A test shorter than the sales cycle measures noise.

Does incrementality replace attribution?

No. They do different jobs. Attribution runs daily for channel operations. Incrementality runs periodically to decide whether major channels deserve their budget. Marketing mix modelling covers the portfolio-level allocation. A defensible measurement approach uses all three and applies judgement across them.

Sources

• Thomas Blake, Chris Nosko and Steven Tadelis, Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment, Econometrica, 2015: eBay's geographic experiments on brand-term and non-brand paid search.
• Randall Lewis and Justin Rao, The Unfavorable Economics of Measuring the Returns to Advertising, Quarterly Journal of Economics, 2015: 25 large field experiments and the statistical difficulty of measuring advertising returns.
• Brett Gordon, Florian Zettelmeyer, Neha Bhargava and Dan Chapsky, A Comparison of Approaches to Advertising Measurement, Marketing Science, 2019: observational attribution methods compared against randomised experiments.
• Test durations, holdout sizes and the annual testing rhythm in this article are practitioner judgement, stated as such.