The evidence on AI productivity is genuinely strong: in controlled studies, professionals using generative AI completed work around 40 per cent faster with measurably higher quality. The same body of research also contains a failure mode almost nobody quotes: on the wrong tasks, AI users performed 19 percentage points worse than colleagues working without it.

Both findings are real. Both come from rigorous experiments. And the gap between them is where marketing teams will win or lose with AI, because the variable that decides which result you get is not the tool. It is the judgement of the person directing it.

Here is what the research shows, where its boundaries sit, and the decisions it should change for a marketing team.

The Productivity Evidence Is Real

Three studies anchor the case. Shakked Noy and Whitney Zhang, in an experiment published in Science, gave professionals realistic writing tasks: those using generative AI finished 40 per cent faster, and the quality of their output improved by 18 per cent. A large field experiment involving 758 Boston Consulting Group consultants found AI users completed more tasks, worked faster and produced higher-quality results on tasks within the technology's capabilities. And a controlled experiment with an AI coding assistant found developers completed a programming task 55 per cent faster than a control group.

For marketing work, the translation is direct. Much of the job is professional writing, analysis, synthesis and production: exactly the task categories where these gains showed up. A team that dismisses the productivity evidence is arguing with well-run experiments, not with hype.

The Jagged Frontier: Where the Same Tool Makes You Worse

The BCG experiment contained a second condition that deserves equal billing. The researchers deliberately included a task designed to sit just outside the technology's capabilities, where the AI would produce convincing but wrong analysis. On that task, consultants using AI were 19 percentage points less likely to reach the correct answer than consultants working unaided.

The researchers called this boundary the “jagged technological frontier”: AI capability does not fail gradually or predictably, it falls off a cliff in places that are hard to see in advance, precisely because the output remains fluent and confident on both sides of the line.

For marketers the implication is uncomfortable but clarifying. The risk isn't that AI produces obviously bad work. It's that it produces plausible work that is wrong in ways only domain judgement can catch: a persuasive strategy built on a misread market, a confident analysis of data that doesn't support the conclusion. AI amplifies whatever it is pointed at, including mistakes.

The Decision the Evidence Should Change

Most teams respond to the productivity evidence with one question: how do we produce more? If a marketer can make five assets in the time one used to take, the organisation asks for five.

That captures the efficiency gain and improves nothing, because volume was rarely the constraint. The evidence supports a better trade: use the recovered hours on the work that execution crowded out. Talking to customers. Studying the category. Sharpening positioning. Diagnosing the actual barrier to growth before optimising anything. These are the tasks that determine whether the faster execution is aimed at anything worth executing.

There is a second decision hiding in the jagged-frontier finding: match tasks to the frontier deliberately. Delegate freely inside it (drafting, summarising, variation, first-pass analysis), and keep human judgement mandatory at the boundary (strategy, interpretation, anything where a wrong answer looks like a right one). Teams that write this down as an explicit working agreement get the 40 per cent without paying the 19 points.

How to Apply This Without the Hype

The honest reading of the evidence is neither “AI will transform everything” nor “AI is overrated”. It is conditional: substantial, measurable gains on suitable tasks, real degradation on unsuitable ones, and the difference decided by the judgement of the person in charge. Which means the highest-return AI investment for a marketing team is not another tool. It is building the marketing fundamentals that let people tell a good output from a fluent one. Evidence informs. Judgement decides.

Ready to Build AI on Top of Judgement?

FP Collectiv's Marketing and AI track teaches exactly this sequence: fundamentals first, then AI as a deliberate layer that improves decisions rather than accelerating noise. Start with Marketing and AI Foundations.

Sources

• Shakked Noy and Whitney Zhang, “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence”, Science, 2023: 40 per cent speed and 18 per cent quality findings.
• Fabrizio Dell'Acqua et al., “Navigating the Jagged Technological Frontier”, Harvard Business School working paper with Boston Consulting Group, 2023: 758-consultant field experiment, including the 19 percentage point degradation outside the frontier.
• GitHub, controlled experiment on AI coding assistance and task-completion speed (55 per cent finding).