does machine generated creative work?

"Every single brand in the study that spent over $1k on MGC was able to discover an MGC ad that outperformed its ROAS baseline."
December 11, 2020 - Announcement
Written by Chase Mohseni

Machine Generated Creative (MGC) means ads that have been generated automatically by an algorithm to achieve goals set out by a human brief.

Pencil generates ads that seek to balance the three competing goals that every marketer has - creative quality, brand relevance and marketing results - and not necessarily in that order! 

As advertising becomes ever more digital and measurable, every ad must “work” and deliver results on a hard metric like return-on-ad-spend (ROAS). It’s all well and good to make a beautiful branded ad, but if it fails to resonate with customers and deliver revenue, it can be a huge waste of time, money and emotional energy. Any performance marketer will tell you that most ads they run fail.

MGC offers performance marketers a remarkable new way to achieve their ROAS goals by generating high quality ads quickly and at low cost. We think it’s such a no brainer that most ads will be made this way in the future. MGC is essentially guaranteed to work for 2 big reasons:

1. MGC takes the fundamental first principle of advertising effectiveness - testing - and vastly accelerates it. Testing always delivers better results. Facebook’s own meta-analysis of 163 campaigns in December 2019 found that testing-and-learning delivered 26% lower costs-per-action than not. Many brands simply miss this opportunity because it’s expensive and slow to create enough ads to do good testing. MGC makes it possible to test more simply because it’s so low cost and so fast.

2. MGC introduces machine-learning based predictions to the creative process.
This is the real game changer, and what the people come for. Algorithms can not only generate ads, but generate ads predicted to work based on what’s worked before.

Pencil’s algorithms analyse historic ads over 50+ different creative “features” and offer a prediction of whether they think each newly generated ad will work for a given goal, targeting and metric. Generated ads, even if completely novel, will receive a good prediction if they share features with ads that worked well in the past. 

In this report, we’ll share some initial ROAS findings of running MGC against “whatever creative the brand was already using” in their Facebook Ad Accounts. This study was conducted across 69 weekly campaign data points for 10 brands involved in our beta program between August and November 2020. Each brand’s weekly campaign spend varied from $1k to $100k or more.

There were 5 key findings of our meta-analysis:

  1. The best MGC ads outperformed their ROAS brand baselines by 52.9%. This is key, as it suggests that an MGC-based approach is able to discover winning ads that win to a significant extent. 
  2. Every single brand in the study that spent over $1k on MGC was able to discover an MGC ad that outperformed its ROAS baseline. This is in line with the first principle that more testing increases the likelihood of something working!
  3. 4 of 10 brands were able to discover MGC ads that outperformed their ROAS baselines by 80-120%. That’s a 2x improvement from ads that cost less and were faster to make. 
  4. Brands with higher spend saw better results from MGC. This makes sense. Higher spends allow brands to run more creative tests and accumulate better signal on those tests. That said, even brands spending only $1-10k on MGC tests were able to find MGC ads that outperformed their ROAS baselines.
  5. Early data shows MGC as beating ROAS baselines across 5 sectors, 3 different spend levels and as soon as the first wave of tests. The sectors were cosmetics, consumer goods, pharmaceuticals, fashion and food & beverage. The spend levels were $1-10k/week, $10-50k/week and $100-500k/week.

Download the full MGC ROAS Results 20Q4 report