Introducing Instant Predictions
Why Instant Predictions?
Of all the questions performance marketers ask themselves, “Which creative should I run next?” is a big one. In the past the answer has been “Let’s try a few and see” or “Run whatever we have” or “Use the same that worked last month.” Nobody has been able to truly, deeply analyse which creative ads have worked for them in the past and use that information to inform what they run next - finding more winning ads and improving results in the process.
This is painful to admit, because every performance marketing team is sitting on a huge pile of hard-won historical ad results data… failed experiments, triumphant ideas and in some cases many millions of dollars of ad spend. And yet, every week means guesswork and testing new creative from what is essentially a scratch start.
This all changes today. With the release of Instant Predictions, Pencil users are now able to leverage predictions powered by their entire Facebook Ads dataset in 1 click. No manual or automated tagging is required. Coupled with the ability to AI-generate new ad ideas, this means a few amazing things for Pencil users:
- Connect Pencil to your Facebook Ads account in one click
- Automatically analyze all the ads you’ve ever run
- AI-generate new creative ideas from your existing assets
- Choose an objective & targeting that you have data for
- Predict which new ad ideas to run next based on 3 prediction categories
Just how deeply can Instant Predictions understand historic ads?
Deeply. It’s very good. Let’s take a look at an example by “quizzing” the deep learning model to see if it understands different conceptual aspects of a single scene from a single ad. We do this by providing the model with an image and 3 tag options in different categories. It responds with different weights on the tags depending on its understanding of the image.
The more conceptual features the model understands, the better it is able to find patterns that link ad creative to positive results. We’re showing just 6 features here, but it’s important to understand that the model has been trained over 400 million+ datapoints and can understand a huge range of concepts and features in each ad - including those invisible to the human eye such as the spacing items on the table and other factors that can’t even be expressed in words. Any feature, no matter how microscopic, may have some impact on performance.
How accurately can Instant Predictions convert understanding into a results prediction?
Here’s a Precision-Recall curve for a model trained to predict the likely performance of a newly generated ad idea given a historic dataset of creative (for which concepts have been understood) and results. This is done with respect to a specific objective, targeting and metric.
It’s for one brand (we train a model for each brand on-the-fly) comprising about ~120 ads in a Facebook Ads account together with CTR results. Similar outcomes are possible with other metrics including ROAS, CPA, CPM, CPC and even brand lift measures in sufficient quantities.
You can see the model achieves about 80% precision at about 80% recall. In layman’s terms this is sufficient to be functionally predictive, and will only improve with more data!
This tells us that the model is able to identify potential ad “winners” for a given objective, targeting and metric before they are run. This is especially powerful when you can generate an unlimited number of new ad ideas to find the biggest “winners” before starting to spend your advertising budget.
What could this look like in terms of real brand outcomes?
Older prediction, “creative insight” or “creative optimization” technologies understood ad creative by manually or automatically “tagging” ads with keywords and meta-data such as the presence of a logo, product image, or offer text.
We’ve already achieved good results even with this older approach. Over November 2020 to July 2021 we showed that almost every brand who ran at least 10 Pencil-generated ads could find “winners” that achieved ~100% ROAS lifts.
However, this system (and other systems like it in the market) are limited in 2 big ways that have now been improved by orders of magnitude:
- Small dataset. It could only analyze tagged ad ideas, so the dataset for prediction was restrictively small
- Low sensitivity. It could only learn based on simple keywords and metadata, and was not sensitive to the universe of semantic meaning & concept within the creative
- Bias. It could only learn based on the tags provided to it - which were often provided by humans - rather than the universe of meaning observable in the dataset
One final important point to make is that predictions are only as good as your ability to act on them. Sending predictions back to a creative team as a critique on their work is likely to be quite annoying. Pencil users can instantly generate new ad ideas based on predictions, making this capability instantly actionable and easy to implement in real world organizations. Imagine what you might learn about your brand and customers, and the results you might achieve, by finally putting all your historical results data to good use.
