Prema Sampath, group product manager at Google, leads a product team that develops measurement applications for brand advertising, including privacy-centric, third-party solutions for YouTube ads. Stephen Mangan is an ROI measurement manager at Google, leading research and measurement partner activations to help advertisers improve their media optimization strategies.
Marketing mix models (MMM) help marketers make apples-to-apples comparisons across all of their different investments. They provide answers to questions like: What drove sales? What was my ROI? How do I optimize my marketing investment?
But determining ROI takes more than a single calculation. Today’s media landscape is only becoming more fragmented and intertwined, presenting MMMs with an unprecedented challenge when predicting future performance.
Incorporating business context to shape the MMM is an art, one with implications for the model’s outcomes and final recommendations.
To make sense of the MMM, advertisers often defer to their measurement providers’ technical expertise. But there is more to the measurement strategy than the science behind the model. Incorporating business context to shape the MMM is an art, one with implications for the model’s outcomes and final recommendations. Advertisers who embrace that art can empower their businesses to make more strategic measurement-based decisions.
Here are three steps marketers can take to embrace the art of the MMM.
Start with granularity
Remember that impressions are not equal across platforms or even within them. When it comes to video-specific measurement, MMMs will evaluate all of your impressions, but platforms can vary widely in terms of watch time, viewability, and audibility. The same data can produce very different results depending on how it’s incorporated into the model. Research we commissioned from Nielsen shows that when CPG brands’ MMMs evaluated individual video platforms rather than aggregated data, return on ad spend (ROAS) varied by as much as 48%.1
Ad formats can vary just as widely. On YouTube, for example, ads can range from unskippable 30-second videos to a 6-second bumper ad. While both formats can deliver ROI, the cost and effectiveness of their impressions won’t be the same.
The same data can produce very different results depending on how it’s incorporated into the model.
So be sure to leverage the most granular data from your publishing partners, and your model will be able to identify the respective value of different impressions. As with any model, the MMM has limits. But every layer of granularity will lead to more informed business decisions.
Add business context
The science of the MMM can tell you what your ROI was, but it can’t tell you why without context. Getting granular, format-level data is a start, but formats are only one driver of ROI. According to Nielsen Catalina Solutions, creative accounts for 47% of video ROI — and yet, MMMs are not designed to evaluate individual creative assets.
Collaboration will help your measurement provider evaluate how these changes in strategy impact ROI. Done right, this can lead to more ROI over time.
"If modeling is done in a vacuum, devoid of context, it could lead to misleading and suboptimal decisions,” said Rajika Karunanayaka, senior manager of media sciences at Hershey. “So context is just as important as the data that goes into modeling and should be a key consideration every step of the way.” Hershey’s Integrated Media team works with IRI to get data from Google and contextualize changes in their media plans, she said.
Work with your publisher partner to gather insights around your media buy and identify any other data needed for the model to answer strategic business questions. Collaboration will help your measurement provider evaluate how these changes in strategy impact marketing ROI. Done right, this can lead to more ROI over time.
“We partner with Google to collect MMM data at the right level of granularity to enable actionable and insightful modeling, and then work with them to review the underlying executional elements that could potentially play into the results,” said Karunanayaka, citing changes to Hershey’s YouTube ad format mix, reach and frequency, audience strategy, and viewability as possible sites of impact. “We also work with our Google team to understand effectiveness at the creative level, while normalizing for ad format so we can get a sense of which Hershey YouTube ads are driving sales.”
In an MMM meta-analysis we commissioned from Nielsen, on average, YouTube ROAS grew by 108% for brands after collaborating2 with Google, a 7X greater increase compared to brands that did not collaborate with Google.3 Sharing your media narrative will allow the modelers to understand what influenced changes in marketing ROI, and will ultimately help you formulate smarter business strategies.
Create more transparency
No model is perfect, MMM included. A team of Google data scientists even published a paper on how MMMs using the same data could report drastically different ROI with the same level of statistical confidence.
With that in mind, assess results from a critical lens. Rather than accept all outputs, ask your measurement provider about the model’s margin of error for specific results and recommendations. Don’t base any important business decisions solely on your MMM as a single source of truth. Instead, take the time to design a test versus control experiment that can give you a second perspective on the model’s accuracy.
The art of enhancing the model is where marketers can step in to help gather data and context from publisher partners and push providers on transparency.
MMM is a valuable tool for measurement, but science alone cannot produce all the right answers. Nuance matters. The art of enhancing the model is where marketers can step in to help gather data and context from publisher partners and push providers on transparency. With these small steps, teams can influence the structure and enhance the accuracy of the MMM, leading to outcomes that enable more informed decision-making.
