Why Always-on Experimentation is the Future of Measurement

In the 10+ years I have been in marketing measurement, I have witnessed change at an alarming pace. Data and analytics have gone from a curiosity to a competitive advantage, to table stakes for marketing organizations. Digital advertising used to be just retargeting and paid search, dwarfed by what was spent on offline marketing. Now, channels such as paid social, online video, and programmatic display are major portions of the marketing mix. Data has evolved from fact-finding expeditions and wayward Excel sheets to cookies, global User IDs, identity, data management platforms, and customer data platforms. I could go on and on.

As far as marketers have come, they still face challenges every day as they execute make or break decisions that drive the bottom line for their businesses. Advances in data and measurement have yielded new challenges, and organizations are still looking for a competitive advantage, especially as more and more marketers crowd into digital acquisition and drive up the online cost per impression by the day. Amidst all this, I’ve come to feel very strongly about a few key trends in marketing measurement.

Multi-touch Attribution has not lived up to the hype.

We all had the best of intentions. We thought attribution via terabytes of user-level data, billions of cookies, and millions of converting and non-converting sequences would unlock the secrets of the customer journey. We thought MTA would facilitate a revolution in marketing decision-making and feed real-time decisions about the next best action at the user level. Some of this was always aspirational, but we’ve come to find out that user-level data is messy at best and significantly flawed at worst.

We expected a treasure trove of insight into the consumer journey, and what we found could be better described as a trail of breadcrumbs in a dark forest.

Even if you ignore just how important non-addressable factors are in the consumer decision-making process, it is incredibly difficult to accurately track users across multiple channels and devices over any reasonable amount of time. And that was before walled gardens like AdWords and Facebook, the black holes of user-level data, became the dominating players they are today. Facebook, Instagram, YouTube, and GDN represent over 80% of non-search digital budgets and those impressions can’t be tracked at the user level. That leaves paid search, and every savvy attribution veteran knows you don’t need MTA to optimize search. The reality is, long before Cambridge Analytica and GDPR, the deck was stacked against user-level MTA.

Now GDPR and privacy laws are driving up the costs of deploying MTA, while the growth of walled gardens is shrinking the budgets it can inform. Here is the current reality of MTA:

• Deployments take 6-12 months.
• Insights provided are only for addressable channels.
• Measurement is limited to clickstream and impression data trackable by third-party cookies (which are on their way out).
• Walled gardens (FB, YouTube, GDN, Instagram, etc.) are off-limits.
• Best-in-class solutions rely upon a DMP, a non-DCM ad server, and an identity provider.

Most of the market has opted to either develop a rules-based approach or simply gulp hard and swallow last-touch metrics no matter how imperfect they may be.

Marketing Mix Modeling is a mature product that hasn’t moved down market.

Practically every enterprise marketer with a $200 million marketing budget either has an MMM vendor or has built an in-house solution. Its value as a strategic tool is rooted in the fact that it is data-driven, encompasses marketing and non-marketing data sources, and informs key strategic marketing decisions. For annual and quarterly marketing planning, it is still a best-in-class approach.

But the value add of an MMM program is driven by several factors, including the size of non-addressable marketing budgets, the number of offline transactions, and the availability of large steady-state multi-year historical data sets. On top of that, much of the value of vendor-provided MMM is driven by consulting services, something that is beyond what many marketing organizations want to pay.

These factors explain why MMM has lower adoption rates outside of the Fortune 1000. Its value is diminished for marketers in rapidly evolving markets with digital-heavy budgets, that are experiencing significant business growth, targeting online conversions, and need agile tactical decision-making.

Incrementality testing is not a nice-to-have, it’s a need-to-have.

Digital natives are on to something. Every marketing team engages in A/B testing; it is ubiquitous in the world of digital marketing. Vendor platforms support split testing, website design wouldn’t exist without it, and many channel managers would tell you that it’s crucial to decision-making. But the quality of testing varies wildly, from ad hoc to dedicated programs supported by large teams of experienced data scientists. Uber and Spotify have spent millions building this capability in-house as their $200M+ digital media budgets have necessitated it. They have entire teams focused exclusively on executing these tests to inform the incrementality of marketing programs down to the campaign level. Meanwhile many digital marketers play amateur data scientists with vendor-provided split testing tools to compare creatives, answer one-off questions from execs, or generally conclude they should spend more on digital acquisition (YES!).

In order to effectively execute digital media, optimize campaign allocation, and scale acquisition, marketers should be running a host of expertly designed incrementality tests in an always-on mode across digital media channels. However, the hard truth is that most marketing organizations don’t have the budgets or the wherewithal to justify developing a best-in-class capability in-house.

Enter the Measured incrementality platform.

Ad-hoc split testing isn’t enough to create a meaningful competitive advantage or to develop a winning customer acquisition strategy. Marketers need an always-on design of experiments that operates across all prospecting and retargeting tactics measuring incrementality and informing decision-making. By partnering with Measured ALL digital marketing teams can build and maintain a comprehensive test-based attribution program.

Our team is built of people who have spent the last decade in marketing attribution and measurement solving these problems for brands. Our product deploys a comprehensive design of experiments that measures incrementality at the ad set, audience, campaign, and program level across marketing channels including Facebook, AdWords, retargeting, and catalog/direct mail.  Unlike MTA, our product is not dependent on third-party cookies and not limited by the myriad problems created by user-level data. Our implementation is painless. In a matter of weeks, we deploy incrementality tests and provide actionable insights. We empower marketing teams with best-in-class experimentation that would take years to build internally. Together with our clients, we are changing the way marketers make decisions and I could not be more excited to be a part of it.