FAQ    Channel Incrementality     What is OTT and How Do You Measure OTT Advertising?

What is OTT and How Do You Measure OTT Advertising?

What is OTT Measurement?

For many years, advertisers have attempted to measure the effectiveness of their television ads, which is much easier said than done. MMM (media mix modeling) or “TV attribution” (analysis done by some MTA providers and agencies) are the primary methods marketers have used over the last few decades when attempting to measure the effectiveness of ads on traditional linear television.

The advent of OTT (Over the Top) has finally given marketers line of sight to user level measurement of ad effectiveness and access to very granular log level data. Log level data includes attributes such as: DMA, time of impression, device characteristics, etc.

What is OTT?

OTT, or Over the Top, is the term for video content that is delivered via connected television devices (think Roku, Amazon Fire TV, Apple TV, etc.) over a high speed broadband internet connection. OTT is a very appealing substitute for satellite or cable television because it can be more affordable and offer a much wider array of content offerings and personalization options.

There are many advantages for consumers as they can access a wide variety of videos, movies and television shows on demand, and on the device of their choice. All you need is a high speed internet connection and a smart TV.

The key advantage to a marketer is that OTT can be a very effective and measurable channel for building brand awareness and acquiring new customers through a wide variety of ad inventory.

How to Measure OTT?

There are several tactics that can be deployed to measure the effectiveness or incremental contribution of OTT advertising. They’re broken up into two categories: audience split tests and matched market tests.

Audience Split Tests:

    • Counterfactual (Ghost Ads): In the ghost ads approach, the ad delivery systems within the OTT provider’s ad platform implements a version of what’s called the ghost ads framework to collect data about audiences who matched a campaign criteria but were not served an ad because of other constraints, like budgets and competitive bids, in the auction. These audiences are then placed into a control audience whose performance is reported alongside the audiences who were exposed to ads within a campaign. This allows marketers to read the lift of a campaign without actually selecting control audiences and executing a control treatment.
  • PSA Ads (Public Service Announcement or charity ad): In the PSA approach the control group receives a placebo ad, which in this case is a charity or public service announcement. The goal is to not stimulate the control group with a brand message, but to present the placebo ad one time for tracking purposes. The conversion rates for the test group (exposed to brand ad) and the control group (not exposed) are observed over a tracking window (typically 30 days) and the difference in conversion rate for each group informs incrementality.

Matched Market Tests:

  • A matched market test is the preferred method when a clean audience split test is not available. A handful of small markets are identified as representative of a large market (ex. California, New York). In these “test markets” the desired treatments are applied for each of the OTT providers in the test such as going dark in one or two media tactics for a set period of time. The results from the test are observed and the difference in performance metrics (like conversion rates, revenue per user) between the test and control are then interpreted to inform incrementality.


James Bance - Director of Growth


Multi-touch attribution is more challenging today due to limited tracking options, identity and cross-device resolution hurdles, data leakage and the massive amount of time it takes to implement.


What is cross-platform attribution (or cross-channel attribution) and why is it difficult?

The goal of cross-platform attribution in marketing is to gain clarity on the interplay and contribution of influence that each channel/tactic/campaign has on driving conversions over and above baseline sales.

It’s a task that has proved to be very difficult for many reasons including but not limited to:

  • Walled gardens are typically inaccessible to third-party tracking of impressions
  • Identity resolution across media platforms is quite low
  • Cross-device tracking is difficult and match rates are extremely low
  • Instrumenting a tracking infrastructure by a third party measurement provider has proved to be fraught with breakage and data leakage
  • It is extremely time consuming to implement without the help of a partner

Video: Landing a source of truth cross-channel media reporting dashboard



What are some cross-channel attribution tools?

MTA – collects individual, or user-level data, for trackable addressable media and conversion events in order to determine the impact of each media event to the desired conversion at the customer level. By summing the impact of each addressable media touchpoint on each customers’ likelihood to convert, MTA quantifies the total media channel lift provided by addressable media. MTA does not account for the impact of non-addressable media, and furthermore much addressable media is either non-trackable or lost due to the innumerable challenges of tracking data at the user level.

Incrementality Measurement – Incrementality in marketing refers to the incremental benefit produced per unit of input stimulation. Incrementality is the lift in desired outcome (awareness, web visits, conversion, subscriptions, revenue, profitability) provided by marketing activity.

Incrementality in marketing is especially needed for channels where ad impressions such as display, Facebook, social, or even TV are hard to measure. To measure incrementality, the audience is broken out into test groups (exposed to the ads) and a control group (suppressed from seeing the ads).

MMM – MMM is a top down (aggregate marketing data) and very artistic statistical exercise where one or more models (e.g. econometric, multi-linear regression) are leveraged to extract key information and insights by deriving information from multiple sources of marketing, economic, weather and financial data. MMM is also a high-touch consultative approach that is very manual with little to no automated data inputs, whereas MTA and Incrementality, when deployed properly, is a very automated approach leveraging preconfigured connectors that extract the required marketing data, across many channels, on regular cadence. (It’s important to note that MTA can take 6 months or more to deploy, whereas Incrementality can be up and running with reporting in 4-6 weeks.) See this article for more on why always-on automated experimentation is the future of marketing measurement.


Trevor Testwuide - CEO

Expert in business strategy and marketing measurement.


Multi-touch attribution is more challenging today due to limited tracking options, identity and cross-device resolution hurdles, data leakage and the massive amount of time it takes to implement.