What is OTT and How Do You Measure OTT Advertising?

OTT, or Over the Top, is the term for video content that is delivered via connected television devices (think Roku, Amazon Fire TV, Apple TV, etc.) over a high-speed broadband internet connection. OTT is a very appealing substitute for satellite or cable television because it can be more affordable and offers consumers a wider variety of on-demand videos, movies, and television shows on the device of their choice. 

Why is OTT good for marketing?

For many years, advertisers have struggled to measure the effectiveness of their television ads. With traditional, linear television, it’s impossible to track who is watching what and when so measuring the performance of advertising is difficult.

The advent of OTT (Over the Top) has finally given marketers line of sight to user-level measurement of ad effectiveness and access to very granular log-level data. Log level data includes attributes such as DMA, time of impression, device characteristics, etc.

OTT can be a very effective and measurable channel for building brand awareness and acquiring new customers through a wide variety of ad inventory.

How do you measure OTT advertising?

There are several methods that can be deployed to measure the effectiveness of incremental contribution of OTT advertising, including audience split tests and geo-matched-market tests.

Audience Split Tests:

Audience split experiments test performance by segmenting the audience into randomly selected, statistically significant test and control cohorts. The test group is served the ad treatment while the control group receives a placebo ad, like a public service announcement or charity ad. The conversion rates for both groups are observed over a tracking window (typically 30 days) and the difference in conversion rate for each group informs incrementality of the campaign. 

An alternative to running a placebo treatment is a counterfactual or ghost ads approach where the ad delivery system within the OTT platform identifies audience groups that matched campaign criteria but weren’t served the ad. By using this group as the “control” and comparing performance alongside the audiences who were exposed to the ads, marketers can read the lift of the campaign without actually selecting control audiences or executing a control treatment.

Geo Matched Market Tests:

A matched market test is the preferred method when a clean audience split test is not available. A handful of small markets are identified as representative of a larger market (ex. California, New York). In these “test markets,” the desired treatments are applied for each of the OTT providers in the test such as going dark in one or two media tactics for a set period of time. The results from the test are observed and the difference in performance metrics (like conversion rates, revenue per user) between the test and control are then interpreted to inform incrementality.

Flighting unbiased, uncontaminated controls is critical to successful experimentation, and every platform has different nuances. Experts at Measured have spent years perfecting complex, statistically sound experiments, tailored for the idiosyncrasies of each channel and platform. The turnkey, automated solutions can be configured to your learning agenda to answer specific questions about your media performance.

Learn more about the Measured Intelligence Suite for incrementality measurement.