FAQ    Incrementality     What is Incrementality Testing and How Do You Measure Incrementality?

What is Incrementality Testing and How Do You Measure Incrementality?

What is Incrementality Testing?

Incrementality testing creates an experiment that systematically withholds media channel exposure to a representative subset of users (the control group) while maintaining normal media channel exposure to the broader user set (the test group). If the control group is both sizable enough to be statistically significant and selected at random such that they are broadly representative of the user base, then the media channel’s incremental contribution can be determined by the difference in business outcome (conversion, revenue, profitability, etc.) between the test and control groups.

Incrementality testing is sometimes referred to as Incremental Sales Lift, Uplift Modeling, or A/B testing.

Difference between Incrementality and Lift

  1. “Incrementality” indicates the portion of last touch conversions that were truly driven by that Media.
  2. “Lift” indicates the increase to the overall business that these incremental conversions represent.

Difference between Incrementality vs Lift

 

How Do You Calculate Incrementality?

In general, the control group should represent a minimum of 10 percent of the total test and control reach. The test group will receive your ad, media in question; the control group will not receive the ad message. There are multiple ways to build a control audience. Some examples are:

  1. Flight a placebo ad or PSA ad (Public Service Announcement) that is irrelevant to your business ad to an exact replica or mirror audience of the exposed audience.
  2. Hold out a subset of the target audience and suppress from the ad exposure.
  3. Programmatic executions have counterfactual bid loss data for the target audience that did not win the media auction.

In order to calculate incrementality, we want to calculate the conversion difference between the test group versus the control.

Incrementality Calculation Formula

(Test Conversion – Control Conversion) / (Test Conversion) = Incrementality

Incrementality Test Calculation Example

Your test group saw 1.5% conversion whereas your control group saw a 0.5% conversion. The control data suggests that, without any media exposure, you would have seen a 0.5% conversion. Keep in mind that conversion could mean leads, sales, profit or whatever metric that is important to your business.

So (1.5% – 0.5%) / 1.5% = 66.7% incrementality in conversions

Incrementality is volatile and it is important to look at it longitudinally over time, within business context. Statistical significance has two primary variables: difference in conversion rates (%CR) and sample sizes. The larger the difference in conversion rates, the less sample you need. As a general rule, we like to see a control reach > 10% of the total test and control reach.

What Techniques Measure Incrementality of Marketing Channels?

Incremental sales driven by a media tactic are calculated using advanced marketing measurement techniques. There are three major types of advanced marketing measurement techniques.

Why is Incrementality Testing Needed?

Advertising where there are a lot of impressions but not immediate action like a click has brand value that ultimately leads to conversions or sales. Channels such as online display, video/YouTube, Facebook, TV, direct mail all provide impressions but it’s not always measured at the click or direct response level. You can read more about the following channels here:

Measured Incrementality Testing

Measured provides incrementality measurement and testing with ease and speed. We are already plugged into 100+ media platforms which allows us to run 100s of audience-level testing with quick set up. Find out your incrementality across your media channels.

Learn more here

Author

Trevor Testwuide - CEO

Expert in business strategy and marketing measurement.

 

Multi-touch attribution is more challenging today due to limited tracking options, identity and cross-device resolution hurdles, data leakage and the massive amount of time it takes to implement.

 

What is cross-platform attribution (or cross-channel attribution) and why is it difficult?

The goal of cross-platform attribution in marketing is to gain clarity on the interplay and contribution of influence that each channel/tactic/campaign has on driving conversions over and above baseline sales.

It’s a task that has proved to be very difficult for many reasons including but not limited to:

  • Walled gardens are typically inaccessible to third-party tracking of impressions
  • Identity resolution across media platforms is quite low
  • Cross-device tracking is difficult and match rates are extremely low
  • Instrumenting a tracking infrastructure by a third party measurement provider has proved to be fraught with breakage and data leakage
  • It is extremely time consuming to implement without the help of a partner

Video: Landing a source of truth cross-channel media reporting dashboard

 

 

What are some cross-channel attribution tools?

MTA – collects individual, or user-level data, for trackable addressable media and conversion events in order to determine the impact of each media event to the desired conversion at the customer level. By summing the impact of each addressable media touchpoint on each customers’ likelihood to convert, MTA quantifies the total media channel lift provided by addressable media. MTA does not account for the impact of non-addressable media, and furthermore much addressable media is either non-trackable or lost due to the innumerable challenges of tracking data at the user level.

Incrementality Measurement – Incrementality in marketing refers to the incremental benefit produced per unit of input stimulation. Incrementality is the lift in desired outcome (awareness, web visits, conversion, subscriptions, revenue, profitability) provided by marketing activity.

Incrementality in marketing is especially needed for channels where ad impressions such as display, Facebook, social, or even TV are hard to measure. To measure incrementality, the audience is broken out into test groups (exposed to the ads) and a control group (suppressed from seeing the ads).

MMM – MMM is a top down (aggregate marketing data) and very artistic statistical exercise where one or more models (e.g. econometric, multi-linear regression) are leveraged to extract key information and insights by deriving information from multiple sources of marketing, economic, weather and financial data. MMM is also a high-touch consultative approach that is very manual with little to no automated data inputs, whereas MTA and Incrementality, when deployed properly, is a very automated approach leveraging preconfigured connectors that extract the required marketing data, across many channels, on regular cadence. (It’s important to note that MTA can take 6 months or more to deploy, whereas Incrementality can be up and running with reporting in 4-6 weeks.) See this article for more on why always-on automated experimentation is the future of marketing measurement.

Author

Trevor Testwuide - CEO

Expert in business strategy and marketing measurement.

 

Multi-touch attribution is more challenging today due to limited tracking options, identity and cross-device resolution hurdles, data leakage and the massive amount of time it takes to implement.