FAQ    Channel Incrementality     Can I Measure Incrementality on Facebook?

Can I Measure Incrementality on Facebook?

Yes, the incremental impact of Facebook advertising on business outcomes like conversions, revenue and profit can be measured.

There are 4 possible advanced marketing measurement techniques that can be used:

  1. Design of Experiments (DoE): Carefully designed experiments that control for targeted audiences, audience overlap, facebook campaign structures, optimization algorithms etc., are the most transparent, granular and accessible way to measure impact of Facebook campaigns on business metrics like sales and revenue. Typically in a Facebook DoE a small portion of audience being targeted is held out for control treatment, while the rest of the campaign audiences are treated to regular campaign creative and campaign treatment. The differences in response rates between the two treatments is analyzed to calculate incrementality.
  2. Counterfactual studies: This approach is an alternate method for measuring incrementality by using an natural experimentation approach (vs designed experimentation approach like #1) to measure incrementality. This approach uses data generated by auction systems within Facebook to observe audiences who “match” the audiences targeted for a campaign, but were not targeted because of various factors (eg: budget, ad auction competitiveness, etc.) and treat them as though they were deliberately held-out for a control treatment. The campaign exposed audiences and the synthetically constructed hold-out audiences are then analyzed for their response rates to calculate incrementality.
  3. Marketing Mix Models (MMM): This approach uses aggregate data rolled up at a week or month-level into a time series which is then fed to a regression model for estimating the impact of Facebook on business metrics. Because of the nature of the approach, results tend to be very macro in nature, providing an average impact of Facebook investments over a quarter. It is not very useful in breaking down the impact estimation by Campaign or tactic so it’s less appropriate for short-term tactical planning. Also in practice, these models take a while to build and stabilize, which could mean 6-12 weeks of lag from end of a quarter to results reporting.
  4. Multi-touch Attribution (MTA): This approach uses user-level data collected via pixels on all ad exposures to construct consumer journeys which are then fed to a machine learning algorithm to decompose the impact of each ad exposure in driving business results. The strength of this approach is extreme granularity of the reporting and the insight into customer journeys. More recently with the advent of privacy regulation and Facebook outlawing user-level 3rd party tracking, the collection of this kind of data has been nearly eliminated except in very special cases. Even when this data was being collected, the measurement would only be correlational out-of-the-box.

For tactical and timely measurement therefore, DoEs and Counterfactual studies are the primary approaches preferred by marketers, especially performance marketers. In many cases marketers use both to get multiple reads and perspectives on the impact of a Facebook advertising investment on their business.

DoE – Pros & Cons:

DoE is typically executed by either the brand or by a 3rd party vendor like Measured. DoEs can be designed to be very granular and shaped to meet learning diverse objectives for marketers. It can be executed independent of Facebook’s account teams, and hence offers the highest levels of control and transparency in executing experiments that match marketers’ learning objectives. Since all of the observations are captured through normal campaign reporting methods, leaving marketers to make inferences about campaign performance without any opaqueness to the methods of data collection. It’s strengths therefore lie in being fully transparent and neutral, while preserving granularity of measurement (eg: ad set-level, audience-level measurements). However, to implement the control treatment, marketers have to serve up a PSA (public service announcement) ad from their Facebook account, which can cost ~1% of their total Facebook budgets. The PSA costs can be minimized by narrowing the learning objectives and focusing on testing the most important audiences.

Counterfactual studies – Pros & Cons:

Counterfactual studies are typically conducted by the platform themselves, in this case Facebook. This feature within Facebook is called Lift testing. The ad delivery systems within Facebook implement a version of what’s called the ghost ads framework to collect data about audiences who matched a campaign criteria but were not served an ad because of other constraints, like budgets and competitive bids, in the auction. These audiences are then synthesized into a control audience whose performance is reported alongside the audiences who were exposed to campaign creatives. This allows marketers to read the lift of a campaign without actually selecting control audiences and executing a control treatment.

The primary advantages of using counterfactual studies is that marketers don’t have to spend any PSA dollars to implement the control treatment (since control audiences are synthesized by Facebook). However, it introduces blind spots into the audience selection and data reporting process, which can be swayed easily by oversampling or undersampling of audiences within an ad set or campaign or date parts or other factors.

The lift reporting within Facebook is also not granular. It does not allow marketers to look at the reporting by week / month, or constituent ad sets and audiences within a campaign (unless it was set up deliberately to be read that way).

For many marketers though, the primary objection of Facebook counterfactual studies is neutrality: having Facebook grade its own homework.

Video: Facebook Incrementality Measurement

Author

Madan Bharadwaj - Cofounder & CTO

Expert in advertising measurement, attribution and analytics

 

Multi-touch attribution is more challenging today due to limited tracking options, identity and cross-device resolution hurdles, data leakage and the massive amount of time it takes to implement.

 

What is cross-platform attribution (or cross-channel attribution) and why is it difficult?

The goal of cross-platform attribution in marketing is to gain clarity on the interplay and contribution of influence that each channel/tactic/campaign has on driving conversions over and above baseline sales.

It’s a task that has proved to be very difficult for many reasons including but not limited to:

  • Walled gardens are typically inaccessible to third-party tracking of impressions
  • Identity resolution across media platforms is quite low
  • Cross-device tracking is difficult and match rates are extremely low
  • Instrumenting a tracking infrastructure by a third party measurement provider has proved to be fraught with breakage and data leakage
  • It is extremely time consuming to implement without the help of a partner

Video: Landing a source of truth cross-channel media reporting dashboard

 

 

What are some cross-channel attribution tools?

MTA – collects individual, or user-level data, for trackable addressable media and conversion events in order to determine the impact of each media event to the desired conversion at the customer level. By summing the impact of each addressable media touchpoint on each customers’ likelihood to convert, MTA quantifies the total media channel lift provided by addressable media. MTA does not account for the impact of non-addressable media, and furthermore much addressable media is either non-trackable or lost due to the innumerable challenges of tracking data at the user level.

Incrementality Measurement – Incrementality in marketing refers to the incremental benefit produced per unit of input stimulation. Incrementality is the lift in desired outcome (awareness, web visits, conversion, subscriptions, revenue, profitability) provided by marketing activity.

Incrementality in marketing is especially needed for channels where ad impressions such as display, Facebook, social, or even TV are hard to measure. To measure incrementality, the audience is broken out into test groups (exposed to the ads) and a control group (suppressed from seeing the ads).

MMM – MMM is a top down (aggregate marketing data) and very artistic statistical exercise where one or more models (e.g. econometric, multi-linear regression) are leveraged to extract key information and insights by deriving information from multiple sources of marketing, economic, weather and financial data. MMM is also a high-touch consultative approach that is very manual with little to no automated data inputs, whereas MTA and Incrementality, when deployed properly, is a very automated approach leveraging preconfigured connectors that extract the required marketing data, across many channels, on regular cadence. (It’s important to note that MTA can take 6 months or more to deploy, whereas Incrementality can be up and running with reporting in 4-6 weeks.) See this article for more on why always-on automated experimentation is the future of marketing measurement.

Author

Trevor Testwuide - CEO

Expert in business strategy and marketing measurement.

 

Multi-touch attribution is more challenging today due to limited tracking options, identity and cross-device resolution hurdles, data leakage and the massive amount of time it takes to implement.