Robust tree-based causal inference for complex ad effectiveness analysis

Pengyuan Wang, Wei Sun, Dawei Yin, Jian Yang, Yi Chang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

As the online advertising industry has evolved into an age of diverse ad formats and delivery channels, users are exposed to complex ad treatments involving various ad characteristics. The diversity and generality of ad treatments call for accurate and causal measurement of ad effectiveness, i.e., how the ad treatment causes the changes in outcomes without the confounding effect by user characteristics. Various causal inference approaches have been proposed to measure the causal effect of ad treatments. However, most existing causal inference methods focus on univariate and binary treatment and are not well suited for complex ad treatments. Moreover, to be practical in the data-rich online environment, the measurement needs to be highly general and efficient, which is not addressed in conventional causal inference approaches. In this paper we propose a novel causal inference framework for assessing the impact of general advertising treatments. Our new framework enables analysis on uni- or multi-dimensional ad treatments, where each dimension (ad treatment factor) could be discrete or continuous. We prove that our approach is able to provide an unbiased estimation of the ad effectiveness by controlling the confounding effect of user characteristics. The framework is computationally efficient by employing a tree structure that specifies the relationship between user characteristics and the corresponding ad treatment. This tree-based framework is robust to model misspecification and highly flexible with minimal manual tuning. To demonstrate the efficacy of our approach, we apply it to two advertising campaigns. In the first campaign we evaluate the impact of different ad frequencies, and in the second one we consider the synthetic ad effectiveness across TV and online platforms. Our framework successfully provides the causal impact of ads with different frequencies in both campaigns. Moreover, it shows that the ad frequency usually has a treatment effect cap, which is usually over-estimated by naive estimation.

Original languageEnglish (US)
Title of host publicationWSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining
PublisherAssociation for Computing Machinery, Inc
Pages67-76
Number of pages10
ISBN (Electronic)9781450333177
DOIs
StatePublished - Jan 1 2015
Externally publishedYes
Event8th ACM International Conference on Web Search and Data Mining, WSDM 2015 - Shanghai, China
Duration: Jan 31 2015Feb 6 2015

Other

Other8th ACM International Conference on Web Search and Data Mining, WSDM 2015
CountryChina
CityShanghai
Period1/31/152/6/15

Fingerprint

Marketing
Tuning
Industry

Keywords

  • Advertising
  • Causal inference
  • Online strategy measurement

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Wang, P., Sun, W., Yin, D., Yang, J., & Chang, Y. (2015). Robust tree-based causal inference for complex ad effectiveness analysis. In WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining (pp. 67-76). Association for Computing Machinery, Inc. https://doi.org/10.1145/2684822.2685294

Robust tree-based causal inference for complex ad effectiveness analysis. / Wang, Pengyuan; Sun, Wei; Yin, Dawei; Yang, Jian; Chang, Yi.

WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 2015. p. 67-76.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, P, Sun, W, Yin, D, Yang, J & Chang, Y 2015, Robust tree-based causal inference for complex ad effectiveness analysis. in WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, pp. 67-76, 8th ACM International Conference on Web Search and Data Mining, WSDM 2015, Shanghai, China, 1/31/15. https://doi.org/10.1145/2684822.2685294
Wang P, Sun W, Yin D, Yang J, Chang Y. Robust tree-based causal inference for complex ad effectiveness analysis. In WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc. 2015. p. 67-76 https://doi.org/10.1145/2684822.2685294
Wang, Pengyuan ; Sun, Wei ; Yin, Dawei ; Yang, Jian ; Chang, Yi. / Robust tree-based causal inference for complex ad effectiveness analysis. WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 2015. pp. 67-76
@inproceedings{6ff480b6abf6420cb351a06d9da8f120,
title = "Robust tree-based causal inference for complex ad effectiveness analysis",
abstract = "As the online advertising industry has evolved into an age of diverse ad formats and delivery channels, users are exposed to complex ad treatments involving various ad characteristics. The diversity and generality of ad treatments call for accurate and causal measurement of ad effectiveness, i.e., how the ad treatment causes the changes in outcomes without the confounding effect by user characteristics. Various causal inference approaches have been proposed to measure the causal effect of ad treatments. However, most existing causal inference methods focus on univariate and binary treatment and are not well suited for complex ad treatments. Moreover, to be practical in the data-rich online environment, the measurement needs to be highly general and efficient, which is not addressed in conventional causal inference approaches. In this paper we propose a novel causal inference framework for assessing the impact of general advertising treatments. Our new framework enables analysis on uni- or multi-dimensional ad treatments, where each dimension (ad treatment factor) could be discrete or continuous. We prove that our approach is able to provide an unbiased estimation of the ad effectiveness by controlling the confounding effect of user characteristics. The framework is computationally efficient by employing a tree structure that specifies the relationship between user characteristics and the corresponding ad treatment. This tree-based framework is robust to model misspecification and highly flexible with minimal manual tuning. To demonstrate the efficacy of our approach, we apply it to two advertising campaigns. In the first campaign we evaluate the impact of different ad frequencies, and in the second one we consider the synthetic ad effectiveness across TV and online platforms. Our framework successfully provides the causal impact of ads with different frequencies in both campaigns. Moreover, it shows that the ad frequency usually has a treatment effect cap, which is usually over-estimated by naive estimation.",
keywords = "Advertising, Causal inference, Online strategy measurement",
author = "Pengyuan Wang and Wei Sun and Dawei Yin and Jian Yang and Yi Chang",
year = "2015",
month = "1",
day = "1",
doi = "10.1145/2684822.2685294",
language = "English (US)",
pages = "67--76",
booktitle = "WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Robust tree-based causal inference for complex ad effectiveness analysis

AU - Wang, Pengyuan

AU - Sun, Wei

AU - Yin, Dawei

AU - Yang, Jian

AU - Chang, Yi

PY - 2015/1/1

Y1 - 2015/1/1

N2 - As the online advertising industry has evolved into an age of diverse ad formats and delivery channels, users are exposed to complex ad treatments involving various ad characteristics. The diversity and generality of ad treatments call for accurate and causal measurement of ad effectiveness, i.e., how the ad treatment causes the changes in outcomes without the confounding effect by user characteristics. Various causal inference approaches have been proposed to measure the causal effect of ad treatments. However, most existing causal inference methods focus on univariate and binary treatment and are not well suited for complex ad treatments. Moreover, to be practical in the data-rich online environment, the measurement needs to be highly general and efficient, which is not addressed in conventional causal inference approaches. In this paper we propose a novel causal inference framework for assessing the impact of general advertising treatments. Our new framework enables analysis on uni- or multi-dimensional ad treatments, where each dimension (ad treatment factor) could be discrete or continuous. We prove that our approach is able to provide an unbiased estimation of the ad effectiveness by controlling the confounding effect of user characteristics. The framework is computationally efficient by employing a tree structure that specifies the relationship between user characteristics and the corresponding ad treatment. This tree-based framework is robust to model misspecification and highly flexible with minimal manual tuning. To demonstrate the efficacy of our approach, we apply it to two advertising campaigns. In the first campaign we evaluate the impact of different ad frequencies, and in the second one we consider the synthetic ad effectiveness across TV and online platforms. Our framework successfully provides the causal impact of ads with different frequencies in both campaigns. Moreover, it shows that the ad frequency usually has a treatment effect cap, which is usually over-estimated by naive estimation.

AB - As the online advertising industry has evolved into an age of diverse ad formats and delivery channels, users are exposed to complex ad treatments involving various ad characteristics. The diversity and generality of ad treatments call for accurate and causal measurement of ad effectiveness, i.e., how the ad treatment causes the changes in outcomes without the confounding effect by user characteristics. Various causal inference approaches have been proposed to measure the causal effect of ad treatments. However, most existing causal inference methods focus on univariate and binary treatment and are not well suited for complex ad treatments. Moreover, to be practical in the data-rich online environment, the measurement needs to be highly general and efficient, which is not addressed in conventional causal inference approaches. In this paper we propose a novel causal inference framework for assessing the impact of general advertising treatments. Our new framework enables analysis on uni- or multi-dimensional ad treatments, where each dimension (ad treatment factor) could be discrete or continuous. We prove that our approach is able to provide an unbiased estimation of the ad effectiveness by controlling the confounding effect of user characteristics. The framework is computationally efficient by employing a tree structure that specifies the relationship between user characteristics and the corresponding ad treatment. This tree-based framework is robust to model misspecification and highly flexible with minimal manual tuning. To demonstrate the efficacy of our approach, we apply it to two advertising campaigns. In the first campaign we evaluate the impact of different ad frequencies, and in the second one we consider the synthetic ad effectiveness across TV and online platforms. Our framework successfully provides the causal impact of ads with different frequencies in both campaigns. Moreover, it shows that the ad frequency usually has a treatment effect cap, which is usually over-estimated by naive estimation.

KW - Advertising

KW - Causal inference

KW - Online strategy measurement

UR - http://www.scopus.com/inward/record.url?scp=84928744232&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928744232&partnerID=8YFLogxK

U2 - 10.1145/2684822.2685294

DO - 10.1145/2684822.2685294

M3 - Conference contribution

SP - 67

EP - 76

BT - WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining

PB - Association for Computing Machinery, Inc

ER -