Multimodal content analysis for effective advertisements on youtube

Nikhita Vedula, Wei Sun, Hyunhwan Lee, Harsh Gupta, Mitsunori Ogihara, Joseph Johnson, Gang Ren, Srinivasan Parthasarathy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

The recent advancement of web-scale digital advertising saw a paradigm shift from the conventional focus of digital advertisement distribution towards integrating digital processes and methodologies and forming a seamless workflow of advertisement design, production, distribution, and effectiveness monitoring. In this work, we implemented a computational framework for the predictive analysis of the content-based features extracted from advertisement video files and various effectiveness metrics to aid the design and production processes of commercial advertisements. Our proposed predictive analysis framework extracts multi-dimensional temporal patterns from the content of advertisement videos using multimedia signal processing and natural language processing tools. The pattern analysis part employs an architecture of cross modality feature learning where data streams from different feature dimensions are employed to train separate neural network models and then these models are fused together to learn a shared representation. Subsequently, a neural network model trained on this joint representation is utilized as a classifier for predicting advertisement effectiveness. Based on the predictive patterns identified between the content features and the effectiveness metrics of advertisements, we have elicited a useful set of auditory, visual and textual patterns that is strongly correlated with the proposed effectiveness metrics while can be readily implemented in the design and production processes of commercial advertisements. We validate our approach using subjective ratings from a dedicated user study, the text sentiment strength of online viewer comments, and a viewer opinion metric of the likes/views ratio of each advertisement from YouTube video-sharing website.

Original languageEnglish (US)
Title of host publicationProceedings - 17th IEEE International Conference on Data Mining, ICDM 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1123-1128
Number of pages6
Volume2017-November
ISBN (Electronic)9781538638347
DOIs
StatePublished - Dec 15 2017
Event17th IEEE International Conference on Data Mining, ICDM 2017 - New Orleans, United States
Duration: Nov 18 2017Nov 21 2017

Other

Other17th IEEE International Conference on Data Mining, ICDM 2017
CountryUnited States
CityNew Orleans
Period11/18/1711/21/17

Fingerprint

Multimedia signal processing
Video signal processing
Neural networks
Websites
Marketing
Classifiers
Monitoring
Processing
Predictive analytics

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Vedula, N., Sun, W., Lee, H., Gupta, H., Ogihara, M., Johnson, J., ... Parthasarathy, S. (2017). Multimodal content analysis for effective advertisements on youtube. In Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017 (Vol. 2017-November, pp. 1123-1128). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2017.149

Multimodal content analysis for effective advertisements on youtube. / Vedula, Nikhita; Sun, Wei; Lee, Hyunhwan; Gupta, Harsh; Ogihara, Mitsunori; Johnson, Joseph; Ren, Gang; Parthasarathy, Srinivasan.

Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. Vol. 2017-November Institute of Electrical and Electronics Engineers Inc., 2017. p. 1123-1128.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Vedula, N, Sun, W, Lee, H, Gupta, H, Ogihara, M, Johnson, J, Ren, G & Parthasarathy, S 2017, Multimodal content analysis for effective advertisements on youtube. in Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. vol. 2017-November, Institute of Electrical and Electronics Engineers Inc., pp. 1123-1128, 17th IEEE International Conference on Data Mining, ICDM 2017, New Orleans, United States, 11/18/17. https://doi.org/10.1109/ICDM.2017.149
Vedula N, Sun W, Lee H, Gupta H, Ogihara M, Johnson J et al. Multimodal content analysis for effective advertisements on youtube. In Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. Vol. 2017-November. Institute of Electrical and Electronics Engineers Inc. 2017. p. 1123-1128 https://doi.org/10.1109/ICDM.2017.149
Vedula, Nikhita ; Sun, Wei ; Lee, Hyunhwan ; Gupta, Harsh ; Ogihara, Mitsunori ; Johnson, Joseph ; Ren, Gang ; Parthasarathy, Srinivasan. / Multimodal content analysis for effective advertisements on youtube. Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. Vol. 2017-November Institute of Electrical and Electronics Engineers Inc., 2017. pp. 1123-1128
@inproceedings{e5872dd06d104aa888017a8b2733eccb,
title = "Multimodal content analysis for effective advertisements on youtube",
abstract = "The recent advancement of web-scale digital advertising saw a paradigm shift from the conventional focus of digital advertisement distribution towards integrating digital processes and methodologies and forming a seamless workflow of advertisement design, production, distribution, and effectiveness monitoring. In this work, we implemented a computational framework for the predictive analysis of the content-based features extracted from advertisement video files and various effectiveness metrics to aid the design and production processes of commercial advertisements. Our proposed predictive analysis framework extracts multi-dimensional temporal patterns from the content of advertisement videos using multimedia signal processing and natural language processing tools. The pattern analysis part employs an architecture of cross modality feature learning where data streams from different feature dimensions are employed to train separate neural network models and then these models are fused together to learn a shared representation. Subsequently, a neural network model trained on this joint representation is utilized as a classifier for predicting advertisement effectiveness. Based on the predictive patterns identified between the content features and the effectiveness metrics of advertisements, we have elicited a useful set of auditory, visual and textual patterns that is strongly correlated with the proposed effectiveness metrics while can be readily implemented in the design and production processes of commercial advertisements. We validate our approach using subjective ratings from a dedicated user study, the text sentiment strength of online viewer comments, and a viewer opinion metric of the likes/views ratio of each advertisement from YouTube video-sharing website.",
author = "Nikhita Vedula and Wei Sun and Hyunhwan Lee and Harsh Gupta and Mitsunori Ogihara and Joseph Johnson and Gang Ren and Srinivasan Parthasarathy",
year = "2017",
month = "12",
day = "15",
doi = "10.1109/ICDM.2017.149",
language = "English (US)",
volume = "2017-November",
pages = "1123--1128",
booktitle = "Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Multimodal content analysis for effective advertisements on youtube

AU - Vedula, Nikhita

AU - Sun, Wei

AU - Lee, Hyunhwan

AU - Gupta, Harsh

AU - Ogihara, Mitsunori

AU - Johnson, Joseph

AU - Ren, Gang

AU - Parthasarathy, Srinivasan

PY - 2017/12/15

Y1 - 2017/12/15

N2 - The recent advancement of web-scale digital advertising saw a paradigm shift from the conventional focus of digital advertisement distribution towards integrating digital processes and methodologies and forming a seamless workflow of advertisement design, production, distribution, and effectiveness monitoring. In this work, we implemented a computational framework for the predictive analysis of the content-based features extracted from advertisement video files and various effectiveness metrics to aid the design and production processes of commercial advertisements. Our proposed predictive analysis framework extracts multi-dimensional temporal patterns from the content of advertisement videos using multimedia signal processing and natural language processing tools. The pattern analysis part employs an architecture of cross modality feature learning where data streams from different feature dimensions are employed to train separate neural network models and then these models are fused together to learn a shared representation. Subsequently, a neural network model trained on this joint representation is utilized as a classifier for predicting advertisement effectiveness. Based on the predictive patterns identified between the content features and the effectiveness metrics of advertisements, we have elicited a useful set of auditory, visual and textual patterns that is strongly correlated with the proposed effectiveness metrics while can be readily implemented in the design and production processes of commercial advertisements. We validate our approach using subjective ratings from a dedicated user study, the text sentiment strength of online viewer comments, and a viewer opinion metric of the likes/views ratio of each advertisement from YouTube video-sharing website.

AB - The recent advancement of web-scale digital advertising saw a paradigm shift from the conventional focus of digital advertisement distribution towards integrating digital processes and methodologies and forming a seamless workflow of advertisement design, production, distribution, and effectiveness monitoring. In this work, we implemented a computational framework for the predictive analysis of the content-based features extracted from advertisement video files and various effectiveness metrics to aid the design and production processes of commercial advertisements. Our proposed predictive analysis framework extracts multi-dimensional temporal patterns from the content of advertisement videos using multimedia signal processing and natural language processing tools. The pattern analysis part employs an architecture of cross modality feature learning where data streams from different feature dimensions are employed to train separate neural network models and then these models are fused together to learn a shared representation. Subsequently, a neural network model trained on this joint representation is utilized as a classifier for predicting advertisement effectiveness. Based on the predictive patterns identified between the content features and the effectiveness metrics of advertisements, we have elicited a useful set of auditory, visual and textual patterns that is strongly correlated with the proposed effectiveness metrics while can be readily implemented in the design and production processes of commercial advertisements. We validate our approach using subjective ratings from a dedicated user study, the text sentiment strength of online viewer comments, and a viewer opinion metric of the likes/views ratio of each advertisement from YouTube video-sharing website.

UR - http://www.scopus.com/inward/record.url?scp=85043975351&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85043975351&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2017.149

DO - 10.1109/ICDM.2017.149

M3 - Conference contribution

AN - SCOPUS:85043975351

VL - 2017-November

SP - 1123

EP - 1128

BT - Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -