Multimodal content analysis for effective advertisements on youtube

Nikhita Vedula, Wei Sun, Hyunhwan Lee, Harsh Gupta, Mitsunori Ogihara, Joseph Johnson, Gang Ren, Srinivasan Parthasarathy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

The recent advancement of web-scale digital advertising saw a paradigm shift from the conventional focus of digital advertisement distribution towards integrating digital processes and methodologies and forming a seamless workflow of advertisement design, production, distribution, and effectiveness monitoring. In this work, we implemented a computational framework for the predictive analysis of the content-based features extracted from advertisement video files and various effectiveness metrics to aid the design and production processes of commercial advertisements. Our proposed predictive analysis framework extracts multi-dimensional temporal patterns from the content of advertisement videos using multimedia signal processing and natural language processing tools. The pattern analysis part employs an architecture of cross modality feature learning where data streams from different feature dimensions are employed to train separate neural network models and then these models are fused together to learn a shared representation. Subsequently, a neural network model trained on this joint representation is utilized as a classifier for predicting advertisement effectiveness. Based on the predictive patterns identified between the content features and the effectiveness metrics of advertisements, we have elicited a useful set of auditory, visual and textual patterns that is strongly correlated with the proposed effectiveness metrics while can be readily implemented in the design and production processes of commercial advertisements. We validate our approach using subjective ratings from a dedicated user study, the text sentiment strength of online viewer comments, and a viewer opinion metric of the likes/views ratio of each advertisement from YouTube video-sharing website.

Original languageEnglish (US)
Title of host publicationProceedings - 17th IEEE International Conference on Data Mining, ICDM 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1123-1128
Number of pages6
Volume2017-November
ISBN (Electronic)9781538638347
DOIs
StatePublished - Dec 15 2017
Event17th IEEE International Conference on Data Mining, ICDM 2017 - New Orleans, United States
Duration: Nov 18 2017Nov 21 2017

Other

Other17th IEEE International Conference on Data Mining, ICDM 2017
CountryUnited States
CityNew Orleans
Period11/18/1711/21/17

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Multimodal content analysis for effective advertisements on youtube'. Together they form a unique fingerprint.

  • Cite this

    Vedula, N., Sun, W., Lee, H., Gupta, H., Ogihara, M., Johnson, J., Ren, G., & Parthasarathy, S. (2017). Multimodal content analysis for effective advertisements on youtube. In Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017 (Vol. 2017-November, pp. 1123-1128). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2017.149