A hierarchical sonification framework based on convolutional neural network modeling of musical genre

Shijia Geng, Gang Ren, Mitsunori Ogihara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Convolutional neural networks have satisfactory discriminative performances for various music-related tasks. However, the models are implemented as "black boxes" and thus their processed representations are non-Transparent for manual interactions. In this paper, a hierarchical sonification framework with a musical genre modeling module and a sample-level sonification module has been implemented for aural interaction. The modeling module trains a convolutional neural network from musical signal segments with genre labels. Then the sonification module performs sample-level modification according to each convolutional layer, where lower sonification levels produce auralized pulses and higher sonification levels produce audio signals similar to the input musical signal. The usage of the proposed sonification framework is demonstrated using a musical stylistic morphing example.

Original languageEnglish (US)
Title of host publication141st Audio Engineering Society International Convention 2016, AES 2016
PublisherAudio Engineering Society
StatePublished - 2016
Event141st Audio Engineering Society International Convention 2016, AES 2016 - Los Angeles, United States
Duration: Sep 29 2016Oct 2 2016

Other

Other141st Audio Engineering Society International Convention 2016, AES 2016
CountryUnited States
CityLos Angeles
Period9/29/1610/2/16

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Modeling and Simulation
  • Acoustics and Ultrasonics

Fingerprint Dive into the research topics of 'A hierarchical sonification framework based on convolutional neural network modeling of musical genre'. Together they form a unique fingerprint.

  • Cite this

    Geng, S., Ren, G., & Ogihara, M. (2016). A hierarchical sonification framework based on convolutional neural network modeling of musical genre. In 141st Audio Engineering Society International Convention 2016, AES 2016 Audio Engineering Society.