Correlation-Assisted imbalance multimedia concept mining and retrieval

Yilin Yan, Mei Ling Shyu

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


In the past decades, we have witnessed an explosion of multimedia data, especially with the development of social media websites and blooming popularity of smart devices. As a result, multimedia semantic concept mining and retrieval whose objective is to mine useful information from the large amount of multimedia data including texts, images, and videos has become more and more important. The huge amount of multimedia data and the semantic gap between low-level features and high-level semantic concepts have made it even more challenging. To address these challenges, the correlations among the classes can provide important context cues to help bridge the semantic gap. Meanwhile, many real-world datasets do not have uniform class distributions while the minority instances actually represent the concept of interests, like frauds in transactions, intrusions in network security, and unusual events in surveillance. Despite extensive research efforts, imbalanced concept retrieval remains one of the most challenging research problems in multimedia data mining. Different from existing frameworks regarding concept correlations among labels, this paper presents a novel concept correlation analysis model using the correlation between the retrieval scores and labels. Experimental results on the TRECVID benchmark datasets demonstrate that the proposed framework can enhance imbalanced concept mining and retrieval even with trivial scores from the minority class.

Original languageEnglish (US)
Pages (from-to)209-227
Number of pages19
JournalInternational Journal of Semantic Computing
Issue number2
StatePublished - Jun 1 2017


  • Imbalanced data
  • concept correlation
  • information integration
  • multimedia big data
  • multimedia semantic retrieval
  • rare class mining

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Linguistics and Language
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Correlation-Assisted imbalance multimedia concept mining and retrieval'. Together they form a unique fingerprint.

Cite this