Effective and efficient video high-level semantic retrieval using associations and correlations

L. I.N. Lin, Mei Ling Shyu

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Two important approaches in multimedia information retrieval are classification and the ranking of the retrieved results. The technique of performing classification using Association Rule Mining (ARM) has been utilized to detect the high-level features from the video, taking advantages of its high efficiency and accuracy. Motivated by the fact that the users are only interested in the top-ranked relevant results, ranking strategies have been adopted to sort the retrieved results. In this paper, an effective and efficient video high-level semantic retrieval framework that utilizes associations and correlations to retrieve and rank the high-level features is developed. The n-feature-value pair rules are generated using a combined measure based on (1) the existence of the (n-1)-feature-value pairs, where n is larger than 1, (2) the correlation between different n-feature-value pairs and the concept classes through Multiple Correspondence Analysis (MCA), and (3) the similarity representing the harmonic mean of the inter-similarity and intra-similarity. The final association classification rules are selected by using the calculated similarity values. Then our proposed ranking process uses the scores that integrate the correlation and similarity values to rank the retrieved results. To show the robustness of the proposed framework, experiments with 15 high-level features (concepts) and benchmark data sets from TRECVID and comparisons with 6 other well-known classifiers are presented. Our proposed framework achieves promising performance and outperforms all the other classifiers. Moreover, the final ranked retrieved results are evaluated by the mean average precision measure, which is commonly used for performance evaluation in the TRECVID community.

Original languageEnglish (US)
Pages (from-to)421-444
Number of pages24
JournalInternational Journal of Semantic Computing
Volume3
Issue number4
DOIs
StatePublished - Dec 1 2009

Fingerprint

video
Semantics
semantics
Classifiers
ranking
Values
Association rules
Information retrieval
class concept
correspondence analysis
information retrieval
performance
multimedia
Experiments
efficiency
experiment
evaluation
community

Keywords

  • Association Rule Mining (ARM)
  • Multiple Correspondence Analysis (MCA)
  • Video retrieval

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Linguistics and Language
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Effective and efficient video high-level semantic retrieval using associations and correlations. / Lin, L. I.N.; Shyu, Mei Ling.

In: International Journal of Semantic Computing, Vol. 3, No. 4, 01.12.2009, p. 421-444.

Research output: Contribution to journalArticle

@article{25b81717db1c4adb9685210e13e0d602,
title = "Effective and efficient video high-level semantic retrieval using associations and correlations",
abstract = "Two important approaches in multimedia information retrieval are classification and the ranking of the retrieved results. The technique of performing classification using Association Rule Mining (ARM) has been utilized to detect the high-level features from the video, taking advantages of its high efficiency and accuracy. Motivated by the fact that the users are only interested in the top-ranked relevant results, ranking strategies have been adopted to sort the retrieved results. In this paper, an effective and efficient video high-level semantic retrieval framework that utilizes associations and correlations to retrieve and rank the high-level features is developed. The n-feature-value pair rules are generated using a combined measure based on (1) the existence of the (n-1)-feature-value pairs, where n is larger than 1, (2) the correlation between different n-feature-value pairs and the concept classes through Multiple Correspondence Analysis (MCA), and (3) the similarity representing the harmonic mean of the inter-similarity and intra-similarity. The final association classification rules are selected by using the calculated similarity values. Then our proposed ranking process uses the scores that integrate the correlation and similarity values to rank the retrieved results. To show the robustness of the proposed framework, experiments with 15 high-level features (concepts) and benchmark data sets from TRECVID and comparisons with 6 other well-known classifiers are presented. Our proposed framework achieves promising performance and outperforms all the other classifiers. Moreover, the final ranked retrieved results are evaluated by the mean average precision measure, which is commonly used for performance evaluation in the TRECVID community.",
keywords = "Association Rule Mining (ARM), Multiple Correspondence Analysis (MCA), Video retrieval",
author = "Lin, {L. I.N.} and Shyu, {Mei Ling}",
year = "2009",
month = "12",
day = "1",
doi = "10.1142/S1793351X09000860",
language = "English (US)",
volume = "3",
pages = "421--444",
journal = "International Journal of Semantic Computing",
issn = "1793-351X",
publisher = "World Scientific Publishing Co. Pte Ltd",
number = "4",

}

TY - JOUR

T1 - Effective and efficient video high-level semantic retrieval using associations and correlations

AU - Lin, L. I.N.

AU - Shyu, Mei Ling

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Two important approaches in multimedia information retrieval are classification and the ranking of the retrieved results. The technique of performing classification using Association Rule Mining (ARM) has been utilized to detect the high-level features from the video, taking advantages of its high efficiency and accuracy. Motivated by the fact that the users are only interested in the top-ranked relevant results, ranking strategies have been adopted to sort the retrieved results. In this paper, an effective and efficient video high-level semantic retrieval framework that utilizes associations and correlations to retrieve and rank the high-level features is developed. The n-feature-value pair rules are generated using a combined measure based on (1) the existence of the (n-1)-feature-value pairs, where n is larger than 1, (2) the correlation between different n-feature-value pairs and the concept classes through Multiple Correspondence Analysis (MCA), and (3) the similarity representing the harmonic mean of the inter-similarity and intra-similarity. The final association classification rules are selected by using the calculated similarity values. Then our proposed ranking process uses the scores that integrate the correlation and similarity values to rank the retrieved results. To show the robustness of the proposed framework, experiments with 15 high-level features (concepts) and benchmark data sets from TRECVID and comparisons with 6 other well-known classifiers are presented. Our proposed framework achieves promising performance and outperforms all the other classifiers. Moreover, the final ranked retrieved results are evaluated by the mean average precision measure, which is commonly used for performance evaluation in the TRECVID community.

AB - Two important approaches in multimedia information retrieval are classification and the ranking of the retrieved results. The technique of performing classification using Association Rule Mining (ARM) has been utilized to detect the high-level features from the video, taking advantages of its high efficiency and accuracy. Motivated by the fact that the users are only interested in the top-ranked relevant results, ranking strategies have been adopted to sort the retrieved results. In this paper, an effective and efficient video high-level semantic retrieval framework that utilizes associations and correlations to retrieve and rank the high-level features is developed. The n-feature-value pair rules are generated using a combined measure based on (1) the existence of the (n-1)-feature-value pairs, where n is larger than 1, (2) the correlation between different n-feature-value pairs and the concept classes through Multiple Correspondence Analysis (MCA), and (3) the similarity representing the harmonic mean of the inter-similarity and intra-similarity. The final association classification rules are selected by using the calculated similarity values. Then our proposed ranking process uses the scores that integrate the correlation and similarity values to rank the retrieved results. To show the robustness of the proposed framework, experiments with 15 high-level features (concepts) and benchmark data sets from TRECVID and comparisons with 6 other well-known classifiers are presented. Our proposed framework achieves promising performance and outperforms all the other classifiers. Moreover, the final ranked retrieved results are evaluated by the mean average precision measure, which is commonly used for performance evaluation in the TRECVID community.

KW - Association Rule Mining (ARM)

KW - Multiple Correspondence Analysis (MCA)

KW - Video retrieval

UR - http://www.scopus.com/inward/record.url?scp=84863052743&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863052743&partnerID=8YFLogxK

U2 - 10.1142/S1793351X09000860

DO - 10.1142/S1793351X09000860

M3 - Article

AN - SCOPUS:84863052743

VL - 3

SP - 421

EP - 444

JO - International Journal of Semantic Computing

JF - International Journal of Semantic Computing

SN - 1793-351X

IS - 4

ER -