Florida International University and University of Miami TRECVID 2010 - semantic indexing

Chao Chen, Qiusha Zhu, Dianting Liu, Tao Meng, Lin Lin, Mei-Ling Shyu, Yimin Yang, Hsin Yu Ha, Fausto Fleites, Shu Ching Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

This paper presents the framework and results of team Florida International University - University of Miami (FIU-UM) for the semantic indexing task of TRECVID 2010. In this task, we submitted four runs of results: • F_A_FIU-UM-1_1: KF+RERANK - apply subspace learning and classification on the key framebased low-level features (KF) and use co-occurrence probability re-ranking method (RERANK) to generate the final ranked results. • F_A_FIU-UM-2_2: LF+KF+SF+RERANK - apply subspace learning and classification on the key frame-based low-level features (KF) and shot-based low-level features (SF) separately. Then co-occurrence probability re-ranking method (RERANK) is used for both key frame based model and shot based model. Finally, a Late Fusion (LF) step combines ranking scores from each model and generates the final ranked shots. • F_A_FIU-UM-3_3: EF+KF+SF+RERANK - apply subspace learning and classification on combined features from the selected key frame-based low-level features (KF) and shot based low-level features (SF) in the Early Fusion (EF) step. Then co-occurrence probability re-ranking method (RERANK) is used. • F_A_FIU-UM-4_4: SF+RERANK - learning and classification based on shot based low-level features (SF). Then co-occurrence probability re-ranking method (RERANK) is used. From the results of different runs, it can be observed that F_A_FIU-UM-1_1 and F_A_FIU-UM-3_3 have better performance than F_A_FIU-UM-2_2 and F_A_FIU-UM-4_4. It implies that adding features from different sources could enhance the effectiveness of the learning and classification model and also visual features seem to be more reliable than audio features for most semantics in TRECVID 2010. The framework aims to handle several challenges in semantic indexing. For the challenge of data imbalance, Multiple Correspondence Analysis (MCA) based pruning method is able to reduce the high ratio between the number of negative instances and the number of positive instances. Meanwhile, for the challenge of semantic gap, the proposed subspace learning and ranking method has adopted a new way to select Principal Components (PCs), which spans a subspace where all instances are projected and classification rules are generated. The scores from one-class positive and negative learning models are further used to rank the classified instances. Then the co-occurrence probability re-ranking approach is utilized to improve the relevance of the retrieved shots. Please note that there is one run that adopts late fusion to combine the scores from key frame-based model and shot-based model. Evaluation results show that more efforts still need to be done to refine each module within our framework and some future directions to be explored are discussed in the conclusion section.

Original languageEnglish
Title of host publication2010 TREC Video Retrieval Evaluation Notebook Papers
PublisherNational Institute of Standards and Technology
StatePublished - Jan 1 2010
EventTREC Video Retrieval Evaluation, TRECVID 2010 - Gaithersburg, MD, United States
Duration: Nov 15 2010Nov 17 2010

Other

OtherTREC Video Retrieval Evaluation, TRECVID 2010
CountryUnited States
CityGaithersburg, MD
Period11/15/1011/17/10

Fingerprint

Semantics
Fusion reactions

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Cite this

Chen, C., Zhu, Q., Liu, D., Meng, T., Lin, L., Shyu, M-L., ... Chen, S. C. (2010). Florida International University and University of Miami TRECVID 2010 - semantic indexing. In 2010 TREC Video Retrieval Evaluation Notebook Papers National Institute of Standards and Technology.

Florida International University and University of Miami TRECVID 2010 - semantic indexing. / Chen, Chao; Zhu, Qiusha; Liu, Dianting; Meng, Tao; Lin, Lin; Shyu, Mei-Ling; Yang, Yimin; Ha, Hsin Yu; Fleites, Fausto; Chen, Shu Ching.

2010 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology, 2010.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chen, C, Zhu, Q, Liu, D, Meng, T, Lin, L, Shyu, M-L, Yang, Y, Ha, HY, Fleites, F & Chen, SC 2010, Florida International University and University of Miami TRECVID 2010 - semantic indexing. in 2010 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology, TREC Video Retrieval Evaluation, TRECVID 2010, Gaithersburg, MD, United States, 11/15/10.
Chen C, Zhu Q, Liu D, Meng T, Lin L, Shyu M-L et al. Florida International University and University of Miami TRECVID 2010 - semantic indexing. In 2010 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology. 2010
Chen, Chao ; Zhu, Qiusha ; Liu, Dianting ; Meng, Tao ; Lin, Lin ; Shyu, Mei-Ling ; Yang, Yimin ; Ha, Hsin Yu ; Fleites, Fausto ; Chen, Shu Ching. / Florida International University and University of Miami TRECVID 2010 - semantic indexing. 2010 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology, 2010.
@inproceedings{1d4df880373f4bb6b9412c8e0cff3ba0,
title = "Florida International University and University of Miami TRECVID 2010 - semantic indexing",
abstract = "This paper presents the framework and results of team Florida International University - University of Miami (FIU-UM) for the semantic indexing task of TRECVID 2010. In this task, we submitted four runs of results: • F_A_FIU-UM-1_1: KF+RERANK - apply subspace learning and classification on the key framebased low-level features (KF) and use co-occurrence probability re-ranking method (RERANK) to generate the final ranked results. • F_A_FIU-UM-2_2: LF+KF+SF+RERANK - apply subspace learning and classification on the key frame-based low-level features (KF) and shot-based low-level features (SF) separately. Then co-occurrence probability re-ranking method (RERANK) is used for both key frame based model and shot based model. Finally, a Late Fusion (LF) step combines ranking scores from each model and generates the final ranked shots. • F_A_FIU-UM-3_3: EF+KF+SF+RERANK - apply subspace learning and classification on combined features from the selected key frame-based low-level features (KF) and shot based low-level features (SF) in the Early Fusion (EF) step. Then co-occurrence probability re-ranking method (RERANK) is used. • F_A_FIU-UM-4_4: SF+RERANK - learning and classification based on shot based low-level features (SF). Then co-occurrence probability re-ranking method (RERANK) is used. From the results of different runs, it can be observed that F_A_FIU-UM-1_1 and F_A_FIU-UM-3_3 have better performance than F_A_FIU-UM-2_2 and F_A_FIU-UM-4_4. It implies that adding features from different sources could enhance the effectiveness of the learning and classification model and also visual features seem to be more reliable than audio features for most semantics in TRECVID 2010. The framework aims to handle several challenges in semantic indexing. For the challenge of data imbalance, Multiple Correspondence Analysis (MCA) based pruning method is able to reduce the high ratio between the number of negative instances and the number of positive instances. Meanwhile, for the challenge of semantic gap, the proposed subspace learning and ranking method has adopted a new way to select Principal Components (PCs), which spans a subspace where all instances are projected and classification rules are generated. The scores from one-class positive and negative learning models are further used to rank the classified instances. Then the co-occurrence probability re-ranking approach is utilized to improve the relevance of the retrieved shots. Please note that there is one run that adopts late fusion to combine the scores from key frame-based model and shot-based model. Evaluation results show that more efforts still need to be done to refine each module within our framework and some future directions to be explored are discussed in the conclusion section.",
author = "Chao Chen and Qiusha Zhu and Dianting Liu and Tao Meng and Lin Lin and Mei-Ling Shyu and Yimin Yang and Ha, {Hsin Yu} and Fausto Fleites and Chen, {Shu Ching}",
year = "2010",
month = "1",
day = "1",
language = "English",
booktitle = "2010 TREC Video Retrieval Evaluation Notebook Papers",
publisher = "National Institute of Standards and Technology",

}

TY - GEN

T1 - Florida International University and University of Miami TRECVID 2010 - semantic indexing

AU - Chen, Chao

AU - Zhu, Qiusha

AU - Liu, Dianting

AU - Meng, Tao

AU - Lin, Lin

AU - Shyu, Mei-Ling

AU - Yang, Yimin

AU - Ha, Hsin Yu

AU - Fleites, Fausto

AU - Chen, Shu Ching

PY - 2010/1/1

Y1 - 2010/1/1

N2 - This paper presents the framework and results of team Florida International University - University of Miami (FIU-UM) for the semantic indexing task of TRECVID 2010. In this task, we submitted four runs of results: • F_A_FIU-UM-1_1: KF+RERANK - apply subspace learning and classification on the key framebased low-level features (KF) and use co-occurrence probability re-ranking method (RERANK) to generate the final ranked results. • F_A_FIU-UM-2_2: LF+KF+SF+RERANK - apply subspace learning and classification on the key frame-based low-level features (KF) and shot-based low-level features (SF) separately. Then co-occurrence probability re-ranking method (RERANK) is used for both key frame based model and shot based model. Finally, a Late Fusion (LF) step combines ranking scores from each model and generates the final ranked shots. • F_A_FIU-UM-3_3: EF+KF+SF+RERANK - apply subspace learning and classification on combined features from the selected key frame-based low-level features (KF) and shot based low-level features (SF) in the Early Fusion (EF) step. Then co-occurrence probability re-ranking method (RERANK) is used. • F_A_FIU-UM-4_4: SF+RERANK - learning and classification based on shot based low-level features (SF). Then co-occurrence probability re-ranking method (RERANK) is used. From the results of different runs, it can be observed that F_A_FIU-UM-1_1 and F_A_FIU-UM-3_3 have better performance than F_A_FIU-UM-2_2 and F_A_FIU-UM-4_4. It implies that adding features from different sources could enhance the effectiveness of the learning and classification model and also visual features seem to be more reliable than audio features for most semantics in TRECVID 2010. The framework aims to handle several challenges in semantic indexing. For the challenge of data imbalance, Multiple Correspondence Analysis (MCA) based pruning method is able to reduce the high ratio between the number of negative instances and the number of positive instances. Meanwhile, for the challenge of semantic gap, the proposed subspace learning and ranking method has adopted a new way to select Principal Components (PCs), which spans a subspace where all instances are projected and classification rules are generated. The scores from one-class positive and negative learning models are further used to rank the classified instances. Then the co-occurrence probability re-ranking approach is utilized to improve the relevance of the retrieved shots. Please note that there is one run that adopts late fusion to combine the scores from key frame-based model and shot-based model. Evaluation results show that more efforts still need to be done to refine each module within our framework and some future directions to be explored are discussed in the conclusion section.

AB - This paper presents the framework and results of team Florida International University - University of Miami (FIU-UM) for the semantic indexing task of TRECVID 2010. In this task, we submitted four runs of results: • F_A_FIU-UM-1_1: KF+RERANK - apply subspace learning and classification on the key framebased low-level features (KF) and use co-occurrence probability re-ranking method (RERANK) to generate the final ranked results. • F_A_FIU-UM-2_2: LF+KF+SF+RERANK - apply subspace learning and classification on the key frame-based low-level features (KF) and shot-based low-level features (SF) separately. Then co-occurrence probability re-ranking method (RERANK) is used for both key frame based model and shot based model. Finally, a Late Fusion (LF) step combines ranking scores from each model and generates the final ranked shots. • F_A_FIU-UM-3_3: EF+KF+SF+RERANK - apply subspace learning and classification on combined features from the selected key frame-based low-level features (KF) and shot based low-level features (SF) in the Early Fusion (EF) step. Then co-occurrence probability re-ranking method (RERANK) is used. • F_A_FIU-UM-4_4: SF+RERANK - learning and classification based on shot based low-level features (SF). Then co-occurrence probability re-ranking method (RERANK) is used. From the results of different runs, it can be observed that F_A_FIU-UM-1_1 and F_A_FIU-UM-3_3 have better performance than F_A_FIU-UM-2_2 and F_A_FIU-UM-4_4. It implies that adding features from different sources could enhance the effectiveness of the learning and classification model and also visual features seem to be more reliable than audio features for most semantics in TRECVID 2010. The framework aims to handle several challenges in semantic indexing. For the challenge of data imbalance, Multiple Correspondence Analysis (MCA) based pruning method is able to reduce the high ratio between the number of negative instances and the number of positive instances. Meanwhile, for the challenge of semantic gap, the proposed subspace learning and ranking method has adopted a new way to select Principal Components (PCs), which spans a subspace where all instances are projected and classification rules are generated. The scores from one-class positive and negative learning models are further used to rank the classified instances. Then the co-occurrence probability re-ranking approach is utilized to improve the relevance of the retrieved shots. Please note that there is one run that adopts late fusion to combine the scores from key frame-based model and shot-based model. Evaluation results show that more efforts still need to be done to refine each module within our framework and some future directions to be explored are discussed in the conclusion section.

UR - http://www.scopus.com/inward/record.url?scp=84905181681&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84905181681&partnerID=8YFLogxK

M3 - Conference contribution

BT - 2010 TREC Video Retrieval Evaluation Notebook Papers

PB - National Institute of Standards and Technology

ER -