Florida International University and University of Miami TRECVID 2008 - high level feature extraction

Guy Ravitz, Lin Lin, Mei-Ling Shyu, Michael Armella, Shu Ching Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper describes the FIU-UMgroup TRECVID 2008 high level feature extraction task submission. We have used a correlation based video semantic concept detection system for this task submission. This system first extracts shot based low-level audiovisual features from the raw data source (audio and video files). The resulting numerical feature set is then discretized. Multiple correspondence analysis (MCA) is then used to explore the correlation between items, which are the feature-value pairs generated by the discretization process, and the different concepts. This process generates both positive and negative rules. During the classification process each instance (shot) is tested against each rule. The score for each instance determines the final classification. We have conducted two runs using two different predetermined values as the score threshold for classification: • A_FIU-UM-FE1 1: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using -2 as the instance score for final classification • A FIU-UM-FE2 2: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using 0 as the instance score for final classification (simple majority) We observed a slight improvement in the A FIU-UM-FE2_2 run over the A FIU-UM-FE1 1 run. Initially it appeared from the training data that using a score of -2 could potentially provide a better performance, however; in order to test a true majority voting concept we have conducted the second run (A_FIU-UM-FE2_2) using 0 as our threshold. Based on the submitted results and our results produced in some of our previous work [6] we believe that theMCA process has the capability to learn the correlation between low-level features such as color, volume, texture etc. and high level features (concepts) and by that help narrow the semantic gap. One of the biggest challenges of this year's high level feature extraction task was the fact that the target high-level feature list has been changed. This year we have used the same low-level features that we used in 2007. We believe that this low level feature set might have not been the best candidate to represent new high level feature list. Therefore, We believe that extracting additional audio-visual features which are a little more relevant to the new concept list would have improved our observed performance. Finally we observed that the problem of imbalanced data is still a major challenge that our system is having difficulties to address. In this paper we will provide more details regarding our system, discuss our observations, and provide some thoughts regarding the future to which this system is heading.

Original languageEnglish
Title of host publication2008 TREC Video Retrieval Evaluation Notebook Papers
PublisherNational Institute of Standards and Technology
StatePublished - Jan 1 2008
EventTREC Video Retrieval Evaluation, TRECVID 2008 - Gaithersburg, MD, United States
Duration: Nov 17 2008Nov 18 2008

Other

OtherTREC Video Retrieval Evaluation, TRECVID 2008
CountryUnited States
CityGaithersburg, MD
Period11/17/0811/18/08

Fingerprint

Feature extraction
Semantics
Textures
Color

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Cite this

Ravitz, G., Lin, L., Shyu, M-L., Armella, M., & Chen, S. C. (2008). Florida International University and University of Miami TRECVID 2008 - high level feature extraction. In 2008 TREC Video Retrieval Evaluation Notebook Papers National Institute of Standards and Technology.

Florida International University and University of Miami TRECVID 2008 - high level feature extraction. / Ravitz, Guy; Lin, Lin; Shyu, Mei-Ling; Armella, Michael; Chen, Shu Ching.

2008 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology, 2008.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ravitz, G, Lin, L, Shyu, M-L, Armella, M & Chen, SC 2008, Florida International University and University of Miami TRECVID 2008 - high level feature extraction. in 2008 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology, TREC Video Retrieval Evaluation, TRECVID 2008, Gaithersburg, MD, United States, 11/17/08.
Ravitz G, Lin L, Shyu M-L, Armella M, Chen SC. Florida International University and University of Miami TRECVID 2008 - high level feature extraction. In 2008 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology. 2008
Ravitz, Guy ; Lin, Lin ; Shyu, Mei-Ling ; Armella, Michael ; Chen, Shu Ching. / Florida International University and University of Miami TRECVID 2008 - high level feature extraction. 2008 TREC Video Retrieval Evaluation Notebook Papers. National Institute of Standards and Technology, 2008.
@inproceedings{6af567cf6fa34b75a625679923b3ee2e,
title = "Florida International University and University of Miami TRECVID 2008 - high level feature extraction",
abstract = "This paper describes the FIU-UMgroup TRECVID 2008 high level feature extraction task submission. We have used a correlation based video semantic concept detection system for this task submission. This system first extracts shot based low-level audiovisual features from the raw data source (audio and video files). The resulting numerical feature set is then discretized. Multiple correspondence analysis (MCA) is then used to explore the correlation between items, which are the feature-value pairs generated by the discretization process, and the different concepts. This process generates both positive and negative rules. During the classification process each instance (shot) is tested against each rule. The score for each instance determines the final classification. We have conducted two runs using two different predetermined values as the score threshold for classification: • A_FIU-UM-FE1 1: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using -2 as the instance score for final classification • A FIU-UM-FE2 2: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using 0 as the instance score for final classification (simple majority) We observed a slight improvement in the A FIU-UM-FE2_2 run over the A FIU-UM-FE1 1 run. Initially it appeared from the training data that using a score of -2 could potentially provide a better performance, however; in order to test a true majority voting concept we have conducted the second run (A_FIU-UM-FE2_2) using 0 as our threshold. Based on the submitted results and our results produced in some of our previous work [6] we believe that theMCA process has the capability to learn the correlation between low-level features such as color, volume, texture etc. and high level features (concepts) and by that help narrow the semantic gap. One of the biggest challenges of this year's high level feature extraction task was the fact that the target high-level feature list has been changed. This year we have used the same low-level features that we used in 2007. We believe that this low level feature set might have not been the best candidate to represent new high level feature list. Therefore, We believe that extracting additional audio-visual features which are a little more relevant to the new concept list would have improved our observed performance. Finally we observed that the problem of imbalanced data is still a major challenge that our system is having difficulties to address. In this paper we will provide more details regarding our system, discuss our observations, and provide some thoughts regarding the future to which this system is heading.",
author = "Guy Ravitz and Lin Lin and Mei-Ling Shyu and Michael Armella and Chen, {Shu Ching}",
year = "2008",
month = "1",
day = "1",
language = "English",
booktitle = "2008 TREC Video Retrieval Evaluation Notebook Papers",
publisher = "National Institute of Standards and Technology",

}

TY - GEN

T1 - Florida International University and University of Miami TRECVID 2008 - high level feature extraction

AU - Ravitz, Guy

AU - Lin, Lin

AU - Shyu, Mei-Ling

AU - Armella, Michael

AU - Chen, Shu Ching

PY - 2008/1/1

Y1 - 2008/1/1

N2 - This paper describes the FIU-UMgroup TRECVID 2008 high level feature extraction task submission. We have used a correlation based video semantic concept detection system for this task submission. This system first extracts shot based low-level audiovisual features from the raw data source (audio and video files). The resulting numerical feature set is then discretized. Multiple correspondence analysis (MCA) is then used to explore the correlation between items, which are the feature-value pairs generated by the discretization process, and the different concepts. This process generates both positive and negative rules. During the classification process each instance (shot) is tested against each rule. The score for each instance determines the final classification. We have conducted two runs using two different predetermined values as the score threshold for classification: • A_FIU-UM-FE1 1: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using -2 as the instance score for final classification • A FIU-UM-FE2 2: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using 0 as the instance score for final classification (simple majority) We observed a slight improvement in the A FIU-UM-FE2_2 run over the A FIU-UM-FE1 1 run. Initially it appeared from the training data that using a score of -2 could potentially provide a better performance, however; in order to test a true majority voting concept we have conducted the second run (A_FIU-UM-FE2_2) using 0 as our threshold. Based on the submitted results and our results produced in some of our previous work [6] we believe that theMCA process has the capability to learn the correlation between low-level features such as color, volume, texture etc. and high level features (concepts) and by that help narrow the semantic gap. One of the biggest challenges of this year's high level feature extraction task was the fact that the target high-level feature list has been changed. This year we have used the same low-level features that we used in 2007. We believe that this low level feature set might have not been the best candidate to represent new high level feature list. Therefore, We believe that extracting additional audio-visual features which are a little more relevant to the new concept list would have improved our observed performance. Finally we observed that the problem of imbalanced data is still a major challenge that our system is having difficulties to address. In this paper we will provide more details regarding our system, discuss our observations, and provide some thoughts regarding the future to which this system is heading.

AB - This paper describes the FIU-UMgroup TRECVID 2008 high level feature extraction task submission. We have used a correlation based video semantic concept detection system for this task submission. This system first extracts shot based low-level audiovisual features from the raw data source (audio and video files). The resulting numerical feature set is then discretized. Multiple correspondence analysis (MCA) is then used to explore the correlation between items, which are the feature-value pairs generated by the discretization process, and the different concepts. This process generates both positive and negative rules. During the classification process each instance (shot) is tested against each rule. The score for each instance determines the final classification. We have conducted two runs using two different predetermined values as the score threshold for classification: • A_FIU-UM-FE1 1: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using -2 as the instance score for final classification • A FIU-UM-FE2 2: train on partial TRECVID2008 development data (all TRECVID2007 development data + partial TRECVID2007 test data) using 0 as the instance score for final classification (simple majority) We observed a slight improvement in the A FIU-UM-FE2_2 run over the A FIU-UM-FE1 1 run. Initially it appeared from the training data that using a score of -2 could potentially provide a better performance, however; in order to test a true majority voting concept we have conducted the second run (A_FIU-UM-FE2_2) using 0 as our threshold. Based on the submitted results and our results produced in some of our previous work [6] we believe that theMCA process has the capability to learn the correlation between low-level features such as color, volume, texture etc. and high level features (concepts) and by that help narrow the semantic gap. One of the biggest challenges of this year's high level feature extraction task was the fact that the target high-level feature list has been changed. This year we have used the same low-level features that we used in 2007. We believe that this low level feature set might have not been the best candidate to represent new high level feature list. Therefore, We believe that extracting additional audio-visual features which are a little more relevant to the new concept list would have improved our observed performance. Finally we observed that the problem of imbalanced data is still a major challenge that our system is having difficulties to address. In this paper we will provide more details regarding our system, discuss our observations, and provide some thoughts regarding the future to which this system is heading.

UR - http://www.scopus.com/inward/record.url?scp=84905186760&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84905186760&partnerID=8YFLogxK

M3 - Conference contribution

BT - 2008 TREC Video Retrieval Evaluation Notebook Papers

PB - National Institute of Standards and Technology

ER -