Using discriminant analysis for multi-class classification: An experimental investigation

Tao Li, Shenghuo Zhu, Mitsunori Ogihara

Research output: Contribution to journalArticle

140 Citations (Scopus)

Abstract

Many supervised machine learning tasks can be cast as multi-class classification problems. Support vector machines (SVMs) excel at binary classification problems, but the elegant theory behind large-margin hyperplane cannot be easily extended to their multi-class counterparts. On the other hand, it was shown that the decision hyperplanes for binary classification obtained by SVMs are equivalent to the solutions obtained by Fisher's linear discriminant on the set of support vectors. Discriminant analysis approaches are well known to learn discriminative feature transformations in the statistical pattern recognition literature and can be easily extend to multi-class cases. The use of discriminant analysis, however, has not been fully experimented in the data mining literature. In this paper, we explore the use of discriminant analysis for multi-class classification problems. We evaluate the performance of discriminant analysis on a large collection of benchmark datasets and investigate its usage in text categorization. Our experiments suggest that discriminant analysis provides a fast, efficient yet accurate alternative for general multi-class classification problems.

Original languageEnglish (US)
Pages (from-to)453-472
Number of pages20
JournalKnowledge and Information Systems
Volume10
Issue number4
DOIs
StatePublished - Nov 2006
Externally publishedYes

Fingerprint

Discriminant analysis
Support vector machines
Pattern recognition
Data mining
Learning systems
Experiments

Keywords

  • Discriminant analysis
  • Multi-class classification

ASJC Scopus subject areas

  • Information Systems

Cite this

Using discriminant analysis for multi-class classification : An experimental investigation. / Li, Tao; Zhu, Shenghuo; Ogihara, Mitsunori.

In: Knowledge and Information Systems, Vol. 10, No. 4, 11.2006, p. 453-472.

Research output: Contribution to journalArticle

@article{9ec2f91199f54f00acdf311ba45da51c,
title = "Using discriminant analysis for multi-class classification: An experimental investigation",
abstract = "Many supervised machine learning tasks can be cast as multi-class classification problems. Support vector machines (SVMs) excel at binary classification problems, but the elegant theory behind large-margin hyperplane cannot be easily extended to their multi-class counterparts. On the other hand, it was shown that the decision hyperplanes for binary classification obtained by SVMs are equivalent to the solutions obtained by Fisher's linear discriminant on the set of support vectors. Discriminant analysis approaches are well known to learn discriminative feature transformations in the statistical pattern recognition literature and can be easily extend to multi-class cases. The use of discriminant analysis, however, has not been fully experimented in the data mining literature. In this paper, we explore the use of discriminant analysis for multi-class classification problems. We evaluate the performance of discriminant analysis on a large collection of benchmark datasets and investigate its usage in text categorization. Our experiments suggest that discriminant analysis provides a fast, efficient yet accurate alternative for general multi-class classification problems.",
keywords = "Discriminant analysis, Multi-class classification",
author = "Tao Li and Shenghuo Zhu and Mitsunori Ogihara",
year = "2006",
month = "11",
doi = "10.1007/s10115-006-0013-y",
language = "English (US)",
volume = "10",
pages = "453--472",
journal = "Knowledge and Information Systems",
issn = "0219-1377",
publisher = "Springer London",
number = "4",

}

TY - JOUR

T1 - Using discriminant analysis for multi-class classification

T2 - An experimental investigation

AU - Li, Tao

AU - Zhu, Shenghuo

AU - Ogihara, Mitsunori

PY - 2006/11

Y1 - 2006/11

N2 - Many supervised machine learning tasks can be cast as multi-class classification problems. Support vector machines (SVMs) excel at binary classification problems, but the elegant theory behind large-margin hyperplane cannot be easily extended to their multi-class counterparts. On the other hand, it was shown that the decision hyperplanes for binary classification obtained by SVMs are equivalent to the solutions obtained by Fisher's linear discriminant on the set of support vectors. Discriminant analysis approaches are well known to learn discriminative feature transformations in the statistical pattern recognition literature and can be easily extend to multi-class cases. The use of discriminant analysis, however, has not been fully experimented in the data mining literature. In this paper, we explore the use of discriminant analysis for multi-class classification problems. We evaluate the performance of discriminant analysis on a large collection of benchmark datasets and investigate its usage in text categorization. Our experiments suggest that discriminant analysis provides a fast, efficient yet accurate alternative for general multi-class classification problems.

AB - Many supervised machine learning tasks can be cast as multi-class classification problems. Support vector machines (SVMs) excel at binary classification problems, but the elegant theory behind large-margin hyperplane cannot be easily extended to their multi-class counterparts. On the other hand, it was shown that the decision hyperplanes for binary classification obtained by SVMs are equivalent to the solutions obtained by Fisher's linear discriminant on the set of support vectors. Discriminant analysis approaches are well known to learn discriminative feature transformations in the statistical pattern recognition literature and can be easily extend to multi-class cases. The use of discriminant analysis, however, has not been fully experimented in the data mining literature. In this paper, we explore the use of discriminant analysis for multi-class classification problems. We evaluate the performance of discriminant analysis on a large collection of benchmark datasets and investigate its usage in text categorization. Our experiments suggest that discriminant analysis provides a fast, efficient yet accurate alternative for general multi-class classification problems.

KW - Discriminant analysis

KW - Multi-class classification

UR - http://www.scopus.com/inward/record.url?scp=33750574349&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33750574349&partnerID=8YFLogxK

U2 - 10.1007/s10115-006-0013-y

DO - 10.1007/s10115-006-0013-y

M3 - Article

AN - SCOPUS:33750574349

VL - 10

SP - 453

EP - 472

JO - Knowledge and Information Systems

JF - Knowledge and Information Systems

SN - 0219-1377

IS - 4

ER -