Randomness in generalization ability

A source to improve it

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Among several models of neurons and their interconnections, feedforward artificial neural networks (FFANN's) are most popular, because of their simplicity and effectiveness. Some obstacles, however, are yet to be cleared to make them truly reliable-smart information processing systems. Difficulties such as long learning time and local minima may not affect FFANN's as much as the question of generalization ability, because a network needs only one training, and then it may be used for a long time. The question of generalization ability of ANN's, however, is of great interest for both theoretical understanding and practical use. This paper reports our observations about randomness in generalization ability of FFANN's. A novel method for measuring generalization ability is defined. This method can be used to identify degree of randomness in generalization ability of learning systems. If an FFANN architecture shows randomness in generalization ability for a given problem, multiple networks can be used to improve it. We have developed a model, called voting model, for predicting generalization ability of multiple networks. It has been shown that if correct classification probability of a single network is greater than half, then as the number of networks in a voting network is increased so does its generalization ability. Further analysis has shown that VC-dimension of the voting network model may increase monotonically as the number of networks in the voting networks is increased. This result is counter intuitive, since it is generally believed that the smaller the VC-dimension, the better the generalization ability.

Original languageEnglish (US)
Pages (from-to)676-685
Number of pages10
JournalIEEE Transactions on Neural Networks
Volume7
Issue number3
DOIs
StatePublished - Dec 1 1996

Fingerprint

Aptitude
Neural networks
Politics
Network architecture
Neurons
Learning systems
Learning
Automatic Data Processing
Information Systems

ASJC Scopus subject areas

  • Software
  • Medicine(all)
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Randomness in generalization ability : A source to improve it. / Sarkar, Dilip.

In: IEEE Transactions on Neural Networks, Vol. 7, No. 3, 01.12.1996, p. 676-685.

Research output: Contribution to journalArticle

@article{707721b4018b49ec8101c1883f9691c7,
title = "Randomness in generalization ability: A source to improve it",
abstract = "Among several models of neurons and their interconnections, feedforward artificial neural networks (FFANN's) are most popular, because of their simplicity and effectiveness. Some obstacles, however, are yet to be cleared to make them truly reliable-smart information processing systems. Difficulties such as long learning time and local minima may not affect FFANN's as much as the question of generalization ability, because a network needs only one training, and then it may be used for a long time. The question of generalization ability of ANN's, however, is of great interest for both theoretical understanding and practical use. This paper reports our observations about randomness in generalization ability of FFANN's. A novel method for measuring generalization ability is defined. This method can be used to identify degree of randomness in generalization ability of learning systems. If an FFANN architecture shows randomness in generalization ability for a given problem, multiple networks can be used to improve it. We have developed a model, called voting model, for predicting generalization ability of multiple networks. It has been shown that if correct classification probability of a single network is greater than half, then as the number of networks in a voting network is increased so does its generalization ability. Further analysis has shown that VC-dimension of the voting network model may increase monotonically as the number of networks in the voting networks is increased. This result is counter intuitive, since it is generally believed that the smaller the VC-dimension, the better the generalization ability.",
author = "Dilip Sarkar",
year = "1996",
month = "12",
day = "1",
doi = "10.1109/72.501725",
language = "English (US)",
volume = "7",
pages = "676--685",
journal = "IEEE Transactions on Neural Networks",
issn = "1045-9227",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "3",

}

TY - JOUR

T1 - Randomness in generalization ability

T2 - A source to improve it

AU - Sarkar, Dilip

PY - 1996/12/1

Y1 - 1996/12/1

N2 - Among several models of neurons and their interconnections, feedforward artificial neural networks (FFANN's) are most popular, because of their simplicity and effectiveness. Some obstacles, however, are yet to be cleared to make them truly reliable-smart information processing systems. Difficulties such as long learning time and local minima may not affect FFANN's as much as the question of generalization ability, because a network needs only one training, and then it may be used for a long time. The question of generalization ability of ANN's, however, is of great interest for both theoretical understanding and practical use. This paper reports our observations about randomness in generalization ability of FFANN's. A novel method for measuring generalization ability is defined. This method can be used to identify degree of randomness in generalization ability of learning systems. If an FFANN architecture shows randomness in generalization ability for a given problem, multiple networks can be used to improve it. We have developed a model, called voting model, for predicting generalization ability of multiple networks. It has been shown that if correct classification probability of a single network is greater than half, then as the number of networks in a voting network is increased so does its generalization ability. Further analysis has shown that VC-dimension of the voting network model may increase monotonically as the number of networks in the voting networks is increased. This result is counter intuitive, since it is generally believed that the smaller the VC-dimension, the better the generalization ability.

AB - Among several models of neurons and their interconnections, feedforward artificial neural networks (FFANN's) are most popular, because of their simplicity and effectiveness. Some obstacles, however, are yet to be cleared to make them truly reliable-smart information processing systems. Difficulties such as long learning time and local minima may not affect FFANN's as much as the question of generalization ability, because a network needs only one training, and then it may be used for a long time. The question of generalization ability of ANN's, however, is of great interest for both theoretical understanding and practical use. This paper reports our observations about randomness in generalization ability of FFANN's. A novel method for measuring generalization ability is defined. This method can be used to identify degree of randomness in generalization ability of learning systems. If an FFANN architecture shows randomness in generalization ability for a given problem, multiple networks can be used to improve it. We have developed a model, called voting model, for predicting generalization ability of multiple networks. It has been shown that if correct classification probability of a single network is greater than half, then as the number of networks in a voting network is increased so does its generalization ability. Further analysis has shown that VC-dimension of the voting network model may increase monotonically as the number of networks in the voting networks is increased. This result is counter intuitive, since it is generally believed that the smaller the VC-dimension, the better the generalization ability.

UR - http://www.scopus.com/inward/record.url?scp=0030149064&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030149064&partnerID=8YFLogxK

U2 - 10.1109/72.501725

DO - 10.1109/72.501725

M3 - Article

VL - 7

SP - 676

EP - 685

JO - IEEE Transactions on Neural Networks

JF - IEEE Transactions on Neural Networks

SN - 1045-9227

IS - 3

ER -