Ortho-ordent initialization of FFANNAs to improve their generalization ability

Research output: Contribution to journalConference article

Abstract

There are several models for neurons and their interconnections. Among them, feedforward artificial neural networks (FFANNs) are very popular for being quite simple. However, to make them truly reliable and smart information processing systems, such characteristics as learning speed, local minima, and generalization ability need more understanding. Difficulties such as long learning-time and local minima, may not affect them as much as the question of generalization ability, because in many applications a network needs only one training, and then it may be used for a long time. However, the question of generalization ability of ANNs is of great interest for both theoretical understanding and practical use, because generalization ability is a measure of a learning system that indicates how closely its actual output approximates to the desired output for an input that it has never seen. We investigate novel techniques for systematic initializations (as opposed to purely random initializations) of FFANN architectures for possible improvement of their generalization ability. Our preliminary work has successfully employed row-vectors of Hadamard matrices to generate initializations; this initialization method has produced networks with better generalization ability.

Original languageEnglish (US)
Pages (from-to)767-778
Number of pages12
JournalProceedings of SPIE - The International Society for Optical Engineering
Volume2492
DOIs
StatePublished - Apr 6 1995
Externally publishedYes
EventApplications and Science of Artificial Neural Networks 1995 - Orlando, United States
Duration: Apr 17 1995Apr 21 1995

Keywords

  • Artificial neural networks (ANNs)
  • Bipolar vectors
  • Error backpropagation learning
  • Feedforward networks
  • Generalization ability of learning systems
  • Hadamard matrices
  • Initialization of anns
  • Orthogonal vectors
  • Recursive algorithm

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Ortho-ordent initialization of FFANNAs to improve their generalization ability'. Together they form a unique fingerprint.

  • Cite this