Modified Kalman filter based method for training state-recurrent multilayer perceptrons

D. Erdogmus, Justin C. Sanchez, J. C. Principe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Kalman filter based training algorithms for recurrent neural networks provide a clever alternative to the standard backpropagation in time. However, these algorithms do not take into account the optimization of the hidden state variables of the recurrent network. In addition, their formulation requires Jacobian evaluations over the entire network, adding to their computational complexity. We propose a spatial-temporal extended Kalman filter algorithm for training recurrent neural network weights and internal states. This new formulation also reduces the computational complexity of Jacobian evaluations drastically by decoupling the gradients of each layer. Monte Carlo comparisons with backpropagation through time point out the robust and fast convergence of the algorithm.

Original languageEnglish (US)
Title of host publicationNeural Networks for Signal Processing - Proceedings of the IEEE Workshop
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages219-228
Number of pages10
Volume2002-January
ISBN (Print)0780376161
DOIs
StatePublished - 2002
Externally publishedYes
Event12th IEEE Workshop on Neural Networks for Signal Processing, NNSP 2002 - Martigny, Switzerland
Duration: Sep 6 2002 → …

Other

Other12th IEEE Workshop on Neural Networks for Signal Processing, NNSP 2002
CountrySwitzerland
CityMartigny
Period9/6/02 → …

    Fingerprint

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Artificial Intelligence
  • Software
  • Computer Networks and Communications
  • Signal Processing

Cite this

Erdogmus, D., Sanchez, J. C., & Principe, J. C. (2002). Modified Kalman filter based method for training state-recurrent multilayer perceptrons. In Neural Networks for Signal Processing - Proceedings of the IEEE Workshop (Vol. 2002-January, pp. 219-228). [1030033] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/NNSP.2002.1030033