Reinforcement learning via kernel temporal difference

Jihye Bae, Pratik Chhatbar, Joseph T. Francis, Justin C. Sanchez, Jose C. Principe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

This paper introduces a kernel adaptive filter implemented with stochastic gradient on temporal differences, kernel Temporal Difference (TD)(λ), to estimate the state-action value function in reinforcement learning. The case λ=0 will be studied in this paper. Experimental results show the method's applicability for learning motor state decoding during a center-out reaching task performed by a monkey. The results are compared to the implementation of a time delay neural network (TDNN) trained with backpropagation of the temporal difference error. From the experiments, it is observed that kernel TD(0) allows faster convergence and a better solution than the neural network.

Original languageEnglish (US)
Title of host publication33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011
Pages5662-5665
Number of pages4
DOIs
StatePublished - Dec 26 2011
Event33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011 - Boston, MA, United States
Duration: Aug 30 2011Sep 3 2011

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
ISSN (Print)1557-170X

Other

Other33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011
CountryUnited States
CityBoston, MA
Period8/30/119/3/11

    Fingerprint

ASJC Scopus subject areas

  • Signal Processing
  • Biomedical Engineering
  • Computer Vision and Pattern Recognition
  • Health Informatics

Cite this

Bae, J., Chhatbar, P., Francis, J. T., Sanchez, J. C., & Principe, J. C. (2011). Reinforcement learning via kernel temporal difference. In 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011 (pp. 5662-5665). [6091370] (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). https://doi.org/10.1109/IEMBS.2011.6091370