Kernel temporal differences for neural decoding

Jihye Bae, Luis G. Sanchez Giraldo, Eric A. Pohlmeyer, Joseph T. Francis, Justin C. Sanchez, José C. Príncipe

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces.

Original languageEnglish (US)
Article number481375
JournalComputational Intelligence and Neuroscience
Volume2015
DOIs
StatePublished - 2015

Fingerprint

Decoding
kernel
Learning
Reinforcement learning
Reinforcement Learning
Positive Definite Kernels
Proper Mapping
Brain-Computer Interfaces
Evaluation
Strictly positive
Value Function
Feasibility Studies
Learning algorithms
Closed-loop
Learning Algorithm
Computational complexity
Brain
Computational Complexity
High-dimensional
Visualization

ASJC Scopus subject areas

  • Computer Science(all)
  • Mathematics(all)
  • Neuroscience(all)

Cite this

Bae, J., Sanchez Giraldo, L. G., Pohlmeyer, E. A., Francis, J. T., Sanchez, J. C., & Príncipe, J. C. (2015). Kernel temporal differences for neural decoding. Computational Intelligence and Neuroscience, 2015, [481375]. https://doi.org/10.1155/2015/481375

Kernel temporal differences for neural decoding. / Bae, Jihye; Sanchez Giraldo, Luis G.; Pohlmeyer, Eric A.; Francis, Joseph T.; Sanchez, Justin C.; Príncipe, José C.

In: Computational Intelligence and Neuroscience, Vol. 2015, 481375, 2015.

Research output: Contribution to journalArticle

Bae, J, Sanchez Giraldo, LG, Pohlmeyer, EA, Francis, JT, Sanchez, JC & Príncipe, JC 2015, 'Kernel temporal differences for neural decoding', Computational Intelligence and Neuroscience, vol. 2015, 481375. https://doi.org/10.1155/2015/481375
Bae J, Sanchez Giraldo LG, Pohlmeyer EA, Francis JT, Sanchez JC, Príncipe JC. Kernel temporal differences for neural decoding. Computational Intelligence and Neuroscience. 2015;2015. 481375. https://doi.org/10.1155/2015/481375
Bae, Jihye ; Sanchez Giraldo, Luis G. ; Pohlmeyer, Eric A. ; Francis, Joseph T. ; Sanchez, Justin C. ; Príncipe, José C. / Kernel temporal differences for neural decoding. In: Computational Intelligence and Neuroscience. 2015 ; Vol. 2015.
@article{7ddf4d51d80f4122874e6f0b3a95aee8,
title = "Kernel temporal differences for neural decoding",
abstract = "We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces.",
author = "Jihye Bae and {Sanchez Giraldo}, {Luis G.} and Pohlmeyer, {Eric A.} and Francis, {Joseph T.} and Sanchez, {Justin C.} and Pr{\'i}ncipe, {Jos{\'e} C.}",
year = "2015",
doi = "10.1155/2015/481375",
language = "English (US)",
volume = "2015",
journal = "Computational Intelligence and Neuroscience",
issn = "1687-5265",
publisher = "Hindawi Publishing Corporation",

}

TY - JOUR

T1 - Kernel temporal differences for neural decoding

AU - Bae, Jihye

AU - Sanchez Giraldo, Luis G.

AU - Pohlmeyer, Eric A.

AU - Francis, Joseph T.

AU - Sanchez, Justin C.

AU - Príncipe, José C.

PY - 2015

Y1 - 2015

N2 - We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces.

AB - We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces.

UR - http://www.scopus.com/inward/record.url?scp=84926372250&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84926372250&partnerID=8YFLogxK

U2 - 10.1155/2015/481375

DO - 10.1155/2015/481375

M3 - Article

C2 - 25866504

AN - SCOPUS:84926372250

VL - 2015

JO - Computational Intelligence and Neuroscience

JF - Computational Intelligence and Neuroscience

SN - 1687-5265

M1 - 481375

ER -