Towards autonomous neuroprosthetic control using Hebbian reinforcement learning

Babak Mahmoudi, Eric A. Pohlmeyer, Noeline W. Prins, Shijia Geng, Justin C. Sanchez

Research output: Contribution to journalArticle

21 Scopus citations

Abstract

Objective. Our goal was to design an adaptive neuroprosthetic controller that could learn the mapping from neural states to prosthetic actions and automatically adjust adaptation using only a binary evaluative feedback as a measure of desirability/undesirability of performance. Approach. Hebbian reinforcement learning (HRL) in a connectionist network was used for the design of the adaptive controller. The method combines the efficiency of supervised learning with the generality of reinforcement learning. The convergence properties of this approach were studied using both closed-loop control simulations and open-loop simulations that used primate neural data from robot-assisted reaching tasks. Main results. The HRL controller was able to perform classification and regression tasks using its episodic and sequential learning modes, respectively. In our experiments, the HRL controller quickly achieved convergence to an effective control policy, followed by robust performance. The controller also automatically stopped adapting the parameters after converging to a satisfactory control policy. Additionally, when the input neural vector was reorganized, the controller resumed adaptation to maintain performance. Significance. By estimating an evaluative feedback directly from the user, the HRL control algorithm may provide an efficient method for autonomous adaptation of neuroprosthetic systems. This method may enable the user to teach the controller the desired behavior using only a simple feedback signal.

Original languageEnglish (US)
Article number066005
JournalJournal of Neural Engineering
Volume10
Issue number6
DOIs
StatePublished - Dec 1 2013

    Fingerprint

ASJC Scopus subject areas

  • Biomedical Engineering
  • Cellular and Molecular Neuroscience

Cite this

Mahmoudi, B., Pohlmeyer, E. A., Prins, N. W., Geng, S., & Sanchez, J. C. (2013). Towards autonomous neuroprosthetic control using Hebbian reinforcement learning. Journal of Neural Engineering, 10(6), [066005]. https://doi.org/10.1088/1741-2560/10/6/066005