Coadaptive brain-machine interface via reinforcement learning

Jack DiGiovanna, Babak Mahmoudi, Jose Fortes, Jose C. Principe, Justin C. Sanchez

Research output: Contribution to journalArticle

96 Citations (Scopus)

Abstract

This paper introduces and demonstrates a novel brain-machine interface (BMI) architecture based on the concepts of reinforcement learning (RL), coadaptation, and shaping. RL allows the BMI control algorithm to learn to complete tasks from interactions with the environment, rather than an explicit training signal. Coadaption enables continuous, synergistic adaptation between the BMI control algorithm and BMI user working in changing environments. Shaping is designed to reduce the learning curve for BMI users attempting to control a prosthetic. Here, we present the theory and in vivo experimental paradigm to illustrate how this BMI learns to complete a reaching task using a prosthetic arm in a 3-D workspace based on the user's neuronal activity. This semisupervised learning framework does not require user movements. We quantify BMI performance in closed-loop brain control over six to ten days for three rats as a function of increasing task difficulty. All three subjects coadapted with their BMI control algorithms to control the prosthetic significantly above chance at each level of difficulty.

Original languageEnglish
Pages (from-to)54-64
Number of pages11
JournalIEEE Transactions on Biomedical Engineering
Volume56
Issue number1
DOIs
StatePublished - Jan 1 2009
Externally publishedYes

Fingerprint

Reinforcement learning
Brain
Prosthetics
User interfaces
Rats

Keywords

  • Brain-machine interface (BMI)
  • Coadaptation
  • Neuroprosthetic
  • Reinforcement learning (RL)

ASJC Scopus subject areas

  • Biomedical Engineering

Cite this

DiGiovanna, J., Mahmoudi, B., Fortes, J., Principe, J. C., & Sanchez, J. C. (2009). Coadaptive brain-machine interface via reinforcement learning. IEEE Transactions on Biomedical Engineering, 56(1), 54-64. https://doi.org/10.1109/TBME.2008.926699

Coadaptive brain-machine interface via reinforcement learning. / DiGiovanna, Jack; Mahmoudi, Babak; Fortes, Jose; Principe, Jose C.; Sanchez, Justin C.

In: IEEE Transactions on Biomedical Engineering, Vol. 56, No. 1, 01.01.2009, p. 54-64.

Research output: Contribution to journalArticle

DiGiovanna, J, Mahmoudi, B, Fortes, J, Principe, JC & Sanchez, JC 2009, 'Coadaptive brain-machine interface via reinforcement learning', IEEE Transactions on Biomedical Engineering, vol. 56, no. 1, pp. 54-64. https://doi.org/10.1109/TBME.2008.926699
DiGiovanna, Jack ; Mahmoudi, Babak ; Fortes, Jose ; Principe, Jose C. ; Sanchez, Justin C. / Coadaptive brain-machine interface via reinforcement learning. In: IEEE Transactions on Biomedical Engineering. 2009 ; Vol. 56, No. 1. pp. 54-64.
@article{8b8bafc8d46b48aa82e985d702795646,
title = "Coadaptive brain-machine interface via reinforcement learning",
abstract = "This paper introduces and demonstrates a novel brain-machine interface (BMI) architecture based on the concepts of reinforcement learning (RL), coadaptation, and shaping. RL allows the BMI control algorithm to learn to complete tasks from interactions with the environment, rather than an explicit training signal. Coadaption enables continuous, synergistic adaptation between the BMI control algorithm and BMI user working in changing environments. Shaping is designed to reduce the learning curve for BMI users attempting to control a prosthetic. Here, we present the theory and in vivo experimental paradigm to illustrate how this BMI learns to complete a reaching task using a prosthetic arm in a 3-D workspace based on the user's neuronal activity. This semisupervised learning framework does not require user movements. We quantify BMI performance in closed-loop brain control over six to ten days for three rats as a function of increasing task difficulty. All three subjects coadapted with their BMI control algorithms to control the prosthetic significantly above chance at each level of difficulty.",
keywords = "Brain-machine interface (BMI), Coadaptation, Neuroprosthetic, Reinforcement learning (RL)",
author = "Jack DiGiovanna and Babak Mahmoudi and Jose Fortes and Principe, {Jose C.} and Sanchez, {Justin C.}",
year = "2009",
month = "1",
day = "1",
doi = "10.1109/TBME.2008.926699",
language = "English",
volume = "56",
pages = "54--64",
journal = "IEEE Transactions on Biomedical Engineering",
issn = "0018-9294",
publisher = "IEEE Computer Society",
number = "1",

}

TY - JOUR

T1 - Coadaptive brain-machine interface via reinforcement learning

AU - DiGiovanna, Jack

AU - Mahmoudi, Babak

AU - Fortes, Jose

AU - Principe, Jose C.

AU - Sanchez, Justin C.

PY - 2009/1/1

Y1 - 2009/1/1

N2 - This paper introduces and demonstrates a novel brain-machine interface (BMI) architecture based on the concepts of reinforcement learning (RL), coadaptation, and shaping. RL allows the BMI control algorithm to learn to complete tasks from interactions with the environment, rather than an explicit training signal. Coadaption enables continuous, synergistic adaptation between the BMI control algorithm and BMI user working in changing environments. Shaping is designed to reduce the learning curve for BMI users attempting to control a prosthetic. Here, we present the theory and in vivo experimental paradigm to illustrate how this BMI learns to complete a reaching task using a prosthetic arm in a 3-D workspace based on the user's neuronal activity. This semisupervised learning framework does not require user movements. We quantify BMI performance in closed-loop brain control over six to ten days for three rats as a function of increasing task difficulty. All three subjects coadapted with their BMI control algorithms to control the prosthetic significantly above chance at each level of difficulty.

AB - This paper introduces and demonstrates a novel brain-machine interface (BMI) architecture based on the concepts of reinforcement learning (RL), coadaptation, and shaping. RL allows the BMI control algorithm to learn to complete tasks from interactions with the environment, rather than an explicit training signal. Coadaption enables continuous, synergistic adaptation between the BMI control algorithm and BMI user working in changing environments. Shaping is designed to reduce the learning curve for BMI users attempting to control a prosthetic. Here, we present the theory and in vivo experimental paradigm to illustrate how this BMI learns to complete a reaching task using a prosthetic arm in a 3-D workspace based on the user's neuronal activity. This semisupervised learning framework does not require user movements. We quantify BMI performance in closed-loop brain control over six to ten days for three rats as a function of increasing task difficulty. All three subjects coadapted with their BMI control algorithms to control the prosthetic significantly above chance at each level of difficulty.

KW - Brain-machine interface (BMI)

KW - Coadaptation

KW - Neuroprosthetic

KW - Reinforcement learning (RL)

UR - http://www.scopus.com/inward/record.url?scp=60549097572&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=60549097572&partnerID=8YFLogxK

U2 - 10.1109/TBME.2008.926699

DO - 10.1109/TBME.2008.926699

M3 - Article

C2 - 19224719

AN - SCOPUS:60549097572

VL - 56

SP - 54

EP - 64

JO - IEEE Transactions on Biomedical Engineering

JF - IEEE Transactions on Biomedical Engineering

SN - 0018-9294

IS - 1

ER -