A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces

Jihye Bae, Luis G. Sanchez Giraldo, Eric A. Pohlmeyer, Justin C. Sanchez, Jose C. Principe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

This paper presents the first attempt to quantify the individual performance of the subject and of the computer agent on a closed loop Reinforcement Learning Brain Machine Interface (RLBMI). The distinctive feature of the RLBMI architecture is the co-adaptation of two systems (a BMI decoder in agent and a BMI user in environment). In this work, an agent implemented using Q-learning via kernel temporal difference (KTD)(λ) decodes the neural states of a monkey and transforms them into action directions of a robotic arm. We analyze how each participant influences the overall performance both in successful and missed trials by visualizing states, corresponding action value Q, and resulting actions in two-dimensional space. With the proposed methodology, we can observe how the decoder effectively learns a good state to action mapping, and how neural states affect the prediction performance.

Original languageEnglish
Title of host publicationProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Pages5402-5405
Number of pages4
DOIs
StatePublished - Oct 31 2013
Event2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2013 - Osaka, Japan
Duration: Jul 3 2013Jul 7 2013

Other

Other2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2013
CountryJapan
CityOsaka
Period7/3/137/7/13

Fingerprint

Brain-Computer Interfaces
Brain
Reinforcement
Reinforcement learning
Learning
Robotic arms
Robotics
Haplorhini
Reinforcement (Psychology)

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Signal Processing
  • Biomedical Engineering
  • Health Informatics

Cite this

Bae, J., Sanchez Giraldo, L. G., Pohlmeyer, E. A., Sanchez, J. C., & Principe, J. C. (2013). A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (pp. 5402-5405). [6610770] https://doi.org/10.1109/EMBC.2013.6610770

A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces. / Bae, Jihye; Sanchez Giraldo, Luis G.; Pohlmeyer, Eric A.; Sanchez, Justin C.; Principe, Jose C.

Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. 2013. p. 5402-5405 6610770.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Bae, J, Sanchez Giraldo, LG, Pohlmeyer, EA, Sanchez, JC & Principe, JC 2013, A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces. in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS., 6610770, pp. 5402-5405, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2013, Osaka, Japan, 7/3/13. https://doi.org/10.1109/EMBC.2013.6610770
Bae J, Sanchez Giraldo LG, Pohlmeyer EA, Sanchez JC, Principe JC. A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. 2013. p. 5402-5405. 6610770 https://doi.org/10.1109/EMBC.2013.6610770
Bae, Jihye ; Sanchez Giraldo, Luis G. ; Pohlmeyer, Eric A. ; Sanchez, Justin C. ; Principe, Jose C. / A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. 2013. pp. 5402-5405
@inproceedings{95862a6f8cb845c483ab3dc5e6e9a8ba,
title = "A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces",
abstract = "This paper presents the first attempt to quantify the individual performance of the subject and of the computer agent on a closed loop Reinforcement Learning Brain Machine Interface (RLBMI). The distinctive feature of the RLBMI architecture is the co-adaptation of two systems (a BMI decoder in agent and a BMI user in environment). In this work, an agent implemented using Q-learning via kernel temporal difference (KTD)(λ) decodes the neural states of a monkey and transforms them into action directions of a robotic arm. We analyze how each participant influences the overall performance both in successful and missed trials by visualizing states, corresponding action value Q, and resulting actions in two-dimensional space. With the proposed methodology, we can observe how the decoder effectively learns a good state to action mapping, and how neural states affect the prediction performance.",
author = "Jihye Bae and {Sanchez Giraldo}, {Luis G.} and Pohlmeyer, {Eric A.} and Sanchez, {Justin C.} and Principe, {Jose C.}",
year = "2013",
month = "10",
day = "31",
doi = "10.1109/EMBC.2013.6610770",
language = "English",
isbn = "9781457702167",
pages = "5402--5405",
booktitle = "Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS",

}

TY - GEN

T1 - A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces

AU - Bae, Jihye

AU - Sanchez Giraldo, Luis G.

AU - Pohlmeyer, Eric A.

AU - Sanchez, Justin C.

AU - Principe, Jose C.

PY - 2013/10/31

Y1 - 2013/10/31

N2 - This paper presents the first attempt to quantify the individual performance of the subject and of the computer agent on a closed loop Reinforcement Learning Brain Machine Interface (RLBMI). The distinctive feature of the RLBMI architecture is the co-adaptation of two systems (a BMI decoder in agent and a BMI user in environment). In this work, an agent implemented using Q-learning via kernel temporal difference (KTD)(λ) decodes the neural states of a monkey and transforms them into action directions of a robotic arm. We analyze how each participant influences the overall performance both in successful and missed trials by visualizing states, corresponding action value Q, and resulting actions in two-dimensional space. With the proposed methodology, we can observe how the decoder effectively learns a good state to action mapping, and how neural states affect the prediction performance.

AB - This paper presents the first attempt to quantify the individual performance of the subject and of the computer agent on a closed loop Reinforcement Learning Brain Machine Interface (RLBMI). The distinctive feature of the RLBMI architecture is the co-adaptation of two systems (a BMI decoder in agent and a BMI user in environment). In this work, an agent implemented using Q-learning via kernel temporal difference (KTD)(λ) decodes the neural states of a monkey and transforms them into action directions of a robotic arm. We analyze how each participant influences the overall performance both in successful and missed trials by visualizing states, corresponding action value Q, and resulting actions in two-dimensional space. With the proposed methodology, we can observe how the decoder effectively learns a good state to action mapping, and how neural states affect the prediction performance.

UR - http://www.scopus.com/inward/record.url?scp=84886542945&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84886542945&partnerID=8YFLogxK

U2 - 10.1109/EMBC.2013.6610770

DO - 10.1109/EMBC.2013.6610770

M3 - Conference contribution

SN - 9781457702167

SP - 5402

EP - 5405

BT - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

ER -