An actor-critic architecture and simulator for goal-directed brain-machine interfaces

Babak Mahmoudi, Jose C. Principe, Justin C. Sanchez

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

The Perception-Action Cycle (PAC) is a central component of goal-directed behavior because it links internal percepts with external outcomes in the environment. Using inspiration from the PAC, we are developing a Brain-Machine Interface control architecture that utilizes both motor commands and goal information directly from the brain to navigate to novel targets in an environment. An Actor-Critic algorithm was selected for decoding the neural motor commands because it is a PAC-based computational framework where the perception component is implemented in the critic structure and the actor is responsible for taking actions. We develop in this work a biologically realistic simulator to analyze the performance of the decoder in terms of convergence and target acquisition. Experience from the simulator will guide parameter selection and assist in understanding the architecture before animal experiments. By varying the signal to noise ratio of the neural input and error signal, we were able to demonstrate how the learning rate and initial conditions affect a motor control target selection task. In this framework, the naïve decoder was able to reach targets in the presence of noise in the error signal and neural motor command with 98% accuracy.

Original languageEnglish (US)
Title of host publicationProceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society
Subtitle of host publicationEngineering the Future of Biomedicine, EMBC 2009
PublisherIEEE Computer Society
Pages3365-3368
Number of pages4
ISBN (Print)9781424432967
DOIs
StatePublished - Jan 1 2009
Event31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009 - Minneapolis, MN, United States
Duration: Sep 2 2009Sep 6 2009

Publication series

NameProceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009

Other

Other31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009
CountryUnited States
CityMinneapolis, MN
Period9/2/099/6/09

ASJC Scopus subject areas

  • Cell Biology
  • Developmental Biology
  • Biomedical Engineering
  • Medicine(all)

Fingerprint Dive into the research topics of 'An actor-critic architecture and simulator for goal-directed brain-machine interfaces'. Together they form a unique fingerprint.

Cite this