Humans and other animate creatures survive in an environment by continually sensing and acting. Illness or injury however may impair or destroy the neural pathways connecting the brain with the external world. These include auditory and visual impairments as well as motor impairment due to stroke, spinal cord injury, Amyotrophic Lateral Sclerosis, or Multiple Sclerosis. There are 250,000 cases of spinal cord injury alone in the United States of America with 11,000 new cases each year.This work is supported by the NSF/NIH Collaborative Research in Computational Neuroscience Program (1R01NS050967-01).
Research on neural prostheses seeks an engineering solution to restoring lost function by providing new, alternate, pathways which restore, to varying degrees, the ability to sense and act on the world. While neural prosthetic research takes many forms, our work focuses on direct cortical control of external devices such as computer displays and robots.
Building a direct, artificial, connection between the brain and the world, requires answers to the following questions:
1. What "signals" can we measure from the brain? From what regions? With what technology?
2. How is information represented (or encoded) in the brain?
3. What algorithms can we use to infer (or decode) the internal ``state'' of the brain?
4. How can we build practical interfaces that exploit the available technology?Current technology allows simultaneous recording from hundreds of cells in the brains of awake, behaving animals. Using this information to understand how the brain represents and processes complex information will enable neural prosthetic technologies that promise a new generation of therapies for the severely disabled. To that end we are developing new mathematical and computational methods for modeling and decoding neural activity. The project has goals:
1) We are developing new probabilistic models of the neural code that exploit machine learning methods and high performance computing resources. These models represent the probabilistic relationship between multiple behavioral variables and the firing activity of a large population of neurons.
2) Neural decoding methods are being developed that model the uncertainty in neural recordings to make sound inferences that can drive neural prostheses.
3) Adaptation and learning of cells in the brain is being studied using the mathematical models developed here and an understanding of this adaptation will be used to design new algorithms for prosthetic applications.
To learn high dimensional probabilistic models from vast amounts of neural data, a new class of mathematical and computational tools is required. In particular we are developing “maximum entropy” methods to learn non-parametric statistical models of high-dimensional, correlated neural data. We have also developed mathematical models to determine the efficacy of neural activity patterns in inducing learning. These new methods address fundamental problems in neural coding that will enable new prosthetic devices through a tight cycle of hypothesis, development, testing, and validation.
Our work exploits neural activity recorded in primary motor cortex using an array of chronically implanted microelectrodes. We adopt a probabilistic formulation of the encoding/decoding problem and have developed a variety of real-time methods for reconstructing hand motion from neural activity. This reconstruction is sufficiently accurate to permit the control of unconstrained 2D cursor movement or simple robotic functions.
Modeling Neural Activity as a Product of Experts
Modeling the information processing performed by a neuronal population is a challenging problem. For a complex task or stimulus and a large population of cells, the dimensionality of the problem is dauntingly large. In any realistic scenario, there will be insufficient experimental data to fully populate this high dimensional space with training examples. Thus, naive approaches for learning probabilistic models relating population activity to the task variables will fail. The standard approach for dealing with this problem is to exploit parameterized models with fewer degrees of freedom. For example, it is common to assume a linear relationship between a cell's ring rate and some task variable. Moreover, the stochastic nature of cell's behavior is often modeled in terms of a parameterized distribution (e.g. Poisson or Gaussian). We know that these assumptions, however, are simplifications. Moving beyond them to uncover the more complex relationships between tasks and the activity of a population of neurons requires new representations and new computational tools. To that end we have developed a non-parametric model that represents the joint probability of neural activity and behavior as a product of experts. This "energy-based" model is learned from training data using a technique called contrastive divergence. By using many "experts" we are able to model complex, high-dimensional, correlated, non-Gaussian probability distributions. These general learning techniques are applicable in many domains beyond neural modeling. We are currently developing decoding algorithms to exploit these models with the goal of improving the accuracy of neural prosthetic control.
Investigators
Michael J. Black, Department of Computer Science, Brown University
Elie Bienenstock, Departments of Applied Math and Neuroscience, Brown University.
John Donoghue, Department of Neuroscience, Brown University.
Mayank Mehta, Department of Neuroscience, Brown University.Post Doctoral Researchers
Former Post Docs
Luk Chong Yeung, Neuroscience and Physics, Brown University.
Students
Frank Wood, Department of Computer Science, Brown University.
Stefan Roth, Department of Computer Science, Brown University.Project Publications
Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia,
Kim, S.-P., Simeral, J., Hochberg, L., Donoghue, J. P., Friehs, G., Black, M. J.,
The 3rd International IEEE EMBS Conference on Neural Engineering, pp. 486-489, May 2-5, 2007.
(pdf)
Decoding grasp aperture from motor-cortical population activity,
Artemiadis, P., Shakhnarovich, G., Vargas-Irwin, C., Black, M. J., Donoghue, J. P.,
The 3rd International IEEE EMBS Conference on Neural Engineering, pp. 518-521, May 2-5, 2007.
(pdf)Nonlinear physically-based models for decoding motor-cortical population activity,
Shakhnarovich, G., Kim, S.-P. and Black, M. J.,
to appear: Advances in Neural Information Processing Systems, NIPS-2006.
(pdf)A non-parametric Bayesian approach to spike sorting,
Wood, F., Goldwater, S., and Black, M. J.,
International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, Aug-Sep, pp. 1165-1169, 2006.
(pdf)Bayesian population coding of motor cortical activity using a Kalman filter.
Wu, W., Gao, Y., Bienenstock, E., Donoghue, J. P., Black, M. J.,
Neural Computation, 18:80-118, 2005.
(pdf preprint), (pdf from publisher)Role of rhythms in facilitating short-term memory.
Mehta, M.,
Neuron 45, 7:9 (2005).
(pdf)Statistical analysis of the non-stationarity of neural population codes.
Kim, S.-P., Wood, F., Fellows, M., Donoghue, J. P., Black, M. J.,
BioRob 2006, The first IEEE / RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 295-299, Pisa, Italy, Feb. 2006.
(pdf)Modeling neural population spiking activity with Gibbs distributions.
Wood, F., Roth, S., and Black, M. J.,
Advances in Neural Information Processing Systems, 18, pp. 1537-1544, 2005.
(pdf)On the spatial statistics of optical flow,
(Marr Prize, Honorable Mention.)
Roth, S., Black, M. J.,
International Conf. on Computer Vision, ICCV, pp. 42-49, 2005.
(pdf)Fields of experts: A framework for learning image priors with applications.
Roth, S. and Black, M. J.,
IEEE Conf. on Computer Vision and Pattern Recognition, vol. II, pp. 860-867, June 2005.
(pdf)Related Publications
Assistive technology and robotic control using MI ensemble-based neural interface systems in humans with tetraplegia,
Donoghue, J. P., Nurmikko, A., Black, M., J., and Hochberg, L.,
Journal of Physiology, Special Issue on Brain Computer Interfaces, 579:603-611, 2007.
(pdf preprint) (pdf from publisher)
Neuromotor prosthesis development,
Donoghue, J.P., Hochberg, L.R., Nurmikko, A.V., Black, M.J., Simeral, J.D., and Friehs, G.,
Medicine & Health Rhode Island, Vol. 90, No. 1, pp. 12-15, Jan. 2007.
(pdf reprint)Probabilistically modeling and decoding neural population activity in motor cortex
Black, M. J. and Donoghue, J. P.,
in Towards Brain-Computer Interfacing, G. Dornhege, J. del R. Mill´an, T. Hinterberger, D. McFarland, K.-R. Muller (eds.), MIT Press, in press.Inferring attentional state and kinematics from motor cortical firing rates,
Wood, F., Prabhat, Donoghue, J. P., Black, M. J.,
Proc. IEEE Engineering in Medicine and Biology Society, pp 1544-1547, Sept. 2005.
(pdf)Motor cortical decoding using an autoregressive moving average model,
Fisher, J and Black, M. J.,
Proc. IEEE Engineering in Medicine and Biology Society, pp. 1469-1472, Sept. 2005.
(pdf)Development of neural motor prostheses for humans.
Donoghue, J., Nurmikko, A., Friehs, G., Black, M.,
(Invited paper) Advances in Clinical Neurophysiology (Supplements to Clinical Neurophysiology, Vol. 57) Editors: M. Hallett, L.H. Phillips, II, D.L. Schomer, J.M. Massey. 2004.
(pdf).On the variability of manual spike sorting,
Wood, F., Black, M. J., Vargas-Irwin, C., Fellows, M., Donoghue, J. P.,
IEEE Trans. Biomedical Engineering, 51(6):912-918, June 2004.
(pdf).Modeling and decoding motor cortical activity using a switching Kalman filter,
Wu, W., Black, M. J., Mumford, D., Gao, Y., Bienenstock, E., Donoghue, J. P.,
IEEE Trans. Biomedical Engineering, 51(6):933-942, June 2004.
(pdf).Closed-Loop Neural Control of Cursor Motion using a Kalman Filter,
Wu, W., Shaikhouni, A., Donoghue, J. P., Black, M.J.,
Proc. IEEE Engineering in Medicine and Biology Society, pp. 4126-4129, Sept. 2004.
(pdf).Automatic Spike Sorting for Neural Decoding,
Wood, F. D., Fellows, M., Donoghue, J. P., Black, M. J.,
Proc. IEEE Engineering in Medicine and Biology Society, pp. 4009-4012, Sept. 2004.
(pdf).A switching Kalman filter model for the motor cortical coding of hand motion,
Wu, W., Black, M. J., Mumford, D., Gao, Y., Bienenstock, E., Donoghue, J. P.,
Proc. IEEE Engineering in Medicine and Biology Society, pp. 2083-2086, Sept. 2003
(pdf).Connecting brains with machines: The neural control of 2D cursor movement,
Black, M. J., Bienenstock, E., Donoghue, J. P., Serruya, M., Wu, W., Gao, Y.,
1st International IEEE/EMBS Conference on Neural Engineering, pp. 580-583, Capri, Italy, March 20-22, 2003.
(pdf).A quantitative comparison of linear and non-linear models of motor cortical activity for the encoding and decoding of arm motions,
Gao, Y., Black, M. J., Bienenstock, E., Wu, W., Donoghue, J. P.,
1st International IEEE/EMBS Conference on Neural Engineering, pp. 189-192, Capri, Italy, March 20-22, 2003.
(pdf).Neural decoding of cursor motion using a Kalman filter,
Wu, W., Black, M. J., Gao, Y., Bienenstock, E., Serruya, M., Shaikhouni, A., Donoghue, J. P.,
Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun and K. Obermayer (Eds.), MIT Press, pp. 117-124, 2003.
(pdf).Inferring hand motion from multi-cell recordings in motor cortex using a Kalman filter,
Wu, W., Black, M. J., Gao, Y., Bienenstock, E., Serruya, M., and Donoghue, J. P.,
SAB'02-Workshop on Motor Control in Humans and Robots: On the Interplay of Real Brains and Artificial Devices, August 10, 2002, Edinburgh, Scotland (UK), pp. 66-73.
(abstract), (postscript), (pdf).Probabilistic inference of arm motion from neural activity in motor cortex,
Gao, Y., Black, M. J., Bienenstock, E., Shoham, S., Donoghue, J.,
Advances in Neural Information Processing Systems 14, T. G. Dietterich, S. Becker, and Z. Ghahramani (Eds.), pp. 221-228, MIT Press, 2002.
(abstract), (postscript), (pdf).Encoding/decoding of arm kinematics from simultaneously recorded MI neurons,
Gao, Y., Bienenstock, E., Black, M., Shoham, S., Serruya, M., Donoghue, J.,
Society for Neuroscience Abst. Vol. 27, Program No. 572.14 2001.
(abstract)Related Talks
Models of Neural Coding in Motor Cortex and their Application to Neural Prostheses. plenary talk, Workshop on Neural Coding, Mathematical Biosciences Institute, The Ohio State University, February 2003.Connecting Brains with Machines: Towards the Neural Control of 2D Cursor Movement. Invited talk, AI Lab, MIT, October 2002.
Related Courses
Topics in Brain Computer Interfaces (CS295-07)Disclaimer
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or National Institutes of Health.