BRAIN - Brain-computer interfaces with Rapid Automated Interfaces for Nonexperts

BCI Tutorial IJCNN2011 : Signal Processing and Machine Learning Approaches in Brain Machine Interfaces



Research on Brain-Machine Interfaces (BMIs) has considerably expanded in the last decade. Such an expansion owes to a large extent to the multidisciplinary and intriguing nature of BMIs which aim at "decoding brain signals to infer the user's intent". Neuro-physiologically inspired signal processing and machine learning are essential pillars to make such decoding possible.

Signal processing algorithms are applied to the brain signals to extract features characterizing the mental activities that the BMI aims at decoding. Machine learning algorithms allow the BMI and its user to adapt to each other by allowing the BMI "to learn" the user particular pattern of brain activity and the user to learn how to modulate her/his brain activity to gain control of the BMI.

This tutorial focuses on the signal processing and machine learning methods that make BMIs possible. The tutorial starts with a basic description of the basic BMI concepts such as brain activity monitoring, a/synchronous operation, and the main neural sources of control (i.e. motor imagery, sensory evoked potentials, and the P300). The processing of brain signals for BMI applications is presented in depth. The multivariate nature of the brain signals combined with neurophysiology knowledge is advantageously taken into account to derive appropriate features to decode motor imagery and attention to sensory stimuli. The machine learning methods in BMIs are subsequently presented to illustrate the mechanisms through which BMI and user mutually adapt. Throughout the tutorial, two aspects are emphasized 1) the practical applicability, and 2) the open research questions.

Tutorial Contents

Part 1. Brain-Machine Interfaces (30 minutes)

    - Brain-Machine Interface: Definition and trends
    - Brain activity monitoring for BMI applications. Time resolution, Spatial resolution, cost, and convenience.
    - Command and Control by merely thinking
    - The machine learns vs. the brain learns

Part 2. Main BMI types (30 minutes)

BMIs can use a variety of neural sources. However, more than 90% of current BCI implementations rely on three main sources. These are presented in this part of the lecture.

    - BMI based on motor imagery
      -- Event related desynchornization/Event related synchronization (ERD/ERS)
      -- Motor imagery and ERD/ERS
      -- Operation of a motor imagery based BMI
    - BMI based on the steady-state visual evoked potential (SSVEP)
      -- Repetitive visual stimuli and EEG
      -- Neurophysiological basis of the SSVEP
      -- Operation of an SSVEP based BMI
    - BMI based on the P300 potential
      -- The oddball paradigm
      -- Single trial P300 detection
      -- Operation of a P300 based BMI

Part 3. Signal Processing and Machine Learning for BMI applications (2 x 45 minutes)

In this part of the lecture, signal processing for BMI is discussed in depth. A common framework to address the detection of patterns related to the three main BMI types is presented.

The tutorial ends with a brief overview of the open research questions and a roadmap towards mainstream adoption.

    - Signal processing. How the requirements on BMI applications differ from that of clinical diagnostics
    - Variability: within session, between session, between participants
    - A common framework to detect relevant ERD/ERS, SSVEP, P300
      -- Common spatial patterns and variations for ERD/ERS characterization.
      -- Spatial filters to detect SSVEPs.
      -- Single trial P300 detection.
    - Challenges for future research

Target Audience

The subjects in this tutorial are comprehensibly presented and are thus suitable for a wide range of scientists. The approaches, methods, and algorithms in this tutorial do not limit to the BMI realm. Indeed, they can be readily applied to domains where other physiological signals are used.

BMI is a multidisciplinary field comprising neurophysiology, signal processing, machine learning, user interaction, and information theory. This tutorial is prepared in such a way as to clearly introduce each concept and its relevance for BMI.


Dr. Gary Garcia Molina has been active in BMI research for a decade. In 2004, he obtained his doctoral degree from the Swiss Federal Institute of Technology Lausanne, Switzerland (EPFL). His thesis entitled "Direct Brain-Computer Communication Through Scalp Recorded EEG Signals" was nominated for the Swiss best thesis award.

Since January 2005, Dr. Garcia is a Senior Researcher at Philips Research Europe Laboratories (Eindhoven, The Netherlands) where he leads a research activity aiming at developing practical Brain-Computer Interfaces for the consumer market. From 2007, Gary Garcia leads the Philips team that participates in the European consortium BRAIN ( which has as main objective the development of BMIs able to automatically adapt to the user and his/her environment, and do not require any expert assistance.

Gary Garcia serves as an adviser and project reviewer for the European-research program ICT (Information and Communication Technologies) on Inclusion and Independent Living.

Dr. Garcia published numerous research papers, and holds four patents on BMI related technology. He has extensive experience with several types of BMIs and he cooperates with numerous institutions in academia and industry to develop BMI technology into home appliances for the disabled and the healthy. From 2008 up to present, Gary Garcia gave several tutorial lectures on BMI technology at different conferences including: IEEE/BiOCAS 2008, ACII 2009 , EUSIPCO 2010 , ISSPA 2010 , and SPPRA 2011 .


    [1] G. Garcia-Molina and V. Mihajlovic, “Spatial Filters to detect Steady State Visual Evoked Potentials elicited by high frequency stimulation: BCI application," Journal of Biomedizinische Technik / Biomedical Engineering, vol. 55, no. 3, pp. 173-182, 2010.
    [2] G. Garcia-Molina, T. Tsoneva, and A. Nijholt, “Emotional brain computer interfaces, " Int. J. Autonomous and Adaptive Communications Systems, vol. In press, 2010.
    [3] D. Zhu, J. Bieger, G. Garcia-Molina, and R. Aarts, “A survey of stimulation methods used in SSVEP-based BCIs," Journal of Computational Intelligence and Neuroscience, vol. 2010, pp. 1-12, 2010.
    [4] G. Garcia-Molina, T. Ebrahimi, and J.-M. Vesin, “Joint Time-Frequency-Space Classification of EEG in a Brain-Computer Interface Application," EURASIP Journal on Applied Signal Processing, vol. 2003, no. 7, pp. 713-729, 2003.
    [5] T. Ebrahimi, J.-M. Vesin, and G. Garcia-Molina, “Brain-Computer Interface in Multimedia Communication," IEEE Signal Processing Magazine, vol. 20, no. 1, pp. 14-24, 2003.
    [6] G. Garcia-Molina, D. Zhu, “Optimal Spatial Filtering for the Steady State Visual Evoked Potential: BCI application,” in Proceedings of the 5th International IEEE EMBS Conference on Neural Engineering, 2011
    [7] G. Garcia-Molina, D. Zhu, V. Mihajlovic, and R. Aarts, “Phase detection of visual evoked potentials for brain computer interfaces," in Proceedings of the 14th Int. Conf. on Human-Computer Interaction, 2011.
    [8] G. Garcia-Molina, D. Zhu, and S. Abtahi, “Phase Detection in a Visual-Evoked-Potential Based Brain Computer Interface," in Proceedings of the 18th European Signal Processing Conference (EUSIPCO 2010), 2010
    [9] G. Garcia-Molina and D. Zhu, “Towards a Practical SSVEP based BCI," in Proceedings of the Fourth International BCI Meeting, 2010
    [10] G. Garcia-Molina, T. Tsoneva, and A. Nijholt, “Emotional brain-computer interfaces," in Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction ACII, 2009