Application of BCI in Mind Controlled Robotic Movements in Intelligent Rehabilitation

Abstract

Driven by society’s growing recognition of the needs of physically challenged people, this paper aims at providing a Smart BCI based solution for the realization of intelligent rehabilitation technique. This system has extremely high impact especially for otherwise critical; Stroke affected and physically challenged individuals. An application that is capable of helping them, perform independently, the essential functions as and when they think is presented here. The system briefed uses a new way (CSSP algorithm) of design thereby overcoming the limitations of conventional systems those uses CSP algorithm. BCI’s users explicitly manipulate their brain activity instead of motor movements. The emphasis is on the development of Non-invasive BCIs that uses EEG signal in order to make the final solution completely robust. Following a general overview, suitable signals, the necessary conditioning techniques and most efficient CSSP method of determining the user’s intention are presented along with further resources for designing brain-computer interfaces.

INDEX TERMS

BCI-Brain Computer Interfacing, CSSP- Combined Spatial Spectral Processing, EEG-Electro EncephaloGram, Phase synchronization, Common Spatial Patterns (CSP).

I.INTRODUCTION

In recent years, there has been an increasing interest in means to enable the control of a computer using thought. These setups for brain-computer interfacing (BCI) have application potential in several areas ranging from gaming and virtual reality to prosthesis control and a communication medium for the paralyzed. The use of noninvasive EEG signals for BCI is one of the most practical methods for signal acquisition from the brain, being a relatively low cost solution when compared to other methods for recording brain activity such as MagnetoEncephaloGraphy (MEG) and functional magnetic resonance imaging (fMRI). Furthermore, the EEG recording provides a good temporal resolution and is not an invasive procedure. Computational method to extract features from the EEG for BCI mental task discrimination is focussed here. Mental tasks can be characterized by neuronal activity in specific spatial locations. Contralateral event related desynchronization (ERD) exhibited during motor imagery tasks is one such typical example.

It is accompanied by excitation and inhibition of distributed neural structures or networks. Additionally, knowing the user’s state as well as the tasks they’re performing might provide key information that would help us design context sensitive systems that adapt themselves for optimal user support. This could prove useful to healthy users also who might be situationally disabled.

How is it possible to control a robot that must make accurate turns at precise moments using signals that arrive at a rate of about one bit per second? The key aspect of our brain-actuated robots is combining the subject’s mental capabilities with the robot’s intelligence.

That is, the subject delivers a few high-level mental commands (for example, “Turn right at the next occasion”), and the robot executes these commands autonomously using the readings of its onboard sensors. In other words, the EEG conveys the subject’s intent, and the robot performs it. This approach makes it possible to continuously control a mobile robot—emulating a motorized wheelchair—along nontrivial trajectories requiring fast and frequent switches between mental task discrimination.

II.SYSTEM OVERVIEW

Human brain consists of millions of neurons, individual nerve cells connected to one another by dendrites and axons. Every time we think, move, feel or remember something, our neurons are at work. That work is carried out by small electric signals that zip from neuron to neuron as fast as 250 mph. The signals are generated by differences in electric potential carried by ions on the membrane of each neuron. Detecting these signals can help interpret what they mean and use them to direct a device of some kind.

a. Brain Computer Interfacing

A brain-computer interface (BCI) is a novel system that translates human thoughts or intentions into a control signal. Using this technique, a BCIS can provide a new non muscular communication channel that could be used for variety of applications, such as assisting people with severe motor disabilities; supporting biofeedback training in people suffering from epilepsy, stroke, or Attention Deficit Hyperactivity Disorders.

The electrodes measure minute differences in the voltage between neurons. The signal is then amplified and filtered. In current BCI systems, it is then interpreted by a computer program (P-CSP Algorithm). In the case of a sensory input BCI, the function happens in reverse. A computer converts a signal, such as one from a video camera, into the voltages necessary to trigger neurons. The signals are sent to an implant in the proper area of the brain, if everything works correctly, the neurons fire and the subject receives a visual image corresponding to what the camera sees.

<

One of the most exciting areas of this paper is the development of devices that can be controlled by thoughts. These devices that would allow severely disabled people to function independently. But how do we turn those tiny EEG voltage measurements into the movement of a robotic arm?

A more difficult task is interpreting the brain signals for movement in someone who can't physically move their own arm.

With a task like this in hand, the subject must be trained well to use the device. With an EEG or implant in place, the subject would visualize closing his or her right hand. After many trials, the software can learn the signals associated with the thought of anticipated movements and interpret it to mean the robotic arm or wheelchair movement. Conventionally, from the topographic patterns of brain rhythm modulations the Common Spatial Patterns (CSP) algorithm extracts subject-specific, discriminative parameters. The recently developed CSSP algorithm is employed for this BCI system, in which very simple frequency filters (with one delay tap) for each channel are optimized together with the spatial filters as they outperform the conventional methods.

b. Suitable brain signals:

Invasive BCI methods place electrodes directly on or inside the cortex. One method records electrical potentials for subsequent analysis of the electrocorticogram (ECoG).

Another method places a multiunit electrode array in the cortex to record the neural firing of a small population of neurons. Both signal types have a superior signalto-noise ratio, need little user training, and are suitable for replacing or restoring lost motor functions in patients with damaged parts of the neuronal system

Noninvasive BCIs, on the other hand, can use a variety of brain signals as input, such as electroencephalograms (EEG), magnetoencephalograms (MEG), bloodoxygen- level-dependent (BOLD) signals, and (de)oxyhemoglobin concentrations. In addition to the above mentioned group MEG, NIRS, fMIR are the other sources that can provide the brain signals necessary for this application

<

III.HARDWARE DESIGN

This system consists of the following Functions Signal acquisition, Pre-processing, Feature extraction, classification, Interfacing and the generation of Control signal, Bio feedback.

The signal Classification block performs the comparison of the signal obtained with that of the various signals corresponding to different mental tasks, subject specific data obtained during training, and generates the control signal necessary. Finally the task is realised. Other components and functions are elaborated in the following sections.

IV.SIGNAL ACQUISITION AND PRE-PROCESSING

EEG based BCI system:

The EEG, which is basically the sum of much post synaptic potential in the cortex, is the most widely used brain signal for operating a BCI system. EEG based BCI systems use this type of brain potentials as source signals for the feature extraction and thought discrimination. Two types of changes can be extracted from the ongoing EEG signals. One is time- and phase-locked (evoked) to an externally or internally paced event, while the other is also time-locked but not phase-locked (induced). The former class belong the event-related potentials (ERPs, steady-state visual evoked potentials (Ssveps), and slow cortical negative shifts; and the latter class belong the event-related desynchronizations (ERDs) and event-related synchronizations (ERSs).

<

Slow cortical potentials. These are Scalp-recorded EEG; it is composed of slow changes in voltage, which are thought to originate from dendrites of pyramidal neurons in superficial layers of the cerebral cortex. These voltage changes occur over a timescale of 0.5–10.0 sec and are called “slow cortical potentials” (SCPs). Negative SCPs - movement and other functions involving cortical activation, while positive SCPs are usually associated with reduced cortical activation.

Event-Related Potential-P300:

It is an infrequent or particularly significant stimulus. When interspersed with frequent or routine stimuli, typically evoke a characteristic response in peoples’ EEG. Because it is positive and often occurs with a latency of around 300 ms, this event-related EEG component is known as the “P300.”This signal enables the BCI to identify a series of choices selected by the user. Only the choice desired by the user evokes a large P300.

Sensorimotor rhythms.

When not engaged in processing sensory input or motor output, the sensory or motor cortices of awake people often produce 8–12 Hz EEG activity. This idling activity, called μ rhythm when focused over sensorimotor cortex or visual alpha rhythm when focused over visual cortex, is thought to be produced by thalamocortical circuits. μ rhythms are usually accompanied by 18–26 Hz beta rhythms, some of which have a different topography and/or timing than μ, and thus constitute an independent feature of EEG. One reason these rhythms provide good signals for EEG-based BCIs is their association with the brain’s normal motor output channels.

Bodily movements are typically accompanied by a decrease in μ and beta rhythms, which is greater over the scalp contra-lateral to a moved hand. This decrease has been labeled ‘event-related de-synchronization’ or ERD. Its opposite, an increase in rhythm, or ‘event-related synchronization’ (ERS) occurs after movement and with relaxation. Especially relevant for use in a BCI is that ERD and ERS do not require overt movement, but occur also during motor imagery and preparation..

The plot on the left shows the ERD curve on one projected CSP channel for the subject whose spectra is shown in Fig. 2. After applying the temporal filter calculated by the CSSSP approach the ERD curve on the right is obtained. Below each plot the r -values as a measure of discriminability are visualized. They show that the ERD curves on the right can be discriminated better

<

The below given plot on the left shows the frequency response for one learned frequency filter for the subject whose spectra is shown in Fig. 2.

In the plot on the right, the resulting spectra are visualized after applying the frequency filter. By this technique the classification error could be reduced from 17.3% to 1.4%.

b. Motor Imagery-related EEG pattern recognition:

Due to the oscillatory character of the MI induced brain phenomena, reflected in the sensorimotor EEG activity, a method of spectral analysis was employed in the signal quantification. The EEG rhythmical content within μ (8–12 Hz) and �� (18–25 Hz) ranges was examined in this regard since it reveals lateralized signal power patterns over C3 and C4 locations characteristic of the MI that a subject is performing (left vs. right MI). In particular, when the sensorimotor area of the brain is activated during the imagination of hand movement, the interplay between contralateral attenuation of the μ rhythm and ipsilateral enhancement of the central �� oscillations in different phases of MI can usually be observed. These processes occur due to the neuro physiological mechanisms of the so-called eventrelated de-synchronization (ERD) and event-related synchronization (ERS).

<

The exact EEG manifestations and frequency bands of ERS and ERD vary from subject to subject. The eventrelated segment of each EEG trial (the last 4 s corresponding to 500 samples) was divided into rectangular windows depending on the settings of two parameters: window length and the amount of overlap. Next, the frequency-related information was extracted from every window with a parametric power spectral density (PSD) method that employs Yule-Walker algorithm for autoregressive (AR) modeling. The EEG features were calculated in each time window as the total energy within the bands of interest (adjusted μ and beta).

This served as an element C3/C4 i x of the feature vector x in (1).

<

Where, i is the window index (i=1,..,Nwin) and Nwin is the number of windows fitted into the 4-s signal interval within each trial.

c. Organizing training and feedback

To employ a BCI successfully, users must first go through several training sessions to obtain control over their brain potentials (waves) and maximize the classification accuracy of different brain states. In general, the training starts with one or two predefined mental tasks repeated periodically in a cue-based mode. In predefined time windows after the cue, we record the brain signals and use them for offline analyses. In this way, the computer learns to recognize the

users’ mental-task-related brain patterns. This learning process is highly subject- specific, so each user must undergo the training individually. The learning phase produces a classifier that we can use to classify the brain patterns online and provide suitable feedback to the users. Visual feedback has an especially high impact on the dynamics of brain oscillations that can facilitate or deteriorate the learning process.

V.SOFTWARE IMPLEMENTATION

One of the most successful algorithms in BCI is the method of common spatial patterns (CSP). The spatial filters, computed using this method are based on a joint diagonalization of the covariance matrices of two classes of data. However, in the standard CSP implementation the influence of amplitude and phase characteristics are not distinguished. In order to investigate phase synchronization links for the scope of task discrimination, so it is necessary to use a method that depends explicitly on phase synchronization information. The algorithm can also be used for the classification of two motor images.

a. Spatial filter design

The common spatial pattern (CSP) algorithm is very useful when calculating spatial filters for detecting ERD effects and for ERD-based BCIs, and is very sound for multi class problems. Given two distributions in a high-dimensional space, the (supervised) CSP algorithm finds directions (i.e., spatial filters) that maximize variance for one class and at the same time minimize variance for the other class. After having band pass filtered the EEG signals to the rhythms of interest, high variance reflects a strong rhythm and low variance a weak (or attenuated) rhythm. Let us take the example of discriminating left hand versus right hand imagery. The spatial filter that focuses on the area of the left hand is characterized by a strong motor rhythm during imagination of right hand movements (left hand is in idle state), and by an attenuated motor rhythm during left hand imagination. This criterion is exactly what the CSP algorithm optimizes: maximizing variance for the class of right hand trials and at the same time minimizing variance for left hand trials. Furthermore the CSP algorithm calculates the dual filter that will focus on the area of the right hand in sensor space. The CSP algorithm thus yields a matrix W, that projects a multichannel signal x(t) such that,

<

The rows of W represent a set of spatial filters, and an associated set of spatial patterns can be derived from the column vectors of the matrix

<

The matrices W and A are sometimes referred to as mixing and de-mixing matrices, or forward and backward models, respectively. Since the variance of a given signal is equivalent to band-power, the method of CSP is particularly effective for discriminating mental states characterized by event related desynchronization / synchronization ERD/ERS) effects . Consider two matrices Xi1 and Xi2 of dimension N_T, representing data from two different classes, where N is the number of channels, T is the number of time samples and i represents the trial number. Let C1 and C2 denote the covariance matrices for each class of data averaged over trials. The composite spatial covariance matrix is then obtained from Cc = C1 + C2, and Cc can be factored as Cc = UcAcUTc where Uc is the matrix of eigenvectors and Ac is the diagonal matrix of eigenvalues. The whitening Transformation

<

is determined such that,

<

Given that

<

S1 and S2 share a common matrix of eigenvectors B, such that

<

and

<

Therefore, the sum of two corresponding eigenvalues from A1 and A2 is always one, and the eigenvector associated with the largest eigenvalue for S1 has the smallest eigenvalue for S2 and vice versa. Defining the projection matrix W =BTP, a trial Xi is mapped to

<

Where, W gives a set of filters such that the variance of the first row of Z i is maximal for one condition and at the same time minimal for the other (vice versa for the last row of Z i). Similar but weaker properties characterize the variance of the rest of the top and bottom rows of Z i

b. Feature extraction and classification using the method of CSP

For feature extraction and classification purposes, typically a small number of filters are selected, corresponding to the first m and the last m rows of the projection matrix W. For a given trial Xi, multiplying the retained rows of the projection matrix W with Xi, a matrix Z i composed of 2m projected signals is obtained. A feature vector f i consisting of the variance values of the projected signals normalized by the total variance of the retained projections is obtained for each trial, such that

<

Where, zp is the pth row of the matrix Z i and p varies from 1 to 2m. The log transformation renders the distribution of the elements of f i normal. The feature vectors obtained from a set of training trials can then be used to estimate the parameters of a classifier, eg: linear discriminant analysis (LDA) classifier. The trained classifier is in turn used for determining the class of unseen trials. Computing the method of CSP on EEG recordings has resulted in promising results for mental task classification, and a number of variants and extensions that handle limitations of the standard algorithm can be found.

However, phase dynamics of the EEG signals that may also contribute to task discrimination, have not been explicitly considered. We are therefore investigating the applicability of this spatial filtering technique on phase interactions in the brain for the additional knowledge it may provide on mental activity and the utility it may have for the classification of mental tasks in BCI systems.

c. Phase locking value

The PLV is one of the most popular methods for the quantification of synchrony levels in the brain. In recent years, this measure has also been employed for task discrimination in BCI systems. The PLV gives a measure of the level of phase synchronization between two signals. In order to obtain the PLV, it is first necessary to obtain the phase of the signals considered. The two most popular methods for acquiring the phase of a scalar signal consist of the analytic signal approach and the wavelet approach. The two methods are known to give very similar results when applied to neurological data, with one advantage of the analytic approach being that it can be computed for broad band signals. In our case we opt for the analytic signal approach, with the analytic signal being given by,

<

Where, s (t) is the scalar signal being considered, sh(t) is the Hilbert transform of the signal, and z(t) is the analytic signal. The instantaneous amplitude A(t) and instantaneous phase can be computed from the analytic signal as follows,

<

The instantaneous phase difference between two signals sx(t) and sy(t) can be computed from

<

By computing the PLV, a measure of the phase locking between the two signals over an interval of N samples is obtained such that,

<

For two time series where the phase differences are randomly distributed, the PLV approaches a value of 0, while for signals where the phase difference is constant; the measure approaches a value of 1. When a dynamic value for the level of synchrony has to be established within single trials, the PLV measure can be applied across a moving window to give the single-trial PLV (S-PLV) where,

<

Delta indicates the number of samples to consider before and after the current time instant for the computation of the PLV. Small values of _ provide better temporal resolution and faster computational speed at the cost of statistical resolution, because compared to long-lasting episodes; short episodes of phase-locking are more likely to arise by chance alone.

d. P-CSP algorithm:

The method of CSP yields a set of spatial filters and spatial patterns that do not specifically indicate whether the differences between the two classes are the result of changes in amplitudes, variations in phase interactions, or a combination of both. In order to investigate which phase interactions are most significant for the discrimination of mental tasks, a method that utilizes the framework of CSP to determine spatial filters that discriminate mental tasks based on phase synchronization information is necessary. The algorithm considered for this application is an useful tool for the classification of mental tasks, and gives spatial patterns that highlight those phase interactions that are relevant for the discrimination of the mental tasks considered. The method we incorporate combines the S-PLV measure and the CSP framework, to give an implementation of the method of CSP based solely on phase information. Firstly, the phase difference for each pair of EEG signals is computed and from these the S-PLV for each channel pair is obtained. The phase information is then encoded by a carrier signal which is amplitude modulated by the computed S-PLV signals and phase modulated by the mean phase difference measured across each window. Therefore, for each channel pair, a signal that captures the phase synchronization information is obtained, where,

<

In the above expression, represents the mean phase difference computed across the S-PLV(t) moving time window and fc is the frequency of the carrier signal. The set of signals that result are then used as inputs for the method of CSP. This modulation is carried out in order to obtain a set of signals that are appropriate for further processing with the CSP algorithm. Through the method of CSP, a set of filters that discriminate two classes of EEG data solely on a basis of phase synchronization information is obtained. We will refer to this procedure as the method of phase-based CSP (P-CSP).

e. Comparison of various BCI systems

The CSP algorithm has proven to be very successful in determining spatial filters which extract discriminative brain rhythms. However the performance can suffer when non-discriminative brain rhythms with an overlapping frequency range interfere. Furthermore, we note that the phase interactions that are highlighted in the spatial patterns from the method of P-CSP cannot be explicitly obtained using the standard CSP implementation. In fact the standard CSP implementation only assigns individual weighting to each spatial location. This can create an ambiguity in the physiological interpretation of the resulting spatial patterns, since what might appear as an increase in signal power at a particular location might in effect be the result of variations in phase interactions. On the other hand, the method of PCSP focuses on the phase synchronization between the channels and is not affected by variations in signal amplitudes. The P-CSP algorithm also takes into account the angle of phase locking between two signals and not just an absolute value for the level of phase synchrony as is the case for the estimation of the standard PLV. Consider as an example two signals that exhibit a high level of phase locking for two classes of data, but where the two classes of data are characterized by a difference in the phase shift between the two signals. In such a case, the standard method of PLV would yield identical values for both classes since this method only considers the absolute level of phase synchrony between two signals. On the other hand, generating the proposed phase synchronization signals and applying the method of CSP on these can yield distinguishable signals for the two sets of data.

Another way to measure brain activity and implement BCI is with a Magnetic Resonance Image (MRI). An MRI machine is a massive, complicated device. It produces very high-resolution images of brain activity, but it can't be used as part of a permanent or semi-permanent BCI. Researchers use it to get benchmarks for certain brain functions or to map where in the brain electrodes should be placed to measure a specific function.

VI.FUTURE DIRECTIONS

For years, the brain of an adult human was viewed as a static organ. When we are a growing, learning , our brain shapes itself and adapts to new experiences, but eventually it settles into an unchanging state -- or so went the prevailing theory. Researchers now proved that the brain actually remains flexible even into old age. This concept is known as cortical plasticity, means that the “brain is able to adapt in amazing ways to new circumstances”. Learning something new or partaking in novel activities forms new connections between neurons and reduces the onset of agerelated neurological problems. If an adult suffers a brain injury, other parts of the brain are able to take over the functions of the damaged portion. In situations where implants are used, it means that the brain can accommodate this seemingly foreign intrusion and develop new connections that will treat the implant as a part of the natural brain. For brain-actuated robots using BCI, fast decision making is critical. In this sense, real-time control of brain-actuated devices, especially robots and neuro-prostheses, is the most challenging BCI application. We still need to improve the BCI’s robustness to make it a more practical and reliable technology.

VI.CONCLUSION

In this paper, Brain computer interfacing (BCI), which is an interdisciplinary area that connects biomedical knowledge on brain physiology with the use of effective computational techniques for mental task discrimination, is presented. The use of such a valuable technology for the realization of intelligent, mind controlled, robotic rehabilitation is also briefed. For this application use of CSSP algorithm is found significant as it overcomes the limitations posted by CSP, one of the most successful algorithms in BCI. In the event of increasing demand for the above presented intelligent systems, it gives me immense pleasure and satisfaction sharing one such application through this paper. Looking forward to my final year project that is going to reveal the practical difficulties and lot more information through its implementation.

References

2008 IEEE International Conference on Systems, Man and Cybernetics (SMC 2008) Brain-Computer Interfacing for Intelligent SystemsAnton Nijholt, University of Twente Desney Tan, Microsoft Research Designing a Robust Type-2 Fuzzy Logic Classifierfor Non-stationary Systems with Application in Brain- Computer Interfacing Pawel Herman, Girijesh Prasad and Thomas Martin McGinnity Intelligent Systems Research Centre, School of Computing and Intelligent Systems, An Algorithm for Brain Computer Interfacing on Phase Synchronization Spatial Patterns by Owen Falzon and Kenneth . Camilleri of Malta