Towards a Brain-Controlled Wheelchair Prototype

In this project, a design for a non-invasive, EEG-based brain-controlled wheelchair has been developed for use by completely paralyzed patients. The proposed design includes a novel approach for selecting optimal electrode positions, a series of signal processing algorithms and an interface to a powered wheelchair. In addition, a 3D virtual environment has been implemented for training, evaluating and testing the system prior to establishing the wheelchair interface. Simulation of a virtual scenario replicating the real world gives subjects an opportunity to become familiar with operating the device prior to engaging the wheelchair.


INTRODUCTION
Certain cases of disease or injury can result in complete loss of muscle control and/or movement despite the subject being fully conscious and aware of his or her surroundings. Recent advancements in brain-computer interfacing (BCI) have presented new opportunities for development of a new wheelchair interface for such patients based on thought.
We are presently investigating a BCI design for a brain-controlled wheelchair (BCW), which comprises three distinct stages: extracting the raw brain waves, processing these signals and classifying them into different control thoughts/action, and interfacing to the powered wheelchair or simulation thereof. This paper will discuss our progress at developing and improving each of these stages.

BRAIN SIGNALS
An appropriate BCW must be able to control each direction of wheelchair movement and thus ideally requires at least 4 control signals, or classes (forward/back/left/right). Although classification accuracy decreases with the number of classes [1], we will present results illustrating that sufficient accuracy can be obtained with 4 classes. The prototype BCW utilises sensorimotor rhythms (SMR) that are generated in sensory and motor cortical regions in association with both real and imagined movement of the limbs and other body parts [2]. The SMRs used in our project are generated by imagined movement of left hand, right hand, either foot, and tongue. The BCW has the capacity to acquire 16channel 24-bit electroencephalogram (EEG), however all of the data used in the experiments in this paper is publicly available EEG data from the BCI Competition III [3]. The motivation for using this data was that it was easily accessible and widely used within the BCI community.

SIGNAL PROCESSING
Several feature extraction (FE) and classification (CL) algorithms were investigated as potential signal classification candidates for the BCW. To assess the utility of these algorithms we evaluated their ability to discriminate four-class SMR EEG data. All of the candidate algorithms have been used successfully in other BCI systems.
The FE methods evaluated were the discrete Fourier transform (DFT) and common spatial patterns (CSP). The DFT is used to transform the EEG into the frequency spectrum by extracting the magnitude of the DFT. The CSP is a spatial filter that seeks to facilitate discrimination of two classes by transforming the data such that the variance of one class is minimized whilst the variance of the other class is maximized [4]. Maximally separating the two classes results in a simpler discrimination task for the classifier.
The classifiers evaluated were a backpropagation artificial neural network (ANN) and support vector machine (SVM). SVMs are linear and seek to separate class instances by the widest margin possible and often include the use of implicit mappings to a higher dimensional space via a kernel function. Two particular SVM kernels are investigated in this project, namely linear and radial basis function (RBF) [5].

Method
The signal-processing environment used in this project for training and evaluation is coded in Matlab and makes use of the LIBSVM library [6] for SVM and a CSP algorithm [4,7] extended to multiclass using the one-versus-rest approach investigated in [8].
All experiments were performed on 60-channel 4-class imagined motor imagery EEG data from subject 'k3b' of dataset IIIa of the BCI Competition III [3]. The EEG was re-sampled to 250 Hz, rereferenced to a common average reference, and segmented in nonoverlapping 0.5s epochs.
The FE methods were applied to each individual epoch and then the classifiers applied to predict the class for each epoch. Training and testing of all algorithms was conducted using a 10-fold cross validation technique repeated 10 times. Performance was measured using bookmaker informedness [9], with the value of '1' representing perfectly correct, '0' being chance and '-1' being perfectly incorrect. The magnitude of this score (equivalent to DeltaP or RoC in the dichotomous case) represents the probability that an informed decision is being made versus a random guess.
A Lillifors test indicated that the data was not normally distributed and as such statistical comparisons were made using the Kruskal-Wallis one-way ANOVA.

Results
The number of CSP eigenvectors to include for best classification performance was investigated empirically. It was observed that classification performance increased logarithmically with the number of eigenvectors included. Thus, all eigenvectors were included in subsequent CSP analyses.
The results for all the different combinations of algorithms tested are summarized in Table 1. There was a main effect for FE characterized by CSP performing significantly better than DFT (p<0.01). There was also a main effect for CL with SVM (RBF) performing significantly better than both SVM (Linear) and ANN (p<0.01). SVM (Linear) was also significantly better than ANN. The best classification performance was 0.66 bookmaker and was achieved with the CSP algorithm combined with SVM with an RBF kernel. This corresponds to a weighted average performance of 75%. Therefore the CSP and SVM (RBF) were the selected as the signal processing algorithms for the BCW prototype.

OPTIMAL ELECTRODE PLACEMENT
Previous work [10] has reported that performance correlates with the number of EEG channels. However, it is our position that high-resolution EEG (>32 channels) is impractical for a BCW device given constraints of size, cost, and user comfort. Performance loss incurred by a smaller EEG montage must be mitigated by optimal electrode placement and application of highperformance classification algorithms. In this section we investigate the optimal number of electrodes and their locations to find a balance between classification accuracy and user comfort/portability. We propose a data-driven method for selecting optimal electrode sites that leverages the CSP algorithm described in section 3.

Method
The CSP algorithm decomposes the input signals into CSP components (spatial filters) that are ordered such that the first and last few maximize the difference in variance of the two classes. The multiclass extension described in section 3.1 uses a oneversus-rest (OVR) approach to calculate CSP for each class versus the grouped remaining classes. Thus the first and last few CSP components will maximize the difference in variance between the target class and the grouped remaining classes.
The CSP algorithm generates a weight matrix that maps the input signals (in EEG space) to the CSP components. In the case of the OVR extension, a weight matrix is produced for each class (control signal). The weight matrix encapsulates the relative contribution of each input signal to each CSP component. By examining the weight matrix we can identify which input signals contribute most strongly to the first and last few CSP components   for the target class. The electrode locations of the highest contributing input signals are deemed to be the optimal electrode sites.
The contribution of each input signal to a CSP component is reflected in the magnitude of the values in the weight matrix. However, the weight of a given input signal is also influenced by the energy of the input signal. For example, a high-energy input signal with a low weight may actually contribute more than a lowenergy signal with a high weight. To account for this, the BCI data was demeaned in each channel and normalized to a uniform RMS for each channel. By removing the variation in energy across the input signals we could then ascertain contribution directly from the values in the weight matrix.
This technique was applied to the same EEG data as used in section 3. A number of subsets of electrodes were selected using this method and the impact on classification performance was assessed using a linear SVM. A Bonferroni-corrected Kruskal-Wallis one-way ANOVA was used to determine statistical significance across the results. Figure 1 shows the classification performance for the full set of 60 electrodes as well several subsets of electrodes selected with the described CSP method. As can be seen, classification performance increases with the number of electrodes in the set, however the rate of improvement diminishes and there is little to be gained beyond 16 electrodes (4-per class). The location of the 16 optimal electrode sites for this data is presented in Figure 2.

Results
Statistical tests were performed between the 16 optimal electrodes (Optimal), all 60 electrodes (All), 16 uniformly selected electrodes and 16 electrodes selected uniformly from channels that contributed little to the CSP components (NonOptimal). The results show that there is no significant difference between the set of all electrodes and the 16 optimally selected electrodes ( Figure  3). Furthermore, the 16 uniformly selected electrodes and the nonoptimal electrodes performed significantly worse.
This result suggests that targeted application of fewer electrodes results in performance equivalent to a standard uniform montage of electrodes. Reducing the number of electrodes required has obvious benefits in terms of comfort and practicality of the system for every day use.

TRAINING AND EVALUATION 5.1 Simulated Environment
Asynchronous evaluation of the system after signal processing is necessary prior to establishing the wheelchair interface, predominantly as a safety measure. An experimental environment based on simulation provides this opportunity and also enables training of subjects on how to control the wheelchair.
For this reason, a virtual 3D environment was created using Java3D with the camera perspective of a first-person viewpoint ( Figure 4) as would be the viewpoint of the person sitting in a wheelchair. The virtual environment simulates the same features of the real wheelchair driving environment, with four different movement commands similar to that of a real wheelchair driving controller. Each command triggers an incremental movement forward or backward or a turn to either left or right. There is also an alarm triggered if the wheelchair collides with a wall.

Wheelchair Interfacing
A modular controller would be particularly suitable for a BCW due to its flexibility and expandability in interfacing with other controllers or systems such as chin control and head arrays. The interfaces to a modular controller are designed for a large variety of input mechanisms corresponding to the large range of specific disabilities and degrees of incapacity. We plan to adapt either the DX-5SW [11], as it best matches our requirements, or the VSI 50A [12], as it is already installed on the Ruby Plus wheelchair that is available to us [13].
Repeated commands and ongoing directional thought are proposed as requirements for the system to maintain its direction and motion. Currently four commands (left, right, forward, back) exist in the system. Human factor evaluation is ultimately necessary for finalizing the type of commands and for determining the necessity of having an emergency stop, and selection of different protocols and choices of command "thought" for an individual subject or to assist with a particular disability. We acknowledge that requiring thought/command repetition to sustain movement maybe too onerous for the operator. Thus it will be important to investigate and evaluate  alternatives such as sustained movement from a single thought/command.

SUMMARY
The brain-controlled wheelchair system proposed in this project identifies the early challenges and successes in realizing a mobility device for disabled or paralyzed individuals based on brain-computer interaction. Specifically we have: 1. Investigated the utility of several signal processing algorithms and identified CSP and SVM as the best candidates for the BCW 2. Developed a novel data-driven method for optimally selecting electrode locations and demonstrated that this performs as well as higher-density electrode montages 3. Developed a 3D virtual wheelchair simulator to provide a safe and controlled environment for users to practice operating the BCW before eventually engaging the real wheelchair

Evaluated electric wheelchair controllers and indentified candidates suitable for operation by a BCI
The BCW is an ongoing research program with several concurrent projects. At this stage the simulated environment and the wheelchair have not been interfaced with the signal processing. However the most immediate goal is to complete a fully operational real-time BCW prototype that interfaces the signal processing discussed earlier with both the developed wheelchair simulator and the real wheelchair. This work is well under way and will be completed in the near-term.

FUTURE DIRECTIONS
A successful BCW is likely to be operated by the user for extended periods of time. As such, a primary HCI consideration in designing a BCW is to minimize the cognitive load imposed on the user by operation of the device. We propose several avenues of future work to address the issue of user load.
A BCI requires the user to generate a pattern of brain activity that can be reliably and reproducibly measured and classified. Unsurprisingly, BCIs typically exploit well-characterized EEG phenomena. For example, the prototype described in this paper exploits the SMR, which can be reliably elicited by imagined movement. However, the task of imagining movements of various limbs is quite abstract from the intended consequence of operating a wheelchair. We believe this dichotomy results in increased load being imposed on the user. Thus we plan to train the BCW on a more goal-relevant task (e.g. 'thinking left' or 'thinking right') and compare the load imposed on the user for the goal-relevant task to that imposed by the SMR task.
It is likely that it will be more difficult to discriminate classes in goal-relevant data and therefore we expect to observe a reduction in classification performance. We will then have to examine the trade-off between cognitive load and classification performance. If there to be a substantial benefit to using a goal-relevant task we will explore methods to improve the subsequent classification performance. One approach that we are investigating is to require the user to repeat command thoughts to operate the device. This would reduce the error by a power of n, where n is the number of times a command must be repeated before the BCW responds. Another potential loading factor is environmental distractions.
When operating a BCW there are all sorts of potential real-world events that could distract the operator and impair their ability to operate the BCW safely. We plan to explore this using the wheelchair simulation environment that we have developed. With the simulator we will be able to introduce a variety of distractors whilst the person operates the BCW in a safe controlled environment.