Research and Reviews - International Journals

International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

Menu

ISSN ONLINE(2278-8875) PRINT (2320-3765)

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

Brainwave Controlled Robot Using Bluetooth

Siliveru Ramesh1, K.Harikrishna2 J.Krishna Chaitanya3
  1. PG Student,Department of ECE, Vardhaman College of Enggineering, Hyderabad, India1
  2. Professor, Department of ECE, Vardhaman College of Enggineering, Hyderabad, India2
  3. Associate professor, Department of ECE, Vardhaman College of Engineering, Hyderabad, India3
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

Abstract

This paper describes about a brain controlled robot based on Brain–computer interfaces (BCI). BCIs are systems that can bypass conventional channels of communication (i.e., muscles and thoughts) to provide direct communication and control between the human brain and physical devices by translating different patterns of brain activity into commands in real time. With these commands a mobile robot can be controlled. Here the robot is selfcontrolled with the ultrasonic sensor. The intention of the project work is to develop a robot that can assist the disabled people in their daily life to do some work independent on others. Here, we are analysing the brain wave signals. Human brain consists of millions of interconnected neurons.

Keywords
EEG Sensor, Relay Driver, Embedded platform, communication module, image acquisition module, BCI-Brain computer interface.

INTRODUCTION
The patterns of interaction between these neurons are represented as thoughts and emotional states. According to the human thoughts, this pattern will be changing which in turn produce different electrical waves. A muscle contraction will also generate a unique electrical signal. All these electrical waves will be sensed by the brain wave sensor and it will convert the data into packets and transmit through Bluetooth medium [1]. Level analyzer unit (LAU) will receive the brain wave raw data and it will extract and process the signal using Mat lab platform. Then the control commands will be transmitted to the robotic module to process. With this entire system, we can move a robot according to the human thoughts and it can be turned by blink muscle contraction.
The basic idea of BCI is to translate user produced patterns of brain activity into corresponding commands. A typical BCI is composed of signal acquisition and signal processing (including preprocessing, feature extraction and classification) [2]. Although some BCI systems do not include all components and others group two or three components into one algorithm, most systems can be conceptually divided into signal acquisition, preprocessing, feature extraction, and classification The brain signals that are widely used to develop EEG-based BCIs include P300 potentials, which are a positive potential deflection on the ongoing brain activity at a latency of roughly 300ms after the random occurrence of a desired target stimulus from non-target stimuli the stimuli can be in visual, auditory, [3] or tactile modality SSVEP, which are visually evoked by a stimulus modulated at a fixed frequency and occur as an increase in EEG activity at the stimulus frequency and the event-related de synchronization (ERD) and event-related synchronization (ERS), which are induced by performing mental tasks, such as motor imagery, mental arithmetic, or mental rotation [4].
Although many researchers have developed various brain-controlled mobile robots, to the best of our knowledge, none of the existing brain-controlled mobile robots is brought out of a controlled laboratory environment. The main reason for this is that the BCI is not stable due to the non-stationary nature of the EEG signals. Thus, to make these mobile robots usable in real-world situations, stable BCI systems need to be explored. If a BCI system is not stable, other techniques should be further developed to improve the overall driving performance. Rebsamen et al., Iturrate et al. also combined a P300 BCI and an autonomous navigation system to develop a robotic wheelchair. The main difference between them is that the latter allows a wheelchair to move in an unknown environment. In addition, the user is able to control the wheelchair to turn left or right at any time by focusing his/her attention on the “turn left” or “turn right” icons at the lower section of the visual display to elicit a corresponding P300.

DESIGN AND IMPLEMENTATION
This paper is implemented in an effective way we divide brain-controlled mobile robots into two categories according to their operational modes. One category is called “direct control by the BCI,” which means that the BCI translates EEG signals into motion commands to control robots directly who first developed a brain-controlled robotic wheelchair whose left or right turning movements are directly controlled by corresponding motion commands translated from user brain signals while imagining left or right limb movements, and tested this system in real-world situations [5]. The robotic platform is illustrated also used a BCI based on motor imagery to build a brain-controlled mobile robot, as illustrated which can perform three motion commands including turning left and right and going forward, and validated this robot in a real world.
As shown in figure 1 the Brain secret card section contains EEG Sensor to Sense the Human brain, and it will be sensed by using the Brainwave Headset which is provided by NeuroSky. Technologies and those signals will be transferred by using Bluetooth which is there in the Brainwave headset, for this Brainwave headset we need to give power using a AAA battery which is shown in figure 2.The Brainwave headset comes with Power switch, a sensor tip, flexible ear arm and a ground connection Ear clip. In this Headset we use Non-invasive sensor that won’t cause any pain to the User who were the headset. After inserting an AAA battery switch on the Brainwave headset using the power switch the LED indicator will blink and if the Red colour light not blinking the headset is powered on but not connected to with the computer’s Bluetooth. If the Blue colour not blinking that means the headset is powered on and connected. If the red or blue colour blinks it shows that the Battery getting low.
As shown in figure 3 the Data transmitted by the Brainwave headset will be received by the Computer’s Bluetooth receiver. And then all these data will be analysed by the Level Analysis platform. The Level Analysis platform will extract the raw data using the MATLAB. After the analysis of this data, this data will be sent to the robot module using serial data transmission i.e. using XBee.
As shown in Figure 4, in the robot module there will be an XBee receiver will receive the data which is transmitted by the XBee transmitter. According to the data received by the XBee the ARM processor will give the directions to the motors and the robot is self-controlled robot with ultrasonic sensor and connected with a relay and a driver circuit. And all this information will be displayed on the LCD display.

SYSTEM HARDWARE

ARM Processor
The ARM processor is 32-bit embedded RISC microprocessor. The ARM7 processor needs very low power, high performance and small size. Here in this paper I’m using ARM7 processor and ARM processor will receive the signals from the Bluetooth receiver and it will process the signals and it will give the signals to the Driver Circuit and according to that signals received from the processor, according to the signals received driver circuit the motors will rotate forward, clockwise and anti-clockwise direction. Here ARM processor will wait until the signals received from the Brainwave headset and after receiving the signals it will moves the robots. The driver circuit will connected to the port 1 of the processor.

Wireless communication
The XBee and XBee-PRO OEM RF Modules were engineered to meet IEEE 802.15.4 standards and support the unique needs of low-cost, low-power wireless sensor networks [6]. The modules require minimal power and provide reliable delivery of data between devices. The modules operate within the ISM 2.4 GHz frequency band. Here we are using XBee to connect Robot with the Computer wirelessly through a logic-level asynchronous serial port. Through its serial port, the module can communicate with any logic and voltage compatible UART [7]. The computer will transform the data which is analysed by using the Level Analyser unit using MAT lab. The data which is transmitted by the XBee module from the computer will be received by the XBee receiver which is connected to the ARM processor.

EEG Signals
EEG signals can be collected with electrodes that are placed on the surface of the scalp. The most widely used electrodes are silver/silver chloride (Ag/AgCl) because they have low cost, low contact impedance, and relatively good stability. Furthermore, there are rather mature commercialized acquisition systems including the amplifier and EEG cap with integrated Ag/AgCl electrodes, which have been successfully applied in scientific research and clinical diagnosis. However, using Ag/AgCl electrodes requires removing outer skin layer and filling gel between electrodes and scalp (and thus, this kind of electrodes is also called “wet” electrodes). These operations take long time and are uncomfortable to users. To address these limitations of “wet” electrodes, some researchers have been exploring “dry” electrodes, which do not need to use gel and skin cleaning [8-11]. The main disadvantage of existing dry electrodes is that the acquired EEG signals are worse than those acquired with conventional electrodes due to the increase of contact impedance [12]. Some companies (such as Quasar, Emotiv Systems Inc., and NeuroSky Inc.) have been commercializing acquisition systems based on dry electrodes [13, 14]. Here we are using NeuroSky Brainwave headset. However, they are not yet mature, and some researchers have doubts about what physiological signals these systems actually acquire [15]. Therefore, until now, all brain-controlled wheelchairs adopt “wet” electrodes to collect brain signals.

NeuroSky Technology
i. Brainwaves:
The last century of neuroscience research has greatly increased our knowledge about the brain and particularly, the electrical signals emitted by neurons firing in the brain. The patterns and frequencies of these electrical signals can be measured by placing a sensor on the scalp. The Mind Tools line of headset products contain Neurosky Think Gear technology, which quantify the analog electrical signals, commonly referred to as brainwaves, and exercise them into digital signals. The Think Gear technology then makes those computations and signals available to games and applications. The Table I give a general synopsis of some of the commonly- identified frequencies that tend to be generated by different types of activity in the brain [16].
ii. ThinkGear
ThinkGear is the technology inside every NeuroSky product or partner product that empowers a device to interface with the wearers’ brainwaves [16]. It includes the sensor that touches the forehead, the contact and reference points located in the ear clip, and the on-board chip that processes all of the data. Both the raw brainwaves and the eSense Meters are computed on the ThinkGear chip.
iii. eSense
eSense is a NeuroSky's proprietary algorithm for representing mental states. To calculate eSense, the NeuroSky ThinkGear technology intensifies the raw brainwave signal and removes the ambient noise and muscle movement. The eSense algorithm is then applied to the remaining signal, resulting in explicated eSense meter values. Please note that eSense meter values do not interpret an exact number, but instead describe ranges of activity. The eSense meters are a way to show how effectively the user is captivating Attention (similar to concentration) or Meditation (similar to relaxation) [16].
(a)Attention eSense:
The eSense Attention meter shows the intensity of a user's level of mental “focus” or “attention”, such as that which occurs during intense concentration and directed (but stable) mental activity. Its value ranges from 0 to 100. Distractions, wandering thoughts, lack of focus, or anxiety may lower the attention meter level.
(b)Meditation eSense:
The eSense Meditation meter shows the level of a user's mental “calmness” or “relaxation”. Its value ranges from 0 to 100. Note that Meditation is a measure of a person's mental states, not physical levels, so simply relaxing all the muscles of the body may not instantly result in an intensified effect meditation level. However, for most people in most normal circumstances, relaxing the body often helps the mind to relax as well. Meditation is related to reduce activity by the active mental processes in the brain. It has long been an observed that closing one's eyes turns of the mental activities which process images from the eyes. So closing the eyes is often an effective method for increasing the Meditation meter level. Distractions, wandering thoughts, anxiety, agitation, and sensory stimuli may lower the Meditation meter levels [16].
iv. eSense Meter - Technical Description
For each different type of eSense (i.e. Attention, Meditation), the meter value is reported on a relative eSense scale of 1 to 100. On this scale, a value between 40 to 60 at any given moment in time is considered “neutral” and is similar in notion to “baselines” that are established in conventional brainwave measurement techniques (though the method for determining a ThinkGear baseline is proprietary and may differ from conventional brainwaves).
A value from 60 to 80 is considered “slightly elevated”, and may be interpreted as levels tending to be higher than normal (levels of Attention or Meditation that may be higher than normal for a given person) [16]. Values from 80 to 100 are considered “elevated”, meaning they are strongly indicative of heightened levels of that eSense. Similarly, on the other end of the scale, a value Between 20 to 40 indicates “reduced” levels of the eSense, while a value Between 1 to 20 indicates “strongly lowered” levels of the eSense. These levels may indicate states of interruption, agitation, or abnormality, according to the opposite of each eSense.
The reason for the somewhat wide ranges for each interpretation is that some parts of the eSense algorithm are dynamically learning and at times employ some “slow-adaptive” algorithms to adjust to natural fluctuations and trends of each user, accounting for and compensating for the fact that brainwaves in the human brain are subject to normal ranges of variance and fluctuation. This is part of the reason why ThinkGear sensors are able to operate on a wide range of individuals under an extremely wide range of personal and environmental conditions, while still giving good accuracy and reliability [16].

DESIGN FLOW
The flow diagram of Brainwave Controlled Robot unit is shown in figure 6. It shows all the step by step functions of robot, how it will be controlled by using brainwave signals. After Switching on the Brainwave headset and the Robot kit, the processor will initialize and the headset will starts sensing the neurons signals and after sensing the signals it will transfer them to through the Bluetooth and the acquisition module will receive the signals in the processor and in the processor the EEG signals comparison will be done if it is yes then the robot will move according to the signals or else it will go to the relay circuit and robot movement will be there and the process will be stopped.

RESULT AND DISCUSSION
The research and development of brain-controlled mobile robots have received a great deal of attention because they can help bring mobility back to people with devastating neuromuscular disorders and thus improve their quality of life.
In this paper, we presented a comprehensive up-to-date review of the complete systems, key techniques, and evaluation issues of brain-controlled mobile robots. After implementing the Mindwave Controlled robot I’ve checked the results with NeuroSky headset, as I expected the headset doesn’t give the 100% accuracy of brainwaves but it is too good for its price and it can give up to 95% accuracy of brainwaves. After installing all the Neurosky software’s in PC, after connecting the Headset with PC through PC using Bluetooth, we need to wear the headset to the head and then we need to open the Mat lab Code and run the program, after clicking run the program in the command window of mat lab it will show the brainwave is connected and blink is detected and after that it will show the attention values and Blink values which is shown in figure 6.
After getting these attention and blink values a graph will be generated and In the graph there will be two signals, the Black Signal is Blinking level and the red signal is Attention signal shown in figure 7. From here these signals will be transferred to the Robot through Zigbee wireless transmission, the signals will be collected by Zigbee receiver and sends to the processor, the processor decodes the signals as per Brainwave signals and according to the signals the Processor gives the commands to the motor wheels of the robot, according to the signals the robot will move forward, right and left and the robot is self controlled.

CONCLUSION
The research and development of brain-controlled mobile robots have received a great deal of attention because they can help bring mobility back to people with devastating neuromuscular disorders and thus improve their quality of life. In this paper, we presented a comprehensive up-to-date review of the complete systems, key techniques, and evaluation issues of brain-controlled mobile robots.

Tables at a glance
Table icon
Table 1
 

Figures at a glance
Figure 1 Figure 2 Figure 3 Figure 4
Figure 1 Figure 2 Figure 3 Figure 4
Figure 5 Figure 6 Figure 7
Figure 5 Figure 6 Figure 7
 

References
  1. Luzheng Bi, Xin-An Fan, Yili Liu “EEG-Based Brain-controlled Mobile Robots: A survey”, Human-Machine Systems, IEEE Transactions on (Volume:43, Issue:2), pp. 161-176, Mar 2013.
  2. Kale Swapnil T, Mahajan Sadanand P, Rakshe Balu G, Prof. N.K.Bhandari “Robot Navigation control through EEG Based Signals” International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 3 Issue 3 March-2014 Page No. 5105-5108.
  3. Priyanka.M Manju Paarkavi.R Dhanasekhar.S “An Intelligent Acoustic Communication System for Aphasia Forbearings” International Conference on Signal Processing, Embedded System and Communication Technologies and their applications for Sustainable and Renewable Energy (ICSECSRE ’14), Vol. 3, Special Issue 3, April 2014.
  4. John Jonides, Patricia A. Reuter-Lorenz, Edward E.Smith, Edward Awh, Lisa L.Barnes, Maxwell drain, Jennifer Glass, Erick J.Lauber, Andrea L.Patalano, Eric H.Schumacher, “Verbal and Spatial Working Memory in Humans” The Psychology of Learning and Motivation, Vol.35.
  5. I.I. Goncharova, D.J. McFarland, T.M. Vaughan, J.R. Wolpaw “EMG contamination of EEG: spectral and topographical characteristics” Clinical Neurophysiology 114 (2003) 1580–1593.
  6. Sparkfun, “Xbee manual” [online] Available: https://www.sparkfun.com/datasheets/Wireless/Zigbee/XBee-Manual.pdf
  7. Cytron Technologies, “SKXbee starter kit” [online] Available: http://www.cytron.com.my/datasheet/WirelessDevice/SKXbee_User's_Manual_v1.pdf
  8. C. Fonseca, J. P. S. Cunha, R.E. Martins, V. M. Ferreira, J. P. M. de Sa, M. A. Barbosa, and A. M. da Silva, “A novel dry active electrode for EEG recording,” IEEE Trans. Biomed. Eng., vol. 54, no. 1, pp. 162–165, Jan. 2007.
  9. F. Popescu, Y. Fazli, S. Badower, B. Blankertz, and K.-R. Muller,¨ “Single trial classification of motor imagination using 6 dry EEG electrodesm,” PLoS ONE, vol. 2, no. 7, 2007.
  10. A. Taheri, R. Knight, and R. Smith, “A dry electrode for EEG recording”, Electroencephalogr. Clin. Neurophysiol. vol. 90, no. 5, pp. 376– 383,May 1994.
  11. A. Searle and L. Kirkup, “A direct comparison of wet, dry and insulating bioelectric recording electrodes,” Physiol. Meas., vol. 21, no. 2, pp. 71– 83, May 2000.
  12. G. Gargiulo, P. Bifulco, R. A. Calvo, M. Cesarelli, C. Jin, A. McEwan, and A. van Schaik, Intelligent and Biosensors. Rijeka, Croatia: In Tech, Jan. 2010, ch. 6.
  13. E. W. Sellers, P. Turner, W. A. Sarnacki, T. Mcmanus, T. M. Vaughan, and B. Matthews, “A novel dry electrode for brain–computer interface,” in Proc. 13th Int. Conf. Human–Computer Interac., San Diego, CA, 2009, pp. 623–631.
  14. T. J. Sullivan, S. R. Deiss, T. P. Jung, and G. Cauwenberghs, “A brain– machine interface using dry-contact low-noise EEG sensors,” in Proc.Conf. Rec. 2008 IEEE Int. Symp. Circuits Syst., Seattle, WA, May 2008,pp. 1986–1989.
  15. J.d.R.Millan, R.Rupp, G.R.Muller-Putz, R. Murray-Smith, C.Giugliemma, M.angermann, C. Vidaurre, F.Cincotti, A.Kubler, R. Leeb, C.Neuper, K.-R. Muller, and D.Mattia, “Combining brain–computer interfaces and assistive technologies state-of-the-art and challenges,” Frontiers Neurosci., vol. 4, pp. 1–15, 2010.
  16. Neurosky, “Mind-set instruction manual”, [online] Available: http://developer.neurosky.com/docs/doku.php?id=mindset_instruction_manual.
  17. X. Perrin, “Semi-autonomous navigation of an assistive robot using low throughput interfaces,” Ph.D. dissertation, ETHZ, Zurich, Switzerland, 2009.
  18. E. Donchin, K. M. Spencer, and R. Wijesinghe, “The mental prosthesis: assessing the speed of a P300-based brain–computer interface,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 8, no. 2, pp. 174–179, Jun. 2000.J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68-73.
  19. N. Birbaumer, N. Ghanayim, T. Hinterberger, I. Iversen, B. Kotchoubey, A. Kubler, J. Perelmouter, E. Taub, and H. Flor, “A spelling device for the paralyzed,” Nature, vol. 398, pp. 297–298, Mar. 1999.
  20. K.-R. M¨uller and B. Blankertz, “Towardb noninvasive brain–computer interfaces,” IEEE Signal Process. Mag., vol. 23, no. 5, pp. 125–128, Sep. 2006.