International Journal of Innovative Research in Computer and Communication Engineering

International Journal of Innovative Research in Computer and Communication Engineering

Menu

ISSN ONLINE(2320-9801) PRINT (2320-9798)

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

Voice Recognition Robot for Visually Impaired People

M. Joshuva Regan, S.R.Barkunan
  1. Graduate Student, Department of ECE, Anna University Regional Centre, Coimbatore, India
  2. Assistant Professor, Department of ECE, Anna University Regional Centre, Coimbatore, India
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering

International Journal of Innovative Research in Computer and Communication Engineering

Abstract

A smart multipurpose human assistance robotic dog is designed to guide the visually impaired and elderly people to some predefined destination avoiding obstacles and traffic. It is also designed to act as an advanced multipurpose human assistance and service robot that is able recognize the words spoken by the user, talk to them and take action according to the spoken voice command. Voice commands are recognized by an android Smartphone and the information is transferred to the main MCU using a Bluetooth serial port that runs Bluetooth SPP protocol stack. The robotic dog has the ability to follow a human when commanded with voice. Touch sensitive e-skin senses human finger touch and helps answering complex user requests such as time, date and weather conditions such as light and temperature. The same can be asked using voice also. It even allows the user to set wake up alarm. A built in audio playback system can play music tracks in MP3 format. One of the music tracks is kept as the alarm tone. It also plays the role of a regular watchdog during night and barks like any normal dog if it finds any abnormal activity. During the day time it can charge itself by moving around within a given region in order to find the maximum sun light, intelligently avoiding the shaded areas, thereby freeing the user completely from maintenance issues such as battery charging.

Keywords
Human walking intention, Voice recognition, speaking robot, digital MEMS, Bluetooth serial port, sonar

INTRODUCTION
Guide dogs are assistance dogs trained to lead elderly or visually impaired people around obstacles. These dogs need to be trained by some special training institute and maintained healthy in order to help people. Being a service dog, they also have a maximum lifetime around 8 to 10 years. These dogs help them in one another way. Elderly people are often left alone and have least amount of interaction with other people. Although a guide dog relieve them from such a situation, they are dumb and lag intelligence of a human companion.
For many visually impaired people, a cane or a stick is a close friend helping them to detect and avoid obstacles in the walking paths. During walking with the cane, they sense and guess directions and locations by hearing sounds surrounding, sniffing smells in the air, feeling touches on skin, counting footsteps they walk, and memorizing events in time and spaces. However, it is difficult for them to do this all the time when surrounding environment could suddenly change, or when they get lost memory of locations. Voice Recognition Robotic Dog Guides Visually Impaired People, Follows Master, Acts as Watchdog, Plays MP3 Music, Finds Sun Light and Charges itself, Respond to e-Skin Touch, Wake up Alarm, and Informs about Time-Date-Light-Temperature

RELATED WORK
AS many countries step into the aging society rapidly, more and more elders suffer from deficits of motor function or disability of the limbs, which are usually caused by neurological problems or lack of muscle strength. In addition, the growing elderly population causes the shortage of people for nursing care. Therefore, there is a great need to develop rehabilitation robots that can partially replace the nurses and the therapists. Currently, plentiful studies on rehabilitation robots can be found, including the applications for the upper limb [1]–[5], for the lower limb [6]–[9], and for the assisting or training of the whole body [10]–[13]. In daily life, the walking is one of the most important human activities. To improve the walking ability of the elderly, the walker-type rehabilitation robot has become a popular research topic over the last decade. There have been many intelligent walker-type robots comprising active or passive wheels and supporting frame. A novel assistive robotic walker called “JAIST active robotic walker (JARoW)” to provide potential users with sufficient ambulatory capability in an efficient, cost effective way was presented in [14]. The Hitomi system to help the blind in outdoor environment was proposed in [15].Apowerassisted walker for physical support during walking was developed in [16]. The Care-O-bot and Nursebot are developed as personal service robots for elderly and disables [17], [18]. The personal aid for mobility and monitoring (PAMM) system to provide mobility assistance and user health status monitoring was proposed in [19]. A new intelligent walker based on passive robotics to assist the elderly, handicapped people, and the blind was proposed in [20]. There are still many deficiencies in the present walker systems. First, many walkers are designed for the indoor environment. Second, most of them are big in size and/or heavy in weight. An indoor robot is often restricted within limited places. Big size makes it impossible to be used in narrow space and heavy weight restricts the man euverability. Many elders and patients are not so weak that they have to be nursed carefully. Nevertheless, sufficient support, such as a cane or stick, is necessary to help them take a walk outside, which enables them to realize high-quality lives or accelerate the rehabilitation. In these cases, an intelligent cane system may be more useful than walkers due to its flexibility and handiness.
In [19], a Smart- Cane system is also proposed, which has a relative smaller size and nonholonomic constraint in kinematics. The nonholonomic constraint is useful for moving along a path stably, but reduces the maneuverability of the system. In [21] and [22], the “GuideCane” and a robotic cane “Roji” are proposed for blind or visually impaired pedestrians to navigate safely and quickly through obstacles and other hazards. In the living environment, including the narrow space, the cane system is expected to be movable in omni directions. Thus, omni directional mobile platform is needed in the robot design. This kind of platform has been considered in some applications [23], [24], whereas their designs are special and not commercially available. Particularly, they are proposed for walker systems but cane systems, which are much smaller in size. Recently, commercial omniwheels are applied in the area of walker systems [25]. The problem that slender rollers of omniwheels have limited load capacities is partly solved by the modern technology. In addition, a small omnidirectional platform can be constructed by this kind of wheels. In our previous study, an intelligent cane system was designed based on a commercially available three-wheeled omnidirectional platform [26]. We also investigated the fall detection and fall-prevention function of the cane robot systems in [27]. The recognition of user’s walking intention plays an important role in the study of the walker-type rehabilitation robots. From the viewpoint of the control system of robot, the walking intention provides a real-time reference trajectory for the robot motion controller. Therefore, the more accurately the walking intention is inferred, the more satisfactory the control performance of the robot may be obtained.
A manufacturing system that is controlled based on the human intention/desire was proposed in [28]. The dance partner robot, which estimates the intention of human dancer, was proposed in [29] and [30]. When we pay attention to the walker-type walking support system, similar researches can be found in [31] and [32]. In the study of motion–intention-recognizing approaches, the EMG-based methods are widely applied [2], [3]. However, the EMG signals are easily influenced by the location of electrodes, the thickness of fattiness, the body temperature, and the perspiration. Meanwhile, the information of the EMG signals is so large that a complicated pre-processing procedure is required before using them as the control input. In this paper, we improved our former intelligent cane system and studied its motion control problems in several situations based on estimating human walking intention. The human walking behavior is described by switching walking modes. To model the human walking intention, an important concept called “intentional direction” (ITD) is proposed, as well as its dynamic model during human walking. Without knowing the ITD accurately, it is not an easy task to design a motion controller of the cane robot for an elderly or a handicapped user. Normally, these people cannot walk along their ITD clearly and smoothly due to their weak or handicapped lower limbs. For instance, even an elderly intends to walk straightforwardly, he/she might finally walk along a zigzag trajectory because of stumbling. Therefore, the interactive forces measured by the cane robot consist of plentiful user’s unintentional walking information, which is part of the observation noise in the dynamic model of ITD. Comparing with a young healthy subject, apparently this observation noise of ITD is much bigger. Thus, it is necessary to pick up the ITD as accurately as possible from the noisy measurement. After that we may design a robot motion controller based on the ITD to aid the user’s walking in accordance with his actual walking intention. Some filtering technologies are used to online estimate the ITD, based on which a new force control scheme called “intention-based admittance control (IBAC)” is proposed to provide a natural and intuitive interface for elderly users.

EXISTING SYSTEM
In existing method they used cane robot for aiding the elderly and handicapped peoples walking. For many visually impaired people, a cane or a stick is a close friend helping them to detect and avoid obstacles in the walking paths. During walking with the cane, they sense and guess directions and locations by hearing sounds surrounding, sniffing smells in the air, feeling touches on skin, counting footsteps they walk, and memorizing events in time and spaces. However, it is difficult for them to do this all the time when surrounding environment could suddenly change, or when they get lost memory of locations. Voice recognition method is not used in existing method. Cane robot is not user friendly.
An intelligent cane robot is designed for aiding the elderly and handicapped people’s walking. The robot consists of a stick, a group of sensors, and a unidirectional basis driven by three Swedish wheels. Recognizing the user’s walking intention plays an important role in the motion control of our cane robot. To quantitatively describe the user’s walking intention, a concept called “intentional direction (ITD)” is proposed. Both the state model and the observation model of ITD are obtained by enumerating the possible walking modes and analyzing the relationship between the human–robot interaction force and the walking intention. From these two models, the user’s walking intention can be online inferred using the Kaman filtering technique. Based on the estimated intention, a new admittance motion control scheme is proposed for the cane robot. Walking experiments aided by the cane robot on a flat ground and slope are carried out to validate the proposed control approach. The experimental results show that the user feels more natural and comfortable when our intention-based admittance control is applied.
image
The direction to which a person intends to move is referred to as the ITD. The ITD can be evaluated by the angle between the forward direction (along with the axis yr) and the ITD itself. Obviously, the ITD is a time-dependent value and is denoted by ρ(n) in the rest of the paper. Furthermore, the quantity of this intention is characterized by the measured resultant force Fρ (n) along the ITD. Note that discrete time scale n is assumed for the requirement of filtering technology.
In this existing method, a new omni directional-type cane robot was developed for the elderly and handicapped. Motion control of this robot was studied based on online estimating human walking intention. The main contribution of this study has been to present dynamic models and online inference algorithm for the human walking intention, which is significant to lead the user’s walking in a natural and comfortable way. An IBAC scheme was also proposed and used to drive the cane robot. Experiments were performed on the flat ground and slope. The effectiveness of the proposed algorithm was confirmed through experiments. It should be pointed out that the interface between the human and the robot is the multiaxis force sensor, which is expensive and fragile. To lower the cost and improve the system reliability, in the future, we would like to construct a low-cost sensing system comprising cheaper force sensors (e.g., force sensing resistors) and range finding sensors for the cane robot. By utilizing some sensor fusion approaches, the state of user can then be reliably recognized and provided to the motion controller.

PROPOSED SYSTEM

A. Android Voice recognition method
Uses android mobiles internal voice recognition to pass voice commands to your robot Pairs with Bluetooth Serial Modules and sends in the recognized voice as a string. For example if you say Hello the android phone will return a sting *Hello# to your Bluetooth module *and # indicate the start and stop bits. Can be used with any microcontroller which can handle strings. Examples Platforms: Arduino, ARM, PICAXE, MSP430, 8051 based and many other processors and controllers HC-05 embedded Bluetooth serial communication module (can be short for module) has two work modes: order-response work mode and automatic connection work mode. And there are three work roles (Master, Slave and Loopback) at the automatic connection work mode. When the module is at the automatic connection work mode, it will follow the default way set lastly to transmit the data automatically. When the module is at the order-response work mode, user can send the AT command tothe module to set the control parameters and sent control order.. The work mode of module can be switched by controlling the module PIN (PIO11) input level.

B. Answering user questions by robot
The Twin MOS Micro SD Memory Card is functionally compatible with the SD Memory card but is smaller in dimensions. It can be inserted into a passive SD or mini SD memory Card Adapter and operate as an SD Memory Card. Twin MOS Micro SD Card TM is ideal for digital devices designed to use Micro SD Card. All the commands spoken by the user is stored in the memory. When the user asked any queries to the robot, the questions directly mapping with memory card. After that the commands processed by micro processor with the help of interfacing sensors like sonar, digital MEMS, light sensor.finally Replied to the user with the help of speaker.

C. ARM processor and its modules
The LPC1311/13/42/43 operate at CPU frequencies of up to 72 MHz. The ARM Cortex-M3 CPU incorporates a 3-stage pipeline and uses a Harvard architecture with separate local instruction and data buses as well as a third bus for peripherals. The ARM Cortex-M3 CPU also includes an internal prefetch unit that supports speculative branching. The LPC1311/13/42/43 operate at CPU frequencies of up to 72 MHz. The ARM Cortex-M3 CPU incorporates a 3-stage pipeline and uses a Harvard architecture with separate local instruction and data buses as well as a third bus for peripherals. The ARM Cortex-M3 CPU also includes an internal prefetch unit that supports speculative branching. The peripheral complement of the LPC1311/13/42/43 includes up to 32 kB of flash memory, up to 8 kB of data memory, USB Device (LPC1342/43 only), one Fast-mode Plus I2C-bus interface, one UART, four general purpose timers, and up to 42 general purpose I/O pins. This application note describes a practical sonar system, using ultrasonic sound waves to determine the distance to obstacles, using a Microchip PIC16F87XA MCU. By measuring the time between sending the pulse and detecting the echo, the distance to the object can be determined.
The MCP9800 is a digital temperature sensor capable of reading temperatures from -55°C to +125°C. Temperature data is measured from an integrated temperature sensor and converted to digital word with a user selectable 9 to 12-bit Sigma Delta Analog to Digital Converter.
Communication with the sensor is accomplished via a two-wire bus that is compatible with industry standard protocols. This permits reading the current temperature, programming the set point and hysteresis and configuring the device.
The LM35 series are precision integrated-circuit temperature sensors, whose output voltage is linearly proportional to the Celsius (Centigrade) temperature. The LM35 thus has an advantage over linear temperature sensors calibrated in ° Kelvin, as the user is not required to subtract a large constant voltage from its output to obtain convenient Centigrade scaling. The LM35 does not require any external calibration or trimming to provide typical accuracies of ±1⁄4°C at room temperature and ±3⁄4°C over a full −55 to +150°C temperature range. Light dependent resistor is used to find the sun light and tells day or night.
I²C is a multi-master serial computer bus invented by Philips that is used to attach low-speed peripherals to a motherboard, embedded system, or cell phone. USART is also known as a Serial Communications Interface or SCI. for system software μVision4. we used μVision4 integrates all the tools need to develop embedded applications including C/C++ compiler, macro assembler, linker/locator, and a HEX file generator.
image

EXPERIMENTAL RESULTS
In this chapter let us discuss about the implementation of this application. First we divide the whole application into small modules. The system comprises the modules like robotic unit, embedding of hardware and software. Let us see about the flowchart and implementation of the systems in the following sections. Information is provided in natural MP3 quality voice interface via an external speaker. Onboard 2-GB memory card allows storing huge number of audio files including MP3 music. Capacitive Touch keypad eliminates finger pain while using older Braille keys and allows blind people to enter notes and control device operation easily. 100 % hands free operation using voice recognition facility. High quality stereo MP3 makes it sounds natural and pleasant to hear. Sun finding based intelligent charging eliminates maintenance issues. GPS less path following based on compass and distance sensor makes it cost effective In addition, the robot provides user information needed, in audio format, including time, calendar, alarm, navigation direction, ambient light and temperature condition. SONAR is capable of measuring object distance upto 3m. Low power 32-bit ARM Cortex-M3 microcontroller enables highly deterministic operation using battery power only. Here the whole modules divided into into small modules. The system comprises the modules like robotic unit, embedding of hardware and software. Let us see about the flowchart and implementation of the systems in the following sections.
image
image

CONCLUSION
In this project a multipurpose assistance robot is designed from the existing technique which effectively helps visually impaired people with zero maintence. Detection of obstacles with the help of sonar sensors gives more accuracy than any other sensors. In this project we have used voice recognition method which helps to recognize human voice. This recognition method is achieved by android mobile phone
Additionally, robotic dog has the ability to follow a human when commanded with voice. It also plays the role of a regular watchdog during night and barks like any normal dog if it finds any abnormal activity. During the day time it can charge itself by moving around within a given region in order to find the maximum sun light, intelligently avoiding the shaded areas, thereby freeing the user completely from maintenance issues such as battery charging. It has a head; eyes and a tail like a real dog which it uses to perform special gestures during human-robot interaction. Thus the results show that the method is also effective in reducing power and maintence.The proposed scheme has been validated by simulation using voice recognition module and implanted to evaluate its performance in terms of receiving voice and recovered.
It is decided to implement the design for the robotic devices to achieve artificial intelligence which is independent and thinking capability of different operations and assist human kinds.

References
  1. S. Ueki, H. Kawasaki, S. Ito, Y. Nishimoto et al., “Development of a hand-assist robot with multi-degrees-of-freedom for rehabilitation therapy,” IEEE/ASME Trans. Mechatronics [Online]. Available: http://ieeexplore.ieee.org, DOI: 10.1109/TMECH.2010.2090353.
  2. K. Kiguchi, K. Iwami, M. Yasuda, K. Watanabe, and T. Fukuda, “An exoskeletal robot for human shoulder joint motion assist,” IEEE/ASME Trans. Mechatronics, vol. 8, no. 1, pp. 125–135, Mar. 2003.
  3. K. Kiguchi, T. Tanaka, and T. Fukuda, “Neuro-fuzzy control of a robotic exoskeleton with EMG signals,” IEEE Trans. Fuzzy Syst., vol. 12, no. 4, pp. 481–490, Aug. 2004.
  4. M. Mihelj, T. Nef, and R. Riener, “ARMin II—7 DOF rehabilitation robot:Mechanics and kinematics,” in Proc. IEEE Int. Conf. Robot. Autom., Roma, Italy, Apr. 2007, pp. 4120–4125.
  5. J. C. Perry, J. Rosen, and S. Burns, “Upper-limb powered exoskeleton design,” IEEE/ASME Trans Mechatronics, vol. 12, no. 4, pp. 408–417, Aug. 2007.
  6. K. Kong and D. Jeon, “Design and control of an exoskeleton for the elderly and patients,” IEEE/ASME Trans. Mechatronics, vol. 11, no. 4,pp. 428– 432, Aug. 2006.
  7. G. S. Sawicki, K. E. Gordon, and D. P. Ferris, “Powered lower limb orthoses: Applications in motor adaptation and rehabilitation,” in Proc. IEEE Int. Conf. Rehabil. Robot., 2005, pp. 206–211.
  8. C. Fleischer and G. Hummel, “Embedded control system for a powered leg exoskeleton,” in Embedded Systems—Modeling, Technology, and Applications. New York: Springer-Verlag, 2006, pp. 177–185.
  9. C. Mavroidis, J. Nikitczuk, B. Weinberg, G. Danaher, K. Jensen, P. Pelletier, J. Prugnarola, R. Stuart, R. Arango, M. Leahey, R. Pavone,A. Provo, and D. Yasevac, “Smart portable rehabilitation devices,” J. Neuroeng. Rehabil., vol. 2, no. 1, pp. 18–32, 2005.
  10. K. Kasaoka and Y. Sankai, “Predictive control estimating operator’s intention for stepping-up motion by exo-skeleton type power assist system HAL,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Oct. 2001, Maui, HI, pp. 1578–1583.
  11. T. Hayashi, H. Kawamoto, and Y. Sankai, “Control method of robotic suit HAL working as operator’s muscle using biological and dynamical information,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2005, pp. 3455–3460.
  12. K. Yamamoto, M. Ishii, K. Hyodo, T. Yoshimitsu, and T. Matsuo, “Development of power assisting suit (miniaturization of supply system to realize wearable suit),” JSME Int. J., Ser. C, vol. 46, no. 3, pp. 923–930, 2003.
  13. J.Yoon, B. Novandy, C.Yoon, andK. J. Park, “A 6-DOF gait rehabilitation robot with upper and lower-limb connections that allows walking velocity updates on various terrains,” IEEE/ASME Trans. Mechatronics, vol. 15, no. 2, pp. 201–215, Apr. 2010.
  14. G. Lee, T. Ohnuma, and N. Y. Chong, “Design and control of JAIST active robotic walker,” Intell. Serv. Robot., vol. 3, no. 3, pp. 125– 135, 2010.
  15. H. Mori and S. Kotani, “A robotic travel aid for the blind—Attention and custom for safe behavior,” in International Symposium of Robotics Research, New York: Springer-Verlag, 1998, pp. 237–245.
  16. M. Fujie, Y. Nemoto, S. Egawa, A. Sakai, S. Hattori, A. Koseki, and T. Ishii, “Power assistedwalking support andwalk rehabilitation,” in Proc. 1st Int. Workshop Humanoid Human Friendly Robot., 1998, pp. 2693–2695.
  17. J. Manuel, H.Wandosell, and B. Graf, “Non-holonomic navigation system of awalking-aid robot,” in Proc. IEEEWorkshop Robot Human Interactive Commun., 2002, pp. 518–523.
  18. N. Roy, G. Baltus, D. Fox, F. Gemperle, J. Goetz, T. Hirsch, and D. Margaritis, “Towards personal service robots for the elderly,” presented at the Workshop Interactive Robot. Entertain., 2000, Pittsburgh, PA.
  19. H. Yu, M. Spenko, and S. Dubowsky, “An adaptive shared control system for an intelligent mobility aid for the elderly,” Auton. Robots, vol. 15, no. 1, pp. 53–66, 2003.
  20. Y. Hirata, A. Hara, and K. Kosuge, “Motion control of passive intelligent walker using servo brakes,” IEEE Trans. Robot., vol. 23, no. 5, pp. 981– 990, Oct. 2007.
  21. I. Ulrich and J. Borenstein, “The guidecane: Applying mobile robot technologies to assist the visually impaired,” IEEE Trans. Syst.,Man, Cybern., A, Syst. Humans, vol. 31, no. 2, pp. 131–136, Mar. 2001.
  22. I. Shim, J. Yoon, and M. Yoh, “A human–robot interactive system RoJi,” Int. J. Control, Autom., Syst., vol. 2, no. 3, pp. 398–405,
  23. H. Yu and S. Dubrowsky, “Omnidirectional mobility using active split offset castors,” presented at the ASME IDETC/CIE 26th Biennial Mech. Robot. Conf., Sep. 10–13, 2006, Baltimore, MD.
  24. Y. Hirata, T. Baba, and K. Kosuge, “Motion control of omnidirectionaltype walking support system “Walking Helper”,” in Proc. IEEEWorkshop Robot Human Interactive Commun., 2003, pp. 85–90.
  25. J. Huang, P. Di, T. Fukuda, and T. Matsuno, “Motion control of omnidirectional-type cane robot based on human intention,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Nice, France, 2008, pp. 273– 278.
  26. J. Huang, P. Di, K. Wakita, T. Fukuda, and K. Sekiyama, “Study of fall detection using intelligent cane based on sensor fusion,” in Proc. IEEE Int. Symp. Micronanomechatron. Human Sci., 2008, pp. 495–500.
  27. Y. Yamada, Y. Umetani, H. Daitoh, and T. Sakai, “Construction of a human/robot coexistence system based on a model of human willintention and desire,” in Proc. IEEE Int. Conf. Robot. Autom., Detroit, MI, 1999, pp. 2861–2867.
  28. K. Kosuge, T. Hayashi, Y. Hirata, and R. Tobiyama, “Dance partner robot: Ms DanceR,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., LasVegas, NV, 2003, pp. 3459–3464.
  29. T. Takeda, Y.Hirata, andK. Kosuge, “Dance step estimation method based on HMM for dance partner robot,” IEEE Trans. Ind. Electron., vol. 54, no. 2, pp. 699–706, Apr. 2007.
  30. K. T. Song and J. Y. Lin, “Behavior fusion of robot navigation using a fuzzy neural network,” in Proc. IEEE Int. Conf. Syst.,Man, Cybern., 2006, pp. 4910–4915.
  31. O. Chuy, Y. Hirata, and K. Kosuge, “A new control approach for a robotic walking support system in adapting user characteristics,” IEEE Trans. Syst., Man, Cybern., vol. 36, no. 6, pp. 725–733, Nov. 2006.
  32. Y. Hirata, A. Muraki, and K. Kosuge, “Motion control of intelligent passive-type walker for fall-prevention function based on estimation of user state,” in Proc. IEEE Int. Conf. Robot. Autom., May 2006, pp. 3498– 3503.
  33. L. Xie, Y. C. Soh, and C. E. de Souza, “Robust Kalman filtering for uncertain discrete-time systems,” IEEE Trans. Automat. Control, vol. 39, no. 6, pp. 1310–1314, Jun. 1994.