K.

Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN 2278-5841, Vol 1, Issue 6, November 2012.

Voice and Gesture Based Electric-Powered Wheelchair Using ARM

K. SUDHEER1, T.V.JANARDHANA RAO2, CH. SRIDEVI3, M.S.MADHAN MOHAN4
M.Tech Student(Embedded Systems), 2Professor , 3,4Associate professor, Department of Electronics and Communication Engineering, B.V.C. Engineering College, Odalarevu, AP, INDIA 4 E- mail : 1sudheer.kanuri@gmail.com, 2dean.admissions@bvcgroup.in, 3chavakula.sridevi@gmail.com, msmm141@gmail.com
ABSTRACT A voice and gesture based system has been developed to control a wheelchair using voice commands and moment of hand i.e. Mems sensor is connected to hand. The system is divided into two main components: speech recognition module with Mems sensor and wheelchair control unit. In the speech recognition module, hidden Markov models are used as the main method in recognizing voice commands. The Mems sensor senses the angle of the hand, i.e. according to the tilt of hand it gives voltages to microcontroller. The wheelchair control unit is developed using ARM controller.
1

for developing an advanced wheelchair. Traditional wheelchairs used by the elderly and handicapped severe disabilities have some limited functions and flexibility several teams are currently at work to develop robotic wheelchairs to overcome all these limitations. In this project we are trying to include some sensors to develop an automated wheelchair which can help the patient to control the direction of the wheelchair based o voice and gesture[3]. In existing systems pc will be used for geture recognition and processing of voice commands[1] . hence along with wheelchair a pc has to be equiped which increases complexity in system. That complexity will be reduced by using voice recognition IC and MEMS accelerometer which is a very small IC placed on the finger tips of the patient. Block Diagram: Implementation of this proposed problem mainly involves three steps. They are voice recognition, gesture recognition and controlling direction of wheelchair using microcontroller based on the received voice or gesture commands by making use of motors. These motors will be driven by controller. The block diagram of the system is shown in Figure 1. DC motors are fixed to the wheels of wheelchair hence based on rotation of motor direction of wheelchair will be controlled. Motors will be interfaced to microcontroller through motor drivers.

Keywords- Voice Recognition, Gesture recognition , ARM controller. I .INTRODUCTION Todays world comprises of a large variety of people. Some of them depend on others for their living. But in todays fast world, everyone is busy and there are less people to care for the increasing number of elderly and the physically challenged people. Also these people find it tough to even navigate inside the home without external aids. The elderly people find automated wheelchairs as an easy way for locomotion. Having known about these facts, our aim was to bring an automated navigation system which can be used by both the elderly and the physically challenged people in a user friendly manner using voices and hand gestures for operation. In biomedical sector, a wheelchair is an important device because of the recent shift in the industrial populations. The demand of the physically handicapped and the aged are ever rising. The present wheelchairs do not have integration of technologies for their working. It either requires constant monitoring by the helper or hence lot of effort. The recent developments in the robotics artificial intelligence or sensor technology promises enormous scope

www.ijrcct.org

chiefeditor@ijrcct.org

Page 278

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN 2278-5841, Vol 1, Issue 6, November 2012.
1. 2. 3. 4.

Figure 1: Block diagram Voice recognition module recieves the voice of the speaker and sends commands corresponding to the recieved voice command to controller. Gesture recognition module sends command to the microcontrolller based on the geture of the user. Control unit controls the direction of rotation of motors. II. SPEECH RECOGNITION MODULE Related Work Various system applications using automatic speech recognition (ASR) technology have been in the market for just over 10 years [7]. In 1994, the Dragon Systems released its DragonDictate for Windows 1.0, using discrete ASR technology, which is a word based ASR. Discrete ASR technology requires the user to pause after each word making the dictation unnatural for the user. In addition, the performance of discrete ASR is slow and usually unacceptable. In 1996, the first continuous speech recognition system, called MedSpeak/Radiology, was released by IBM. Continuous speech technology allows users to speak in a natural manner conversationally, as they are speaking to another person. At the time of their releases, these systems were often priced to be over 10,000 dollars and required very expensive computer systems to support them to function as desired [7]. Now instead of using cost effective computer systems directly speech recognition modules can be used which reduces complexity as well as size of the total system There are two essential characteristics for any effective ASR system: accuracy and speed. In addition, to meeting these demands, ASR systems face a number of additional challenges including the large variance that exists among individual human speech patterns (e.g. pitch, rate, inflection). A successful ASR system requires extraordinary flexibility to accommodate these variances. The process of ASR typically follows these steps:

5.

Acoustic speech is captured by a microphone and undergoes analog-to-digital conversion. Short time frames of speech are converted into signal features that can be used to discern speech components. Series of frames are mapped to phonemes, the smallest building blocks of speech, and series of phonemes are mapped to words. Contextual information is simultaneously processed to increase the accuracy of the ASR system. For example, contextual information would be able to assist in the selection of a series of homonyms (e.g. merry, marry, and Mary). The series of words are determined based on the probabilistic measure and provided in a format suitable for the next stage in the application (e.g. text format).

Hidden Markov Model Hidden Markov models (HMMs) are widely used in the area of ASR [8]. HMMs are comprised of two main components: a Markov chain of states, which is the hidden portion of the model, and statistical descriptors for each state in the Markov chain that are associated with the probabilistic nature of the observed data. This structure enables the HMM to simultaneously characterize local signal properties (i.e. regularities in the signal that occur in a short time span) and global signal properties by modeling local signal properties as a sequence of events. As speech can be considered a quasistatic signal, the structure of HMMs make it ideal for ASR. Proposed model From the above procedures, to implement speech recognition system it requires a pc but it is very difficult for a patient on wheelchair to carry a PC along with him hence for speech recognition HM2007 IC based SR07 circuit can be used[3] which reduces circuit complexity. This circuit recieves user voice commands through microphone and processes and then sends 8 bit data output corresponding to the recieved command which can be interfaced with any microcontroller Features: 1 It has a stand alone speech recognition ciruit. 2 User programmable 3 It can recognize up to 20 word vocabulary of duration two second each 4 Multi-lingual

www.ijrcct.org

chiefeditor@ijrcct.org

Page 279

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN 2278-5841, Vol 1, Issue 6, November 2012.
Non-volatile memory back up with 3V battery onboard. Will keep the speech recognition data in memory even after power off. 6 Easily interfaced to control external circuits & appliances The heart of the speech recognition circuit is the HM2007 speech recognition IC. 5

HM2007 can recognize 20 words each of length 1.92 second or equivalently it can recognize up to 40 words each of length 0.98 seconds[4]. In this ciruit along with HM2007 there are decoders keypads and digital display. The keypad and digital display are used to communicate and program the HM2007 chip. The keypad have 12 normally open contact switches. When the circuit is turned on, 00 is on the digital display, the red LED (READY) is lit and the circuit waits for a command.

Speech recognition is divided into two broad processing categories; speaker dependent and speaker independent. Speaker dependent systems are trained by the individual who will be using the system. These systems are capable of achieving a high command count and better than 95% accuracy for word recognition. The drawback to this approach is that the system only responds accurately only to the individual who trained the system. Speaker independent is a system trained to respond to a word regardless of who speaks. Therefore the system must respond to a large variety of speech patterns, inflections and enunciation's of the target word. The command word count is usually lower than the speaker dependent however high accuracy can still be maintain within processing limits. Speech recognition circuit is shown in Figure 3.

Figure 2: SR07 Speech Recognition System Press 1 (display will show 01 and the LED will turn off) on the keypad, then press the TRAIN key ( the LED will turn on) to place circuit in training mode, for word one. Say the target word into the onboard microphone (near LED) clearly. The circuit signals acceptance of the voice input by blinking the LED off then on. If any errors occur the following codes will be displayed based on the type of error Error Codes: The chip provides the following error codes. 55 = word to long 66 = word to short 77 = no match Once stored commands or data can be tested. Data present in IC can be cleared by pressing 99 and then CLR. Then all the numbers will quickly scroll on the digital display and data will be erased Figure 3: Speech Recognition IC III. GESTURE RECOGNITION Gesture means A movement of part of the body, esp. a hand or the head, to express an idea or meaning. Here the project is implemented based on hand gesture. Based on the hand movement wheelchair will be controlled. To recognise this hand gesture MEMS accelerometer[2] is ussed. Micro Electro Mechanical Systems (MEMS) is the integration of mechanical elements, sensors, actuators, and electronics on a common silicon substrate through micro fabrication technology. MEMS is an enabling technology allowing the development of smart products, augmenting the computational ability of microelectronics In most cases, the physics behind the behavior of MEMS devices can be expressed by mathematical expressions. MEMSolver works by creating a mathematical model of the system and generates analytical solutions to explain the behavior of the MEMS device. The user just has to enter the input parameters like length and width of the beam for example in a user friendly GUI, and the software will immediately calculate the relevant

www.ijrcct.org

chiefeditor@ijrcct.org

Page 280

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN 2278-5841, Vol 1, Issue 6, November 2012.
results and plot graphs that fully explain the MEMS device or part of it.

MEMS accelerometer is a single chip with small size and low cost. Because of their small size and weight. Accelerometers are attached to the fingertips and back of the hand. Arrows on the hand show the location of accelerometers and their sensitive directions. That the sensitive direction of the accelerometer is in the plane of the hand. In this praposed model MMA7361L accelerometer is used. An accelerometer is an electromechanical device that measures acceleration forces. The device can be used for sensor data changes, product orientation, and gesture detection. This is 3-axes accelerometer which produces three voltages in three directions based on the tilt[4] of the accelerometer. Orientation can be detected based on the capacitance produced between parallel plates. The figure 4 shows the detection of orientation. In this there are two fixed and one movable plates based on the orientation, position of movable plate changes which results in change in capacitance Figure 5: Voltages corresponding to orientation Finally this accelerometer produces three output votages in three directions corresponding to each orientation which are in analog form hence they will be interfaced to controller through analog to digital conversion pins as shown in figure 6.

Figure 4: Orientation detection Based on the tilt of the accelerometer different voltages will be generated in three directions i.e., X,Y and Z. Voltages that will be produced corresponding to tilt are shown in figure 5.

Figure 6: Accelerometer interfacing to controller VI. WORKING Speech recognition module receives voice commands of the user and sends binary code corresponding to the recieved command to the microcontroller. Similarly gesture recognition module sends analog voltages in three directions corresponding to each orientation

www.ijrcct.org

chiefeditor@ijrcct.org

Page 281

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN 2278-5841, Vol 1, Issue 6, November 2012.
to the microcontroller. Output of the speech recognition module and gesture recognition module are interfaced to controller. This controller controls the wheelchair based on recieved gesture or voice command. To control wheelchair DC motors will be drived through the microcontroller where DC motors are connected to the wheels of wheelchair, but DC motors will not be driven directly by microcontroller as thier operating voltages are not compatible. Hence to drive motor L293D motor driver is used. V. RESULTS AND DISCUSSION By using above procedure hardware setup is done figure 7 shows input part that is interfacing of speech recognition module and accelerometer to ARM controller.

Motors will be fixed to front wheels of wheelchair and hence based on the rotation of motors direction of wheelchair will be controlled.

In this system for the operation of hardware software is needed. Controller has to be programmed to perform the required task. the code was written on assembly language using Keil Vision-3 software. The assembly code now converted into Hex file. Now the code was burn onto the controller. Table 1: Tesa results based on voice commands No.of words spoken 20 45 No.of times system correctly responded 19 43

Speaker Speaker-I Speaker-II

Accuracy 95% 95.5%

The recognition rate of the voice commands is 72.5% using the Speech recognized electronic device controller, where as it is improved by using HM2007 IC based SR07 voice recognition module and it produces 95% accuracy .in voice recognition as shown in Table 1. Gesture recognition also produces more than 90% of accuracy from test results. VI. CONCLUSION Figure 7: Hardware setup input part Controller controls wheelchair by properly controlling the direction of rotation of motors hence motors will be interface to controller through the motor driver as shown in figure 8. By using this sytem physically challenged people finds easy way to navigate within the home using wheelchair without any external aid. This provides ease of operation as the system is user friendly and size is small. Voice commands will be received through microphone and for gesture recognition small IC will be placed at finger tips. This system canbe extended by including GSM which sends an SMS during emergency by assigning pirticular voice or gesture command. By including GPS position of the wheelchair can also be known. VII. REFERENCES 1. Rajesh Kannan Megalingam, Ramesh Nammily Nair, Sai Manoj Prakhya, Automated Voice based Home Navigation System for the Elderly and the Physically Challengedproc.ICACT2011P .603-08,Feb.2011. O.Mirabella, M.Brischetto, G.Mastroeni Mems based gesture recognition, proc.HSI P.599 604, May

2.

2010
Figure 8: hardware setup output part

www.ijrcct.org

chiefeditor@ijrcct.org

Page 282

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN 2278-5841, Vol 1, Issue 6, November 2012.
3. 4. 5. 6. R.Posada Gomez, L.H.Sanchez Medel, A Hand Gesture System Of Control For An Intelligent Wheelchair,Proc.ICEEE,P.68-71,5-7,sept.2007. http://www.analog.com/en/content/over_five_motion_sen ses/fca.html,1995-2012. J.Z. Yi , Y.K. Tan Z.R. Ang, Microcontroller Based Voice-Activated Powered Wheelchair Control, ACM publishers, 2007. C.Chandramouli and Vivek Agarwal, Speech Recognition based Computer Keyboard Replacement for the Quadriplegics, Paraplegics, Paralytics and Amputees, ACM publishers, Munro, Jay. "Watch What You Say." PC Magazine Online. March 10, 1998. http://www.zdnet.com/pcmag/ features/speech/intro1.html (23 October 1998). Rabiner L. R., A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proc

7. 8.

www.ijrcct.org

chiefeditor@ijrcct.org

Page 283