Voice controlled Robotic hand for disabled people

1.INTRODUCTION
When we say about voice control, the first thing come into our mind is Speech recognition which
makes system to understand the human voice/commands. Now, depending upon this voice
commands our robotic hand is going to be controlled. First thing has to be cleared is model of the
voice recognition module. There are two types of voice recognition modules namely V2 and V3.
To build up the strong robotic hand for lifting high weights high torque motors should be used
where DOF’s are preferred. As of the small robotic hand servo motors are preferred. The
controlling of the robotic hand is controlled with micro controller namely Arduino UNO. When
we give a voice command it is driven to controller to perform the operation of the robotic hand.

1.1. BLOCK DIAGRAM:

Figure 1.1 Block diagram of Robotic hand

The block diagram shows all the major components that are required for implementation of the
robotic hand. Here, we have used Micro phone which is connected to the voice recognition module
of V3 model interfaced with micro controller connected. Power supply required for voice
recognition module, Arduino and Servos is 5V. Driving of 5V from Arduino as input for servos is
not preferable, as power supply drops. Microcontroller will generate the control signals to operate
the five servos of the robotic arm.

Components used:

➢ Voice Recognition Module

➢ Micro phone
➢ Arduino UNO
➢ Servo Motors
➢ Power Supply (9V)
➢ Physical Structure of Robotic hand
➢ 7805 Regulators

Department of ECE, GMRIT Page 1

 Voice controlled Robotic hand for disabled people

2.LITERATURE SURVEY
In the robotics field, several research efforts have been directed towards recognizing human gestures. Few
popular systems are:

2.1. VISION-BASED GESTURE RECOGNITION:

This recognition system basically worked in the field of Service Robotics and the researchers finally
designed a Robot performing the cleaning task. They designed a gesture-based interface to control a
mobile robot equipped with a manipulator. The interface uses a camera to track a person and recognize
gestures involving arm motion. A fast, adaptive tracking algorithm enables the robot to track and follow
a person reliably through office environments with changing lighting conditions. Two gesture
recognition methods i.e. a template-based approach and a neural based approach were compared and
combined with the Viterbi algorithm for the recognition of gestures defined through arm motion. It
results in an interactive clean-up task, where the user guides the robot to go to the specific locations
that need to be cleaned and also instructs the robot to pick up trash.

2.2. MOTION CAPTURE SENSOR RECOGNITION:

This recognition technique made it possible to implement an accelerometer-based system to
communicate with an industrial robotic arm wirelessly. In this particular project the robotic arm is
powered with ARM7 based LPC1768 core. MEMS is a three-dimensional accelerometer sensor which
captures gestures of human-arm and produces three different analog output voltages in three
dimensional axes. And two flex sensors are used to control the gripper movement.

2.3. ACCELEROMETER BASED GESTURE RECOGNITION:

This Gesture Recognition methodology has become increasingly popular in a very short span of time.
The low moderate cost and relatively small size of the accelerometers are the two factors that makes it
an effective tool to detect and recognize human body gestures. Several studies have been conducted on
the recognition of gestures from acceleration data using Artificial Neural Networks (ANNs).

Department of ECE, GMRIT Page 2

 Voice controlled Robotic hand for disabled people

3.THEORETICAL STUDY

3.1. VOICE RECOGNITION MODULE:

• This module is a compact and easy-control speaking recognition board. It supports up to
80 voice commands in all with each voice of 1500ms. Maximum 7 voice commands could
work at a time.

• These 80 voice commands are stored in one large group like a library. Any 7 voice
commands in the library could be imported into recognizer.

Figure 3.1 Voice Recognition module (V3 Model)

3.1.1. PARAMETERS:

• Input voltage: 4.5-5.5V

• Current: <40mA

• Analog interface: 3.5mm mono-channel microphone connector + microphone interface.

• Recognition accuracy: 99% (under ideal environment)

• It supports multi-languages and Arduino library is supplied.

While speech recognition is the process of converting speech to digital data, voice recognition is
aimed toward identifying the person who is speaking. Voice recognition works by analyzing the
features of speech that differ between individuals. Everyone has a unique pattern of speech
stemming from their anatomy (the size and shape of the mouth and throat) and behavioral patterns
(their voice’s pitch, their speaking style, accent, and so on). The applications of voice recognition
are markedly different from those of speech recognition. Most commonly, voice recognition
technology is used to verify a speaker’s identity or determine an unknown speaker’s identity.
Speaker verification and speaker identification are both common types of voice recognition.

Department of ECE, GMRIT Page 3

 Voice controlled Robotic hand for disabled people

Speaker verification is the process of using a person’s voice to verify that they are who they say
they are. Essentially, a person’s voice is used like a fingerprint. Once a sample of their speech is
recorded, a person’s speech patterns are tested against a database to see if their voice matches their
claimed identity.

3.2. TRAINING OF VOICE MODULE:

Figure 3.2 Settings of voice recognition module

After all the setting above, you can speak or send voice instruction to it. If identified successfully,
result will be returned via serial port in the format:

• <Sigtrain> <sigtrain voice_command_no> < voice_command_string>

Eg: sigtrain 0 hold

• Load – load r0 r1….

• Clear – clear

• Record – record/record r0 r1... – it checks recognizer train status vr – vr – it checks

recognizer status.

Department of ECE, GMRIT Page 4

 Voice controlled Robotic hand for disabled people

3.3. TRAINING STATUS:

Figure 3.3 Training Voice recognition module with commands

3.3.1. RECORDING STAGE:

• Record indication: D1 (RED) flashes 3 times within the 600ms, then off for400ms,
and then flashes quickly for 4 times within 600ms. Now the recording indication is
over.
• Begin to speak: D1 (RED) is off for 400ms, and then is on. Voice during the time
while D1 (RED) is on will be recorded by this module.
• Recording a voice instruction successfully for the first time: D1 (RED) off, D2
(ORANGE) on for 300ms.
• Recording a voice instruction successfully for the first time: D1 (RED) off, D2
(ORANGE) on for 700ms.
• Recording failure: D2 (ORANGE) flashes 4 times within the 600ms. In cases that
voice instructions detected twice don’t match, or the sound is too large, or there is
no sound, recording will fail. You need to start over the recording process for that
instruction.

Department of ECE, GMRIT Page 5

 Voice controlled Robotic hand for disabled people

Initially we have trained the voice recognition module with few of the voice commands[1], namely

HOLD, PINCH, INDEX, THUMB, RELEASE.

3.4. ALGORITHM:

start

Check input
port

Yes HOLD
If
Condition1

No

Yes PINCH
If
Condition2

No

Yes THUMB
If
Condition3

No

Yes INDEX
If
Condition4
No

Yes RELEASE
If
Condition5

Figure 3.4 Algorithm of Robotic hand

Department of ECE, GMRIT Page 6

 Voice controlled Robotic hand for disabled people

3.5. SERVO MOTOR:

Servo motors are DC motors that allows for precise control of angular position. They are actually
DC motors whose speed is slowly lowered by the gears. The servo motors usually have a revolution
cut off from 90° to 180°. A few servo motors also have revolution cut off of 360° or more. But servo
motors do not rotate constantly. Their rotation is limited in between the fixed angles. A servo motor
consists of three wires- a black wire connected to ground, a white/yellow wire connected to control
unit and a red wire connected to power supply. The function of the servo motor is to receive a
control signal that represents a desired output position of the servo shaft and apply power to its DC
motor until its shaft turns to that position.

3.5.1. CONTROLLING A SERVO MOTOR

A servo motor is controlled by controlling its position using Pulse Width Modulation Technique.
The width of the pulse applied to the motor is varied and send for a fixed amount of time. The
pulse width determines the angular position of the servo motor. For example, a pulse width of 1ms
causes an angular position of 0 degrees, whereas a pulse width of 2ms causes an angular width of
180 degrees.

Commands on Arduino:

• servo.attach(pin) - servo.attach(9);
• servo.write(angle) – servo.write(90);

Figure 3.5 Internal diagram of Servo motor

Department of ECE, GMRIT Page 7

 Voice controlled Robotic hand for disabled people

3.6. ARDUINO:

• Arduino Uno is a microcontroller board developed by Arduino.cc which is an open-source

electronics platform mainly based on AVR microcontroller Atmega328.

• The current version of Arduino Uno comes with USB interface, 6 Analog input pins, 14 I/O
digital ports that are used to connect with external electronic circuits. Out of 14 I/O ports, 6
pins can be used for PWM output.

• Arduino Uno are the most official versions that come with Atmega328 8-bit AVR Atmel
microcontroller where RAM memory is 32KB.

• Atmega328 microcontroller is placed on the board that comes with a number of features like
timers, counters, interrupts, PWM, CPU, I/O pins and based on a 16MHz clock that helps in
producing more frequency and number of instructions per cycle.

Figure 3.6 Arduino

Department of ECE, GMRIT Page 8

 Voice controlled Robotic hand for disabled people

4. EXPERIMENTAL STUDY
4.1. CONNECTIONS:

-
POWER
SUPPLY
+
Figure 4.1 Circuit diagram of Robotic hand

Voice recognition module connections with Arduino:

Servo connections to Arduino and Power Supply(5V):

✓ RED wire : +Vcc – 5v.
✓ BROWN wire : GND.
✓ ORANGE wire : Control pin of Arduino.

Department of ECE, GMRIT Page 9

 Voice controlled Robotic hand for disabled people

4.2. METHODOLGY:
• This project presents the working of a robotic hand using voice control. The main aim of
this project is that it can be very useful for paralyzed people or people with impairments.

• The first term to be considered in voice control is Speech Recognition i.e. making the
system to understand human voice. Speech recognition is a technology where the system
understands the words (not its meaning) given through speech which are recognized by a
mic.

• The speech recognizer depends basically on HM2007 processor which is the heart of the
speech recognizer. The Analog voice input is transformed to digital commands. These
commands are used as input to the Arduino.

• Microcontroller also provides a PWM signal for controlling the speed and direction of
rotation of the motor. The PWM output of the controller is fed to the servo motors that
drive the motors of the robotic arm.

• During working, when the user says a particular trained word into the microphone of the
speech recognition module, the words are recognized by it and the digital output
corresponding to the voice command is provided to the microcontroller.

• Initially the output from the voice recognition module is of the state 00

• The corresponding BCD output of the speech recognition kit is sent to the Arduino
microcontroller. According to the inputs received by the Arduino, appropriate signals are
now sent to the motors to rotate the required motor in the specific direction.

• In this way, one can operate this robotic arm similar to the real hand such as holding and
releasing the objects, moving fingers freely. All these operations can be done by
recognizing the corresponding voice commands which is very realistic.

Department of ECE, GMRIT Page 10

 Voice controlled Robotic hand for disabled people

4.3. CODE:

Training Program of Voice Recognition Module:

Figure 4.2 Software Library to train module

Have to include the library VoiceRecognitionV3-mastertrain and select vr_sample_train to train

voice commands.

4.3.1. TRAINING COMMANDS:

Figure 4.3 Training commands

Department of ECE, GMRIT Page 11

 Voice controlled Robotic hand for disabled people

4.4. SOURCE CODE:

#include <SoftwareSerial.h>

#include "VoiceRecognitionV3.h"

#include <Servo.h>

/** .*

Connection

Arduino VoiceRecognitionModule

2 -------> TX

3 -------> RX

*/

VR myVR(2,4); // 2:RX 3:TX, you can choose your favourite pins.

uint8_t records[7]; // save record

uint8_t buf[64];

Servo servo1;

Servo servo2;

Servo servo3;

Servo servo4;

Servo servo5;

int led = 13;

#define hold (0)

#define relese (1)

#define thumb (2)

Department of ECE, GMRIT Page 12

 Voice controlled Robotic hand for disabled people

#define index (3)

/**

@brief Print signature, if the character is invisible,

print hexible value instead.

@param buf --> command length

len --> number of parameters

*/

void printSignature(uint8_t *buf, int len)

int i;

for(i=0; i<len; i++){

if(buf[i]>0x19 && buf[i]<0x7F){

Serial.write(buf[i]);

else{

Serial.print("[");

Serial.print(buf[i], HEX);

Serial.print("]");

/**

@brief Print signature, if the character is invisible,

Department of ECE, GMRIT Page 13

 Voice controlled Robotic hand for disabled people

print hexible value instead.

@param buf --> VR module return value when voice is recognized.

buf[0] --> Group mode(FF: None Group, 0x8n: User, 0x0n:System

buf[1] --> number of record which is recognized.

buf[2] --> Recognizer index(position) value of the recognized record.

buf[3] --> Signature length

buf[4]~buf[n] --> Signature

*/

void printVR(uint8_t *buf)

Serial.println("VR Index\tGroup\tRecordNum\tSignature");

Serial.print(buf[2], DEC);

Serial.print("\t\t");

if(buf[0] == 0xFF){

Serial.print("NONE");

else if(buf[0]&0x80){

Serial.print("UG ");

Serial.print(buf[0]&(~0x80), DEC);

else{

Serial.print("SG ");

Serial.print(buf[0], DEC);

Department of ECE, GMRIT Page 14

 Voice controlled Robotic hand for disabled people

Serial.print("\t");

Serial.print(buf[1], DEC);

Serial.print("\t\t");

if(buf[3]>0){

printSignature(buf+4, buf[3]);

else{

Serial.print("NONE");

Serial.println("\r\n");

void setup()

/** initialize */

myVR.begin(9600);

servo1.attach(3);

servo2.attach(5);

servo3.attach(6);

servo4.attach(9);

servo5.attach(11);

Department of ECE, GMRIT Page 15

 Voice controlled Robotic hand for disabled people

Serial.begin(115200);

Serial.println("Elechouse Voice Recognition V3 Module\r\nControl LED sample");

pinMode(led, OUTPUT);

if(myVR.clear() == 0){

Serial.println("Recognizer cleared.");

}else{

Serial.println("Not find VoiceRecognitionModule.");

Serial.println("Please check connection and restart Arduino.");

while(1);

if(myVR.load((uint8_t)hold) >= 0){

Serial.println("hold loaded");

if(myVR.load((uint8_t)relese) >= 0){

Serial.println("relese loaded");

if(myVR.load((uint8_t)thumb) >= 0){

Serial.println("thumb loaded");

if(myVR.load((uint8_t)index) >= 0){

Serial.println("index loaded");

Department of ECE, GMRIT Page 16

 Voice controlled Robotic hand for disabled people

void loop()

int ret;

ret = myVR.recognize(buf, 50);

if(ret>0){

switch(buf[1]){

case hold:

/** turn on LED */

servo1.write(0);

servo2.write(0);

servo3.write(0);

servo4.write(0);

servo5.write(0);

break;

case relese:

/** turn on LED */

servo1.write(180);

servo2.write(180);

servo3.write(180);

servo4.write(180);

servo5.write(180);

break;

Department of ECE, GMRIT Page 17

 Voice controlled Robotic hand for disabled people

case thumb:

/** turn on LED */

servo1.write(0);

servo2.write(0);

servo3.write(0);

servo4.write(0);

servo5.write(180);

break;

case index:

/** turn on LED */

servo1.write(0);

servo2.write(180);

servo3.write(0);

servo4.write(0);

servo5.write(0);

break;

default:

Serial.println("Record function undefined");

break;

/** voice recognized */

printVR(buf);

Department of ECE, GMRIT Page 18

 Voice controlled Robotic hand for disabled people

5. RESULTS AND DISCUSSIONS:

Figure 5.1 Hardware Robotic Hand

Department of ECE, GMRIT Page 19

 Voice controlled Robotic hand for disabled people

6. CONCLUSIONS AND FUTURE SCOPE

Our Robotic hand can be very useful for paralyzed people or people with impairments. It can be
controlled as a realistic hand which performs the tasks easily.

REFERENCES:
[1] International Research Journal of Engineering and Technology (IRJET) Volume: 04
Issue: 04 | Apr -2017.

Department of ECE, GMRIT Page 20

 Voice controlled Robotic hand for disabled people

CERTIFICATES:

Department of ECE, GMRIT Page 21