COMSATS Institute of Information Technology

Hand Gesture Controlled Robotic Vehicle

A Thesis Presented to
COMSATS University of Information Technology, Abbottabad

In partial fullfillment of the requirement for the degree of

BS (Electrical Computer Engineering)

By

Muhammad Ilyas
CUIT/SP15-BCE-047/ATD

Mujahid Hussain
CUIT/SP15-BCE-055/ATD

Farhad hussian
CUIT/SP15-BCE-021/ATD

SPRING, 2018
Table of Contents

1.INTRODCUTION...................................................................................................................... 3
1.1 Robot ........................................................................................................................... 3
1.2 Human Machine .......................................................................................................... 3
1.3 Motivation For Project ................................................................................................... 3
1.4 Gesture Controlled Robot ............................................................................................. 4
1.5 Applications................................................................................................................. 4

2.LITERATURE REVIEW………………………………………………………………….6
2.1HAND Gesture Controlled Robot………………………………………………………….6
2.2Transmitting Section……………………………………………………………………… 6
2.3Receiving Section………………………………………………………………………….7
2.4Accelerometer …………………………………………………………………………….7
2.5Arduino UNO………………………………………………………………………………8
2.6Communication ………………………………………………………………………….10
2.7RF PAIR …………………………………………………………………………..10
2.8Motor Driver (L293D)…………………………………………………………………...11
2.9Dc Motor ………………………………………………………………………………13
 CHAPTER 1
HAND GESTURE CONTROLLED MOBILE ROBOTIC ARM
VEHICLE
INTRODUCTION

Nowadays, robots are used in the different domains ranging from search and rescue in the
dangerous environments to the interactive entertainments. The more the robots are employed
in our daily life, the more a natural communication with the robot is required. Since the
introduction of the most common input computer devices not a lot have changed. This is
probably because the existing devices are adequate. It is also now that computers have been
so tightly integrated with everyday life, that new applications and hardware are constantly
introduced. The means of communicating with computers at the moment are limited to
keyboards, mice, light pen, trackball, keypads etc. These devices have grown to be familiar
but inherently limit the speed and naturalness with which we interact with the computer. On
the other hand, hand gesture, as a natural interface means, has been attracting so much
attention for interactive communication with robots in the recent years.

1.1 Robot

A machine that resembles a living creature in being capable of moving

independently (as by walking or rolling on wheels) and performing complex actions
(such as grasping and moving objects). A device that automatically performs
complicated, often repetitive tasks (as in an industrial assembly line) .

1.2 Hand Gesture

Everyone “talks” with their hands at least sometimes. Some people’s hand-talking or
gesturing matches their message well. Other people have a tendency to make overly large
gestures that can be distracting. And still others don’t use their hands much at all. No matter
which camp you fall into, it’s important to pay attention to your hand gestures while you are
communicating or making a presentation. You may be unconsciously communicating in ways
you don't realize. Many gestures are universal across cultures and geographies.

1.3 Motivation For Project

Our motivation to work on this project came from a disabled person who was driving his
wheel chair by hand with quite a lot of difficulty. So we wanted to make a device which
would help such people drive their chairs without even having the need to touch the wheels of
their chairs. Andour objective is to make this device simple as well as cheap so that it could
be mass produced and can be used for a number of purposes.
1.4 Gesture Controlled Robot

The paper describes robustness of Sensors based Gesture Controlled Robot which is a kind of
robot that can be controlled by our hand gestures rather than an ordinary old switches or
keypad. In future there is a chance of making robots that can interact with humans in a natural
manner. Hence my target interest is with hand motion based gesture interfaces. An innovative
formula for gesture recognition is developed for identifying the distinct action signs made
through hand movements. An Acceleration Sensor was used to carry out this and also a Flex
sensor for convinced operation. For communication between hand and radio module has
been used which has a significant transmission range, consumes low power and has multiple
advantages over conventional RF or Bluetooth transmission. In order to full-fill our
requirement a software program has been written and executed using a microcontroller on the
robot. Gestures control robots can extensively be employed in human non-verbal
communication. They allow to express orders, or to transmit some basic cardinal information.
In addition, in some special situations they can be the only way of communicating, as in the
cases of deaf people (sign language) and traffic coordination in the absence of traffic lights, a
real-time continuous gesture recognition system for sign language Face and Gesture
recognition. Upon noticing the results of experimentation proves that our gesture formula is
very competent and it also enhances the natural way of intelligence and is also assembled in a
cheap simple hardware circuit.
1.5 Applications

The applications of the accelerometer based gesture controlled robot include

 These robots are used in military applications to operate robots
 These robots are used in medical applications for the purpose of surgery
 These robotics are used in the construction field
 These robotics are used in industries to control trolly and lift.
 CHAPTER 2
LITERATURE REVIEW
LITERATURE REVIEW

Our hand gesture controlled robot works on the principle of accelerometer which records
hand movements and sends that data to Arduino Uno that records the movements. That
information is then transferred to a encoder which makes it ready for RF transmission. On
the receiving end, the information is received wirelessly via RF, decoded and then passed
onto the microcontroller which takes various decisions based on the received information.
These decisions are passed to the motor driver which triggers the motors in different
configurations to make the robot move in a specific direction. The figure 2.1 shows to help
understanding of the whole working of the robot: Accelerometer Arduino Uno NRF
transceiver (TX/RX) Motor Driver and DC Motors.

Figure 2.1: System Block Diagram

2.1 HAND GESTURE CONTROLLED ROBOT

Gesture Controlled Robot is divided into two sections

2.2 Transmitting Section.

In transmitter part an accelerometer and a RF transmitter unit is used. As we have already
discussed that accelerometer gives an analog output so here we need to convert this analog
data in to digital. For this purpose we have used 4 channel comparator circuit in place of any
ADC. By setting reference voltage we gets a digital signal and then apply this signal to
HT12E encoder to encode data or converting it into serial form and then send this data by
using RF transmitter into the environment.

The second is the receiving end which comprises of following main components:

2.3 Receiving Section.

At the receiver end we have used RF receiver to receive data and then applied to HT12D
decoder. This decoder IC converts received serial data to parallel and then read by using
Arduino . According to received data we drive robot by using two DC motor in forward,
reverse, left, right and stop direction.

2.4 Accelerometer
The most important component is accelerometer. Accelerometer is a 3 axis acceleration
measurement device with +-3g range. This device is made by using polysilicon surface
sensor and signal conditioning circuit to measure acceleration. The output of this device is
Analog in nature and proportional to the acceleration. This device measures the static
acceleration of gravity when we tilt it. And gives an result in form of motion or vibration.

Figure 2.2:Accelerometer
2.5 Arduino UNO

The Arduino Uno ("Uno" means one in Italian) is a microcontroller board based on the
ATmega328 (datasheet). It has 14 digital input/output pins (of which 6 can be used as PWM
outputs), 6 analog inputs, a 16 MHz ceramic resonator, a USB connection, a power jack, an
ICSP header, and a reset button. It contains everything needed to support the microcontroller;
simply connect it to a computer with a USB cable or power it with a AC-to-DC adapter or
battery to get started.
Arduino is an open source platform hardware and software, project, and user community that
designs and manufactures single-board microcontrollers and microcontroller kits for building
digital devices and interactive objects that can sense and control objects in the physical
world. The project's products are distributed as open-source hardware and software, which
are licensed under the GNU Lesser General Public License (LGPL) or the GNU General
Public License (GPL), permitting the manufacture of Arduino boards and software
distribution by anyone. Arduino boards are available like figure 2.3 commercially in
preassembled form, or as do-it-yourselfkits.

Arduino board designs use a variety of microprocessors and controllers. The boards are
equipped with sets of digital and analog input/output (I/O) pins that may be interfaced to
various expansion boards (shields) and other circuits. The boards feature serial
communications interfaces, including Universal Serial Bus (USB) on some models, which are
also used for loading programs from personal computers. The microcontrollers are typically
programmed using a dialect of features from the programming languages C,and C++. In
addition to using traditional compiler toolchains, the Arduino project provides an integrated
development environment(IDE) based on the Processing language project.
Figure 2.3: Arduino UNO

Table 2.2Arduino UNO Specifications

ATmega328
Microcontroller
5V
Operating Voltage

Input Voltage (recommended) 7-12V

Input Voltage (limits) 6-20V

Digital I/O Pins 14 (of which 6 provide PWM output)

6
Analog Input Pins
40 mA
DC Current per I/O Pin
 50 mA
DC Current for 3.3V Pin

Flash Memory 32 KB (ATmega328) of which 0.5 KB used

by bootloader

SRAM 2 KB (ATmega328)

EEPROM 1 KB (ATmega328)

Clock Speed 16 MHz

2.6Communication.

The Arduino Uno has a number of facilities for communicating with a computer, another
Arduino, or other microcontrollers. The ATmega328 provides UART TTL (5V) serial
communication, which is available on digital pins 0 (RX) and 1 (TX). The ATmega328 also
supports I2C (TWI) and SPI communication.

2.7 RF PAIR

 The RF module, as the name suggests, operates at Radio Frequency. The

corresponding frequency range varies between 30 kHz & 300 GHz. In this RF system,
the digital data is represented as variations in the amplitude of carrier wave. This kind
of modulation is known as Amplitude Shift Keying (ASK).

 Transmission through RF is better than IR (infrared) because of many reasons. Firstly,
signals through RF can travel through larger distances making it suitable for long
range applications. Also, while IR mostly operates in line-of-sight mode, RF signals
can travel even when there is an obstruction between transmitter & receiver. Next, RF
transmission is more strong and reliable than IR transmission. RF communication
uses a specific frequency unlike IR signals which are affected by other IR emitting
sources.

  This RF module comprises of an RF Transmitter and an RF Receiver. The
transmitter/receiver (Tx/Rx) pair operates at a frequency of 434 MHz. An RF
transmitter receives serial data and transmits it wirelessly through RF through its
antenna connected at pin4. The transmission occurs at the rate of 1Kbps - 10Kbps.The
transmitted data is received by an RF receiver operating at the same frequency as that
of the transmitter.

Figure 2.4: RF PAIR

2.8 MOTOR DRIVER (L293D)

L293D is a typical Motor driver or Motor Driver IC which allows DC motor to drive on
either direction. L293D is a 16-pin IC which can control a set of two DC motors
simultaneously in any direction. It means that you can control two DC motor with a single
L293D IC. It works on the concept of H-bridge. H-bridge is a circuit which allows the
voltage to be flown in either direction. As you know voltage need to change its direction for
being able to rotate the motor in clockwise or anticlockwise direction, Hence H-bridge IC are
ideal for driving a DC motor.

In a single L293D chip there are two h-Bridge circuit inside the IC which can rotate two dc
motor independently. Due its size it is very much used in robotic application for controlling
DC motors. Given below is the pin diagram of a L293D motor controller.
There are two Enable pins on l293d. Pin 1 and pin 9, for being able to drive the motor, the pin
1 and 9 need to be high. For driving the motor with left H-bridge you need to enable pin 1 to
high. And for right H-Bridge you need to make the pin 9 to high. If anyone of the either pin1
or pin9 goes low then the motor in the corresponding section will suspend working. It’s like a
switch.

Figure 2.5: MOTOR DRIVER

IC Turning in figure 2.6 shows that a motor ON and OFF requires only one switch to control
a single motor in a single direction. We can reverse the direction of the motor by simply
reversing its polarity. This can be achieved by using four switches that are arranged in an
intelligent manner such that the circuit not only drives the motor, but also controls its
direction. Out of many, one of the most common and clever design is a H-bridge circuit
where transistors are arranged in a shape that resembles the English alphabet "H”
 Figure 2.6: H-Bridge Block
diagram
H-Bridge As seen in the image, the circuit has four switches A, B, C and D. Turning these
switches ON and OFF can drive a motor in different ways.
• When switches A and D are on, motor rotates clockwise.
• When B and C are on, the motor rotates anti-clockwise.
• When A and B are on, the motor will stop.
• Turning off all the switches gives the motor a free wheel drive.
• Turning on A & C at the same time or B & D at the same time shorts the entire circuit.
So, never try to do it

2.9 DC Motor

A machine that converts DC power into mechanical power is known as a DC motor. Its
operation is based on the principle that when a current carrying conductor is placed in a
magnetic field, the conductor experiences a mechanical force. DC motors have a revolving
armature winding but non-revolving armature magnetic field and a stationary field winding or
permanent magnet. Different connections of the field and armature winding provide different
speed/torque regulation features. The speed of a DC motor can be controlled by changing the
voltage applied to the armature or by changing the field current.
Figure 2.7: DC Motor