HAND GESTURE RECOGNITION USING MACHINE LEARNING

Statement about the Problem:

Worldwide, at least 360 million people are likely to be deaf and dumb (or simply
deaf). Most of these people do not have access to public services. In the case of
hearing impairment, mutism has created the need of using sign language. A sign
language consists of a set of manual gestures that use fingers, palms, arms, and
even body movements to represent letters, numbers, and words. This type of
language allows people who cannot speak or hear to communicate with others.

Hand gesture recognition for human computer interaction is an area of active

research in computer vision and machine learning. The primary goal of gesture
recognition research is to create a system, which can identify specific human
gestures and use them to convey information or for device control.

One of the primary goals of gesture recognition research is to create systems,

which can identify specific gestures and use them to convey information or to
control a device. Being hand-pose one of the most important communication
tools in humans daily life, and with the continuous advances of image and video
processing techniques, research on human-machine interaction through gesture
recognition led to the use of such technology in a very broad range of possible
applications.
Reason for choosing the particular topic:
The reason for choosing this particular topic is to start and expand my knowledge
in the field of Machine Learning. Machine learningand AI in generalhas been
around for a while. But its recently started accelerating at a rate thats surprised a
lot of people. As recently as 2014, most experts thought it would be 10 years
before a machine beat the worlds best players at Go. DeepMind proved them
wrong. Its becoming increasingly apparent that many tasks we once thought
would be the domain of humans alone for the foreseeable futureif not
foreverwill be accomplished by machine learning systems much sooner than
expected.
So, choice of this field for our minor/major project was a decision based on how
our interest start developing in interesting projects that had been accomplished
already and how the direction of IT industry is changing in the current time.

Objective and Scope of the project:

Objective of the project is to make a system where if a person communicates
using a sign language (American Standard), the translation of that hand gesture to
a text will be displayed on the screen. This will help bridging the communication
gap between people with hearing/speaking disability to make communication
even easier to conduct when hand gestures are used so that communication with
a person is possible who doesnt know how to interpret the hand gestures.

Scope of this project lies in where it is used. Organizations where people with
speaking disabilities go for various purposes like getting their government
identities, bank loans etc. can use this system as an external translator where
they can use hand gestures to communicate directly with the officials which
reduces their communication gap that wouldve existed earlier due to the officials
in the organization not knowing anything about how to interpret hand gestures.

Similarly, it can also be embedded in a mobile application to interpret gestures on

a fly.
Methodology:

1. Training Set/Database values collection Using the integrated or external

webcam to take pictures of various hand gestures and training the
algorithm/adding it to database for processing of result after a hand
gesture is placed in front of camera for interpretation. For this part, we use
a data set comprising raw images and a corresponding csv file.

2. Classification Algorithm Implementation - Train a binary classifier to

detect hand/not-hand images using the above data set.

3. Implementation of recognition and decision making models In Machine

Learning, support vector machines (SVMs, also support vector networks)
are supervised learning models with associated learning algorithms that
analyze data used for classification and regression analysis. Therefore,
decision of using a custom learning model or a one that has been previously
made has to be taken on the basis of accuracy of results.

4. Displaying results Assigning a front end to the project that will help
display the results of interpretation of hand gestures to the user is to be
done.

5. Improving accuracy and Testing A Machine Learning model requires

constant improvement and training to improve the accuracy of the result.
So, considerable testing of the system has to be done to improve the result.
Schedule(for minor project):

Hardware/Software Resources:
Python 2.x/3, PyCharm/Jupyter IDE, Ubuntu/Windows operating systems for
establishing working environments, external softwares for testing and training
the model.

Conclusion:
Making a persons life easier in a way that helps them a lot is what an engineer is
all about. Machine Learning is a fast growing field and has applications in many
areas, including our project. Designing our Machine Learning model, training and
improving it will be our goal for the first part of our project. This project can be
deployed at places to make peoples life easier or can be used in a mobile
application as a further improvement to it. End goal of this project is to have a
system working well, displaying accurate results and tested to be ready for
deployment.
Bibliography:
1. https://repositorium.sdum.uminho.pt/bitstream/1822/39162/1/L12)%20
Hand%20Gesture%20Recognition%20System%20based%20in%20Comput
er%20Vision%20and%20Machine%20Learning.pdf
2. https://medium.freecodecamp.org/weekend-projects-sign-language-and-
static-gesture-recognition-using-scikit-learn-60813d600e79
3. http://cs229.stanford.edu/proj2011/ChenSenguptaSundaram-
SignLanguageGestureRecognitionWithUnsupervisedFeatureLearning.pdf
4. http://answers.opencv.org/question/98410/hand-posture-recognition-
using-machine-learning/
5. https://link.springer.com/chapter/10.1007/978-981-10-4086-3_59
6. www.google.com
7. https://en.wikipedia.org/wiki/Gesture_recognition
8. www.stackoverflow.com