REPORT ON

SIXTH SENSE TECHNOLOGY

Submitted By,
Name- AMAN GOYAL
Enroll No 510/11
Roll No 69
Branch E&C
 SIXTHSENSE TECHNOLOGY

Abstract
Every one of us are aware of the five basic senses seeing, feeling,
smelling, tasting and hearing. These senses have evolved through millions of
years.

Whenever we encounter a new object/experience our natural senses tries to

analysis that experience and the information that is obtained is used to modify our
interaction with the environment. But in this new age of technologies the most
important information that helps one to make right decision is something that
cannot be perceived and analysed by our natural senses. That information is the
data in the digital form, and it is available to everyone through sources like
internet.

The sixth sense technology concept is an effort to connect this data in the
digital world in to the real world.

-1-
 SIXTHSENSE TECHNOLOGY

INDEX
Abstract 1

1. Introduction
2. History 3
3. Sixth Sense Devices 4
4. Working 9
5. Technology Used in Sixth Sense 10
6. Advantages 14
7. Application 16
8. Conclusion 20
9. References 21
10. Certificate 22

-2-
 SIXTHSENSE TECHNOLOGY

1. Introduction
Although miniaturized versions of computers help us to connect to the
digital world even while we are travelling there arent any device as of now which
gives a direct link between the digital world and our physical interaction with the
real world. Usually the informations are stored traditionally on a paper or a digital
storage device.

Sixth sense technology helps to bridge this gap between tangible and non-
tangible world. Sixth Sense device is basically a wearable gestural interface that
connects the physical world around us with digital information and lets us use
natural hand gestures to interact with this information .The sixth sense technology
was developed by Pranav Mistry, a PhD student in the Fluid Interfaces Group at
the MIT Media Lab.

The sixth sense technology has a Web 4.0 view of human and machine
interactions. Sixth Sense integrates digital information into the physical world and
its objects, making the entire world your computer. It can turn any surface into a
touch-screen for computing, controlled by simple hand gestures. It is not a
technology which is aimed at changing human habits but causing computers and
other machines to adapt to human needs. It also supports multi user and multi
touch provisions.

-3-
 SIXTHSENSE TECHNOLOGY

1. The History

The history of "6th sense technology" goes back into 1990's when Steve Mann first
attempted to propose a neck worn projector and a camera combination.

The idea of implementing computer technologies to daily tasks as our "6th sense"
was further developed by Pranav Mistry who also appears to be an MIT student as
well as Steve Mann. The first prototype of the 6th sense technology was actually
bigger than what it looks like today and it was not working properly to use in daily
life. In an article called The sixth sense technology Arjun K. R says that "They
started with a larger projector that was mounted on a helmet. But that proved
cumbersome if someone was projecting data onto a wall then turned to speak to
friend- the data would project on the friend's face" (Arjun K. R, 2011). Therefore,
Mistry came up with a different and more convenient device which is a neck worn
portable camera that allows users more space for their daily actions.

On the other hand, Mistry first tried out his idea on a simple computer mouse. First,
he put two rollers into one mouse and see if he could obtain data and guide the
movements of the mouse. Two rollers did not work properly so he decided to use four
rollers and see if it could work better. Four rollers gave him the idea that he could use
the same idea on fingers and thats what he next moved on to. The picture below
shows how four rolls were tried out in Mistry's experiment.

The 6th sense technology was a bit pricey back in 1990's because the specifications
that the device needed were barely entering the market and it was really costly to
convert them into commercial use. But today, with the fast pace of technological
innovations, the cost of this technology is getting ready to enter the commercial
market which will allow people to actually use it in their daily life in a few years.

-4-
 SIXTHSENSE TECHNOLOGY

2. Sixth Sense Devices

'SixthSense' is a wearable gestural interface that augments the physical
world around us with digital information and lets us use natural hand gestures to
interact with that information. The hardware components are coupled in a pendant
like mobile wearable device.

The SixthSense prototype is comprised of a pocket projector, a mirror,

colored marker and a camera. The camera, mirror and projector is connected
wirelessly to a blue tooth smart phone device that can easily fit into the users
pocket. A software then processes the data that is collected by the capturing device
and produce analysis. The software that is used in sixth sense device is open source
type.

-5-
 SIXTHSENSE TECHNOLOGY

Camera

It captures the image of the object in view and track the users hand gesture.
The camera recognizes individuals, images, pictures, gestures that user makes with
his hand. The camera then sends this data to a smart phone for processing.
Basically the camera forms a digital eye which connects to the world of digital
information.

Mobile Components

The SixthSense device consists of a web enabled smart phone which

process the data send by the camera. The smart phone searches the web and
interprets the hand gestures with help of the colored markers placed at the finger
tips.

-6-
 SIXTHSENSE TECHNOLOGY

Projector

The information that is interpreted through the smart phone can be projected
into any surface. The projector projects the visual information enabling surfaces
and physical objects to be used as interfaces. The projector itself consists of a
battery which have 3 hours of battery life .A tiny LED projector displays the data
sent from the smart phone on any surface in view- object, wall or person. The
downward facing projector projects the image on to a mirror.

-7-
 SIXTHSENSE TECHNOLOGY

Mirror
The usage of a mirror is important as the projector dangles pointing
downwards from the neck. The mirror reflects the image on to a desire surface.
Thus finally the digital image is freed from its confines and placed in the physical
world.

Coloured Marker

There are colour markers placed at the tip of users finger. Marking the users
fingers with red, yellow green and blue coloured tape helps the webcam to
recognize the hand gestures. The movements and arrangement of these markers are
interpreted into gestures that act as a interaction instruction for the projected
application interfaces.

-8-
 SIXTHSENSE TECHNOLOGY

4.Working

-9-
 SIXTHSENSE TECHNOLOGY

The hardware that makes Sixth Sense work is a pendant like mobile wearable interface
It has a camera, a mirror and a projector and is connected wirelessly to a Bluetooth smart
phone that can slip comfortably into ones pocket
The camera recognizes individuals, images, pictures, gestures one makes with their hands
Information is sent to the Smartphone for processing
The downward-facing projector projects the output image on to the mirror
Mirror reflects image on to the desired surface
Thus, digital information is freed from its confines and placed in the physical world

3. Technology Used in Sixth Sense

- 10 -
 SIXTHSENSE TECHNOLOGY

Augmented Reality

The augmented Reality is a visualization technology that allows the user to experience
the virtual experience added over real world in real time. With the help of advanced AR
technology the information about the surrounding real world of the user becomes interactive and
digitally usable. Artificial information about the environment and the objects in it can be stored
and retrieved as an information layer on top of the real world view.

When we compare the spectrum between virtual reality, which creates immersive,
computer-generated environments, and the real world, augmented reality is closer to the real
world. Augmented reality adds graphics, sounds, haptic feedback and smell to the natural world
as it exists. Both video games and cell phones are driving the development of augmented reality.

The augmented systems will also superimpose graphics for every perspective available
and try adjust to every movement of the user's head and eyes .The three basic components of an
augmented reality system are the head-mounted display, tracking system and mobile computer
for the hardware. The main goal of this new technology is to merge these three components into
a highly portable unit much like a combination of a high tech Walkman and an ordinary pair or
eyeglasses.

The head-mounted display used in augmented reality systems will enable the user to view
superimposed graphics and text created by the system .Another component of an augmented
reality system is its tracking and orientation system. This system pinpoints the user's location in
reference to his surroundings and additionally tracks the user's eye and head movements.
Augmented reality systems will need highly mobile computers. As of now many computers
arent there to satisfy to provide this option.

- 11 -
 SIXTHSENSE TECHNOLOGY

Gesture Recognition

It is a technology which is aimed at interpreting human gestures with the help of

mathematical algorithms. Gesture recognition technique basically focuses on the emotion
recognition from the face and hand gesture recognition.

Gender recognition technique enables humans to interact with computers in a more direct
way without using any external interfacing devices. It can provide a much better alternative to
text user interfaces and graphical user interface which requires the need of a keyboard or mouse
to interact with the computer. Interfaces which solely depends on the gestures requires precise
hand pose tracking. In the early versions of gesture recognition process special type of hand
gloves which provide information about hand position orientation and flux of the fingers.

In the Sixth Sense devices coloured bands are used for this purpose. Once hand pose has
been captured the gestures can be recognised using different techniques. Neural network
approaches or statistical templates are the commonly used techniques used for the recognition
purposes. This technique have an high accuracy usually showing accuracy of more than 95%.
Time dependent neural network will also be used for real time recognition of the gestures.

Computer Vision

Computer vision is the technology in which machines are able to interpret/extract

necessary information from an image. Computer vision technology includes various fields like
image processing, image analysis and machine vision. It includes certain aspect of artificial
intelligence techniques like pattern recognition. The machines which implement computer vision

- 12 -
 SIXTHSENSE TECHNOLOGY

techniques require image sensors which detect electromagnetic radiation which are usually in the
form of ultraviolet rays or light rays.

The computer vision find itself applicable in varies field of interest. One such field is bio
medical image processing. Its also used in autonomous vehicle like SUVs. The computer
vision technique basically includes four processes.

1. Recognition: One of the main task of computer vision technique is to determine whether
the particular object contain the useful data or not.
2. Motion Analysis: Motion analysis includes several tasks related to estimation of motion
where an image sequence is processed continuously to detect the velocity at each point of
the image or in the 3D scene.
3. Scene Reconstruction: Computer vision technique employs several methods to recreate a
3D image from the available images of a scene.
4. Image Restoration: The main of aim of this step is to remove noise from an given image.
The simplest method involves using simple filters like low pass or median filters. In order
to get better quality images more complex methods like in painting are used.

Radio Frequency Identification

Radio frequency identification systems transmit the identity of an object wirelessly, using
radio magnetic waves. The main purpose of a radio frequency identification system is to enable
the transfer of a data via a portable device.

The portable device is commonly known as tag. The data send by the tag is received and
processed by a reader according to the needs of the application. The data send by the tag contains
various informations identification or location of the information, or specifics about the product
that has been tagged, for example price, color , date of purchase, etc. This technology

gained importance because of its ability to track moving object.

- 13 -
 SIXTHSENSE TECHNOLOGY

A typical radio frequency tag consists of a microchip attached to a radio antenna which is
mounted on a substrate. To retrieve the data from the tag a reader is needed. A typical radio
frequency reader consists of two antennas that emit radio waves and at the same are capable of
accepting the signals from the tag. The reader then passes the information that it has received to a
computer device in digital form.

The computer then interprets this digital data and processes it. Radio frequency
identification techniques are widely used in the fields like asset tracking, supply chain
management, manufacturing, payment systems etc.

One of the major advantages of devices using radio frequency technology over other
similar devices is that RFID devices need not be positioned precisely relative to the scanner. But
till then there are certain areas of concern for this technology. Some problem related to this
technology is tag collision and reader collision. Usually the reader collision occurs when the
signals from two or more readers overlap, while tag collision occurs when many tags are present
in a small area.

- 14 -
 SIXTHSENSE TECHNOLOGY

5. Advantages

Portable
One of the main advantages of the sixth sense devices is its small size and portability. It
can be easily carried around without any difficulty. The prototype of the sixth sense is designed
in a such a way that it gives more importance to the portability factor. All the devices are light in
weight and the smart phone can easily fit into the users pocket.

Support multi touch and multi user interaction

Multi touch and multi user interaction is another added feature of the sixth sense devices.
Multi sensing technique allows the user to interact with system with more than one than one
finger at a time.. Sixth sense devices also in-cooperates multi user functionality. This is typically
useful for large interaction scenarios such as interactive table tops and walls.

Cost effective

The cost incurred for the construction of the sixth sense proto type is quiet low. It was
made from parts collected together from common devices. And a typical sixth sense device cost
up to $300. The sixth sense devices have not been made in large scale for commercial purpose.
Once that happens its almost certain that the device will cost much lower than the current price.

Connectedness between real world and digital world

Forming a connection between the real world and the digital world was the main aim of
the sixth sense technology.

- 15 -
 SIXTHSENSE TECHNOLOGY

Data access directly from the machines in real time

With help of a sixth sense device the user can easily access data from any machine at real
time speed. The user doesnt require any machine-human interface to access the data. The data
access through recognition of hand gestures is much easier and user friendlier compared to the
text user interface or graphical user interface which requires keyboard or mouse.

Mind map the idea anywhere

With the advent of the sixth sense device, requirement of a platform or a screen to
analyses and interpret the data has become obsolete. We can project the information into any any
surface and can work and manage the data as per our convenience.

Open source software

The software that is used to interpret and analysis the data collected by the device is made
open source. This enables other developers to contribute to the development of the system

- 16 -
 SIXTHSENSE TECHNOLOGY

6. Application
The sixth sense technology finds a lot of application in the modern world. The sixth sense
devices bridge the gap by bringing the digital world into the real world and in that process
allowing the users to interact with the information without the help of any machine interfaces.
According to the words of its developers the extend of use of this new device is only limited by
the imagination of human beings. Some practical applications of the sixth sense technology is
given below:

Taking Pictures

Another application of sixth sense devices is the implementation of a gestural camera.

This camera takes the photo of the location user is looking at by detecting the framing gesture.
After taking the desired number of photos we can project them onto any surfaces and then use
gestures to sort through those photos and organize and resize them.

The user taking making a pose gesture The camera takes the photo of the location

Map Application

- 17 -
 SIXTHSENSE TECHNOLOGY

With the help of a map application the user can call upon any map of his/her choice and
navigate through them by projecting the map on to any surface. By using the thumb and index
fingers movements the user can zoom in, zoom out or pan the selected map

Making Calls

We can make calls with help of sixth sense device .The sixth sense device is used to
project the keypad into your palm and the using that virtual keypad we can make calls to anyone.

Drawing Application

- 18 -
 SIXTHSENSE TECHNOLOGY

The drawing application allows the user you to draw on any surface by tracking the
fingertip movements of the users index finger. The pictures that are drawn by the user can be
stored and replaced on any other surface. The user can also shuffle through various pictures and
drawing by using the hand gesture movements

Zooming features

The user can zoom in or zoom out using intuitive hand movements

- 19 -
 SIXTHSENSE TECHNOLOGY

Interacting with physical objects

The Sixth Sense system also helps to interact with physical objects we use in a better
way. It augments physical objects by projecting more information about these objects projected
on them.

For example, a gesture of drawing a circle on the users wrist projects an analog watch on
the users hand. Similarly a newspaper can show live video news or dynamic information can be
provided on a regular piece of paper.

Newspaper showing live feeds

an analog watch projected into the wrist

- 20 -
 SIXTHSENSE TECHNOLOGY

7. Conclusion

Sixth Sense technology is the science of tomorrow with the aim of connecting the
digital world with the physical world seamlessly, eliminating hardware devices. This
combination of devices and software together create a reality in which the digital
world is merged with the physical world. This technology can be used in a plethora
of positive ways, especially by enhancing daily tasks. It facilitates easily learn
information about products while shopping, find instructions while building or
constructing and being able to easily identify areas while travelling are all extremely
positive results of Sixth Sense technology. It is essentially a wearable computer that
can surf the web, make phone calls and even connect to other computing devices. It
is more portable and more interactive than any Smartphone, laptop or tablet available
today. It does, however, pose a threat to society in the form of privacy issues. It
allows for the user to easily record and photograph anything without being notice
and it can also allow for the user to gain available online information about anything
just by glancing at it and performing an online search. This will surely bring up many
social, legal, ethical and security concerns. Sixth Sense technology is remarkable in
its intended purpose for daily task efficiency and interactive computing. Even though
the sixth sense devices are currently in development stages and have not been used
in large now, its quiet predictable that this technology have the potential to
revolutionise the way in which we interacts with the digital world.

- 21 -
 SIXTHSENSE TECHNOLOGY

8. References

http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthsense_technolo
gy.htm
http://www.commoncraft.com/augmented-reality-video
http://www.arcane-technologies.com/en/
http://en.wikipedia.org/wiki/Augmented_reality
http://www.rfidjournal.com/
http://www.ted.com/talks/pattie_maes_demos_the_sixth_sense.html
http://timesofindia.indiatimes.com/Health__Science/Sixth_sense_gizmo_a_sci-
fi_reality/articleshow/4083978.cms
http://www.gesturetek.com/

- 22 -