Eye Movement Based Human Computer Interaction
Gautham Dev k. Ms Ashmy Antony.
Department of Computer Science and Engineering
Sreepathy Institute of Management & Technology,
Vavanoor, Kerala, India
gauthamdevadas@gmail.com
Abstract—With the advancement in technology the number of
computing devices that a person uses is increasing and there is a
need of faster and non-intrusive methods of communicating with
these devices. In this regard, eye movement is a promising input
medium for human computer interaction. In this paper discussed
various eye tracking techniques that can be used to find the line of
gaze of the user.Then discussed some of the algorithms that can
be used to implement these eye tracking techniques.The main
aim of this paper is to propose new applications utilizing eye
gaze that are suitable for standard user.These applications will
enhance the reading experience of the user and make reading a
more joyous task.
Keywords-Eye movements; Eye tracking; Dwell time;
Eye gaze; Fixation
I. INTRODUCTION
In the present times,interaction with computer is not only
with keyboards and printers anymore.Many more advanced
input technologies have been proposed and implemented in-
order to interact with computer more easily and efficient.For
e.g touch screen,speech recogonition system etc.Eye gaze is
also a recent input technique which has the potential to be
widely used in future. Using eye movement for controlling
the computer improves the experience of working with the
computer as it is faster and gives the illusion that the computer
is complying with the users thought. It can be used either
exclusively or in combination with other input technologies
such as eye movement can be used along with a button so
that it confirms the users intentions for performing critical
tasks and reduce the chances of error. It does not require any
training and can thus be used by a layman. It can act as a boon
for a person with motor disability as it does not require any
motion but simple eye movements. It can give them a greater
controlled over their surrounding and help them in interacting
with the world.
In previous years a lot of work has been done on this
technology with the aim of making it available for general
public. Firstly eye trackers were developed based on three
kinds of technologies to track the movement of the eye. These
three technologies are: eye attached tracking-where a device
like a contact lens is attached to the eye, optical tracking-which
makes use of light for measuring eye motion, electric potential
measurement-where the dipole moment of the eye is calculated
for determining the eyemotion. Next, some algorithms were
developed to get the x,y coordinates of the portion of screen
where the user is looking from the data obtained by the tracker.
Ass.proffessor
Department of Computer Science and Engineering
Sreepathy Institute of Management & Technology,
Vavanoor, Kerala, India
AshmyAntony@simat.ac.in
This data was then utilized to perform many generic tasks in
human computer dialogue such as object selection, moving an
object, scrolling text, etc.
In this paper i will first give the background necessary
to understand this topic. Here i will describe some of the
techniques that can be used for measuring eye movement,some
previous work done in this field by other people.i will also
give the characteristics of the eye which can help or create a
problem in its use as an input technique.Here, i will also briefly
discuss a problem named as Midas Touch Problem (named by
Robert J.K. Jacob) that is commonly encountered while using
eye movement as an input medium. We have also discussed
some of the algorithms for determining the iris center which is
helpful in finding the x, y coordinate of the screen where the
user is looking.i had proposed and designed some applications
based on eye gaze as an input technique. These applications
are named as Effortless Scrolling-used to achieve movement
of the screen along with the movement of our eye gaze; and
Enriched Reading- can help in improving reading experience
of the user by assisting the user in understanding the text.
These techniques can be implemented in modern day E-books
which would give them a slight advantage over the old paper
books.
II. ABBREVIATIONS AND ACRONYMS USED
• POG- point of gaze.
• EOG- Electro Oculography.
• HCI- Human Computer interaction.
• VOG- Video Oculography.
• IR-PCR- Infrared Pupil corneal reflection.
III. BACKGROUND
A.Methods For Measuring Eye Movements
Eye tracking is the process of measuring either the point of
gaze (where one is looking) or the motion of an eye relative
to the head. It can be carried out by the use of eye trackers.
Many techniques are available for tracking the movement of
the eye; while none is perfect, some are considerably more
suitable for user computer interaction than others. An eye
tracker gives the X and Y coordinates of the point on the
screen at which the user is looking. This data is converted into
a stream of tokens. These tokens represent some meaningful
user computer dialogues and can be used in the same way as
the tokens generated by mouse or keyboard events.
 Some of the commonly heard methods are Electronic
method,Mechanical method,optical/video method single point
and optical video method two point.
B.Previous Work
The use of eye movement in human computer interaction
has been promising for the last 50 years but it is still
not available for use to the standard user. This means that
something has held it up in its merely promising state for
such a long time.The labeling of a technology as merely
promising for such a long time proves that the technology
has a lot of potential; otherwise it would have been discarded
by now.Here i quickly summerise the work that has been
done in this regard.Firstly natural eye movements were studied
and characterized into two types.The main problem that is
encountered in using eye movement as an input technique
was identified by Robert J.K. Jacob, who named it the Midas
Touch Problem.After that eye trackers were developed based
on different kinds of eye tracking techniques to track the eye
movements of the user.Then many tracking algorithms were
designed to track the pupil of the eye or the iris center which
helps in finding the x, y coordinates of the users point of
gaze on screen.Then various experiments were conducted in
the field of eye movement based human computer interaction
so as to explore the use of eye gaze as an input to computer.
Now, many generic tasks in human computer dialogue such
as object selection, moving or copying an object, choosing
menu items, etc. can be done using eye movement alone or
in combination with other input modalities.
C. Characteristics of Eye Movements
In order to view an object clearly it must appear on the
fovea, a small area at the center of the retina. Because of this
the eye provides an accurate indication of the object being
observed. The eye typically moves in the form of saccades
and fixations. A saccade is a sudden and instantaneously
movement of eye. It is typically followed by a fixation of
200-600ms which is a period of relative stability. However,
even during a fixation the eye is not completely stable .
Several small, jittery motions, generally less than one degree
in size takes place during a fixation. There is a sequence of
a slow drift followed by a sudden, tiny saccade-like jump to
correct the effect of the drift (a micro saccade). These jittery
movements have to be eliminated as the user is not aware of
these movements and hence raw eye movement data requires
filtering to be useful.
D. The Midas Touch Problem
The main use of eye is for scanning the area in front of the
user, when the eyes are to be used for giving input as well it
becomes difficult to determine whether a command has to be
executed or not. This problem was identified by Robert J.K.
Jacob (1990) and was termed as the Midas Touch problem.
It refers to a situation in which a command is initiated even
when the user is just looking at the screen and does not
wish to perform any function. Ideally, acommand should be
initiated only when the user wishes to perform a function
and otherwise the user should be able to just look around
without issuing any command, but generally the two cases
are impossible to distinguish. This problem can be avoided
by using eye movement in combination with other input
methods for giving important commands.
E. Eye Tracking Techniques
Eye tracking plays a central role in eye movement based
human-computer interaction. It is the process of tracking the
absolute point of gaze i.e., the point where the users gaze is
focused. Eye tracking techniques can be classified as: Videooc-
culography or Video based tracking, Infrared Pupil Corneal
reflection, Electrooculography [5]. All the above techniques
are widely used in commercial and research fields but each
one of them has their own advantages and disadvantages. The
use of a particular technique depends on the requirements of
the application. We will discuss all the three techniques, their
pros and cons and their suitability for eye movement based
HCI.
Fig. 1. Corneal reflection plus pupil eye tracker
E.1 Videooculography
Videooculography (Fig.2.) tracks the movement of the
eye using cameras that can be either remote or mounted
on the head. The data received from these cameras is then
analyzed by a computer. For good quality of eye tracking the
camera is zoomed-in on the eye. This reduces the working
angle of the camera and hence demands that the user sits
still. It also requires an unhindered view of the eye. So,
the accuracy of tracking gets easily affected by lighting
conditions, glasses, heavy makeup, or squinting of the eye. It
provides an accuracy of 4degrees, which is not satisfactory.
Its main advantage is that it is comfortable and non-intrusive.
E.2 Infra-red Pupil-Corneal Reflection
IR-PCR (Fig.3) makes use of the corneal reflection of IR
waves as a reference point along with the pupil to calculate
Fig. 2. First-generation video-based eye trackers: Mobile Eye (photo courtesy
of Applied Science Laboratories)
the point of gaze. This type of tracking provides a fairly good
accuracy of 0.5 degree and also allows considerable head
movements. Its disadvantage is that it cannot be used under
changing light conditions such as ambient light.
E.3 Electrooculography
This technique (Fig.4) measures the electric potential from
the cornea-retina dipole. It uses electrodes placed near the
eye for measuring changes in potential. It can continue eye
tracking even during blinks or when the user squints. The
disadvantage of EOG is that it provides a lower accuracy and
is invasive.
From the above described techniques, the best suited for
human computer interaction is the infra-red pupil-corneal
reflection technique as it gives a good accuracy, allows
considerable head movements and is non-invasive.
F. Eye Tracking Algorithms
After the tracker tracks the eye, the eye tracking algorithms
are needed to find the iris center which gives the point of
gaze. Here we briefly mention two algorithms for finding
the iris center, one is the Longest Line Scanning Algorithm
given by Kyung-Nam Kim and R. S. Ramakrishna [6] and the
other algorithm is known as the Starburst Algorithm given by
Dongheng Li, David Winfield and Derrick J. Parkhurst.
F.1 Longest Line Scanning
LLS algorithm (Fig.5) uses limbus detection to track the
Fig. 3. IR is used to illuminate the eye and its reflection on the cornea provides
an additional reference point that improves accuracy (2008, www.cogain.org.)
center of the iris. The iris is nearly circular and its projection
is elliptical in shape. To track the iris center this algorithm uses
the property that the center of an ellipse lies on the center of
the longest horizontal line inside the boundary of the ellipse.
F.2 Starburst Algorithm
Starburst algorithm (Fig.6) uses pupil rather than limbus to
track the iris center. Tracking the contour of the pupil is more
convenient than tracking the limbus as the pupil contour is
smaller and is defined more clearly.
IV. SOME PROPOSED APPLICATIONS
In this section we present some applications that use eye
movement as an input to interact with computers.
A. Effortless Scrolling
Eye controlled scrolling of text can be achieved by
detecting vertical and horizontal gaze gestures of the eye.
The gaze gestures relevant to our application are shown
in (Fig.7). Our aim is to achieve movement of the screen
corresponding to the eye movement as long as his point of
gaze is within the margins. That means the amount of text
that will be scrolled in the desired direction is variable and
relative to the eye movement. Secondly if the user gazes at
some point that is beyond the margins, it indicates that he
has read all the text that was displayed within that frame, and
now a large and fixed portion of the screen should be moved

Fig. 4. Wearable EOG goggles (Bulling et al.The Swiss Federal Institute of
Technology (ETH) Zurich developed these goggles.
Fig. 5. LLS Algorithm
in the concerned direction. Also, this should not happen with
random gaze gestures of the user. Hence, it becomes necessary
to determine a threshold value of time. The scrolling will
take place only if the dwell time exceeds this threshold value.
A.1 Algorithm
A gesture typically consists of strokes and the main chal-
lenge in developing this application is to recognize gaze
gestures from natural eye movements. The following algorithm
was devised for the same. We have designed this algorithm to
recognize only horizontal and vertical gaze gestures, diagonal
gestures will be ignored. So what happens when the user looks
in the diagonal direction? In that case, we calculate separately
the horizontal and vertical distances between the old and new
points of gaze. The direction of scrolling is then determined by
Fig. 6. Starburst Algorithm
Fig. 7. Relevent Gaze Gestures
comparing the two values (either vertical or horizontal only).
I first present the algorithm (Fig.8),and then i have provided
its explanation in the next subsection.
A.2 Explanation of Proposed Algorithm
This algorithm uses two kinds of eye movements to enable
scrolling:
First is a fixation at any point beyond the margin for a
few msec. (known as dwell time). This dwell time should
be greater than a threshold time which is to be calculated
experimentally. This type of eye movement causes a large
portion of the text to move in the desired direction.
Second is a series of fixations in a particular direction. As
long as the user keeps looking in the same direction the text
will keep on scrolling along with the eye movement. The
scrolling will cease if the user suddenly changes direction.
The value of t in the algorithm ensures the implementation of
this constraint.
The conditions in the algorithm are based on the fact that
the origin lies at the top-left corner of the computer screen.
The X-coordinate increases in the right direction and the Y-
coordinate increases in the downward direction.
The red part of the algorithm implements the first kind of
scrolling. We check whether the point of gaze lies beyond the
margin for a dwell time greater than threshold time. If the
 Fig. 9. Word highlighted

Fig. 10. Pop-up box showing the meaning

above two conditions are satisfied scrolling takes place. This

is done for all four directions- upward, downward, left and
right.
The blue part implements the second kind of scrolling. As
has been explained, we calculate separately the horizontal and
vertical distances between the old and new points of gaze and
compare the two values to determine whether the scrolling
has to be done in vertical or horizontal direction. Suppose we
found that the text has to be scrolled in the vertical direction.
Next we would determine whether the direction is upwards
or downwards and store and respectively. Same thing would
be done for horizontal direction. Then we have to ensure
that this is not just a random eye movement, but the users
gaze has been aligned in this direction consecutively. For this
we compare this stored character with the old character and
assign a value to t (0 or 1). Then if the above condition is
satisfied (t==1), scrolling takes place in the desired direction,
indicated by the stored character.

B. Enriched Reading

Reading experience of the user can be enhanced using eye

movements. We can detect the word the user is currently
reading, if he fixes his gaze on a word for a sufficiently
long time. If this time is longer than required, the computer
recognizes that he is struggling with the word and provides
him with its meaning. This technique if applied in modern
day e-books will raise their standard and give them a slight
advantage over the old paper books.

Steps involved:
• If dwell time on a word is greater than threshold value,
the word will be highlighted.(Fig.9)

• Its meaning will be searched in the dictionary and dis-

played to the user in a pop-up box.(Fig.10)
Fig. 8. Proposed Algorithm
 • The pop-up box can be closed by simply looking at the
close icon for a longer dwell time.
Although we have not devised an algorithm for this ap-
plication, it can be implemented using the modified version
of the reading detection algorithm given in the thesis of
Heiko Drewes titled Eye Gaze Tracking for Human Computer
Interaction.
V. FUTURE WORK
Many new innovative applications can be developed based
on eye movement based human computer interaction. First
of all, we wish to see our proposed applications enriched
reading and effortless scrolling being used in daily life. Both
of these applications can be applied in modern day e-books
can give them a slight advantage over old style paper books
as we would be able to readily and easily get the meaning
of any word which we do not understand without having to
search for it in the dictionary. Further eye movement can be
used in the following applications in the near future.
R EFERENCES
[1] Robert J. K. Jacob, The Use of Eye Movements in Human-Computer
Interaction Techniques: What You Look At is What You Get. Naval
Research Laboratory, Washington, DC. 20375. ACM Transactions on
Information Systems, Vol. 9, No 3, April 1991, Pages 152-169.
[2] Linda E. Sibert, Robert J.K. Jacob, Evaluation of Eye Gaze Interaction
ACM Digital Library April 2000. CHI ’2000 The Hague, Amsterdam
1-58113-216-6/00/04. CHI Letters volume 2 issue 1.
[3] Heiko Drewes, Eye Gaze Tracking for Human Computer Interaction,
LMU Mnchen 2010.
[4] Robert J. K. Jacob and Keith S. Karn, Eye Tracking in Human Computer
Interaction and Usability Research: Ready to Deliver the Promises Journal
of Eye Movement Research. The Minds Eye: Cognitive and Applied
Aspects of Eye Movement Research. Hyona, Radach Deubel (eds.)
Oxford, England. Copyright 2003 by Elsevier Science BV.

• Password/ATM pin can be entered using eye gaze. It will

provide greater confidentiality to the user and would deter
shoulder surfing.
• The use of eye gaze in mobiles for outgoing calls can be
helpful in avoiding accidental calls.
• Washing machines, refrigerators, ovens and other embed-
ded devices can be operated by simple eye gestures.
• Will be able to turn off/on the lights and fans without
having to move.
• Eye movement can be used to make computer games
more interactive.
• Eye movement can be helpful in saving energy by switch-
ing off the display if the user is not looking at the screen
for a particular time.
Once eye movement has been established as a convenient
and reliable source of input there is no stopping the benefits
we can extract from it and soon our marketplaces will be
flooded by such devices. The use of eye movement as an
input in electronic gadgets will make life easier.

VI. CONCLUSION
Eye movement as an input medium has many advantages.
It is faster than any other current input media and requires no
training or particular coordination for normal users. Moreover,
it is convenient for disabled users. It provides an illusion that
the computer is acting on the users intentions rather than
its explicit commands. Because of these advantages, many
new applications have been designed in the past are still
being developed in spite of the limitations of eye trackers.
Taking a step further, we have proposed some applications
and hope to see them implemented in the future. We think
these applications can be incorporated in the modern day e-
books which would raise their standards and give them a slight
advantage over normal paper books.