A COLLOQUIUM REPORT

ON

COMPUTER VISION

SUBMITTED BY

(HIMANI)

(1403213041)

INFORMATION TECHNOLOGY DEPARTMENT,

ABES ENGINEERING COLLEGE, GHAZIABAD

* 2015-2016 *

1
 ABES ENGINEERING COLLEGE, GHAZIABAD

Department of Information Technology

CERTIFICATE

This is to certify that Ms. HIMANI, Roll No. 1403213041, has successfully
completed Colloquium seminar on COMPUTER VISION upto satisfaction level
and submitted the same during the academic year 2015-2016 towards the Course
requirement, under the Department of Information Technology, ABES Engineering
College, Ghaziabad.

Prof.(Dr.) P.C.Vashist

Colloquium Coordinator HOD-IT

2
 ACKNOWLEDGEMENT

I am very grateful to Dr. P.C. VASHIST head of Department, Department of Information

Technology,Abes Engineering College Ghaziabad for providing all the required resources for
the successful completion of my seminar.
My heartfelt gratitude to my internal guide Mr. SANJEEV KAPOOR and Mr. SAURABH
SRIVASTAVA, Assistant professors, for their valuable suggestions and guidance in the
preparation of the seminar report.
I will be failing in duty if I do not acknowledge with grateful thanks to the authors of the
references and other literatures referred to in this seminar.
I express my thanks to class coordinators and all staff members and friends for all the help and
co-ordination extended in bringing out this seminar successfully in time.
Last but not the least; I am very much thankful to my parents who guided me in every step which
I took.

HIMANI
1403213041

3
 INDEX

Page No

1. Certificate 02
2. Acknowledgement 03
3. Table of Figures 05
4. Abstract 06
5. Computer Vision: Overview 07
6. Introduction 08
7. History 10
8. Application 11
9. Related Fields 14
10. Computer Vision System Method 15
11. Human Computer Interaction 16
12. Eye Tracking 18
13. Conclusion 23
14. References 23

4
 TABLE OF FIGURES

Page No

1. Hardware Component of CV System 08

2. TimeLine Diagram Of CV System 10
3. Application of Computer Vision 11
4. Relation b/w CV and Other Fields 14
5. Medical Application Of Eye Tracking 18
6. Other Application Of Eye Tracking 19
7.

5
 ABSTRACT

Vision consists of a multiplicity of tasks, of which object identification is only one. We in the
computer vision community have concentrated our efforts on object identification, and have
thereby ensured that the formulation of the problem of vision provides methods which are not of
general utility for vision. Ironically one consequence of this is that computer vision may not even
be of use for object identification. An analysis of why computer vision has become synonymous
with object identification is presented. The implications of this analysis for object identification
and for interpreting neurophysiological evidence in terms of 'feature detectors' are presented. A
formulation of the problem of vision in terms of spatio-temporal characteristics is proposed.

6
Computer Vision

The purpose of this section is to describe computer vision and its application.

Computer vision covers the core technology of automated image analysis which is used in many
fields. Computer vision is a field that includes methods for acquiring, processing, analyzing, and
understanding images and, in general, high-dimensional data from the real world in order to
produce numerical or symbolic information. The development of this field has been to duplicate
the abilities of human vision by electronically perceiving and understanding an image.

7
 INTRODUCTION

Computer vision is the science and technology of machines that see. Vision is therefore one of the
problems of computer science most worthy of investigation because we know that it can be solved, yet
we do not know how to solve it well. In fact, to solve the "general vision problem" we will have to come
up with answers to deep and fundamental questions about representation and computation at the core of
human intelligence.

Computer vision is concerned with modeling and replicating human vision using computer software and
hardware (Figure1). It combines knowledge in computer science, electrical engineering, mathematics,
physiology, biology, and cognitive science. It needs knowledge from all these fields in order to
understand and simulate the operation of the human vision system.

Fig1.Typical Hardware Components of a Computer Vision System

Goal of Computer Vision is to:

- Write computer programs that can interpret images .

- Process images acquired with cameras in order to produce a representation of objects in the
world.

8
 What is Computer Vision?

Trucco and Verri: computing properties of the 3D world from one or more digital images
Sockman and Shapiro: To make useful decisions about real physical objects and scenes based
on sensed images.
Ballard and Brown: The construction of explicit, meaningful description of physical objects
from images.
Forsyth and Ponce: Extracting descriptions of the world from pictures or sequences of pictures

So, Computer vision (Automatic understanding of images and video .) is a discipline that studies
how to reconstruct, interpret and understand a 3D scene from its 2D images in terms of the
properties of the structures present in the scene.
As a Scientific Discipline, computer vision is concerned with the theory behind artificial
systems that extract information from images.
As a Technological Discipline, computer vision seeks to apply its theories and models to the
construction of computer vision systems

Examples of Computer Vision Systems are used for:

Controlling processes (e.g., an industrial robot).

Navigation (e.g. by an autonomous vehicle or mobile robot).
Detecting events (e.g., for visual surveillance or people counting).
Organizing information (e.g., for indexing databases of images and image sequences).
Interaction (e.g., as the input to a device for computer-human interaction).
Automatic Modeling objects or environments (e.g., medical image analysis or
topographical modeling).
Inspection ( e.g. in manufacturing applications).

Sub-domains of computer vision include scene reconstruction, event detection, video tracking,
object recognition, learning, indexing, motion estimation, and image restoration.

Why is Vision So Difficult?

9
Because vision is an inverse problem (that mean: to describe the world that we see in one or
more images and to reconstruct its properties, such as shape, illumination, and color
distributions), in which we seek to recover some unknowns given in sufficient information to
fully specify the solution. We must therefore resort to physics-based and probabilistic models to
disambiguate between potential solutions .

History of Computer Vision

A brief Personal Synopsis of the main developments in computer vision over the last 30 years
figure (2)
Fig
.2 (A

rough time line of some of the most active topics of research in computer vision)

1970s. When computer vision first started out in the early 1970s, it was viewed as the visual
perception component of an ambitious agenda to mimic human intelligence and to endow robots
with intelligent behavior.
1980s. In the 1980s, a lot of attention was focused on more sophisticated mathematical
techniques for performing quantitative image and scene analysis.
1990s. While a lot of the previously mentioned topics continued to be explored, few of them
become significantly more active.
2000s.This past decade has continued to see a deepening inter play between the vision and
graphics fields.

10
 Computer Vision Applications :

Computer vision is being used today in a wide variety of real-world applications,

which appeared in figure (3) and include:

Robotics
Robotics Applications:
1. Navigation and visual serving.
2. Obstacles avoidance.
3. Localization-determine robot location automatically.
4. Assembly (peg-in-hole, welding, painting).
5. Manipulation (e.g. PUMA robot manipulator).
6. Intelligent robotics to interact with and serve people.

Medicine
Medicine Applications:
1. Classification and detection (e.g. lesion or cells classification and tumor detection.
2. 3D human orange construction (MRI or ultra sound).
3. Vision-guided robotics surgery.
4. 2D/3D segmentation.

11
 Security
Security Applications:
1. Biometrics (iris, finger print, face recognition).
2. Surveillance-detecting certain suspicious activities or behaviors (like monitoring for intruders,
analyzing high way traffic, and monitoring pools for drowning victims).

Transportation
Transportation Applications:
Automotive safety: detecting unexpected obstacles such as pedestrians on the street, under
conditions where active vision techniques such as radar or lieder do not work as well.
1. Autonomous vehicle.
2. Safety, e.g., driver vigilance monitoring.

Industrial automation
Industrial Applications:
1. Industrial inspection (defect detection and menstruation).
- Machine inspection: rapid parts inspection for quality assurance using stereo vision with
specialized illumination to measure tolerances on aircraft wings or auto body parts, or looking
for defects in steel castings using X-ray vision.
2. Assembly.
3. Barcode and package label reading.
4. Object sorting.
5. Document understanding [e.g. (OCR) Optical character recognition: reading hand written
postal codes on letters and automatic number plate recognition (ANPR). OCR: Technology to
convert scanned docs to text].

12
 Image/video databases
Image/video databases Applications:
1. It is mainly used for image retrieval based on image content.

Human Computer Interface

Human Computer Interface Applications:
1. Gaze estimation.
2. Face expression recognition.
3. Head and hand gesture recognition, and American Sign Language.

Other Applications: Touching Our Life

Football.
Movies.
Road monitoring.
Autonomous driving.
Space: planetary exploration, docking.
Medicine pathology, surgery, diagnosis (Medical imaging: registering pre-operative and
intra-operative imagery or performing long-term studies of peoples brain morphology as they
age).
Microscopy.
Military.
Remote Sensing.
Retail: object recognition for automated checkout lanes.
3D model building (photogrammetry): fully automated 3D model building from aerial
photographs used in systems such as Virtual Earth.
Match move: merging computer generated imagery (CGI) with live action foot age by
tracking feature points in the source video to estimate the 3Dcamera motion and shape of the
environment.
Motion capture (mocap): of actors for computer animation, using retroreflective markers
viewed from multiple cameras or other vision based techniques.

13
 Related Fields :

Image Processing
Computer Graphics
Pattern Recognition
Perception
Robotics
AI

All above fields are related disciplines for Computer vision and figure (4) show the relation
between them and it.

Fig4. (Relation between Computer Vision and Various Other Fields)

A brief description about relation between these fields and Computer

Vision:

Computer vision is sometimes seen as a part of the Artificial Intelligence field or the computer
science field in general. [Computer Vision is used to allow AI to identify and analyze a picture or
components in a picture taken by sensory instruments. After identifying edges, objects, or
patterns the AI can then act according to the stimulus given].

14
Physics is another field that is closely related to computer vision. [Computer vision is a tool used
in physics to extend our understanding beyond things that we cannot physically see] .
A third field which plays an important role is Neurobiology, specifically the study of the
biological vision system. [Computer vision is tied closely to the research of human vision.
Computer vision used to mimic and simulate the behavior of biological optics].

Another field related to computer vision is Signal Processing . The fields most closely related to
computer vision are Image Processing(), Image Analysis(Image processing and image analysis
tend to focus on 2D images, how to transform one image to another, e.g., by pixel-wise
operations such as contrast enhancement, local operations such as edge extraction or noise
removal, or geometrical transformations such as rotatin g the image. This characterization
implies that image processing/analysis neither require assumptions nor produce interpretations
about the image content.) , and Machine Vision.

Computer Vision System Methods

The organization of a computer vision system is highly application dependent. Some systems are
stand-alone applications which solve a specific measurement or detection problem, while others
for control of mechanical actuators, planning, information databases, man-machine interfaces,
etc.
There are, however, typical functions which are found in many computer vision systems:

Image acquisition: A digital image is produced by one or several image sensors, which,
besides various types of light-sensitive cameras, include range sensors, tomography devices,
radar, ultra-sonic cameras, etc. Depending on the type of sensor, the resulting image data is an
ordinary 2D image, a 3D volume, or an image sequence. The pixel values typically correspond to
light intensity in one or several spectral bands (gray images or color images), but can also be
related to various physical measures, such as depth, absorption or reflectance of sonic or
electromagnetic
waves, or nuclear magnetic resonance.

15
Pre-processing: Before a computer vision method can be applied to image data in order to
extract some specific piece of information, it is usually necessary to process the data in order to
assure that it satisfies certain assumptions implied by the method.

Feature extraction: Image features at various levels of complexity are extracted from the
image data, more complex features may be related to texture, shape or motion.

Detection/segmentation: At some point in the processing a decision is made about which

image points or regions of the image are relevant for further processing.

High-level processing: At this step the input is typically a small set of data, for example a
set of points or an image region which is assumed to contain a specific object. The remaining
processing deals with, for example:
- Verification that the data satisfy model-based and application specific assumptions.
- Estimation of application specific parameters, such as object poses or objects size.
- Image recognition: classifying a detected object into different categories.
- Image registration: comparing and combining two different views of the same object.

Decision making: Making the final decision required for the application (like: matched or
unmatched recognition).

Human Computer Interaction

The purpose of this section is to describe HCI which is one of computer vision realworld
applications.
The research in this area has been growing very fast in the last few decades. In recent years there
has been a growing interest in improving all aspects of the interaction between humans and
computers. The growth in Human-Computer Interaction (HCI) field has not only been in
quality of interaction, it has also experienced different branching in its history.Human-computer
intelligent interaction as opposed to simple human computer interaction [20]. Research in
Human-Computer Interaction (HCI) has been spectacularly successful, and has fundamentally
changed computing.

16
Human computer interaction (HCI) lies at the crossroads of many scientific areas including
artificial intelligence, computer vision, face recognition, motion tracking, etc. The new direction
of research is to replace common regular methods of interaction with intelligent, adaptive,
multimodal, natural methods.

Human-Computer Interaction: Definition, Terminology

Sometimes called as Man-Machine Interaction or Interfacing, concept of Human-Computer

Interaction/Interfacing (HCI) was automatically represented with the emerging of computer, or
more generally machine, itself. The main terms that should be considered in the design of HCI:
functionality and usability.
Functionality of a system is defined by the set of actions or services that it provides to its users.
However, the value of functionality is visible only when it becomes possible to be efficiently
utilized by the user.
HCI is a design that should produce a fit between the user, the machine and the
required services in order to achieve a certain performance both in quality and
optimality of the services.

Overview on HCI .
Existing HCI Technologies
HCI design should consider many aspects of human behaviors and needs to be useful. The
complexity of the degree of the involvement of a human in interaction with a machine is
sometimes invisible compared to the simplicity of the interaction method itself. The existing
interfaces differ in the degree of complexity both because of degree of functionality /usability
and the financial
and economical aspect of the machine in market. In design of HCI, the degree of activity that
involves a user with a machine should be thoroughly thought. The user activity has three
different levels: physical, cognitive and affective. The physical aspect determines the
mechanics of interaction between human and computer while, The cognitive aspect deals with
ways that users can understand the system and interact with it. The affective aspect is a more
recent issue and it tries not only to make the interaction a pleasurable experience for the user but

17
also to affect the user in a way that make user continue to use the machine by changing attitudes
and emotions toward the user.

Eye Tracking

Eye Tracking is a part of the HCI Practice track, and in this section we will describe eye
tracking technique and its real practices.

Researchers predict that eye tracking will become widely used in HCI research and practice,
because it provides objective data on the physiological and perceptual impact of interaction that
have so far remained untapped . Eye tracking used for the evaluation and design of digital
interfaces such as websites, games, iTV, mobile phones and more . Eye tracking is a technology
that enables some feature(s) of the eye to be tracked visually by a camera or imaging system,
delivering real-time coordinates of the users gaze . It has been used over the past two decades in
a number of military and academic research institutes .But, with Technologys Eye gaze system
and development of equipments increasing number of HCI researchers and practitioners can
consider employing eye- tracking. The majority of studies on eye-tracking to date have been
concerned with using eye-tracking as an input device, rather than as a data collection tool .

What is Eye Tracking?

Eye tracking is the process of measuring either the point of gaze ("where we are looking") or the
motion of an eye relative to the head.

18
 Introduction to Eye Tracking:
Eye tracking is one of the most salient features of the human face, eyes and their movements
play an important role in expressing a persons desires, needs, cognitive processes, emotional
states, and interpersonal relations. Eye-movement tracking is a method that is increasingly being
employed to study usability issues in HCI contexts. An eye tracking systems are a systems that
can track the movements of a user's eyes, or it is Method for calculating what user is looking at.
Eye movement reflects the human thought processes.
Eye tracking (ET) is a technique for capturing eye movements as a person looks at a computer
interface. Eye movement research is of great interest in the study of neuroscience and psychiatry,
as well as ergonomics, advertising and design. Since eye movements can be controlled

19
volitionally, to some degree, and tracked by modern technology with great speed and precision,
they can now be used as a powerful input device, and have many practical applications in
human-computer interactions. Eye tracking is a technique whereby an individuals eye
movements are measured so that the researcher knows both where a person is looking at any
given time and the sequence in which their eyes are shifting from one location to another.
Tracking peoples eye movements can help HCI researchers understand visual and display-based
information processing and the factors that may impact upon the usability of system interfaces
[8]. Eye movements can also be captured and used as control signals to enable people to interact
with interfaces directly without the need for mouse or keyboard input, which can be a major
advantage for certain populations of users such as disabled individuals[8].

Eye Tracking Definition .

Eye tracking is a technique allowing testers to determine eye movement and eye-fixation patterns
of a person. Eye tracking can be used in both passive and active modes. In usability testing, eye
tracking helps software designers to evaluate the usability of their screen layouts. It is an
example of passive, monitoring use of eye tracking, because the eye-tracking devices simply
monitor eye activity for later study and analysis.
Other examples of passive mode are:
Behavioral scientists can monitor what pilots or control room operators look at when given
certain tasks or placed in certain situations.
Reading specialists can use eye tracking to recognize when a person is reading and when
he/she has fixated on a word longer than normal, in order to create a highly individualized
reading aid.
Marketing researchers can determine what features of product advertising and packaging
attracts buyer attention.
Eye tracking can also be used to actively direct a computer through the motions of the eyes
(active, control mode). Examples are:
Disabled people who cannot use their hands to operate a computer can do so with their eyes,
using on-screen keyboards and mouse controllers.
Hospitals can provide an eye-aware communication program to people who have lost their
ability to move and speak, either temporarily or permanently through a traumatic accident.

20
 Applications of Eye Tracking:
Active research, eye detection and tracking remain a very challenging task dueto several unique
issues, including occlusion of the eye by the eyelids, degree of openness of the eye, variability in
either size, reflectivity or head pose, etc. Applications of computer vision, such as People
tracking, face detection, and various medical applications, encounter occlusions and shape
variations, but
rarely of the same order of magnitude and frequency as seen with the eyes .
Application Fields of Eye Tracking:
Cognitive Studies.
Medical Research.
Human Factors.
Computer Usability.
Translation Process Research.
Vehicle Simulators.
In-vehicle Research.
Training Simulators.
Virtual Reality.
Adult Research.
Infant Research.
Adolescent Research.
Geriatric Research.
Primate Research.
Sports Training.
FMRI / MEG / EEG.
Commercial eye tracking (web usability, advertising, marketing, automotive, etc).
Finding good clues.
Communication systems for disabled.
Improved image and video communications.
Computer Science: Activity Recognition.

Quick tour on real world applications in Eye Tracking:

21
Eye tracking can be used in a variety of different applications:
Academic Research:
Research that analyzes human behavior and patterns, Fields of Research:
Psychology Eye movements provide objective data on how subjects perceive the world
and how they react to stimuli.
Neuroscience Eye tracking devices combined with physiological data such as brain
imaging can help identify how the information is processed in the brain. Eye tracking can be
used to analyze visual development and link it to developmental aspects of neurological
functions, neurological diseases and brain damage.
Reading research Reading patterns can be cross referenced with different demographics
of people and therefore provide insight intohow they gather information.
Computer science Human computer interaction allows users to input information in a
more natural way into their computers. Eye tracking can be used as a control medium, like
moving the cursor and clicking on icons on the screen, as well as creating adaptive user
interfaces, where the computer reacts to the eye gaze of the user and creates an interactive
environment.
Psycholinguistics Using eye tracking can help analyze how people acquire, comprehend
and use language. Eye tracking can also be used in translation process studies.
Sports Research Eye tracking is great for coaches who want to train their players on
effectively gathering information from the field through simulations.
Training and Learning Research Learning to analyze an environment quickly can be a
valuable skill in air traffic control, radar control, medical X-ray examinations, video surveillance,
industrial process control, driving, army or police field work, surgical training and others.
Teaching this skill through simulations using an eye tracker can eliminate the time-consuming
process of only learning it with experience.

22
 CONCLUSION

In this report we have attempted to give an overview of eye-tracking technology; how the
techniques work, what the history and background is, what present-day implementations are like
and the most famous application of it, what limitations of the eye tracking technique. and as well
as we spoke about eye tracking it have delivered details about Human Computer Interaction as a
big field contains eye tracking and Computer Vision as the general field contains HCI and ET.

REFERENCE
1. Walter G. Kropatsch "History of Computer Vision: A Personal Perspective", 2008.
2. http://cvlab.epfl.ch/~fua/courses/vision/intro/vision.html
3. http://en.wikipedia.org/wiki/Computer_vision
4. http://www.downloadpdffree.com/vision.pdf/20
5. http://www.coursehero.com/file/2041671/lec1/
6. Daniel C. Richardson "Eye-Tracking: Characteristics and Methods & Eye-Tracking: Research
Areas and Applications", .
7. Andrew T. Duchowski "A Breadth-First Survey of Eye Tracking Applications", 2002.
8. Alex Poole and Linden J. Ball "Eye Tracking in Human-Computer Interaction and Usability
Research: Current Status and Future Prospects", 2005.
9. Robert J. K. Jacob and Keith S. Karn "Eye Tracking in HumanComputer Interaction and
Usability Research: Ready to Deliver the Promises", 2003.
10. Dan Witzner Hansen and Qiang Ji "In the Eye of the Beholder: A Survey of Models for Eyes
and Gaze", 2010.
11. Kara Pernice and Jakob Nielsen "Eye tracking Methodology: How to Conduct and Evaluate
Usability Studies Using Eye tracking ", 2009.
12. http://www.uxbooth.com/blog/a-brief-history-of-eye-tracking/

23
13. Daniel C. Richardson "Part 1: Eye-Tracking: Characteristics and Methods & Part 2: Eye-
Tracking: Research Areas and Applications", 2004.
14. Crystal Chin, Shirley Lee, and Judy Ramey "An Orientation to Eye Tracking in Usability
Studies", 2005.

24