BlueEyes

Human-Operator Monitoring System

Presented By –
Name Roll No.

Abha Singh
(0612031001)
Anupma Gupta (0612031018)

Institute Of Technology &

Management
 Agenda
• Motivation
• What is BlueEyes?
• Benefits
• Physiological foundations
• Eye movement sensor
• System design
• Personal Area Network
• Application
• Conclusion
Institute Of Technology &
Management
 Motivation
Human error - a frequent reason of
catastrophes and ecological disasters
– temporal indisposition
– weariness
– oversight

Some industrial environment can create a

danger of exposing the operator to toxic
substances, which can affect his cardiac,
circulatory and pulmonary systems. Thus,
on the grounds of plethysmographic signal
taken from the forehead skin surface, the
system computes heart beat rate and blood
oxygenation. Institute Of Technology &
Management
 What is BlueEyes ?
• The complex solution for human-operator
monitoring:
– conscious brain involvement monitoring
– triggering user-defined actions
– data recording

What is BlueEyes not?

• Doesn’t predict nor interfere with operator’s thoughts
• Cannot force directly the operator to work

Institute Of Technology &

Management
 BlueEyes - benefits
• Prevention from dangerous incidents
• Minimization of
– ecological consequences
– financial loss
– a threat to a human life

• The reconstruction
of the course of
operator’s work

Institute Of Technology &

Management
 Physiological foundations

Human brain

Preconscious brain Conscious brain

• alogical • manages the knowledge
• „automatic” • logical reasoning
• makes decisions

Institute Of Technology &

Management
 Eye movements

• Central vision and

the conscious brain

Saccade - a rapid eye jump

to a new location
assigned by the
conscious attention
process.

Saccades presence ⇔ Exploratory mode

Institute Of Technology &
Management
 The eye movement sensor
• Eye position measuring - direct infrared
occulography
• Oxy- and deoxyhemoglobin measurement
• Two axial
accelerometer
• Ambient light
sensor

Institute Of Technology &

Management
 Wires, wires, wires...

A serious limitation of:

– mobility
– comfort
– safety

Institute Of Technology &

Management
 Personal Area Network !

Solution:

A wireless Personal Area

Network

Institute Of Technology &

Management
BlueEyes emphasizes – Bluetooth
technology and the movements of the eyes.
Bluetooth provides reliable wireless
communication whereas the eye movements
enable us to obtain a lot of interesting and
important information.

Institute Of Technology &

Management
 System overview
Data Acquisition Unit Central System Unit

Atmel 89C52 Connection Visualization

Bluetooth Bluetooth Manager Module
microcontroller

Jazz Data
Data Analysis
Multisensor Logger

Bluetooth technology provides means

for creating a Personal Area Network
linking the operators and the central
system.

Institute Of Technology &

Management
 DAU - features
Data Acquisition Unit
• Lightweight
• Runs on batteries - low
Atmel 89C52
microcontroller
Bluetooth power consumption
Jazz • Easy to use - does not
Multisensor disturb the operator
working
• ID cards for operator
authorization
• Voice transmission using
hardware PCM codec

Institute Of Technology &

Management
 CSU - features
• Connection Central System Unit
management
• Data processing Connection Visualization
Bluetooth Manager Module
• Visualization
• Data recording Data
Data Analysis
Logger

• Access
verification
• System maintenance

Institute Of Technology &

Management
 DAU - components
microphone MC 145483
earphone PCM codec

Jazz Multisensor Bluetooth

module
Beeper Atmel 89C52

UART
microcontroller
LCD display
UART MAX232
LED indicators
BlueEyes
Data Acquisition Unit
ID card Simple
interface keyboard
ID card

Institute Of Technology &

Management
CSU - intermodule communication
Connection
Manager

Operator manager (OM)

... OM OM

Alerts Separated physiological

data streams

Visualization Data
Data Analysis
Module Logger

Processed data

Recorded (off-line) data

Institute Of Technology &

Management
 CSU - Connection Manager

Operator • Initializes Bluetooth

Data •Data demultiplexer
Manager modules
•Data producer queues
• Inquires for new devices
in range
Separated
Bluetooth •Operator Raw Data
Establishes connections
Buffer data streams
•Manager
Supports operator
Alert buffer Alerts
roaming
... • Feeds ... Operator Manager
Bluetooth
with physiological data
... Connection ...
Manager

Raw Data
Operator Buffer
Manager
Bluetooth Alert buffer Alerts

Institute Of Technology &

Management
CSU - intermodule communication
Connection
Manager

Operator manager (OM)

... OM OM

Alerts Separated physiological

data streams

Visualization Data
Data Analysis
Module Logger

Processed data

Recorded (off-line) data

Institute Of Technology &

Management
 CSU - Data Analysis

Data analysis module consist of saccade

detector, pulse rate generator, Custom
analyzer. It analysis the data for
obtaining the information about the
operator’s physiological condition . The
saccade detector is used for calculating
the velocity of eye movement.

Institute Of Technology &

Management
 CSU - saccade detector
• Saccade detector and conscious brain
involvement level calculation
Eye movement velocity

Raw eye 250

200
saccade

movement data 150

100

Velocity
50
0
-50
-100
-150
-200
-250
Differentiation 0 8000 16000 24000 32000
Time [ms]

Visual attention level

60
Attention level (Lva)

50
40 Conscious
30 brain
20
involvement
10
0
level
0 8000 16000 24000 32000
Time [ms]

Institute Of Technology &

Management
CSU - intermodule communication
Connection
Manager

Operator manager (OM)

... OM OM

Alerts Separated physiological

data streams

Visualization Data
Data Analysis
Module Logger

Processed data

Recorded (off-line) data

Institute Of Technology &

Management
 CSU - Visualization Module
• Raw and processed data visualization using:

– VU-meters
– pie-charts
– time series

• audio and video

data playback

Institute Of Technology &

Management
CSU - intermodule communication
Connection
Manager

Operator manager (OM)

... OM OM

Alerts Separated physiological

data streams

Visualization Data
Data Analysis
Module Logger

Processed data

Recorded (off-line) data

Institute Of Technology &

Management
 Data security
• Only registered mobile devices
can connect to the system
• Bluetooth connection authentication
• Bluetooth connection encryption
• Access rights restrictions
• Personal and physiological data encryption

Institute Of Technology &

Management
Implementation and
engineering consideration
There are three user
Operator:
Operator is a person whose physiological
parameters are supervised.
The operator wears the DAU.
The only functions offers to the operator are
Authorization to the system and receiving alarm
alerts.
Supervisor:
 He is the person responsible for analyzing
operators’ condition and performance
System Administrator:
 He is the user that maintains the system to
delivered tools for adding new operators to the
database.
Institute Of Technology &
Management
BlueEyes system can be applied in every
working environment requiring
permanent operator's attention:
• at power plant control rooms
• at captain bridges
• at flight control centers

Institute Of Technology &

Management
 Conclusion

In future it is possible to create a computer

which can interact with us as we interact each
other with the use of blue eye technology. It
seems to be a fiction, but it will be the life lead
by “BLUE EYES” in the very near future. Ordinary
household devices -- such as televisions,
refrigerators, and ovens -- may be able to do
their jobs when we look at them and speak to
them. By using Blue Eyes the dangerous
accidents & ecological disaster will be minimize
in industrial field.

Institute Of Technology &

Management
 References
• www.seminar4u.com
• www.howstuffswork.com
• www.cs.put.poznan.com
• www.whitepapers.com
• www.ibmresearchcentre.com

Institute Of Technology &

Management
Institute Of Technology &
Management
Institute Of Technology &
Management