Blue Eyes Ppt 2km33

Blue eye technology ( blue eyes – Human operator monitoring system )

BY

Agenda n n n n n n n n n n

Motivation What is BlueEye technology ? What is BlueEyes ? System deg System overview DAU CSU Data security IBM research Conclusion

Motivation n

Human error - a frequent reason of catastrophes and ecological disasters – temporal indisposition

•

•

– weariness – oversight Monitoring of conscious brain involvement

Lack of complex solution exploiting wireless link technology, database systems and AI algorithms

What is blue eye technology ? n

 

The BLUE EYES technology aims at creating computational machines that have perceptual and sensory ability like those of human beings.

How can we make computers “see” and “feel” ? n

n

Blue eyes uses sensing technology to identify a ’s actions and to extract key information. Information is then analyzed to determine the ’s physical ,emotional ,or informational state.

What is Blue Eyes ? n

The complex solution for humanoperator monitoring: – – – –

Visual attention monitoring Physiological condition Operator’s position detection Wireless data acquisition using bluetooth

What is BlueEyes not? n

n

Doesn’t predict nor interfere with operator’s thoughts Cannot force directly the operator to work

IBM research BLUE EYE – EMOTIONAL MOUSE sensors in the mouse ,sense the physiological attributes which are correlated to emotions using correlation model -by simply touching the mouse ,the computer will be able to determine a person’s emotional state. n BLUE EYE enabled TELEVISION – could become active when the makes an eye . n

Physiological foundations Human brain

Preconscious brain • •

alogical „automatic”

Conscious brain • • •

manages the knowledge logical reasoning makes decisions

Conscious brain - operation modes

Exploratory mode

Planning mode

The eye movement sensor n

n

-

Off-shelf eye movement sensor – Jazz multisensor Supplies raw digital data regarding Eye position Level of blood oxygenation Acceleration along horizontal and vertical axes Ambient light intensity

Eye movements •

Retina - central vision (macula)

•

Central vision and the conscious brain

Saccade - a rapid eye jump to a new location assigned by the conscious attention process.

Saccades presence

Exploratory mode

Jazz multisensor

Jazz Multisensor

Jazz

Design objectives n

A mobile data acquisition device

• A reliable real-time buffering, processing and recording • A clear visualization interface • Mechanisms for creating custom data analyzers • To ensure a possibility of distributing the processing among a number of computers

Implementation and engineering considerations -THREE groups of s n

n

n

OPERATOR -a person whose physiological parameters are supervised - authorization in the system - receiving alerts SUPERVISOR – a person responsible for analyzing operator’s condition and performance. SYSTEM S- maintains the system deliver tools for adding - new operator’s to the database - defining alarm conditions - configuring logging tools - creating new analyzer modules

Deg A personal area network for linking all the operators and the supervising system n Two major units - DAU (data acquisition unit ) - CSU (central system unit ) n

System overview Data Acquisition Unit Atmel 89C52 microcontroll er Jazz Multisensor

Bluetoot h

Central System Unit Bluetoo th

Connection Manager

Visualizati on Module

Data Analysis

Data Logger

Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.

DAU The DAU consists of the following components n ATMEL 8952 microcontroller n BLUE TOOTH MODULE – s synchronous voice data transmission n PCM CODEC –used to transmit operator’s voice and central system sound n UART –communication between bluetooth module and microcontroller (115200 bps) n MAX232 –level shifter n ALPHAUNUMERIC LCD display n LED indicators n ID CARD interface

DAU - features is

Data Acquisition Unit

n Atmel 89C52 microcontroll er

Bluetoot h

n

Jazz Multisensor

n

n

Lightweight Runs on batteries low power consumption Easy to use - does not disturb the operator working ID cards for operator authorization

DAU - components microphone

MC 145483 PCM codec

earphone

Bluetoot h modul

Jazz Multisensor

LCD display

Atmel 89C52 microcontroller

UART

Beeper

UART

LED indicators

MAX2 32

BlueEyes ID card interface ID card

Simple keyboar d

Data Acquisition Unit

Microcontroller & blue tooth module

EEPROMS & the PCB

Implementation - DAU n

•

Made from scratch by the team

n

Bluetooth module inside Jazz sensor interface Microphone and earphone sockets 5-key keyboard Two lines, 8 characters LCD Powered from 6 AA batteries

n

n

n n

n

Implementation - DAU PCM codec board Microcontroller board

Batteries and module voltageBluetooth level monitor

CSU - features n

n

n n n

n

Central System Unit

Connection management Bluetoo th Data processing Visualization Data recording Access verification System maintenance

Connection Manager

Visualizati on Module

Data Analysis

Data Logger

CSU – components n

n

n

n

CONNECTION MANAGER – main task to perform low-level blue tooth communication DATA ANALYSIS MODULE – performs the analysis of the raw sensor data in order to obtain information about operator’s physiological condition DATA LOGGER MODULE – provides for storing the monitored data. VISULAIZATION MODULE – provides interface for the supervisors

CSU - Visualization Module n

Raw and processed data visualization using: – – –

n

VU-meters pie-charts time series

audio and video data playback

CSU - saccade detector Saccade detector and conscious brain involvement level calculation

n

Raw eye movement data

Velocity

Eye movement velocity

Differentiation

250 200 150 100 50 0 -50 -100 -150 -200 -250 0

8000

16000

24000

32000

Time [ms]

Visual attention level

Attention level (Lva)

60 50

Conscious brain involvement level

40 30 20 10 0 0

8000

16000 Time [ms]

24000

32000

CSU - Connection Manager Operator Data Manager

Bluetoo th

... ...

Bluetoo th

Bluetooth Connection Manager

•

Initializes Bluetooth

•Data demultiplexer modules •Data producer queues

Inquires for new devices in range Raw connections • Establishes Separated Data • Operator s operator data streams Buffer Manager roamingAlert Alerts buffer • Feeds Operator Manager with...physiological data ... •

Operator Manager

Raw Data Buffer Alert buffer

Alerts

CSU - Data Analysis •

Custom data analyzers - decision tree based classifiers

•

Automatic decision tree coefficients generation - C4.5 based algorithm

Data samples

Assigned category “Yes”

...

No ... answer “Yes” “No” No answer

Decision tree coefficients

CSU - Data Analysis n

Custom data analyzers - decision tree based classifiers Example: Operator gesture recognition (simplified) Vertical Acceleration variance value high Horizontal Acceleration variance high

N “YES”

Y

N Horizontal Acceleration variance high

Y

N

No answer

Y “NO”

CSU - intermodule communication n

Uses buffered thread-safe queues and producer-consumer scheme at Data consumer

creates Data producer

Data queue

Data Data consumer & Processing consumer producer

Data Data consumer & Processing consumer producer

D ata q ueu e D ata q ueu e

Data consumer

CSU - intermodule communication Connection Connection Manager Manager Operator Operatormanager manager (OM) (OM) Alerts

Data DataAnalysis Analysis

... ...

OO MM

OO MM

Separated physiological data streams

Visualization Visualization Module Module

Processed data

Recorded (off-line) data

Data Logger

Implementation - CSU •

Standard MDI interface

n

Custom-built Bluetooth communication routines Intel USB camera ID card programmer Developed using Visual C++ 6.0 Deployment environment: Windows 2000, MS SQL Server

n n n n

Future improvements DAU small CMOS camera to monitor the operator’s point of gaze single PCB (SMD technology) low voltage ICs - LiIO batteries power CSU data mining algorithms advanced database encryption using e.g. AES algorithm

Data security n

n n n n

Only ed mobile devices can connect to the system Bluetooth connection authentication Bluetooth connection encryption Access rights restrictions Personal and physiological data encryption

App licat ions of Blue Eyes n

Generic control rooms

(System can be applied in every working environment requiring permanent operator’s attention) – Power station – Captain bridge – Flight control centers – Operating theatres – anesthesiologists

Blue E yes - benefits n

Prevention from dangerous incidents

n

Minimization of – – –

n

ecological consequences financial loss a threat to a human life

The reconstruction of the course of operator’s work

References n n n n n n

www.seminar4u.com www.howstuffswork.com www.cs.put.poznan.com www.whitepapers.com www.ibmresearchcenter.com www.almaden.ibm.com

Conclusion n

n

In the near future ,ordinary household devices- such as television , refrigerators ,ovens may be able to do their jobs when we look at them and speak to them. Future applications of blue eye technology is limitless

Thank you