Sunteți pe pagina 1din 144

Image Processing

Chapter 1: Introduction

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Personal Data: Joachim Denzler

Office Room 1220, Ernst-Abbe-Platz 1-3

Phone 03641/946420

Fax 03641/946372

E-Mail joachim.denzler@uni-jena.de

URL http://inf-cv.uni-jena.de/

Office Tuesday, 4pm-5pm, Room 1220 (please contact me before by


hours email)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Goals of todays lecture

l Terms and definitions


l Structure and goals of the course
l Expectations
l Exercises
l Requirements and exams
l Course material and literature
l Introduction and motivation

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
What is Computer Vision?

l General Goals:
– Computation of properties of the 3d world from one or several
digital images
– Properties: geometric or dynamic
– Geometric properties: shape of an object, position in the world
– Dynamic properties: speed, velocity, and motion direction
l Several disciplines are involved
– Physics and Optics
– Mathematics and Computer Science
– Image Processing
– Artificial Intelligence
– Pattern Recognition
– Signal Processing, Control Theory
– Robotics
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Related Areas

l Other terms: image analysis, scene analysis, image


understanding
l Differentiation possible:
– Image Processing: related to image-to-image transformations,
seeks for general properties in images (for example, lines,
regions) and methods to improve digital images with respect to
further processing
– Pattern Recognition: detection and classification of objects in
digital images (2d as well as 3d objects)
– Photogrammetry: contact free computation of measures from
digital images with high precision

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Application examples

l Industrial inspection and quality control


l Reverse engineering
l Surveillance and security applications
l Face and gesture recognition
l Surveillance in road traffic (e.g. electronic toll systems)
l Autonomous driving and driver assisting systems
l Hand-Eye robot systems (in medical applications)
l Space and military applications
l Medical image processing (CT, MR, X-ray)
l Image data bases and image retrieval
l Augmented and virtual reality (close connection to
computer graphics)
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Image Retrieval: unsupervised clustering

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Optimal Sensor Data Acquisition

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Active Scene Exploration

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Object Tracking I

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Object Tracking II

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Object Tracking III

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Object Tracking IV

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
3d Reconstruction I

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
3d Reconstruction II

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
3d Reconstruction III

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
3d Reconstruction IV

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Visualisation I

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Visualization II

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Visualization III

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Visualization IV

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Visualization V

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
3d State Estimation I

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
3d State Estimation II

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Augmented Reality

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Augmented Reality

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Grasp Planning of Objects

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Future of Computer Vision?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Some Examples from Own Research

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Further Examples

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Further Examples

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Further Examples

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Further Examples

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Image Processing and Computer Vision in Industry

l Selection of US companies:
http://www.cs.ubc.ca/spider/lowe/vision.html
– Inspection and quality control (IC inspection)
– Fingerprint, iris, and face recognition software
– Construction and design of new sensor (3d cameras)
– Image processing for industrial robots
l Selection of German companies and research institutes:
http://kogs-www.informatik.uni-
hamburg.de/~koethe/german_vision.html
– Quality control
– Remote sensing
– Medical image processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Chair for Computer Vision

l Laboratory for Computer


Vision and Robotics
l Students‘ laboratory: work
places (PCs, cameras, pan/tilt
systems) for Bachelor and
Master's thesis
l Distributed image processing
system for surveillance
applications
l The robot Talos

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Chair for Computer Vision

l Pan-Tilt Units (Directed


Perception)
l Speed up to 300
degree/second
l Computer controlled via
RS232
l Accuracy up to 3 arc minutes

Copyright of image by directed perception

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Chair for Computer Vision

l Sony DFW VL 500 (Firewire


zoom camera)
l Digital,1/3 inch color camera,
progressive scan
l motor controlled aperture,
focus, and 12x zoom
l 640x480 quadratic pixels, 30
fps, external trigger

Copyright of image by Sony

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sensors in Motion

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sensors in Motion

l Mobile system Talos


l Part of the Distributed
Computer Vision System
l Research Goals: actively move
cameras to solve tasks like
object recognition, tracking,
and 3d reconstruction using a
bunch of cameras

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Research Areas

l Active Sensor Data Acquisition: can we solve problems in computer


vision by active, goal-oriented sensor data acquisition more efficiently,
with higher quality or even at all?
l Generic Object Recognition: how does a human knows what is
specific for an object of category „cup“ (my cup, your cup, nice cups,
coffee cups, tea cups, kids cups, broken cups)?
l Learning from View Examples: is it possible to learn new objects from
just a few examples in the order of 2-5 sample images
l 3d Reconstruction: how can we reconstruct geometric properties of
objects like surfaces in real time and online?
l Image Based Modelling: how can we reconstruct photorealistic
models of objects or scenes automatically and how can such models be
applied in model based computer vision?
l Real-Time Object Tracking: how can we track interesting events in
video streams and keep such events (for example, people) in the field
of view of a camera
l Calibration of Sensor Networks: how can we decide for the relative
pose of cameras without explicit camera calibration, i.e. without use of
calibration objects placed in the scene

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Cooperation with Industry

l Recognition of passports and automatic reading


l Optical character recognition in difficult environments
l License plate recognition and automatic reading
l Automotive Applications (pedestrian and road sign
recognition)
l Partners: Siemens Health Care (Forchheim), Desko
(Bayreuth), Crossmatch Technologies (USA, Jena),
Robot/Jenoptik (Hildesheim), Continental Deutschland
(Lindau), Zeiss AG (Jena)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Typical Steps in Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Typical Steps in Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Typical Steps in Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Typical Steps in Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Typical Steps in Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Typical Steps in Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Structure of the Course

l Chapter 2: Basics of Digital Images


– Comparison with the human “visual system”
– Image Sensing and Acquisition
– Sampling and Quantization
l Chapter 3: Image Enhancement in the Spatial
Domain
– Gray level transformation
– Histogram processing
– Linear Systems and spatial filtering
– Smoothing spatial filters
– Gradients in the image plane and sharpening filters

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Structure of the Course II

l Chapter 4: Image Enhancement in the Frequency


Domain
– Fourier transform, FFT
– Smoothing and sharpening frequency domain filters
– Filter Analysis and Design
l Chapter 5: Image Compression
– Models
– Element of Information Theory
– Error free and lossy coding
– Current standards

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Structure of the Course III

l Chapter 6: Segmentation
– Detection of discontinuities
– Edge links, boundary detection, Contour tracking
– Thresholding
– Region-based segmentation
– Motion in image segmentation
l Chapter 7: Representation of Segmentation Results
– Chain codes and interpolation techniques
– Signatures
– Contour/boundary descriptors
– Regional descriptors
– Principal component analysis

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Structure of the Course IV

l Chapter 8: Object Recognition


– Relations to pattern recognition
– Distance measuring classifiers
– Statistical approaches
– Neural Networks

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Procedure During the Semester

l Reminder Questions: every class we start with some


questions concerning the content of the previous class
(answers hopefully from you!)
l Exercises: apply your knowledge to design small image
processing solutions using Matlab Image Processing
Toolbox
– accounts necessary: will be organized next week
l Slides: will be designed just in time and provided in CAJ
(registration necessary, https://caj.informatik.uni-jena.de)
l Feedback and discussions are welcome (and important!)
l Important Dates:
– July 30th : examination
– October, 1st : if you do not pass, second chance
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Preparation for the exam

l Important: do not rely just on the slides


l Better: read textbook in parallel
l Much better: Learn in a team, ask, explain
l Strategy for preparing for the exam:
1. Short talk preparation: for each chapter prepare a 5min talk, in
which you summarize the content from a general overview to
the details of individual techniques
2. Try to explain details to another person
l If you have problems doing 1.: you need to continue
preparation
l If you have problems in doing 2.: there is a high risk that
you did not understand everything (at least the details)
l Examination date: July 20th (oral)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
References in WWW

l Webpages with infos/tutorials


– Computer Vision Homepage: http://www-
2.cs.cmu.edu/~cil/vision.html
– CV Online: http://homepages.inf.ed.ac.uk/rbf/CVonline/
l Free software:
– Vigra: http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/
– VXL: http://vxl.sourceforge.net/
– OpenCV: http://www.intel.com/research/mrl/research/opencv/
– Gandalf: http://gandalf-library.sourceforge.net/
– Libraries to convert image formats:
ftp://wuarchive.wustl.edu/graphics/graphics/packages/pbmplus
– PUMA: http://www.uni-koblenz.de/~puma/
– Further Overview: http://peipa.essex.ac.uk/info/software.html

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Commercial Software

l Matlab Image Processing Toolbox:


http://www.mathworks.com/products/matlab/
l Intel Performance Primitives:
http://www.intel.com/software/products/perflib/
l Halcon: http://www.mvtec.com/halcon/
l NeatVision: http://www.neatvision.com/

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Literature:

l Textbook:
– Gonzalez, Woods: Digital Image Processing. Prentice Hall.
2002.
l Slides: will be available in CAJ
– Please register with http://caj.informatik.uni-jena.de

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Is the human visual system perfect?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Is the human visual system perfect?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Is the human visual system perfect?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Is the human visual system perfect?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
A Puzzle: what can you see?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Local Context: now you see more from the image

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Global Context: you see other parts of the object

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
More Global Context: even more

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
More Global Context: and more …

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
The „Scene“: now you recognize „tigger“ easily

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Attention and Object Recognition

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Attention and Object Recognition

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Attention and Object Recognition

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Task Driven Attention

What time shows the clock on the


previous images?

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Task Driven Attention

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Image Comparison: find differences between images

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Image Comparison: easy job for the machine

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Variations in Images

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Noise

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Different Illumination

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Different Illumination

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Different Illumination

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Different Illumination

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Variation along a scan line

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
What does the machine already better?

l Exhausting tasks: counting parts, people, cars


l Measuring tasks: for example, difference between CAD
model and an individual manufactured part
l Tasks that need high speed and precission: for example,
postal address reading

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Accuracy: 3d Measuring

Ground truth distance between axes: 2,715 m


Reconstructed: 2,728 m
(Diploma Thesis Michael Trummer, 2004)
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Speed: Automatic Reading of Documents

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Speed: Automatic Reading of Documents

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Speed: Automatic Reading of Documents

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Speed: Automatic Reading of Documents

Letter recognition rate: 98.13%


Recognition rate for document: 92.16%
with 7.50% rejection rate (error rate: 0.34%)
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Exhausting: Licencs Plate Reading

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Exhausting: Licencs Plate Reading

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Medical Image Processing

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Introduction: Images

l What kind of images shall we expect?


– Gray value images
– Color images
– Multispectral images
– Ultrasonic images
– Depth images
l What is the „hardware“ behind the image formation
process?
– Gray value cameras
– Color cameras
– Satellites
– Sonar sensors
– Laser sensors
l We will in general deal with so called intensity images
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Images: Computer Science Perspective

l In computer science we have to work with digital images,


i.e. 2-D arrays or matrices, which represent the intensity
measured by the sensor
l Important 1: although we assume digital images for
most of the algorithms, for some tasks (for example, high
precise measurement) it is necessary to look at images
as continuous functions in 2D.
l Important 2: in addition, most algorithms can only be
completely understood, if we develop the methods
assuming 2D functions as input.

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Why is Computer Vision Difficult?

l The goal is: reconstruct properties of the 3D world based


on 2D projections, distortions by the optical system,
discretization, noise, and numerical problems during
processing the discrete data
l The information coded in the image can be
– Pure intensity
– Color value (either interpolated or real)
– Measured intensities in certain frequency ranges
l Consequence:
– To solve tasks in computer vision, we have to take into
consideration all the different aspects including the image
formation process
– Note: algorithms have to be developed such that they can
reconstruct necessary information using discrete representation
of the image with a finite, discrete domain of values

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
A digital image – Can you see anything?

15 15 14 15 16 16 16 16 16 16 17
14 15 15 14 15 16 15 16 16 16 19
14 15 15 15 15 16 16 16 16 17 26
14 15 15 17 20 23 22 19 18 21 34
14 15 18 24 34 42 42 33 25 30 51
14 15 19 32 42 53 55 49 36 36 57
15 15 20 33 39 50 66 71 53 38 59
15 17 23 35 38 49 69 78 56 36 58
15 18 33 51 50 53 71 79 57 35 56
15 23 41 61 57 53 69 78 56 35 56
15 24 47 69 64 55 67 77 57 36 56
14 21 39 61 58 50 59 67 52 33 49
14 17 24 35 35 31 34 39 33 23 27
13 14 19 23 22 21 21 23 21 18 19
13 14 16 17 16 16 16 16 16 16 16

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
A digital image – special „preprocessing“

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
The original image

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
The process of image formation

l Relevant parameters that influence image formation:


– Optical parameters of the sensor/lenses (type of the lense, focal
length, field of view) – you are the experts in this
– Photometric parameters, which describe how light reaches the
sensor after it has been reflected from object surfaces in the
scene (type, intensity and direction of illumination, reflectivity of
the surfaces, kind and structure of the sensor)
– Geometric parameters, which describe at which position on the
sensor the image of a 3D point of the scene will be projected
(projections models, pose and orientation of the camera with
respect to the scene, distortion models for lenses, etc.)
l These three parameters influence the image formation
process for any intensity image

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Further important properties

l Further properties play an important role during image


formation:
– The physical properties of the light sensitive matrix in the sensor
– The discrete nature of the photoreceptor
– The quantization which defines how many bits are used to
represent the continuous energy measured at each pixel position
l In the following we will ignore these properties which
need to be taken into consideration for measuring tasks

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sensors I

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sensors II

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
„Our“ Sensor

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Simple model for image generation

0 ≤ f (x, y) < ∞ l f(x,y): recorded image


f (x, y) = i(x, y)r(x, y) l i(x,y): illumination of the
surface point
0 < i(x, y) < ∞
l r(x,y): reflection at the
0 ≤ r(x, y) ≤ 1
surface point

l Values for i(x,y):


– 90000 lm/m2 (sunshine)
– 10000 lm/m2 (cloudy condition)
– 0.1 lm/m2 bei (full moon)
– 1000 lm/m2 (office)
l Values für r(x,y): 0.01 for black velvet, 0.93 for snow
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Important terms

l Although we measure a continuous function in 2D, we


can only represent discrete values in the computer
l Definition Sampling: The discretization of the
coordinate values of the continuous function f is called
sampling
l Definition Quanitzation: The discretization of the
amplitude values of the continuous function f is called
quantization
l Usually we do a normalization of the discrete amplitude
values such that the smallest value is equal to zero.

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sampling and Quantization

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sampling in Practice

l Sampling is usually give by the sensor itself


– A single sensor element which is moved can controll any
sampling rate (depending on the mechanics)
– Line sensors which are moved have a certain sampling rate
given by the sensor arranged in the line. Sampling perpendicular
to the linear sensor strip depends on the movement of the strip
– Sensor arrays (CCD-Chips) have a predefined sampling rate in
both directions of the 2D coordinate system
l After recording the image sampling rate can be artificially
changed:
– subsampling: resolution is reduced
– resampling: resolution is increased under certain assumptions
(one important area: subpixel accuracy or computation)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Result of Sampling and Quantization

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Representation of images in the computer

l In the computer images are usually stored as matrices or


arrays
– One element of the marix is called Pixel, Picture Element, Pel
– Dimension of the matrix in general MxN
– The coordinates (x,y) in general have no direct relations to the
position on the sensor element (CCD-Chip)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sensor elements and arrangement

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Quantization

l The continuous amplitude needs to be represented with


discrete values, too
l Typically we use L=2k quantization levels, i.e. the
continuous intensity is mapped to the range [0,L-1]
l Important terms:
– Dynamical range of an image: part out of [0,L-1], in which most
of the pixel values are located
– High contrast in an image: significant portions of the pixel values
are equally distributed over [0,L-1]
– Low contrast: Most pixel values are located in a small subset of
[0,L-1]

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Storage requirements

l Image size: NxN


l Quantization in L levels

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Spatial resolution and subsampling

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Spatial resolution and resampling

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Quantization and image quality/information

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Quantization and image quality/information

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Sub- und resampling in practice

l Both steps are applied to images in certain applications


l The goal is to increase (resampling, zooming) or reduce
(subsampling, shrinking) the spatial resolution
l There are two general steps necessary
1. Definition of a new pixel locations on a new M‘xN‘ grid
2. Computation of new intensity values for each position at the
new grid
l Depending on the distance between two grid elements
compared to the original image we get zooming or
shrinking

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Zoomin/Shrinking

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Zoomin/Shrinking

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Nearest Neighbor Interpolation

l Principle: use for the new grid position the intensity value
stored at the nearest neighbor on the original grid
l Advantage: very efficient
l Disadvantage: checker board effect

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Pixel replication

l Applicable if you want to increase the size of an image


by an integer number of times
l Principle: duplicate each row and column of the image
l Advantage: very fast and efficient
l Disadvantage: as a special case of nearest neigbor
interpolation you will also observe checker board effects,
especially for large magnifications

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Bilinear interpolation

l Principle: Approximate the function of intensities around


a pixel position by a bilinear funktion
l Procedure:
1. Search for the four neigbors of the pixel
2. Estimate a bilinear function given by the four neighboring
intensity values
3. Compute the new amplitude using the coordinates and the
estimated bilinear function
v ( xˆ, yˆ ) = axˆ + byˆ + cxˆyˆ + d
(0,0) (0,1)
v (0,0) = d v (1,0) = a + d
v (0,1) = b + d v (1,1) = a + b + c + d
(1,0) (1,1)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Subsamling

l Similar procedure as for resampling:


– Equivalent to pixel replication: remove lines and columns
– In practice: define new grid with distance between grid elements
larger than original grid; new grid results in lower resolution
– To reduce or eliminate aliasing effect: apply low-pass filter
before

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Resampling: Examples

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Relations between pixels I

l Neighborhood I: a pixel at position (x,y) in the image


has four neighbors in horizontal and vertical direction
– horizontal: (x+1,y), (x-1,y)
– vertical: (x,y+1), (x,y-1)
– These four neighbors are called 4-neighbors
– Distance between center pixel and neighbors is one
l Neighborhood II: a pixel at position (x,y) in the image
has four diagonal neighbors
– (x+1,y+1), (x+1,y-1), (x-1, y-1), (x-1,y+1)
– These four neighbors together with the 4-neighbors are called 8-
neighbors

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Relations between pixels II

l Adjacency: it is of enormous interest in computer vision


whether two pixels are connected or not
– A connection between pixels is the basis for the definition of a
contour
– Besides neighborhoods we also have to check certain conditions
(C) that must be fulfilled, usually given be intensity values at or
in a small area around the pixels
– 4-adjacent: a pixel is a 4-neighbor of another pixel and fulfills C
– 8-adjacent: a pixel is a 8-neighbor of another pixel and fulfills C
– mixed adjacency :
l Either: a pixel is in 4-neighbor of another pixel
l Or: the pixel is diagonal neighbor and the intersection of the set of
4-neigbors does not meet C

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Relations between pixels III

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Relations between pixels IV

l Path: A path between two pixels p and q is given by a


sequence of disjoint pixels with coordinates
(x0,y0)…(xn,yn) and
– p = (x0,y0) and q = (xn,yn)
– All pixels (xi,yi) and (xi+1,yi+1) are adjacent, for 0<=i<n
– The length of the path is n
– If p = q the path is called closed
l Region: a region is a set of adjacent pixels
l Contour: the contour of a region is a set of adjacent
pixels with the condition, that each pixel of the set has at
least one neighboring pixel that does not belong to the
region

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Distances

l Given three pixels p = (x,y), q = (s,t) und z = (v,w)


l A distance function D is called a metric if the following
conditions hold:
– D(p,q) >= 0 und D(p,q) = 0 iif p = q
– D(p,q) = D(q,p)
– D(p,z) <= D(p,q) + D (q,z)
l Special metrics on pixel grids
– Euclidean distance: De(p,q) = [(x-s)2+(y-t)2]1/2
– City-Block-Distance (Manhattan-Distance): D4 = |x-s|+|y-t|
– Checkerboard-Distance: D8 = max(|x-s|,|y-t|)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Excursion in probability theory

l Probability theory important for a lot of image processing


algorithms and strategies
l Terms:
– Relative frequency
– Discrete random variable (RV)
– Bayesian formula (or law)
– Probability density function
– Cumulative density function
– Properties of RV
– Normal distribution

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Relative frequency and probabiliy

l Given an experiment, where a coin is flipped n times and


we get results for heads (nH) and tails (nT)
l Obviously we will have :
l Deviding both sides by n we get

l The fraction on the left side are called relative frequency


for heads and tails
l For large number of n we call the fraction probability P
for the outcome E (head or tail) mit 0<=P(E)<=1 und
P(H)+P(Z)=1

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Joint Event

l We observe two events A and B during an experiment of


length n. The following combinations of outcomes are
possible:
– A occurs (n1 times) or
– B occurs (n2 times) or
– A and B occurs (n3 times) or
– Neither A nor B occurs (n4 times)
and as a consequence n1+n2+n3+n4 = n

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Joint Event

l The event that either A or B or both events are observed


is directly given by:

l This results in the important rule:

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Conditional Probability

l The relative frequency that A is observed if B happens is


given by (we say: with probability P(A|B))

l Similar to that we can define P(B|A)

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Bayesian Law and Statistical Independency

l We now get

l and it follows that

l Two events are called statistically independent, if

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Random Variables

l We extend discrete events to a continuous space where


we associate events with outcomes of an experiment
that lies in a certain continuous interval (in contrast to
the discrete nature of a coin flipping experiment)
l The outcome of an experiment is represented by a so
called random variable (RV)
– We might also talk about RV in the discrete case
l Main difference between discrete and continuous RV:
– The probability for a certain value of a continuous RV is equal to
zero

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Cumulative Probability Function

l The cumulative probability function F(a) of a continuous


RV a is defined as

l Properties:

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Probability Density Function

l The probability density function (short: density, pdf – do


not mix up this term with the term density from physics) p
of a random variable x is defined as

l Properties of a pdf:

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Transformations of RV

l Let y=T(x) be a transformation of a random variable x


with T being monotonical decreasing
l For the densities of the two RV x and y we get the
following important relation

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Attributes of random variables I

l Expected value of a RV

l Expected value of a transformed RV:

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Attributes of random variables II

l Variance of a RV:

l Example of a pdf of a continuous RV

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Optimal Quantization

a1 a2 a3 aL aL+1
fj

b1 b2 bL-1 bL fj‘

l Goal: Quantization which minimizes the so called


quantization error
l Approach (see figure): all continuous values of the
function f with ai <= fj < ai+1 are mapped to discrete values
fj‘ = bi
l We do not consider other criterions of quality (e.g. visual
quality, compression, etc.)
Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Results: Optimal Quantization

L an +1
e =å (f - b )
ò n p(f )df ,
2
p(f = an ) ¹ 0
n =1 an

an +1

¶e
an +1 !
ò fp(f )df
ò - 2(f - bn )p(f )df = 0 Þ bn =
an
=
¶bn an +1

ò p(f )df
an

an

n = 1...L
¶e !
bn -1 + bn
= (an - bn -1 ) p(an ) - (an - bn )
2 2
p(an ) = 0 Þ an =
¶an 2
n = 2...L, a1 = fmin , aL +1 = fmax

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Example: low contrast

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Histogram

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
Result after requantization

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena
New Histogramm

Copyright © 2017 Joachim Denzler, Chair for Computer Vision, Uni Jena

S-ar putea să vă placă și