Basics of digital image processing

Lecture 4
September 23, 2006
 What is image processing
„ Is enhancing an image or extracting
information or features from an image
„ Computerized routines for information
extraction (eg, pattern recognition,
classification) from remotely sensed
images to obtain categories of information
about specific features.
„ Many more
 Image Processing Includes
„ Image quality and statistical evaluation
„ Radiometric correction
„ Geometric correction
„ Image enhancement and sharpening
„ Image classification
„ Pixel based
„ Object-oriented based
„ Accuracy assessment of classification
„ Post-classification and GIS
„ Change detection
 Image Quality
„ Many remote sensing datasets contain high-quality,
accurate data. Unfortunately, sometimes error (or
noise) is introduced into the remote sensor data by:
„ the environment (e.g., atmospheric scattering,
cloud),
„ random or systematic malfunction of the remote
sensing system (e.g., an uncalibrated detector
creates striping), or
„ improper pre-processing of the remote sensor
data prior to actual data analysis (e.g., inaccurate
analog-to-digital conversion).
154 155

Cloud

155

160
162
MODIS
True
143

163
164
Clouds in ETM+
Striping Noise and Removal

CPCA

Combined Principle
Component Analysis

Xie et al. 2004

 Speckle Noise and
Removal
Blurred objects
and boundary

G-MAP
Gamma Maximum
A Posteriori Filter
 Univariate descriptive image statistics

„ The mode is the value that

occurs most frequently in a
distribution and is usually the
highest point on the curve
(histogram). It is common,
however, to encounter more
than one mode in a remote
sensing dataset.
„ The median is the value midway
in the frequency distribution.
One-half of the area below the
distribution curve is to the right
of the median, and one-half is to
the left
„ The mean is the arithmetic
average and is defined as the n
sum of all brightness value
observations divided by the ∑ BV ik
number of observations.
µk = i =1
n
 Cont’
n

∑ (BV − µk )
2
„ Min ik

„ Max vark = i =1
n −1
„ Variance
„ Standard deviation
„ Coefficient of
variation (CV)
sk = σ k = vark
„ Skewness
„ Kurtosis σ
CV = k
„ Moment µ k
 Multivariate Image Statistics

„ Remote sensing research is often concerned

with the measurement of how much radiant
flux is reflected or emitted from an object in
more than one band. It is useful to compute
multivariate statistical measures such as
covariance and correlation among the several
bands to determine how the measurements
covary. Variance–covariance and correlation
matrices are used in remote sensing principal
components analysis (PCA), feature
selection, classification and accuracy
assessment.
 Covariance
„ The different remote-sensing-derived spectral measurements
for each pixel often change together in some predictable
fashion. If there is no relationship between the brightness
value in one band and that of another for a given pixel, the
values are mutually independent; that is, an increase or
decrease in one band’s brightness value is not accompanied
by a predictable change in another band’s brightness value.
Because spectral measurements of individual pixels may not
be independent, some measure of their mutual interaction is
needed. This measure, called the covariance, is the joint
variation of two variables about their common mean.
n n

∑ BV ∑ BV SPkl
cov kl =
n ik il
SPkl = ∑ (BVik ×BVil ) − i =1 i =1

i =1 n n −1
 Correlation
To
Toestimate
estimatethe thedegree
degreeof ofinterrelation
interrelationbetween
betweenvariables
variablesininaamanner
mannernotnot
influenced
influencedby bymeasurement
measurementunits, units,the
thecorrelation
correlationcoefficient,
coefficient,isis
commonly
commonlyused. used.The
Thecorrelation
correlationbetween
betweentwo
twobandsbandsofofremotely
remotelysensed
sensed
data,
data,rrklkl,,isisthe
theratio
ratioof
oftheir
theircovariance
covariance(cov
(covklkl))to
tothe
theproduct
productofoftheir
their
standard
standarddeviations
deviations(s(skkssl); thus:
l); thus:

cov kl
rkl =
sk sl
IfIf we
we square
square the
the correlation
correlation coefficient
coefficient (r(rklkl),), we
we obtain
obtain the
the sample
sample coefficient
coefficient ofof
determination 2
determination(r(r2),),which
whichexpresses
expressesthe
theproportion
proportionof ofthe
thetotal
totalvariation
variationininthethevalues
valuesof of
“band
“bandl”l”that
thatcan
canbebeaccounted
accountedforforor
orexplained
explainedby byaalinear
linearrelationship
relationshipwith withthe
thevalues
values
of
ofthe
therandom
randomvariable
variable“band
“bandk.”
k.”Thus
Thusaacorrelation coefficient(r(rklkl))of
correlationcoefficient of0.70
0.70results
resultsininan
an
2
rr2value
valueof
of0.49,
0.49,meaning
meaningthat
that49%
49%of ofthe
thetotal
totalvariation
variationofofthe
thevalues
valuesof of“band
“bandl”l”ininthe
the
sample
sampleisisaccounted
accountedforforby
byaalinear
linearrelationship
relationshipwith withvalues
valuesofof“band
“bandk”. k”.
Pixel Band 1
(green)
Band 2
(red)
Band 3
(ni)
Band 4
(ni) example
(1,1) 130 57 180 205
(1,2) 165 35 215 255
(1,3) 100 25 135 195
SP12 = (31,860) −
(675)(232 )
(1,4) 135 50 200 220
5
(1,5) 145 65 205 235
540
cov12 = = 135
4
Band 1 (Band 1 x Band Band 2
2)
130 7,410 57
165 5,775 35
100 2,500 25
135 6,750 50
145 9,425 65
675 31,860 232
 Band 1 Band 2 Band 3 Band 4
Mean (µk) 135 46.40 187 222
Variance (vark) 562.50 264.80 1007 570
(s k ) 23.71 16.27 31.4 23.87
(mink) 100 25 135 195
(maxk) 165 65 215 255
Range (BVr) 65 40 80 60
Univariate statistics

Band 1 Band 2 Band 3 Band 4 Band Band Band 3 Band

1 2 4
Band 1 562.25 - - -
Band 1 - - - -
Band 2 135 264.8 - -
0 Band 2 0.35 - - -
Band 3 718.75 275.25 1007.5 -
Band 3 0.95 0.53 covariance
- -
0
Band 4 537.50 64 663.75 570 Band 4 0.94 0.16 0.87 -

Covariance Correlation coefficient

 Types of radiometric correction

„ Detector error or sensor error (internal

error)
„ Atmospheric error (external error)
„ Topographic error (external error)
 Atmospheric correction
Various Paths of
Satellite Received Radiance

„ There are several ways to Remote

atmospherically correct sensor
remotely sensed data. Total radiance L
detector

Some are relatively at the sensor S

straightforward while Solar
others are complex,
E
irradiance 0

being founded on 90Þ Lp LT

physical principles and
requiring a significant
amount of information to Tθ
60 miles
function properly. This or
0
Tθ
discussion will focus on 2 v 100km
two major types of Diffuse sky
atmospheric correction: irradiance Ed 1
4
1,3,5 Atmosphere

θv
„ Absolute atmospheric 3
θ0
correction, and LI
„ Relative atmospheric
correction. 5

Scattering, Absorption Reflectance from Reflectance from

Refraction, Reflection neighboring area, study area,
r rλ
λn
 Absolute atmospheric correction

„ Solar radiation is largely unaffected as it travels through the

vacuum of space. When it interacts with the Earth’s atmosphere,
however, it is selectively scattered and absorbed. The sum of
these two forms of energy loss is called atmospheric attenuation.
Atmospheric attenuation may 1) make it difficult to relate hand-
held in situ spectroradiometer measurements with remote
measurements, 2) make it difficult to extend spectral signatures
through space and time, and (3) have an impact on classification
accuracy within a scene if atmospheric attenuation varies
significantly throughout the image.

„ The general goal of absolute radiometric correction is to turn

the digital brightness values (or DN) recorded by a remote sensing
system into scaled surface reflectance values. These values can
then be compared or used in conjunction with scaled surface
reflectance values obtained anywhere else on the planet.
a)a)Image
Imagecontaining
containingsubstantial
substantialhaze
hazeprior
priortotoatmospheric
atmosphericcorrection.
correction.b)b)Image
Imageafter
after
atmospheric
atmosphericcorrection
correctionusing
usingATCOR
ATCOR(Courtesy
(CourtesyLeica
LeicaGeosystems
Geosystemsand andDLR,
DLR,the
the
German
GermanAerospace
AerospaceCentre).
Centre).
 relative radiometric correction

„ When required data is not available for

absolute radiometric correction, we can
do relative radiometric correction
„ Relative radiometric correction may be
used to
„ Single-image normalization using histogram
adjustment
„ Multiple-data image normalization using
regression
 Single-image normalization using
histogram adjustment

„ The method is based on the fact that infrared

data (>0.7 µm) is free of atmospheric
scattering effects, whereas the visible region
(0.4-0.7 µm) is strongly influenced by them.
„ Use Dark Subtract to apply atmospheric
scattering corrections to the image data. The
digital number to subtract from each band
can be either the band minimum, an average
based upon a user defined region of interest,
or a specific value
Dark Subtract using band minimum
 Topographic correction
„ Topographic slope and aspect also introduce
radiometric distortion (for example, areas in
shadow)
„ The goal of a slope-aspect correction is to
remove topographically induced illumination
variation so that two objects having the same
reflectance properties show the same
brightness value (or DN) in the image despite
their different orientation to the Sun’s position
„ Based on DEM, sun-elevation
 Conceptions of geometric correction

„ Geocoding: geographical referencing

„ Registration: geographically or nongeographically (no coordination system)
„ Image to Map (or Ground Geocorrection)
The correction of digital images to ground coordinates using ground control
points collected from maps (Topographic map, DLG) or ground GPS points.
„ Image to Image Geocorrection
Image to Image correction involves matching the coordinate systems or
column and row systems of two digital images with one image acting as a
reference image and the other as the image to be rectified.
„ Spatial interpolation: from input position to output position or coordinates.
„ RST (rotation, scale, and transformation), Polynomial, Triangulation
„ Root Mean Square Error (RMS): The RMS is the error term used to
determine the accuracy of the transformation from one system to another. It is
the difference between the desired output coordinate for a GCP and the actual.
„ Intensity (or pixel value) interpolation (also called resampling): The process of
extrapolating data values to a new grid, and is the step in rectifying an image that
calculates pixel values for the rectified grid from the original data grid.
„ Nearest neighbor, Bilinear, Cubic
 Image enhancement
„ image reduction,
„ image magnification,
„ transect extraction,
„ contrast adjustments (linear and non-linear),
„ band ratioing,
„ spatial filtering,
„ fourier transformations,
„ principle components analysis,
„ texture transformations, and
„ image sharpening
 Contrast Enhancement (stretch)

„ Materials or objects reflect or emit similar amounts of radiant flux (so

similar pixel value)
„ Low-contrast imagery with pixel range less than the designed
radiometric range
„ 20-100 for TM less than the designed 0-255
„ To improve the contrast:
„ Linear technique
„ Minimum-maximum contrast stretch
„ Percentage linear contrast stretch
„ Standard deviation contrast stretch
„ Piecewise linear contrast stretch
„ Non-linear technique
„ Histogram equalization
„ Contrast enhancement is only intended to improve the visual quality
of a displayed image by increasing the range (spreading or
stretching) of data values to occupy the available image display range
(usually 0-255). It does not change the pixel values, unless save it as
a new image. It is not good practice to use saved image for
classification and change detection.
 Minimum-maximum contrast
stretch

⎛ BVin − min k ⎞
BVout = ⎜⎜ ⎟⎟quant k
⎝ max k − min k ⎠
where:
where:
--BV
BVininisisthe
theoriginal
originalinput
inputbrightness
brightnessvalue
value
--quant
quantkkisisthe
therange
rangeofofthe
thebrightness
brightnessvalues
valuesthat
thatcan
canbe
be
displayed
displayedon onthe
theCRT
CRT(e.g.,
(e.g.,255),
255),
--min
minkkisisthe
theminimum
minimumvaluevaluein
inthe
theimage,
image,
--max
maxkkisisthe
themaximum
maximumvaluevaluein
inthe
theimage,
image,and
and
--BV
BVout is the output brightness value
out is the output brightness value
 Percentage linear and
standard deviation contrast
stretch
„ X percentage (say 5%) top or low values of the image
will be set to 0 or 255, rest of values will be linearly
stretched to 0 to 255
„ ENVI has a default of a 2% linear stretch applied to each
image band, meaning the bottom and top 2% of image
values are excluded by positioning the range bars at the
appropriate points. Low 2% and top 2% will be saturated
to 0 and 255, respectively. The values between the range
bars are then stretched linearly between 0 and 255
resulting in a new image.
„ If the percentage coincides with a standard deviation
percentage, then it is called a standard deviation contrast
stretch. For a normal distribution, 68%, 95.4%, 99.73%
values lie in ±1σ, ±2 σ, ±3 σ. So 16% linear contrast
stretch is the ±1σ contrast stretch.
 original Saturating the water Saturating the land
Stretching the land Stretching the water

Special linear contrast stretch

Or Stretch on demand
 Piecewise linear contrast stretch

„ When the histogram of an image is not

Gaussian (bimodal, trimodal, …), it is
possible to apply a piecewise linear contrast
stretch.
„ But you better to know what each mode in
the histogram represents in the real world.
 Stretch both
land and water
 Principle Components Analysis (PCA)
„ There are large correlations among remote sensing bands. PCA will result
in another uncorrelated datasets: principal component images (PCs). PC1
contains the largest variance
„ The first two or three components (PCs) contain over 90% of information
from the original many bands. It is a great compress operation
„ The new principal component images that may be more interpretable than
the original data.
Purposes of image classification

Land use and land cover (LULC)

Vegetation types
Geologic terrains
Mineral exploration
Alteration mapping
…….
 What is image classification or
pattern recognition

„ Is a process of classifying multispectral (hyperspectral) images into

patterns of varying gray or assigned colors that represent either
„ clusters of statistically different sets of multiband data, some of which
can be correlated with separable classes/features/materials. This is the
result of Unsupervised Classification, or
„ numerical discriminators composed of these sets of data that have been
grouped and specified by associating each with a particular class, etc.
whose identity is known independently and which has representative
areas (training sites) within the image where that class is located. This is
the result of Supervised Classification.
„ Spectral classes are those that are inherent in the remote sensor
data and must be identified and then labeled by the analyst.

„ Information classes are those that human beings define.

unsupervised classification, The
computer or algorithm automatically
group pixels with similar spectral
characteristics (means, standard
deviations, covariance matrices,
correlation matrices, etc.) into unique
clusters according to some statistically
determined criteria. The analyst then
re-labels and combines the spectral
clusters into information classes.
supervised classification. Identify known a priori
through a combination of fieldwork, map
analysis, and personal experience as training
sites; the spectral characteristics of these sites are
used to train the classification algorithm for
eventual land-cover mapping of the remainder of
the image. Every pixel both within and outside the
training sites is then evaluated and assigned to the
class of which it has the highest likelihood of
being a member.
 Hard vs. Fuzzy classification
„ Supervised and unsupervised classification
algorithms typically use hard classification logic
to produce a classification map that consists of
hard, discrete categories (e.g., forest,
agriculture).

„ Conversely, it is also possible to use fuzzy set

classification logic, which takes into account the
heterogeneous and imprecise nature (mix
pixels) of the real world. Proportion of the m
classes within a pixel (e.g., 10% bare soil, 10%
shrub, 80% forest). Fuzzy classification
schemes are not currently standardized.
 Pixel-based vs. Object-oriented
classification
„ In the past, most digital image classification was based on
processing the entire scene pixel by pixel. This is commonly
referred to as per-pixel (pixel-based) classification.

„ Object-oriented classification techniques allow the

analyst to decompose the scene into many relatively
homogenous image objects (referred to as patches or
segments) using a multi-resolution image segmentation
process. The various statistical characteristics of these
homogeneous image objects in the scene are then subjected
to traditional statistical or fuzzy logic classification. Object-
oriented classification based on image segmentation is often
used for the analysis of high-spatial-resolution imagery (e.g.,
1 × 1 m Space Imaging IKONOS and 0.61 × 0.61 m Digital
Globe QuickBird).
 Unsupervised classification
„ Uses statistical techniques to group n-dimensional data into their natural
spectral clusters, and uses the iterative procedures
„ label certain clusters as specific information classes
„ K-mean and ISODATA
„ For the first iteration arbitrary starting values (i.e., the cluster properties)
have to be selected. These initial values can influence the outcome of the
classification.
„ In general, both methods assign first arbitrary initial cluster values. The
second step classifies each pixel to the closest cluster. In the third step the
new cluster mean vectors are calculated based on all the pixels in one
cluster. The second and third steps are repeated until the "change" between
the iteration is small. The "change" can be defined in several different ways,
either by measuring the distances of the mean cluster vector have changed
from one iteration to another or by the percentage of pixels that have
changed between iterations.
„ The ISODATA algorithm has some further refinements by splitting and
merging of clusters. Clusters are merged if either the number of members
(pixel) in a cluster is less than a certain threshold or if the centers of two
clusters are closer than a certain threshold. Clusters are split into two
different clusters if the cluster standard deviation exceeds a predefined value
and the number of members (pixels) is twice the threshold for the minimum
number of members.
 Supervised classification:
training sites selection
„ Based on known a priori through a combination of fieldwork,
map analysis, and personal experience

„ on-screen selection of polygonal training data (ROI), and/or

„ on-screen seeding of training data (ENVI does not have

this, Erdas Imagine does).
„ The seed program begins at a single x, y location and evaluates
neighboring pixel values in all bands of interest. Using criteria
specified by the analyst, the seed algorithm expands outward like
an amoeba as long as it finds pixels with spectral characteristics
similar to the original seed pixel. This is a very effective way of
collecting homogeneous training information.

„ From spectral library of field measurements

Selecting
ROIs

Alfalfa

Cotton

Grass

Fallow
 Supervised classification methods
„ Various supervised classification algorithms may be used to assign an unknown pixel to one
of m possible classes. The choice of a particular classifier or decision rule depends on the
nature of the input data and the desired output. Parametric classification algorithms
assumes that the observed measurement vectors Xc obtained for each class in each spectral
band during the training phase of the supervised classification are Gaussian; that is, they are
normally distributed. Nonparametric classification algorithms make no such assumption.

„ Several widely adopted nonparametric classification algorithms include:

„ one-dimensional density slicing
„ parallepiped,
„ minimum distance,
„ nearest-neighbor, and
„ neural network and expert system analysis.

„ The most widely adopted parametric classification algorithms is the:

„ maximum likelihood.

„ Hyperspectral classification methods

„ Binary Encoding
„ Spectral Angle Mapper
„ Matched Filtering
„ Spectral Feature Fitting
„ Linear Spectral Unmixing
Supervised
classification
method:

Spectral Feature
Fitting

Source: http://popo.jpl.nasa
.gov/html/data.html
 Accuracy assessment of classification

„ Remote sensing-derived thematic information are

becoming increasingly important. Unfortunately, they
contain errors.
„ Errors come from 5 sources:
„ Geometric error still there
„ None of atmospheric correction is perfect
„ Clusters incorrectly labeled after unsupervised classification
„ Training sites incorrectly labeled before supervised
classification
„ None of classification method is perfect
„ We should identify the sources of the error, minimize it,
do accuracy assessment, create metadata before being
used in scientific investigations and policy decisions.
„ We usually need GIS layers to assist our classification.
 training vs. ground reference
„ Several ways to do error evaluation
„ Based on training pixels (areas)
„ The problem is that the locations of training sites are usually not
random. They are biased by analyst’s a priori knowledge of
where certain LULC types exist in the scene.
„ This will results in higher classification accuracies than the one
below
„ Based on ground reference pixels
„ These sites are not used to train the classification algorithm and
therefore represent unbiased reference information
„ It is possible to collect some ground sites prior to the
classification, perhaps at the same time as the training data
„ But majority of test reference is often collected after
classification.
„ Landscape often change rapidly. Therefore, it is best to
collect both the training and ground reference as close
to the data of remote sensing data acquisition as
possible. (for example, agriculture crops change fast)
 Error (Confusion) Matrix
„ Producer (analyst) accuracy is a measure indicating the probability that
the classifier has labeled an image pixel into Class A given that the
ground truth is Class A. it is the probability of a reference pixel being
correctly classified.
„ Omission error represent pixels that belong to the ground truth class but
that the classification technique has failed to classify them into the
proper class.
„ User accuracy is a measure indicating the probability that a pixel is Class
A given that the classifier has labeled the pixel into Class A. it is the
probability that a pixel classified on the map actually represents that
category on the ground.
„ Commission error represent pixels that belong to another class but are
labeled as belonging to the class.
„ Overall accuracy is total classification accuracy.
„ Kappa coefficient (Khat) is a discrete multivariate technique of use in
accuracy assessment. Khat>80% represent strong agreement and good
accuracy. 40%-80% is middle, <40% is poor.
Example: they took 407 samples
(pixels) based on the stratified random
sampling after classification. First
made 5 files (each contain one class),
using a random number generator to
get points.
Post-classification and GIS

salt-
and-
pepper
 types

„ Majority/Minority Analysis
„ Clump Classes
„ Morphology Filters
„ Sieve Classes
„ Combine Classes
„ Classification to vector (GIS)
 Change detection
„ Change detect involves the use of multi-temporal datasets to
discriminate areas of land cover change between dates of imaging.
„ Ideally, it requires
„ Same or similar sensor, resolution, viewing geometry, spectral bands,
radiomatric resolution, acquisition time of data, and anniversary dates
„ Accurate spatial registration (less than 0.5 pixel error)
„ Methods
„ Independently classified and registered, then compare them
„ Classification of combined multi-temporal datasets,
„ Principal components analysis of combined multi-temporal datasets
„ Image differencing (subtracting), (needs to find change/no change threshold,
change area will be in the tails of the histogram distribution)
„ Image ratioing (dividing), (needs to find change/no change threshold,
change area will be in the tails of the histogram distribution)
„ Change vector analysis
„ Delta transformation
Example: stages of development
 Sun
Sun City
City ––
Hilton
Hilton Head
Head

1994
1994

1996
1996
 1974

1,040 urban
hectares

1994

3,263 urban
hectares

315%
increase