Documente Academic
Documente Profesional
Documente Cultură
Contents
1 Hypertools toolbox 1
2 Installing hypertools 2
3 Data handling 2
3.1 Data handling in hypertools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.2 Spectral images in PRTools dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.3 Conversion between DIP image and dataset representation . . . . . . . . . . . . . . . . . . 4
3.4 Importing binary BioRad FTIR data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
5 Preprocessing 9
5.1 Baseline subtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
5.2 Smoothing of spectral data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5.3 Unmixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6 Dissimilarity measures 12
6.1 Dissimilarity measures implemented in hypertools . . . . . . . . . . . . . . . . . . . . . . . 12
6.2 Visualization using dissimilarity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
6.3 Building dissimilarity representation for pattern recognition . . . . . . . . . . . . . . . . . 13
8 Image segmentation 20
8.1 Segmentation combining spatial and spectral domain . . . . . . . . . . . . . . . . . . . . . 20
8.2 ECHO segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
9 Classifiers 22
1 Hypertools toolbox
Hypertools is A Matlab toolbox for analysis of hyperspectral images. It contains algorithms for visualiza-
tion, preprocessing, representation and classification of spectral data. This toolbox is being developed at
TU Delft in The Netherlands within the Hyperspectral Image Analysis project, sponsored by the Dutch
technology foundation STW. Hypertools is available under academic or commercial license. Although we
are especially targeting spectral images, number of routines may be used also for generic spectral datasets
1
without spectral. It is based on the PRTools toolbox version 4. This document briefly describes how to
use hypertools for analyzing of spectral images.
2 Installing hypertools
Hypertools require Matlab version 6.1 or higher and PRTools version 4.x. Many routines is hypertools
require also DIPimage version 1.4 or higher. In order to install hypertools, extract the archive into a
directory and add its path into the Matlab environment.
3 Data handling
3.1 Data handling in hypertools
Hyperspectral images consist of spectral measurements organized in a spatial setup:
In hypertools, image cubes may be stored either in PRTools dataset or in a dip_image. It depends
on actual type of processing we want to execute on the data cube which data representation is better
at the moment. In case of extensive filtering or 2D-connected processing, dip_image is a good choice.
For pattern recognition tasks, such as clustering, image segmentation, feature extraction or classification,
dataset representation offers far more flexibility. Because of availability of additional meta information,
hypertools uses PRTools dataset as the primary data representation.
In this tutorial, we illustrate various data analysis approaches on a spectral image from a plastic sorting
application (NIR spectra). The image depicts four types of plastics and a background class. Through
the text, the image a1 is used for training and a2 as a test set.
2
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> a1.objsize
ans =
33 40
>> a1.featsize
ans =
240
optional labels per spectrum in nlab. Labelling of spectral images may be visualized as an image
using
getli (get label image) function:
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> getli(a1)
Displayed in figure 3
unique identifier per spectrum (pixel) in ident. Identifiers are useful to back trace what pixels are
present in a subset of the original spectral image.
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
3
Figure: Highlighting the prototypes
version info capturing the PRTools version, used for dataset generation and date of dataset cre-
ation. Dataset name may closely identify the dataset content or project name:
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> a1.version1
ans =
>> a1.version2
ans =
22-Sep-2003 15:33:40
>> a1.name=projectA
projectA, 1320 by 240 dataset with 5 classes: [372 360 258 161 169]
additional info in the user structure. Additional information, such as spectral range, units, or
spectral imaging technique description, may be stored in the user field.
4
>> fim=ftir_load(a1.dat,910.399010, 15.43049)
4096 by 512 dataset with 0 classes: []
>> fim.user
ans =
type: spectra
format: FTIR converted from bio-rad
mode: transmission
units: cm^-1
start_wavelength: 910.3990
step: 15.4305
>> plots(gendat(fim,10))
>> a1
projectA, 1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> sih=showsi(a1);
5
Figure: Visualizing the spectral image using showsi
Moving a mouse pointer over the spatial image, a spectrum at the current point is visible in the spectral
plot. When a left mouse button is clicked in the spectral plot, a spectral wavelength may be selected by
dragging the mouse over the plot. The spatial plot is updated accordingly.
By clicking over the spatial image, points may be selected. Corresponding spectra are also plotted in
the spectral pane. Three buttons in the spectral figure allow to choose three different colors or classes.
Right mouse click in the spatial window cancels the point selection.
6
Figure: Highlighting the points of interest
Selected points may be retrieved from the spectral image view using the spectral image handle sih
and used e.g. as prototypes. A dissimilarity may be computed from all the data points to the selected
prototypes:
>> proto=si_get_spectra(sih)
5 by 240 dataset with 3 classes: [2 2 1]
>> D=dasam(a,proto)
5280 by 5 dataset with 5 classes: [1461 1426 1025 664 704]
>> fig=scatterdui(D);
% fig is a figure handle of the scatter plot which we will use later.
7
Figure: Scatter plot of a dissimilarity space representation
Buttons along each scatter axis allow us to step easily between different feature space dimensions. By
clicking at the data points, sample index is shown nearby. This enables us to find back a sample or a
pixel from a feature space.
In order to see the correspondence between different representations of spectral data, scatter plots may
be attached to the spectral image. We need the handles of both spectral image (sih) and of the scatter
plot (fig):
>> si_attach_display(sih,fig)
Now, we may move the mouse over the spatial image and observe the corresponding data sample in
the feature space, denoted by the yellow circle.
8
5 Preprocessing
5.1 Baseline subtraction
Baseline may be subtracted from a dataset using a
basesubm mapping. First, a single spectrum must be identified which will be used for baseline correc-
tion and then the baseline region must be identified. This may be achieved using an interactive showsi
tool, included in hypertools:
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
9
Figure: Spectral image with subtracted baseline
The baseline subtraction mapping reduced the dimensionality of our dataset from original 240 to 179
wavelengths. Please note the zero values in the tails of corrected spectra resulting from the default
clipping.
>> w=smoothm(sg,7,2)
Spectral smoothing mapping (Savitsky-Golay ws=7,p=2,d=0), fixed mapping --> smoothm
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> b=gendat(a1,1)
1 by 240 dataset with 5 classes: [0 1 0 0 0]
>> plots(b)
% smoothing
>> w=smoothm(sg,7,2)
Spectral smoothing mapping (Savitsky-Golay ws=7,p=2,d=0), fixed mapping --> smoothm
10
Figure: Smoothing spectra. Original spectrum (blue) and the smoothed spectrum (red)
5.3 Unmixing
Other names for unmixing are blind source separation (in signal processing), multy-curve resolution (in
chemometrics) or factor analysis. The goal of unmixing is to represent dataset as a product of two
matrices: concentrations (scores) and spectra (loadings) of pure components. For spectral data we can
make use of the nonegativity of both matrices (scores and loadings). Unmixing routines which are
implemented in hypertools:
varimax : VARIMAX, given the loadings found by PCA, tries to find the rotation after which they
look as sparse as possible, i.e. it assumes that pure spectra consist of the number of compact peaks.
Varimax is provided in two versions,
varimaxfm performing feature selection and
varimaxom implementing object selection.
opa : OPA (orthogonal projection approach) looks for the set of the most dissimilar (orthogonal)
spectra
opam implements selection of most orthogonal features
opaps selection of the most orthogonal prototypes (examples).
simplisma : is similar to the OPA but also takes into account the purity of the candidate spectra
(the pure spectrum is supposed to have large variance)
simplismam implements selection of most pure features
simplismaps selection of the most pure prototypes (examples).
als : (alterating least squares) is the last step in unmixing procedure. Taking as an input data
and candidate loadings (pure spectra) found by previous routines, it decomposes data into positive
concentration and spectra matrices.
todo:unmixing examples
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
11
>> xOPAOptions.maxsn = inf;
>> OPAOptions.eps = 0.05;
>> OPAOptions.include_mean = 0;
>> OPAOptions.verbose = 0;
>> [Y, Y_ind, dis_max, dis] = opa(X,OPAOptions);
>> ALSOptions.mode = samples; % samples | features
>> ALSOptions.maxiter = 100;
>> ALSOptions.crit = rec; % conv | rec
>> ALSOptions.eps = 1e-2;
>> ALSOptions.verbose = 0;
>> [Zp,Yp] = als(X,Y,ALSOptions);
6 Dissimilarity measures
Dissimilarity measures define the scalar-valued measurement of a dissimilarity between two spectra. The
dissimilarity values may be used for visualization or for building dissimilarity representation for pattern
recognition.
12
6.2 Visualization using dissimilarity measures
Dissimilarities may be computed from a complete spectral image to a set of prototype spectra. In the
following example, we choose randomly a set of prototypes from a labelled spectral image and compute
Spectral Angle Mapper dissimilarity between all image spectral and these prototypes. The result of
dissimilarity computation is a dataset with 15 features (each measuring dissimilarity to a corresponding
prototype). Because the dataset originates from a spectral image, it may be converted into dip_image
object and visualized.
>> a1
projectA, 1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> proto=gendat(a1,0.01)
projectA, 15 by 240 dataset with 5 classes: [4 4 3 2 2]
The image contains 15 bands, (features in a dataset) rendering dissimilarities to the 15 prototypes.
By pressing n or p keys, we can move back and forth (dip_image feature). The following Figure shows
the dissimilarity to the 5th prototype (indices in DIP image package are zero-based).
13
% compute mean class spectra - use them as class prototypes
>> m=meancov(a1)
5 by 240 dataset with 5 classes: [1 1 1 1 1]
% execute the trained mapping on the test set and get the average class error:
>> dts*w*testc
ans =
0.0992
Prototypes (representation set) may be also selected randomly or via the interactive tool, as shown
above.
14
applied to a training dataset, results in a set of non-overlapping wavelength groups and corresponding
Fisher projection mappings. Applied to a test set, a new feature space is generated by projecting each
wavelength group into the 1D Fisher space. In order to mitigate the influence of non-informative data, the
authors recommend to run a subsequent feature selection or extraction algorithm after applying GLDB
method.
gldbm routine implements the bottom-up two-class GLDB feature extractor as proposed by Kumar
et.al.. Selected wavelength groups may be visualized by
plot_gldb_groups function.
As you can see, the GLDB feature extraction algorithm decomposes a complete spectral range into
a set of wavelength groups. Many of these are probably not adding any discriminatory information in
the classification problem at hand. In order to identify the informative groups, a second stage feature
selection may be carried on.
In the following example, we select the best features generated by the GLDB extraction using a
sequential forward selection procedure. Because GLDB is effectively both feature selection (groups of
adjacent wavelengths) and feature extraction (within each group), the feature selection result may be
combined with the trained GLDB mapping leading to a reduced GLDB mapping:
15
% select the best subset of GLDB features in the dataset c
>> wfsel=featself_simple(c1)
Forward Feature Selection, 50 to 1 fixed mapping --> cmapm
% let us now derive a reduced GLDB mapping retaining only wavelength groups,
% selected by the feature selection
>> wnew=gldbm_featsel(w,wfsel)
240 to 1 trained mapping --> gldbm
% we can look into the wnew mapping to see only the group of wavelengths 87 to 170
% is used in this reduced GLDB mapping:
>> getdata(wnew)
clf: [1x1 struct]
l: 87
u: 170
Hypertools provides also impementation of the top-down GLDB algorithm using the Log-odds probability-
based criterion ( gldbm_td_prob ) as proposed by the GLDB authors. We have also implemented the
top-down GLDB using the apparent error criterion ( gldbm_td_ae ) and using the combined Fisher
separability and correlation criterion used also in the bottom-up case ( gldbm_td ).
% train the multi-class GLDB extraction on a training set a1 with five classes:
>> a1
1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> w=gldbm_multi(a1)
Best Bases mapping, 240 to 42 trained mapping --> gldbm_multi
% let us look into the mapping (asking for a user-specific data of the trained mapping):
>> getdata(w)
clf: [1x26 struct]
l: [1x26 double]
u: [1x26 double]
info_iter: 94
We can observe that the multi-class GLDB mapping is composed of 26 wavelength groups but yields
42 features.
Similarly to the two-class GLDB, the second-stage feature selection may be carried on and the resulting
feature selection mapping combined with the trained GLDB mapping so only the informative wavelength
groups are retained.
16
>> c1=a1*w
1320 by 42 dataset with 5 classes: [372 360 258 161 169]
Note, that the feature selection operates on the level of output features not the wavelength groups.
Finally, Hypertools provides a multi-class GLDB extractor leveraging the non-linear Fisher criterion,
introduced in:
Marco Loog, R.P.W. Duin, R. Haeb-Umbach, Multiclass Linear Dimension Reduction by Weighted
Pairwise Fisher Criteria, IEEE PAMI, vol. 23, no. 7, July 2001
The non-linear Fisher criterion is beneficial in situations where some classes are very distant from
others in the feature space. While the classical Fisher projection would emphasize the distant class,
the non-linear will re-weight the class contributions and offer a better overall performance. In PRTools,
the non-linear Fisher mapping is implemented in nlfisherm function. The Hypertools implements the
non-linear multi-class GLDB in the nlgldbm routine.
17
>> a1
projectA, 1320 by 240 dataset with 5 classes: [372 360 258 161 169]
% training set
>> b=seldat(a1,[2 3])
projectA, 618 by 240 dataset with 2 classes: [360 258]
>> w=genfeatsel(b,100,5)
Feature Selection, 240 to 2 fixed mapping --> cmapm
Figure: Scatter plot of a test set with features, identified by genetic algorithm
>> a1
projectA, 1320 by 240 dataset with 5 classes: [372 360 258 161 169]
>> w=maf(a1,0.9)
240 to 2 trained mapping --> affine
>> scatterdui(a1*w)
18
Figure: Scatter plot of the spectral data projected by MAF mapping
19
maximum possible correlation and at the same time are orthogonal to the previous columns in both
matrices. Columns of T and U are normalized to be univariate. Because X and Y can be not full-ranked,
pseudo inverse pinv is used in the algorithm.
>> plot_cca(b,bts)
Figure: Superimposed training (lighter colors) and test data (dark colors), projected by the CCA mapping
We can see the bright unimodal clusters corresponding to the training data, projected by CCA map-
ping. The darker markers denote the test set, projected using the same mapping. It is apparent, that in
this case the training set is not representative of the problem.
8 Image segmentation
Image segmentation is an unsupervised pattern recognition technique producing a unique assignment of
image pixels (spectra) into a set of classes. Because even the number of classes is usually unknown, image
segmentation is, in fact, an ill-posed clustering problem.
20
Firstly, a set of labels is created by clustering the spectral data domain. Then, in a loop, separate
classifiers are trained and executed in both domains: by default the nearest mean classifier (nmc) in the
spectral domain and Parzen classifier with a Gaussian kernel in the spatial domain (implemented by
convolution). Both domains are combined using the product combination rule. The process is repeated
until stability.
First, we segment a spectral image using a raw spectral data with 240 spectral wavelengths. We
cluster the data using k-means algorithm and using the combined spectra-spatial segmentation algorithm
(starting from the output of k-means). In the results bellow, we can see that some spatial inconsistencies
are improved using the combined segmentation algorithm because the spatial information is employed.
>> a1
projectA, 1320 by 240 dataset with 5 classes: [372 360 258 161 169]
Figure: Result of kmeans clustering (left) and combined spectral-spatial algorithm (right) using raw
spectra
In the second experiment, we build a dissimilarity-based representation of the spectral image using
Spectral Angle Mapper (SAM) distance measure. A set of five, randomly selected, prototypes forms a
representation set. Again, both the k-means and the combined spectral-spatial algorithms generate image
labelling:
>> proto=gendat(+a1,5)
5 by 240 dataset with 1 class: [5]
>> drawident(a1(:,100),proto)
>> D=dasam(a1,proto)
1320 by 5 dataset with 5 classes: [372 360 258 161 169]
21
Figure: Five randomly selected prototype pixels (spectra)
>> lab=kmeans(D,5);
>> getli(lab,a1) % left figure
Figure: Result of kmeans clustering (left) and combined spectral-spatial algorithm (right) using SAM
distances
k-means clustering algorithm doesnt take into account spatial connectivity and, therefore, provides a
noisy solution which may be homogenized using the combined spectra-spatial algorithm.
9 Classifiers
Hypertools implements several classifiers, traditionally used by the spectral community:
mindistc : Minimum distance classifier for dissimilarity to prototypes. An example is available
here
corrc : correlation classifier
samc : Spectral Angle Mapper classifier
todo:simca desciption + simcac and simcam examples
22