40
1, JANUARY 2006
Classification of Remote Sensing Images From Urban
Areas Using a Fuzzy Possibilistic Model
Jocelyn Chanussot, Senior Member, IEEE, Jon Atli Benediktsson, Fellow, IEEE, and Mathieu Fauvel
Abstract—The classification of very high-resolution remotely
sensed images from urban areas is addressed. Previous studies
have shown the interest of exploiting the local geometrical in-
formation of each pixel to improve the classification. This is
performed using the derivative morphological profile (DMP) ob-
tained with a granulometric approach, using opening and closing
operators. For each pixel, this DMP constitutes the feature vector
on which the classification is based. In this letter, we present an
interpretation of the DMP in terms of a fuzzy measurement of
the characteristic size and contrast of each structure. This fuzzy
measure can be compared to predefined possibility distributions to
derive a membership degree for a set of given classes. The decision
is taken by selecting the class with the highest membership degree.
This model is illustrated and validated in a classification problem
using IKONOS images.
Index Terms—Classification, fuzzy sets, granulometry, mathe-
matical morphology, possibility distribution, urban area, very high
resolution.
I. INTRODUCTION
A MONG the region-based approaches for segmentation,
classification at the pixel-level is very popular in the field
of remote sensing. It basically consists of assigning each pixel
to a semantically or physically meaningful class. The decision
is generally taken by only considering the value of the pixel
(be it gray-level or multispectral), the “physical” nature of each
pixel being inferred from this value. However, when one is
not only interested in the “physical” nature of each pixel, but
also in the structure of the image features, more information
is needed. In this letter, we address the case of the analysis of
urban areas, using panchromatic very high-resolution remotely
sensed images. Some local geometrical information is clearly
required to enable accurate classification: for instance, pixels
belonging to the roof either of small or of large buildings will
have the same value, and a classification based on these values
will not be able to distinguish between these two classes (small
and large buildings). To solve this problem, a classification
method has been proposed in [1] and [2]. It is composed of the
following two main steps.
Manuscript received May 18, 2005; revised July 11, 2005. This research was
supported in part by the Icelandic Research Fund, in part by the Research Fund
of the University of Iceland, and in part by the Jules Verne Program of the French
and Icelandic Governments (PAI EGIDE).
J. Chanussot is with the Signals and Images Laboratory (LIS, Grenoble),
LIS/ENSIEG, Domaine Universitaire, 38402 Saint-Martin-d’Hères Cedex,
France (e-mail: jocelyn.chanussot@lis.inpg.fr).
J. A. Benediktsson is with the Department of Electrical and Computer Engi-
neering, University of Iceland, 107 Reykjavik, Iceland (e-mail: benedikt@hi.is).
M. Fauvel is with the Signals and Images Laboratory (LIS, Grenoble),
LIS/ENSIEG, Domaine Universitaire, 38402 Saint-Martin-d’Hères
Cedex, France and also with the Department of Electrical and Computer
Engineering, University of Iceland, 107 Reykjavik, Iceland (e-mail:
mathieu.fauvel@lis.inpg.fr).
Digital Object Identifier 10.1109/LGRS.2005.856117
1545-598X/$20.00 © 2006 IEEE
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 3, NO.
1) Feature extraction: This is based on the construction of
a derivative morphological profile, which characterizes
each pixel, both in terms of intensity and in terms of local
geometry. This step is described in Section II.
2) Classification: This step was initially based on a neural
network. In this letter, we propose to use a fuzzy possi-
bilistic model instead. This is presented in Section III.
The results obtained on IKONOS remote sensing data are
presented in Section IV. Fig. 4 presents one original panchro-
matic image from Reykjavik, Iceland (1-m resolution, size
975 639). Six classes of interest are defined: large buildings,
houses, open areas, large roads, streets, and shadows (see [2]
for more details). The proposed method is positively compared
to the previously published method.
II. FEATURE EXTRACTION AND GRANULOMETRY
Granulometriesarepopularandpowerfultoolsderivedfromthe
mathematicalmorphologytheory.Theyareclassicallyusedforthe
analysisofthesizedistributionofparticlesinanimage.Theycanbe
applied in various applications, ranging from the study of porous
media to texture segmentation [3]. More information on mathe-
maticalmorphologycanbefoundin[4].Soille[5]isanapplication
oriented book on morphological image analysis, from the princi-
ples to recent developments, and Soille and Pesaresi [6] is a survey
paper investigating the use of advanced morphological operators
in the general frame of satellite remote sensing.
Granulometries have recently been introduced in remote
sensing image processing for the classification of urban areas
[1], [2]. As a matter of fact, traditional pixel classification
techniques used for remotely sensed rural areas turned to be
ill-suited for urban areas, especially at very high resolution.
Beyond the spectral signature, there is actually a strong need for
incorporating spatial information in the classification process.
That can be achieved using granulometries.
The classical granulometry by opening is obtained by ap-
plying morphological opening operations with structuring ele-
ments (SEs) of increasing size. The consequence is a progres-
sive simplification of the image with a gradual disappearance
of the features that are brighter than their immediate neighbor-
hood. Each structure is removed when it becomes smaller than
the SE. Using connected operators, no shape noise is introduced
[7]. The opening being an antiextensive operation, the evolution
of the gray level for each pixel can be plotted as a monotonously
decreasing curve as the size of the SE increases. Noting
as the original image and , the morphological opening by re-
construction using a disc SE of radius , the morphological pro-
file is defined for each pixel by
(1)
CHANUSSOT et al.: CLASSIFICATION OF REMOTE SENSING IMAGES FROM URBAN AREAS 41

Granulometric curves are often interpreted by computing

their discrete derivative. After each morphological filter, the
difference with the result of the previous operation is computed.
For each pixel, this results in the differential morphological
profile DMP , which is also known as the “pattern spectrum”
[3]. For each pixel, DMP is given by
(a) (b)
DMP (2)

A structure is removed when the size of the SE reaches its

characteristic size. Corresponding pixels are then assigned the
gray-level value of the surrounding darker region. For all these
pixels, this generates a large value on the DMP, a value rep-
resenting the local contrast between the removed structure and
its surrounding region. Furthermore, the index of the itera-
tion where the structure is removed provides an estimation of
the size of the structure. Opening operations only affect struc-
tures that are brighter than their immediate neighborhood. Other
structures are left unchanged and thus lead to a flat DMP, with
a null value.
Similarly, a granulometry by closing is obtained using mor-
phological closing operations, with the same set of structuring
elements. In the same way, a derivative profile DMP is ob-
tained. Noting the morphological closing by reconstruction,
it is given by
DMP (3)
By duality to the opening operation, the closing-based pro-
file provides information regarding the structures that are darker
than their immediate surrounding and does not affect structures
that are brighter than their surroundings.
Finally, in order to process simultaneously the bright and the
dark structures of the image, the two DMPs are concatenated
DMP DMP for
DMP for (4)
Fig. 1 presents several examples of profiles obtained for var-
ious pixels from the IKONOS image. One typical example is
presented for each class. On each profile, the left part of the
curves corresponds to the granulometries by closing,
and the right part to the granulometries by opening.
In this case, the used structuring elements were discs with in-
creasing radius (successively 3, 6, 9, 12, 15, 18, 21, and 24 pixels
long radius). Using the Euclidean distance, a discrete disc of ra-
dius is defined as the set of pixels of coordinates such
that . In the case of urban area remote sensing,
the objects of interest have fairly uniform gray level. Further-
more, the size of the structuring element remains bounded, en-
suring that only local contrasts are measured. Consequently, the
DMP of most pixels has one side with only zero values (either
the “opening” side for objects darker than their immediate sur-
roundings or the “closing” side for objects brighter than their
immediate surroundings). This assumption falls if larger neigh-
borhood are considered, or if the objects are textured.
As a conclusion, the DMP provides for each pixel a 16-di-
mensional vector of attributes. Following this feature extraction,
the classification of each pixel is based on this vector. Previous
studies have investigated the use of a neural network, eventu-
ally after a dimension reduction and a feature selection [1], [2].
(c) (d)
(e) (f)
Fig. 1. Examples of typical differential morphological profiles (DMP(i))
obtained for various pixels. (a) A street pixel, (b) a large road pixel, (c) a
shadow pixel, (d) a house pixel, (e) a large building pixel, and (f) an open area
pixel.
In this letter, we propose to perform a classification based on a
fuzzy interpretation of the DMP, using possibilistic models for
the different classes. This is discussed in Section III
III. CLASSIFICATION: POSSIBILISTIC MODEL
A. Interpretation of the DMP
As previously stated, when the size of the structuring ele-
ment reaches the size of one given structure, the structure is
removed. This induces a noticeable change of the gray-level
values of the corresponding pixel and, as a consequence, a peak
in the DMP. Therefore, the position DMP , of the
greatest value within the DMP, is an estimate of the charac-
teristic size of the structure the pixel belongs to. However, for
physical reasons or because of the spatial sampling, the different
structures in the image do not have perfectly sharp edges. Con-
sequently, the DMP is not constituted of a single peak with a
one-sample-wide support, and the estimation of the size remains
imprecise. Fuzzy sets theory is the appropriate frame to handle
imprecise or uncertain information [8].
In this letter, we propose to interpret the normalized DMP,
defined as DMP DMP , as a fuzzy measure of the
size of each structure. In addition, information regarding the
contrast of the structure is provided by the maximum value of
the DMP: DMP .
The examples of different DMPs presented in Fig. 1 are con-
sistent with this interpretation. For instance, looking at corre-
sponding DMPs, it clearly appears that a house is smaller than a
large building or that a large road is larger than a street. Re-
garding the contrast information, shadows are usually highly
contrasted and dark (they appear on the left side of the DMP)
42
whereas as open areas usually are less contrasted and bright
(right part of the DMP). Furthermore, the need for both the in-
formation related to the size of the structure and the information
related to its contrast is clearly established: e.g., the support of
the DMPs presented in Fig. 1(b) and (c) largely overlaps. The
dinstiction between the shadow class and the large road class
requires the consideration of the contrast. This is described in
Sections III-B–E.
B. Fuzzy Possibilistic Models
For any semantic classification, the expert has to model the
knowledge related to the considered classes. For instance, when
classifying using a neural network, this knowledge consists in
manually labeling some pixels in order to train the network. The
design of this ground truth can be difficult. Furthermore, the
training process itself remains to a large degree uncontrolled,
and the whole process is not very flexible. For instance, the re-
moval or the addition of one class requires the whole process
to be restarted from the beginning: modification of the ground
truth map and new training of the network.
In the frame of urban remote sensing, the expert usually has
a clear mental representation of the objects of interest. This
knowledge can be formalized as simple statements such as the
following.
• A shadow is a dark object with a high contrast but unknown
size.
• A house is a bright object with a small size and a high
contrast.
• A large building is a bright object with a large size and a
high contrast.
The quantitative meaning of words such as “small,” “large,” or
“narrow,” “wide,” etc. has to be determined. However, these
concepts cannot be crisply defined. For instance if “small”
means smaller than 12, it does not make sense to consider 13 as
“big,” and not “small” at all. As a consequence, a fuzzy model
seems more appropriate to model the expert knowledge. In this
letter, we propose to use possibility distributions [9]–[11]. Such
a model fits the intuition and is consistent with the interpreta-
tion of the DMP described in the previous section, as will be
described in the following.
C. Information About the Size
First, the information related to the size of the different struc-
ture is modeled. One possibility distribution is designed
for each class in the shape of a generalized trapezoidal func-
tion ranging from 0 (absolutely impossible) to 1 (completely
possible). The transitions are linear, which is the most popular
choice in the frame of fuzzy logic. Of course, other transitions
could be considered but the expert usually has no rationale for
the setting of more complex shapes and the influence on the
results anyway remains limited. Fig. 2 presents the six possi-
bility distributions for the six information classes. For instance,
Fig. 2(a) models the expert knowledge for a “large building.”
In this case, considering a structure that is brighter than its sur-
rounding (here, without any consideration of the absolute value
of the contrast), this corresponds to the following assertion: “if
the characteristic size of this structure is greater than 15, it is
fully possible that it is a large building. If the size is smaller than
9, it is strictly impossible that it is a large building. Inbetween,
the possibility increases with the size.” In a second step, for each
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 3, NO. 1, JANUARY 2006
(a) (b)
(c) (d)
(e) (f)
Fig. 2. Possibility distributions designed by the expert to characterize the
possible size of each kind of structure. (a) The street class, (b) the large road
class (c), the shadow class, (d) the house class, (e) the large building class, and
(f) the open area class.
pixel, a degree of membership in each class is computed.
The degree of membership indicates, to what degree, the current
pixel is likely to belong to each of the classes. This is defined as
the normalized scalar product between the DMP and each dis-
tribution
DMP
(5)
DMP
If for a given pixel and a given class the support of the DMP
and the support of the fuzzy set are disjoint, the membership
degree of this pixel to class is zero. On the contrary, if the
support of the DMP is included in the core of the fuzzy set, the
membership degree is maximum, and it is fully possible that the
pixel belongs to the class . Since the cores of the different pos-
sibility distributions are not necessarily disjoint, one pixel can
be considered as possibly belonging to several different classes.
For instance, with the defined models, a pixel belonging to a
large dark structure can equally belong to a shadow or to a large
street. The difference is made when aggregating the contrast in-
formation. This is discussed in Sections III-D and E.
D. Information About the Contrast
In parallel, the expert designs the characteristic contrast of
each class. Is the local contrast “high” or “low”? Again, this can
be easily modeled using fuzzy sets. Fig. 3 presents the fuzzy def-
initions for low and large contrast, respectively. The expert de-
cides which contrast model holds for each class (e.g., a shadow
should have a large contrast whereas a street usually has a low
contrast). Comparing the measured local contrast, i.e., the max-
imum value DMP of the DMP, with the corresponding
model (respectively low or high), another membership degree
CHANUSSOT et al.: CLASSIFICATION OF REMOTE SENSING IMAGES FROM URBAN AREAS 43

(a) (b)
Fig. 3. Possibility distributions modeling (a) low and (b) high contrast.

is derived for each class, based on the sole contrast informa-

tion. These values are ranging from 0 to 1.

E. Aggregating Size and Contrast Information

In a last step, the two membership degrees (respectively de-
rived from the size and from the contrast information) are ag-
(a) (b)
gregated. This is done using a simple product. This is a con-
junctive fusion operator [12]: a class is likely to be selected if
this choice is consistent both with the size and with the contrast
information. The final decision is taken by selecting the class
maximizing this product

class (6)

Note that the idea of jointly processing the size and the contrast
of the different features in an image has also been addressed in
[13] for the analysis of oriented patterns such as fingerprints or
fiber structures.

IV. RESULTS
In our experiments, we used the following parameters: eight
closings and eight openings were computed with increasing
structuring element sizes (the increment is three pixels). This is
a tradeoff to obtain both a feature vector with a reasonable size (c) (d)
and SE sizes ranging from small (3) to rather large Fig. 4. (a) Original Ikonos image. (b) Manually labeled ground truth. (c)
. The setting of these parameters must ensure that Classification result using the neural network. (d) Classification result using
all the structures of interest fall within this range. The increment the proposed fuzzy approach.
should be fine enough to ensure the separation between the
classes of interest, consistently with the expert’s definition of
TABLE I
the corresponding distributions of possibility. As previously CONFUSION MATRIX—NEURAL CLASSIFIER—OVERALL ACC. 40.3%
mentioned, the structuring elements are discs. Fig. 4(b) presents
the ground truth obtained by a manual labeling. It is especially
large to ensure a fair and significant evaluation. For visual
inspection, Fig. 4(c) presents the classification results obtained
with the neural network (algorithm presented in [2]). Fig. 4(d)
presents the results obtained with the proposed fuzzy possi-
bilistic model. The following lookup table is used: shadows are
in brown, large buildings in red, houses in gray, large roads
in greenish blue, streets in blue, and open areas in green. For
a quantitative evaluation, Tables I and II, respectively, present
the confusion matrices obtained with the two methods for the
six classes. It is interesting to see that the overall accuracy
increases from 40.3% with the neural network to 52.1% with
the proposed fuzzy approach. However, one should also note
that the neural network outperforms the proposed method in
terms of accuracies for the streets class. postfiltering could increase these accuracies. Although the rep-
It should be underlined that the numerical results (Tables I resentation of the data in terms of DMP has proven its poten-
and II) should only be considered in a relative way, in order tial, it also has some intrinsic limitations (accuracy results are
to compare the performances of the methods. Some pre- and still far below 100%, meaning that some further developments
44
TABLE II
CONFUSION MATRIX—FUZZY CLASSIFIER—OVERALL ACC. 52.1%
are still needed to solve the problem of automatic classifica-
tion of very high-resolution images from urban areas). In par-
ticular, the DMP only provides information regarding the size
and the contrast of the objects within the image. To achieve
a more accurate classification, additional information may be
needed, especially in the context of complex natural images.
Various methods incorporating contextual, structural, or percep-
tual information have been proposed [14]–[16]. However, this is
beyond the scope of this letter whose aim is to propose and eval-
uate an alternative way of handling the DMP and extract valu-
able information.
V. CONCLUSION
The contribution of this letter lies in the two following items.
• Fuzzy interpretation of the DMP: Based on its basic defi-
nition, the differential morphological profile is interpreted
as a fuzzy measurement of the characteristic size of the
objects. It also provides information on the contrast of the
structures.
• Possibilistic model: We show how the expert’s knowledge
regarding the size and the contrast of the structures of in-
terest can be modeled as possibility distributions.
For classification purposes, the measured DMPs (interpreted as
fuzzy sets) are compared with the previously designed possi-
bility distributions. For each pixel, this provides a membership
degree to each of the classes. The final decision is taken by
selecting the class with the highest resulting membership de-
gree. This proposed classification method has several advan-
tages, which are summarized in the three following items.
• Simplicity: The trapezoidal model used for each possi-
bility distribution is simple and uses few parameters. Being
fuzzy, its settings are not too sensitive. Fitting intuition,
the model does not require any manual ground truth nor
training.
• Flexibility: In the proposed method, it is very easy to add
a new class of interest without restarting the whole clas-
sification process. There is only one possibility distribu-
tion to define and the corresponding membership degree
to compute.
• Complementarity: Finally, it is interesting to note that the
proposed algorithm and the previously published one [2]
provide complementary results. Therefore, there is a poten-
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 3, NO. 1, JANUARY 2006
tial for further improvement by adaptively combining these
results [17]. The fusion with other classification methods,
taking advantage of the DMP representation, but also over-
coming its limitations by incorporating other strategies, is
another perspective of this work.
Further developments also include the use of spectral informa-
tion. See [18] and [19] for some work using the DMP in the
frame of hyperspectral data. Another issue lies in the use of
other shapes for the structuring elements, such as line segments
with various orientations. That could be useful to address the
“streets” class for instance. See [20] and [13] for the use of
such structuring elements in different contexts, the latter also
addressing theoretical and algorithmic issues.
REFERENCES
[1] M. Pesaresi and J. A. Benediktsson, “A new approach for the morpho-
logical segmentation of high resolution satellite imagery,” IEEE Trans.
Geosci. Remote Sens., vol. 39, no. 2, pp. 309–320, Feb. 2001.
[2] J. A. Benediktsson, M. Pesaresi, and K. Arnason, “Classification and
feature extraction for remote sensing images from urban areas based on
morphological transformations,” IEEE Trans. Geosci. Remote Sens., vol.
41, no. 9, pp. 1940–1949, Sep. 2003.
[3] R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed: Prentice
Hall, 2002.
[4] J. Serra, Mathematical Morphology, Theoretical Advances. Orlando,
FL: Academic, 1988, vol. 2.
[5] P. Soille, Morphological Image Analysis, Principles and Applications,
2nd ed. Berlin, Germany: Springer-Verlag, 2003.
[6] P. Soille and M. Pesaresi, “Advances in mathematical morphology ap-
plied to geoscience and remote sensing,” IEEE Trans. Geosci. Remote
Sens., vol. 40, no. 9, pp. 2042–2055, Sep. 2002.
[7] J. Crespo, J. Serra, and R. Schafer, “Theoretical aspects of morpholog-
ical filters by reconstruction,” Signal Process., vol. 47, pp. 201–225,
1995.
[8] L. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, pp. 338–353, 1965.
[9] , “Fuzzy sets as the basis for a theory of possibility,” Fuzzy Sets
Syst., vol. 1, pp. 3–28, 1978.
[10] D. Dubois and H. Prade, Possibility Theory. New York: Plenum, 1988.
[11] Z. Wang and G. Klir, Fuzzy Measure Theory. New York: Plenum,
1991.
[12] I. Bloch, “Information combination operators for data fusion: A compar-
ative review with classification,” IEEE Trans. Syst., Man, Cybern., vol.
26, no. 1, pp. 52–67, Jan. 1996.
[13] P. Soille and H. Talbot, “Directional morphological filtering,” IEEE
Trans. Pattern Anal. Mach. Intell., vol. 23, no. 11, pp. 1313–1329, Nov.
2001.
[14] A. Shackelford and C. Davis, “A combined fuzzy pixel-based and ob-
ject-based approach for classification of high resolution multispectral
data over urban areas,” IEEE Trans. Geosci. Remote Sens., vol. 41, no.
10, pp. 2354–2363, Oct. 2003.
[15] P. Gamba, J. A. Benediktsson, and G. Wilkinson, “Foreword to the spe-
cial issue on urban remote sensing by satellite,” IEEE Trans. Geosci.
Remote Sens., vol. 41, no. 9, pp. 1903–1906, Sep. 2003.
[16] C. Unsalan and K. Boyer, “A system to detect houses and residential
street network in multispectral satellite images,” Comput. Vis. Image Un-
derstanding, 2005, to be published.
[17] M. Fauvel, J. Chanussot, and J. A. Benediktsson, “Fusion of methods for
the classification of remote sensing images from urban areas,” presented
at the IGARSS, Seoul, Korea, 2005.
[18] J. A. Benediktsson, J. A. Palmason, and J. R. Sveinsson, “Classification
of hyperspectral data from urban areas based on extended morphological
profiles,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 3, pp. 480–491,
Mar. 2005.
[19] A. Plaza, P. Martinez, J. Plaza, and R. Perez, “Dimensionality reduc-
tion and classification of hyperspectral image data using sequences of
extended morphological transformations,” IEEE Trans. Geosci. Remote
Sens., vol. 43, no. 3, pp. 466–479, Mar. 2005.
[20] J. Chanussot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for
linear features detection in multitemporal sar images,” IEEE Trans.
Geosci. Remote Sens., vol. 37, no. 3, pp. 1292–1305, May 1999.