Image Compression Using Wavelet Transform and Self-development Neural Network

Jung-Hua Wang and Mer-Jiang Gou

Department of Electrical Engineering

National Taiwan Ocean University
Keelung 202, Taiwan, ROC
Email: bQM&&ul.ntou.edu.hy

ABSTRACT
In this paper, *e propose a novel method of encoding an
image without blocky effects. The method mcorporates
the wavelet transform and a self-development neural
network-Vitality Conservabon (VC) network [11 to
acheve sigtllficant mprovement in mage compression
performance The implementationconsists of three steps.
First, the image is decomposed at different scales using
wavelet transform to obtain an orthogonal wavelet
representation of the image Each band can be
subsequently processed in parallel. At the second step,
the discrete Karhunen-Loeve transform is used to extract
the pmcipal components of the wavelet coefficients.
Thus, the processing speed can be much faster than
otherwise. Fmally, results of the second step are used as
input to the VC network for vector quanbzation Our
simdabon results show that such implementauon can, in
much less time, achieve superior reconstructed images to
other methods [2].

1.

efficient image compressor using wavelet transform and

the VC network. Before training the VC network, we
first apply the dlscrete Karhunen-Loeve transform to
wavelet coefficients to obtain an optimal set of basis data.
Doing so can greatly reduce the quantizatoin errors and
training time. The block diagram of the compression
system is shown in Fig. 1. The organization of h s paper
is as follow. Section 2 briefly describes the wavelet
transforms used in t h ~ spaper and a quick review of
wavelets in general. In Section 3 we briefly describe the
discrete Karhunen-Loeve (K-L) "form
which is used
to transform the wavelet coefficients. Section 4
introduces the VC network, followed by simulation
results in Section 5.

2. THE WAVELET TRANSFO

Wavelet transforms are multiresolution decompositions
that can be used to analyze signals and images. They
describe a signal by the power at each scale and position
Edges can be located very effectively in the wavelet
transform domain. For image processing applications,
one can use the hierarchical wavelet decomposition by
Mallat [4]. The G and H filters are applied to the image
in both the horizontal and vertical directions, and the
filter outputs subsampled by a factor of two, generating
three orientation selective high-pass subbands, GG, GH,
HG, and a low-pass subband HH. The process is then
repeated on the HH band to generate the next level of the
decomposition, and so on. As shown in Fig. 2, one
octave of decomposition leads to four subbands. Fig. 3
shows the wavelet image of Lena decomposed in this
manner. Therefor, ten subbands are obtained by iterating
three times of such decomposition. The low-resolution

INTRODUCTION

Appllcatlons, whlch need to store large database and lor

tranmt digtal images requving high bit-rates over
channels mth lmted bandwdth, have demanded
improved image compresslon techruques Neural
network and wavelet transform [5] have been used for
mage compression with some success in the past.
Unsupemsed networks based on compeatwe learning
algorithms [3] have been successfully applied to learmng
vector quantlzatlon In th~s paper, we apply a
compebtive type self-development neural network,
namely the VC network [l] to l e m g vector
quanUzatlon The purpose of this study is to design an

0-7803-4778-1198 $10.00 0 199

4104

vector quantization. During competitive learning process,

the frequency of a node being accessing by input vectors
largely depends on input distribution. In practice, the real
winning probability of each node cannot be possibly
known. So one can define vitality vdo for an arbitrary kh
node as the measure of wining frequency &er an input
presentation at time t. Since the winning node is the only
node that has the right to update its weight vector, after
the kh node is a selected as the winning node, we not
only update its weight vector, but also add Av+(Q to uk(r).
On the other han4 a quantity of do-(@ is subtracted
from vital~ty of rest non-wining nodes. Vitality
consewation is defined as

version of the image, A t ' , is fed back to the input to

produce one low-resollution version of the image and six
subbands which contain image details.
The wavelet decomposition is an alternative
representation of images. To compress an input image
data using transform, we have to decide which
coefficients to send and how many bits to encode them.
Our compression algorithm consists of talung the lowpass subband in full, and then deciding which
coefficients within tlhe remaining subbands to keep.
Hence, the basic idea is this: the smaller the support of
the wavelet, the less nonzero wavelet coefficients will
correspond to an edge:, so the more efficient will be our
compression scheme. In this work, we choose the
Daubechies-4 wavelet [5]. The second step of the
compression system is to apply the discrete KarhunenLoeve transform to the coefficients other than the wresolution subband. .

M/t
k=l

where M(Q is the total number of nodes at time t. Given

an initial vitality I.@), the vitality U&) at time t can be
formulated as

3. THE DIMSCRETE K-L TRANSFOR

where

The Karhunen-Loeve Transform (KLT), also known as

principal component analysis, is an optimal transform in
the sense of mean-squareerror and is useful in a variety
of applications such as data compression, image
processing, and pattem recognition. IUT is an
orthogonal transform whose basis vectors are
eigenvectors of the specified covariance matrix of input
data. At the second stage in Fig.l, we use the discrete
KLT to extract the remaining subbands. Then the
Karhunen-Loeve Expansion (KLE) will give the optunal
set of basis vectors [6]. The KLE projects the ndimensional data vectors onto the k-dimensional
subspace spanned .by the k eigenvectors which
correspond to the k largest eigenvalues of the covariance
matrix of the data set ( n > k). Due to the decrement in
dimensions of the data, the processing speed at next
stage can be much faster, and the resulting compression
ratio higher than other methods. In the following, we
propose a selfdevelqpment neural network to quantize
the results of the second stage.

Wk(i) =

1, if k , node wins at time i

otherwise

0,

(3)

Assuming a node has won many times and its vitality

exceeds a threshold Ob,, in order to achieve better
codeword representations, a new node is generated from
the high vitahty node. Note that a mother node is always
a winning node, but the reverse is not true. Also assume
at t=m-Z the vitality of a winning node udm-1)2 q-.
Given the fact that equal parhtioning the probability
space yields maximum entropy, the new born node and
its mother node should share the vitality equally.
Therefore, after a node-generation the vitality of the new
node (and its mother node) is given by
(4)

In most competitive learning algorithms, a global

learning rate q(!, is used and its initialization has great
effect on the final results. Normally any initial value ??,&
(0< qw, < I) monotonically decayed by a factor y (e.g.
0.9) can do the job. In order to speed up the learning
process, the original region represented by nwrnshould be
partitioned as quickly as possible into two equal regions.
This can be achieved if larger values of q(t) is assigned

4. THE VC NEURAL NETWORK

In this Section we de!;crilx the selfdevelopment neural
network based on competitive leaming in conjunction

to the mothedson and their neighboring nodes. Also,

with a vitality consvration [l] for incremental learning

4105

becausetheVC network learns as it grows, hasty

growing in node number may result in early convergence
(at
10.001), and ends up incomplete training. Thus,
it is vital to keep growing speed and quality of l e a m g
in balance during the traimng process. To avoid the
insufficient learning, it is mportant that the next new
node not be generated until after sufficient number of
input vectors has been presented.

31.673 dE3. Using a bit rate of 0.058 bpp, the PSNR can
be 30.012dJ3, as shown in Fig.4(c). Therefore, our
proposed method indeed can preserve the image details
and edges very well, whtch is particularly evident in the
ham of the compressed Lena image in Fig.4(b) and
Fig.4(c)

5. EXPERIMENTAL RESULTS

In ~s paper, we have proposed a method of image

compression that incorporate the wavelet transform and
VC neural network. The wavelet transform used here
attempts to exploit the blocky effect of the human eye,
pelding encouraging results. The discrete KLT gives the
optimal set of basis vector so that image quality and
speed of training network can be improved sigIllficantly.
After KLT, we train the VC network for learning VQ
applied to image compression. We have empirically
shown that the proposed method can achieve high
compression ratio while maintaining good visual quality
through the use of bit allocation in the subimages.

6. CONCLUSIONS

In all our simulations presented in this paper,

performance is measured by using the Mean-Squared
Error (MSE)
MSE =

1
-E

E(?;

- f j ) 2

,=1

Another useful measure is the PSNR (peak-signal-tonoise) defined as

psm

= 10 log 1o

[-]

where f r denotes the original image,

f (m, )

is the

7. REFERENCES

reconstructed . In measuring the quality of output

compressed images, we will use the PSNR.

J. H. Wang and C. P. Hsiao, Representationburden conservation nework applied to learning

VQ, Neural Processing Letters. Vol. 5, 3, pp. 209217, 1997.
T. De* K. Pwhi, and V. Cherkassky, Combining
neural networks and the wavelet transform for
image compression, Proc. IEEE Intnl Con$,
pp.637-640, 1993.
S. C. Ahalt, A. K. Krishnamuthy, P. Chen, and D. E.
Melton, Competitive learning algorithms for
vector quantization, Neural Networks., vol. 3, pp.
277-290, Oct. 1990.
S. Mallat, A theory for multiresolution signal
decomposition; The wavelet representation, IEEE
Trans. Pattern Anal. Mach. Intell., vol. 11, pp. 674693, 1989.
I. Daubechies, Orthonormal bases of compactly

We have applied our method to the input gray image,

Lena, as shown in Fig. 4(a). The image is 512x512
pixels in size. The PSNR values are calculated. Our
results achieve better picture quality for a given
compression ratio than the standard image compression
methods. The performance of our approach is compared
to other compression approaches. Among the diflerent
methods reported, we consider four well-known methods:
Ho and Gersho obtained a result of PSNR =30.93dl3 at
0.36 bpp, using the variable-rate multistage VQ [7].
High PSNR values were obtained by Amir Averbuch,
Danny Lazar and Moshe Israeli used entropy coded and
LGB algorithtq and they achieved PSNR = 31.76dE3 at
0.136 bpp [SI. Also,Mare Antonini, and Michel Barlaud
had PSNR=30.85-dB at 0.37 bpp [ 9 ] .Finally, the EZW
method achmed PSNR=33.17dl3 at 0.25bpp and 27.54
dl3 at 0.0625 bpp [lo].

supported wavelets, Commun. Pure Appl. Math.,

vol. 41, pp. 909-996, 1988.

R. C. Gonzalez and P. Wintz, Digital Image
Processing, Addison-Wesley, 1987.
Y. Ho and A. Gersho, Variable-rate multi-stage
vector quatization for image coding, in Proc.
IEEE ICASP, New York, Apr. 1988.
A. Averbuch, D. Lazar and M. Israeli, Image

Quotes of PSNR for the 512*512 Lena image are so

abundant throughout the image compression literature.
Lena is compressed using out method to a bit rate of
0.150 bpp, as shown in Fig. 4(b). The resulting PSNR is

4106

1, pp. 205-220, Apr. 1992.

[lo] J. M. Shapiro, Embedded image codmg using
zerotrees of wavelet coefficients, Signal proc.
IEEE Trans. vol. 41, pp. 3445-3462,Dec. 1993.

compression ming wavelet transform and

multiresolution dwomposition, Image proc. IEEE
Trans. vol. 5, pp. 4-15, Jan. 1996.
[9] M. Antonini and M. Barlaud, Image coding using
wavelet transfom~,Image Proc. IEEE Trans. vol.

Low-resolution
subband

-7bFHGd
KLTI

Image

==$

VCNI

ReconstImage
ruction %ssed

Waveet
decomp
osi-tion

==$

KLT3

==$

VCN3

=-=)

Figure 1. Block diagram of the image compressionsystem.

LL subband
oflewl8

512

HL subband
~

jj

256

:
LH subband
Ofleve18

I1

HH wbband

le$8

256

256

256

Figure 2. First stage of a discrete wavelet transform: The

image is divided into four subbands using separate filters.

4107

Figure 3. Wavelet decomposition of Lena image

@@-pass coefficientstimes 4).

Figure 4. (a) orignal512 by 512 image. @) compressed image at 0.150 bpp,

PSNR = 3 1.673. (c) compressed image at 0.058 bpp, PSNR = 30.0 12

41 08