Data Compression

Topic : Source Coding

Data Compression Why ?

Storing or transmitting multimedia data requires large
space or bandwidth
The size of one hour 44K sample/sec 16-bit stereo (two
channels) audio is 3600x44000x2x2= 633.6MB, which can
be recorded on one CD (650 MB). MP3 compression can
reduce this number by factor of 10
The size of a 500x500 color image is 750KB without
compression (JPEG can reduced this by a factor of 10 to 20)
The size of one minute real-time, full size, color video clip is
60x30x640x480x3= 1.659GB. A two-hour movie requires
200GB. MPEG2 compression can bring this number down
to 4.7 GB (DVD)

the need for compression

Image: 6.0 million pixel camera, 3000x2000
18 MB per image 56 pictures / 1GB

Video: DVD Disc 4.7 GB

video 720x480, RGB, 30 f/s 31.1MB/sec
audio 16bits x 44.1KHz stereo 176.4KB/s
1.5 min per DVD disc

Send video from cellphone:

352*240, RGB, 15 frames / second
3.8 MB/sec $38.00/sec levied by AT&T

what can we compress?

Goals of compression
Remove redundancy
Reduce irrelevance

irrelevance or perceptual redundancy

not all visual information is perceived by
eye/brain, so throw away those that are not.

what can we compress?

Goals of compression
Remove redundancy
Reduce irrelevance

redundant : exceeding what is necessary or

normal
symbol redundancy
the common and uncommon values cost the same to
store

spatial and temporal redundancy

adjacent pixels are highly correlated.

Introduction
Compression is a way to reduce the number of
bits in a frame but retaining its meaning.
Decreases space, time to transmit, and cost
Technique is to identify redundancy and to
eliminate it
If a file contains only capital letters, we may
encode all the 26 alphabets using 5-bit
numbers instead of 8-bit ASCII code

Introduction
If the file had n-characters, then the savings
= (8n-5n)/8n => 37.5%

Compression can be categorised in

two broad ways:
Lossless Compression
where data is compressed and can be reconstituted
(uncompressed) without loss of detail or information.
These are referred to as bit-preserving or reversible
compression systems also.

Lossy Compression
where the aim is to obtain the best possible fidelity for a
given bit-rate or minimizing the bit-rate to achieve a
given fidelity measure. Video and audio compression
techniques are most suited to this form of compression.

Broad Classification
Entropy Coding (statistical)
lossless; independent of data characteristics
e.g. RLE, Huffman, LZW, Arithmetic coding

Source Coding
lossy; may consider semantics of the data
depends on characteristics of the data
e.g. DCT, DPCM, ADPCM, color model transform

Hybrid Coding (used by most mm systems)

combine entropy with source encoding
e.g., JPEG, H.263, MPEG-1, MPEG-2, MPEG-4

Compression methods
Run-length Coding
Entropy Coding

Huffman Coding
Arithmetic Coding
DPCM

Prediction

Source Coding

DM
FFT

Transformation

DCT
Bit Position

Layered Coding

Sub-sampling
Sub-band Coding

Vector Quantization
JPEG
MPEG
Hybrid Coding

H.261
DV1 RTV, DV1 PLV

Source Coding vs. Channel Coding

Source Coding

Channel Coding

Decoding

Source Encoding Techniques

Transformation encoding
Transform the bit-stream into another domain
Data in the new domain more amenable to
compression
Type of transformation depends on data

Image/video transformed from time domain

into frequency domain (DCT)

Source Coding (Data Compression)

Exploit the redundancy in the source to reduce the
data required for storage or for transmission
Highly complex encoders are required for
compression (MPEG, H.264 )
However, Simple decoders !
The Highly complex encoders require
Bulky handsets
Power consumption
Battery Life

How Source Coding Works

Types of redundancy
Spatial redundancy - Transform or predictive coding
Temporal redundancy - Predictive coding

In predictive coding, the next value in the sequence is

predicted from the past values and the predicted value is
subtracted from the actual value
The difference is only sent to the decoder
Let the Past values are in C, the predicted value is y = f(C).
If the actual value is x, then (x y) sent to the decoder.

How Source Coding Works

Decoder, knowing the past values C, can
also predict the value of y.
With the knowledge of (x y), the decoder
finds the value of x, which is the desired
value

x-y
+

x-y
+

x
-

Prediction

Prediction

Past
Values (C)

Past
Values (C)

Encoder

Decoder

Differential/Predictive Encoding
Encoding the difference between actual value and a
prediction of that value
How they work?
When consecutive change little
Suited for audio and video

Predictive coding simply means transmitting differences

Predict the next sample as being equal to the current sample
More complex prediction schemes can be used

Instead of sending the current sample, send the error involved

in the previous assumption

Predictive Coding
Number of Techniques
Differential Pulse Code Modulation (DPCM)
Delta Modulation (DM)
Adaptive Pulse Code Modulation (APCM)

Lossless Predictive Coding

Formally, define the integer signal as the set of values fn. Then,
we predict values f^n and compute the error en as follows:
t

fn an k f n k
k 1

en f n fn
when t = 1, we get ...
Usually, t is between 2 and 4 (in this case it is called a linear predictor
We might need to have a truncating or rounding operation following
the prediction computation

Lossless Predictive Coding

Lossless Predictive Coding: Example

Consider the following predictor:

f 1 f f
n
n2
2 n 1

e f f
n

Show how to code the following sequence

f1 , f 2 , f 3 , f 4 , f 5 21, 22, 27, 25, 22.

Differential Encoding for Audio

Differential Pulse Code Modulation(DPCM)
When we use PCM, we get a sequence of PCM coded
samples.
Represent first PCM sample as a whole and all the
following samples as differences from the previous one.
Sample

+
-

DPCM
Encoder

Previous
Sample
Difference
+

DPCM
Decoder

difference

+
Previous
Sample

Sample

Lossy Predictive Coding: DPCM

DPCM = Differential Pulse Code Modulation
Form the prediction f ^n
Form an error en
Quantize the error

Lossy Predictive Coding: DPCM

Lossy Predictive Coding: DPCM

Example

~
1 ~
fn
f n 1 f n 2
2

e f f
n

255 en
e~n Q[en ] 16 *
256 8
16

Differential Pulse Code

Modulation (DPCM)
Encode the changes between consecutive samples
Example
f (n)

f (n)

n
f (n) 156,157,158,158,156,156,154,154,155

f (n) 156,1,1,0,1,0,1,0,1

The value of the differences between samples are much

smaller than those of the original samples. Less bits are
used to encode the signal (e.g. 7 bits instead of 8 bits)

DPCM decoding
The difference is added to the previous
sample to obtain the value of the current
sample.

Delta Modulation
Modification of DPCM
Uses only 1 bit to encode difference.
Sets 1 if the difference increases
Sets 0 if the difference decreases

Leads to inaccurate coding

Delta Modulation
DM (Delta Modulation) is a simplified
version of DPCM that is used as a quick
analog-to-digital converter.

Note that the prediction simply involves a delay

Adaptive DPCM (ADPCM)

One observation is that small difference
between samples happens more often than
large changes
Entropy coding method such the Huffman
coding scheme can be used to encode the
difference for additional efficiency

JPEG Encoding- DCT

DCT: Discrete Concise Transform
DCT transforms the 64 values in 8x8 pixel block in a way
that the relative relationships between pixels are kept but
the redundancies are revealed.
Example:
A gradient grayscale

Discrete Cosine Transform

(DCT)
DCT is the real part of the 2D Fourier transform

The inverse DCT is

DCT Transform of 2D Images

DCT Example

DCT of images can also be considered as the

projection of the original image into the DCT basis
functions. Each basis function is in the form of
2 y 1
2x 1
cos(v
) cos(u
)
2N
2N

JPEG Discrete Cosine

Transform
DCT transforms the data from a spatial domain to a
frequency domain.
It removes redundancy in the data.
It is proven to be the optimal transform for large classes
of images.
The DCT algorithm is symmetrical, and an inverse DCT
algorithm can be used to decompress an image.
The DCT coefficients of each 8x8 blocks are calculated
using the formula below.

JPEG Discrete Cosine

Transform
Here is an example. On the left is the 8 x 8
block of pixels, and on the right is the DCT
coefficients.

JPEG - Quantization
The DCT output matrix is quantized to reduce the precision of the
coefficients.
This increases the compression DCT(0, 0) is known as the DC
coefficient which represents the basic colour, i.e., wave-length, of the
image block
The other DCT coefficients are known as AC coefficients which
represent the frequency components of the data block.
AC coefficients further away from the DC coefficient can be dropped
to reduce the data size
JPEG baseline algorithm defines a set of quantization tables
Each element q in the table, known as quantum is used in the
following formula to calculate the quantized coefficients Q:

Vector Quantization
Divide the data stream into blocks or vectors
One or two dimensional blocks

Use codebooks
Find the closest symbol in codebook for a
given sample
Transmit the reference to that symbol
Codebook present at sender/receiver
When no exact match, could send the error
Lossy or lossless

Useful with known signal characteristics

Construct codebooks that can match a wide

Vector Quantization
Two Dimensional Voronoi Diagram

Codewords in 2-dimensional space. Input vectors are

marked with an x, codewords are marked with red circles,
and the Voronoi regions are separated with boundary lines.

Vector Quantization
Source Image

..

..

Training Vector

Index (j)
1
2
3

16,384

Training Vector (xj(k))

x1(1) x1(16)
x2(1) x2(16)
x3(1) x3(16)

x16384(1) x16384(16)

Vector Quantization
Training Vector
Index (j)
1
2
3

16,384

Training Vector (xj(k))

x1(1) x1(16)
x2(1) x2(16)
x3(1) x3(16)

x16384(1) x16384(16)

Codebook Design

Codebook Table

Vector Quantization
New Image (Index Table)

i1

i5

i39

i10

i45

i45

i200

i12

i54

i256

i96

i78

i256

i54

i5

i9

Codebook Table

The Schematic of a Vector Quantizer

Q:R Y
k

.(1)

Block diagram of Vector Quantization

Intro

The Schematic of a Vector Quantizer

Compression Formula
Amount of compression:
Codebook size is K, input vector of dimension L
In order to inform the decoder of which code vector is selected, we
need to use log 2 K bits.
E.g. need 8 bits to represent 256 code vectors.

Rate: each code vector contains the reconstruction value of L

source output samples, the number of bits per sample would be:
.
log K / L

Sample: a scalar value in vector.

K: level of vector quantizer.

Summary
For Audio and Vedio compression , we use
the technique of Differential /Predictive
Coding with quantization.
For an Image compression, we can use DCT
methode with quantization.

Thanks For Your Attention!

Question..?
Submitted by,
Nikheesh
Fifth Semester MCA
CCSIT,CU Campus