JPEG

Still Image Data Compression Standard

JPEG Introduction - The background

JPEG stands for Joint Photographic Expert Group A standard image compression method is needed to enable interoperability of equipment from different manufacturer It is the first international digital image compression standard for continuous-tone images (grayscale or color) Why compression is needed?
Ex) VGA(640x480) with compression degradation 640x480x8x3=7,372,800bits 200,000bits without any visual

JPEG Introduction whats the objective?

very good or excellent compression rate, reconstructed image quality, transmission rate be applicable to practically any kind of continuous-tone digital source image good complexity have the following modes of operations:
sequential encoding Progressive encoding lossless encoding

JPEG Overview
Source image data Encoder model descriptors encoder statistical model symbols entropy encoder compressed image data

model tables

entropy coding tables

The basic parts of an JPEG encoder

JPEG Baseline System

JPEG Baseline System

JPEG Baseline system is composed of:
Sequential DCT-based mode Huffman coding

88 blocks

DCT-based encoder
FDCT quantizer statistical model table specification entropy encoder compressed image data

Source image data

table specification

The basic architecture of JPEG Baseline system

JPEG Baseline System Why does it work?

Lossy encoding HVS is generally more sensitive to low frequencies Natural images
s e n s itivity

fre q u e n c y

Frequency sensitivity of Human Visual System

The Baseline System DCT

The Discrete Cosine Transform (DCT) separates the frequencies contained in an image. The original data could be reconstructed by Inverse DCT.
The mathematical representation of FDCT (2-D):
7 7 1 F (u, v ) = C (u )C ( v ) f ( x, y ) cos( 2(i + 1)u / 16) cos( 2( j + 1)v / 16) 4 x =0 y =0

Where

1 / 2 x=0 C( x) = 1 otherwise f(x,y): 2-D sample value F(u,v): 2-D DCT coefficient

Basis of DCT transform

7 7 1 F (u, v) = C(u)C(v) f ( x, y) cos(2(i + 1)u / 16) cos(2( j + 1)v / 16) 4 x =0 y = 0

The Baseline System-DCT (cont.)

An example of 1-D DCT decomposition
(b)
150 100 50 150 100 50

(c)

f(x)

0 1 -50 -100 -150 2 3 4 5 6 7 8

S(u)

0 1 -50 -100 -150 2 3 4 5 6 7 8

Before DCT (image data)

1
1
1

After DCT (coefficients)

1

Amplitude

0 1 2 3 4 5 6 7 8

Amplitude

Amplitude

Amplitude

0 1 2 3 4 5 6 7 8

0 1 2 3 4 5 6 7 8

0 1 2 3 4 5 6 7 8

-1

-1

-1

-1

U=0
1
1

U=1
1

U=2
1

U=3

Amplitude

Amplitude

Amplitude

0 1 2 3 4 5 6 7 8

0 1 2 3 4 5 6 7 8

0 1 2 3 4 5 6 7 8

Amplitude

0 1 2 3 4 5 6 7 8

-1

-1

-1

-1

U=4

U=5

U=6

U=7

The 8 basic functions for 1-D DCT

The Baseline System-DCT (cont.)

DC coefficient

The DCT coefficient values can be regarded as the relative amounts of the 2-D spatial frequencies contained in the 88 block the upper-left corner coefficient is called the DC coefficient, which is a measure of the average of the energy of the block Other coefficients are called AC coefficients, coefficients correspond to high frequencies tend to be zero or near zero for most natural images

The Baseline System Quantization

F (u, v) F ' (u, v) = Round ( ) Q(u , v)
F(u,v): original DCT coefficient F(u,v): DCT coefficient after quantization Q(u,v): quantization value

Why quantization? .
to achieve further compression by representing DCT coefficients with no greater precision than is necessary to achieve the desired image quality

Generally, the high frequency coefficients has larger quantization values Quantization makes most coefficients to be zero, it makes the compression system efficient, but its the main source that make the system lossy

The Baseline System-Quantization (cont.)

1 6 1 2 1 4 1 4 1 8 2 4 4 9 7 2 1 1 1 2 1 3 1 7 2 2 3 5 6 4 9 2 1 0 1 4 1 6 2 2 3 7 5 5 7 8 9 5 1 6 1 9 2 4 2 9 5 6 6 4 8 7 9 8 2 4 2 6 4 0 5 1 6 8 8 1 13 0 12 1 4 0 5 8 5 7 8 7 19 0 14 0 11 2 10 0 5 1 6 0 6 9 8 0 13 0 13 1 10 2 13 0 6 1 5 5 5 6 6 2 7 7 9 2 11 0 9 9

JPEG Luminance quantization table

A simple example
O O X X X X O O O X X X X X X O O X X X X X X O X X X X X X X X X X X X X X X X O X X X X X X O O X X X X X X O O O X X X X O O

Original image pattern

-10 -10 10 10 10 10 -10 -10

-10 10 10 10 10 10 10 -10

-10 10 10 10 10 10 10 -10

10 10 10 10 10 10 10 10

10 10 10 10 10 10 10 10

-10 10 10 10 10 10 10 -10

-10 10 10 10 10 10 10 -10

-10 -10 10 10 10 10 -10 -10

40 0 -45 0 -20 0 -3 0

0 -26 0 0 0 -24 0 0 0 0 0 0 0 10 0 0

0 0 0 0 0 8 0 0 0 20 0 0 0 18 0 0

0 -11 0 0 0 -10 0 0 0 0 0 0 0 4 0 0

0 0 0 0 0 0 0 0

Digitized image

After FDCT(DCT coefficients)

A simple example(cont.)
16 12 14 14 18 24 49 72 11 12 13 17 22 35 64 92 10 14 16 22 37 55 78 95 16 19 24 29 56 64 87 98 24 26 40 51 68 81 103 112 40 58 57 87 109 104 121 100 51 60 69 80 103 113 120 103 61 55 56 62 77 92 101 99

40 0 -45 0 -20 0 -3 0

0 -26 0 0 0 -24 0 0 0 0 0 0 0 10 0 0

0 0 0 0 0 8 0 0 0 20 0 0 0 18 0 0

0 -11 0 0 0 -10 0 0 0 0 0 0 0 4 0 0

0 0 0 0 0 0 0 0

3 0 -3 0 -1 0 0 0

0 -3 0 0 0 -2 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

DCT coefficients

Quantized coefficients

Baseline System - DC coefficient coding

Since most image samples have correlation and DC coefficient is a measure of the average value of a 88 block, we make use of the correlation of DC coefficients

quantized DC coefficients

DPCM

DC difference

Differential pulse code modulation

Baseline System - AC coefficient coding

AC coefficients are arranged into a zig-zag sequence:
Horizontal frequency
3 0 -3 0 -1 0 0 0 0 0 0 0 0 0 0 0 -3 0 -2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 2

1 4 8 11 19 22 34 36

5 7 12 18 23 33 37 49

6 13 17 24 32 38 47 48

14 16 25 31 39 46 50 57

15 26 30 40 45 51 56 58

27 29 41 44 52 55 59 62

28 42 43 53 54 60 61 63

Vertical frequency

3 9 10 20 21 35

3 0 0 -3 0 -3 0 0 0 0 -1 0 -2(EOB)

Baseline System - Statistical modeling

Statistical modeling translate the inputs to a sequence of symbols for Huffman coding to use Statistical modeling on DC coefficients:
symbol 1: different size (SSSS) symbol 2: amplitude of difference (additional bits)

Statistical modeling on AC coefficients:

symbol 1: RUN-SIZE=16*RRRR+SSSS symbol 2: amplitude of difference (additional bits)

Additional bits for sign and magnitude

Huffman AC statistical model run-length/amplitude combinations

Huffman coding of AC coefficients

An examples of statistical modeling

quantized DC value DPCM difference SSSS Additional bits
zigzag index AC descriptor RRRR SSSS RUN-SIZE Additional bits

Example 1: Huffman symbol assignment to DC descriptors +8 +9 +8 -6 -8 -3 0 +1 -1 -14 -2 +5 0 1 1 4 2 3 -1 0 0001 00 101

8 9 +1 0 1 33 1

+3 +6 3 110
EOB 0 0 --

+3 0 0 -63 0

Example 2: Huffman symbol assignment to AC descriptors 1 2 3 4 5 6 7 0 0 0 0 -14 0 0 4 2 4 68 0001

Other Operation Modes: JPEG2000 ROI coding

JPEG 2000
Allow efficient lossy and lossless compression within a single unified coding framework Progressive transmission by quality, resolution, component, or spatial locality Compressed domain processing Region of Interest coding JPEG2000 is NOT an extension of JPEG Wavelet Transform An extremely flexible bitstream structure

DCT Transform vs. Space-Scale Transform

JPEG2000 ROI coding

Bit plane shift Finer Quantization level used

Experiment
http://www.sfu.ca/~cjenning/toybox/hj peg/index.html