R 255

G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
ICT -Source encoding & channel encoding


Presented By:
Deep Chandra Bhatt
SCET-2049
ECE DEPARTMENT
M.TECH BATCH 2013-2015
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Content


o Overview of Coding
Source and channel coding definition.
o Code Length and Code Efficiency
o Source Coding Techniques
Shannon-Feno coding
Huffman coding
o Channel Coding
Error Control coding ( ARQ and FEC)

R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
The main purpose of coding is to improve the efficiency of communication system.

Coding is a procedure for mapping a given set of messages [m1,m2mN] into a new set
of encoded message [C1,C2CN] in such a manner that the transformation is one to
one. This is called source coding.


Main goal is to get minimum average length of code to increase the efficiency of
transmission.


It is also possible to devise codes to reduce the probability of error by detecting and
correcting errors (a.k.a. error control codes ) this coding is called channel coding.
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Code Length and Coding Efficiency:

Let M be number of symbols in there be N messages [m1,m2..MN] with probabilities
[P(m1),P(m2)P(mN)] and Let ni be the number of symbols in i th message.

The average code length of the message is then given by :

_ N
L = ni p(ni) letters/message
i=1
_
L should be minimum to have efficient transmission

Coding efficiency ,then can be defined as ;

_
= Lmin
______
__
L
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Let H(x) be the entropy of the source in bits per message, and let Log M is the max.
information associated to each letter in bits /Letter. Then H(x)/Log (m) gives the minimum
average no. of letters per message Hence coding efficiency is :
_
= Lmin = H(x)
______ ______
__ __
L L Log M



R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Classification of Codes:

Fixed length codes : code word length is fixed.

Variable Length codes: code word length is variable

Distinct codes: each code word is distinguishable from other.

Prefix free codes: a code in which no code word can be formed by adding code symbol to
another code ( in prefix free code no code word is prefix of another).

Uniquely decodable codes: A distinct code is uniquely decodable if original signal can be
reconstructed from encoded binary sequence.

Instantaneous codes: uniquely decodable codes and prefix codes are instantaneous if the end
of any code word is recognizable without examining the subsequent code symbols.

Optimal Codes: A code is said to be Optimal if it is instantaneous and has minimum average
length L
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Shannon-Fano Encoding: Algorithm

The technique was proposed in Shannon's "A Mathematical Theory of communication", his 1948 article
introducing the field of information theory. The method was attributed to Fano, who later published it as
a technical report
Procedure of coding is as follows:
-Arrange the character set in order of decreasing probability
-While a probability class contains more than one symbol:
Divide the probability class in two so that the probabilities in the two halves are as
nearly as possible equal
-Assign a '1' to the first probability class,
and a '0' to the second
Character
X6
X3
X4
X5
X1
X7
X2
Probability
0.25
0.2
0.15
0.15
0.1
0.1
0.05
1
0
1
0
1
0
1
0
1
0
1
0
Code
11
10
011
010
001
0001
0000

R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Shannon-Fano Encoding: Example
Message x
1
x
2

x
3

x
4

x
5

x
6

x
7

x
8

Probability 0.25 0.25 0.125 0.125 0.0625 0.0625 0.0625 0.0625
x1,x2,x3,x4,x5,x6,x7,x8
x1,x2 x3,x4,x5,x6,x7,x8
x1 x2 x3,x4 x5,x6,x7,x8
x3 x4 x5,x6 x7,x8
01 00
1100
100
10
0
1
11
x5 x6 x7 x8
101
110 111
1101 1110 1111
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Shannon-Fano Encoding: Example



Entropy

Average length of the encoding vector




Message x
1
x
2
x
3
x
4
x
5
x
6
x
7
x
8
Probability 0.25 0.25 0.125 0.125 0.0625 0.0625 0.0625 0.0625
Encoding
vector
00 01 100 101 1100 1101 1110 1111
1 1 1 1 1 1
2 log 2 log 4 log 2.75
4 4 8 8 16 16
H
| |
| | | | | |
= + + =
| | | |
\ . \ . \ .
\ .
{ }
1 1 1
2 2 2 3 4 4 2.75
4 8 16
i i
L P x n
| |
| | | | | |
= = + + =
| | | |
\ . \ . \ .
\ .

8
H(x)= Pk Log Pk
K=1
Letters/message
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Huffman Encoding

This encoding algorithm has been proposed by David A. Huffman in 1952, and it is still
the main loss-less compression basic encoding algorithm.
Huffman code leads to lowest possible values of code length and thus also called as
minimum redundancy code or optimum code.
Procedure of coding is as follows:
1.Messages are arranged in decreasing order of probability
2. The probability of least likely messages are combined
and the resulting Probabilities are rearranged in
decreasing order this step is called reduction.

Character
X6
X3
X4
X5
X1
X7
X2
Probability
0.25
0.2
0.15
0.15
0.1
0.1
0.05
0.25
0.35
0.25
0.15
0.6
0.4
1.0
Character
X6
X3
X4
X5
X1
X7
X2
Probability
00
010
011
100
101
110
111
0
1
0
1
0
1
0
1
0
1
0
1

R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Channel coding deals with error control techniques. If the data at
the output of a communications system has errors that are too
frequent for the desired use, the errors can often be reduced by
the use of a number of techniques.

Coding permits an increased rate of information transfer at a fixed
error rate, or a reduced error rate for a fixed transfer rate. The two
main methods of error control are:

Automatic Repeat Request (ARQ) when a receiver circuit
detects errors in a block of data, it requests that the data is
retransmitted.

Forward Error Correction (FEC) the transmitted data is encoded
so that the data can correct as well as detect errors caused by
channel noise.
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
The two main categories of channel codes are:
Block codes a block code is a code having all its words of same
length .a block of k information bits is encoded to give a codeword
of n bits (n > k). For each sequence of k information bits, there is a
a distinct codeword of n bits.
Examples of block codes include Hamming Codes and Cyclic
Codes.
A Cyclic Redundancy Check (CRC) code can detect any
error burst up to the length of the CRC code itself.
Convolution Codes the coded sequence of n bits depends not
only on the present k information bits but also on the previous
information bits.

These coding schemes will be covered in error control codes in details.!!!
R 255
G 204
B 0
R 255
G 130
B 0
R 127
G 16
B 162
R 163
G 166
B 173
R 104
G 113
B 122
R 234
G 234
B 234
R 170
G 15
B 30
R 0
G 0
B 0
R 255
G 255
B 255
Supporting colors:
R 60
G 170
B 0
Primary colors:
To change the document information in the footer, press [Alt + F8] and use the Nokia_Siemens_Networks__Change_Document_Information macro or click
You can setup the color/gradient and outline style of selected shapes with the Shape_set_color_and_gradient macro, or by clicking
Thank you