DC Error Correcting Codes

Digital Communication and Error
Correcting Codes
Timothy J. Schulz
Professor and Chair
Engineering Exploration
Fall, 2004
Department of Electrical and Computer Engineering

Digital Data
• ASCII Text
A 01000001
B 01000010
C 01000011
D 01000100
E 01000101
F 01000110
. .
. .
. .
Digital Coding for Error Correction 00101001110101101010101000

Digital Sampling
000010001000001011011011010000111110111111111
011
010
001
000
111
110
101
100

Digital Communication
• Example: Frequency Shift Keying (FSK)

– Transmit a tone with a frequency determined by each bit:
s  t   b cos  2 f 0t    1  b  cos  2 f1t 

Digital Channels
Binary Symmetric Channel
1-p
0 0
1 1
1-p
Error probability: p

Error Correcting Codes

3 channel bits per 1 information bit: rate = 1/3
encode book
information bits channel bits
0 000
1 111
decode book
channel bits information bits
000 0
001 0
010 0
011 1
100 0
101 1
110 1
111 1

information bits 0 0 1 0 1
channel code 000 000 111 000 111
received bits 010 000 100 001 110
decoded bits 0 0 0 0 1
5 channel errors; 1 information error

• An error will only be made if the channel makes 2 or

three errors on a block of 3 channel bits
0.5
ccc
situation
no errors
probability
(1-p)(1-p)(1-p)
= 1-3p+3p -p 2 3 0.45
cce one error (1-p)(1-p)(p) = p-2p2+p3 0.4

cec one error (1-p)(p)(1-p) = p-2p2+p3
cee two errors (1-p)(p)(p) = p2-p3 0.35
ecc one error (p)(1-p)(1-p) = p-2p2+p3
bit error probability
ece two errors (p)(1-p)(p) = p2-p3 0.3
eec two errors (p)(p)(1-p) = p2-p3
0.25
eee three errors (p)(p)(p) = p3
0.2
error probability = 3p2 – 2p3 0.15
0.1
0.05
0
0 0.1 0.2 0.3 0.4 0.5
channel error probability

• Codes are characterized by the number of channel bits

(M) used for (N) information bits. This is called an N/M
code.
• An encode book has 2N entries, and each entry is an
M-bit codeword.
• A decode book has 2M entries, and each entry is an
N-bit information-bit sequence.

Linear Block Codes
EE576 Dr. Kousa Linear Block Codes 15

Basic Definitions
• Let u be a k-bit information sequence

v be the corresponding n-bit codeword.
A total of 2k n-bit codewords constitute a (n,k) code.
• Linear code: The sum of any two codewords is a codeword.
• Observation: The all-zero sequence is a codeword in every
linear block code.

Generator Matrix
• All 2k codewords can be generated from a set of k linearly independent
codewords.
• Let g0, g1, …, gk-1 be a set of k independent codewords.
 g 0   g 00 g 01  g 0,n1 
G          
 
g k -1   g k 1,0 g k 1,1  g k 1,n1 
• v = u·G

Systematic Codes
• Any linear block code can be put in systematic form
n-k k
check bits information bits
• In this case the generator matrix will take the form

G = [ P Ik]
• This matrix corresponds to a set of k codewords
corresponding to the information sequences that have a
single nonzero element. Clearly this set in linearly
independent.

Generator Matrix (cont’d)
• EX: The generating set for the (7,4) code:
1000 ===> 1101000; 0100 ===> 0110100
0010 ===> 1110010; 0001 ===> 1010001
• Every codeword is a linear combination of these 4 codewords.
That is: v = u •G, where
 
1 1 0 1 0 0 0
 
 0 1 1 0 1 0 0
G   P | Ik 
1 1 1 0 0 1 0
 
1 0 1
  
0 0
  
0
 
1
 k( nk ) k k 
• Storage requirement reduced from 2k(n+k) to k(n-k).

Parity-Check Matrix
• For G = [ P | Ik ], define the matrix H = [In-k | PT]
• (The size of H is (n-k)xn).
• It follows that GHT = 0.
• Since v = u•G, then v•HT = u•GHT = 0.
• The parity check matrix of code C is the generator matrix
of another code Cd, called the dual of C.
1 0 0 1 0 1 1
H  0 1 0 1 1 1 0
 
0 0 1 0 1 1 1

Encoding Using H Matrix
(Parity Check Equations)
1 0 0
0 1 0
0 0 1
 v1 v2 v3 v4 v5 v6 v7  1 1 0  0
0 1 1
information 1 1 1
1 0 1
v1+v4  v6  v7  0 v1=v4  v6  v7
v2+v4  v5  v6  0  v2=v4  v5  v6
v3+v5  v6  v7  0 v3=v5  v6  v7

Encoding Circuit

Minimum Distance
• DF: The Hamming weight of a codeword v , denoted by
w(v), is the number of nonzero elements in the codeword.
• DF: The minimum weight of a code, wmin, is the smallest
weight of the nonzero codewords in the code.
wmin = min {w(v): v  C; v ≠0}.
• DF: Hamming distance between v and w, denoted by
d(v,w), is the number of locations where they differ.
Note that d(v,w) = w(v+w)
• DF: The minimum distance of the code
dmin = min {d(v,w): v,w  C, v ≠ 0}
• TH3.1: In any linear code, dmin = wmin

Minimum Distance (cont’d)
• TH3.2 For each codeword of Hamming weight l there

exists l columns of H such that the vector sum of these
columns is zero. Conversely, if there exist l columns of H
whose vector sum is zero, there exists a codeword of
weight l.
• COL 3.2.2 The dmin of C is equal
to the minimum numbers of
columns in H that sum to zero. 1 0 0 1 0 1 1
H  0 1 0 1 1 1 0 
• EX:  
0 0 1 0 1 1 1

Decoding Linear Codes
• Let v be transmitted and r be received, where

r=v+e v + r
e  error pattern = e1e2..... en, where e

th
ei   1 if the error has occured in the i location
 0 otherwise
The weight of e determines the number of errors.
• We will attempt both processes: error detection, and error

correction.

Error Detection
• Define the syndrome
s = rHT = (s0, s1, …, sk-1)
• If s = 0, then r = v and e =0,
• If e is similar to some codeword,
then s = 0 as well, and the error is undetectable.
• EX 3.4:
1 0 0
0 1 0 
0 0 1 
 s0 s1 s2    r1 r2 r3 r4 r5 r6 r7  1 1 0  0 s 0 = r0  r3  r5  r6
0 1 1  s1 = r1  r3  r4  r5
1 1 1
1 0 1 s 2 = r2  r4  r5  r6

Error Correction
• s = rHT = (v + e) HT = vHT + eHT = eHT

• The syndrome depends only on the error pattern.
• Can we use the syndrome to find e, hence do the
correction?
• Syndrome digits are linear combination of error digits.
They provide information about error location.
• Unfortunately, for n-k equations and n unknowns there are
2k solutions. Which one to use?

Example 3.5
• Let r = 1001001
• s = 111
• s0 = e0+e3+e5+e6 =1
• s1 = e1+e3+e4+e5 =1
• s2 = e2+e4+e5+e6 =1
• There are 16 error patterns that satisfy the above equations,
some of them are
0000010 1101010 1010011 1111101
• The most probable one is the one with minimum weight.
Hence v* = 1001001 + 0000010 = 1001011

Standard Array Decoding
• Transmitted codeword is any one of:

v1, v2, …, v2k
• The received word r is any one of 2n n-tuple.
• Partition the 2n words into 2k disjoint subsets D1, D2,…, D2k
such that the words in subset Di are closer to codeword vi
than any other codeword.
• Each subset is associated with one codeword.

Standard Array Construction
1. List the 2k codewords in a row, starting with the all-zero codeword v1.
2. Select an error pattern e2 and place it below v1. This error pattern will
be a correctable error pattern, therefore it should be selected such that:
(i) it has the smallest weight possible (most probable error)
(ii) it has not appeared before in the array.
3. Add e2 to each codeword and place the sum below that codeword.
4. Repeat Steps 2 and 3 until all the possible error patterns have been
accounted for. There will always be 2n / 2k = 2 n-k rows in the array.
Each row is called a coset. The leading error pattern is the coset
leader.
• Note that choosing any element in the coset as coset leader does not
change the elements in the coset; it simply permutes them.

Standard Array
v1  0 v2 v3  v 2k
e2 e2 + v2 e 2 + v3  e 2  v 2k
e3 e3 + v 2 e3 + v3  e3  v 2k
   
e 2n - k e 2n -k  v 2 e 2n - k  v3 e 2n -k  v 2k
• TH 3.3
No two n-tuples in the same row are identical.
Every n-tuple appears in one and only one row.

Standard Array Decoding is Minimum
Distance Decoding
• Let the received word r fall in Di subset and lth coset.
• Then r = el + vi
• r will be decoded as vi. We will show that r is closer to vi
than any other codeword.
• d(r,vi) = w(r + vi) = w(el + vi + vi) = w(el)
• d(r,vj) = w(r + vj) = w(el + vi + vj) = w(el + vs)
• As el and el + vs are in the same coset, and el is selected to
be the minimum weight that did not appear before, then
w(el)  w(el + vs)
• Therefore d(r,vi)  d(r,vj)

Standard Array Decoding (cont’d)
• TH 3.4
Every (n,k) linear code is capable of correcting exactly 2n-k
error patterns, including the all-zero error pattern.
• EX: The (7,4) Hamming code
# of correctable error patterns = 23 = 8
# of single-error patterns = 7
Therefore, all single-error patterns, and only single-error
patterns can be corrected. (Recall the Hamming Bound, and
the fact that Hamming codes are perfect.

EX 3.6: The (6,3) code defined by the H matrix:

Codewords
 1 0 0 0 1 1
  000000
H   0 1 0 1 0 1 110001
  101010
 0 0 1 1 1 0
011011
011100
v1=v5  v6
101101
v2=v4  v6 110110
v3=v4  v5 000111
d min  3
• Can correct all single errors and one double error pattern
000000 110001 101010 011011 011100 101101 110110 000111
000001 110000 101011 011010 011101 101100 110111 000110
000010 110011 101000 011001 011110 101111 110100 000101
000100 110101 101110 011111 011000 101001 110010 000011
001000 111001 100010 010011 010100 100101 111110 001111
010000 100001 111010 001011 001100 111101 100110 010111
100000 010001 001010 111011 111100 001101 010110 100111
100100 010101 001110 111111 111000 001001 010010 100011

The Syndrome
• Huge storage memory (and searching time) is required by standard array
decoding.
• Recall the syndrome
s = rHT = (v + e) HT = eHT
• The syndrome depends only on the error pattern and not on the transmitted
codeword.
• TH 3.6
All the 2k n-tuples of a coset have the same syndrome. The syndromes of
different cosets are different.
(el + vi )HT = elHT (1st Part)
Let ej and el be leaders of two cosets, j<l. Assume they have the same
syndrome.
ejHT = elHT (ej +el)HT = 0.
This implies ej +el = vi, or el = ej +vi
This means that el is in the jth coset. Contradiction.

The Syndrome (cont’d)
Error Pattern Syndrome

• There are 2n-k rows and
2n-k syndromes (one-to- 0000000 000
one correspondence). 1000000 100
• Instead of forming the 0100000 010
standard array we form a 0010000 001
decoding table of the 0001000 110
0000100 011
correctable error patterns 0000010 111
and their syndromes. 0000001 101

Syndrome Decoding
Decoding Procedure:
1. For the received vector r, compute the syndrome s = rHT.
2. Using the table, identify the coset leader (error pattern) el .
3. Add el to r to recover the transmitted codeword v.
• EX:
r = 1110101 ==> s = 001 ==> e = 0010000
Then, v = 1100101
• Syndrome decoding reduces storage memory from nx2n to
2n-k(2n-k). Also, It reduces the searching time considerably.

Hardware Implementation
• Let r = r0 r1 r2 r3 r4 r5 r6 and s = s0 s1 s2
• From the H matrix:
s0 = r0 + r3 + r 5 + r 6
s1 = r1 + r3 + r 4 + r 5
s2 = r2 + r4 + r 5 + r 6
• From the table of syndromes and their corresponding
correctable error patterns, a truth table can be constructed.
A combinational logic circuit with s0 , s1 , s2 as input and
e0 , e1 , e2 , e3 , e4 , e5 , e6 as outputs can be designed.

Decoding Circuit for the (7,4) HC

Error Detection Capability
• A codeword with dmin can detect all error patterns of weight dmin – 1 or
less. It can detect many higher error patterns as well, but not all.
• In fact the number of undetectable error patterns is 2k-1 out of the 2n -1
nonzero error patterns.
• DF: Ai  number of codewords of weight i.
• {Ai; i=0,1,…,n} = weight distribution of the code.
• Note that Ao=1; Aj =0 for 0 < j < dmin
n
Pu   i
A p i
i  d min
(1  p ) n i

• EX: Undetectable error probability of (7,4) HC
A0 =A7 = 1; A1 =A2 =A5 =A6=0; A3=A4=7
Pu(E) =7p3(1-p)4 + 7p4(1-p)3 + p7
For p = 10-2 Pu(E) = 7x10-6 n
• Define the weight enumerator: A( z ) 

i 0
Ai z i

• Then i
n n
 p 
Pu   Ai p (1  p)
i n i
 (1  p)  Ai p 
n i

i 1 i 1 1 p 
• Let z = p/(1-p), and noting that A0=1
i
 p  n
 p    p  
A   1   Ai   ; Pu  (1  p)  A
n
  1
1 p  i 1 1 p   1 p  

• The probability of undetected error can as well be found from the
weight enumerator of the dual code
Pu  2 n  k B (1  2 p )  (1  p) n
where B(z) is the weight enumerator of the dual code.

• When either A(z) and B(z) are not available, Pu may be upper bounded
by
Pu ≤ 2-(n-k) [1-(1-p)n]
• For good channels (p 0) Pu ≤ 2-(n-k)

Error Correction Capability
• An (n,k) code of dmin can correct up to t errors where
t   (d min  1) / 2
• It may be able to correct higher error patterns but not all.

• The total number of patterns it can correct is 2n-k
t
n
If     2 n  k
i 0  i  the code is perfect
n t
 n  p i (1  p) n i  1   n  p i (1  p) n i
Pu   i
i  t 1 
 i
i 0  

Hamming Codes
• Hamming codes constitute a family of single-error correcting codes
defined as:
n = 2m-1, k = n-m, m  3
• The minimum distance of the code dmin = 3
• Construction rule of H:
H is an (n-k)xn matrix, i.e. it has 2m-1 columns of m tuples.
The all-zero m tuple cannot be a column of H (otherwise dmin=1).
No two columns are identical (otherwise dmin=2).
Therefore, the H matrix of a Hamming code of order m has as its
columns all non-zero m tuples.
The sum of any two columns is a column of H. Therefore the sum of
some three columns is zero, i.e. dmin=3.

Systematic Hamming Codes
• In systematic form:
H =[ Im Q]
• The columns of Q are all m-tuples of weight  2.
• Different arrangements of the columns of Q produces
different codes, but of the same distance property.
• Hamming codes are perfect codes
t
 n   2nk
 i
i 0  
Right side = 1+n; Left side = 2m =n+1

Decoding of Hamming Codes
• Consider a single-error pattern e(i), where i is a number

determining the position of the error.
• s = e(i) HT = HiT = the transpose of the ith column of H.
• Example:
 1 0 0
 0 1 0
 0 0 1
 0 1 0 0 0 0 0 1 1 0   0 1 0
 0 1 1
 1 1 1
1 0 1

Decoding of Hamming Codes (cont’d)
• That is, the (transpose of the) ith column of H is the

syndrome corresponding to a single error in the ith position.
• Decoding rule:
1. Compute the syndrome s = rHT
2. Locate the error ( i.e. find i for which sT = Hi)
3. Invert the ith bit of r.

Weight Distribution of Hamming Codes
• The weight enumerator of Hamming codes is:

A( z ) 
1
n 1

(1  z ) n  n(1  z )( z 2 ) ( n 1) / 2 
• The weight distribution could as well be obtained from the
recursive equations:
A0=1, A1=0
(i+1)Ai+1 + Ai + (N-i+1)Ai-1 = CNi i=1,2,…,N
• The dual of a Hamming code is a (2m-1,m) linear code. Its
weight enumerator is B( z )  1  (2  1) z
m 2 m1

History
• In the late 1940’s Richard Hamming recognized that

the further evolution of computers required greater
reliability, in particular the ability to not only detect
errors, but correct them. His search for error-
correcting codes led to the Hamming Codes, perfect
1-error correcting codes, and the extended Hamming
Codes, 1-error correcting and 2-error detecting
codes.

Uses
• Hamming Codes are still widely used in computing,

telecommunication, and other applications.
• Hamming Codes also applied in
– Data compression
– Some solutions to the popular puzzle The Hat
Game
– Block Turbo Codes

A [7,4] binary Hamming Code
• Let our codeword be (x1 x2 … x7) ε F27

• x3, x5, x6, x7 are chosen according to the message
(perhaps the message itself is (x3 x5 x6 x7 )).
• x4 := x5 + x6 + x7 (mod 2)
• x2 := x3 + x6 + x7
• x1 := x3 + x5 + x7

[7,4] binary Hamming codewords

A [7,4] binary Hamming Code
• Let a = x4 + x5 + x6 + x7 (=1 iff one of these bits is in error)
• Let b = x2 + x3 + x6 + x7
• Let c = x1 + x3 + x5 + x7
• If there is an error (assuming at most one) then abc will be
binary representation of the subscript of the offending bit.
• If (y1 y2 … y7) is received and abc ≠ 000, then we

assume the bit abc is in error and switch it. If
abc=000, we assume there were no errors (so if
there are three or more errors we may recover the
wrong codeword).

Definition: Generator and Check
Matrices
• For an [n, k] linear code, the generator matrix is a
k×n matrix for which the row space is the given code.
• A check matrix for an [n, k] is a generator matrix for
the dual code. In other words, an (n-k)×k matrix M for
which Mx = 0 for all x in the code.

A Construction for binary
Hamming Codes
• For a given r, form an r × 2r-1 matrix M, the columns of which
are the binary representations (r bits long) of 1, …, 2r-1.
• The linear code for which this is the check matrix is a [2r-1, 2r-1
– r] binary Hamming Code = {x=(x1 x2 … x n) : MxT = 0}.
Example Check Matrix

• A check matrix for a [7,4] binary Hamming Code:

Syndrome Decoding
• Let y = (y1 y2 … yn) be a received codeword.

• The syndrome of y is S:=LryT. If S=0 then there was
no error. If S ≠ 0 then S is the binary representation
of some integer 1 ≤ t ≤ n=2r-1 and the intended
codeword is
x = (y1 … yr+1 … yn).

Example Using L3
• Suppose (1 0 1 0 0 1 0) is received.
100 is 4 in binary, so the intended codeword was (1 0

1 1 0 1 0).

Extended [8,4] binary Hamm.
Code
• As with the [7,4] binary Hamming Code:
– x3, x5, x6, x7 are chosen according to the message.
– x4 := x5 + x6 + x7
– x2 := x3 + x6 + x7
– x1 := x3 + x5 + x7
• Add a new bit x0 such that
– x0 = x1 + x2 + x3 + x4 + x5 + x6 + x7 . i.e., the new bit
makes the sum of all the bits zero. x0 is called a
parity check.

Extended binary Hamming Code
• The minimum distance between any two codewords
is now 4, so an extended Hamming Code is a 1-error
correcting and 2-error detecting code.
• The general construction of a [2r, 2r-1 - r] extended
code from a [2r –1, 2r –1 – r] binary Hamming Code
is the same: add a parity check bit.

Check Matrix Construction of
Extended Hamming Code
• The check matrix of an extended Hamming Code can
be constructed from the check matrix of a Hamming
code by adding a zero column on the left and a row
of 1’s to the bottom.

q-ary Hamming Codes
• The binary construction generalizes to Hamming

Codes over an alphabet A={0, …, q}, q ≥ 2.
• For a given r, form an r × (qr-1)/(q-1) matrix M over A,
any two columns of which are linearly independent.
• M determines a [(qr-1)/(q-1), (qr-1)/(q-1) – r] (= [n,k])
q-ary Hamming Code for which M is the check matrix.

Example: ternary [4, 2] Hamming
• Two check matrices for the some [4, 2] ternary
Hamming Codes:

Syndrome decoding: the q-ary
case
• The syndrome of received word y, S:=MyT, will be a
multiple of one of the columns of M, say S=αmi, α
scalar, mi the ith column of M. Assume an error
vector of weight 1 was introduced y = x + (0 … α …
0), α in the ith spot.

Example: q-ary Syndrome
• [4,2] ternary with check matrix , word (0

1 1 1) received.
• So decode (0 1 1 1) as
(0 1 1 1) – (0 0 2 0) = (0 1 2 1).

Perfect 1-error correcting
• Hamming Codes are perfect 1-error correcting codes.

That is, any received word with at most one error will
be decoded correctly and the code has the smallest
possible size of any code that does this.
• For a given r, any perfect 1-error correcting linear
code of length n=2r-1 and dimension n-r is a
Hamming Code.

Proof: 1-error correcting
• A code will be 1-error correcting if
– spheres of radius 1 centered at codewords cover the
codespace, and
– if the minimum distance between any two codewords ≥ 3,
since then spheres of radius 1 centered at codewords will be
disjoint.
• Suppose codewords x, y differ by 1 bit. Then x-y is a
codeword of weight 1, and M(x-y) ≠ 0. Contradiction. If x, y
differ by 2 bits, then M(x-y) is the difference of two multiples of
columns of M. No two columns of M are linearly dependent, so
M(x-y) ≠ 0, another contradiction. Thus the minimum distance
is at least 3.

Perfect
• A sphere of radius δ centered at x is

Sδ(x)={y in An : dH(x,y) ≤ δ}. Where A is the alphabet,
Fq, and dH is the Hamming distance.
• A sphere of radius e contains words.
• If C is an e-error correcting code then
, so .

Perfect
• This last inequality is called the sphere packing
bound for an e-error correcting code C of length n
over Fm:
where n is the length
of the code and in this case e=1.
• A code for which equality holds is called perfect.

Proof: Perfect
• The right side of this, for e=1 is qn/(1+n(q-1)).

• The left side is qn-r where n= (qr-1)/(q-1).
qn-r(1+n(q-1)) = qn-r(1+(qr-1)) = qn.
Applications
• Data compression.
• Turbo Codes
• The Hat Game

Data Compression
• Hamming Codes can be used for a form of lossy

compression.
• If n=2r-1 for some r, then any n-tuple of bits x is within
distance at most 1 from a Hamming codeword c. Let
G be a generator matrix for the Hamming Code, and
mG=c.
• For compression, store x as m. For decompression,
decode m as c. This saves r bits of space but
corrupts (at most) 1 bit.

The Hat Game
• A group of n players enter a room whereupon they each receive
a hat. Each player can see everyone else’s hat but not his own.
• The players must each simultaneously guess a hat color, or

pass.
• The group loses if any player guesses the wrong hat color or if
every player passes.
• Players are not necessarily anonymous, they can be numbered.
• Assignment of hats is assumed to be random.
• The players can meet beforehand to devise a strategy.
• The goal is to devise the strategy that gives the highest
probability of winning.

EE 551/451, Fall, 2007
Communication Systems

Zhu Han
Class 25
Dec. 6th, 2007

Outline
 Project 2
 ARQ Review
 Linear Code
– Hamming Code Revisit
– Reed–Muller code
 Cyclic Code

– CRC Code
– BCH Code
– RS Code
EE 541/451 Fall 2007

ARQ, FEC, HEC
 ARQ
Error detection code
tx rx
ACK/NACK
 Forward Error Correction (error correct coding)
tx

Error correction code
rx
 Hybrid Error Correction

Error detection/
Correction code
tx rx
ACK/NACK
EE 541/451 Fall 2007

Hamming Code
 H(n,k): k information bit length, n overall code length
 n=2^m-1, k=2^m-m-1:
 H(7,4), rate (4/7); H(15,11), rate (11/15); H(31,26), rate (26/31)
 H(7,4): Distance d=3, correction ability 1, detection ability 2.
 Remember that it is good to have larger distance and rate.

 Larger n means larger delay, but usually better code
EE 541/451 Fall 2007

Hamming Code Example
 H(7,4)
 Generator matrix G: first 4-by-4 identical matrix
 Message information vector p
 Transmission vector x

 Received vector r
and error vector e
 Parity check matrix H
EE 541/451 Fall 2007

Error Correction
 If there is no error, syndrome vector z=zeros
 If there is one error at location 2
 New syndrome vector z is

which corresponds to the second column of H. Thus, an error

has been detected in position 2, and can be corrected
EE 541/451 Fall 2007
Exercise
 Same problem as the previous slide, but p=(1001)’ and the error
occurs at location 4 instead.
 Pause for 5 minutes
 Might be 10 points in the finals.

EE 541/451 Fall 2007

Important Hamming Codes
 Hamming (7,4,3) -code. It has 16 codewords of length 7. It can
be used to send 27 = 128 messages and can be used to correct 1
error.
• Golay (23,12,7) -code. It has 4 096 codewords. It can be used to
transmit 8 3888 608 messages and can correct 3 errors.
 Quadratic residue (47,24,11) -code. It has 16 777 216

codewords and can be used to transmit 140 737 488 355 238

messages and correct 5 errors.
EE 541/451 Fall 2007

Reed–Muller code

EE 541/451 Fall 2007

Cyclic code
 Cyclic codes are of interest and importance because
– They posses rich algebraic structure that can be utilized in a
variety of ways.
– They have extremely concise specifications.
– They can be efficiently implemented using simple shift register
– Many practically important codes are cyclic

 In practice, cyclic codes are often used for error detection

(Cyclic redundancy check, CRC)
– Used for packet networks
– When an error is detected by the receiver, it requests
retransmission
– ARQ
EE 541/451 Fall 2007

BASIC DEFINITION of Cyclic Code

EE 541/451 Fall 2007

FREQUENCY of CYCLIC CODES

EE 541/451 Fall 2007

EXAMPLE of a CYCLIC CODE

EE 541/451 Fall 2007

POLYNOMIALS over GF(q)

EE 541/451 Fall 2007

EXAMPLE

EE 541/451 Fall 2007

Cyclic Code Encoder

EE 541/451 Fall 2007

Cyclic Code Decoder
 Divider

 Similar structure as multiplier for encoder
EE 541/451 Fall 2007

Cyclic Redundancy Checks (CRC)

EE 541/451 Fall 2007

Example of CRC

EE 541/451 Fall 2007

Checking for errors

EE 541/451 Fall 2007

Capability of CRC
 An error E(X) is undetectable if it is divisible by G(x). The
following can be detected.
– All single-bit errors if G(x) has more than one nonzero term
– All double-bit errors if G(x) has a factor with three terms
– Any odd number of errors, if P(x) contain a factor x+1
– Any burst with length less or equal to n-k
– A fraction of error burst of length n-k+1; the fraction is 1-2^(-(-n-

k-1)).
– A fraction of error burst of length greater than n-k+1; the fraction
is 1-2^(-(n-k)).
 Powerful error detection; more computation complexity
compared to Internet checksum
 Page 652
EE 541/451 Fall 2007

BCH Code
 Bose, Ray-Chaudhuri, Hocquenghem
– Multiple error correcting ability
– Ease of encoding and decoding
– Page 653
 Most powerful cyclic code
– For any positive integer m and t<2^(m-1), there exists a t-error

correcting (n,k) code with n=2^m-1 and n-k<=mt.

 Industry standards
– (511, 493) BCH code in ITU-T. Rec. H.261 “video codec for
audiovisual service at kbit/s” a video coding a standard used for
video conferencing and video phone.
– (40, 32) BCH code in ATM (Asynchronous Transfer Mode)
EE 541/451 Fall 2007

BCH Performance

EE 541/451 Fall 2007

Reed-Solomon Codes
 An important subclass of non-binary BCH
 Page 654
 Wide range of applications
– Storage devices (tape, CD, DVD…)
– Wireless or mobile communication
– Satellite communication

– Digital television/Digital Video Broadcast(DVB)

– High-speed modems (ADSL, xDSL…)
EE 541/451 Fall 2007

Examples
 10.2 page 639
 10.3 page 648
 10.4 Page 651
 Might be 4 points in the final

EE 541/451 Fall 2007

1971: Mariner 9
 Mariner 9 used a [32,6,16] Reed-Muller
code to transmit its grey images of Mars.

camera rate:
100,000 bits/second
transmission speed:
16,000 bits/second
EE 541/451 Fall 2007

1979+: Voyagers I & II
 Voyagers I & II used a [24,12,8] Golay code
to send its color images of Jupiter and Saturn.

 Voyager 2 traveled further to Uranus

and Neptune. Because of the higher
error rate it switched to the more
robust Reed-Solomon code.
EE 541/451 Fall 2007
Modern Codes
 More recently
Turbo codes
were invented,
which are used in
3G cell phones,
(future) satellites,
and in the Cassini-

Huygens space
probe [1997–].
 Other modern codes: Fountain, Raptor, LT, online codes…

 Next, next class
EE 541/451 Fall 2007

imperfectness of a given code as the difference between the code's required Eb/No to
attain a given word error probability (Pw), and the minimum possible Eb/No required to
attain the same Pw, as implied by the sphere-packing bound for codes with the same
block size k and code rate r.

EE 541/451 Fall 2007

Radio System Propagation

EE 541/451 Fall 2007

Satellite Communications
 Large communication area. Any two
places within the coverage of radio
transmission by satellite can
communicate with each other.
 Seldom effected by land disaster
( high reliability)
 Circuit can be started upon
establishing earth station (prompt
circuit starting)
 Can be received at many places

simultaneously, and realize broadcast,

multi-access communication
economically( feature of multi-
access)
 Very flexible circuit installment , can
disperse over-centralized traffic at
any time.
 One channel can be used in different
directions or areas (multi-access
connecting).
EE 541/451 Fall 2007

GPS
 Just a timer, 24 satellite
 Calculation position

EE 541/451 Fall 2007

IV054
CHAPTER 3: Cyclic and convolution
codes
Cyclic codes are of interest and importance because
• They posses rich algebraic structure that can be

utilized in a variety of ways.
• They have extremely concise specifications.
• They can be efficiently implemented using simple
shift registers.
Cyclic codes
• Many practically important codes are cyclic.
Convolution codes allow to encode streams od data

(bits).
IV054
BASIC DEFINITION AND EXAMPLES
•Definition A code C is cyclic if
•(i) C is a linear code;
•(ii) any cyclic shift of a codeword is also a codeword, i.e. whenever a0,… an -1  C, then
also an -1 a0 … an –2  C.
Example
(i) Code C = {000, 101, 011, 110} is cyclic.
(ii) Hamming code Ham(3, 2): with the generator matrix
1 0 0 0 0 1 1
 
0 1 0 0 1 0 1
G 
0 0 1 0 1 1 0
 
0 0 0 1 1 1 1 
is equivalent to a cyclic code. 
(iii) The binary linear code {0000, 1001, 0110, 1111} is not a cyclic, but it is
equivalent to a cyclic code.
(iv) Is Hamming code Ham(2, 3) with the generator matrix
1 0 1 1
 
 0 1 1 2 
(a) cyclic?
(b) equivalent to a cyclic code?
Cyclic codes
IV054 FREQUENCY of CYCLIC CODES
•Comparing with linear codes, the cyclic codes are quite scarce. For, example there are 11 811
linear (7,3) linear binary codes, but only two of them are cyclic.
•Trivial cyclic codes. For any field F and any integer n >= 3 there are always the following cyclic
codes of length n over F:
• No-information code - code consisting of just one all-zero codeword.
• Repetition code - code consisting of codewords (a, a, …,a) for a  F.
• Single-parity-check code - code consisting of all codewords with parity 0.
• No-parity code - code consisting of all codewords of length n
•For some cases, for example for n = 19 and F = GF(2), the above four trivial cyclic codes are
the only cyclic codes.
Cyclic codes
IV054 EXAMPLE of a CYCLIC CODE
•The code with the generator matrix
1 0 1 1 1 0 0
 
•has codewords
G  0 1 0 1 1 1 0
0 0 1 0 1 1 1
 
• c1 = 1011100 c2 = 0101110 c3 =0010111
• c1 + c2 = 1110010 c1 + c3 = 1001011 c2 + c3 = 0111001
• c1 + c2 + c3 = 1100101
•and it is cyclic because the right shifts have the following impacts
• c1  c2, c2  c3, c 3  c 1 + c3
• c1 + c2  c2 + c3, c1 + c3  c1 + c2 + c3, c 2 + c3  c1
• c 1 + c2 + c 3  c 1 + c 2
Cyclic codes
IV054
POLYNOMIALS over GF(q)
• Fq[x] denotes the set of all polynomials over GF(q ).

• deg (f(x )) = the largest m such that xm has a non-zero coefficient in f(x).
Multiplication of polynomials If f(x), g(x) Fq[x], then

deg (f(x) g(x)) = deg (f(x)) + deg (g(x)).
Division of polynomials For every pair of polynomials a(x), b(x)  0 in Fq[x] there
exists a unique pair of polynomials q(x), r(x) in Fq[x] such that
a(x) = q(x)b(x) + r(x), deg (r(x)) < deg (b(x)).
Example Divide x3 + x + 1 by x2 + x + 1 in F2[x].
Definition Let f(x) be a fixed polynomial in Fq[x]. Two polynomials g(x), h(x) are said
to be congruent modulo f(x), notation
g(x)  h(x) (mod f(x)),
if g(x) - h(x) is divisible by f(x).
Cyclic codes
IV054 RING of POLYNOMIALS
•The set of polynomials in Fq[x] of degree less than deg (f(x)), with addition and multiplication modulo f(x) forms a ring
denoted Fq[x]/f(x).
•Example Calculate (x + 1)2 in F2[x] / (x2 + x + 1). It holds

•(x + 1)2 = x2 + 2x + 1  x2 + 1  x (mod x2 + x + 1).
•How many elements has Fq[x] / f(x)?

•Result | Fq[x] / f(x) | = q deg (f(x)).
•Example Addition and multiplication in F2[x] / (x2 + x + 1)

+ 0 1 x 1+x  0 1 x 1+x
0 0 1 x 1+x 0 0 0 0 0
1 1 0 1+x x 1 0 1 X 1+x
x x 1+x 0 1 x 0 x 1+x 1
1+x 1+x x 1 0 1+x 0 1+x 1 x
Definition A polynomial f(x) in Fq[x] is said to be reducible if f(x) = a(x)b(x), where a(x), b(x)  Fq[x] and
deg (a(x)) < deg (f(x)), deg (b(x)) < deg (f(x)).
If f(x) is not reducible, it is irreducible in Fq[x].
Theorem The ring Fq[x] / f(x) is a field if f(x) is irreducible in Fq[x].
Cyclic codes
IV054
FIELD Rn, Rn = Fq[x] / (xn - 1)
•Computation modulo xn – 1
•Since xn  1 (mod xn -1) we can compute f(x) mod xn -1 as follow:
•In f(x) replace xn by 1, xn +1 by x, xn +2 by x2, xn +3 by x3, …
•Identification of words with polynomials

•a0 a1… an -1  a0 + a1 x + a2 x2 + … + an -1 xn -1
•Multiplication by x in Rn corresponds to a single cyclic shift

•x (a0 + a1 x + … an -1 xn -1) = an -1 + a0 x + a1 x2 + … + an -2 xn -1
Cyclic codes
IV054 Algebraic characterization of cyclic codes
• Theorem A code C is cyclic if C satisfies two conditions
• (i) a(x), b(x)  C  a(x) + b(x)  C
• (ii) a(x)  C, r(x)  Rn  r(x)a(x)  C
• Proof
• (1) Let C be a cyclic code. C is linear  (i) holds.
• (ii) Let a(x)  C, r(x) = r0 + r1x + … + rn -1xn -1
• r(x)a(x) = r0a(x) + r1xa(x) + … + rn -1xn -1a(x)
• is in C by (i) because summands are cyclic shifts of a(x).
• (2) Let (i) and (ii) hold

•  Taking r(x) to be a scalar the conditions imply linearity of C.
•  Taking r(x) = x the conditions imply cyclicity of C.
Cyclic codes
IV054 CONSTRUCTION of CYCLIC CODES
•Notation If f(x)  Rn, then
 f(x) = {r(x)f(x) | r(x)  Rn}
•(multiplication is modulo xn -1).
•Theorem For any f(x)  Rn, the setf(x) is a cyclic code (generated by f).
•Proof We check conditions (i) and (ii) of the previous theorem.
•(i) If a(x)f(x)  f(x) and b(x)f(x)  f(x), then

• a(x)f(x) + b(x)f(x) = (a(x) + b(x)) f(x) f(x)
•(ii) If a(x)f(x)  f(x), r(x)  Rn, then

• r(x) (a(x)f(x)) = (r(x)a(x)) f(x)  f(x).
Example C = 1 + x2 , n = 3, q = 2.
We have to compute r(x)(1 + x2) for all r(x)  R3.
R3 = {0, 1, x, 1 + x, x2, 1 + x2, x + x2, 1 + x + x2}.
Result C = {0, 1 + x, 1 + x2, x + x2}

C = {000, 011, 101, 110}
Cyclic codes
IV054
Characterization theorem for cyclic codes
•We show that all cyclic codes C have the form C = f(x) for some f(x)  Rn.
•Theorem Let C be a non-zero cyclic code in Rn. Then

• there exists unique monic polynomial g(x) of the smallest degree such that
• C = g(x)
• g(x) is a factor of xn -1.
Proof
(i) Suppose g(x) and h(x) are two monic polynomials in C of the smallest degree.
Then the polynomial g(x) - h(x)  C and it has a smaller degree and a multiplication
by a scalar makes out of it a monic polynomial. If g(x)  h(x) we get a contradiction.
(ii) Suppose a(x)  C.
Then
a(x) = q(x)g(x) + r(x) (deg r(x) < deg g(x))
and
r(x) = a(x) - q(x)g(x)  C.
By minimality
r(x) = 0
and therefore a(x) g(x).
Cyclic codes
IV054
Characterization theorem for cyclic codes
•(iii) Clearly,
•xn –1 = q(x)g(x) + r(x) with deg r(x) < deg g(x)
•and therefore r(x)  -q(x)g(x) (mod xn -1) and

•r(x)  C  r(x) = 0  g(x) is a factor of xn -1.
GENERATOR POLYNOMIALS
Definition If for a cyclic code C it holds
C = g(x),
then g is called the generator polynomial for the code C.
Cyclic codes
IV054 HOW TO DESIGN CYCLIC CODES?
•The last claim of the previous theorem gives a recipe to get all cyclic codes of given length n.
•Indeed, all we need to do is to find all factors of
• xn -1.
•Problem: Find all binary cyclic codes of length 3.
•Solution: Since
• x3 – 1 = (x + 1)(x2 + x + 1)
• both factors are irreducible in GF(2)
•we have the following generator polynomials and codes.
• Generator polynomials Code in R3 Code in V(3,2)

• 1 R3 V(3,2)
• x+1 {0, 1 + x, x + x2, 1 + x2} {000, 110, 011, 101}
• x2 + x + 1 {0, 1 + x + x2} {000, 111}
• x3 – 1 ( = 0) {0} {000}
Cyclic codes
IV054 Design of generator matrices for cyclic codes
• Theorem Suppose C is a cyclic code of codewords of length n with the generator polynomial
• g(x) = g0 + g1x + … + grxr.
• Then dim (C) = n - r and a generator matrix G1 for C is
 g 0 g1 g 2 ... gr 0 0 0 ... 0
 
 0 g 0 g1 g 2 ... gr 0 0 ... 0 
G1   0 0 g 0 g1 g2 ... gr 0 ... 0 
 
 .. .. .. 
Proof  0 0 ... 0 0 ... 0 g0 ... g r 

(i) All rows of G1 are linearly independent.
(ii) The n - r rows of G represent codewords
g(x), xg(x), x2g(x),…, xn -r -1g(x)
(*)
(iii) It remains to show that every codeword in C can be expressed as a linear combination of
vectors from (*).
Inded, if a(x)  C, then
a(x) = q(x)g(x).
Since deg a(x) < n we have deg q(x) < n - r.
Hence
q(x)g(x) = (q0 + q1x + … + qn -r -1xn -r -1)g(x)
= q0g(x) + q1xg(x) + … + qn -r -1xn -r -1g(x).
Cyclic codes
IV054 EXAMPLE
•The task is to determine all ternary codes of length 4 and generators for them.
•Factorization of x4 - 1 over GF(3) has the form
• x4 - 1 = (x - 1)(x3 + x2 + x + 1) = (x - 1)(x + 1)(x2 + 1)
•Therefore there are 23 = 8 divisors of x4 - 1 and each generates a cyclic code.
• Generator polynomial Generator matrix
• 1 I4
• x  1 1 0 0
0  1 1 0

 0 0  1 1
• x+1 1 1 0 0
0 1 1 0

0 0 1 1
• x2 + 1
1 0 1 0
0 1 0 1
• (x - 1)(x + 1) = x2 - 1 
 1 0 1 0
0  1 0 1 
• (x - 1)(x2 + 1) = x3 - x2 + x - 1 [ -1 1 -1 1 ] 
• (x + 1)(x2 + 1) [1111]
• x4 - 1 = 0 [0000]
Cyclic codes
IV054
Check polynomials and parity check matrices for cyclic codes
•Let C be a cyclic [n,k]-code with the generator polynomial g(x) (of degree n - k). By the last theorem
g(x) is a factor of xn - 1. Hence
• xn - 1 = g(x)h(x)
•for some h(x) of degree k (where h(x) is called the check polynomial of C).
•Theorem Let C be a cyclic code in Rn with a generator polynomial g(x) and a check polynomial h(x).
Then an c(x)  Rn is a codeword of C if c(x)h(x)  0 - this and next congruences are modulo xn - 1.
Proof Note, that g(x)h(x) = xn - 1  0

(i) c(x)  C  c(x) = a(x)g(x) for some a(x)  Rn
 c(x)h(x) = a(x) g(x)h(x)  0.
 0
(ii) c(x)h(x)  0
c(x) = q(x)g(x) + r(x), deg r(x) < n – k = deg g(x)
c(x)h(x)  0  r(x)h(x)  0 (mod xn - 1)
Since deg (r(x)h(x)) < n – k + k = n, we have r(x)h(x) = 0 in F[x] and therefore
 c(x)
r(x) = 0Cyclic = q(x)g(x)  C.
codes
IV054
POLYNOMIAL REPRESENTATION of DUAL CODES
•Since dim (h(x)) = n - k = dim (C) we might easily be fooled to think that the check
polynomial h(x) of the code C generates the dual code C.
•Reality is “slightly different'':
•Theorem Suppose C is a cyclic [n,k]-code with the check polynomial

• h(x) = h0 + h1x + … + hkxk,
•then
•(i) a parity-check matrix for C is
 hk hk 1 ... h0 0 ... 0
 
0 hk ... h1 h0 ... 0
H  
.. ..
 
0 0 ... 0 hk ... h0 

•(ii) C is the cyclic code generated by the polynomial
h x   hk  hk 1 x  ...  h0 x k
•i.e. the reciprocal polynomial of h(x).
Cyclic codes
IV054 POLYNOMIAL REPRESENTATION of DUAL CODES
•Proof A polynomial c(x) = c0 + c1x + … + cn -1xn –1 represents a code from C if c(x)h(x) = 0.

For c(x)h(x) to be 0 the coefficients at xk,…, xn -1 must be zero, i.e.
c0 hk  c1hk 1  ...  ck h0  0
c1hk  c2 hk 1  ...  ck 1h0  0
.. ..
cn  k 1hk  cn  k hk 1  ...  cn 1h0  0
•Therefore, any codeword c0 c1… cn -1  C is orthogonal to the word hk hk -1…h000…0 and to
its cyclic shifts.
•Rows of the matrix H are therefore in C. Moreover, since hk = 1, these row-vectors are
linearly independent. Their number is n - k = dim (C). Hence H is a generator matrix for
C, i.e. a parity-check matrix for C.
•In order to show that C is a cyclic code generated by the polynomial
h x   hk  hk 1 x  ...  h0 x k
•it is sufficient to show that h x  is a factor of xn -1.
k
 
•Observe that h x   x h x and since
1
h(x -1)g(x -1) = (x -1)n -1
•we have that xkh(x -1)xn -kg(x -1) = xn(x –n -1) = 1 – xn
•and therefore h x  is indeed a factor of xn -1.
Cyclic codes
IV054
ENCODING with CYCLIC CODES I
ENCODING
•Encoding using with CYCLIC CODES I
a cyclic code can be done by a multiplication of two polynomials - a
message polynomial and the generating polynomial for the cyclic code.
•Let C be an (n,k)-code over an field F with the generator polynomial

•g(x) = g0 + g1 x + … + gr –1 x r -1 of degree r = n - k.
•If a message vector m is represented by a polynomial m(x) of degree k and m is encoded

by
• m  c = mG1,
•then the following relation between m(x) and c(x) holds

• c(x) = m(x)g(x).
•Such an encoding can be realized by the shift register shown in Figure below, where
input is the k-bit message to be encoded followed by n - k 0' and the output will be the
encoded message.
•Shift-register encodings of cyclic codes. Small circles represent multiplication by the

corresponding constant,  nodes represent modular addition, squares are delay elements
Cyclic codes
IV054 ENCODING of CYCLIC CODES II
•Another method for encoding of cyclic codes is based on the following (so called
systematic) representation of the generator and parity-check matrices for cyclic codes.
•Theorem Let C be an (n,k)-code with generator polynomial g(x) and r = n - k. For i = 0,1,
…,k - 1, let G2,i be the length n vector whose polynomial is G2,i(x) = x r+I -x r+I mod g(x). Then
the k * n matrix G2 with row vectors G2,I is a generator matrix for C.
•Moreover, if H2,J is the length n vector corresponding to polynomial H2,J(x) = xj mod g(x),
then the r * n matrix H2 with row vectors H2,J is a parity check matrix for C. If the message
vector m is encoded by
• m  c = mG2,
•then the relation between corresponding polynomials is

• c(x) = xrm(x) - [xrm(x)] mod g(x).
•On this basis one can construct the following shift-register encoder for the case of a
systematic representation of the generator for a cyclic code:
•Shift-register encoder for systematic representation of cyclic codes. Switch A is closed for
first k ticks and closed for last r ticks; switch B is down for first k ticks and up for last r ticks.
Cyclic codes
IV054 Hamming codes as cyclic codes
•Definition (Again!) Let r be a positive integer and let H be an r * (2r -1)
matrix whose columns are distinct non-zero vectors of V(r,2). Then the
code having H as its parity-check matrix is called binary Hamming
code denoted by Ham (r,2).
•It can be shown that binary Hamming codes are equivalent to cyclic
codes.
Theorem The binary Hamming code Ham (r,2) is equivalent to a cyclic code.
Definition If p(x) is an irreducible polynomial of degree r such that x is a primitive element

of the field F[x] / p(x), then p(x) is called a primitive polynomial.
Theorem If p(x) is a primitive polynomial over GF(2) of degree r, then the cyclic code
p(x) is the code Ham (r,2).
Cyclic codes
IV054
Hamming codes as cyclic codes
•Example Polynomial x3 + x + 1 is irreducible over GF(2) and x is

primitive element of the field F2[x] / (x3 + x + 1).
•F2[x] / (x3 + x + 1) =
•{0, x, x2, x3 = x + 1, x4 = x2 + x, x5 = x2 + x + 1, x6 = x2 + 1}
•The parity-check matrix for a cyclic version of Ham (3,2)

1 0 0 1 0 1 1
 
H  0 1 0 1 1 1 0
0 0 1 0 1 1 1
 
Cyclic codes
IV054
PROOF of THEOREM
•The binary Hamming code Ham (r,2) is equivalent to a cyclic code.
•It is known from algebra that if p(x) is an irreducible polynomial of degree r, then the ring F2[x] / p(x) is a field of
order 2r.
•In addition, every finite field has a primitive element. Therefore, there exists an element  of F2[x] / p(x) such that
• F2[x] / p(x) = {0, 1, , 2,…, 2r –2}.
•Let us identify an element a0 + a1 + … ar -1xr -1 of F2[x] / p(x) with the column vector
• (a0, a1,…, ar -1)T
•and consider the binary r * (2r -1) matrix

• H = [ 1 2 … 2^r –2 ].
•Let now C be the binary linear code having H as a parity check matrix.
•Since the columns of H are all distinct non-zero vectors of V(r,2), C = Ham (r,2).
•Putting n = 2r -1 we get
• C = {f0 f1 … fn -1  V(n, 2) | f0 + f1 + … + fn -1 n –1 = 0 (2)
• = {f(x)  Rn | f() = 0 in F2[x] / p(x)} (3)
•If f(x)  C and r(x)  Rn, then r(x)f(x)  C because

• r()f() = r()  0 = 0
•and therefore, by one of the previous theorems, this version of Ham (r,2) is cyclic.
Cyclic codes
IV054 BCH codes and Reed-Solomon codes
•To the most important cyclic codes for applications belong BCH codes and Reed-Solomon codes.
•Definition A polynomial p is said to be minimal for a complex number x in Zq if p(x) = 0 and p is

irreducible over Zq.
Definition A cyclic code of codewords of length n over Zq, q = pr, p is a prime, is

called BCH code1 of distance d if its generator g(x) is the least common multiple of
the minimal polynomials for
 l,  l +1,…,  l +d –2
for some l, where
 is the primitive n-th root of unity.
If n = qm - 1 for some m, then the BCH code is called primitive.
primitive
Definition A Reed-Solomon code is a primitive BCH code with n = q - 1.
Properties:
• Reed-Solomon codes are self-dual.
1
BHC stands for Bose and Ray-Chaudhuri and Hocquenghem who discovered
these codes. Cyclic codes
IV054
CONVOLUTION CODES
• Very often it is important to encode an infinite stream or several streams of data – say
bits.
• Convolution codes, with simple encoding and decoding, are quite a simple
• generalization of linear codes and have encodings as cyclic codes.
• An (n,k) convolution code (CC) is defined by an k x n generator matrix,

• entries of which are polynomials over F2
• For example,
G1  [ x 2  1, x 2  x  1]
• is the generator matrix for a (2,1) convolution code CC 1 and
1  x 0 x  1
G2  
 0 
 1 x 
• is the generator matrix for a (3,2) convolution code CC 2
Cyclic codes
IV054 ENCODING of FINITE POLYNOMIALS
• An (n,k) convolution code with a k x n generator matrix G can be usd to encode

a
• k-tuple of plain-polynomials (polynomial input information)
• I=(I0(x), I1(X),…,Ik-1(x))
• to get an n-tuple of crypto-polynomials
• C=(C0(x), C1(x),…,Cn-1(x))
• As follows
• C= I . G
Cyclic codes
EXAMPLES
• EXAMPLE 1
• (x3 + x + 1).G1 = (x3 + x + 1) . (x2 + 1, x2 + x + 1]

• = (x5 + x2 + x + 1, x5 + x4 + 1)
• EXAMPLE 2
 1 0 x  1
( x  x, x  1).G2  ( x  x, x  1).
2 3 2
3
01 x 
Cyclic codes
IV054
ENCODING of INFINITE INPUT STREAMS
• The way infinite streams are encoded using convolution codes will be
• Illustrated on the code CC1.
• An input stream I = (I0, I1, I2,…) is mapped into the output stream
• C= (C00, C10, C01, C11…) defined by
• C0(x) = C00 + C01x + … = (x2 + 1) I(x)

• and
• C1(x) = C10 + C11x + … = (x2 + x + 1) I(x).
• The first multiplication can be done by the first shift register from the next
• figure; second multiplication can be performed by the second shift register
• on the next slide and it holds
• C0i = Ii + Ii+2, C1i = Ii + Ii-1 + Ii-2.
• That is the output streams C0 and C1 are obtained by convolving the input
• stream with polynomials of G1’
Cyclic codes
IV054 ENCODING
The first shift register output

input
1 x x2
will multiply the input stream by x2+1 and the second shift register
output

input
1 x x2
will multiply the input stream by x2+x+1.
Cyclic codes
IV054 ENCODING and DECODING
The following shift-register will therefore be an encoder for the

code CC1
 C00,C01,C02
I 1 x x2 Output streams
 C10,C11,C12
For encoding of convolution codes so called
Viterbi algorithm
Is used.
Cyclic codes
Cyclic Linear Codes
Rong-Jaye Chen

OUTLIN
E
 [1] Polynomials and words
 [2] Introduction to cyclic codes
 [3] Generating and parity check matrices for cyclic codes
 [4] Finding cyclic codes
 [5] Dual cyclic codes

Cyclic Linear Codes
• [1] Polynomials and words
– 1. Polynomial of degree n over K
K [x ]  {a 0  a1x  a 2x 2  a 3x 3  ....  an x n }
a 0 ,...., an  K , deg(f (x ))  n
– 2. Eg 4.1.1
Let f ( x )  1  x  x 3  x 4 g ( x )  x  x 2  x 3 h (x )  1  x 2  x 4 then
(a ) f (x )  g (x )  1  x 2  x 4
(b ) f (x )  h (x )  x  x 2  x 3
(c ) f (x )g (x )  (x  x 2  x 3 )  x (x  x 2  x 3 )  x 3 (x  x 2  x 3 ) 
x 4 (x  x 2  x 3 )  x  x 7

Cyclic Linear Codes
– 3. [Algorithm 4.1.8]Division algorithm
Let f (x ) and h ( x ) be in K [x ] with h ( x )  0. Then there exist

unique polynomial q ( x ) and r ( x ) in K [x ] such that
f ( x )  q ( x )h ( x )  r ( x ),
with r (x )  0 or deg(r ( x ))  deg(h ( x ))

– 4. Eg. 4.1.9
f ( x )  x  x 2  x 6  x 8 , h (x )  1  x  x 2  x 4
q (x )  x 3  x 4 , r (x )  x  x 2  x 3
f ( X )  h (x )(x 3  x 4 )  (x  x 2  x 3 )
deg(r ( x ))  deg(h ( x ))  4

Cyclic Linear Codes
– 5. Code represented by a set of polynomials
• A code C of length n can be represented as a set of polynomials over K of
degree at most n-1
– f ( x )  a  a x  a x 2  ....  a x n 1 over K
0 1 2 n 1
– c  a 0a1a 2 ...a n 1 of length n in K n
– 6. E.g 4.1.12
Codeword c Polynomial c(x)

0000 1+x2
1010 x+x3
0101 1+x+x2+
1111 x3
Cyclic Linear Codes
– 7. f(x) and p(x) are equivalent modulo h(x)
f (x ) mod h (x )  r (x )  p (x ) mod h (x )
ie . f ( x )  p ( x )(mod h ( x ))
– 8.Eg 4.1.15
f (x )  1  x 4  x 9  x 11 , h (x )  1  x 5 , p (x )  1  x 6
f ( x ) mod h ( x )  r ( x )  1  x  p ( x ) mod h ( x )
=>f(x) and p(x) are equivalent mod h(x)!!
– 9. Eg 4.1.16
f (x )  1  x 2  x 6  x 9  x 11 , h (x )  1  x 2  x 5 , p (x )  x 2  x 8
f ( x )mod h ( x )  x  x 4 , p ( x )mod h ( x )  1  x 3
=>f(x) and p(x) are NOT equivalent mod h(x)!!

Cyclic Linear Codes
– 10. Lemma 4.1.17

If f ( x )  g ( x )(mod h ( x )), then
f ( x )  p ( x )  g ( x )  p ( x )(mod h ( x ))
and
f ( x ) p ( x )  g ( x ) p ( x )(mod h ( x ))
– 11. Eg. 4.1.18
f ( x )  1  x  x 7 , g (x )  1  x  x 2 , h ( x )  1  x 5 , p (x )  1  x 6
so f ( x )  g (x )(mod h (x )), then
f (x )  p (x ) and g ( x )  p ( x ) :
((1  x  x 7 )  (1  x 6 )) mod h (x )  x 2  ((1  x  x 2 )  (1  x 6 ))mod h ( x )
f (x ) p (x ) and g ( x ) p ( x ) :
((1  x  x 7 )(1  x 6 )) mod h ( x )  1  x 3  ((1  x  x 2 )(1  x 6 ))mod h ( x )
Cyclic Linear Codes
• [2]Introduction to cyclic codes

– 1. cyclic shift π(v)
• V: 010110,  (v ) : 001011
v 10110 111000 0000 1011

 (v )
01011 011100 0000 1101
– 2.cyclic code
• A code C is cyclic code(or linear cyclic code) if (1)the cyclic shift of each
codeword is also a codeword and (2) C is a linear code
• C1=(000, 110, 101, 011} is a cyclic code
• C2={000, 100, 011, 111} is NOT a cyclic code
– V=100, =010 is not in C2
 (v )

Cyclic Linear Codes
– 3. Cyclic shiftπis a linear transformation
• Lemma 4.2.3  (v  w )   (v )   (w ),
and  (av )  a  (v ), a  K  {0,1}
Thus to show a linear code C is cyclic
it is enough to show that  (v )  C
for each word v in a basis for C
• S={v, π(v), π2(v), …, π n-1(v)}, and C=<S>,

then v is a generator of the linear cyclic code C

Cyclic Linear Codes
– 4. Cyclic Code in terms of polynomial

v   (v ), v (x )  xv (x )
Eg 4.2.11 v=1101000, n=7, v(x)=1+x+x 3

word polynimial(mod 1+x 7 )
----------- -----------------------------
0110100 xv ( x )  x  x 2  x 4
0011010 x 2v ( x )  x 2  x 3  x 4
0001101 x 3v ( x )  x 3  x 4  x 6
1000110 x 4v ( x )  x 4  x 5  x 7  1  x 4  x 5 mod(1  x 7 )
0100011 x 5v ( x )  x 5  x 6  x 8  x  x 5  x 6 mod(1  x 7 )
1010001 x 6v ( x )  x 6  x 7  x 9  1  x 2  x 6 mod(1  x 7 )

Cyclic Linear Codes
– 5. Lemma 4.2.12
Let C be a cyclic code let v in C. Then for any polynomial a(x),
c(x)=a(x)v(x)mod(1+xn) is a codeword in C
– 6. Theorem 4.2.13
C: a cyclic code of length n,
g(x): the generator polynomial, which is the unique nonzero
polynomial of minimum degree in C.
degree(g(x)) : n-k,
• 1. C has dimension k
• 2. g(x), xg(x), x2g(x), …., xk-1g(x) are a basis for C
• 3. If c(x) in C, c(x)=a(x)g(x) for some polynomial a(x)
with degree(a(x))<k

Cyclic Linear Codes
– 7. Eg 4.2.16
the smallest linear cyclic code C of length 6 containing g(x)=1+x 3 <-> 100100 is
{000000, 100100, 010010, 001001, 110110,
101101, 011011, 111111}
– 8. Theorem 4.2.17
g(x) is the generator polynomial for a linear cyclic code of length n if only if g(x) divides 1+x n
(so 1+xn =g(x)h(x)).
– 9. Corollary 4.2.18
The generator polynomial g(x) for the smallest cyclic code of length n containing
the word v(polynomial v(x)) is g(x)=gcd(v(x), 1+x n)
– 10. Eg 4.2.19
n=8, v=11011000 so v(x)=1+x+x3+x4
g(x)=gcd(1+x+x3+x4 , 1+x8)=1+x2
Thus g(x)=1+x2 is the smallest cyclic linear code containing
v(x), which has dimension of 6.
Cyclic Linear Codes
• [3]. Generating and parity check matrices for cyclic code

– 1. Effective to find a generating matrix
• The simplest generator matrices (Theorem 4.2.13)
 g (x ) 
 xg ( x ) 
G   , n: length of codes, k=n-deg(g(x))
 : 
 k-1 
 x g (x ) 

Cyclic Linear Codes
2. Eg 4.3.2
• C: the linear cyclic codes of length n=7 with generator polynomial
g(x)=1+x+x3, and deg(g(x))=3, => k = 4
g (x )  1  x  x 3 1101000 
xg (x )  x  x 2  x 4 0110100
 G=  
x 2g (x )  x 2  x 3  x 5 0011010
 
x 3g (x )  x 3  x 4  x 6  0001101 

Cyclic Linear Codes
3. Efficient encoding for cyclic codes

Let C be a cyclic code of length n and dimension k
(so the generator polynomial g(x) has degree n - k).
message polynomial a ( x )  a0  a1 x    ak 1 x k 1
( representing source message ( a0 , a1 ,, ak 1 ))
Encoding algorithm : c(x)  a (x) g (x)

more time efficient compared with that of
a general linear code ( c  aG )

Cyclic Linear Codes
– 4. Parity check matrix
• H : wH=0 if only if w is a codeword
• Symdrome polynomial s(x)
– c(x): a codeword, e(x):error polynomial, and w(x)=c(x)+e(x)
– s(x) = w(x) mod g(x) = e(x) mod g(x), because c(x)=a(x)g(x)
– H: i-th row ri is the word of length n-k
=> ri(x)=xi mod g(x)
– wH = (c+e)H => c(x) mod g(x) + e(x) mod g(x) = s(x)

Cyclic Linear Codes
– 5. Eg 4.3.7
• n=7, g(x)=1+x+x3, n-k = 3
r0 (x )  1mod g ( x )  1    100 100 

r1 ( x )  x mod g (x )  x 010
   010  
r2 (x )  x mod g (x )  x
2 2
   001 001
 
r3 (x )  x 3 mod g (x )  1  x    110  H  110 
   011 011
r 4 ( x )  x 4 mod g ( x )  x  x 2  
   111  111 
r5 (x )  x 5 mod g (x )  1  x  x 2
101 
r6 (x )  x 6 mod g ( x )  1  x 2    101

Cyclic Linear Codes
• [4]. Finding cyclic codes

– 1. To construct a linear cyclic code of length n
• Find a factor g(x) of 1+xn, deg(g(x)) = n-k
• Irreducible polynomials
– f(x) in K[x], deg(f(x)) >= 1
– There are no a(x), b(x) such that f(x)=a(x)b(x),
deg(a(x))>=1, deg(b(x))>=1
• For n <= 31, the factorization of 1+xn
(see Appendix B)
• Improper cyclic codes: Kn and {0}

Cyclic Linear Codes
– 2. Theorem 4.4.3
s 2r
if n=2 s then 1+x  (1  x )
r n
– 3. Coro 4.4.4
Let n  2 r s, where s is odd and let 1  x s be
the product of z irreducible polynomials.
Then there are (2 r  1) z  2 proper linear
cyclic codes of length n.

Cyclic Linear Codes
– 4. Idempotent polynomials I(x)
• I(x) = I(x)2 mod (1+xn) for odd n

• Find a “basic” set of I(x)
Ci= { s=2j i (mod n) | j=0, 1, …, r}

where
k
1 = 2r mod n
I (x )   ai c i (x ), ai  {0,1} where ci ( x )   x j
jCi
i 0

Cyclic Linear Codes
– 5. Eg 4.4.12
For n=7,
C 0  {0}, so c 0 ( x )  x 0  1
C1  {1, 2, 4} = C 2  C 4 , so c1 ( x )  x 1  x 2  x 4
C 3  {3, 5, 6} = C 5 = C 7 , so c 2 ( x )  x 3  x 6  x 5
 I(x)=a0c 0 ( x )  a1c1 ( x )  a3c 3 ( x ), ai  {0,1},

I(x)  0
– 6. Theorem 4.4.13
Every cyclic code contains a unique idempotent
polynomial which generates the code.(?)

Cyclic Linear Codes
– 7. Eg. 4.4.14 find all cyclic codes of length 9
C 0  {0}, C1  {1, 2, 4, 8, 7,5}, C3  {3, 6}

 c 0 ( x )  1, c1 ( x )  x  x 2  x 4  x 5  x 7  x 8 , c3 ( x )  x 3  x 6
==> I(x)=a0c 0 ( x )  a1c1 ( x )  a3c 3 ( x )
Idempotent polynomial The generator

I(x) polynomial
g(x)=gcd(I(x), 1+x9)
1 1
x+x2+x4+x5+x7+x8 1+x+x3+x4+x6+x7
x3+x6 1+x3
1+x+x2+x4+x5+x7+x8 1+x+x2
: :
Cyclic Linear Codes
• [5].Dual cyclic codes
– 1. The dual code of a cyclic code is also cyclic
– 2. Lemma 4.5.1
a > a(x), b > b(x) and b’ > b’(x)=xnb(x-1) mod 1+xn
then
a(x)b(x) mod 1+xn = 0 iff πk(a) ． b’=0
for k=0,1,…n-1
– 3. Theorem 4.5.2
C: a linear code, length n, dimension k with generator g(x)
If 1+xn = g(x)h(x) then
C⊥: a linear code , dimension n-k with generator x kh(x-1)

Cyclic Linear Codes
– 4. Eg. 4.5.3
g(x)=1+x+x3, n=7, k=7-3=4
h(x)=1+x+x2+x4
h(x)generator for C⊥ is
g⊥ (x)=x4h(x-1)=x4(1+x-1+x-2+x-4 )=1+x2+x3+x4
– 5. Eg. 4.5.4
g(x)=1+x+x2, n=6, k=6-2=4
h(x)=1+x+x3+x4
h(x)generator for C⊥ is g⊥ (x)=x4h(x-1)=1+x+x3+x4

Modulation, Demodulation and
Coding Course
Period 3 - 2005
Sorour Falahati
Lecture 8

Last time we talked about:
 Coherent and non-coherent detections
 Evaluating the average probability of symbol error for different bandpass

modulation schemes
 Comparing different modulation schemes based on their error

performances.
Lecture 8
Today, we are going to talk about:
• Channel coding
• Linear block codes

– The error detection and correction capability
– Encoding and decoding
– Hamming codes
– Cyclic codes
2005-02-09 Lecture 8
What is channel coding?
• Channel coding:
– Transforming signals to improve communications
performance by increasing the robustness against
channel impairments (noise, interference, fading, ..)
– Waveform coding: Transforming waveforms to
better waveforms
– Structured sequences: Transforming data
sequences into better sequences, having structured
redundancy.
• “Better” in the sense of making the decision
process less subject to errors.
2005-02-09 Lecture 8
Error control techniques
• Automatic Repeat reQuest (ARQ)
– Full-duplex connection, error detection codes
– The receiver sends a feedback to the transmitter,
saying that if any error is detected in the received
packet or not (Not-Acknowledgement (NACK) and
Acknowledgement (ACK), respectively).
– The transmitter retransmits the previously sent
packet if it receives NACK.
• Forward Error Correction (FEC)
– Simplex connection, error correction codes
– The receiver tries to correct some errors
• Hybrid ARQ (ARQ+FEC)
– Full-duplex, error detection and correction codes
2005-02-09 Lecture 8
Why using error correction coding?
– Error performance vs. bandwidth
– Power vs. bandwidth
– Data rate vs. bandwidth PB
– Capacity vs. bandwidth Coded
A
F
Coding gain:
For a given bit-error probability, C B
the reduction in the Eb/N0 that can be
realized through the use of code: D
E
 Eb   Eb  Uncoded

G [dB]   
 [dB]    [dB]
 N0 u  N 0 c Eb / N 0 (dB)
2005-02-09 Lecture 8
Channel models
• Discrete memoryless channels

– Discrete input, discrete output
• Binary Symmetric channels
– Binary input, binary output
• Gaussian channels
– Discrete input, continuous output
2005-02-09 Lecture 8
Linear block codes
Some definitions – cont’d
• Binary field :
– The set {0,1}, under modulo 2 binary addition and
multiplication forms a field.
Addition Multiplication
00  0 00  0
0 1  1 0 1  0
1 0  1 1 0  0
11  0 1 1  1
– Binary field is also called Galois field, GF(2).
2005-02-09 Lecture 8
• Fields :
– Let F be a set of objects on which two operations ‘+’ and ‘.’ are
defined.
– F is said to be a field if and only if
1. F forms a commutative group under + operation. The
additive identity element is labeled “0”.
a, b  F  a  b  b  a  F
2. F-{0} forms a commutative group under . Operation. The
multiplicative identity element is labeled “1”.
a, b  F  a  b  b  a  F
3. The operations “+” and “.” distribute:
a  (b  c)  (a  b)  (a  c)
2005-02-09 Lecture 8
• Vector space:
– Let V be a set of vectors and F a fields of
elements called scalars. V forms a vector space
over F if:
1. Commutative: u, v  V  u  v  v  u  F
2. a  F , v  V  a  v  u  V
3. Distributive:
(a  b)  v  a  v  b  v and a  (u  v)  a  u  a  v
4. Associative: a, b  F , v  V  (a  b)  v  a  (b  v )
5. v  V, 1  v  v
2005-02-09 Lecture 8
– Examples of vector spaces
• The set of binary n-tuples, denoted by Vn
V4  {(0000), (0001), (0010), (0011), (0100), (0101), (0111),

(1000), (1001), (1010), (1011), (1100 ), (1101), (1111 )}
• Vector subspace:
– A subset S of the vector space Vn is called a
subspace if:
• The all-zero vector is in S.
• The sum of any two vectors in S is also in S.
• Example:
{(0000), (0101), (1010), (1111)} is a subspace of V4 .
2005-02-09 Lecture 8
• Spanning set:
– A collection of vectors G   v1 , v 2 ,  , v n,
the linear combinations of which include all vectors in
a vector space V, is said to be a spanning set for V or
to span V.
• Example:
 (1000), (0110), (1100), (0011), (1001) spans V4 .
• Bases:
– A spanning set for V that has minimal cardinality is
called a basis for V.
• Cardinality of a set is the number of objects in the set.
• Example:
 (1000), (0100), (0010), (0001) is a basis for V4 .
2005-02-09 Lecture 8
Linear block codes
• Linear block code (n,k)
– A set C  Vnwith cardinality 2 k is called a linear block
code if, and only if, it is a subspace of the vector Vn
space .
Vk  C  Vn
• Members of C are called code-words.
• The all-zero codeword is a codeword.
• Any linear combination of code-words is a
codeword.
2005-02-09 Lecture 8
Linear block codes – cont’d
mapping Vn
Vk
C
Bases of C
2005-02-09 Lecture 8
• The information bit stream is chopped into blocks of k bits.
• Each block is encoded to a larger block of n bits.
• The coded bits are modulated and sent over channel.
• The reverse procedure is done at the receiver.
Channel
Data block Codeword
encoder
k bits n bits
n-k Redundant bits

k
Rc  Code rate
n
Lecture 8
2005-02-09
• The Hamming weight of vector U, denoted by
w(U), is the number of non-zero elements in
U.
• The Hamming distance between two vectors
U and V, is the number of elements in which
they differ.
d (U, V )  w(U  V )
• The minimum distance of a block code is

d min  min d (U i , U j )  min w(U i )
i j i
2005-02-09 Lecture 8
• Error detection capability is given by
ed 1
min which is defined as the
• Error correcting-capability t of a code,
maximum number of guaranteed correctable errors per codeword, is
 d min  1
t
 2 
2005-02-09 Lecture 8
• For memory less channels, the probability that the
decoder commits an erroneous decoding is
– p is the transition probability or bit error probability

over channel.
• The decoded bit error probability is
n
n j
PM     p (1  p ) n  j
j t 1  j 
1 n n j
PB   j   p (1  p) n  j
n j t 1  j 
2005-02-09 Lecture 8
• Discrete, memoryless, symmetric channel model
1 1-p 1
p
Tx. bits Rx. bits
p
0 1-p 0
– Note that for coded systems, the coded bits are
modulated and transmitted over channel. For example,
for M-PSK modulation on AWGN channels (M>2):
2  2 log 2 M  Ec     2  2 log 2 M  Eb Rc   
p 
Q sin     Q sin  
log 2where
M  isN 0energy per M  coded
log 2 M
bit, given by N0 M 
Ec Ec  Rc Eb
2005-02-09 Lecture 8
Linear block codes –cont’d
mapping Vn
Vk
C
Bases of C
– A matrix G is constructed by taking as its rows the vectors

on the basis, .
{V1 , V2 ,  , Vk }
 v11 v12  v1n 
 V1  
   v21 v22  v2 n 
G    
   
Vk   
 vk 1 vk 2  vkn 
2005-02-09 Lecture 8
• Encoding in (n,k) block code
U  mG  V1 
V 
(u1 , u2 , , un )  (m1 , m2 , , mk )   2 
 
 
Vk 
(u1 , u2 , , un )  m1  V1  m2  V2    m2  Vk
– The rows of G, are linearly independent.
2005-02-09 Lecture 8
• Example: Block code (6,3)
Message vector Codeword
000 000000
 V1  1 1 0 1 0 0 100 110100
G   V2   0 1 1 0 1 0 010 011010
 V3  1 0 1 0 0 1  110 1 011 1 0
001 1 01 0 0 1
101 0 111 0 1
011 1 1 0 011
1 11 0 0 0 111
2005-02-09 Lecture 8
• Systematic block code (n,k)
– For a systematic code, the first (or last) k elements in
the codeword are information bits.
G  [P I k ]
I k  k  k identity matrix
Pk  k  (n  k ) matrix
U  (u1 , u2 ,..., un )  ( p1 , p2 ,..., pn  k , m1 , m2 ,..., mk )
          
parity bits message bits
2005-02-09 Lecture 8
• For any linear code we can find an G
matrix H ( n  k )n , which its rows are
orthogonal to rows of :
GH  0 T
• H is called the parity check matrix and

its rows are linearly independent.
• For systematic linear block codes:
H  [I n  k P T ]
2005-02-09 Lecture 8
Data source Format
m Channel U Modulation
encoding
channel
Channel Demodulation
Data sink Format
decoding r Detection
m̂
r  Ue
r  (r1 , r2 ,...., rn ) received codeword or vector
e  (e1 , e2 ,...., en ) error pattern or vector
• Syndrome testing:
– S is syndrome of r, corresponding to the error
pattern e. S  rH T  eH T
2005-02-09 Lecture 8
• Standard array
1. For row i  2,3,...,2 n, k find a vector in of
minimum weight which is not already Vn listed in the
array.
2. Call this patterne i and form the i : th row as the
corresponding coset
zero
codeword U1 U2  U 2k
e2 e2  U2  e 2  U 2k
coset
   
e 2 n k e 2 n k  U 2  e 2 n k  U 2 k
coset leaders
2005-02-09 Lecture 8
• Standard array and syndrome table decoding
1. Calculate S  rHT
ˆ  r leader,
U
2. Find the coset  eˆ eˆ  ei , corresponding to S.
3. Calculate and corresponding m̂.
– Note that U ˆ  r  eˆ  (U  e)  eˆ  U  (e  eˆ )
• If eˆ  e, error is corrected.
• If ˆ , undetectable decoding error occurs.
ee

codewords
• Example: Standard array for the (6,3) code
000000 110100 011010 101110 101001 011101 110011 000111
000001 110101 011011 101111 101000 011100 110010 000110
000010 110111 011000 101100 101011 011111 110001 000101
000100 110011 011100 101010 101101 011010 110111 000110
001000 111100   
010000 100100 coset
100000 010100 
010001 100101   010110
Coset leaders

Error pattern Syndrome
000000 000 U  (101110) transmitted.
000001 101
r  (001110) is received.
000010 011
The syndrome of r is computed :
000100 110
001000 001 S  rH T  (001110) H T  (100)
010000 010 Error pattern corresponding to this syndrome is
100000 100
eˆ  (100000)
010001 111
The corrected vector is estimated
ˆ  r  eˆ  (001110)  (100000)  (101110)
U

Hamming codes
• Hamming codes
– Hamming codes are a subclass of linear block codes and
belong to the category of perfect codes.
– Hamming codes are expressed as a function of a single
integer .
m2
Code length : n  2m  1
Number of information bits : k  2 m  m  1
Number of parity bits : n-k  m
Error correction capability : t  1
– The columns of the parity-check matrix, H, consist of all
non-zero binary m-tuples.

Hamming codes
• Example: Systematic Hamming code (7,4)
1 0 0 0 1 1 1
H  0 1 0 1 0 1 1  [I 33 PT ]
0 0 1 1 1 0 1
0 1 1 1 0 0 0 
1 0 1 0 1 0 0
G   [P I 44 ]
1 1 0 0 0 1 0
 
1 1 1 0 0 0 1 

Cyclic block codes
• Cyclic codes are a subclass of linear block codes.

• Encoding and syndrome calculation are easily
performed using feedback shift-registers.
– Hence, relatively long block codes can be
implemented with a reasonable complexity.
• BCH and Reed-Solomon codes are cyclic codes.

Cyclic block codes
• A linear (n,k) code is called a Cyclic code if all cyclic

shifts of a codeword are also a codeword.
– Example:
U  (u0 , u1 , u2 ,..., un 1 ) “i” cyclic shifts of U
U (i )
 (un i , un i 1 ,..., un 1 , u0 , u1 , u2 ,..., un i 1 )
U  (1101)
U (1)  (1110 ) U ( 2)  (0111) U (3)  (1011) U ( 4)  (1101)  U

Cyclic block codes
• Algebraic structure of Cyclic codes, implies expressing
codewords in polynomial form
U( X )  u0  u1 X  u 2 X 2  ...  un 1 X n 1 degree (n-1)
• Relationship between a codeword and its cyclic shifts:
XU( X )  u0 X  u1 X 2  ..., un  2 X n 1  un 1 X n
 un 1  u0 X  u1 X 2  ...  un  2 X n 1  u n 1 X n  un 1
                 
U (1 ) ( X ) u n1 ( X n 1)
 U (1) ( X )  un 1 ( X n  1)
U (1) ( X )  XU( X ) modulo ( X n  1)
– Hence:
By extension
U ( i ) ( X )  X i U( X ) modulo ( X n  1)

Cyclic block codes
• Basic properties of Cyclic codes:
– Let C be a binary (n,k) linear cyclic code
1. Within the set of code polynomials in C, there
is a unique monic polynomial g ( X ) with
minimal degree r  n. g ( X ) is called the
generator polynomials.
g ( X )  g 0  g1 X  ...  g r X r
2. Every code polynomialU( X ) in C, can be
expressed uniquely as U( X )  m ( X )g ( X )
3. The generator polynomial g ( X ) is a factor of
X n 1

Cyclic block codes
4. The orthogonality of G and H in polynomial
form is expressed as g ( X )h( X )  X  .1This
n
means h( X ) is also a factor of X n  1
5. The rowi, i  1,..., k , of generator matrix is

formed by the coefficients of the " i  1"cyclic
shift of the generator polynomial.
 g0 g1  g r 0
 g( X )   
 Xg ( X )   g0 g1  gr 
G      
    
 k 1   g0 g1  g r 
 X g ( X )  
0 g0 g1  g r 

Cyclic block codes
• Systematic encoding algorithm for an (n,k) Cyclic
code:
nk
1. Multiply the message polynomial m( X ) by X
2. Divide the result of Step 1 by the generator

polynomial g ( X ) . Let p( X ) be the reminder.
3. Add p ( X ) to X nk
m( X ) to form the codewordU( X )

Cyclic block codes
• Example: For the systematic (7,4) Cyclic code with
generator polynomial g( X )  1  X  X 3
1. Find the codeword for the message
m  (1011)
n  7, k  4, n  k  3
m  (1011)  m( X )  1  X 2  X 3
X n k m( X )  X 3m( X )  X 3 (1  X 2  X 3 )  X 3  X 5  X 6
Divide X n  k m( X ) by g ( X) :
X 3  X 5  X 6  (1  X  X 2  X 3 )(1  X  X 3 )  1
              
quotient q(X) generator g(X) remainder p ( X )
Form the codeword polynomial :

U ( X )  p( X )  X 3m ( X )  1  X 3  X 5  X 6
U  (1 0 0 1 0 1 1 )
parity bits message bits

Cyclic block codes
2. Find the generator and parity check matrices, G and H,
respectively.
g( X )  1  1 X  0  X 2  1 X 3  ( g 0 , g1 , g 2 , g 3 )  (1101)
1 1 0 1 0 0 0
0 Not in systematic form.
1 1 0 1 0 0
G We do the following:
0 0 1 1 0 1 0 row(1)  row(3)  row(3)
 
0 0 0 1 1 0 1 row(1)  row(2)  row(4)  row(4)
1 1 0 1 0 0 0
0 1 0 0 1 0 1 1
1 1 0 1 0 0
G H  0 1 0 1 1 1 0
1 1 1 0 0 1 0
  0 0 1 0 1 1 1
1 0 1 0 0 0 1
I 33 PT
P I 44

Cyclic block codes
• Syndrome decoding for Cyclic codes:
– Received codeword in polynomial form is given by
Received r ( X )  U ( X )  e( X ) Error
codewor pattern
d
– The syndrome is the reminder obtained by dividing the
received polynomial by the generator polynomial.
r ( X )  q ( X )g ( X )  S ( X ) Syndrome
– With syndrome and Standard array, error is estimated.
• In Cyclic codes, the size of standard array is considerably reduced.

Example of the block codes
PB
8PSK
QPSK
Eb / N 0 [dB]
ADVANTAGE of GENERATOR MATRIX:
 we need to store only the k rows of G instead of 2k

vectors of
the code.
 for the example we have looked at generator array of

(36)
replaces the original code vector of dimensions (8  6).
This is a definite reduction in complexity .

Systematic Linear Block Codes
 Systematic (n,k) linear block codes has such a mapping
that
part of the sequence generated coincides with the k
message
digits.
 Remaining (n-k) digits are parity digits
A systematic linear block code has a generator matrix of

p p  p 1 0  0
the form :  11 12 1,(nk ) 
p p  p 0 1 0
G  P I    21
 22 2,(nk ) 
 k 
     

 pk1 p  p 0 0  1
 k2 k ,(nk ) 

 P is the parity array portion of the generator matrix
pij = (0 or 1)
Ik is the (kk) identity matrix.
 With the systematic generator encoding complexity is

further
reduced since we do not need to store the identity matrix.
p p  p 1 0  0
 11 12 1,(nk ) 
Since U =m G p p  p 0 1 0
u , u ,........,un  m , m ,........,mn    21 22 2,(nk ) 
1 2  1 2  
     

 pk1 p  p 0 0  1
 k2 k ,(nk ) 

 where
u  m p  m p  .......  m p for i  1,....., (n  k )
i 1 1i 2 2i k ki
m for i  (n  k  1),...., n
ink
And the parity bits are
p  m p  m p  ........  m p
1 1 11 2 21 k k1
p  m p  m p  ........  m p
2 1 12 2 22 k k2
p m p m p  ........  m p
nk 1 1,(nk ) 2 2,(nk ) k k ,(nk )
 given the message k-tuple

m= m1,……..,mk
And the general code vector n-tuple
U = u1,u2,……,uk
Systematic code vector is:
U = p1,p2……..,pk , m1,m2,……,mk

Example:
For a (6,3) code the code vectors are described as
1 1 0 1 0 0
 
U  
  m , m , m  0 1 1 0 1 0
 1 2 3  
1 0 1 0 0 1
P I3
 U = m1+m3, m1+m2, m2+m3, m1, m2, m3
u1 , u2 , u3, u 4, u 5, u6

Parity Check Matrix (H)
We define a parity-check matrix since it will enable us
to
decode the received vectors.
For a (kn) generator matrix G

There exists an (k-n) n matrix H
Such that rows of G are orthogonal to the rows of H
i.e G HT = 0
To satisfy the orthogonality requirement H matrix is
written as: H  I PT 
 nk 

1 0 0 

01  0 
Hence  
  
  
I 

nk
  0
0  1 
H 
T

  p p  p 
 P  
1 ,( n  k )

11 12
pp  p
21 22  2 ,( n  k )
  
 
 p
p  p
k1 k2  k ,( n  k )
The product UHT of each code vector is a zero vector.

UH T  p  p , p  p ,......, p p 0
1 1 2 2 nk nk
Once the parity check matrix H is formed we can use it to

test whether a received vector is a valid member of the
codeword set. U is a valid code vector if and only if
UHT=0.

Syndrome Testing
Let r = r1, r2, ……., rn be a received code vector (one of 2n n-
tuples)
Resulting from the transmission of U = u1,u2,…….,un (one of
the 2k n-tuples).
 r=U+e
where e = e1, e2, ……, en is the error vector or error pattern
introduced by the channel
 In space of 2n n-tuples there are a total of (2n –1) potential
nonzero error patterns.
 The SYNDROME of r is defined as:
S = r HT
The syndrome is the result of a parity check performed on r to
determine whether r is a valid member of the codeword set.
 If r contains detectable errors the syndrome has some non-
zero value
 syndrome of r is seen as
S = (U+e) HT = UHT + eHT
 since UHT = 0 for all code words then :

S = eHT
An important property of linear block codes, fundamental to

the
decoding process, is that the mapping between correctable
error
patterns and syndromes is one-to-one.

Parity check matrix must satisfy:
1. No column of H can be all zeros, or else an error

in the
corresponding code vector position would not
affect the
syndrome and would be undetectable
2. All columns of H must be unique. If two columns

are identical errors corresponding to these code
word locations will be indistinguishable.

Example:
Suppose that code vector U = [ 1 0 1 1 1 0 ] is transmitted and
the vector r = [ 0 0 1 1 1 0 ] is received.
Note one bit is in error..
Find the syndrome vector,S, and verify that it is equal to eHT.
(6,3) code has generator matrix G we have seen before:

1 1 0 1 0 0 
G  0 1 1 0 1 0 
 
1 0 1 0 0 1
P I
P is the parity matrix and I is the identity matrix.

 1 0 0 1 0 0
 0   
 0 1 0 
1 0

 0 0 1  
T
H p   0 0 1 
 1,1 p p   
1,2 1,3 1 1 0 
p p p  0 1 1 
 2,1 2,2 2,3 
p  
p p  1 0 1
 3,1 3,2 3,3 
1 0 0 
0 1 0 
 
0 0 1 
S=rH =
T [ 001110 ] 
 1 1 0 
0 1 1 
 
1 0 1 
= [ 1, 1+1, 1+1 ] = [ 1 0 0]
(syndrome of corrupted code vector)
Now we can verify that syndrome of the corrupted code vector is the same as
the syndrome of the error pattern:
S = eHT = [1 0 0 0 0]HT = [ 1 0 0 ]
( =syndrome of error pattern )
Error Correction
Since there is a one-to-one correspondence between correctable
error patterns and syndromes we can correct such error patterns.
Assume the 2n n-tuples that represent possible received vectors
are arranged in an array called the standard array.
1. The first row contains all the code vectors starting with all-
zeros
vector
2. First column contains all the correctable error patterns
The standard
U array
U for a (n,k) code
U is:
 U
1 2 i 2k
e U e  U e  U e
2 2 2 i 2 2k 2
 
e U e 
j i j
 
e U e  U e
2nk 2 2nk 2k 2nk
Each row called a coset consists of an error pattern in the first
column, also known as the coset leader, followed by the code
vectors perturbed by that error pattern.
The array contains 2n n-tuples in the space Vn

each coset consists of 2k n-tuples
2n
 there are k  2nk cosets
2
If the error pattern caused by the channel is a coset leader, the

received vector will be decoded correctly into the transmitted
code vector Ui. If the error pattern is not a coset leader the
decoding will produce an error.

Syndrome of a Coset
If ej is the coset leader of the jth coset then ;
Ui + e j is an n-tuple in this coset
Syndrome of this coset is:

S = (Ui + ej)HT = Ui HT + ejHT
= e jH T
All members of a coset have the same syndrome and in fact the
syndrome is used to estimate the error pattern.

Error CorrectionDecoding
The procedure for error correction decoding is as follows:
1. Calculate the syndrome of r using S = rHT

2. Locate the coset leader (error pattern) , ej, whose
syndrome equals rHT
3. This error pattern is the corruption caused by the channel
4. The corrected received vector is identified as U = r + ej .
We retrieve the valid code vector by subtracting out the
identified error
Note: In modulo-2 arithmetic subtraction is identical to that of

addition

Example:
Locating the error pattern:
For the (6,3) linear block code we have seen before the standard
array can be arranged as:
000000 110100 011010 101110 101001 011101 110011

000111
000001 110101 011011 101111 101000 011100 110010 000110
000010 110110 011000 101100 101011 011111 110001 000101
000100 110000 011110 101010 101101 011001 110111 000011
001000 111100 010010 100110 100001 010101 111011 001111
010000 100100 001010 111110 111001 001101 100011 010111
100000 010100 111010 001110 001001 111101 010011 100111
010001 100101 001011 111111 111000 001100 100010 010110

The valid code vectors are the eight vectors in the first row and
the correctable error patterns are the eight coset leaders in the
first column.
Decoding will be correct if and only if the error pattern caused

by the channel is one of the coset leaders
We now compute the syndrome corresponding to each of the

correctable error sequences by computing ejHT for each coset
leader 1 0 0
0 1 0
 
0 0 1
Se  
j 1 1 0

0 1 1
 
1 0 1

Syndrome lookup table..
error pattern Syndrome

000000 000
000001 101
000010 011
000100 110
001000 001
010000 010
100000 100
010001 111

Error Correction
We receive the vector r and calculate its syndrome S
We then use the syndrome-look-up table to find the
corresponding error pattern.
This error pattern is an estimate of the error, we denote it as ê
The decoder then adds ê to r to obtain an estimate of the
transmitted
code vector û
Û = r + ê = (U + e) + ê = U + (e ê)
If the estimated error pattern is the same as the actual error
pattern that is if ê = e then û =U
If ê  e the decoder will estimate a code vector that was not
transmitted and hence we have an undetectable decoding error.

Example
Assume code vector U = [ 1 0 1 1 1 0 ] is transmitted and the
vector r=[0 0 1 1 1 0] is received.
The syndrome of r is computed as:

S = [0 0 1 1 1 0 ]HT = [ 1 0 0 ]
From the look-up table 100 has corresponding error pattern:
ê = [1 0 0 0 0 0 ]
The corrected vectors is the Û = r + ê = 0 0 1 1 1 0 + 1 0 0 0 0
0
= 1 0 1 1 1 0 (corrected)
In this example actual error pattern is the estimated error
pattern,
Hence û=U

3F4 Error Control Coding
Dr. I. J. Wassell

Introduction
• Error Control Coding (ECC)

– Extra bits are added to the data at the transmitter
(redundancy) to permit error detection or
correction at the receiver
– Done to prevent the output of erroneous bits
despite noise and other imperfections in the
channel
– The positions of the error control coding and
decoding are shown in the transmission model

Transmission Model
Error Modulator
Digital Source
Control
Line X()
(Transmit
Source Encode Coding
r Coding Filter, etc)
Hc() Channe
Transmitter l
N() Noise
Error Demod
+
Digital Source Line
Control (Receive
Sink Decode
Decoding
Decoding
Filter,
Y()
r
etc)
Receiver
Error Models
• Binary Symmetric Memoryless Channel
– Assumes transmitted symbols are binary
– Errors affect ‘0’s and ‘1’s with equal probability
(i.e., symmetric)
– Errors occur randomly and are independent from
bit to bit (memoryless)
1-p
0 0 p is the probability
p
of bit error or the Bit
IN OUT Error Rate (BER) of
p
1 1 the channel
1-p
Error Models
• Many other types
• Burst errors, i.e., contiguous bursts of bit errors
– output from DFE (error propagation)
– common in radio channels
– Insertion, deletion and transposition errors
• We will consider mainly random errors
Error Control Techniques
• Error detection in a block of data
– Can then request a retransmission, known as automatic repeat request
(ARQ) for sensitive data
– Appropriate for
• Low delay channels
• Channels with a return path
– Not appropriate for delay sensitive data, e.g., real time speech and data
• Forward Error Correction (FEC)
– Coding designed so that errors can be corrected at the receiver
– Appropriate for delay sensitive and one-way transmission (e.g., broadcast
TV) of data
– Two main types, namely block codes and convolutional codes. We will only
look at block codes
•
Block Codes
We will consider only binary data
• Data is grouped into blocks of length k bits (dataword)
• Each dataword is coded into blocks of length n bits (codeword), where
in general n>k
• This is known as an (n,k) block code
• A vector notation is used for the datawords and codewords,
– Dataword d = (d1 d2….dk)
– Codeword c = (c1 c2……..cn)
• The redundancy introduced by the code is quantified by the code rate,
– Code rate = k/n
– i.e., the higher the redundancy, the lower the code rate
Block Code - Example

• Dataword length k = 4
• Codeword length n = 7
• This is a (7,4) block code with code rate = 4/7
• For example, d = (1101), c = (1101001)
Error Control Process
Source code Codeword
data chopped (n bits)
10110 1000 into blocks
1 Chann
1000
Datawor el
d (k bits) coder
Codeword +
Datawor possible
d (k bits) errors (n bits)
Chann Chann
el el
decode
Error flags r
• Decoder gives corrected data
• May also give error flags to
– Indicate reliability of decoded data
– Helps with schemes employing multiple layers of error correction
Parity Codes
• Example of a simple block code – Single Parity Check Code
– In this case, n = k+1, i.e., the codeword is the dataword with
one additional bit
– For ‘even’ parity the additional bit is,
q  i 1 d i (mod 2)
k
– For ‘odd’ parity the additional bit is 1-q

– That is, the additional bit ensures that there are an ‘even’ or
‘odd’ number of ‘1’s in the codeword
Parity Codes – Example 1

• Even parity
(i) d=(10110) so,
c=(101101)
(ii) d=(11011) so,
c=(110110)
Parity Codes – Example 2
• Coding table for (4,3) even parity code
Datawor Codewor
0d 0 0 0d 0 0 0
0 0 1 0 0 1 1
0 1 0 0 1 0 1
0 1 1 0 1 1 0
1 0 0 1 0 0 1
1 0 1 1 0 1 0
1 1 0 1 1 0 0
1 1 1 1 1 1 1

Parity Codes
• To decode
– Calculate sum of received bits in block (mod 2)
– If sum is 0 (1) for even (odd) parity then the dataword is the first k bits of
the received codeword
– Otherwise error
• Code can detect single errors
• But cannot correct error since the error could be in any bit
• For example, if the received dataword is (100000) the transmitted dataword
could have been (000000) or (110000) with the error being in the first or
second place respectively
• Note error could also lie in other positions including the parity bit
• Known as a single error detecting code (SED). Only useful if probability
of getting 2 errors is small since parity will become correct again
• Used in serial communications
• Low overhead but not very powerful
• Decoder can be implemented efficiently using a tree of XOR gates

Hamming Distance
• Error control capability is determined by the Hamming distance
• The Hamming distance between two codewords is equal to the number
of differences between them, e.g.,
10011011
11010010 have a Hamming distance = 3
• Alternatively, can compute by adding codewords (mod 2)
=01001001 (now count up the ones)
• The Hamming distance of a code is equal to the minimum Hamming
distance between two codewords
• If Hamming distance is:
1 – no error control capability; i.e., a single error in a received
codeword yields another valid codeword
XXXXXXX X is a valid codeword
Note that this representation is diagrammatic only.
In reality each codeword is surrounded by n codewords. That is,
one for every bit that could be changed

Hamming Distance
2 – can detect single errors (SED); i.e., a single error will yield an
invalid codeword
XOXOXO X is a valid codeword
O in not a valid codeword
See that 2 errors will yield a valid (but incorrect) codeword

3 – can correct single errors (SEC) or can detect double errors (DED)
XOOXOOX X is a valid codeword
O in not a valid codeword
See that 3 errors will yield a valid but incorrect codeword

Hamming Distance - Example
• Hamming distance 3 code, i.e., SEC/DED

– Or can perform single error correction (SEC)
10011011 X This code corrected this

11011011 way
11010011
O This code corrected this
11010010
O X way
X is a valid codeword
O is an invalid codeword

Hamming Distance
• The maximum number of detectable errors is
d min  1
• That is the maximum number of correctable
errors is given by,
 d min  1
t 
 2 
where dmin is the minimum Hamming distance
between 2 codewords and  . means the
smallest integer

Linear Block Codes
• As seen from the second Parity Code example, it is possible to use a
table to hold all the codewords for a code and to look-up the
appropriate codeword based on the supplied dataword
• Alternatively, it is possible to create codewords by addition of other
codewords. This has the advantage that there is now no longer the
need to held every possible codeword in the table.
• If there are k data bits, all that is required is to hold k linearly
independent codewords, i.e., a set of k codewords none of which can
be produced by linear combinations of 2 or more codewords in the set.
• The easiest way to find k linearly independent codewords is to choose
those which have ‘1’ in just one of the first k positions and ‘0’ in the
other k-1 of the first k positions.

Linear Block Codes
• For example for a (7,4) code, only four codewords are
required, e.g.,
1 0 0 0 1 1 0
0 1 0 0 1 0 1
0 0 1 0 0 1 1
0 0 0 1 1 1 1
• So, to obtain the codeword for dataword 1011, the first, third and fourth
codewords in the list are added together, giving 1011010
• This process will now be described in more detail

Linear Block Codes
• An (n,k) block code has code vectors
d=(d1 d2….dk) and
c=(c1 c2……..cn)
• The block coding process can be written as c=dG
where G is the Generator Matrix  a a ... a1n   a1 
11 12
a a ... a  a 
• G   21 22 2n   2 

Thus,  . . ... .   . 
   
k
 ak1 ak 2 ... akn  a k 
c   dia i
i 1
• ai must be linearly independent, i.e.,
Since codewords are given by summations of the ai vectors, then to
avoid 2 datawords having the same codeword the ai vectors must be
linearly independent
Linear Block Codes
• Sum (mod 2) of any 2 codewords is also a codeword, i.e.,
Since for datawords d1 and d2 we have;
d 3  d1  d 2
So, k k k k
c 3   d 3i a i   (d1i  d 2i )a i  d1i a i   d 2i a i
i 1 i 1 i 1 i 1
c3  c1  c 2
• 0 is always a codeword, i.e.,
Since all zeros is a dataword then,
k
c   0 ai  0
i 1

Error Correcting Power of LBC
• The Hamming distance of a linear block code (LBC) is
simply the minimum Hamming weight (number of 1’s or
equivalently the distance from the all 0 codeword) of the
non-zero codewords
• Note d(c1,c2) = w(c1+ c2) as shown previously
• For an LBC, c1+ c2=c3
• So min (d(c1,c2)) = min (w(c1+ c2)) = min (w(c3))
• Therefore to find min Hamming distance just need to
search among the 2k codewords to find the min
Hamming weight – far simpler than doing a pair wise
check for all possible codewords.

Linear Block Codes – example 1
• For example a (4,2) code, suppose;
1 0 1 1 a1 = [1011]
G   
 0 1 0 1 a2 = [0101]
• For d = [1 1], then;
1 0 1 1
 0 1 0 1
c 
_ _ _ _
 1 1 1 0

Linear Block Codes – example 2
• A (6,5) code with

1 0 0 0 0 1
0 1 0 0 0 1
 
G  0 0 1 0 0 1
 
0 0 0 1 0 1

0 0 0 0 1 1

• Is an even single parity code

Systematic Codes
• For a systematic block code the dataword appears unaltered in
the codeword – usually at the start
• The generator matrix has the structure,
k R R=n-k
1 0 .. 0 p11 p12 .. p1R 
0 1 .. 0 p21 p22 .. p2 R 
G   I | P
.. .. .. .. .. .. .. .. 
 
0 0 .. 1 pk1 pk 2 .. pkR 
• P is often referred to as parity bits
• I is k*k identity matrix. Ensures dataword appears as beginning
of codeword
• P is k*R matrix.

Decoding Linear Codes
• One possibility is a ROM look-up table
• In this case received codeword is used as an address
• Example – Even single parity check code;
Address Data
000000 0
000001 1
000010 1
000011 0
……… .
• Data output is the error flag, i.e., 0 – codeword ok,
• If no error, dataword is first k bits of codeword
• For an error correcting code the ROM can also store datawords
• Another possibility is algebraic decoding, i.e., the error flag is computed
from the received codeword (as in the case of simple parity codes)
• How can this method be extended to more complex error detection and
correction codes?

Parity Check Matrix
• A linear block code is a linear subspace Ssub of all length n vectors
(Space S)
• Consider the subset Snull of all length n vectors in space S that are
orthogonal to all length n vectors in Ssub
• It can be shown that the dimensionality of Snull is n-k, where n is the
dimensionality of S and k is the dimensionality of Ssub
• It can also be shown that Snull is a valid subspace of S and
consequently Ssub is also the null space of Snull
• Snull can be represented by its basis vectors. In this case the generator
basis vectors (or ‘generator matrix’ H) denote the generator matrix for
Snull - of dimension n-k = R
• This matrix is called the parity check matrix of the code defined by G,
where G is obviously the generator matrix for Ssub- of dimension k
• Note that the number of vectors in the basis defines the dimension of
• thethe
So subspace
dimension of H is n-k (= R) and all vectors in the null space are
orthogonal to all the vectors of the code
• Since the rows of H, namely the vectors bi are members of the null
space they are orthogonal to any code vector
• So a vector y is a codeword only if yHT=0
• Note that a linear block code can be specified by either G or H
Parity Check Matrix
• So H is used to check if a codeword is valid,
 b11 b12 ... b1n   b1 

b b22 ... b2 n   b 2 
H   21  R=n-k
 . . ... .   . 
   
bR1 bR 2 ... bRn  b R 
• The rows of H, namely, bi, are chosen to be
orthogonal to rows of G, namely ai
• Consequently the dot product of any valid
codeword with any bi is zero

Parity Check Matrix
• This is so since,
k
c   di a i
i 1
and so,
k k
b j .c  b j . d i a i   d i (a i .b j )  0
i 1 i 1
• This means that a codeword is valid (but not necessarily

correct) only if cHT = 0. To ensure this it is required that the rows
of H are independent and are orthogonal to the rows of G
• That is the bi span the remaining R (= n - k) dimensions of the
codespace

Parity Check Matrix
• For example consider a (3,2) code. In this case G has 2 rows, a1 and a2
• Consequently all valid codewords sit in the subspace (in this case a
plane) spanned by a1 and a2
• In this example the H matrix has only one row, namely b1. This vector is
orthogonal to the plane containing the rows of the G matrix, i.e., a1 and
a2
• Any received codeword which is not in the plane containing a1 and a2
(i.e., an invalid codeword) will thus have a component in the direction of
b1 yielding a non- zero dot product between itself and b1

Parity Check Matrix
• Similarly, any received codeword which is in the
plane containing a1 and a2 (i.e., a valid codeword) will
not have a component in the direction of b1 yielding a
zero dot product between itself and b1
c1
a2
c2
a1
c3
b1
Error Syndrome
• For error correcting codes we need a method to compute the
required correction
• To do this we use the Error Syndrome, s of a received
codeword, cr
s = crHT
• If cr is corrupted by the addition of an error vector, e, then
cr = c + e
and
s = (c + e) HT = cHT + eHT
s = 0 + eHT
Syndrome depends only on the error

Error Syndrome
• That is, we can add the same error pattern to different
codewords and get the same syndrome.
– There are 2(n - k) syndromes but 2n error patterns
– For example for a (3,2) code there are 2 syndromes and 8
error patterns
– Clearly no error correction possible in this case
– Another example. A (7,4) code has 8 syndromes and 128
error patterns.
– With 8 syndromes we can provide a different value to
indicate single errors in any of the 7 bit positions as well as
the zero value to indicate no errors
• Now need to determine which error pattern caused the
syndrome

Error Syndrome
• For systematic linear block codes, H is constructed
as follows,
G = [ I | P] and so H = [-PT | I]
where I is the k*k identity for G and the R*R identity
for H
• Example, (7,4) code, dmin= 3
1 0 0 0 0 1 1
0 0 1 1 1 1 0 0 
1
G   I | P  
1 0 0 1 0
 
H  - P T | I  1 0 1 1 0 1 0
0 0 1 0 1 1 0
  1 1 0 1 0 0 1
0 0 0 1 1 1 1

Error Syndrome - Example
• For a correct received codeword cr = [1101001]
In this case,
0 1 1
1 0 1 

1 1 0
 
s  c r H T  1 1 0 1 0 0 1 1 1 1   0 0 0
1 0 0
 
0 1 0
0 0 1 


Error Syndrome - Example
• For the same codeword, this time with an error in the
first bit position, i.e.,
cr = [1101000]
0 1 1
1 0 1

1 1 0
 
s  c r H T  1 1 0 1 0 0 0 1 1 1   0 0 1
1 0 0
 
0 1 0
0 0 1

• In this case a syndrome 001 indicates an error in bit 1 of the
codeword

Comments about H
• The minimum distance of the code is equal to the minimum number of
columns (non-zero) of H which sum to zero
• We can express
 d0 
d 
c r H T  [cr 0 , cr1 ,..., cr n 1 ] 1   cr 0 d 0  cr1d1  ...  cr n 1d n 1
 . 
 
d
 n 1 
Where do, d1, dn-1 are the column vectors of H
• Clearly crHT is a linear combination of the columns of H

• For a codeword with weight w (i.e., w ones), then crHT is a linear
combination of w columns of H.
• Thus we have a one-to-one mapping between weight w codewords and
linear combinations of w columns of H
• Thus the min value of w is that which results in crHT=0, i.e., codeword cr
will have a weight w (w ones) and so dmin = w
Comments about H
• For the example code, a codeword with min weight (dmin = 3) is
given by the first row of G, i.e., [1000011]
• Now form linear combination of first and last 2 cols in H, i.e.,
[011]+[010]+[001] = 0
• So need min of 3 columns (= dmin) to get a zero value of cHT in
this example
Standard Array
• From the standard array we can find the most likely transmitted
codeword given a particular received codeword without having
to have a look-up table at the decoder containing all possible
codewords in the standard array
• Not surprisingly it makes use of syndromes

Standard Array
• The Standard Array is constructed as follows,
c1 (all zero) c2 …… cM s0
e1 c2+e1 …… cM+e1 s1 All patterns in
e2 c2+e2 …… cM+e2 s2 row have same
e3 c2+e3 …… cM+e3 s3 syndrome
… …… …… …… … Different rows
eN c2+eN …… cM+eN sN have distinct
syndromes
• The array has 2k columns (i.e., equal to the number

of valid codewords) and 2R rows (i.e., the number of
syndromes)
Standard Array
• The standard array is formed by initially choosing ei to be,
– All 1 bit error patterns
– All 2 bit error patterns
– ……
• Ensure that each error pattern not already in the array has a
new syndrome. Stop when all syndromes are used
• Imagine that the received codeword (cr) is c2 + e3 (shown in bold in the
standard array)
• The most likely codeword is the one at the head of the column
containing c2 + e3
• The corresponding error pattern is the one at the beginning of the row
containing c2 + e3
• So in theory we could implement a look-up table (in a ROM) which
could map all codewords in the array to the most likely codeword (i.e.,
the one at the head of the column containing the received codeword)
• This could be quite a large table so a more simple way is to use
syndromes

Standard Array
• This block diagram shows the proposed

implementation
s e
Compute Look-up
syndrome table
cr c
+

Standard Array
• For the same received codeword c2 + e3, note that the
unique syndrome is s3
• This syndrome identifies e3 as the corresponding error
pattern
• So if we calculate the syndrome as described previously,
i.e., s = crHT
• All we need to do now is to have a relatively small table
which associates s with their respective error patterns. In
the example s3 will yield e3
• Finally we subtract (or equivalently add in modulo 2
arithmetic) e3 from the received codeword (c2 + e3) to yield
the most likely codeword, c2

Hamming Codes
• We will consider a special class of SEC codes (i.e.,
Hamming distance = 3) where,
– Number of parity bits R = n – k and n = 2R – 1
– Syndrome has R bits
– 0 value implies zero errors
– 2R – 1 other syndrome values, i.e., one for each bit
that might need to be corrected
– This is achieved if each column of H is a different
binary word – remember s = eHT

Hamming Codes
• Systematic form of (7,4) Hamming code is,
1 0 0 0 0 1 1
0 0 1 1 1 1 0 0 
1 0 0 1 0 1
G   I | P  
0 0 1 0 1 1 0
 
H  - P T | I  1 0 1 1 0 1 0
  1 1 0 1 0 0 1
0 0 0 1 1 1 1
• The original form is non-systematic,

1 1 1 0 0 0 0 
1 0 0 1 1 0 0  0 0 0 1 1 1 1
G   H  0 1 1 0 0 1 1
0 1 0 1 0 1 0 
  1 0 1 0 1 0 1
1 1 0 1 0 0 1 
• Compared with the systematic code, the column orders of both
G and H are swapped so that the columns of H are a binary
count
Hamming Codes
• The column order is now 7, 6, 1, 5, 2, 3, 4, i.e., col. 1 in the non-
systematic H is col. 7 in the systematic H.
Hamming Codes - Example
• For a non-systematic (7,4) code
d = 1011
c = 1110000
+ 0101010
+ 1101001
= 0110011
e = 0010000
cr= 0100011
s = crHT = eHT = 011

• Note the error syndrome is the binary address of the bit to be
corrected
Hamming Codes
• Double errors will always result in wrong bit being
corrected, since
– A double error is the sum of 2 single errors
– The resulting syndrome will be the sum of the
corresponding 2 single error syndromes
– This syndrome will correspond with a third single
bit error
– Consequently the ‘corrected’ codeword will now
contain 3 bit errors, i.e., the original double bit
error plus the incorrectly corrected bit!

Bit Error Rates after Decoding
• For a given channel bit error rate (BER), what is the BER after
correction (assuming a memoryless channel, i.e., no burst
errors)?
• To do this we will compute the probability of receiving 0, 1, 2, 3,
…. errors
• And then compute their effect
• Example – A (7,4) Hamming code with a channel BER of 1%, i.e., p = 0.01
P(0 errors received) = (1 – p)7 = 0.9321
P(1 error received) = 7p(1 – p)6 = 0.0659
76 2
P(2 errors received)  p (1  p ) 5  0.002
2
P(3 or more errors) = 1 – P(0) – P(1) – P(2) = 0.000034

• Single errors are corrected, so,

0.9321+ 0.0659 = 0.998 codewords are correctly
detected
• Double errors cause 3 bit errors in a 7 bit
codeword, i.e., (3/7)*4 bit errors per 4 bit dataword,
that is 3/7 bit errors per bit.
Therefore the double error contribution is 0.002*3/7
= 0.000856

• The contribution of triple or more errors will be less

than 0.000034 (since the worst that can happen is
that every databit becomes corrupted)
• So the BER after decoding is approximately
0.000856 + 0.000034 = 0.0009 = 0.09%
• This is an improvement over the channel BER by a
factor of about 11

Perfect Codes
• If a codeword has n bits and we wish to correct up to
t errors, how many parity bits (R) are needed?
• Clearly we need sufficient error syndromes (2R of
them) to identify all error patterns up to t errors
– Need 1 syndrome to represent 0 errors
– Need n syndromes to represent all 1 bit errors
– Need n(n-1)/2 to syndromes to represent all 2 bit
errors
– Need nCe = n!/(n-e)!e! syndromes to represent all e
bit errors

Perfect Codes
• So,
2R  1  n to correct up to 1 error
n(n - 1)
 1 n  to correct up to 2 errors
2
n(n - 1) n(n - 1)(n - 2)
 1 n   to correct up to 3 errors
2 6
If equality then code is Perfect
• Only known perfect codes are SEC Hamming codes and TEC Golay
(23,12) code (dmin=7). Using previous equation yields
23(23 - 1) 23(23 - 1)(23 - 2)

1  23    2048  211  2 ( 2312)
2 6
Summary
• In this section we have

– Used block codes to add redundancy to messages
to control the effects of transmission errors
– Encoded and decoded messages using Hamming
codes
– Determined overall bit error rates as a function of
the error control strategy

Error Correction Codes &
Multi-user Communications
Agenda
• Shannon Theory
• History of Error Correction Code
• Linear Block Codes
• Decoding
• Convolution Codes
• Multiple-Access Technique
• Capacity of Multiple Access
• Random Access Methods
2006/07/07 Wireless Communication Engineering I 270
Shannon Theory: 　　
　 R < C → Reliable communication Redundancy (Parity bits) in
transmitted data stream
　→ error correction capability
Encoding
Block Code Convolutional Code
Code length is fixed Coding Rate is fixed

Decoding
Hard-Decoding Soft-Decoding
Digital Information Analog Information

History of Error Correction Code
• Shannon (1948):
Random Coding, Orthogonal Waveforms
• Golay (1949): Golay Code, Perfect Code
• Hamming (1950):
Hamming Code
(Single Error Correction, Double Error
Detection)
• Gilbert (1952): Gilbert Bound on Coding Rate

• Muller (1954):
Combinatorial Digital Function and
Error Correction Code
• Elias (1954): Tree Code, Convolutional Code
• Reed and Solomon (1960):
Reed-Solomon Code (Maximal Separable Code)
• Hocquenghem (1959) and Bose and
Chaudhuri (1960):
BCH Code (Multiple Error Correction)
• Peterson (1960):
Binary BCH Decoding, Error Location
Polynomial
• Wozencraft and Reiffen (1961):
Sequencial decoding for convolutional code
• Gallager (1962) LDPC
• Fano (1963):
Fano Decoding Algorithm for convolutional code
• Ziganzirov (1966):
Stack Decoding Algorithm for convolutional code
• Forney (1966):
Generalized Minimum Distance Decoding
(Error and Erasure Decoding)
• Viterbi (1967):
Optimal Decoding Algorithm for convulutional code
• Berlekamp (1968): Fast Iterative BCH Decoding
• Forney (1966): Concatinated Code
• Goppa (1970):
Goppa Code (Rational Function Code)
• Justeson (1972):
Justeson Code (Asmptotically Good Code)
• Ungerboeck and Csajka (1976):
Trellis Code Modulation,
Bandwidth-constraint channel
• Goppa (1980): Algebraic-Geometry Code
• Welch and Berlekamp (1983):
Remainder Decoding Algorithm without using
Syndrome
• Araki, Sorger and Kotter (1993):
Fast GMD Decoding Algorithm
• Berrou (1993): Turbo Code,
Parallel concatinated convolutional code

Basics of Decoding
a) Hamming distance d (ci , c j )  2t  1

b) Hamming distance d (ci , c j )  2t
The received vector is denoted by r.
t → errors correctable
Linear Block Codes
(n, k, dmin) code n : code length
k : number of information bits
d min : minimum distance
k n : coding rate
d ↑ Good Error correction capability
k / n ↓ Low rate
r = n-k ↑ → d ↑
　 (n, k, d) Linear Block Code is
　 Linear Subspace with k-dimension in n-dimension linear space.

Arithmetic operations (, , , /) for encoding
and decoding over an finite field GF(Q)
where Q = pr, p: prime number r: positive
integer
Example GF(2):
+ 0 1  0 1
addition 0 0 1 multiplication 0 0 0
1 1 0 1 0 1
XOR AND

[Encoder]
• The Generator Matrix G and the Parity Check Matrix H
k information bits X → encoder G → n-bits codeword C
C  XG
• Dual (n, n - k) code
Complement orthogonal subspace

Parity Check Matrix H = Generator Matrix of Dual code
CHt = 0
GHt = 0

error vector & syndrome
c : codeword vector
e : error vector
r : received vector (after Hard decision)
s : syndrome
s  rH t   c  e  H t  eH t
s  e (decoding process)

[Minimum Distance]
Singleton Bound
If no more than dmin - 1 columns of H are linearly
independent.
dmin ≤ n - k + 1 (Singleton Bound)
Maximal Separable Code:
dmin = n - k + 1, e.g. Reed-Solomon Code
• Some Specific Linear Block Codes

– Hamming Code (n, k, dmin) = (2m - 1, 2m - 1 - m, 3)
– Hadamard Code (n, k, dmin) = (2m, m + 1, 2m - 1)

Easy Encoding
• Cyclic Codes
C = (cn -1, …, c0) is a codeword → (cn - 2, …, c0, cn
- 1) is also a codeword
Codeword polynomial: C(p) = cn - 1 pn - 1 + ...+ cn
pC  p  mod p  1  Cyclic Shift

n

Encoding: Message polynomial
X  p   xk 1 p    xn 
k 1
Codeword polynomial C  p   X  p  g  p 
where g(p): generator polynomial of degree nk
p 1  g  p  h p 
n
h(p): Parity polynomial
　 Encoder is implemented by Shift registers.

Encoder for an (n, k) cyclic code.
Syndrome calculator for (n, k) cyclic code.
Digital to Analog (BPSK)

c  1  s  1
0  s  1
 s  2c  1
Soft-Decoding & Maximum Likelihood
( k)
r = s +n
(
= r1 , , rn)
= ( s ,, s ) + ( n ,, n )
1 n 1 n
  k 
Prob r s : Likelihood
 
Max Prob r s
k

  Min r  s 
k
k
k 2
 Max : Correlation r, c 

  k
k
• Optimum Soft-Decision Decoding of Linear
Block Codes
Optimum receiver has M = 2k Matched Filter →
M correlation metrics
C  r, Ci     2cij  1 rj
n
j 1
Ci : i - th codeword
wher c : j - th position bit of the i - th codeword
ij
e
rj : j - th received signal
→ Largest matched filter output is selected.

Error probability for soft-decision decoding
(Coherent PSK)
PM  exp   b Rc d min  k ln 2
 b : SNR per bit
wher R : Coding rate   k n 
c
e
Uncoded binary PSK
1
Pe  exp   b 
2
Coding gain:
Cg  10 log Rc d min  k ln 2  b 
dmin ↑ → Cg ↑

• Hard-Decision Decoding
Discrete-time channel =
modulator + AWGN channel + demodulator
→ BSC with crossover probability
 
p  Q 2 b Rc : coherent PSK
 
Q  b Rc : coherent FSK
1
2 exp  12  b Rc  : noncoherent FSK
Maximum-Likelihood Decoding →
Minimum Distance Decoding
Syndrome Calculation by Parity check matrix H
S  YH t
  Cm  e H t
 eH t
where C m : transmitted codeword
Y : received codeword at the demodulator
e : binary error vector
• Comparison of Performance between Hard-Decision and Soft-Decision
Decoding
→ At most 2dB difference
• Bounds on Minimum Distance of Linear Block Codes (Rc vs. dmin)

– Hamming upper bound (2t < dmin)
1 t
n
1  Rc  log 2   
n i 0  i 
– Plot kin upper bound
d min  1  1 2
1  log 2 d min   1  Rc  
n  2d min  2 n

– Elias upper bound
d min
 2 A1  A
n
Rc  1  A log 2 A  1  A log 2 1  A
– Gilbert-Varsharmov lower bound
d min

n
Rc  1  H  

• Interleaving of Coded Data for Channels
with Burst Errors
Multipath and fading channel → burst error
Burst error correction code: Fire code
Correctable burst length b
1 
b    n  k 
2 
Block and Convolution interleave is
effective for burst error.
Convolution Codes
Performance of convolution code > block code
shown by Viterbi’s Algorithm.
 nE ( R )
P (e)  z
E(R) : Error Exponent

Constraint length-3, rate-1/2 convolutional encoder.
• Parameter of convolution code:
Constraint length, K
Minimum free distance
• Optimum Decoding of Convolution
Codes –
The Viterbi Algorithm
For K ≤ 10, this is practical.
• Probability of Error for Soft-Decision
Decoding
Trellis for the convolutional encoder
 a Q 

Pe  d 2 b Rc d
d  d free
where ad : the number of paths of

distance d
• Probability of Error for Hard-Decision
Decoding Hamming distance is a metric
for hard-decision
Turbo Coding

RSC Encoder

Shannon Limit & Turbo Code

Multi-user Communications
Multiple Access Techniques
1. A common communication channel is shared by many users.
up-link in a satellite communication, a set of terminals →
a central computer, a mobile cellular system
2. A broadcast network
down-links in a satellite system, radio and TV broadcast systems
3. Store-and-forward networks
4. Two-way communication systems
-FDMA (Frequency-division Multiple Access)
-TDMA (Time-division Multiple Access)
- CDMA (Code-division Multiple Access):
for burst and low-duty-cycle information transmission
Spread spectrum signals → small cross-correlations
For no spread random access, collision and interference occur.
Retransmission Protocol

Capacity of Multiple Access
Methods
In FDMA, normalized total capacity Cn = KCK / W
(total bit rate for all K users per unit of bandwidth)
 Eb 
Cn  log 2 1  Cn 
 N0 
where W : Bandwidth
Eb : Energy per bit
N 0 : Noise power spectrum desity
Normalized capacity as a function of εb / N0 for FDMA.
Total capacity per hertz as a function of εb / N0 for FDMA.
In TDMA, there is a practical limit for the
transmitter power
In no cooperative CDMA,
1
Cn  log 2 e 
Eb N 0

Normalized capacity as a function of εb / N0 for noncooperative CDMA.

Capacity region for multiple users
Capacity region of two-user CDMA multiple access Gaussian channel.

Code-Division Multiple Access
• CDMA Signal and Channel Models
• The Optimum Receiver
Synchronous Transmission
Asynchronous Transmission
- Sub optimum Detectors
Computational complexity grows linearly
with the number of users, K.
Conventional Single-user Detector
Near-far problem
Decor relation Detector
Minimum Mean-Square-Error Detector
Other Types of Detectors
- Performance Characteristics of Detectors
Random Access Methods
• ALOHA Systems and Protocols Channel access
protocol
synchronized (slotted) ALOHA
unsynchronized (unslotted) ALOHA
Throughput for slotted ALOHA

Throughput & Delay Performance

• Carrier Sense Systems and Protocols
CSMA / CD
(carrier sense multiple access with
collision detection)
No persistent CSMA
1-persistent CSMA
p-persistent CSMA

Chapter 10
Error Detection
and
Correction
10.322
10-1 INTRODUCTION
some issues related, directly or indirectly, to error

detection and correction.
Topics discussed in this section:

Types of Errors
Redundancy
Detection Versus Correction
Modular Arithmetic
10.323
Figure 10.1 Single-bit error
In a single-bit error, only 1 bit in the data unit has

changed.
10.324
Figure 10.2 Burst error of length 8
A burst error means that 2 or more bits in the data unit

have changed.
10.325
Error detection/correction
 Error detection
 Check if any error has occurred
 Don’t care the number of errors
 Don’t care the positions of errors
 Error correction
 Need to know the number of errors
 Need to know the positions of errors
 More difficult
10.326
Figure 10.3 The structure of encoder and decoder
To detect or correct errors, we need to send extra

(redundant) bits with data.
10.327
Modular Arithmetic
 Modulus N: the upper limit
 In modulo-N arithmetic, we use only the
integers in the range 0 to N −1, inclusive.
 If N is 2, we use only 0 and 1
 No carry in the calculation (sum and
subtraction)
10.328
Figure 10.4 XORing of two single bits or two words
10.329
10-2 BLOCK CODING
In block coding, we divide our message into blocks,

each of k bits, called datawords. We add r redundant
bits to each block to make the length n = k + r. The
resulting n-bit blocks are called codewords.

Error Detection
Error Correction
Hamming Distance
Minimum Hamming Distance
10.330
Figure 10.5 Datawords and codewords in block coding
10.331
Example 10.1
The 4B/5B block coding discussed in Chapter 4 is a good

example of this type of coding. In this coding scheme, k =
4 and n = 5.
As we saw, we have 2k = 16 datawords and 2n = 32

codewords. We saw that 16 out of 32 codewords are used
for message transfer and the rest are either used for other
purposes or unused.
10.332
Figure 10.6 Process of error detection in block coding
10.333
Table 10.1 A code for error detection (Example 10.2)
10.334
Figure 10.7 Structure of encoder and decoder in error correction
10.335
Table 10.2 A code for error correction (Example 10.3)
10.336
Hamming Distance
 The Hamming distance between two
words is the number of differences
between corresponding bits.
 The minimum Hamming distance is the
smallest Hamming distance between
all possible pairs in a set of words.
10.337
We can count the number of 1s in the Xoring of two words
1. The Hamming distance d(000, 011) is 2 because
2. The Hamming distance d(10101, 11110) is 3 because
10.338
Example 10.5
Find the minimum Hamming distance of the coding scheme

in Table 10.1.
Solution
We first find all Hamming distances.
The dmin in this case is 2.

10.339
Example 10.6
Find the minimum Hamming distance of the coding scheme

in Table 10.2.
Solution
We first find all the Hamming distances.
The dmin in this case is 3.

10.340
Minimum Distance for
Error Detection
 To guarantee the detection of up to s errors in all cases, the
minimum Hamming distance in a block code must be dmin = s + 1.
 Why?
Example 10.7
•The minimum Hamming distance for our first code scheme (Table 10.1) is 2. This code guarantees
detection of only a single error.
•For example, if the third codeword (101) is sent and one error occurs, the received codeword does not
match any valid codeword. If two errors occur, however, the received codeword may match a valid
codeword and the errors are not detected.
10.341
Example 10.8
•Table 10.2 has dmin = 3. This code can detect up to two

errors. When any of the valid codewords is sent, two errors
create a codeword which is not in the table of valid
codewords. The receiver cannot be fooled.
•What if there are three error occurrance?
10.342
Figure 10.8 Geometric concept for finding dmin in error detection
10.343
Figure 10.9 Geometric concept for finding dmin in error correction
To guarantee correction of up to t errors in all cases, the

minimum Hamming distance in a block code must be dmin
= 2t + 1.
10.344
Example 10.9
A code scheme has a Hamming distance dmin = 4. What is

the error detection and correction capability of this
scheme?
Solution
This code guarantees the detection of up to three errors
(s = 3), but it can correct up to one error. In other words,
if this code is used for error correction, part of its capability is wasted. Error
correction codes need to have an odd minimum distance (3, 5, 7, . . . ).
10.345
10-3 LINEAR BLOCK CODES
•Almost all block codes used today belong to a subset called

linear block codes.
•A linear block code is a code in which the exclusive OR
(addition modulo-2 / XOR) of two valid codewords creates
another valid codeword.
10.346
Example 10.10
Let us see if the two codes we defined in Table 10.1 and

Table 10.2 belong to the class of linear block codes.
1. The scheme in Table 10.1 is a linear block code

because the result of XORing any codeword with any
other codeword is a valid codeword. For example, the
XORing of the second and third codewords creates the
fourth one.
2. The scheme in Table 10.2 is also a linear block code.

We can create all four codewords by XORing two
other codewords.
10.347
Minimum Distance for
Linear Block Codes
 The minimum hamming distance is the number of 1s in
the nonzero valid codeword with the smallest number
of 1s
10.348
Linear Block Codes
 Simple parity-check code
 Hamming codes
Table 10.3 Simple parity-check code C(5, 4)

•A simple parity-check code is a single-bit
error-detecting code in which n = k + 1 with dmin = 2.
•The extra bit (parity bit) is to make the total number
of 1s in the codeword even
•A simple parity-check code can detect an odd
number of errors.
10.349
Figure 10.10 Encoder and decoder for simple parity-check code
10.350
Example 10.12
Let us look at some transmission scenarios. Assume the

sender sends the dataword 1011. The codeword created
from this dataword is 10111, which is sent to the receiver.
We examine five cases:
1. No error occurs; the received codeword is 10111. The

syndrome is 0. The dataword 1011 is created.
2. One single-bit error changes a1 . The received
codeword is 10011. The syndrome is 1. No dataword
is created.
3. One single-bit error changes r0 . The received codeword
is 10110. The syndrome is 1. No dataword is created.
10.351
Example 10.12 (continued)
4. An error changes r0 and a second error changes a3 .

The received codeword is 00110. The syndrome is 0.
The dataword 0011 is created at the receiver. Note that
here the dataword is wrongly created due to the
syndrome value.
5. Three bits—a3, a2, and a1—are changed by errors.
The received codeword is 01011. The syndrome is 1.
The dataword is not created. This shows that the simple
parity check, guaranteed to detect one single error, can
also find any odd number of errors.
10.352
Figure 10.11 Two-dimensional parity-check code
10.353
10.354
10.355
Table 10.4 Hamming code C(7, 4)
1. All Hamming codes discussed in this book have dmin = 3.

2. The relationship between m and n in these codes is
n = 2m − 1.
10.356
Figure 10.12 The structure of the encoder and decoder for a Hamming code
10.357
Table 10.5 Logical decision made by the correction logic analyzer
r0=a2+a1+a0 S0=b2+b1+b0+q0
r1=a3+a2+a1 S1=b3+b2+b1+q1
r2=a1+a0+a3 S2=b1+b0+b3+q2
10.358
Example 10.13
Let us trace the path of three datawords from the sender to the
destination:
1. The dataword 0100 becomes the codeword 0100011.
The codeword 0100011 is received. The syndrome is
000, the final dataword is 0100.
The codeword 0011001 is received. The syndrome is \
011. After flipping b2 (changing the 1 to 0), the final
dataword is 0111.
The codeword 0001000 is received. The syndrome is
101. After flipping b0, we get 0000, the wrong dataword.
This shows that our code cannot correct two errors.
10.359
10-4 CYCLIC CODES
Cyclic codes are special linear block codes with one

extra property. In a cyclic code, if a codeword is
cyclically shifted (rotated), the result is another
codeword.

Cyclic Redundancy Check
Hardware Implementation
Polynomials
Cyclic Code Analysis
Advantages of Cyclic Codes
Other Cyclic Codes
10.360
Table 10.6 A CRC code with C(7, 4)
10.361
Figure 10.14 CRC encoder and decoder
10.362
Figure 10.15 Division in CRC encoder
10.363
Figure 10.16 Division in the CRC decoder for two cases
10.364
Figure 10.21 A polynomial to represent a binary word
10.365
Figure 10.22 CRC division using polynomials
The divisor in a cyclic code is normally called the

generator polynomial or simply the generator.
10.366
10-5 CHECKSUM
The last error detection method we discuss here is

called the checksum. The checksum is used in the
Internet by several protocols although not at the data
link layer. However, we briefly discuss it here to
complete our discussion on error checking

Idea
One’s Complement
Internet Checksum
10.367
Example 10.18
Suppose our data is a list of five 4-bit numbers that we

want to send to a destination. In addition to sending these
numbers, we send the sum of the numbers. For example, if
the set of numbers is (7, 11, 12, 0, 6), we send (7, 11, 12, 0,
6, 36), where 36 is the sum of the original numbers. The
receiver adds the five numbers and compares the result
with the sum. If the two are the same, the receiver assumes
no error, accepts the five numbers, and discards the sum.
Otherwise, there is an error somewhere and the data are
not accepted.
10.368
Example 10.19
We can make the job of the receiver easier if we send the

negative (complement) of the sum, called the checksum. In
this case, we send (7, 11, 12, 0, 6, −36). The receiver can
add all the numbers received (including the checksum). If
the result is 0, it assumes no error; otherwise, there is an
error.
10.369
Example 10.20
How can we represent the number 21 in one’s

complement arithmetic using only four bits?
Solution
The number 21 in binary is 10101 (it needs five bits). We
can wrap the leftmost bit and add it to the four rightmost
bits. We have (0101 + 1) = 0110 or 6.
10.370
Example 10.21
How can we represent the number −6 in one’s

complement arithmetic using only four bits?
Solution
In one’s complement arithmetic, the negative or
complement of a number is found by inverting all bits.
Positive 6 is 0110; negative 6 is 1001. If we consider only
unsigned numbers, this is 9. In other words, the
complement of 6 is 9. Another way to find the complement
of a number in one’s complement arithmetic is to subtract
the number from 2n − 1 (16 − 1 in this case).
10.371
Figure 10.24 Example 10.22
1 1 1 1
0 0 0 0
10.372
Note
Sender site:
1. The message is divided into 16-bit words.
2. The value of the checksum word is set to 0.
3. All words including the checksum are
added using one’s complement addition.
4. The sum is complemented and becomes the
checksum.
5. The checksum is sent with the data.
10.373
Note
Receiver site:
1. The message (including checksum) is
divided into 16-bit words.
2. All words are added using one’s
complement addition.
3. The sum is complemented and becomes the
new checksum.
4. If the value of checksum is 0, the message
is accepted; otherwise, it is rejected.
10.374
Example 10.23
Let us calculate the checksum for a text of 8 characters

(“Forouzan”). The text needs to be divided into 2-byte (16-
bit) words. We use ASCII (see Appendix A) to change each
byte to a 2-digit hexadecimal number. For example, F is
represented as 0x46 and o is represented as 0x6F. Figure
10.25 shows how the checksum is calculated at the sender
and receiver sites. In part a of the figure, the value of
partial sum for the first column is 0x36. We keep the
rightmost digit (6) and insert the leftmost digit (3) as the
carry in the second column. The process is repeated for
each column. Note that if there is any corruption, the
checksum recalculated by the receiver is not all 0s. We
leave this an exercise.
10.375
Figure 10.25 Example 10.23
10.376
Modern Coding Theory: LDPC Codes
Hossein Pishro-Nik
University of Massachusetts Amherst
November 7, 2006
Outline
 Introduction and motivation

 Error control coding
 Block codes
 Minimum distance
 Modern coding: LDPC codes
 Practical challenges
 Application of LDPC codes to
holographic data storage
378
Errors in Information Transmission
Digital Communications: Received bits = Corrupted version

of the transmitted bits
Transporting information from
one party to another, using a
sequence of symbols, e.g. bits. …. 0100010101 …
Noise & interference:

received sequence may be
different from the transmitted
one.
…. 0110010101 …
Transmitted bits 379

Errors in Information Transmission: Cont.
Magnetic recording Track
Sector
Some of the bits may change 010101011110010101001
during the transmission from the
disk to the disk drive
380
Errors in Information Transmission: Cont.
These communication systems can be modeled

as Binary Symmetric Channels (BSC):
Receiver
Sender
BSC
Information bits Corrupted bits
10010…10101… 10110…00101…
0 1-p 0
Each bit is flipped p
with probability p: 0<p<0.5

p
1-p 1
1
381
Pioneers of Coding Theory
Bell Telephone Laboratories
Richard Hamming Claude Shannon

382
Error Control Coding: Repetition Codes
Error Control Coding:

Use redundancy to reduce the bit error rate
10010…10101… BSC 10110…00101…
Bit error probability p=0.01
Example:
Three-fold repetition code: send each bit three times
383
Repetition Codes: Cont.
( x1 ) ( y1 , y2 , y3 )  ( x1 , x1 , x1 )
Encoder: Repeat
each bit three times
codeword
BSC
( x1 ) ( z1 , z 2 , z3 )
Decoder: majority
voting
Corrupted codeword
384
Repetition Codes: Cont.
( x1 )  (0) (0,0,0)
Encoder
codeword
BSC
(1,0,0)
(0)
Decoder
Successful Corrupted codeword

decoding!
Decoding: majority voting

385
Decoding Error
Decoding Error Probability pe :

= Prob{2 or 3 bits in the codeword received in error}
pe  p 3  3 p 2 (1  p),
p  0.01  pe  3 10-4
 Advantage: reduced bite error rate
 Disadvantage: we lose bandwidth because each bit

should be sent three times
386
Error Control Coding: Block Codes
( x1 , x2 ,..., xk ) ( y1 , y 2 ,..., y n )
Encoder
Information block codeword
n>k BSC
( x1 , x2 ,..., xk ) ( z1 , z 2 ,..., z n )
Decoder
Corrupted codeword
 Encoding: mapping the information block to the
corresponding codeword
 Decoding: an algorithm for recovering the

information block from the corrupted codeword
 Always n>k: redundancy in the codeword

387
Error Control Coding: Block Codes
( x1 , x2 ,..., xk ) ( y1 , y 2 ,..., y n )
Encoder
Information block Codeword
BSC
( x1 , x2 ,..., xk ) ( z1 , z 2 ,..., z n )
Decoder
Corrupted codeword
K: code dimension, n: code length
This is called an (n,k) block code
388
Code Rate
 In general an (n,k) block code is a 1-1 mapping

from k bits to n bits: ( x1 , x2 ,..., xk )  (y1 , y2 ,..., yn )
Dimension k
R  Code rate  
Code length n
0  R 1
 R shows the amount of redundancy in the codeword
 Higher R = Lower redundancy
389
Repetition Codes Revisited
For repetition code: k=1, n=3, R=1/3
( x1 ) ( y1 , y 2 , y3 )  ( x1 , x1 , x1 )
Encoder
K=1 n=3
BSC
( x1 ) ( z1 , z 2 , z3 )
Decoder
The repetition code is a (3,1) block code
390
Block Codes: Cont.
There are two valid codewords in the repetition code:

(0)  (0,0,0)
(1)  (1,1,1) Valid codewords
 A (5,3) block code: (n=5, k=3, R=3/5):
(0 0 0)  ( 0 0 0 0 0)
(0 0 1)  (0 1 1 0 0)
(0 1 0)  (1 0 0 1 0)
(0 1 1)  (0 1 0 1 1)
(1 0 0)  (1 0 1 1 1)
8 valid codewords
(1 0 1)  (0 1 1 1 0)
(1 1 0)  (0 1 0 0 1)
(1 1 1)  (1 1 0 0 0)
 The number of valid codewords is equal

.2
k
to the number of possible information blocks,
391
Block Codes: Cont.
( x1 , x2 ,..., xk )  (y1 , y 2 ,..., yn )

Valid codewords
All n-tuples
All k-tuples
(0,0)
(0,1)
(1,0)
(1,1)
2 k points 2 n points
392
Good Block Codes
 There exist more efficient and more powerful

codes.
 Good Codes:
 High rates = lower redundancy (depends on the

channel error rate p)
 Low error rate at the decoder
 Simple and practical encoding and decoding
393
Linear Block Codes
C : ( x1 , x2 ,..., xk )  (y1 , y 2 ,..., y n )
A linear mapping
 g11 g12 ... g1n 
 
 g 21 g 22 .... g 2 n 
( y1 , y2 ,..., y n )  ( x1 , x2 ,..., xk )   . . . . 
 
 . . . . 
 g k1 gk 2 ... g kn 
Linear block codes:

Generator matrix G
 Simple structure: easier to analyze
 Simple encoding algorithms.
394
Linear Block Codes
 There are many practical linear block codes:

 Hamming codes
 Cyclic codes
 Reed-Solomon codes
 BCH codes
 …
395
Channel Capacity
( x1 , x2 ,..., xk ) ( y1 , y2 ,..., y n )
Encoder
Noisy
channel
( x1 , x2 ,..., xk ) ( z1 , z 2 ,..., z n )
Decoder
Channel capacity (Shannon):

The maximum achievable data rate
Shannon capacity is achievable using random codes
396
Shannon Codes
( x1 , x2 ,..., xk )  (y1 , y 2 ,..., yn ) Random mapping

Valid codewords
All n-tuples
All k-tuples
(0,0)
(0,1)
(1,0)
(1,1)
2 k points 2 n points
397
Shannon Random Codes
As n (block length) goes to infinity,

random codes achieve the channel
capacity, i.e,
Code rate R approaches C, while the

decoding error probability goes to zero
398
Error Control Coding:
Low-Density Parity-Check (LDPC) Codes
 Ideal codes
 Have efficient encoding
 Have efficient decoding
 Can approach channel capacity
 Low-density parity-check (LDPC) codes

 Random codes: based on random graphs
 Simple iterative decoding
399
t-Error-Correcting Codes
 The repetition codes can correct one error in the

codeword; however, it fails to correct higher number
of errors.
 A code that is capable of correcting t errors in the

codewords is called a t-error-correcting code.
 The repetition code is a 1-error-correcting code.
400
Minimum Distance
The minimum distance of a code is the minimum

Hamming distance between its codewords:
d min= Min{dist(u,v): u and v are codewords}

For the repetition code, since there are only two
valid codewords,
c1  (0,0,0) and c2  (1,1,1)
d min  dist (c1 , c2 )  3
401
Minimum Distance: Cont.
All vectors of length n (n-tuples)
402
Minimum Distance: Cont.
Higher minimum distance = Stronger code
Example: For repetition code t=1, and d min  3.
403
Modern Coding Theory
• Random linear codes can achieve

channel capacity
• Linear codes can be encoded efficiently
• Decoding of linear codes: NP hard
• Gallager’s idea:
– Find a subclass of random linear codes that
can be decoded efficiently
404
 Iterative coding schemes: LDPC codes, Turbo codes
Encoder
BSC
Iterative
Decoder
 Iterative decoding instead of distance-based decoding
405
Introduction to Channel Coding
 Noisy channels:
Noisy
Information bits Corrupted bits
10010…10101…
channel 10e10…e01e1…
Example: binary erasure channel (BEC)
(0,1) BEC (0,e)
Other channels: Gaussian channel, binary symmetric channel,…
406
Low-Density Parity-Check Codes
 Defined by random sparse graphs (Tanner graphs)
y1  y2  y3  0 Check (message) nodes
Variable (bit) nodes
y1 y2 y3 yn
Simple iterative decoding: message-passing algorithm
407
Important Recent Developments
 Luby et al. and Richardson et al.

 Density evolution
 Optimization using density evolution
 Shokrollahi et al.
 Capacity-achieving LDPC codes for the binary erasure
channel (BEC)
 Richardson et al. and Jin et al.

 Efficient encoding
 Irregular repeat-accumulate codes
408
Standard Iterative Decoding over the BEC
Codeword Received word
01101001  01e0ee01
Standard Iterative Algorithm: Check node
Repeat for any check node

{
If only one of the
neighbors is missing, Neighbors
0 1 e
recover it y1 y2 y3
}
y3  y1  y2  0  1  1
409
Standard Iterative Decoding: Cont.
f f f f
0 1 e =0 e =1 1 e =1
Decoding is successful!
410
Algorithm A: Cont.
 The algorithm may fail
0 e e e 1 1
Stopping Set: S
411
Practical Challenges: Finite-Length Codes
• In practice, we need to use short or

moderate-length codes
• Short or moderate length codes do not
perform as well as long codes
412
Error Floor of LDPC Codes
10-1
Capacity-
approaching LDPC
Codes suffer from
the error floor
10-5
problem
BER
High Error
10 -7
Floor Low Error Floor
10-9
Average erasure probability of the channel

413
Volume Holographic Memory (VHM) Systems
414
Noise and Error Sources
• Thermal noise, shot noise,…

• Limited diffraction
• Aberration
• Misalignment error
• Inter-page interference (IPI)
• Photovoltaic damage
• Non-uniform erasure
415
The Scaling Law
• The magnitudes of systematic errors and thermal noise

are assumed to remain unchanged with respect to M
1
(number of pages) and SNR is proportional to 2
M

SNR  2
M
• SNR decreases as the number of pages increases.
• There exists an optimum number of pages that maximizes

the storage capacity.
416
Raw Error Distribution over a Page
• Bits in different regions

of a page are affected
by different noise
powers.
• The noise power is

higher at edges.
417
Properties and Requirements
• Can use large block lengths
• Non-uniform error correction
12
• Error floor: Target BER< 10
• Simple implementation: Simple decoding
418
Ensembles for Non-uniform Error Correction
Check Nodes
Variable Nodes …
c1 c2 ck
Bits from the first region
419
Ensemble Properties
• Threshold effect
• Concentration
theorem
• Density evolution:
420
Ensemble Properties
• Stability condition (BEC):
421
Design Methodology
• The performance of the decoder is not directly related to

the minimum distance.
• However, the minimum distance still plays an important

role:
– Example: error floor effect
• To eliminate error floor, we avoid degree-two variable

nodes to have large minimum distance.
• For efficient decoding, and also for simplicity of design,

we use low degrees for variable nodes.
422
Performance on VHM
10-2
Rate=.85
Avg degree=6
Gap from capacity

10-5
at BER 1e-9:
0.6dB
BER
n= 105
n= 10
10-7
4
10-9
423
Storage Capacity
Information theoretic capacity
for soft-decision decoding: .95Gb
1
LDPC: soft
.84 Gb
LDPC: hard
0.8 .76 Gb
Storage capacity (Gbits)
RS: hard
.52Gb
0.6
0.4
0.2
0
2000 4000 6000
Number of pages 424
Conclusion
• Carefully designed LDPC codes can result in

significant increase in the storage capacity.
• By incorporating channel information in
design of LDPC codes
- Small gap from capacity
- Error floor reduction
- More efficient decoding
425
 The performance of the decoder is not directly

related to the minimum distance.
 However, the minimum distance still plays an

important role:
 Example: error floor effect
426
Chapter 4
Digital
Transmission
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004
4.1 Line Coding
Some Characteristics
Line Coding Schemes
Some Other Schemes

Figure 4.1 Line coding

Figure 4.2 Signal level versus
data level

Figure 4.3 DC component

Example 1
A signal has two data levels with a pulse duration of 1 ms. We calculate the
pulse rate and bit rate as follows:
Pulse Rate = 1/ 10-3= 1000 pulses/s
Bit Rate = Pulse Rate x log2 L = 1000 x log2 2 = 1000 bps
Example 2
A signal has four data levels with a pulse duration of 1 ms. We calculate the
pulse rate and bit rate as follows:
Pulse Rate = = 1000 pulses/s

Bit Rate = PulseRate x log2 L = 1000 x log2 4 = 2000 bps

Figure 4.4 Lack of
synchronization

Example 3
In a digital transmission, the receiver clock is 0.1 percent

faster than the sender clock. How many extra bits per
second does the receiver receive if the data rate is 1
Kbps? How many if the data rate is 1 Mbps?
Solution
At 1 Kbps:
1000 bits sent 1001 bits received1 extra bps
At 1 Mbps:
1,000,000 bits sent 1,001,000 bits received1000 extra bps

Figure 4.5 Line coding
schemes

Figure 4.6 Unipolar encoding
Note:
Unipolar encoding uses only one voltage level.

Figure 4.7 Types of polar
encoding
Note:
Polar encoding uses two voltage levels (positive and negative).

Note:
In NRZ-L the level of the signal is dependent upon the state of the
bit.
Note:
In NRZ-I the signal is inverted if a

1 is encountered.

Figure 4.8 NRZ-L and NRZ-I
encoding

Figure 4.9 RZ encoding

Note:
A good encoded digital signal

must contain a provision for
synchronization.

Figure 4.10 Manchester
encoding

Note:
In Manchester encoding, the

transition at the middle of the bit
is used for both synchronization
and bit representation.

Figure 4.11 Differential
Manchester encoding

Note:
In differential Manchester encoding, the transition at the middle of

the bit is used only for synchronization.
The bit representation is defined by the inversion or noninversion
at the beginning of the bit.
Note:
In bipolar encoding, we use three levels: positive, zero,

and negative.

Figure 4.12 Bipolar AMI
encoding

Figure 4.13 2B1Q

Figure 4.14 MLT-3 signal

4.2 Block Coding
Steps in Transformation
Some Common Block Codes

Figure 4.15 Block coding

Figure 4.16 Substitution in
block coding

Table 4.1 4B/5B encoding
Data Code Data Code
0000 11110 1000 10010
0001 01001 1001 10011
0010 10100 1010 10110

0011 10101 1011 10111
0100 01010 1100 11010
0101 01011 1101 11011
0110 01110 1110 11100
0111 01111 1111 11101
Table 4.1 4B/5B encoding (Continued)
Data Code
Q (Quiet) 00000
I (Idle) 11111
H (Halt) 00100
J (start delimiter) 11000
K (start delimiter) 10001
T (end delimiter) 01101
S (Set) 11001
R (Reset) 00111
Figure 4.17 Example of 8B/6T
encoding

4.3 Sampling
Pulse Amplitude Modulation

Pulse Code Modulation
Sampling Rate: Nyquist
Theorem
How Many Bits per Sample?
Bit Rate
Figure 4.18 PAM

Note:
Pulse amplitude modulation has some

applications, but it is not used by itself in data
communication. However, it is the first step in
another very popular conversion method called
pulse code modulation.

Figure 4.19 Quantized PAM
signal

Figure 4.20 Quantizing by using sign
and magnitude

Figure 4.21 PCM

Figure 4.22 From analog signal to
PCM digital code

Note:
According to the Nyquist theorem,

the sampling rate must be at least 2
times the highest frequency.

Figure 4.23 Nyquist theorem

Example 4
What sampling rate is needed for a signal with a
bandwidth of 10,000 Hz (1000 to 11,000 Hz)?
Solution
The sampling rate must be twice the highest
frequency in the signal:
Sampling rate = 2 x (11,000) = 22,000 samples/s

Example 5
A signal is sampled. Each sample requires at least 12

levels of precision (+0 to +5 and -0 to -5). How many bits
should be sent for each sample?
Solution
We need 4 bits; 1 bit for the sign and 3 bits for the
value. A 3-bit value can represent 23 = 8 levels (000
to 111), which is more than what we need. A 2-bit
value is not enough since 22 = 4. A 4-bit value is too
much because 24 = 16.

Example 6
We want to digitize the human voice. What is the bit rate, assuming 8 bits per sample?
Solution
The human voice normally contains frequencies from 0 to 4000 Hz.
Sampling rate = 4000 x 2 = 8000 samples/s
Bit rate = sampling rate x number of bits per sample

= 8000 x 8 = 64,000 bps = 64 Kbps
Note:
Note that we can always change a band-pass signal to a low-pass

signal before sampling. In this case, the sampling rate is twice the
bandwidth.

4.4 Transmission Mode
Parallel Transmission
Serial Transmission

Figure 4.24 Data transmission

Figure 4.25 Parallel
transmission

Figure 4.26 Serial
transmission

Note:
In asynchronous transmission, we send 1 start bit (0) at the

beginning and 1 or more stop bits (1s) at the end of each byte.
There may be a gap between each byte.
Note:
Asynchronous here means “asynchronous at the byte level,” but

the bits are still synchronized; their durations are the same.

Figure 4.27 Asynchronous
transmission

Note:
In synchronous transmission,
we send bits one after another
without start/stop bits or gaps.
It is the responsibility of the
receiver to group the bits.

Figure 4.28 Synchronous
transmission

PART III
Data Link Layer

Position of the data-link layer

Data link layer duties

LLC and MAC sublayers

IEEE standards for LANs

Chapters
Chapter 10 Error Detection and Correction

Chapter 11 Data Link Control and Protocols
Chapter 12 Point-To-Point Access
Chapter 13 Multiple Access
Chapter 14 Local Area Networks
Chapter 15 Wireless LANs
Chapter 16 Connecting LANs
Chapter 17 Cellular Telephone and Satellite Networks
Chapter 18 Virtual Circuit Switching

Chapter 10
Error Detection
and
Correction
Note:
Data can be corrupted during

transmission. For reliable
communication, errors must be
detected and corrected.

10.1 Types of Error
Single-Bit Error
Burst Error

Note:
In a single-bit error, only one bit in

the data unit has changed.

10.1 Single-bit error
Note:
A burst error means that 2 or more bits in the data unit have changed.

10.2 Burst error of length 5

10.2 Detection
Redundancy
Parity Check
Cyclic Redundancy Check (CRC)
Checksum

Note:
Error detection uses the concept of

redundancy, which means adding
extra bits for detecting errors at the
destination.

10.3 Redundancy

10.4 Detection methods

10.5 Even-parity concept

Note:
In parity check, a parity bit is added

to every data unit so that the total
number of 1s is even
(or odd for odd-parity).

Example 1
Suppose the sender wants to send the word world. In ASCII the five characters
are coded as
1110111 1101111 1110010 1101100 1100100
The following shows the actual bits sent
11101110 11011110 11100100 11011000 11001001
Example 2
Now suppose the word world in Example 1 is received by the receiver without
being corrupted in transmission.
11101110 11011110 11100100 11011000 11001001
The receiver counts the 1s in each character and comes up with even numbers
(6, 6, 4, 4, 4). The data are accepted.

Example 3
Now suppose the word world in Example 1 is corrupted during transmission.
11111110 11011110 11101100 11011000 11001001
The receiver counts the 1s in each character and comes up with even and odd
numbers (7, 6, 5, 4, 4). The receiver knows that the data are corrupted, discards
them, and asks for retransmission.
Note:
Simple parity check can detect all single-bit errors. It can detect burst
errors only if the total number of errors in each data unit is odd.

10.6 Two-dimensional parity

Example 4
Suppose the following block is sent:
10101001 00111001 11011101 11100111 10101010
However, it is hit by a burst noise of length 8, and some bits are corrupted.
10100011 10001001 11011101 11100111 10101010
When the receiver checks the parity bits, some of the bits do not follow the
even-parity rule and the whole block is discarded.
10100011 10001001 11011101 11100111 10101010
Note:
In two-dimensional parity check, a block of bits is divided into

rows and a redundant row of bits is added to the whole block.
10.7 CRC generator and checker

10.8 Binary division in a CRC
generator

10.9 Binary division in CRC checker

10.10 A polynomial
10.11 A polynomial representing a

divisor

Table 10.1 Standard polynomials
Name Polynomial Application
CRC-8 x 8 + x2 + x + 1 ATM header

CRC-10 x10 + x9 + x5 + x4 + x 2 + 1 ATM AAL
ITU-16 x16 + x12 + x5 + 1 HDLC
x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10
ITU-32 LANs
+ x8 + x7 + x5 + x4 + x2 + x + 1

Example 5
It is obvious that we cannot choose x (binary 10) or x2 + x (binary 110) as the

polynomial because both are divisible by x. However, we can choose x + 1
(binary 11) because it is not divisible by x, but is divisible by x + 1. We can also
choose x2 + 1 (binary 101) because it is divisible by x + 1 (binary division).
Example 6
The CRC-12
x12 + x11 + x3 + x + 1
which has a degree of 12, will detect all burst errors affecting an odd number of
bits, will detect all burst errors with a length less than or equal to 12, and will
detect, 99.97 percent of the time, burst errors with a length of 12 or more.

10.12 Checksum

10.13 Data unit and checksum

Note
:
The sender follows these steps:
•The unit is divided into k sections, each of n bits.
•All sections are added using one’s complement to get the sum.
•The sum is complemented and becomes the checksum.
•The checksum is sent with the data.
Note
:
The receiver follows these steps:
•The sum is complemented.
•If the result is zero, the data are accepted: otherwise, rejected.
Note:
The receiver follows these steps:

•The sum is complemented.
•If the result is zero, the data are accepted: otherwise, rejected.

Example 7
Suppose the following block of 16 bits is to be sent using a
checksum of 8 bits.
10101001 00111001
The numbers are added using one’s complement
10101001
00111001
------------
Sum 11100010
Checksum 00011101
The pattern sent is 10101001 00111001 00011101

Example 8
Now suppose the receiver receives the pattern sent in Example 7 and there is no
error.
10101001 00111001 00011101
When the receiver adds the three sections, it will get all 1s, which, after
complementing, is all 0s and shows that there is no error.
10101001
00111001
00011101
Sum 11111111
Complement 00000000 means that the pattern is OK.

Example 9
Now suppose there is a burst error of length 5 that affects 4 bits.
10101111 11111001 00011101
When the receiver adds the three sections, it gets
10101111
11111001
00011101
Partial Sum 1 11000101
Carry 1
Sum 11000110
Complement 00111001 the pattern is corrupted.

10.3 Correction
Retransmission
Forward Error Correction
Burst Error Correction

Table 10.2 Data and redundancy bits
Number of Number of Total
data bits redundancy bits bits
m r m+r
1 2 3
2 3 5
3 3 6
4 3 7
5 4 9
6 4 10
7 4 11

10.14 Positions of redundancy bits in
Hamming code

10.15 Redundancy bits calculation

10.16 Example of redundancy bit
calculation

10.17 Error detection using Hamming
code

10.18 Burst error correction example

Chapter 11
Chapter 11 : Error-Control
Coding
Lecture edition by K.Heikkinen

Chapter 11 goals
• To understand error-correcting codes in use theorems and their
principles
– block codes, convolutional codes, etc.
Chapter 11 contents
• Introduction
• Discrete Memoryless Channels
• Linear Block Codes
• Cyclic Codes
• Maximum Likelihood decoding of Convolutional Codes
• Trellis-Coded Modulation
• Coding for Compound-Error Channels

Introduction
• Cost-effective facility for transmitting information at a

rate and a level of reliability and quality
– signal energy per bit-to-noise power density ratio
– achieved practically via error-control coding
• Error-control methods
• Error-correcting codes

Discrete Memoryless Channels

Discrete Memoryless Channels
• Discrete memoryless channles (see fig. 11.1)

described by the set of transition probabilities
– in simplest form binary coding [0,1] is used of which
BSC is an appropriate example
– channel noise modelled as additive white gaussian
noise channel
• the two above are so called hard-decision
decoding
– other solutions, so called soft-decision coding

Linear Block Codes
• A code is said to be linear if any twowords in

the code can be added in modulo-2 arithmetic
to produce a third code word in the code
• Linear block code has n bits of which k bits are
always identical to the message sequence
• Then n-k bits are computed from the message
bits in accordance with a prescribed encoding
rule that determines the mathematical structure
of the code
– these bits are also called parity bits

Linear Block Codes
• Normally code equations are written in the form of

matrixes (1-by-k message vector)
– P is the k-by-(n-k) coefficient matrix
– I (of k) is the k-by-k identity matrix
– G is k-by-n generator matrix
• Another way to show the relationship between the
message bits and parity bits
– H is parity-check matrix

Linear Block Codes
• In Syndrome decoding the generator matrix (G) is

used in the encoding at the transmitter and the parity-
check matrix (H) atthe receiver
– if corrupted bit, r = c+e, this leads to two important
properties
• the syndrome is dependant only on the error
pattern, not on the trasmitted code word
• all error patterns that differ by a code word, have
same syndrome

Linear Block Codes
• The Hamming distance (or minimum) can be used to

calculate the difference of the code words
• We have certain amount (2_power_k) code vectors, of
which the subsets constitute a standard array for an
(n,k) linear block code
• We pick the error pattern of a given code
– coset leaders are the most obvious error patterns

Linear Block Codes
• Example : Let us have H as parity-check matrix which

vectors are
– (1110), (0101), (0011), (0001), (1000), (1111)
– code generator G gives us following codes (c) :
• 000000, 100101,111010, 011111
– Let us find n, k and n-k ?
– what will we find if we multiply Hc ?

Linear Block Codes
Examples of (7,4) Hamming code

words and error patterns

Cyclic Codes
• Cyclic codes form subclass of linear block codes

• A binary code is said to be cyclic if it exhibits the two
following properties
– the sum of any two code words in the code is also a code
word (linearity)
• this means that we speak linear block codes
– any cyclic shift of a code word in the code is also a code
word (cyclic)
• mathematically in polynomial notation

Cyclic Codes
• The polynomial plays major role in the generation of cyclic

codes
• If we have a generator polynomial g(x) of an (n,k) cyclic code
with certain k polynomials, we can create the generator
matrix (G)
• Syndrome polynomial of the received code word corresponds
error polynomial

Cyclic Codes

Cyclic Codes
• Example : A (7,4) cyclic code that has a block length of 7, let us find the
polynomials to generate the code (see example 3 on the book)
– find code polynomials
– find generation matrix (G) and parity-check matrix (H)
• Other remarkable cyclic codes
– Cyclic redundancy check (CRC) codes
– Bose-Chaudhuri-Hocquenghem (BCH) codes
– Reed-Solomon codes

Convolutional Codes
• Convolutional codes work in serial manner, which suits better to such kind of
applications
• The encoder of a convolutional code can be viewed as a finite-state machine that
consists of an M-stage shift register with prescribed connections to n modulo-2
adders, and a multiplexer that serializesthe outputs of the address
• Convolutional codes are portrayed in graphical form by using three different

diagrams
– Code Tree
– Trellis
– State Diagram

Maximum Likelihood Decoding of
Convolutional Codes
• We can create log-likelihood function to a convolutional code
that have a certain hamming distance
• The book presents an example algorithm (Viterbi)
– Viterbi algorithm is a maximum-likelihood decoder, which
is optimum for a AWGN (see fig. 11.17)
• initialisation
• computation step
• final step

Trellis-Coded Modulation
• Here coding is described as a process of imposing certain

patterns on the transmitted signal
• Trellis-coded modulation has three features
– Amount of signal point is larger than what is required,
therefore allowing redundancy without sacrificing
bandwidth
– Convolutional coding is used to introduce a certain
dependancy between successive signal points
– Soft-decision decoding is done in the receiver

Coding for Compound-Error
Channels
• Compound-error channels exhibit independent and burst
error statistics (e.g. PSTN channels, radio channels)
• Error-protection methods (ARQ, FEC)

The Logical Domain
Chapter 6
Error Control in the

Binary Channel
Fall 2007 Telecommunications Technology

The Exclusive OR
A
• If A and B are binary
variables, the XOR of A
XOR 0 1
and B is defined as:
0+0=1+1=0
0 0 1
0+1=1+0=1
B
XOR with 1 complements 1 1 0
variable
• Note dij = w(xi XOR xj)
Telecommunications Technology
Hamming Distance
• The weight of a code word is the number of 1’s in it.

• The Hamming Distance between two code words is
equal to the number of digits in which they differ.
• The distance dij between xi = 1110010 and xj =
1011001 is 4.
x1 = 1110010
x2 = 1100001
y = 1100010
d(x1, y) = w(1110010 + 1100010)

= w(0010000)
=1
d(x2, y) = w(110001 + 1100010)

= w(000011)
=2
The Binary Symmetric Channel
1- p
x0 y0
p
p
x1 y1
1- p
BSC and Hamming Distance
• If x is the input code word to a BSC, and y is the

output, y = x + n, where the noise vector n has a 1
wherever an error has occurred:
x= 1110010, n = 0010000 y = 1100010
• An error causes a distance of 1 between input and
output code words.
Geometric Point-of-View
Code set of
all 8, 3-digit 010 011
words
Minimum 110 111
distance = 1 000 001
100 101
In the BSC, any error changes a code word into a code word
Reduced Rate Source
010 011
Code set of
4, 3-digit 110 111
words
000 001
Minimum
distance = 2
100 101
Error Correction and Detection
Capability
• The distance between two code words is the
number of places in which they differ.
• dmin is the distance between the two codes
which are closest together.
• A code with minimum distance dmin may be
used
– to detect dmin-1 errors, or
– to correct (dmin-1)/2 errors.
1 2
x1 y x2
The probability that y came from x1 = Pr{1 error} = pq6
The probability that y came from x2 = Pr{2 errors} = p2q5
p(y/x1) > p(y/x2); p <1/2
Received word is more likely to have come from closest code word
Decode received vector as closest code word => correction
Error Detection and Correction
Shannon’s Noisy Channel Coding Theorem:
• To achieve error-free communications
– Reduce Rate below Capacity
– Add structured redundancy
• Increase the distance between code words

• Add redundancy in a structured way
• For example, add a single parity check digit
• Choose value of the appended digit to make number of 1’s even
(or odd)
Fall 2007
Parity Check Equation
m0 m1 m2 m3 c1
i  n 1
ck  m
i 0
i
A
+ 0 1
Addition is modulo 2 B 0 0 1
1 1 0

An (n,k) Group Code
Code word x7 x6 x5 x4 x3 x2 x1
m4 m3 m2 c3 m1 c2 c1
c1  m1  m2  m4
c2  m1  m3  m4
c3  m2  m3  m4
m4 m3 m2 m1 c3 c2 c1
1 1 1 0 1 0 0
1 1 0 1 0 1 0
1 0 1 1 0 0 1
Parity Check and Generator Matrices
1 0 0 0 1 1 1
0 1 0 0 1 1 0
H
0 0 1 0 1 0 1
0 0 0 1 0 1 1
 1 1 1 0 1 0 0
G 1 1 0 1 0 1 0
1 0 1 1 0 0 1
Codes
 m3 m2 m1 m0
Message m

Code word x x6 x5 x4 x3 x2 x1 x0
 
Transmitted code word x  mH
  
Received word y  xe

Error event e has a 1 wherever an error has occurred.

Syndrome Calculation
  
y  mH  e
  T
s  yG
  T  T
s  mHG  e G
  T
s  eG
Hamming Code
• In the example, n = 7 and k = 4.
• There are r = n - k, or 3, parity check digits.
• This code has a minimum distance of 1, thus all single errors can be
“corrected”.
• If no error occurs the syndrome is 000.
• If more errors occur, the syndrome is another 3-bit sequence.
• Each single error gives a unique syndrome.
• Any single error is more likely to occur than any double, triple, or higher
order error.
• Any non-zero syndrome is most likely to have occurred because the
single error that could cause it occurred, than for any other reason.
• Therefore, deciding that the single error occurred is most likely the
correct decision.
• Hence, the term error correction.
Properties of Binomial Variables
• Given n bits with a probability of error p and a probability of no error q =
1-p.
– The probability of no errors is qn
– The probability of one error is pqn-1
– The probability of k errors is pkqn-k
• It is no problem to show that if p<1/2 then any k-error event is more
likely than any k+1-error event.
– The most likely number of errors is np. When p is very low then the
most likely error event is NO ERRORS, single errors are next most
likely.
• Single-error-correcting codes can be very effective!
Hamming Codes
• Hamming codes are (n,k) group codes where n = k+r is the
length of the code words k is the number of data bits. R is the
number of parity check bits, and 2r = n + 1.
• Typical codes are
• (7,4), r = 3
• (15,11), r = 4 (24 = 16)
• (63, 57), r = 6
• Hamming codes are ideal, single error correcting codes.
Hamming Code Performance
• If the probability of bit error without coding is pu and pc with
coding
 
• the probability of a word error without coding is 1  1  1  pu 
n

 the probability of a word error using a (7,4) Hamming Code is:
1  (1  p )
c
7
 7 pc (1  pc ) 6  
• pu is the uncoded channel error probability.
• pc is the probability of bit error when Eb/No is reduced to 4/7 ths of
that at which pu was calculated.

Cyclic Codes
• Cyclic codes are algebraic group codes in which the code words
form an ideal.
• If the bits are considered coefficients of a polynomial, every code
word is divisible by a generator polynomial.
• The rows of the generator matrix are cyclic permutations of one
another.
An ideal is a group in which every member is the product of two others.
Cyclic Code Generation
A message M = 110011 can be expressed as the polynomial M(X) = X5 + X4

+X+1
(code digits are coefficients of polynomial)
With a generator polynomial P(X) = X4 + X3 + 1, the code word can be

generated as: T(X) = XnM(X) + R(X),
where R(X) is the remainder when XnM(X) is divided by P(X), i.e.

XnM(X)/P(X) = Q(X) + R(X)/P(X),
where n is the order of P(X).
Applications of Cyclic Codes
• Cyclic codes (or cyclic redundancy check CRC) are used routinely to
detect errors in data transmission.
• Typical codes are the
– CRC-16: P(X) = X16 + X15 + X2 + 1
– CRC-CCITT: P(X) = X16 + X12 + X5 + 1
Cyclic Code Capabilities

• A cyclic code will detect:
– All single-bit errors.
– All double-bit errors.
– Any odd number of errors.
– Any burst error for which the length of the burst is less than the
CRC.
– Most larger burst errors.
Convolutional Codes
• Block codes are memoryless codes - each output depends only the current k-bit block
being coded.
• The bits in a convolutional code depend on previous source bits.
• The source bits are convolved with the impulse response of a filter.
Why convolutional codes? Because the code set grows exponentially with code length –
the hypothesis being that the Rate could be maintained as n grew, unlike all block codes
– the Wozencraft contribution.
Convolutional Coder
1 1 0 1 0 1 ... 11 10 11 01 01 01 ...
O1
Input
Xi Xi-1 Xi-2
Encoded output
O2
• Rate 1/2 convolutional coder
0
Trellis Diagram
1
00
01
10
11
1 1 0 1 0 1
11 10 11 01 01 01
00
11
01
01 01 01
10 10
11
11
Decoding
00
01
10
11
Insert an error in a sequence of transmitted bits and try and decode it.
Sequential Decoding
• Decoder determines the most likely output sequence.
• Compares the received sequence with all possible sequences that might have
been obtained with the coder
• Selects the sequence that is closest to the received sequence.
Viterbi Decoding
• Choose a decoding-window width b in excess of the block length.
• Compute all code words of length b and compare each to the received code
word
• Select that code word closest to the received word.
• Re-encode the decoded frame and subtract from the received word.
Turbo Codes
• Turbo codes were invented by Berrou, Clavieux and Thimajshima in 1993
• Turbo codes achieve excellent error correction capabilities at rates very close to
the Shannon bound
• Turbo codes are concatenated or product codes
• Sequential decoding is used.
Interleaved Concatenated Code
Information
Inner checks on information
Outer checks on information
Checks on checks
Coding and Decoding
Inner Coder Outer Coder

(Block) (Convolutional)
Outer Decoder Inner Decoder
Turbo Code Performance
= spectral efficiency = bits per second per Hertz

W
h
The Problem: Noise
Information
Source Transmitter Reciever Destination
Signal Received
Signal
Message Message
Message = [1 1 1 Message = [1 1 0
1] 1]
Noise
NoiseSource
= [0 0
1 0]

W
h
a
Poor solutions
• Repeats –
Single CheckSum -
Data = [1 1 1 1]
• Truth table:
Message=
A B X-OR
0 0 0 [1 1 1 1]
0 1 1 [1 1 1 1]
1 0 1
1 1 0 [1 1 1 1]
• General form:
Data=[1 1 1 1]
Message=[1 1 1 1 0]

W
h
a
Why they are poor t
Repeat 3 times:
Shannon Efficiency •This divide W by 3
•It divides overall capacity by at
C  W log 2 1  S / N  least a factor of 3x.
C is Channel Capacity
W is raw Channel Capacity Single Checksum:
•Allows an error to be detected
S/N is the signal to noise ratio
but requires the message to be
discarded and resent.
•Each error reduces the channel
capacity by at least a factor of 2
because of the thrown away
message.

W
h
a
Hammings Solution t
i
Encoding:
• Multiple Checksums
Message=[a b c d] Message=[1 0 1 0]
r= (a+b+d) mod 2 r=(1+0+0) mod 2 =1
s= (a+b+c) mod 2 s=(1+0+1) mod 2 =0
t= (b+c+d) mod 2
t=(0+1+0) mod 2 =1
Code=[r s a t b c d]
Code=[ 1 0 1 1 0 1 0 ]

W
h
a
Simulation t
i
Stochastic Simulation:
Fig 1: Error Detection s
100%
100,000 iterations
Add Errors to (7,4) data
Percent Errors Detected (%)
90%
No repeat randoms
Measure Error Detection 80%
Results: 70%
Error Detection
•One Error: 100% 60%
•Two Errors: 100%

•Three Errors: 83.43% 50%
1 2 3 4
•Four Errors: 79.76% Errors Introduced

W
h
How it works: 3 dots a
t
Only 3 possible words i

Distance Increment = 1 s
A B C
t
One Excluded State (red)
A B C T
wo valid code words (blue)
It is really a checksum.
 Single Error Detection
 No error correction
A C
This is a graphic representation of the “Hamming Distance”
W
h
a
Hamming Distance t
Definition: i
The number of elements that need to be changed (corrupted) to s
turn one codeword into another.
The hamming distance from: t
• [0101] to [0110] is 2 bits
h
• [1011101] to [1001001] is 2 bits
• “butter” to “ladder” is 4 characters
• “roses” to “toned” is 3 characters

W
h
Another Dot a
t
The code space is now 4. i

The hamming distance is still 1. s
t
h
e
Allows:
Error DETECTION for For Hamming distances greater than 1
Hamming Distance = 1. an error gives a false correction.
Error CORRECTION for
Hamming Distance =1
W
h
Even More Dots a
t
i
s
t
h
e
Allows:
Error DETECTION for • For Hamming distances greater than M
Hamming Distance = 2. 2 an error gives a false correction.
• For Hamming distance of 2 there is
Error CORRECTION for an error detected, but it can not be
Hamming Distance =1. corrected.

W
h
a
Multi-dimensional Codes t
Code Space: i
• 2-dimensional s
• 5 element states
Circle packing makes more t
efficient use of the code-space h
e
M
a

W
h
a
Cannon Balls t
i
s
Efficient Circle packing is the same Efficient Sphere packing is the same
as efficient 2-d code spacing as efficient 3-d code spacing t
h
e
M
a
t
Efficient n-dimensional sphere packing is the same as n-code spacing
• http://wikisource.org/wiki/Cannonball_stacking
• http://mathworld.wolfram.com/SpherePacking.html
W
More on Codes h
a
• Hamming (11,7)
t
• Golay Codes
• Convolutional Codes
• Reed-Solomon Error Correction i
• Turbo Codes s
• Digital Fountain Codes
t
An Example h
We will e
• Encode a message
• Add noise to the transmission
M
• Detect the error
• Repair the error
a
t
r
W
h
Encoding the message a

t
i
To encode our message But why? s
we multiply this matrix
You can verify that: t
1 0 0 0 0 1 1 h
0 e
1
Hamming[1 0 0 0]=[1 0 0 0 0 1 1]
1 0 0 1 0
H  Hamming[0 1 0 0]=[0 1 0 0 1 0 1]
0 0 1 0 1 1 0 Hamming[0 0 1 0]=[0 0 1 0 1 1 0] M
  Hamming[0 0 0 1]=[0 0 0 1 1 1 1] a
0 0 0 1 1 1 1
t
r
By our message i
code  H  message x
?
Where multiplication is the logical AND
And addition is the logical XOR

Add noise
• If our message is 1 0 0 0 0 1 1
Message = [0 1 1 0] 0 1 0 0 1 0 1
   0 1 1 0
0 0 1 0 1 1 0
• Our Multiplying yields  
0 0 0 1 1 1 1
Code = [0 1 1 0 0 1 1]
 0  1 0 0 0 0 1 1
Lets add an error,  1  0 1 0 0 1 0 1
 1   0 0 1 0 1 1 0
so Pick a digit to mutate  0   0 0 0 1 1 1 1
  0 1 0 0 1 0 1
  0 0 1 0 1 1 0
  0 1 1 0 0 1 1
Code => [0 1 0 0 0 1 1]

Testing the message
• We receive the erroneous The matrix used to decode is:
string:
Code = [0 1 0 0 0 1 1] 0 0 0 1 1 1 1
• We test it: Decoder  0 1 1 0 0 1 1
Decoder*CodeT 1 0 1 0 1 0 1
=[0 1 1] To test if a code is valid:
• And indeed it has an error • Does Decoder*CodeT
=[0 0 0]
– Yes means its valid
– No means it has error/s

Repairing the message
• To repair the code we find the • Decoder*codeT is
collumn in the decoder matrix [ 0 1 1] 0 0 0 1 1 1 1
whose elements are the row Decoder  0 1 1 0 0 1 1
results of the test vector 1 0 1 0 1 0 1
• We then change
• This is the third element of our
code
• Our repaired code is
[0 1 1 0 0 1 1]
Decoding the message

We trim our received code by 3 elements and we have our original
message.
[0 1 1 0 0 1 1] => [0 1 1 0]

Channel Coding in
IEEE802.16e
Student: Po-Sheng Wu
Advisor: David W. Lin

Outline
• Overview
• RS code
• Convolution code
• LDPC code
• Future Work
Overview

RS code
• The RS code in 802.16a is derived from a systematic

RS (N=255, K=239, T=8) code on GF(2^8)

RS code

RS code
• This code then is shortened and punctured to enable

variable block size and variable error-correction
capability.
• Shorten ： (n, k) → (n-l, k-l)
• Punctured ： (n, k) → (n-l, k)
• In general, the generator polynomial
in IEEE802.16a h=0

RS code
• They are shortened to K’ data bytes and punctured to permit T’

bytes to be corrected.
• When a block is shortened to K’, the first 239-K’ bytes of the
encoder input shall be zero
• When a codeword is punctured to permit T’ bytes to be
corrected, only the first 2T’ of the total 16 parity bytes shall be
employed.
• When shortened and punctured to (48,36,6) the first 203(239-
36) information bytes are assigned 0.
• And only the first 12(2*6) bytes of R(X) will be employed in the
codeword.

Shortened and Punctured

RS code

RS code
• Decoding : The Euclid’s (Berlekamp) algorithm is a

common decoding algorithm for RS code.
• Four step:
-compute the syndrome value
-compute the error location polynomial
-compute the error location
-compute the error value

Convolution code
• Each RS code is encoded by a binary convolution

encoder, which has native rate of ½, a constraint
length equal to 7.

Convolution code
• “1” means a transmitted bit and “0” denotes a

removed bit, note that the has been changed
from that of the native convolution code with rate ½ .

Convolution code
• Decoding: Viterbi algorithm

Convolution code
• The convolution code in IEEE802.16a need to be

terminated in a block, and thus become a block code.
• Three method to achieve this termination
– Direct truncation
– Zero tail
– Tail biting

RS-CC code
• Outer code: RS code
• Inner code: convolution code
• Input data streams are divided into RS blocks, then each RS
block is encode by a tail-biting convolution code.
• Between the convolution coder and modulator is a bit
interleaver.

LDPC code
• low density parity checks matrix

• LDPC codes also linear codes. The codeword can be
expressed as the null space of H, Hx=0
• Low density enables efficient decoding
– Better decoding performance to Turbo code
– Close to the Shannon limit at long block length

LDPC code
• n is the length of the code, m is the number of parity

check bit

LDPC code
• Base model

LDPC code
• if p(f,i,j) = -1
– replace by z*z zero matrix
else
– p(f,i,j) is the circular shift size
 p (i, j ), p(i,j)  0

p  f , i, j     p(i,j)z f 
 z  , p(i,j)>0
 0 

LDPC code
• Encoding
[u p1 p2]
• Decoding
– Tanner Graph
– Sum Product Algorithm

LDPC code
• Tanner Graph

LDPC code
• Sum Product Algorithm

LDPC code

LDPC code
Future Work
• Realize these algorithm in computer
• Find some decoding algorithm to speed up
Chapter 11
Data Link
Control
and
Protocols
11.1 Flow and Error Control
Flow Control
Flow control refers to a set of procedures used to restrict the amount of
data that the sender can send before waiting for acknowledgment.
Error Control
Error control in the data link layer is based on automatic repeat
request, which is the retransmission of data.

11.2 Stop-and-Wait ARQ
Operation
Bidirectional Transmission

11.1 Normal operation

11.2 Stop-and-Wait ARQ, lost frame

11.3 Stop-and-Wait ARQ, lost ACK frame

Note:
In Stop-and-Wait ARQ, numbering

frames prevents the retaining of
duplicate frames.

11.4 Stop-and-Wait ARQ, delayed ACK

Note:
Numbered acknowledgments are

needed if an acknowledgment is
delayed and the next frame is lost.

11.5 Piggybacking

11.3 Go-Back-N ARQ
Sequence Number
Sender and Receiver Sliding Window

Control Variables and Timers
Acknowledgment
Resending Frames
Operation
11.6 Sender sliding window

11.7 Receiver sliding window

11.8 Control variables

11.9 Go-Back-N ARQ, normal operation

11.10 Go-Back-N ARQ, lost frame

11.11 Go-Back-N ARQ: sender window
size

Note:
In Go-Back-N ARQ, the size of the

sender window must be less than 2m;
the size of the receiver window is
always 1.

11.4 Selective-Repeat ARQ
Sender and Receiver Windows
Operation
Sender Window Size
Bidirectional Transmission
Pipelining

11.12 Selective Repeat ARQ, sender and receiver
windows

11.13 Selective Repeat ARQ, lost frame

Note:
In Selective Repeat ARQ, the size of the

sender and receiver window must be at
most one-half of 2m.

11.14 Selective Repeat ARQ, sender
window size

Example 1
In a Stop-and-Wait ARQ system, the bandwidth of the line is 1 Mbps, and 1 bit
takes 20 ms to make a round trip. What is the bandwidth-delay product? If the
system data frames are 1000 bits in length, what is the utilization percentage of
the link?
Solution
The bandwidth-delay product is
1  106  20  10-3 = 20,000 bits
The system can send 20,000 bits during the time it takes for the data to go from
the sender to the receiver and then back again. However, the system sends only
1000 bits. We can say that the link utilization is only 1000/20,000, or 5%. For
this reason, for a link with high bandwidth or long delay, use of Stop-and-Wait
ARQ wastes the capacity of the link.
Example 2
What is the utilization percentage of the link in Example 1 if the link uses Go-
Back-N ARQ with a 15-frame sequence?
Solution
The bandwidth-delay product is still 20,000. The system can send up to 15
frames or 15,000 bits during a round trip. This means the utilization is
15,000/20,000, or 75 percent. Of course, if there are damaged frames, the
utilization percentage is much less because frames have to be resent.

11.5 HDLC
Configurations and Transfer Modes
Frames
Frame Format
Examples
Data Transparency

11.15 NRM

11.16 ABM

11.17 HDLC frame

11.18 HDLC frame types

11.19 I-frame

11.20 S-frame control field in HDLC

11.21 U-frame control field in HDLC

Table 11.1 U-frame control command and response
Command/response Meaning
SNRM Set normal response mode
SNRME Set normal response mode (extended)
SABM Set asynchronous balanced mode
SABME Set asynchronous balanced mode (extended)
UP Unnumbered poll
UI Unnumbered information
UA Unnumbered acknowledgment
RD Request disconnect
DISC Disconnect
DM Disconnect mode
RIM Request information mode
SIM Set initialization mode
RSET Reset
XID Exchange ID
FRMR Frame reject
Example 3
Figure 11.22 shows an exchange using piggybacking where is no
error. Station A begins the exchange of information with an I-
frame numbered 0 followed by another I-frame numbered 1.
Station B piggybacks its acknowledgment of both frames onto an I-
frame of its own. Station B’s first I-frame is also numbered 0 [N(S)
field] and contains a 2 in its N(R) field, acknowledging the receipt
of A’s frames 1 and 0 and indicating that it expects frame 2 to
arrive next. Station B transmits its second and third I-frames
(numbered 1 and 2) before accepting further frames from station A.
Its N(R) information, therefore, has not changed: B frames 1 and 2
indicate that station B is still expecting A frame 2 to arrive next.

11.22 Example 3

Example 4
In Example 3, suppose frame 1 sent from station B to
station A has an error. Station A informs station B to
resend frames 1 and 2 (the system is using the Go-Back-
N mechanism). Station A sends a reject supervisory frame
to announce the error in frame 1. Figure 11.23 shows the
exchange.

11.23 Example 4

Note:
Bit stuffing is the process of adding one

extra 0 whenever there are five
consecutive 1s in the data so that the
receiver does not mistake the
data for a flag.

11.24 Bit stuffing and removal

11.25 Bit stuffing in HDLC

Chapter 11
Data Link Control

11.648
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
11-1 FRAMING
The data link layer needs to pack bits into frames, so

that each frame is distinguishable from another. Our
postal system practices a type of framing. The simple
act of inserting a letter into an envelope separates one
piece of information from another; the envelope serves
as the delimiter.
Fixed-Size Framing
Variable-Size Framing
11.649
Figure 11.1 A frame in a character-oriented protocol
11.650
Figure 11.2 Byte stuffing and unstuffing
Byte stuffing is the process of adding 1 extra byte

whenever there is a flag or escape character in the text.
11.651
Figure 11.3 A frame in a bit-oriented protocol
Bit stuffing is the process of adding one extra 0

whenever five consecutive 1s follow a 0 in the
data, so that the receiver does not mistake
the pattern 0111110 for a flag.
11.652
Figure 11.4 Bit stuffing and unstuffing
11.653
11-2 FLOW AND ERROR CONTROL
The most important responsibilities of the data link

layer are flow control and error control. Collectively,
these functions are known as data link control.

Flow Control
Error Control
11.654
Note
 Flow control refers to a set of procedures used to

restrict the amount of data that the sender can
send before waiting for acknowledgment.
 Error control in the data link layer is based on

automatic repeat request, which is the
retransmission of data.
11.655
11-3 PROTOCOLS
Now let us see how the data link layer can combine
framing, flow control, and error control to achieve the
delivery of data from one node to another.
The protocols are normally implemented in software by

using one of the common programming languages.
To make our discussions language-free, we have written

in pseudocode a version of each protocol that
concentrates mostly on the procedure instead of delving
into the details of language rules.
11.656
Figure 11.5 Taxonomy of protocols discussed in this chapter
11.657
11-4 NOISELESS CHANNELS
Let us first assume we have an ideal channel in which

no frames are lost, duplicated, or corrupted. We
introduce two protocols for this type of channel.

Simplest Protocol
Stop-and-Wait Protocol
11.658
Figure 11.6 The design of the simplest protocol with no flow or error control
11.659
Figure 11.7 Flow diagram for Example 11.1
Figure 11.7 shows an example

of communication using this
protocol. It is very simple. The
sender sends a sequence of
frames without even thinking
about the receiver. To send
three frames, three events
occur at the sender site and
three events at the receiver site.
Note that the data frames are
shown by tilted boxes; the
height of the box defines the
transmission time difference
between the first bit and the
last bit in the frame.
11.660
Figure 11.8 Design of Stop-and-Wait Protocol
11.661
Figure 11.9 shows an

example of communication
using this protocol. It is still
very simple. The sender
sends one frame and waits
for feedback from the
receiver. When the ACK
arrives, the sender sends the
next frame. Note that
sending two frames in the
protocol involves the sender
in four events and the
receiver in two events.
11.662
11-5 NOISY CHANNELS
Although the Stop-and-Wait Protocol gives us an idea

of how to add flow control to its predecessor, noiseless
channels are nonexistent. We discuss three protocols in
this section that use error control.

Stop-and-Wait Automatic Repeat Request
Go-Back-N Automatic Repeat Request
Selective Repeat Automatic Repeat Request
11.663
Note
Error correction in Stop-and-Wait ARQ is done by
keeping a copy of the sent frame and retransmitting of the
frame when the timer expires.
 In Stop-and-Wait ARQ:
 we use sequence numbers to number the frames.
The sequence numbers are based on modulo-2
arithmetic.
 In Stop-and-Wait ARQ, the acknowledgment
number always announces in modulo-2 arithmetic the
sequence number of the next frame expected.
11.664
Figure 11.11 Flow diagram for an example of Stop-and-Wait ARQ.
Frame 0 is sent and

acknowledged. Frame 1
is lost and resent after
the time-out. The resent
frame 1 is
acknowledged and the
timer stops.
Frame 0 is sent and
acknowledged, but the
acknowledgment is lost.
The sender has no idea
if the frame or the
acknowledgment is lost,
so after the time-out, it
resends frame 0, which
is acknowledged.
11.665
Example 11.4
Assume that, in a Stop-and-Wait ARQ system, the bandwidth of the

line is 1 Mbps, and 1 bit takes 20 ms to make a round trip. What is
the bandwidth-delay product? If the system data frames are 1000 bits
in length, what is the utilization percentage of the link?
Solution
The bandwidth-delay product is
The system can send 20,000 bits during the time it takes for the data
to go from the sender to the receiver and then back again. However,
the system sends only 1000 bits. We can say that the link utilization is
only 1000/20,000, or 5 percent. For this reason, for a link with a
high bandwidth or long delay, the use of Stop-and-Wait ARQ wastes
the capacity of the link.
11.666
Example 11.5
What is the utilization percentage of the link in Example

11.4 if we have a protocol that can send up to 15 frames
before stopping and worrying about the
acknowledgments?
Solution
The bandwidth-delay product is still 20,000 bits. The
system can send up to 15 frames or 15,000 bits during a
round trip. This means the utilization is 15,000/20,000, or
75 percent. Of course, if there are damaged frames, the
utilization percentage is much less because frames have to
be resent.
11.667
Note
In the Go-Back-N Protocol, the sequence numbers are
modulo 2m,
where m is the size of the sequence number field in bits.
11.668
Figure 11.12 Send window for Go-Back-N ARQ
11.669
Note
The send window is an abstract concept defining an
imaginary box of size 2m−1 with three variables: Sf, Sn, and
Ssize.
 The send window can slide one or more slots when a

valid acknowledgment arrives.
11.670
Figure 11.13 Receive window for Go-Back-N ARQ
11.671
Note
The receive window is an abstract concept defining an

imaginary box
of size 1 with one single variable Rn.
The window slides
when a correct frame has arrived; sliding occurs one slot
at a time.
11.672
Figure 11.15 Window size for Go-Back-N ARQ
11.673
Note
In Go-Back-N ARQ, the size of the send window must be

less than 2m;
the size of the receiver window
is always 1.
11.674
This is an
example
of a case
where the
forward
channel is
reliable,
but the
reverse is
not. No
data
frames
11.675
Scenario
showing what
happens when
a frame is
lost.
11.676
Note
Stop-and-Wait ARQ is a special case of Go-Back-N ARQ

in which the size of the send window is 1.
11.677
Figure 11.18 Send window for Selective Repeat ARQ
11.678
Figure 11.19 Receive window for Selective Repeat ARQ
11.679
Figure 11.21 Selective Repeat ARQ, window size
11.680
Note
In Selective Repeat ARQ, the size of the sender and

receiver window
must be at most one-half of 2m.
11.681
Figure 11.22 Delivery of data in Selective Repeat ARQ
11.682
Scenario
showing how
Selective Repeat
behaves when a
frame is lost.
11.683
11-6 HDLC
High-level Data Link Control (HDLC) is a bit-oriented

protocol for communication over point-to-point and
multipoint links. It implements the ARQ mechanisms we
discussed in this chapter.

Configurations and Transfer Modes
Frames
Control Field
11.684
Figure 11.25 Normal response mode
Figure 11.26 Asynchronous balanced mode
11.685
Figure 11.27 HDLC frames
Control field
format for the
different frame
types
11.686
Table 11.1 U-frame control command and response
11.687
Figure 11.31 Example of piggybacking with error
Figure 11.31 shows an exchange in which a

frame is lost. Node B sends three data
frames (0, 1, and 2), but frame 1 is lost.
When node A receives frame 2, it discards it
and sends a REJ frame for frame 1. Note
that the protocol being used is Go-Back-N
with the special use of an REJ frame as a
NAK frame. The NAK frame does two things
here: It confirms the receipt of frame 0 and
declares that frame 1 and any following
frames must be resent. Node B, after
receiving the REJ frame, resends frames 1
and 2. Node A acknowledges the receipt by
sending an RR frame (ACK) with
acknowledgment number 3.
11.688
11-7 POINT-TO-POINT PROTOCOL
Although HDLC is a general protocol that can be used

for both point-to-point and multipoint configurations,
one of the most common protocols for point-to-point
access is the Point-to-Point Protocol (PPP). PPP is a
byte-oriented protocol.

Framing
Transition Phases
Multiplexing
Multilink PPP
11.689
Figure 11.32 PPP frame format
PPP is a byte-oriented protocol using byte stuffing with the

escape byte 01111101.
11.690
Figure 11.33 Transition phases
11.691
Figure 11.35 LCP packet encapsulated in a frame
11.692
Table 11.2 LCP packets
11.693
Table 11.3 Common options
11.694
Figure 11.36 PAP packets encapsulated in a PPP frame
11.695
Figure 11.37 CHAP packets encapsulated in a PPP frame
11.696
Figure 11.38 IPCP packet encapsulated in PPP frame
Code value
for IPCP
packets
11.697
A Survey of Advanced
FEC Systems
Eric Jacobsen
Minister of Algorithms, Intel Labs
Communication Technology Laboratory/
Radio Communications Laboratory
July 29, 2004
With a lot of material from Bo Xia, CTL/RCL
www.intel.com/labs
Communication and Interconnect Technolo
Outline
 What is Forward Error Correction?
 The Shannon Capacity formula and what it means
 A simple Coding Tutorial
 A Brief History of FEC

 Modern Approaches to Advanced FEC
 Concatenated Codes
 Turbo Codes
 Turbo Product Codes
 Low Density Parity Check Codes
www.intel.com/labs 699
Information Theory Refresh

The Shannon Capacity Equation
C = W log2(1 + P / N)
Channel Transmit Noise
Channel Bandwidth Power Power
Capacity (Hz)
(bps)
2 fundamental ways to increase data rate

C is the highest data rate that can be transmitted error free under
the specified conditions of W, P, and N. It is assumed that P
is the only signal in the memoryless channel and N is AWGN.
®
A simple example
A system transmits messages of two bits each through a channel
that corrupts each bit with probability Pe.
Tx Data = { 00, 01, 10, 11 } Rx Data = { 00, 01, 10, 11 }
The problem is that it is impossible to tell at the receiver whether the

two-bit symbol received was the symbol transmitted, or whether it
was corrupted by the channel.
Tx Data = 01 Rx Data = 00
In this case a single bit error has corrupted the received symbol, but
it is still a valid symbol in the list of possible symbols. The most
fundamental coding trick is just to expand the number of bits
transmitted so that the receiver can determine the most likely
transmitted symbol just by finding the valid codeword with the
minimum Hamming distance to the received symbol.
®
Continuing the Simple
Example
A one-to-one mapping of symbol to codeword is produced:
Symbol:Codeword The result is a systematic block code

00 : 0010 with Code Rate R = ½ and a minimum
01 : 0101 Hamming distance between codewords
10 : 1001 of dmin = 2.
11 : 1110
A single-bit error can be detected and corrected at the receiver by

finding the codeword with the closest Hamming distance. The
most likely transmitted symbol will always be associated with the
closest codeword, even in the presence of multiple bit errors.
This capability comes at the expense of transmitting more bits,

usually referred to as parity, overhead, or redundancy bits.
®
Coding Gain
The difference in performance between an uncoded and a coded
system, considering the additional overhead required by the code,
is called the Coding Gain. In order to normalize the power required
to transmit a single bit of information (not a coded bit), Eb/No is used
as a common metric, where Eb is the energy per information bit, and
No is the noise power in a unit-Hertz bandwidth.
Uncoded … … The uncoded symbols require a certain

amount of energy to transmit, in this
Symbols case over period Tb.
Coded … … The coded symbols at R = ½ can be

Symbols transmitted within the same period if
the transmission rate is doubled. Using
with R = ½
Tb Time No instead of N normalizes the noise
considering the differing signal
bandwidths.
www.intel.com/labs
®
703
Coding
Coding Gain
Gain and
and Distance
Distance to
to Channel
Channel Capacity
Capacity Example
Example
C, R = 3/4 C, R = 9/10
0.1
1.62 3.2 Uncoded
“Matched-Filter
0.01 Bound”
Performance
3
1 10
These curves
Compare the
performance of
BER = Pe
4
1 10
two Turbo
Codes with a
1 10
5 concatenated
Viterbi-RS
system. The
d = ~1.4dB Coding Gain = ~5.95dB
1 10
6 TC with R =
d = ~2.58dB Coding Gain = ~6.35dB 9/10 appears to
be inferior to
1 10
7
1 2 3 4 5 6 7 8 9 10 11
the R = ¾ Vit-
Eb/No (dB) RS system, but
R = 3/4 w/RS
R = 9/10 w/RS Capacity for R = 3/4 is actually
VitRs R = 3/4
Uncoded QPSK
operating
closer to
capacity.
®
FEC Historical Pedigree

1950 1960 1970
Shannon’s Paper
1948
Early practical
Hamming
defines basic implementations
binary codes of RS codes for tape
Gallager’s Thesis
and disk drives
On LDPCs
BCH codes Berlekamp and Massey
Proposed Viterbi’s Paper rediscover Euclid’s
On Decoding polynomial technique
Convolutional Codes and enable practical
Reed and Solomon algebraic decoding
define ECC Forney suggests
Technique concatenated codes
FEC Historical Pedigree II

1980 1990 2000
Ungerboeck’s
TCM Paper - 1982
LDPC beats
RS codes appear
in CD players Turbo Codes
For DVB-S2
Berrou’s Turbo Code
First integrated Standard - 2003
Paper - 1993
Viterbi decoders Renewed interest
(late 1980s) Turbo Codes in LDPCs due to TC
Adopted into Research
Standards
TCM Heavily (DVB-RCS, 3GPP, etc.)
Adopted into
Standards
Block
Block Codes
Codes
Generally, a block code is any code defined with a finite codeword length.
Systematic Block Code If the codeword is constructed by

appending redundancy to the
Data Field Parity payload Data Field, it is called a
“systematic” code.
Codeword
The “parity” portion can be actual parity bits, or generated by some other means, like
a polynomial function or a generator matrix. The decoding algorithms differ greatly.
The Code Rate, R, can be adjusted by shortening the data field (using zero padding)
or by “puncturing” the parity field.
Examples of block codes: BCH, Hamming, Reed-Solomon, Turbo Codes,

Turbo Product Codes, LDPCs
Essentially all iteratively-decoded codes are block codes.
Convolutional
Convolutional Codes
Codes
Convolutional codes are generated using a shift register to apply a polynomial to a

stream of data. The resulting code can be systematic if the data is transmitted in
addition to the redundancy, but it often isn’t.
This is the convolutional encoder for
The p = 133/171 Polynomial that is in
very wide use. This code has a
Constraint Length of k = 7. Some
low-data-rate systems use k = 9 for
a more powerful code.
This code is naturally R = ½, but

deleting selected output bits, or
“puncturing” the code, can be done
to increase the code rate.
Convolutional codes are typically decoded using the Viterbi algorithm, which increases in
complexity exponentially with the constraint length. Alternatively a
sequential decoding algorithm can be used, which requires a much longer constraint length
for similar performance.
Diagram from [1] ®
Convolutional
Convolutional Codes
Codes -- II
II
This is the code-trellis, or state diagram of a k = 2

Convolutional Code. Each end node represents a code
state, and the branches represent codewords selected
when a one or a zero is shifted into the encoder.
The correcting power of the code comes from the

sparseness of the trellis. Since not all transitions from
any one state to any other state are allowed, a state-
estimating decoder that looks at the data sequence can
estimate the input data bits from the state relationships.
The Viterbi decoder is a Maximum Likelihood

Sequence Estimator, that estimates the encoder
state using the sequence of transmitted codewords.
This provides a powerful decoding strategy, but

when it makes a mistake it can lose track of the
sequence and generate a stream of errors until
it reestablishes code lock.
Diagrams from [1] ®
Concatenated
Concatenated Codes
Codes
A very common and effective code is the concatenation of an inner convolutional

code with an outer block code, typically a Reed-Solomon code. The convolutional
code is well-suited for channels with random errors, and the Reed-Solomon code is
well suited to correct the bursty output errors common with a Viterbi decoder. An
interleaver can be used to spread the Viterbi output error bursts across multiple RS
codewords.
Data
RS Conv. Viterbi
Interleaver Channel
Encoder Encoder Decoder
Inner Code
Outer Code
Data
De- RS
Interleaver Decoder
Concatenating
Concatenating Convolutional
Convolutional
Codes
Codes
Parallel and serial
Data
CC CC Viterbi/APP
Interleaver Channel
Encoder1 Encoder2 Decoder
Serial Concatenation
Data
De- Viterbi/APP
Interleaver Decoder
Data Data
Viterbi/APP
Channel Combiner
Decoder
CC
Encoder1
De- Viterbi/APP
CC Interleaver Decoder
Interleaver
Encoder2
Iterative
Iterative Decoding
Decoding of
of CCCs
CCCs
Rx Data
Viterbi/APP
Interleaver
Decoder Data
De- Viterbi/APP
Interleaver Decoder
Turbo Codes add coding diversity by encoding the same data twice through
concatenation. Soft-output decoders are used, which can provide reliability update
information about the data estimates to the each other, which can be used during a
subsequent decoding pass.
The two decoders, each working on a different codeword, can “iterate” and continue
to pass reliability update information to each other in order to improve the probability
of converging on the correct solution. Once some stopping criterion has been met,
the final data estimate is provided for use.
These Turbo Codes provided the first known means of achieving decoding
performance close to the theoretical Shannon capacity.
®
MAP/APP decoders
 Maximum A Posteriori/A Posteriori Probability
 Two names for the same thing
 Basically runs the Viterbi algorithm across the data sequence in both
directions
 ~Doubles complexity
 Becomes a bit estimator instead of a sequence estimator
 Optimal for Convolutional Turbo Codes
 Need two passes of MAP/APP per iteration
 Essentially 4x computational complexity over a single-pass Viterbi
 Soft-Output Viterbi Algorithm (SOVA) is sometimes substituted as a
suboptimal simplification compromise
Turbo Code Performance
Turbo
Turbo Code
Code Performance
Performance II
II
0.1
The performance curves shown here 1.629 2.864
were end-to-end measured

0.01
performance in practical modems. The
black lines are a PCCC Turbo Code, and
3
1 10
The blue lines are for a concatenated
Viterbi-RS decoder. The vertical
dashed lines show QPSK capacity for 1 10
4
R = ¾ and R = 7/8. The capacity for
BER
QPSK at R = ½ is 0.2dB. 1 10
5
The TC system clearly operates much 1 10

6
closer to capacity. Much of the

observed distance to capacity is due 7
1 10
to implementation loss in the modem.
8
1 10
0 1 2 3 4 5 6 7 8 9
Eb/No (dB)
Uncoded
Vit-RS R = 1/2
Vit-RS R = 3/4
Vit-RS R = 7/8
Turbo Code R = 1/2
Turbo Code R = 3/4
Turbo Code R = 7/8
Tricky
Tricky Turbo
Turbo Codes
Codes
Repeat-Accumulate codes use simple repetition followed by a differential encoder

(the accumulator). This enables iterative decoding with extremely simple codes.
These types of codes work well in erasure channels.
Repeat Accumulate
Section Section
1:2 Interleaver D +
R = 1/2 R=1
Outer Code Inner Code
Since the differential encoder has R = 1, the final code rate is determined by the
amount of repetition used.
Turbo
Turbo Product
Product Codes
Codes
Horizontal Hamming Codes
The so-called “product codes” are codes

Vertical Hamming Codes
Created on the independent dimensions

Of a matrix. A common implementation
2-Dimensional
Parity
Arranges the data in a 2-dimensional array,
Data Field
and then applies a hamming code to each
row and column as shown.
The decoder then iterates between decoding

Parity the horizontal and vertical codes.
Parity Parity
Since the constituent codes are Hamming codes, which can be decoded simply, the
decoder complexity is much less than Turbo Codes. The performance is close to capacity
for code rates around R = 0.7-0.8, but is not great for low code rates or short blocks. TPCs
have enjoyed commercial success in streaming satellite applications.
Low Density Parity Check
Codes
 Iterative decoding of simple parity check codes
 First developed by Gallager, with iterative decoding, in 1962!
 Published examples of good performance with short blocks
 Kou, Lin, Fossorier, Trans IT, Nov. 2001
 Near-capacity performance with long blocks

 Very near! - Chung, et al, “On the design of low-density parity-check codes within
0.0045dB of the Shannon limit”, IEEE Comm. Lett., Feb. 2001
 Complexity Issues, especially in encoder

 Implementation Challenges – encoder, decoder memory
LDPC Bipartite Graph
Check Nodes
Edges
Variable Nodes
(Codeword bits)
This is an example bipartite graph for an irregular LDPC code.
Iteration
Iteration Processing
Processing
1st half iteration, compute ’s,’s, and r’s for each edge.
Check Nodes
i+1 = maxx(i,qi) Edges
i = maxx(i+1,qi) (one per parity bit)
ri = maxx(i,i+1)
ri
qi
mVn
Variable Nodes mV = mV0 + r’s qi = mV – ri

(one per code bit)
2nd half iteration, compute mV, q’s for each
variable node.
LDPC
LDPC Performance
Performance Example
Example
LDPC Performance can

Be very close to capacity.
The closest performance
To the theoretical limit
ever was with an LDPC,
and within 0.0045dB of
capacity.
The code shown here is

a high-rate code and
is operating within a few
tenths of a dB of capacity.
Turbo Codes tend to work

best at low code rates and
not so well at high code rates.
LDPCs work very well at high
code rates and low code rates.
Figure is from [2] ®
Current State-of-the-Art
 Block Codes
 Reed-Solomon widely used in CD-ROM, communications standards.
Fundamental building block of basic ECC
 Convolutional Codes
 K = 7 CC is very widely adopted across many communications standards
 K = 9 appears in some limited low-rate applications (cellular telephones)
 Often concatenated with RS for streaming applications (satellite, cable, DTV)
 Turbo Codes
 Limited use due to complexity and latency – cellular and DVB-RCS
 TPCs used in satellite applications – reduced complexity
 LDPCs
 Recently adopted in DVB-S2, ADSL, being considered in 802.11n, 802.16e
 Complexity concerns, especially memory – expect broader consideration
Cyclic Codes for Error Detection
W. W. Peterson and D. T. Brown
by
Maheshwar R Geereddy

Definition

A code is called cyclic if [xnx0x1...xn-1] is a
codeword whenever [x0x1...xn-1xn] is also a codeword.
Notations
k = Number of binary digits in the message
before encoding
n = Number of binary digits in the encoded
message
n – k = number of check bits
n k

b = length of a burst of errors.
G (X) = message polynomial
P (X) = generator polynomial
R (X) = remainder on dividing X n-k G (X) by P(X).
F (X) = encoded message polynomial
E (X) = error polynomial
H (X) = Received encoded message polynomial

H (X) = F (X) + E (X)

Polynomial Representation of Binary
Information
It is convenient to think of binary digits as
coefficients of a polynomial in the dummy variable
X.
Polynomial is written low-order-to-high-order.
Polynomials are treated according to the laws of
ordinary algebra with an exception addition is to
be done modulo two.

Algebraic Description of Cyclic Codes
A cyclic code is defined in terms of a generator

polynomial P(X) of degree n-k.
If P(X) has X as a factor then every code

polynomial has X as a factor and therefore
zero-order coefficient equal to zero.
Only codes for which P(X) is not divisible by X
are considered.

Encoded Message Polynomial F(X)
• Computer X n – k G (X)
• R(X) = X n – k G (X) / P(X)
• Add the remainder to the X n – k G (X)
– F(X) = X n – k G (X) + R(X)
X n – k G (X) = Q(X) P(X) + R(X)
Also F(X) = Q(X) P(X)

Principles of Error Detection and Error
Correction
• An encoded message containing errors can be
represented by
H (X) = F (X) + E (X)
H (X) = Received encoded message polynomial

F (X) = encoded message polynomial
E (X) = error polynomial

Principles of Error Detection and Error
Correction Contd…
• To detect error, divide the received, possible
erroneous message H(X) by P(X) and test the
remainder.
• If the remainder is nonzero an error has been
detected.
• If the remainder is zero, either no error or an
undetectable error has occurred

DETECTION OF SINGLE ERRORS
• Theorem 1: A cyclic code generated by an polynomial

P(X) with more than one term detects all single errors.
• Proof:
– A single error in the i’th position of an encoded message
corresponds to an error polynomial X i.
– For detection of single errors, it is necessary that P(X) does
not divide X i.
– Obviously no polynomial with more than one term divides X I.

DETECTION OF SINGLE ERRORS
Contd….
• Theorem 2: Every polynomial divisible by 1 + X has

an even number of terms.
– Proof:
• Also if P(X) contains a factor 1 + X any odd numbers

of errors will be detected.

Double and Triple Error Detecting
Codes (Hamming Codes)
• Theorem 3: A code generated by the polynomial
P(X) detects all single and double errors if the length
n of the code is no greater than the exponent e to
which P(X) belongs.
– Detecting double errors requires that P(X) does not divisible
by
X i + X j for any i, j < n

Double and Triple Error Detecting
Codes Contd…
• Theorem 4: A code generated by P (X) = (1 + X)
P1(X) detects all single, double, and triple errors if the
length n of the code is no greater than the exponent e
to which P1(X) belongs.
– Single and triple errors are detected by presence of factor 1
+ X as proved in Theorem 2.
– Double errors are detected because P1(X) belong to the
exponent
e >= n as proved in Theorem 3
– Q.E.D

Detection of a Burst-Error
• A burst error of length b will be defined as any pattern of errors for
which the number of symbols between first and last errors including
these errors, is b.
• Theorem 5: Any cyclic code generated by a polynomial of degree n-k
detects any burst –error of length n-k or less.
– Any burst polynomial can be factored as E(X) = Xi E1(X)
– E1(X) is of degree b-1
– Burst can be detected if P(X) does not divide E(X)
– Since P(X) is assumed not to have X as a factor, it could divide
E(X) only if it could divide E1(X).
– But b < = n – k
• There fore P(X) is of higher degree than E1(X) which implies that P(X)
could not divide E1(X)
• Q.E.D
• Theorem 6: The fraction of bursts of length b > n-k that are undetected
is
2-(n-k) if b > n – k + 1
2-(n – k – 1) if b = n – k + 1
Detection of Two Bursts of Errors
(Abram son and Fire Codes)
• Theorem 7: The cyclic code generated by P (X) = (1 + X) P1 (X)
detects any combination of two burst-errors of length two or less
if the length of the code, n, is no go greater than e, the exponent
to which P1 (X) belongs.
– Proof
– There are four types of error patterns
• E(X) = X i + X j
• E(X) = (X i + Xi+1) + X j
• E(X) = X j (X j + Xj+1)
• E(X) = (X i + Xi+1) + (X j + Xj+1)

Other Cyclic Codes
• There are several important cyclic codes which have not been
discussed in this paper.
– BCH codes (developed by Bose, Chaudhuri, and
Hocquenghem) are a very important type of cyclic codes.
– Reed-Solomon codes are a special type of BCH codes that
are commonly used in compact disc players.
Implementation
• Briefly, to encode a message, G (X), n-k zeros are annexed (I.e.
multiplication of Xn-1G (X) is performed) and then Xn-1G (X) is
divided by the polynomial P (X) of degree n-k. The remainder is
then subtracted from Xn-1G (X). (It replaces the n-k zeroes).
• This encoded message is divisible by P (X) for checking out
errors

Implementation Contd…
• It can be seen that modulo 2 arithmetic has simplified the division
considerably.
• Here we do not require the quotient, so the division to find the remainder
can be described as follows.
• 1) Align the coefficient of the highest degree terms of the divisor and
dividend and subtract (same as addition)
• 2) Align the coefficient of the highest degree terms of the divisor and
difference and subtract again
3) Repeat the process until the difference has the lower degree than the
divisor
• The hardware to implement this algorithm is a shift register and a
collection of modulo two adders.
• The number of shift register positions is equal to the degree of the
divisor, P (X), and the dividend is shifted through high order first and left
to right.

• As the first one (the coefficient of the high order term of
the dividend) shifts off the end we subtract the divisor by
the following procedure:
1. In the subtraction the high-order terms of the divisor and
dividend always cancel. As the higher order term of the dividend
is shifted off the end of the register, this part of the subtraction is
done automatically.
2. Modulo two adders are placed so that when a one shifts off the
end of the register, the divisor is subtracted from the contents of
the register. The register than contains a difference that is
shifted until another comes off the end and then the process is
repeated. This continues until the entire dividend is shifted into
the register.

Input 100010001101011
0 -> 10 00 1
0 -> 11 10 1
0 -> 11 01 1
1 -> 11 00 0
1 -> 11 10 0
0 -> 11 11 0
1 -> 01 11 1
0 -> 00 01 0
1 -> 00 00 1
1 -> 00 10 1

• To minimize the hardware it is desirable to use the same register for
both encoding and error detection.
• If circuit of fig : 3 is used for error detection, the remainder on dividing X
n-k H(X) by P(X) instead of remainder on dividing H(X) by P(X)
• This makes no difference, because if H(X) is not evenly divisible by

P(X) than obviously X n-k H(X) will not be divisible either.
• Error Correction: It is a much more difficult task than error detection.
• It can be shown that each different correctable error pattern must give a
different remainder after division buy P(X).
• There fore error correction can be done.
Conclusion
• Cyclic codes for error detection provides high efficiency and the ease of
implementation.
• It provides standardization like CRC-8 and CRC-32

The Viterbi Algorithm
• Application of Dynamic Programming-the Principle of
Optimality
• -Search of Citation Index -213 references since 1998
• Applications
– Telecommunications
• Convolutional codes-Trellis codes
• Inter-symbol interference in Digital Transmission
• Continuous phase transmission
• Magnetic Recording-Partial Response Signaling-
• Divers others
– Image restoration
– Rainfall prediction
– Gene sequencing
– Character recognition

Milestones
• Viterbi (1967) decoding convolutional codes
• Omura (1968) VA optimal
• Kobayashi (1971) Magnetic recording
• Forney (1973) Classic survey recognizing the
generality of the VA
• Rabiner (1989) Influential survey paper of hidden
Markov chains

Example-Principle of Optimality
Find optimal path to each Professor X chooses an
bridge optimum
Peris path on his trip to lunch
Publis h
h
1.2 N .
. 2
7 1.2
.5 N
Faculty
1. Club
. 2
5
.
EE Bld Optimal: 6 adds N bridges 3
Brute force:8 Optimal: 4(N+1) adds
adds . Brute force: (N-1)2N
5 adds
S
. .8 1.0
8 S .
8
Digital Transmission with
Convolutional Codes
Information a1 , a2 ,..., aN Convolutional

Source
c1 , c2 ,..., cN
AN Encoder
BSC
p
p
 1 , a 2 ,..., a N
Information a b1 , b2 ,..., bN
Viterbi
Sink Algorithm
BN

Maximum a Posteriori (MAP)
Estimate
Define
D( B N , AN )  Hamming distance between sequences
Maximum Aposteriori Probability

N
,BN ) N
,BN )
max P (b1 , b2 ,..., bN / a1 , a2 ,..., a N )  max p D ( A (1  p) N  D ( A
a1 , a2 ,..., aN a1 , a2 ,..., a N
p  bit error probability
Equivalently
min D ( A N , B N ) log( p /(1  p)
a1 , a2 ,..., a N
Brute force = Exponential

Growth with N
Convolutional codes-Encoding a
sequence
Example(3,1) code (output,input)
efficiency=input/output
Initial state - s1  s2  0
Output
Input
T T 111 100 010 110 011 001 000
110100
S1 S2
State 01  i
0 2  i  s1
0 3  i  s1  s2
Initial state - s1  s2  0

Markov chain for Convolutional code
0-000
input -output state

0 -001
00 Fig.2.14
01
1 -111 0 -010
0 -011
1 -110
10 1 -100
11 1 -101
Trellis Representation
State output Next state
0 input
s1s2
1 input
00 000 00
111
01 01
001
110
10 011 10
100
11 11
010
101

Iteration for Optimization
min D( A N , B N )  min D( AN , B N )
a1 , a2 ,..., a N s1 , s2 ,..., s N
Shift register contents
N
min D( AN , B N )  min
s1 , s2 ,..., s N s1 , s2 ,..., s N
 d (a , b )-
i 1
i i BSC
memorylessness
N 1 N 1
min ( D( A , B )  N N
min ( D( A ,B )  d (aN , bN ))
s1 , s2 ,..., s N s1 , s2 ,..., s N 1 , s N
N 1 N 1
min ( D( A , B )  min N N
min ( D( A ,B )  d (aN , bN ))
s1 , s2 ,..., s N sN s1 , s2 ,..., s N 1/ S N
min ( D( A N , B N )  min( d ( a N , bN )  min D( A N 1, B N 1 ))

s1 , s2 ,..., s N sN s1 , s2 ,..., sN 1/ S N

Key step!
min D( A N  2 , B N  2 ))  min D( A N  2 , B N  2 ))
s1 , s2 ,..., sN 2 / S N 1 ,S N s1 , s2 ,..., s N 2 / S N 1
Redundant
N 1 N 1
min D( A ,B )
s1 , s2 ,..., s N 1 / s N
N 2 N 2
min ( d ( aN 1 , bN 1 )  min D( A ,B ))
s N 1 / s N Incremental distance s1 , s2 ,..., sN 2 / sN 1 Accumulated distance
Linear growth in N

Deciding Previous State
min D( Ai , B i ) 
s1 , s2 ,..., si / si 1
min(d (ai , bi )  min D( Ai 1 , B i 1 ))

si / si 1 s1 , s2 ,..., si 1 / si
State i-1 d (ai , bi ) State i

D( Ai 1 , B i 1 ) 00 bi  010
1 00
4 ai  000 4
Search previous states
10 2
2 ai  001

Viterbi Algorithm-shortest path
First step to detect sequence
s0
s1
s2
s3
Trace though successive states
shortest path-Hamming distance to s0

Trellis codes-Euclidean distance
Optimum Sequence Detection

Inter-symbol Interference
z (t )
N
Transmitter Channel Equalizer
 a p(t  iT )
i 1
i
Decisions VA
N
z (t )   ai h(t  iT )  n(t )-Received signal
i 1

ri  j   h(t  iT )h(t  jT )dt
0
ri  j  0; i  j  m  Finite memory channel

AWGN Channel-MAP Estimate
 2
 N

min   z (t )   ai h(t  iT )  dt -
0  
a1 , a2 ,..., a N
i 1
Euclidean distance between received and possible signals

Simplification
 N N N 
min  2 a i Z i   ai a j ri  j 
a1 , a2 ,..., aN
 i 1 i 1 j 1 
where

Z i   y (t )h(t  iT )dt -Output of Matched Filter
0
Viterbi Algorithm for ISI
Define:s k  {ak  m 1 ,..., ak }-states
Memory m
 k k k

D( Z1 ,..., Z k , sk  m 1 ,..., sk )   2 a i Z i   ai a j ri  j   Accumulated distance
 i 1 i 1 j 1 
k 1
d ( Zk ; sk 1 , sk )  2ak Z k  2ak 
i k m
ai rk i  ak2r0  Incremental distance
State = number of symbols in memory

min D( Z1 ,..., Z k 1 , sk  m 1 ,..., sk 1 ) 
s1 , s2 ,..., sk 1 / sk
min (d (Zk ; sk 1 , sk )  min D( Z1 ,..., Z k  2 , sk  m 1 ,..., sk  2 ))

sk 1 / sk s1 , s2 ,..., sk 2 / sk 1

Magnetic Recording
Magnetization pattern

m(t )   (ak  1)u (t  kT )  1(t )
k 0
Magnetic flux passes over heads

Differentiation of pulses
Controlled ISI
Same model applies to
Output
Partial Response signaling
 d m(t ) 
e(t )    * h(t )
 dt 

 2 xk h(t  kT ) where xk  ak  ak 1
k 0 Nyquist pulse
EE576 Dr. Kousa Linear Block Codes Sample 760

Continuous Phase FSK
Digital Input Sequence
a1 , a2 ,..., aN
Tranmitted Signal
yk  cos( (ak )t  xk ); kT  t  (k  1)T
Constraint-Continuous Phase
 (ak 1 )t  xk 1   (ak )t  xk ; mod 2
Example-Binary signaling
  odd number ½ cycles

Whole cycles
0.8
0.3
0;even no. ones
-0.2
0 0.2 0.4 0.6 0.8 1
xk  
-0.7
 1;odd no. ones
-1.2
EE576 Dr. Kousa Signaling interval Linear Block Codes 761
Merges and State Reduction
Optimal paths through trellis
All paths merge
Force merges to reduce complexity
Computations order of (No states)2
Carry only high probability states

Input Pixel Effect of Blurring
Optical output signal

Blurring Analogous to ISI
L L
s (i, j )    a(i  l , j  m)
l  L m  L Input pixel
h(l , m)  n(i, j )
Optical channel AWGN
EE576
where L  optical blur width
Dr. Kousa Linear Block Codes 763
Row Scan
VA for optimal row sequence
Known state transitions

And Decision Feed back
Utilized for state reduction

Hidden Markov Chain
• Data suggests Markovian structure

• Estimate initial state probabilities
• Estimate transition probabilities
• VA used for estimation of Probabilities
• Iteration

Rainfall Prediction
Rainy Rainy
wet dry
No rain
Showery Showery
wet dry
Rainfall observations

DNA Sequencing
• DNA-double helix
– Sequences of four nucleotides, A,T,C and G
– Pairing between strands
– Bonding
•Genes
A  T and C  G
–Made up of Cordons, i.e. triplets of adjacent nucleotides
–Overlapping of genes
Nucleotide sequence
CGGATTC
Gene 1
Cordon A in three genes
Gene 2
Gene 3
Hidden Markov Chain
Tracking genesM S
1 P1
S-start first cordon of gene
P1-4- +1,…,+4 from start
Gene M2 P2
E-stop
H-gap
M1-4 -1,…,-4 from start
M3
P3
Initial and
Transition M4
Probabilities known P4
EE576
H
.
Dr. Kousa Linear Block Codes
E
768

Recognizing Handwritten Chinese Characters
Text-line images
Estimate stroke width
Set up m X n grid
Estimate initial and transition probabilities
Detect possible segmentation paths by VA
Next Slide
Results
Example Segmenting Handwritten
Characters
Eliminating
All possible Redundant
segmentation Paths
paths
Removal of Discarding
Overlapping near paths
paths

DC Error Correcting Codes

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

DC Error Correcting Codes

Încărcat de

Drepturi de autor:

Formate disponibile

Digital Communication and Error

Department of Electrical and Computer Engineering

Digital Coding for Error Correction 00101001110101101010101000

Digital Coding for Error Correction 00101001110101101010101000

• Example: Frequency Shift Keying (FSK)

s  t   b cos  2 f 0t    1  b  cos  2 f1t 

Digital Coding for Error Correction 00101001110101101010101000

Digital Coding for Error Correction 00101001110101101010101000

Error Correcting Codes

Digital Coding for Error Correction 00101001110101101010101000

Error Correcting Codes

5 channel errors; 1 information error

Digital Coding for Error Correction 00101001110101101010101000

Error Correcting Codes

• An error will only be made if the channel makes 2 or

cce one error (1-p)(1-p)(p) = p-2p2+p3 0.4

error probability = 3p2 – 2p3 0.15

Digital Coding for Error Correction 00101001110101101010101000

Error Correcting Codes

• Codes are characterized by the number of channel bits

Digital Coding for Error Correction 00101001110101101010101000

EE576 Dr. Kousa Linear Block Codes 15

• Let u be a k-bit information sequence

EE576 Dr. Kousa Linear Block Codes 16

EE576 Dr. Kousa Linear Block Codes 17

• In this case the generator matrix will take the form

EE576 Dr. Kousa Linear Block Codes 18

EE576 Dr. Kousa Linear Block Codes 19

EE576 Dr. Kousa Linear Block Codes 20

EE576 Dr. Kousa Linear Block Codes 21

EE576 Dr. Kousa Linear Block Codes 22

EE576 Dr. Kousa Linear Block Codes 23

• TH3.2 For each codeword of Hamming weight l there

EE576 Dr. Kousa Linear Block Codes 24

• Let v be transmitted and r be received, where

• We will attempt both processes: error detection, and error

EE576 Dr. Kousa Linear Block Codes 25

EE576 Dr. Kousa Linear Block Codes 26

• s = rHT = (v + e) HT = vHT + eHT = eHT

EE576 Dr. Kousa Linear Block Codes 27

EE576 Dr. Kousa Linear Block Codes 28

• Transmitted codeword is any one of:

EE576 Dr. Kousa Linear Block Codes 29

EE576 Dr. Kousa Linear Block Codes 30

EE576 Dr. Kousa Linear Block Codes 31

EE576 Dr. Kousa Linear Block Codes 32

EE576 Dr. Kousa Linear Block Codes 33

EX 3.6: The (6,3) code defined by the H matrix:

EE576 Dr. Kousa Linear Block Codes 35

EE576 Dr. Kousa Linear Block Codes 36

Error Pattern Syndrome

EE576 Dr. Kousa Linear Block Codes 37

EE576 Dr. Kousa Linear Block Codes 38

EE576 Dr. Kousa Linear Block Codes 39

EE576 Dr. Kousa Linear Block Codes 40

EE576 Dr. Kousa Linear Block Codes 41

• Define the weight enumerator: A( z ) 

EE576 Dr. Kousa Linear Block Codes 42

where B(z) is the weight enumerator of the dual code.

EE576 Dr. Kousa Linear Block Codes 43

• It may be able to correct higher error patterns but not all.

EE576 Dr. Kousa Linear Block Codes 44

EE576 Dr. Kousa Linear Block Codes 45

Right side = 1+n; Left side = 2m =n+1