11/29/2013

Information Theory and Coding

ECE533

DP
Status of DP ?
Number Theory

Feedback
Fill Feedback on tinyurl.com/FeedbackNikB
Extra Lecture on
MATLAB for DP
Nikesh Bajaj
nikesh.14730@lpu.co.in

j
Asst. Prof., ECE Dept.
Lovely Professional University 2 By Nikesh Bajaj

ja
Ba
Overview Divisibility
Divisor
Divisibility b divides a if a = mb for some m, where a, b, and m are integers
Division Algorithm b divides a b|a if not divisible b/|a
b|a then b|ca for c=int
sh
b|a & a|c then b|c
If a|b and b|a, then a = b & If a|1, then a = 1
b|a a > 0 and b >0 so a <= b
Any b 0 , divides 0 b|0
If b|g and b|h, then b|(mg + nh) for arbitrary integers m and n
if a 0 (mod n), then n|a
If a(x) , b(x)F[x] : b(x) | a(x) if a(x) =b(x)c(x)
ke

If a(x)|b(x) & b(x)|a(x) then a(x) =cb(x) c non0 of F[x]

3 By Nikesh Bajaj 4 a(x)|b(x) deg(a) <= deg(b) By Nikesh Bajaj
Ni

Division Algorithm Greatest Common Divisor

Division Algorithm
P(x) = q(x)d(x) +r(x), d(x) =\ 0 deg(r) < deg(d)
Ex p(x) =x3 +x + 1 and d(x) = x2 +x+1
GCD
The positive integer c is said to be the greatest common
divisor of a and b if
c is a divisor of a and of b;
any divisor of a and b is a divisor of c.
all nonzero integers divide 0, we have gcd(a, 0) = |a|.
gcd(18, 12) = gcd(12, 6) = gcd(6, 0) = 6
Relative Prime
If gcd(a, b) = 1
gcd(11, 10) = gcd(10, 1) = gcd(1, 0) = 1

5 By Nikesh Bajaj 6 By Nikesh Bajaj

1
 11/29/2013

GCD (a,b) What you should know

1. (ma, mb) = m(a, b) Why we need to compute GCD?
2. If d|a & d|b (a/d, b/d) = 1/d(a, b) How to compute it easily?
3. (a,b) =g then (a/g, b/g) =1
4. (a,c) =(b,c) =1 then (ab,c) =1
5. If c|ab and (b,c) =1 then c|a
6. Every divisor d of a and b divides (a,b)
7. (a,b) =|a| iff a|b
8. (a,(b,c)) = ((a,b),c)

j
7 By Nikesh Bajaj 8 By Nikesh Bajaj

ja
Ba
Euclidean Algorithm Theorems
To find the gcd or to check the relative prime
If g =(a,b) then there exist s and t such that
Example: 1970 1066, gcd(966,851)=? g =(a,b) = as+bt
To find gcd(1970, 1066)
1970 = 1 x 1066 + 904 gcd(1066, 904)
For any integer n, (a,b) = (a, b+na)
sh
1066 = 1 x 904 + 162 gcd(904, 162)
904 = 5 x 162 + 94 gcd(162, 94)
162 = 1 x 94 + 68 gcd(94, 68)
94 = 1 x 68 + 26 gcd(68, 26)
68 = 2 x 26 + 16 gcd(26, 16)
26 = 1 x 16 + 10 gcd(16, 10)
16 = 1 x 10 + 6 gcd(10, 6)
10 = 1 x 6 + 4 gcd(6, 4)
ke

6 =1x4+2 gcd(4, 2)
4 =2x2+0 gcd(2, 0)
Therefore, gcd(1970, 1066) = 2
9 By Nikesh Bajaj 10 By Nikesh Bajaj
Ni

Congruence Congruence
a b (mod m) iff m|(a-b)
if (a mod n) = (b mod n) b a 0 mod m
Then a b (mod n) If a b (mod m)
73 4 (mod 23); and b c (mod m) then a c (mod m)
21 -9 (mod 10) and c d (mod m) then ax+cy bx+dy (mod m )
and c d (mod m) then ac bd (mod m )
a(x) b(x) mod(m(x)) a b (mod m)
n n

and d|m and d >0 then a b (mod d)

then for c>0 ac bd (mod mc )
then (a,m) = (b, m)
If ax ay (mod m)
and (a,m) =1 then x y (mod m)
x y (mod m/(a,m))
11 By Nikesh Bajaj 12 By Nikesh Bajaj

2
 11/29/2013

!!!!! Euler Totient Function

Test (n) : # relative prime to n (<n)
DP
Extra Lecture
Feedback

j
13 By Nikesh Bajaj 14 By Nikesh Bajaj

ja
Ba
Euler Totient Function Fermats Little Theorem
if n and a are integer st (a,n) =1 then
sh

If p is prime and a st (a,p)=1 then p|(ap-1 -1)

or
ke

15 By Nikesh Bajaj 16 By Nikesh Bajaj

Ni

What you should know? Extension Field

How Fermats Little theorem is used to Primitive Element
compute inverse? Example
Computing Totient Function of GF(5)
Primitive Elements = 2 and 3
17216
2782 Importance
To construct Field
10^5 -1
347^2 3
343766732
636

17 By Nikesh Bajaj 18 By Nikesh Bajaj

3
 11/29/2013

Extension Field: Construction Example

Primitive Polynomial
Primitive poly. Of every degree exist for every GF
Can be used to construct extension field
Link for polynomial:
http://theory.cs.uvic.ca/gen/poly.html
Example
Construct GF(8) using p(x) =x3+ x+ 1

j
19 By Nikesh Bajaj 20 By Nikesh Bajaj

ja
Ba
TRY Theorem: Factoring xn -1
Construct Theorem: for 1 q-1 as non-zero element of
GF(8) from GF(2) see example in book GF(q)
GF(9) from GF(3) xq-1-1 =(x-1) (x-2) (x-3) (x-q-1)
sh

Example for GF(5)
GF(16) from GF(2) 4
x -1 =(x-1) (x-2) (x-3) (x-4)

GF(16) from GF(4) see example in Book For n= qm -1 ; GF(qm)

ke

21 By Nikesh Bajaj 22 By Nikesh Bajaj

Ni

Factoring xn -1 Minimal Poly. & Conj root

GF(23) - GF(8) GF(2) with x3 + x -1 (x-1) - (x-1)
(x3+x+1) - (x-z)(x-z2) (x-z2-z)
(x3+x2+1) - (x-z-1) (x-z2-z-1)(x-z2 -1)
z, z2, z+1, z2 +z, z2 +z+1, z2 +1, 1
M.Poly roots: roots:
x7 -1 =(x-1)(x-z)(x-z2)(x-z-1)(x-z2-z)(x-z2-z-1)(x-z2 -1)
(x-1) - 1 0 (= 7)

x7 -1 =(x-1)(x3+x+1)(x3+x2+1)
(x3+x+1) - z, z2 z2+z 1, 2, 4
(x3+x2+1) - z+1, z2+z+1, z2 +1 3, 6, 5

23 By Nikesh Bajaj 24 By Nikesh Bajaj

4
 11/29/2013

Minimal Polynomial Example

xn-1 = xqn-1-1 = f1(x) f2(x) f3(x) fp(x) Over GF(q) Generate GF(16) from GF(2) and find all the
Also valid on GF(qm ) Minimal Polynomial of each element
xqn-1-1 = (x-j)
j = non zero elements of GF(qm )

Concludes
Every fj(x) can be repr in GF(qm ) as product of some term
Every j is a zero of exactly one fj(x)
fj(x) = Minimal Polynomial
Smallest degree poly. With co-efficient in GF(q) and zeros in

j
GF(qm) is called a minimal poly of j

25 By Nikesh Bajaj 26 By Nikesh Bajaj

ja
Ba
Practice Sheets You should know
Generate GF(8) from GF(2) and find all the Remember
Minimal Polynomial of each element Primitive Polynomial of GF(2) upto 5 degree
Generate GF(9) from GF(3) and find all the 2nd Degree
sh
Minimal Polynomial of each element [2 1]
3rd Degree Trick to
Generate GF(16) from GF(4) and find all the

remember
[3 1] [3 2]
Minimal Polynomial of each element Taps, These
4th Degree are power
Generate GF(25) from GF(5) and find all the [4 1] [4 3] [4 3 2 1]
Minimal Polynomial of each element 5th Degree
ke

[5 2] [5 3] [5 4 32] [ 5 4 2 1] [ 5 3 2 1] [5 4 3 1]

27 By Nikesh Bajaj 28 By Nikesh Bajaj

Ni

Information Theory and Coding

ECE533

Chinese Remainder Theorem

See example in TODD K MOOK
BCH Codes
Bose-Chaudhuri-Hocquenghem

Nikesh Bajaj
nikesh.14730@lpu.co.in
Asst. Prof., ECE Dept.
29 By Nikesh Bajaj Lovely Professional University

5
 11/29/2013

BCH-Introduction BCH
Raj Chandra Bose (19 June 1901 31 October 1987) One of the most powerful class of linear cyclic block
Indian American mathematician and statistician
codes

Dwijendra Kumar Ray-Chaudhuri (Born November 1, 1933)

Bengali Indian , mathematician and a statistician Subclass of Cyclic codes
Alexis Hocquenghem (1908?-1990)
French mathematician Multiple error correcting codes
Approach is other way round
Decoding
Reed -Solomon Codes: Subset of BCH

j
31 By Nikesh Bajaj 32 By Nikesh Bajaj

ja
Ba
Mathematical Background Mathematical Background
Primitive Element Theorem: for 1 q-1 as non-zero element of GF(q)
Primitive Polynomial xq-1-1 =(x-1) (x-2) (x-3) (x-q-1)
Minimal Polynomial Example for GF(5)
sh

Set of Conj x4-1 =(x-1) (x-2) (x-3) (x-4)

ke

33 By Nikesh Bajaj 34 By Nikesh Bajaj

Ni

Primitive Cyclic codes Minimal Polynomial

xn-1 =f1(x) f2(x) f3(x) fp(x) xn-1 = xqn-1-1 = f1(x) f2(x) f3(x) fp(x)
Number of cyclic codes =2p -2 Over GF(q)
Good Codes? Also valid on GF(qm )
Primitive Block-length xqn-1-1 = (x-j)
n = qm -1 over GF(q) j = non zero elements of GF(qm )
Primitive Cyclic codes Concludes
Every fj(x) can be repr in GF(qm ) as product of some term
Every j is a zero of exactly one fj(x)
fj(x) = Minimal Polynomial
Smallest degree poly. With co-efficient in GF(q) and zeros in
GF(qm) is called a minimal poly of j

35 By Nikesh Bajaj 36 By Nikesh Bajaj

6
 11/29/2013

Example Minimal Polynomial

GF(2)---GF(8) Conjugates:
Two elements of GF(qm ) that share same minimal polynomial
over GF(q)
Conjugacy depends on base field e.g. GF(16)---GF(2) or GF(4)
Set of Conjugates
2
q q ... q
r 1

r 1
where r is smallest int. s.t q
2 r 1
f ( x) ( x )( x q )( x q )....( x q )

j
37 By Nikesh Bajaj 38 By Nikesh Bajaj

ja
Ba
Set of Conjugates: Example BCH codes: Designing
GF(256) GF(2) Primitive Block Length n = qm -1
primitive elements BCH codes defined over GF(q) with block-length of
qm -1 are called Primitive BCH Codes
Then set of conjugates
sh

{1 2 4 8 16 32 64 128} Designing Steps

{3 6 12 24 48 96 192 129} For primitive blocklength n =qm -1 to
{5} correct t-errors
1. Choose Primitive poly. Of deg. m and
construct GF(qm )
ke

2. Find minimal Polys. fi(x) of i for i= 1,2, 2t

3. Compute Generator Poly. g(x)
39 By Nikesh Bajaj 40 g(x) = LCM {f1(x)f2(x)f3(x)..f2t(x) } By Nikesh Bajaj
Ni

BCH: Designing Proof of Designing

41 By Nikesh Bajaj 42 By Nikesh Bajaj

7
 11/29/2013

BCH Design: Example g(x) for t-error for n=15

For block length n=?, For t=1
and t-errors For t=2
For m=4, GF(16) <-
GF(2)
P(x) =x4 + x +1
Minimal Poly. For t=3
Set of Conjugats

For t=4

j
43 By Nikesh Bajaj 44 By Nikesh Bajaj

ja
Ba
Questions Other Example
Design BCH Code for Block length n=7 :GF(8) <- GF(2)
Block length n=15 :GF(16)<- GF(4)
n = 8 to correct t =1 bit error, t= 2 and t=3

n = 7 to correct t =1 bit error, t= 2 and t=3

sh

n = 15 to correct t =1 bit error, t= 2 and t=3

n = 8 to correct t =1 bit error, t= 2 and t=3
ke

45 By Nikesh Bajaj 46 By Nikesh Bajaj

Ni

Decoding BCH: Gorenstein-Z Decoding BCH

Sj = v() = e() syndrome Error Locator Poly.

S1 =Y1X1 +Y2X2+ .. YvXv

S2 =Y1X12 +Y2X22+ .. YvXv2

e(x) = en-1xn-1 + en-2xn-2 + .. + e0x0. S2t =Y1X12t +Y2X22t+ .. YvXv2t

- Error location (i1 i2 .. .iv)

- Error Magnitude (ei1 ei2 eiv)

47 By Nikesh Bajaj 48 By Nikesh Bajaj

8
 11/29/2013

Example
S1 S 2 S 3 det(M)
M S 2 S 3 S 4
v=v-1
S 3 S 4 S 5
Received vector is det(M)
S1 S 2
v(x) = x5 + x3 M
S1()= v() S 2 S 3

S1 = 5 + 3 = 11
M 1
S2 = 10 + 6 = 7
S3= 15 + 9 = 7 2 8 1 11
2 1 S 3 2
S4= 20 + 12 = 14 1 M S 4 1 ( x) 8 x 2 11x 1
S5= 25 + 15 = 5

( 5 x 1)( 3 x 1)

j
S6= 30 + 18 = 14

49 By Nikesh Bajaj 50 By Nikesh Bajaj

ja
Gorenstein-Zierler
Ba
Algorithm Lets Think
Generator Poly g(x) of BCH code
Has element of base field and zeros in
Extension field
sh

What if it has element in extension field too??

Or
If g(x) has zeros in base field only???
ke

51 By Nikesh Bajaj 52 By Nikesh Bajaj

Ni

Reed Solomon Codes Reed Solomon

Subset of non-binary BCH ---- Wide range of Apps Coding for large number of bits together than individual
Storage Devices (CD, DVD, Blu-ray, Barcodes etc) bits. Non-Binary
So here GF(q) GF(q
m ) m=1
Wireless/Mobile Comm. (Cell., Telepn, Microwave links etc)
n= q
m -1 =q -1
Satellite Comm.
So minimal poly of any is in same field
Digital TV, Digital Video Broadcast
f(x) = x -
High-speed modem: ADSL, xDSL etc.
Made so many things possible..
g(x) = LCM {f1(x)f2(x)f3(x)..f2t(x) }
= xqn-1-1 = (x-j)
Polynomial codes over Certain Finite Field = (x-) (x-2 ) (x-3) . . .. (x- 2t)
Irving S. Reed and Gustave Solomon in Journal of Society g(x) = (x-i) (x-i+1 )(x-i+2 ) (x-i+3) . . .. (x- i+2t)
for industrial and Applied Mathematics, 1960 deg(g(x)) ==2t
53 By Nikesh Bajaj 54 By Nikesh Bajaj

9
 11/29/2013

Example:
RS-Code Q: Reed Solomon Codes
For block length n =15 Construct Single error correcting code for
And t = 2 block length =7
g(x)= (x-) (x-2 ) (x-3)(x-4)

=x4 + 13 x3 + 6 x2 + 3x +10
k=?
Code(15,?)
k- symbols to n-symbols

j
55 By Nikesh Bajaj 56 By Nikesh Bajaj

ja
Ba
Performance Encoding
g(x)= (x-) (x-2 )
= x2 + 4x +3
i=[ 1 0 5 2]
sh
i(x) = x4 + x2 + 5x +2
x2 i(x) =
x2 i(x)/g(x)
Reminder = x +2
c(x) = x6 + x4 + 5 x3 +2 x2 +x +2
c=[1 0 5 2 12]
ke

57 By Nikesh Bajaj 58 By Nikesh Bajaj

Ni

Circuit implementation Circuit Implementation

Working IEEE 802.15.4a : RS(63,55)
g(x) = x8 + 43x7 + 59x6 +31x5 + 10x4 +40 x3 + 14x2 +7x+36

59 By Nikesh Bajaj 60 By Nikesh Bajaj

10
 11/29/2013

RS Code RS Code
RS is MDS Table 5.7 in RB
Design distance d = 2t+1 Code t d* r
Singleton bound d* <= n - k +1 (3 , 1) 1
RS : 2t = n-k so d* = n k + 1 (7, 5) 1
(7, 1) 3
(15, 13) 1
Hardware

(15, 1) 7
(255, 245) 5
Software (on 1.6GHz Pentium PC) (255, 223) 16
RS(255,251) ~ 120Mbps t=2 (255, 155) 50
RS(255,239) ~ 30Mbps 8

j
RS(255, 223) ~10Mbps 16
61 By Nikesh Bajaj 62 By Nikesh Bajaj

ja
Ba
Next
Convolutional Codes
Turbo/TCM
Bounds
sh

ke

63 By Nikesh Bajaj 64 By Nikesh Bajaj

Ni

65 By Nikesh Bajaj 66 By Nikesh Bajaj

11
 11/29/2013

j
67 By Nikesh Bajaj 68 By Nikesh Bajaj

ja
Ba
Cyclic Code
Cyclic Code
Linear and
cyclic shift
sh
Ring
Polynomial in GF(q)
Deg, monic, + , * mod
Exp GF(3)
ke

69 By Nikesh Bajaj 70 By Nikesh Bajaj

Ni

12