Lecture 3

¾ Introduction to Vector Space Theory

¾ Matrices

1
 Block codes: basic definitions
• An alphabet is a discrete (usually finite) set of
symbols.
example: B = { 0; 1} is the binary alphabet
• Definition: A block code of blocklength n over an
alphabet X is a nonempty set of n-tuples of
symbols from X.
• The n-tuples of the code are called codewords.
• Codewords are vectors whose components are
symbols in X.

2
 Block codes: basic definitions

• Codewords of length n are typically generated

by encoding messages of k information bits
using an invertible encoding function.
• Number of codewords is M = 2k , Rate R = k/n
• The rate is a dimensionless fraction; the fraction
of transmitted symbols that carry information.
• A code with blocklength n and rate k/n is called
an (n; k) code

3
Linear Block Codes

c = mG
− −

where,
c − code word (vector)
−

m − message word (vector)

−

G − Generator matrix

4
 Vector Space-Introduction
• An n-dimensional vector has a form
x = ( x1, x2, x3, …, xn ) .
• The set Rn of n-dimensional vectors is a vector
space.
• Any set V is called a vector space if it contains
objects that behave like vectors:
• ie, they add & multiply by scalars according to
certain rules. In particular, they must be closed
under vector addition and scalar multiplication.
• But addition & scalar multiplication need not be
defined conventionally!
5
 Contd…
• Let V denote the vector space.The addition on V
is vector addition.The scalar multiplication
combines a scalar from a Field F and a vector
from V. Hence V is defined over a field F.
• V must form a commutative group under addition
• For any element a in F and any element v in V,
a.V is an element in V.
• Distributive law- a.(u+v)=a.u+a.v
• Associative law- (a.b).v=a.(b.v)

6
 Contd….
Important vector spaces:
• R, R2, R3, Rn with usual + and scalar multn.
• Mmn ; the set of all m x n matrices
• Pn; all polynomials of degree ≤ n
• Consider a vector space over binary field
F2.Consider the sequence u=u0…un-1 where the
ui ‘s are from {0,1}.We can construct such 2n n-
tuples over F2.Let Vn denote this set. Vn is a
Vector space over F2

7
 Subspaces
• A set W of vectors is a subspace of vector
space V if and only if W is a subset of V and
W is itself a vector space under the same
addition and scalar multiplication.
• For any two vectors u,v ε W, (u+v) ε W.
• For any element a in F and any u in W, a.u
must be in W.

8
 Contd…
To test if W is a Subspace
• We should, but need not, check all the properties
of a vector space in W: most hold because W’s
vectors are also in the bigger vector space V.
• But we must check closure in W: linear
combinations of vectors in W must also lie in W.
• This means the zero & additive inverses must
be in W too.

9
 Examples
• Let u1,….,uk be a set of k vectors in V over
a field F. The set of all linear combinations
of u1,….,uk forms a subspace of V.
• The set of polys of degree 2 or less is a
subspace of the set of polynomials of degree
3 or less.
• The set of integers is not a subspace of R,
because the set of scalars includes fractions,
eg 1/2.
10
 Spanning Sets &Linear Independence

• A set S = {u1,u2,.......,un} of vectors is said to span a

vector space V if every vector in V can be
expressed as a linear combination of the vectors in
S.
• Ex:( x, y, z ) = x i + y j + z k, so every vector in R3 is
a linear combination of i, j & k.
• If any vector in a set can be expressed as a
linear combination of the others, we call the
set linearly dependent. If not, the set is linearly
independent.
11
 Basis set
• A set of linearly independent vectors is a
basis for a Vector space V if each vector in V
can be expressed in one and only one way as a
linear combination of the set.
• In any Vector space or subspace there exists at
least one set B of linearly independent vectors
which span the space.
• The no. of vectors in the Basis of a Vector
space is the dimension of the Vector space.
• One example of a basis are the vectors
(1,0,…,0), (0,1,…,0),…, (0,0, …, 1).
12
 Orthogonality
Let u= (u0 , u1 ,.....un −1 ) and
v= (v0 , v1 ,....vn −1 )
be two n-tuples in Vn. We define the inner product(dot product) as
u.v= u0 v0 + u1v1 + ........ + un −1vn −1 where the multiplication and addition are
carried out in mod-2.. The inner product is a scalar. If u.v=0, then u and v
are said to be orthogonal to each other
The inner product has the following properties
(1) u.v=v.u
(2) u.(v+w)=u.v+u.W
(3)(au).v=a(u.v)
.
13
 Matrices
A k x n matrix over F2 is a rectangular array with
k rows and n columns.
⎡ g00 g01 g02 ..... g0,n−1 ⎤
⎢ g g g ..... g ⎥
⎢ 10 11 12 1,n−1 ⎥

G=⎢ . . . . . ⎥
⎢ ⎥
⎢ . . . . . ⎥
⎢ gk −1,0 gk −1,1 gk −1,2 ..... gk −1,n−1 ⎥
⎣ ⎦

where each gij with 0 ≤ i ≤ k and 0 ≤ j ≤ n

is an element from the binary field F2.
14
 • G is also represented by its k rows
g 0 , g 0 ,.....g k −1 as
⎡ g0 ⎤
⎢ g ⎥
⎢ 1 ⎥
G=⎢ . ⎥
⎢ ⎥
⎢ . ⎥
⎢⎣ g k −1 ⎥⎦

Each row of G is an n-tuple and each column is a k-tuple over F2.

15
• If k (with k ≤ n ) rows of G are linearly
independent, then the 2k linear combinations
of of these rows form a k dimensional
subspace of the vector space Vn of all the n-
tuples over F2. This subspace is called the
row space of G

• Elementary row operations will not change

the row space of G

16
• Let S be the row space of a k x n matrix G over
F2 whose rows are linearly independent. Let Sd be
the null space of S. Then the dimension of Sd is
n-k. Consider (n-k) linearly independent vectors
in Sd. These vectors span Sd. We can form an
(n-k) x n matrix H as
⎡ h0 ⎤ ⎡ h00 h01 h02 ..... h0,n −1 ⎤
⎢ h ⎥ ⎢ h h11 h12 ..... h1,n −1 ⎥⎥
⎢ 1 ⎥ ⎢ 10
H =⎢ . ⎥=⎢ . . . . . ⎥
⎢ ⎥ ⎢ ⎥
⎢ . ⎥ ⎢ . . . . . ⎥
⎢⎣ g n − k −1 ⎥⎦ ⎢⎣ hn − k −1,0 hn − k −1,1 hn − k −1,2 ..... hn − k −1,n −1 ⎥⎦
17
The row space of H is Sd
• Since each row gi is a vector in S and each
row hj of H is a vector in Sd, the inner
product of gi and hj must be zero. As the
row space S of G is the null space of the
row space Sd of H, S is called the null space
or dual space of H.

18
 Theorem
• For any k x n matrix G over F2, with k linearly
independent rows, there exists an (n-k) x n
matrix over the same field with (n-k) linearly
independent rows such that for any row gi in
G and any hj in H, gi.hj = 0. The row space of
G is the null space of H and vice versa.

19