Linear Regression Model in Matrix Form

V c Hong V
University of Economics HCMC

June 2015

V c Hong V (UEH)

Applied Econometrics

June 2015

1 / 13

Basic Definitions

Definition D. 1 (Matrix). A matrix is a rectangular array of

numbers. More precisely, an m x n matrix has m row and n
columns. The positive integer m is called the row dimension, and
n is called the column dimension
We can write an m x n matrix generically as

a11 a12 a13 . . . a1n

a21 a22 a23 . . . a2n

A = [aij ] = ..

am1 am2 am3 . . . amn

V c Hong V (UEH)

Applied Econometrics

June 2015

2 / 13

Definition D. 2 (Square matrix). A square matrix has the

same number of rows and columns. The dimension of a square
matrix is its number of rows and columns.
Definition D. 3 (Vectors): A 1 x m matrix is called a row
vector (of demension m) and can be written as
x (x1 , x2 , . . . , xm )
An nx1 matrix is called a column
and can be written as

vector
y1
y2

y ..
.
yn

V c Hong V (UEH)

Applied Econometrics

June 2015

3 / 13

Definition D. 4 (Diagonal Matrix). A square matrix A is a

diagonal matrix when al of its off-diagonal are zero, that is,
aij = 0 for all i 6= j. We can always write a diagonal matrix as

a11 0 0 . . . 0
0 a22 0 . . . 0

A = ..

0
0 0 . . . ann

V c Hong V (UEH)

Applied Econometrics

June 2015

4 / 13

Definition D. 5 (Identity and Zero Matrix). The n x n

identity matrix, denoted I, to emphasize its dimension, is the
diagonal matrix with unity (one) in each diagonal position, and
zero elsewhere:

1 0 0 ... 0
0 1 0 . . . 0

I I n ..

0 0 0 ... 1

The m x n zero matrix, denoted 0, is the m x n matrix with

zero for all entries. This need not be a square matrix.

V c Hong V (UEH)

Applied Econometrics

June 2015

5 / 13

Matrix Operations

Matrix Addition
Two matrices A and B, each having dimension mxx, can be added
element by element: A + B = [aij + bij ]. More precisely,

a11 + b11 a12 + b12 . . . a1n + b1n

a21 + b21 a22 + b22 . . . a2n + b2n

A +B =

..

.
am1 + bm1 am2 + bm2 . . . amn + bmn

V c Hong V (UEH)

Applied Econometrics

June 2015

6 / 13

Matrix Operations (cont.)

Scalar Multiplication
Given any real number (often called a scalar), scalar
A [aij ], or
multiplication is defined as A

a11 a12 . . . a1n

a21 a22 . . . a2n

A = ..
A

am1 am2 . . . amn

V c Hong V (UEH)

Applied Econometrics

June 2015

7 / 13

Matrix Operations (cont.)

Matrix Multiplication
To multiply matrix A by matrix B to form the product AB, the
column dimension of A must equal the row dimension of B.
Therefore, let A be an m x n matrix and let B be an n x p matrix.
Then, matrix multiplication is defined as

n
X
AB = [
aik bkj ]
k=1

V c Hong V (UEH)

Applied Econometrics

June 2015

8 / 13

Matrix Multiplication (cont.)

V c Hong V (UEH)

Applied Econometrics

June 2015

9 / 13

Transpose

Definition D. 6 (Transpose). Let A = [aij ] be an m x n

matrix. The transpose of A, denoted A (called A prime), is
the n x m matrix obtained by interchanging the rows and
columns of A. We can write this as A 0 [aji ]


2 1 7
A=
4 5 0

2 4
A 0 = 1 5
7
0

V c Hong V (UEH)

Applied Econometrics

June 2015

10 / 13

Properties of Transpose

I
I
I

A 0 )0 = A
(A
A)0 = A
A0
(A
AB
(AB
AB)0 = B 0A 0

V c Hong V (UEH)

Applied Econometrics

June 2015

11 / 13

The Linear Regression Model in Matrix Form

The multiple linear regression model with k parameters is expressed
as follows:
yi = 0 + 1 x1i + 2 x2i + . . . + k xki , i = 1, 2, . . . , n

y1
Let y = . . .,
yn

1 x12 . . . x1k

.. ,
X = ... ...
.
1 xn2 . . . xnk

1
..
b= . ,
k

e1
..
e=.
en

So, y = Xb + e

V c Hong V (UEH)

Applied Econometrics

June 2015

12 / 13

OLS in Matrix Form

Minimize:

Pn

2
i=1 ei

= e 0e

MinS(b) = e0 e = (y Xb)0 (y Xb) = y0 y 2y0 Xb + b0 XXb (1)

Take derivatie of S(b) w.r.t. b:
S(b)
= 2(X 0 y X 0 Xb) = 0 X 0 Xb = X 0 y b = [X 0 X ]1 X 0 y
b

V c Hong V (UEH)

Applied Econometrics

June 2015

13 / 13