G Lecture05

5.
Week 5
Regression through the origin

&
Multiple Regression
(Partial and overall significance test)
Gujarati(2003): Chapter 6, 7
All rights reserved by Dr.Bill Wan Sing Hung - HKBU
5.2
The intercept term is absent or zero.
i.e., Yi = b 2 X i + ui
Yi ^ ^
SRF : Yi = b 2 X i
^
b2
0 Xi

5.3
The estimated model:
~ ~
Y = b2 X i
~ ~
or Y = b2 X i +  i
Applied OLS method:
( )
^2
s
~  X iYi ~
and Var b2 =  X2
b2 =
 Xi 2 i
and  u~2
s^ 2 =
N -1
5.4
Some feature of interceptless model
1. u
~
i need not be zero
can be occasions turn out to be negative.
2. R2 may not be appropriate for the summary
of statistics.
3. df does not include the constant term,

i.e., (n-k)
In practice:
•1. A very strong priori or theoretical expectation, otherwise stick
to the conventional intercept-present model.
•2. If intercept is included in the regression model but it turns out
to be statistically insignificant, then we have to drop the intercept
to re-run the regression.
Regression through origin 5.5
Y = b 1 + b 2 X+ u i Y = b2 X + u’i
^  xy  XY
b2 = ~
=
 x2 b2  X2
^ s
^2
Var (b2 ) =
^
s2
Var (b2 )=
~
x2  X2
s^ 2 = 
^2
^ 2 =  u’
u ~2
s
n-2 n -1
=
[ (X- X)(Y- Y)]
2
 ( XY )
2
R
 (X- X)  (Y- Y)
2
2 2
raw R = 2
( xy ) 2  X2  Y2
or R =2
x y 2 2
5.6
Y
False SFR:
^ ^ X
Y = β’ 2
True and best SFR:

^ ^
Y = β1+ β^2X
^
b1
X
10 50

5.7
Y
True and best SFR:

^ ^
Y = β1+ β^2X
False SFR:
^
Y = ^β2X
5
^
b1
X
10 50

5.8
Y
True and best SFR:
^
Y = ^β X 2
False SFR:
^ ^
Y =β1+ β^2X False SFR:
^ ^ ^X
Y = β1+ β’ 2
^
b1
X
10 50

5.9
Example 1: Capital Asset Pricing Model (CAPM) security i’s
expect risk premium=expected market risk premium
( ER i
- r f
)= b2 ( ER m
- r f
)
expected risk free expected

rate of of rate of
return on return return on
security i market
portfolio
b 2 as a measure of systematic risk.

b 2 >1 ==> implies a volatile or aggressive security.
b 2 <1 ==> implies a defensive security.
5.10
Example 1:(cont.)
ER i - r f Security market line
b2
1
ERm - f

5.11
Example 2: Covered Interest Parity
F - e
(i - i ) = = fN
* N
e
International interest rate differentials equal exchange rate
forward premium.
- e
i.e., (i - i ) = b2 (
* F
)
e
i - i*
Covered interest
parity line
b2
1
F -e
e
5.12
Example 2:(Cont.)
in regression: E (b1 ) = 0
F -e
(i - i ) = b 1 + b 2(
*
) + ui
e
If covered interest parity holds, b1 is expected to be zero.

5.13
y: Return on A Future Fund, %
X: Return on Fisher Index, %
^=
Formal report: Y 1.0899 X R2=0.714 N=10
(5.689) SEE=19.54
5.14
H0: b 1 = 0
1.279 - 0
7.668
^=
Y 1.2797 + 1.0691X R2=0.715 N=10
(0.166) (4.486) SEE=20.69
The t-value shows that b1Allisrightsstatistically insignificant different from zero

reserved by Dr.Bill Wan Sing Hung - HKBU
5.15
Multiple Regression
Y = b1 + b2 X2 + b3 X3 +…+ bk Xk + u

5.16
Derive OLS estimators of multiple regression
^ ^ ^
Y = b1 + b2X2 + b3X3 + ^u
^ ^ ^
u^ = Y - b1 - b2X2 - b3X3
OLS is to minimize the SSR( ^2)
^ ^ ^
min. RSS = min.  ^2 = min. (Y - b1 - b2X2 - b3X3)2
RSS ^ ^ ^
^ =2  ( Y - b - b X - b 3X3)(-1) = 0
b1 1 2 2
RSS ^ ^ ^
^ =2  ( Y - b - b X - b 3X3)(-X2) = 0
b2 1 2 2
RSS ^ ^ ^
^ =2  ( Y - b - b 2 2 b3X3)(-X3) = 0
X -
b3 1

5.17
rearranging three equations:
^ ^ ^
nb1 + b2 X2 + b3  X3 = Y
^ X + b^ X 2 + b^  X X = X Y
b 2 2 2 3 3 2 3 2
^ X + b^ X X + b^  X 2 = X Y
b 1 3 2 2 3 3 3 3
rewrite in matrix form:

n X2 X3 ^
b 1
Y 2-variables Case
X2 X22 X2X3 ^
b 2 =  X2Y
3-variables Case
X3 X2X3 X3 2 ^
b 3
X3Y
^
(X’X) b = X’Y Matrix notation

Cramer’s rule: 5.18
n Y X3
X2 X2Y X2X3
X3 X3Y X32 (yx2)(x32) - (yx3)(x2x3)
^ =
b2 =
n X2 X3 (x22)(x32) - (x2x3)2
X2 X22 X2X3
X3 X2X3 X32
n X2 Y
X2 X22 X2Y
X3 X2X3 X3Y (yx3)(x22) - (yx2)(x2x3)
^ =
b3 =
n X2 X3 (x22)(x32) - (x2x3)2
X2 X22 X2X3
X3 X2X3 X32
_ ^_ ^_
^
b1 = Y - b2X2 - b3X3
5.19
or in matrix form:
(X’X) b^ = X’Y
3x3 3x1 3x1
^ = (X’X)-1 (X’Y)
==> b
3x1 3x3 3x1
^ ^ ^  ^
u 2
Var-cov(b) = su2 (X’X)-1 and su2 =
n-3
Variance-Covariance matrix
^ ^ ^ ^ ^
Var(b1) Cov(b1 b2) Cov(b1 b3)
^ ^ ^
Var-cov(b) = Cov (b2 b1) Var(b^2) Cov(b^ b
2 3
^)
Cov (b^3 b^1) Cov(b
^ ^b )
3 2 Var(b^ )
3
^
= su2(X’X)-1
5.20
-1
n X2 X3
^ X2 X22 X2X3
= su2
X3 X3X2 X32
^ 2= u^2 u^2
and s u n-3 = n- k
k=3
# of independent variables
( including the constant term)

5.21
The meaning of partial regression coefficients
Y = b1 + b2X2 + b3X3 + u (suppose this is a true model)
Y
= b2 : b2 measures the change in the mean
X2 values of Y, per unit change in X2,
holding X3 constant.
or The ‘direct’ or ‘net’ effect of a unit change in
X2 on the mean value of Y
Y holding X2 constant, the direct effect
= b3
X3 of a unit change in X3 on the mean
value of Y.
Holding constant:
To assess the true contribution of X2 to the change in
Y, we control the influence of X3.
5.22
Properties of multiple OLS estimators
_ _ _
1. The regression line(surface)passes through the mean of Y1, X2, X3
_ _ _
i.e., ^ ^ ^
b1 = Y - b2X2 - b3X3 Linear in parameters
_
^ ^_ ^ _ Regression through the mean
==> Y = b1 + b2X2 + b3X3
_
2. ^ = Y + b^ x + b^ x
Y 2 2 3 3
Unbiased: E(b^)=b
i i
or y =^ b x +^
2 2 bx 3 3
Zero mean of error

3. u=0
^
4. ^ = uX
uX1
^ =0
2
^ k=0 )
(uX constant Var(ui) = s2
5. ^^
uY=0 random sample
Properties of multiple OLS estimators 5.23
^
6. As X2 and X3 are closely related ==> var(b2) and var(b^3)
become large and infinite. Therefore the true values of b2
and b3 are difficult to know.
All the normality assumptions in the two-variables case regression
are also applied to the multiple variable regression.
But one addition assumption is
No exact linear relationship among the independent variables.
(No perfect collinearity, i.e., Xk  Xj )
7. The greater the variation in the sample values of X2 or
X3, the smaller variance of b^2 and b
^ , and the
3
estimations are more precisely.
8. BLUE (Gauss-Markov Theorem)

The adjusted R2 (R2) as one of indicators of the overall fitness 5.24
ESS RSS u^2
R2 = =1- =1-
TSS TSS y2
_ ^2 / (n-k)
u
R2 = 1 - k : # of independent
y / (n-1)
2
variables plus the
_ ^s2 constant term.
R2 = 1 -
SY2
n : # of obs.
_ ^
u 2 (n-1)
2
R =1-
y2 (n-k)
_
R2 = 1 - (1-R2) n-1
n-k
_
R2  R2 0 < R2 < 1
Adjusted R2 can be negative: R2  0
Note: Don’t misuse the adjusted R2, Gujarati(2003) pp. 222
5.25
Y = b1 + b2 X2 + b3 X3 + u
Y TSS
sY n-1
s^ u

5.26
Suppose X4 is not an explanatory

Variable but is included in regression
C
X2
X3
X4

5.27
Hypothesis Testing in multiple regression:
1. Testing individual partial coefficient
2. Testing the overall significance of all coefficients
3. Testing restriction on variables (add or drop): Xk = 0 ?
4. Testing partial coefficient under some restrictions
Such as b2+ b3 = 1;
or b2 = b3 (or b2+ b3 = 0); etc.
5. Testing the functional form of regression model.
6. Testing the stability of the estimated regression model

-- over time
-- in different cross-sections

5.28
1. Individual partial coefficient test
1 holding X3 constant: Whether X2 has the effect on Y ?
Y
H0 : b2 = 0 = b2 = 0?
X2
H1 : b2  0
^
b2 - 0 0.726
t= = = 14.906
Se (b^ )2 0.048
Compare with the critical value tc0.025, 12 = 2.179
Since t > tc ==> reject Ho
^
Answer : Yes, b2 is statistically significant and is
significantly different from zero.

5.29
1. Individual partial coefficient test (cont.)
2 holding X2 constant: Whether X3 has the effect on Y?
Y
H0 : b3 = 0 = b3 = 0?
X3
H1 : b3  0
^
b3 - 0 2.736-0
t= = = 3.226
Se (b^ ) 0.848
3
Critical value: tc0.025, 12 = 2.179
Since | t | > | tc | ==> reject Ho
^
Answer: Yes, b3 is statistically significant and is
significantly different from zero.
5.30
2. Testing overall significance of the multiple regression
3-variable case: Y = b1 + b2X2 + b3X3 + u

H0 : b2 = 0, b3 = 0, (all variable are zero effect)
H1 : b2  0 or b3  0 (At least one variable is not zero)
1. Compute and obtain F-statistics

2. Check for the critical Fc value (F c , k-1, n-k)
3. Compare F and Fc , and

if F > Fc ==> reject H0

Analysis of Variance: Since y=^y+u 5.31
^2
^2 + u
==> y2 = y
TSS = ESS + RSS
ANOVA TABLE
(SS) (MSS)
Source of variation Sum of Square df Mean sum of Sq.
Due to regression(ESS)  y^2 k-1 y^2
k-1
Due to residuals(RSS)  u^2 n-k u^2 ^2
n-k = s u
Total variation(TSS)  y2 n-1

Note: k is the total number of parameters including the intercept term.
MSS of ESS  ^y2/(k-1)

ESS / k-1
F= = = ^ 2 /(n-k)
MSS of RSS RSS / n-k  u
H0 : b2 = … = bk = 0
if F > Fck-1,n-k ==> reject Ho
H1 : b2  …  bk  0
5.32
^ ^
Three- y = b2x2 + b3x3 + u^
variable ^ ^
 y2 = b2  x2 y + b3  x3 y +  u^2
case
TSS = ESS + RSS
ANOVA TABLE
Source of variation SS df(k=3) MSS
ESS ^ x y + b^ x y
b 3-1 ESS/3-1
2 2 3 3
RSS u
^2 n-3 RSS/n-3
(n-k)
TSS y2 n-1
ESS / k-1 (b^2 x2y + b^3 x3y) / 3-1

F-Statistic = =
RSS / n-k u
^2 / n-3
5.33
An important relationship between R2 and F
ESS / k-1 ESS (n-k)
F= =
RSS / n-k RSS (k-1)
ESS n-k
=
TSS-ESS k-1 For the three-variables case :
ESS/TSS n-k
= R2 / 2
ESS F=
1 - TSS k-1 (1-R2) / n-3
R2 n-k
=
1 - R2 k-1
R2 / (k-1) (k-1) F
F = Reverse : R2 =
(1-R2) / n-k (k-1)F + (n-k)

R2 and the adjusted R2 (R2) 5.34
ESS RSS u^2

R2 = =1- =1-
TSS TSS y2
_ ^2 / (n-k)
u
R2 = 1 - k : # of independent
y / (n-1)
2
variables
_ ^s2 including the
R2 = 1 -
SY2 constant term.
n : # of obs.
_ ^2
u (n-1)
2
R =1-
y2 (n-k)
_
R2 = 1 - (1-R2) n-1
n-k
_
R2  R2 0 < R2 < 1
Adjusted R2 can be negative: R2  0
Overall significance test: 5.35
H0 : b2 = b3 = b4 = 0
H1 : at least one coefficient
is not zero.
b2  0 , or b3  0 , or b4  0
R2 / k-1
F* = 2
=
(1-R ) / n- k
0.9710 / 3
=
(1-0.9710) /16
= 179.13
Fc(0.05, 4-1, 20-4) = 3.24
 k-1 n-k
Since F* > Fc ==> reject H0.
Construct the ANOVA Table (8.4) .(Information from EViews) 5.36
Source of SS Df MSS
variation
2 2 2 2
Due to R (y ) k-1 R (y )/(k-1)
regression =(0.971088)(28.97771)2x19
(SSE) =15493.171 =3 =5164.3903
(1- R )(y ) or (   )
2 2 2 2 2
Due to n-k (1- R )(y )/(n-k)
Residuals =(0.0289112)(28.97771) )2x19
(RSS) =461.2621 =16 =28.8288
2
Total (y ) n-1
(TSS) =(28.97771) 2x19
=15954.446 =19
Since (sy)2 = Var(Y) = y2/(n-1) => (n-1)(sy)2 = y2
MSS of regression 5164.3903

F* = = = 179.1339
MSS of residual 28.8288
Example:Gujarati(2003)-Table6.4, pp.185) 5.37
H0 : b 1 = b 2 = b 3 = 0
* ESS / k-1 R2 / k-1 0.707665 / 2
F = = =
RSS/(n- k) 2
(1-R ) / n- k (1-0.707665)/ 61
F* = 73.832
Fc(0.05, 3-1, 64-3) = 3.15
 k-1 n-k
Since F* > Fc
==> reject H0.

Construct the ANOVA Table (8.4) .(Information from EVIEWS) 5.38
Source of SS Df MSS
variation
2 2 2 2
Due to R (y ) k-1 R (y )/(k-1)
regression =(0.707665)(75.97807)2x64
(SSE) =261447.33 =2 =130723.67
(1- R )(y ) or (   )
2 2 2 2 2
Due to n-k (1- R )(y )/(n-k)
Residuals =(0.292335)(75397807)2x64
(RSS) =108003.37 =61 =1770.547
2
Total (y ) n-1
(TSS) =(75.97807)2x64
=369450.7 =63
Since (sy)2 = Var(Y) = y2/(n-1) => (n-1)(sy)2 = y2
MSS of regression 130723.67

F* = = = 73.832
MSS of residual 1770.547
5.39
Y = b1 + b2 X2 + b3 X3 + u
H0 : b2 = 0, b3= 0,
H1 : b2  0 ; b3  0
Fc0.01, 2, 61 = 4.98
Compare F* and Fc, checks the F-table:
Fc0.05, 2, 61 = 3.15
Decision Rule:
Since F*= .73.832 > Fc = 4.98 (3.15) ==> reject Ho
Answer : The overall estimators are statistically significant

different from zero.

G Lecture05

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

G Lecture05

Încărcat de

Drepturi de autor:

Formate disponibile

5.

Regression through the origin

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

3. df does not include the constant term,

True and best SFR:

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

True and best SFR:

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

expected risk free expected

b 2 as a measure of systematic risk.

ER i - r f Security market line

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

The t-value shows that b1Allisrightsstatistically insignificant different from zero

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

rewrite in matrix form:

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

Zero mean of error

8. BLUE (Gauss-Markov Theorem)

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

Suppose X4 is not an explanatory

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

6. Testing the stability of the estimated regression model

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

Compare with the critical value tc0.025, 12 = 2.179

Since t > tc ==> reject Ho

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

Critical value: tc0.025, 12 = 2.179

Since | t | > | tc | ==> reject Ho

3-variable case: Y = b1 + b2X2 + b3X3 + u

1. Compute and obtain F-statistics

3. Compare F and Fc , and

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

Total variation(TSS)  y2 n-1

MSS of ESS  ^y2/(k-1)

ESS / k-1 (b^2 x2y + b^3 x3y) / 3-1

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

ESS RSS u^2

Since (sy)2 = Var(Y) = y2/(n-1) => (n-1)(sy)2 = y2

MSS of regression 5164.3903

Fc(0.05, 3-1, 64-3) = 3.15

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

Since (sy)2 = Var(Y) = y2/(n-1) => (n-1)(sy)2 = y2

MSS of regression 130723.67

Answer : The overall estimators are statistically significant

All rights reserved by Dr.Bill Wan Sing Hung - HKBU

S-ar putea să vă placă și