EMET2007 Notes

Jaron Lee
April 23, 2014
1 Introduction
1.i Parameters
For the model y =
0
+
1
x +u,
1
=
y
x
as long as
u
x
= 0.
For the multiple linear model,
1
=
y
x
1
if all other factors are held constant.
1.ii Logarithms
In the log-log models,
1
represents an elasticity i.e.
1
=
%y
%x
.
In lin-log models (Y = . . . +1 log(X)) the interpretation of
1
is
1
= 100
y
%x
In log-lin models
1
= 100
%y
x
.
2 Mathematical review
2.i Summations
Claim:

X
i
(Y
i


Y ) =

(X
i


X)(Y
i


Y )
Proof:
RHS =

(X
i


X)(Y
i


Y )
=

(X
i
(Y
i


Y )

X(Y
i


Y ))
=

X
i
(Y
i


Y )

X

(Y
i


Y )
=

X
i
(Y
i


Y )

X(

Y
i
n

Y )
=

X
i
(Y
i


Y )

X(n

Y n

Y ) recall denition of mean of y

= LHS
2.ii Descriptive statistics
V ar(Y
1
, Y
2
, . . . , Y
n
) =
1
n

(Y
i


Y )
2
=
1
n

Y
2
i


Y
2
Cov(X, Y ) =
1
n

X
i
Y
i


X

Y
Cor(X, Y ) =
Cov(X, Y )

y
V ar(X) = E[(X E(X))
2
] = E[X
2
] (E[X])
2
V ar(

Y
i
) =

V ar(Y
i
) +

Cov(Y
i
, Y
j
)
1
3 Monte Carlo Simulations
These simulations produce the range of possible values which we can obtain for our estimator when
we sample from a given model and population. The simulation mimcs the sampling distribution
of the estimator.
3.i Data Generating Process
E(
i
) = 0
V ar(
i
) =
2
Importantly, this is constant.
Cov(
i
,
j
) = 0 for i = k
Xs are xed across samples.
3.ii Criteria for selecting estimators
Mean Error = E[
g
] Measures the bias
Mean Absolute Error = E[|
g
|] Cost of mistake proportional to size
Mean Square Error = E[(
g
)
2
] Cost of mistake proportional to square of size
4 Simple Linear Regression
DGP assumptions are the assumptions we make about the way the underlying process works and
how it leads to the data. Problems occur if 1. the DGP assumptions are not a useful approximation
or 2. the estimator used did a poor job for the DGP given.
4.i SLM, No intercept
We want to nd the Best Linear Unbiased Estimator for the SLM with no intercept.
Linear: if

=

w
i
Y
i
Unbiased: if E[

] =
We start with the linear estimators. Then we impose the condition that they be unbiased
(

w
i
X
i
= 1). Finally we calculate the variance of the linear estimator (V ar(

w
i
Y
i
) =
2

w
2
i
).
For our DGP, the best estimators will place more weight on observations with high values of
X. The reason behind this is that inferences form those observations will be less sensitive to the
eects of the same - recall that V ar() is constant.
The mean squared error looks at the spread of possible values of the estimator around the true
value. It is given as MSE = E[(
g
)
2
] = V ar(
g
) + (Bias(
g
))
2
.
V ar(

w
i
Y
i
) =
2

w
2
i
V ar(

OLS
) =

2

(X
2
k
)
2
4.ii SLM, Intercept
The OLS estimator is also the BLUE in this case.
E(

0
) =
0

w
0i
+
1

w
0i
X
i
E(

1
) =
0

w
1i
+
1

w
1i
X
i
For an unbiased

1
we can set

w
i
= 0 and

w
i
X
i
= 1.
Alternatively we can reach the OLS by dening residuals, forming the SSR, and minimising
SSR over
0
,
1
. The tted regression always passes through (

X,

Y ).
The formulas are
1
=

(X
i

X)Y
i

(X
i

X)
2
.
0
=

Y
1

X.
4.iii Summary
Given the Gauss Markov assumptions, OLS is unbiased, ecient (gives the smallest possible vari-
ance).
5 Interpreting results
5.i Residuals
We want to use the resdiuals to estimate the variance of the errors. The unbiased estimator for
the standard error of the regression s is s =

1
nk1

e
2
i
. k denotes the number of independent
variables.
5.ii Variance
The estimated standard error of the slope is e.s.e.(

1
) =

s
2

(X
i

X)
2
. This is distinct from the
standard error of the slope which is s.e.(

1
) =

(X
i

X)
2
.
5.iii Condence Intervals
C.I.(1 ) =

t
nk1,

2
e.s.e(

)
6 Hypothesis Testing
6.i Type I,II errors
TYPE I: Reject null when it is true
TYPE II: Dont reject null when it is false
6.ii T - statistics
For H
0
:
1
=

1
the t-stat is as follows

t =

1
e.s.e(

1
)
This is distributed on a t-distribution with n k 1 degrees of freedom.
3
6.iii Signicance
The maximum chance we are willing to risk committing a type I error (reject the null when it is
true) is .
6.iv Summary
1. State the null and alternative hypotheses
2. Choose a test
3. Choose a signicance level
4. Find critical region
5. Find test statistic
6. Reject the null if the test statistic falls in the region
7 R
2
SST =

(Y

Y )
2
SSR =

(Y

Y )
2
SSM =

Y

Y )
2
SST = SSM +SSR
R
2
=
SSM
SST
= 1
SSR
SST
Important notes about R
2
:
R
2
measures the fraction of total variation in Y that can be explained by the independent
variables.
Adding more variables always increases R
2
R
2
measures how well the model does in explaining the variation of the sample.
R
2
does not measure how well the model does in estimating the parameters of the DGP.
High values of R
2
does not mean the model does a good job.
Adjusted R
2
is a meausre used to compare models with dierent numbers of independent
variables. Adjusted R
2
increases if the added variable has |t
stat
| > 1.
8 Specication Errors
8.i Superuous variables
If we regress an equation which contains a variable X
i
not present in the DGP, X
i
is superuous.
The true parameter value is
i
= 0. The model is unbiased, but the variance of the estimated
coecients for other variables is increased.
4
8.ii Omitted variables
Regressing an equation which does not contain variables of the DGP will generally produce a biased
estimate of the parameters.
If we have a biased estimator
1
of
1
such that E(
1
) =
1
+
2

w
i
X
2i
then the bias will
depend on two elements: 1. the eect X
2
has on Y and 2. the correlation between X
2
and X
1
. If
both are nonzero then bias occurs.
Calculating the sign of each of these elements can allow one to calculate the sign of the bias.
Omission of variables will cause the variance of the estimator to decrease. Consider that
V ar(
1
) =

2
SST
1
(1R
2
1
. The variance of the term
1
is approximated by V ar(
1
) =

2
SST
1
. Then it
is clear that V ar(
1
) < V ar(
1
).
Omitted variables bias is a problem when you want the relationship between X
1
and Y with
X
2
(the omitted variable) held constant. If you want the total eect of X
1
on Y the omitting X
2
is permissible.
9 Topics on variables
9.i Multicollinearity
Consider the variance of an estimator in a multiple linear regression: V ar(

j
) =
s
2
SST
j
(1R
2
j
)
. Note
that SST
j
=

(X
ji


X
j
)
2
.
If X
1
and other Xs are closely related then the R
2
term will be close to one. The variance
and s.e. of

1
will be large. The corresponding t-statistic (calculated against a hypothesis that the
coecient is zero) may be small, not reecting whether the variable is signicant.
If X
1
and other Xs are perfectly related then OLS fails.
Finally, the relationship between any number of Xs will not aect the estimation of the coef-
cient some other uninvolved X.
If two variables are heavily correlated, but either one is removed, not much explanatory power
is lost since the other variable picks up the slack via omitted variable bias. Yet each coecient
will have a large standard error.
9.ii Dummy variables
A binary variable used to quantify qualitative data. Consider: log(Y ) = . . . +
i
X
i
, where X
i
is a
dummy variable taking values 0 or 1. Then
i
represents the percentage dierence in Y, given the
dierent values of X
i
.
Regression of Y against only dummy variables is equivalent to regression against a constant.
The estimated constant is the mean of Y , given which state the dummy variable is in.
If we allow for dierent slopes we can use an interaction term by mutiplying one of the dummy
variables with an explanator.
9.iii Interaction terms
Any two explanators may be multiplied together. We would do this if we believed that the eect
of X
1
on Y changed as X
2
changed, for example. Then we can test for whether this coecient is
zero - i.e. whether this hypothesized relationship does in fact exist.
Any explanator may be multiplied by itself to produce polynomial terms.
10 Joint hypothesis tests
Consider the equation y =
0
+
1
x
1
+
2
x
2
1
+. . . +. To test that x
1
is not an explanator of y we
cannot run t-tests individually on x
1
, x
2
1
, since this only tests for the eect of each factor alone.
5
To test the hypothesis that neither factor has explanatory power we need to remove both at
once.
10.i F-test
The idea here is that we run the regression and calculate the SSRs, rst with the variables included
(SSR
r
) and then without (SSR
u
). If the variables excluded did not explain much then the change
in SSR should be minimal.
The F - statistic is given by F =
(nk1)(SSR
r
SSR
u
)
rSSR
u
. Here, r is the number of restrictions
imposed on the model (the number of equals signs).
If s are distributed normally then this statistic is distributed according to the F-distribution
with (r, n k 1) df.
10.ii RESET
The regression specication error test is used to detect general functional form misspecication.
We add to the original MLS equation (y =
0
+
1
x
1
+. . . +
k
+u) two terms:
1
y
2
and
2
y
2
.
Then the new equation is y =
0
+
1
x
1
+. . . +
k
x
k
+
1
y
2
+
2
y
2
+u. If the original equation
is correctly specied then the corresponding F-test for H
0
:
1
=
2
= 0 should return a low F
statistic.
6