Generalized Least Squares Heteroskedasticity

Generalized Least Squares and Heteroskedasticity

Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009

February 4, 2009

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

The Classical Linear Model:

1 2 3 4

Linearity: Y = X + u. Strict exogeneity: E (u|X ) = 0 No Multicollinearity: (X ) = K , w.p.1. No heteroskedasticity/ serial correlation: V (u|X ) = 2 In .

= (X X )1 X Y is best linear unbiased. Gauss/Markov: |X ) = 2 (X X )1 Variance: S 2 (X X )1 is unbiased for V ( Valid Inference: with the normality assumption, we use t and F tests. Now we will focus on the consequences of relaxing V (u|X ) = 2 In .

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

The Generalized Linear Model

Suppose all classical assumptions hold, but now V (u|X ) = 2 where is any symmetric, positive denite n n matrix We are now allowing for heteroskedasticity (elements of the diagonal of are not restricted to be all equal) and/or serial correlation (o-diagonal elements may now be = 0.), but we are not imposing any structure to yet (besides symmetry and pd). Plan
1

Explore consecuences on previous results of relaxing V (u|X ) = 2 In . Find optimal estimators and valid inference strategies for this case (GLS).

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Consequences of relaxing V (u|X ) = 2 In

is still linear and unbiased (why?) but the Gauss Markov Theorem does not hold anymore. We will show constructively is now innecient by nding the BLUE for the that generalized linear model. |X ) will now be 2 (X X )1 (X X )1 (check it). V ( |X ) is no longer 2 (X X )1 , and S 2 (X X )1 will be a V ( ). biased estimator for V ( t tests no longer have the t distribution, F tests no longer valid too. So, ignoring heteroskedasticity or serial correlation, that is, the use and V |X ) = S 2 (X X )1 , keeps estimation of unbiased ( of though inecient, and invalidates all standard inference.
Walter Sosa-Escudero Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Generalized Least Squares

First we need a simple result: if is n n symmetric and pd, there is an n n nonsingular matrix C such that 1 = C C
What does this mean, intuitively?

Consider now the following tranformed model Y = X + u where Y = CY , X = CX and u = Cu.

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Now check:
1 2 3 4

Y = X + u , so the transformed model is trivially linear. E (u |X ) = CE (u |X ) = 0 (X ) = (CX ) = K , w.p.1. V (u |X ) =

(CX is a rank preserving tranformation of X !).

V (Cu|X ) = E (CuuC |X ) = CE (uu |X )C = C 2 C = 2 C [1 ]1 C = 2 C (C C )1 = In So...

2 1

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Since the transformed model satises all the classical assumption, the Gauss-Markov Theorem hold for the transformed model, hence the best linear unbiased estimator is: gls = (X X )1 X Y This the generalized least squares estimator Careful: the GLS estimator is an OLS estimator of a transformed isomorphic model (the generalized linear model). It provides the BLUE under heteroskedasticity/ serial correlation. is inecient in the generalized context Now it is clear that (why?) It is important to see that the statistical properties depend on the underlying structure (they are not properties of an estimator per-se).
Walter Sosa-Escudero Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Note that gls = (X X )1 X Y = (X C CX )1 X C CY = (X 1 X )X 1 Y gls = . When = In , gls requires that we know The practical implementation of 2 (though not .) gls ) = 2 (X X )1 . It is easy to check that V (

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Feasible GLS

. Then, replacing Suppose there is an estimate for , label it : by f gls = (X 1 X )X 1 Y This is the feasible GLS estimator. Is it linear and unbiased? Ecient?

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Heteroskedasticity
Suppose we keep the no-serial correlation assumption, but allow for heteroskedasticity, that is:
2 2 2 V (u|X ) = 2 diag(1 , 2 , . . . , n )

It is easy to see that in this case

1 1 1 ) C = diag(1 , 2 , . . . , n

Hence X = CX
1 is a matrix with typical element Xik /i , i = 1, . . . , n, k = 1, . . . , K . Y = CY is constructed in the same fashion. gls for this particular case is called the weighted least squares

estimator.
Walter Sosa-Escudero Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

In some cases the FGLS takes a very simple form. Consider Y = X + u with 2 V (ui |X ) = 2 Xij Consider the transformed model Y = X + u where X is a matrix with typical element
Xik =

Xik Xij

u and Y are dened in a similar fashion. Note u i = ui Xij

2 so V (u i |X ) = . Hence the transformed model is homoskedastic.

Walter Sosa-Escudero Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

In this case the FGLS is a GLS estimator since there is no need to estimate unknown parameters. In any case, the implementation of a FGLS or GLS requires detailed knowledge of the structure of heteroskedasticity in order to propose a homoskedastic transformed model. This is very rarely available. Later on we will explore another approach: keep the OLS estimator (we will loose eciency) but replace S 2 (X X )1 by some other estimator that behaves correctly even under heteroskedasticity. (we need an asymptotic framework for this).

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Testing for heteroscedasticity

We will explore two strategies, with no derivations and emphasizing intuitions. Later on we will prove all results (we need an asymptotic framework for this). a) White test H0 : no heteroscedasticity, HA : there is heterocedasticity of some form. Consider a simple case with K = 3: Yi = 1 + 2 X2i + 3 X3i + ui 1, . . . , n

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Steps to implement the test:

1 2

Estimate by OLS, save squared residuals in e2 . Regress e2 on all variables, their squares and all possible non-redundant cross-products. In our case, regress e2 on 2 , X 2 , X X , and obtain R2 in this auxiliar 1, X2 , X3 , X2 2 3 3 regression. Under H0 , nR2 2 (p). p = number of explanatory variables in the auxiliar regression minus one.

Reject H0 if nR2 is too large.

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Intuition: Note that under homoskedasticity

2 E (u2 i |X ) =

The auxiliar model can be seen as trying to model the variance of the error term. If the R2 of this auxiliar regression were high, then we could explain the behavior of the squared residuals, providing evidence that they are not constant. Comments:
Valid for large samples. Informative if we do not reject the null (no heterocedasticity). When it rejects the null: there is heterocedasticity. But we do not have any information regarding what causes heterocedasticity. This will cause some trouble when trying to construct a GLS estimator, for which we need to know in a very specic way what causes heterocedasticity.
Walter Sosa-Escudero Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

b) The Breusch-Pagan/Godfrey/Koenker test Mechanically very similar to Whites test. Checks if certain variables cause heterocedasticity. Consider the linear model where all classical assumptions hold, but: V (ui |X ) = h(1 + 2 Z2i + 3 Z3i + . . . + p Zpi ) where h( ) is any positive function with two derivatives. When 2 = . . . = p = 0, V (ui |X ) = h(1 ), a constant!! Then, homoscedasticity means H0 : 2 = 3 = . . . = p = 0, and HA : 2 = 0 3 = 0 . . . p = 0.

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Steps to implement the test:

1 2

Estimate by OLS, and save squared residuals e2 i. Regresss e2 i on the Zik variables, k = 2, . . . , p and get (ESS). The test statistic is: 1 ESS 2 (p 1) 2 (p) 2

under H0 , asymptotically. We reject if it is too large.

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity

Generalized Least Squares Heteroskedasticity

Comments: Intuition is as in the White test (a model for the variance). By focusing on a particular group, if we reject the null we have a better idea of what causes heterocedasticity. Accepting the null does not mean there isnt heterocedasticity (why?). Also a large sample test.
2 as a test, which is Koenker (1980) has proposed to use nRA still valid if errors are non-normal.

Walter Sosa-Escudero

Generalized Least Squares and Heteroskedasticity