Multicollinearity and Selection of Explanatory Variables

Partial Correlation Coefficients of Partial Correlation

Multicollinearity
This problem occurs when the explanatory variables are very highly correlated with each other, Perfect multicollinearity Cannot estimate all the coefficients - e,g, suppose x3 = 2x2 and the model is yt = 1 + 2x2t + 3x3t + 4x4t + ut Problems if near Multicollinearity is present but ignored - R2 will be high but the individual coefficients will have high standard errors, - The regression becomes very sensitive to small changes in the specification, - Thus confidence intervals for the parameters will be very wide, and significance tests might therefore give inappropriate conclusions,

Measuring Multicollinearity
The easiest way to measure the extent of multicollinearity is simply to look at the matrix of correlations between the individual variables, e,g,
Corr x2 x3 x4 x2 0.2 0.8 x3 0.2 0.3 x4 0.8 0.3 -

But another problem: if 3 or more variables are linear - e,g, x2t + x3t = x4t Note that high correlation between y and one of the xs is not muticollinearity,

Solutions to the Problem of Multicollinearity

Traditional approaches, such as ridge regression or principal components, But these usually bring more problems than they solve, Some econometricians argue that if the model is otherwise OK, just ignore it The easiest ways to cure the problems are - drop one of the collinear variables - transform the highly correlated variables into a ratio - go out and collect more data e,g, - a longer run of data - switch to a higher frequency

Partial Correlation, in Econometric Models

a combination of explanatory variables, with a high value of Multiple Correlation Coefficient with the explained variable, y, and less correlated between them Partial Correlation Coefficients show the contribution of each explanatory variable to explaining the variation of y variable yt = a0 + a1x1t + a2x2t + t - simple correlation coefficients: ryx1, between y and x1,

ryx2, between y and x2,

rx1x2, between x1 and x2; - partial correlation coefficients : ryx1.x2 between y and x1, when the influence of x2 is retired (constant), ryx2.x1 between y and x2, when the influence of x1 is retired (constant), Partial correlation coefficients measure the intensity of the link between two variables when the influence of the third is maintained constant (is secluded),

Partial Correlation (1)

0 a 1 x 1t a 2 x 2 t et yt a
y x2 a) Variables x1, x2 are independent x1 x2 y x1 x2 b) Variables x1, x2 are correlated (collinear) x1 y

In case a) simple coefficients can be calculated: ryx1 between y and x1

ad ryx2 between y and x2, The determination coefficient is:

R2yx1x2 = r2yx1+ r2yx2, And the multiple correlation coefficient is:

R yx1x 2 r

2 yx1

2 yx2

Partial Correlation (2)

In case b) the explanatory variables are interdependent and the determination coefficient is the sum of intersection surfaces between:

y and x1 separately without the common influence of x1 and x2 over y,

y and x2 separately without the common influence of x1 and x2 over y, y, x1 and x2, simultaneously influence of both explanatory variables over y, To obtain the determination coefficient , respectively the multiple correlation coefficient, the inter influence between the explanatory variables must be eliminated from the sum of their determination coefficient,

Partial Correlation (3)

Generalizing the concept of partial correlation, for the model with k explanatory variables, it measures the intensity of correlation between two variables, when the influence of one or more variables is kept constant, The number of the secluded variables establishes the order of the partial correlation coefficient, The simple correlation coefficients are called of order 0, In a model with k explanatory variables, the maximum order to calculate partial correlation coefficient is k1, because there cannot be secluded the influence of all explanatory variables,

Partial Correlation (4)

Let y, the explained variable and x1, x2, x3, the explanatory variables (k = 3), The determination coefficients for the regressions between y and each explanatory variable separately, are R2yx1, R2yx2, R2yx3, These are the squared values of the simple correlation coefficients, because there are linear links, r2yx1, r2yx2, r2yx3 they show the proportion of each x variables contribution to explain the y variation,

There are 6 partial correlation coefficients of order 1:

ryx1,x2; ryx1,x3; ryx2,x1; ryx2,x3; ryx3,x1; ryx3,x2; 3 partial correlation coefficients of order 2: ryx1,x2x3; ryx2,x1x3; ryx3,x1x2; The last order to calculate partial correlation coefficients is 2, when k=3,

How to Calculate the Partial Correlation Coefficients (1)

Two ways:
a) Starting from the residuals e1 and e2 resulted after two regressions: Regression between y and the secluded variables, Regression between the explanatory variable xi whose influence is analized, and the secluded variables, following the steps: - e,g, for r2yx3,x1x2:

regression y = f(x1, x2) , yt = a0 + a1x1t + a2x2t + 1t, t=1, n

estimation of residuals
t y t (a 0 a 1 x1t a 2 x 2t ) e1t y t y

regression x3= f(x1, x2) , x3t = b0 + b1x1t + b2x2t + 2t , t=1, n;

b x b x ) Estimation of residuals e2t x3t x 3t x3t (b 0 1 1t 2 2t

The simple correlation coefficient between e1 i e2: ryx3,x1x2=re1e2

How to Calculate the Partial Correlation Coefficients (2)

b) Starting from Student test - Only for the partial correlation coefficients of order k-1,
2 t i r2yxi.(the rest of k-1 variables) = 2 t i (n k 1)

, where

ti is the empirical Student ratio for xi variable whose influence over y, is analyzed

Exercise the partial correlation coefficients

y x1 83,7 88,8 100,7 105,5 118,5 131,4 148,5 162 174,5 185,3 x2 18 21,5 25,6 29,5 34,6 40,5 44,4 49,8 51,5 53,8 x3 92,5 93,6 96,5 94 100,2 101,5 105,4 112,8 112,6 112,7 x4 92,5 95,6 97,5 97,4 100,2 101,4 104,6 109,8 111,5 112,2

9,5 10,7 11,5 12,5 13,3 15,3 16,8 18,8 19,5 21,5

Calculate the partial correlation coefficients, one for each order : ryx3.x1, ryx3.x1x2, ryx3.x1x2x4,
r
2 yx3 . x1 x2 x4

ti2 2 ti (n k 1)