OLS Assumptions

1. Linear Regression Model:

The regression model is linear in the parameters, though it may or may not be linear in the
variables. That is the regression model as shown in

Yi = β1 + β2 X1+β3 X3 +……+βk Xk+Ui

2. Fixed X Values or X Values Independent of the Error Term:

Values taken by the regressor X may be considered fixed in repeated samples (the case of
fixed regressor) or they may be sampled along with the dependent variable Y (the case of stochastic
regressor). In the latter case, it is assumed that the X variable(s) and the error term are independent,
that is, cov (Xi, ui) =0.

3. Zero Mean Value of Disturbance Ui:

Given the value of Xi, the mean, or expected, value of the random disturbance term ui is
zero.

E(ui|Xi) =0 Or, if X is non-stochastic, E(ui) =0

4. Homoscedasticity or Constant Variance of Ui:

The variance of the error, or disturbance, term is the same regardless of the value of X.

Symbolically,

var(ui) =E[ui −E(ui|Xi)]2 =E(u2 i|Xi)

5. No Autocorrelation between the Disturbances:

Given any two X values, Xi and Xj(I,j), the correlation between any two ui and uj(I, j) is
zero. In short, the observations are sampled independently. Symbolically, cov(ui, uj|Xi, Xj) =0,

6. The Number of Observations n Must Be Greater than the Number of Parameters to Be

Estimated:

Alternatively, the number of observations must be greater than the number of explanatory
variables.

1
7. The Nature of X Variables:

The X values in a given sample must not all be the same. Technically, var (X) must be a
positive number. Furthermore, there can be no outliers in the values of the X variable, that is,
values that are very large in relation to the rest of the observations.

OLS properties

Small-Sample Properties

i. Unbiasedness

An estimator ê is said to be an unbiased estimator of e if the expected value of ê is equal to

the true e; that is

E (ê) = e

E (ê) - e = 0

If this equality does not hold, then the estimator is said to be biased, and the bias is

Calculated as

bias (ê) = E(ê) - e

The average value of these estimates is expected to be equal to the true value if the
estimator is unbiased.

ii. Minimum Variance

ê 1 is said to be a minimum-variance estimator of e if the variance of ê1 is smaller than or

at most equal to the variance of ê2, which is any other estimator of e.

iii. Best Unbiased, or Efficient, Estimator

If ê1 and ê2 are two unbiased estimators of e, and the variance of ê1 is smaller than or at
most equal to the variance of ê2, then ê1 is a minimum-variance unbiased, or best Unbiased, or
efficient, estimator.

iv. Linearity

2
 An estimator ê is said to be a linear estimator of e if it is a linear function of the sample
observations. Thus, the sample mean defined as

X = 1/n Ʃ Xi = 1/n (x1 + x2 + ···+ xn)

is a linear estimator because it is a linear function of the X values.

v. Best Linear Unbiased Estimator (BLUE)

If ê is linear, is unbiased, and has minimum variance in the class of all linear unbiased
estimators of e, then it is called a best linear unbiased estimator, or BLUE for short.

vi. Minimum Mean-Square-Error (MSE) Estimator

The MSE of an estimator ê is defined as

MSE (ê) = E (ê− e)²

This is in contrast with the variance of ê, which is defined as

var (ê) = E[ê− E(ê)]²

The difference between the two is that var (ê) measures the dispersion of the distribution of ê
around its mean or expected value, whereas MSE(ê) measures dispersion around the true value of
the parameter.

Large-Sample Properties

It often happens that an estimator does not satisfy one or more of the desirable statistical
properties in small samples. But as the sample size increases indefinitely, the estimator possesses
several desirable statistical properties. These properties are known as the large sample, or
asymptotic, properties.

i. Asymptotic Unbiasedness

An estimator ê is said to be an asymptotically unbiased estimator of e if

lim n→∞E(ê n) = e

3
where ên means that the estimator is based on a sample size of n and where lim means limit and n
→ ∞ means that n increases indefinitely. In words, ê is an asymptotically unbiased estimator of e
if its expected, or mean, value approaches the true value as the sample size gets larger and larger.

ii. Consistency

ê is said to be a consistent estimator if it approaches the true value e as the sample size gets

larger and larger.

iii. Asymptotic Efficiency:

Let estimated θ be an estimator of θ. The variance of the asymptotic distribution of

estimated θ is called the asymptotic variance of estimated θ. If estimated θ is consistent and its
asymptotic variance is smaller than the asymptotic variance of all other consistent estimators of θ,
estimated θ is called asymptotically efficient.

iv. Asymptotic Normality

An estimator e is said to be asymptotically normally distributed if its sampling distribution

tends to approach the normal distribution as the sample size n increases indefinitely. For example,
statistical theory shows that if X1, X2, ... , Xn are independent normally distributed variables with
the same mean µ and the same variance σ2, the sample mean ƩX is also normally distributed with
mean µ and variance σ2/n in small as well as large samples.

Problems:

1. Multicollinearity:

Multicollinearity occurs when independent variables in a regression model are

correlated. If the degree of correlation between variables is high enough, it can
cause problems when you fit the model and interpret the results. For the k-variable regression
involving explanatory variables X1, X2, ..., Xk ,an exact linear relationship is said to exist if the
following condition is satisfied:
λ 1X1 +λ2 X2 +···+λk Xk =0

4
  Test for Multicollinearity:

For measure of Multicollinearity if there is high R square value and t is statistical

insignificant than it is confirmed that there is multicollinearity. One way to measure
multicollinearity is the variance inflation factor (VIF), which assesses how much the variance of
an estimated regression coefficient increases if your predictors are correlated. If the VIF value
greater than 10 it is confirmed that there is perfect multicollinearity.

2. Heteroscedasticity:

The concept of heteroscedasticity the opposite being homoscedasticity, is used in

statistics, especially in the context of linear regression or for time series analysis, to describe the
case where the variance of errors or the model is not the same for all observations, while often one
of the basic assumption in modeling is that the variances are homogeneous and that the errors of
the model are identically distributed.
OLS is still unbiased and consistent, even if we do not assume homoskedasticity. The
standard errors of the estimates are biased if we have heteroskedasticity.

Testing for Heteroscedasticity:

If it is suspected that the variances are not homogeneous (a representation of the residuals
against the explanatory variables may reveal heteroscedasticity), it is therefore necessary to
perform a test for heteroscedasticity. Several tests have been developed, with the following null
and alternative hypotheses:

 H0: The residuals are homoscedastic

 H1: The residuals are heteroscedastic

Tests:

 Breusch-Pagan test
 White test

5
  Breusch-Pagan test:

The Breusch-Pagan test will detect any linear forms of heteroskedasticity.

1. First estimate the model by OLS.

2. Find residuals from the OLS regression.
3. Regress residuals on independent variables.
4. After regressing the residuals squared on all of the x’s, use the R2 to form an F or LM
test. The F statistic is just the reported F statistic for overall significance of the
regression,
5. F = [R2 /k]/[(1 –R2 )/(n – k – 1)], which is distributed F .
k, n – k – 1

6. The LM statistic is LM = nR2, which is distributed ck2. If the p-value is sufficiently

small, that is, below the chosen significance level, then we reject the null hypothesis of
homoskedasticity.

3. White Test:

White test allows for nonlinearities by using squares and crossproducts of all the x’s.

1. Estimate your model using OLS:

2.
3. Obtain the predicted Y values after estimating your model.

4. Estimate the model using OLS:

5.
6. Retain the R-squared value from this regression:

7.
8. Calculate the F-statistic or the chi-squared statistic:

6
3. Autocorrelation:
The term autocorrelation may be defined as “correlation between members of series of
observations ordered in time [as in time series data] or space [as in cross-sectional data].In the
regression context, the classical linear regression model assumes that such autocorrelation does
not exist in the disturbances ui. Symbolically,

cov(ui, uj|xi, xj) = E(ui uj) =0

If this assumption is violating than there is autocorrelation in the model.

Test For Autocorrelation:

Durbin Watson Test:

The most celebrated test for detecting serial correlation is that developed by statisticians
Durbin and Watson. It is popularly known as the Durbin–Watson d statistic. assumptions
underlying the d statistic.

1. The regression model includes the intercept term. If it is not present, as in the case of the
regression through the origin, it is essential to rerun the regression including the intercept term to
obtain the RSS

2. The explanatory variables, the X’s, are non-stochastic, or fixed in repeated sampling.

3. The disturbances ut are generated by the first-order autoregressive scheme: ut= ρut-1 +εt.
Therefore, it cannot be used to detect higher-order autoregressive schemes.

4. The error term ut is assumed to be normally distributed.

5. The regression model does not include the lagged value(s) of the dependent variable as one of
the explanatory variables.

7
Method

1. Run the OLS regression and obtain the residuals.

2. Compute d

3. For the given sample size and given number of explanatory variables, find out the critical dL
and dU values.

Durbin Watson value lies between 0 to 4 if it is near to 2 or 2 there is no auto correlation if it is

less than 2 than there is positive auto correlation and if greater than 2 or near to 4 than there is
negative autocorrelation which we see from the table.

Decision Rule:

Given the level of significance α,

1. H0:ρ =0 versus H1:ρ>0. Reject H0 at α level if d < dU. That is, there is statistically significant
positive autocorrelation.

2. H0:ρ =0 versus H1:ρ<0. Reject H0 at α level if the estimated (4−d) < dU, that is, there is
statistically significant evidence of negative autocorrelation.

3. H0:ρ =0 versus H1:ρ =0. Reject H0 at 2α level if d < dU or (4−d) < dU, that is, there is statistically
significant evidence of autocorrelation, positive or negative.