Autocorrelation in Time

Series
KNNL Chapter 12

Issues in Autocorrelated Data

When error terms are correlated (not
independent), problems occur when using
ordinary least squares (OLS) estimates
Regression Coefficients are Unbiased, but not
Minimum Variance
MSE underestimates 2
Standard errors of regression coefficients based
on OLS underestimate the true standard error
Inflated t and F statistics and artificially
narrow confidence intervals
Autocorrelated Errors (1st Order) t t 1 ut
where ut uncorrelated disturbances (typically assumed to be normal)

First-Order Model - I
First-Order Autoregressive Model ( AR(1)) :
Simple Regression: Yt 0 1 X t t t 1,..., n

t t 1 ut

autoregression parameter with 1

ut ~ N 0, 2 and independent
Generalizes to Multiple Regression:
Yt 0 1 X t1 ... p 1 X t , p 1 t t 1,..., n

t t 1 ut

Properties of Errors (assumption regarding 1 for model consistency):

2
1 ~ N 0,
2
1
2 1 u2 E 2 E 1 E u2 0
Covariance: 2 , 1 1 u2 , 1 2 2

Corrrelation: 2 , 1

2 , 1

2 1

2
2
2
2 1 u2

2
1

1 2

2
2
u2 , 1 2 0
1 2
2

2
1 2

1 2 1 2
2

First-Order Model - II
In General:

t t 1 ut t 2 ut 1 ut t 2 ut 1 ut ... s ut s
2

s 0

E t 0

Covariance: t , t s

s 0
s 2

1 2

Correlation: t , t s s

ut s
s 0

2s

ut s
2

s 0

2s

1 2

s0

s0

2
2

1
2
1
M
M
M
n 1 n 2 n 3

AR(2) : t 1 t 1 2 t 2 ut

L n 1

L n 2
L n 3

O M
L 1
Even Higher or models can be fit as well.

Test For Independence - Durbin-Watson

Test
Yt 0 1 X t1 ... p 1 X t , p 1 t

ut ~ NID 0, 2

t t 1 ut

H 0 : 0 Errors are uncorrelated over time

H A : 0 Positively correlated
1) Obtain Residuals from Regression
2) Compute Durbin-Watson Statistic (given below)
3) Obtain Critical Values from Table B.7, pp. 1330-1331 (R will provide a p-value)
If DW d L p 1, n Reject H 0
n

Test Statistic: DW

et et 1

t 2

e
t 1

E t 0 E t t 1

If DW dU p 1, n Conclude H 0

2
t

1 2

2
et et 1 e e 2 et et 1 2 e 2n
1 2
t 2
t 2
t 2
t 2
t 1
Under H 0 , expect DW 2
n

2
t

2
t 1

2
t

Otherwise Inconclusive

Autocorrelation - Remedial Measures

Determine whether a missing predictor variable
can explain the autocorrelation in the errors
Include a linear (trend) term if the residuals show
a consistent increasing or decreasing pattern
Include seasonal dummy variables if data are
quarterly or monthly and residuals show cyclic
behavior
Use transformed Variables that remove the
(estimated) autocorrelation parameter
(Cochrane-Orcutt and Hildreth-Lu Procedures)
Use First Differences
Estimated Generalized Least Squares

Transformed Variables
Suppose is known: Yt 0 1 X t t

t t 1 ut

Let Yt ' Yt Yt 1 0 1 X t t 0 1 X t 1 t 1

0 1 1 X t X t 1 t t 1 0 1 1 X t X t 1 ut

Yt ' 0' 1' X t' ut

(Standard Simple linear regression with independent errors)

where:
Yt ' Yt Yt 1

X t' X t X t 1

0' 0 1

1' 1

In Practice, we need to estimate with a sample based value r

Yt ' Yt rYt 1

X t' X t rX t 1

Fit: Y ' b0' b1' X ' and if errors are uncorrelated, back transform to:
b0'
Y b0 b1 X where: b0
1 r
^

s b0

s b0'
1 r

b1 b1'

s b1 s b1'

Cochrane-Orcutt Method
Start by estimating in Model: t = t-1 + ut by
regression through the origin for residuals (see below)
Fit transformed regression model (previous slide)
Check to see if new residuals are uncorrelated
(Durbin-Watson test), based on the transformed
model
If uncorrelated, stop and keep current model
If correlated, repeat process with new estimate r
based on current regression residuals from the
original (back transformed) model
n

e
t 2
n

t 1 t

e
t 2

2
t 1

Hildreth-Lu and First Difference

Methods
Hildreth-Lu Method
Find value of r (between 0 and 1) that minimizes the SSE
for the transformed model by grid search
Apply the transformed analysis based on the estimated r

First Differences Method

Uses = 1 in transformed model (Yt = Yt Yt-1 Xt = Xt
Xt-1 )
Set b0 = 0 and fits regression through origin of Y on X
When back-transforming:

b0 Y b X
'
1

b1 b

'
1

Forecasting with Autocorrelated

Errors
Makes use of any of the 3 estimation techniques (C-O, H-L, First Differences):
Yt 0 1 X t t

t t 1 ut
Yt 0 1 X t t 1 ut Yn 1 0 1 X n 1 n un 1
3 Elements:
^

1. Expected Value: 0 1 X n 1 Estimated as Y n 1 b0 b1 X n 1

2. Multiple of period n Error Term: n

Estimated as ren

3. Current disturbance un 1 ~ N 0, 2

Forecast for period n 1 (note the notation is "Forecast", not F -distribution :

^

Fn 1 Y n 1 ren
Standard Error of the Prediction (based on transformed model):

X X'
1
s pred MSE ' 1
n
n

X i' X '

i2
2

'
n 1

Approximate 95% PI: Fn 1 t 1 2 ; n 3 s pred

(First Differences has n - 2 df)