Frequency Domain Analysis

Fourier analysis: investigate periodic and cyclic phenomena

economics: (daily, weekly, monthly, yearly) seasonality
biology: diurnal, solar, lunar eects
environmentrics and ecology: climate,
engineering: rotating machinery
astronomy: rotation of stars, life cycle of planets
Idea: think of x
t
as a function x(t) and express the function in terms of the sinu-
soids with dierent frequencies/periods (deterministic function versus random
function)
Interpretation: describe the periodic phenomenon of the series according to the
magnitude of the coecients associated with the sinusoids
1
Time Series Analysis: Frequency Domain Approach
2010/5/17
Fourier Analysis for Deterministic Functions
Assume f(t) is a periodic function with period 2 and
_

f
2
(t)dt < .
f(t) =
1
2
a
0
+

j=1
(a
j
cos(jt) + b
j
sin(jt)) ,
where
a
j
=
1

f(t) cos(jt)dt, b
j
=
1

f(t) sin(jt)dt.
let f
m
(t) =
1
2
a
0
+

m
j=1
(a
j
cos(jt) + b
j
sin(jt)), then
_

(f
m
(t) f(t))
2
dt 0,
as m .
f L
2
(, ), where L
2
(a, b) forms a (innite dimensional) Hilbert space with
the inner product < f, g >=
_
b
a
f(t)g(t)dt
{cos(jt), sin(jt) : j = 0, 1, . . .} is an orthogonal basis for L
2
(, ), i.e.,
_

cos(jt) cos(kt)dt =
_

sin(jt) sin(kt)dt =
j,k
,
_

cos(jt) sin(kt)dt = 0.
2
Time Series Analysis: Frequency Domain Approach
2010/5/17
Discrete Fourier Transform
Assume (X(0), X(1), X(n 1))

IR
n
C
n
and dene
X

(t) = X(t mod n), Y (t) = X

_
tn
2
_
.
X

(t) is a periodic function with period n embedded from X(t)

Y (t) is a periodic function with period 2 and therefore Y (t) can be written as
Y (t) =
1
2
a
0
+

j=1
(a
j
cos(jt) + b
j
sin(jt))
since X(t) = X

(t) = Y (2t/n), we have

X(t) =
1
2
a
0
+

j=1
(a
j
cos(
j
t) + b
j
sin(
j
t)) ,
j
= 2j/n.
for a stochastic process X
t
(), a
j
() and b
j
() are random variables
3
Time Series Analysis: Frequency Domain Approach
2010/5/17
Orthonormal Basis for C
N
Dene
e

j
= n
1/2
_
e
i0
j
, e
i1
j
, , e
i(n1)
j
_
,
j
=
2j
n
(, ],
j F
n

_

_
n 1
2
_
, . . . ,
_
n 1
2
__
.
{e
j
: j F
n
} is an orthonormal basis for C
n
with < x, y >=

x
i
y
i
, i.e.,
< e
i
, e
j
>=
ij
.
X() = (X
0
(), . . . , X
n1
())

can be represented as
X() =

jF
n
d
j
()e
j
, d
j
=< X, e
j
> .
{d
j
} is called discrete Fourier transform (DFT) of X
I
j
= I(
j
) 2|d
j
|
2
=
2
n

n1
t=0
X
t
e
i
j
t

2
is called periodogram ordinate
4
Time Series Analysis: Frequency Domain Approach
2010/5/17
ANOVA in Frequency Domain
Total variation of X can be decomposed into nearly uncorrected components
according to dierent frequencies:
||x||
2
= < x, x >=
_

d
j
e
j
,

d
k
e
k
_
=

j
d
j

d
j
< e
j
, e
j
>=

jF
n
|d
j
|
2
=
I(
0
)
2
+
1
2

jF
n
\{0}
I(
j
)
5
Time Series Analysis: Frequency Domain Approach
2010/5/17
Periodogram Ordinate
I(
j
) =
2
n

n1

t=0
X
t
e
i
j
t

2
= 2

|h|<n
(h) e
ih
j
.
I(
j
) = I(
j
) = I(
j
)
I(
0
) = 2n
1
|

t
X
t
|
2
= 2n

X
2
n
corresponds to the mean behavior of {X
t
}
I(
j
), j = 0 corresponds to the period behavior with period T = 2/
j
, which
is invariant to the mean of {X
t
} (verify!)
I(
j
) can be explained as the contribution of the frequency
j
. The periodic
eect with frequency
j
is strong if I(
j
) is large.
{I(
j
)} is the DFT of (h) (upto a scalar)
What is the DFT of (h)? spectral density! (a population version of I(
j
))
6
Time Series Analysis: Frequency Domain Approach
2010/5/17
Spectral Density
Assume {X
t
} is a stationary process with ACF (h). Then the spectral density of
{X
t
} is dened as
f() =
1
2

h=
(h)e
ih
f() is a periodic function with period 2
f() is the Fourier transform of (h) and (h) =
_

f()e
ih
d is the inverse
Fourier transform of f() (positive denite property)
f() can be explained as the relative contribution to the variability of process
from the sinusoid with frequency
I(
j
) can be an estimator for 4f(
j
)
the long-term variance satises nvar(

X
n
)

|h|<
(h) = 2f(0)
7
Time Series Analysis: Frequency Domain Approach
2010/5/17
Spectral Density for Linear Processes
Assume {X
t
} satises X
t
= (B)Z
t
, {Z
t
} are WN(0,
2
). Then, the spectral
density of {X
t
} is
f() =

2
2
(e
i
) (e
i
) =

2
2
|(e
i
)|
2
proof: verify!
WN: f() =
2
/(2) is a constant function
ARMA: f() =

2
2
|(e
i
)|
2
|(e
i
)|
2
AR(1): f() =

2
2
(1 +
2
1
2
1
cos )
1
for a linear lter Y
t
= (B)X
t
, f
Y
() = |(e
i
)|
2
f
X
()
{
j
} is called time-invariant linear lter
(e
i
) is called transfer function of the lter
|(e
i
)|
2
is called power transfer function (PTF)
8
Time Series Analysis: Frequency Domain Approach
2010/5/17
Linear Filter
How to design a lter to block, reduce or enhance certain signals according to
their frequencies?
How the PTF look like for smoothing and dierencing?
dierencing reduce the power around zero frequency: eliminates trend (smooth)
structures
smoothing reduce the power of high frequencies: eliminates noisy structures
Design a symmetric moving average lter which eliminates seasonal components
with period 3 and at the same time passes quadratic trend functions without
distortion. (homework! TSTM Problem 1.5)
9
Time Series Analysis: Frequency Domain Approach
2010/5/17
Sampling Properties of Periodogram Ordinates
Assume {X
t
} is WS with mean and

h
|(h)| < (holds for ARMA) and
re-dene I
n
() as
I
n
(0) = n

X
2
n
, I
n
(
j
) =

|h|<n
(h)e
ih
j
=

|h|<n
(h) cos(h
j
), j = 0.
E[I
n
(0) n
2
] = nE[

X
2
n

2
] = nvar(

X
n
)

(h) = 2f(0)
For j = 0, EI
n
(
j
) 2f(
j
).
Is I
n
(
j
)/(2) a good estimator of f()? (what is required to be a good esti-
mator?)
10
Time Series Analysis: Frequency Domain Approach
2010/5/17
Sampling Properties of Periodogram Ordinates (cont.)
I
n
(
j
)/(2) Exp(f(
j
)) as n (asymptotic unbiased but not consis-
tent!)
(I
n
(
1
), . . . , I
n
(
k
))/(2) converges to a vector of indep. exponential random
variables in which the ith component has mean f(
i
)
How to get a consistent estimator for f()? smoothing!
Idea: If f() is continuous, then the average of I
n
(
j
) in a small neighborhood
of converges to the mean (central limit theorem).
11
Time Series Analysis: Frequency Domain Approach
2010/5/17
Smoothed Periodogram Ordinates
Dene

f(
j
) =

|k|M
n
W
n
(k)
I
n
(
j+k
)
2
,
where the weight function W
n
(k) and M
n
satisfy M
n
, M
n
/n 0 and
W
n
(k) = W
n
(k) 0,

|k|M
n
W
n
(k) = 1,

|k|M
n
W
2
n
(k) 0.
E

f() f() (asymptotic unbiased)
lim
n
_

|k|M
n
W
2
n
(k)
_
1
cov
_

f(),

f()
_
=
_
_
_
2f
2
(), = = 0, ,
f
2
(), = = 0, ,
0, = .

f() f() (consistency)
12
Time Series Analysis: Frequency Domain Approach
2010/5/17
Non-parametric Inference for Spectral Density
condence interval: approximate the distribution of

f() by
2
distribution
(Tukey, 1949)
let

f(
j
) c
2

match rst two moments to solve c and : = 2(

k
W
2
n
(k))
1
1 P
_

(/2)


f(
j
)
f(
j
)

2

(1 /2)
_
testing for the presence of hidden periodicity:
Kolmogorov-Smirno
Fishers test
13
Time Series Analysis: Frequency Domain Approach
2010/5/17
Models Dened in Frequency Domain
Bloomeld model (1973):
f() = exp
_
_
_
K

j=0

j
cos(j)
_
_
_
a positive and periodic function with period 2
no constraint on {
j
}
Fourier representation based on cosine functions
ARMA can be represented as Bloomeld models with innite order
14
Time Series Analysis: Frequency Domain Approach
2010/5/17
Parametric Inference in Frequency Domain
Whittle likelihood:
L

() =
[n/2]

j=1
log f

(
j
)
[n/2]

j=1
I
n
()
f

(
j
)
,
where I
n
(
j
) =
1
2n

n
t=1
X
t
e
i
j
t

2
denotes the normalized periodogram at the
Fourier frequencies
j
= 2j/n for j = 1, 2, . . . , [n/2].
Whittle likelihood in the frequency domain is equivalent to the likelihood in the
time domain
Whittle estimator is equivalent to MLE
implementation (computation) is easy: no matrix inversion
local Whittle estimator for semi-parametric inference
15
Time Series Analysis: Frequency Domain Approach
2010/5/17