Documente Academic
Documente Profesional
Documente Cultură
Homework 7 Solution
Written by: Fares Qeadan, Revised by: Anna Panorska
Problem 5.5.2.
Let Y1 , Y2 , ..., Yn be a random sample from fY (y; θ) = θ1 e−y/θ , y > 0. We will compare the Cramer-Rao lower bound for
− −
fY (y; θ) to the variance of the maximum likelihood estimator for θ, θ̂ = n1 ni=1 Yi = Y and examine whether Y is a
P
1 −y/θ 1 y
ln fY (y; θ) = ln e = ln( ) + ln e−y/θ = − ln θ − .
θ θ θ
Thus
∂ ln f(y; θ) 1 y
=− + 2
∂θ θ θ
and
∂ 2 ln f(y; θ) 1 2y
2
= 2 − 3.
∂θ θ θ
Therefore (assuming you know how to obtain E(Y ) = θ)
1 2Y 1 2 1 2θ 1
I(θ) = −E − 3 =− − E(Y ) = − 2 + 3 = 2.
θ2 θ θ2 θ3 θ θ θ
Thus,
1 1 θ2
CRLB = = 2 = .
nI(θ) n θ2 n
Now, (assuming you know how to obtain V ar(Y ) = θ2 )
− V ar(Y ) θ2
V ar(Y ) = = = CRLB.
n n
Hence, θ̂ is a best estimator for θ.
1
2
Problem 5.5.4.
Suppose a random sample of size n is taken from a normal distribution with mean µ and variance σ 2 , where σ 2 is
− Pn
known. We will compare the Cramer-Rao lower bound with the variance of µ̂ = Y = n1 i=1 Yi and examine whether
−
Y is an efficient estimator for µ.
√ (y − µ)2
1 2 2
ln fY (y; µ) = ln √ e−(y−µ) /2σ = − ln σ 2π − .
σ 2π 2σ 2
Thus
∂ ln f(y; µ) (y − µ)2
=
∂µ σ2
and
∂ 2 ln f(y; µ) 1
2
= − 2.
∂µ σ
Therefore
1 1
I(µ) = −E − 2 = 2
σ σ
and,
1 1 σ2
CRLB = = 1 = .
nI(µ) n σ2 n
Now,
− 1 σ2
V ar(µ̂) = V ar(Y ) = V ar(Y ) = = CRLB.
n n
Hence, µ̂ is an efficient estimator for µ.
Problem 5.6.7.
Let Y1 , Y2 , ..., Yn be a random sample of size n from the pdf fY (y; θ) = θyθ−1 , 0 ≤ y ≤ 1
Note that !θ−1
n n
× 1 = u(T, λ)v(−
→
Y Y
L(θ) = θyiθ−1 =θ n
yi y)
i=1 i=1
and v(−→
Qn n θ−1
Qn
where T = i=1 yi , u(T, θ) = θ (T ) y ) = 1. That is, T = i=1 Yi is a sufficient statistic for θ.
Qn
Now, to examine whether the MLE θ̂ of θ is a function of T = i=1 Yi , it’s more convenient to work with the log
likelihood function
3
n
X
ln L(θ) = n ln θ + (θ − 1) ln yi .
i=1
Further
n
d ln L n X
= + ln yi .
dθ θ
i=1
d ln L
Setting dθ = 0 gives
n n n
θ̂ = − Pn =− Qn =− .
i=1 ln Yi ln i=1 Yi ln T
Qn
That is the MLE of θ, θ̂ is a function of T = i=1 Yi .
Problem 5.6.9.
We will write the pdf fY (y; λ) = λe−λy , y > 0 in its exponential form and try to deduce a sufficient statistic for λ
assuming that the data consist of a random sample of size n.
= λ × 1 × e(−λ)(y)
= a(λ)b(y)ec(λ)d(y)
n
Y
L(λ) = λe−λyi
i=1
Pn
= λn e−λ i=1 yi
×1
= u(T, λ)v(−
→
y)
Problem 5.6.10.
θ
fY (y; θ) = , 0 < y < ∞, 0 < θ < ∞.
(1 + y)θ+1
We will write the pdf in its exponential form and try to deduce a sufficient statistic for θ
4
θ
f(Y |θ) = = θ(1 + y)−(θ+1)
(1 + y)θ+1
θ × 1 × eln(1+y)
−(θ+1)
=
= θ × 1 × e−(θ+1) ln(1+y)
= a(θ)b(y)ec(θ)d(y) .
n n
Y θ Y
L(θ) = θ+1
= θe−(θ+1) ln(1+yi )
i=1
(1 + yi ) i=1
Pn
= θn e−(θ+1) i=1 ln(1+yi )
×1
= u(T, λ)v(−
→
y ),
where T =
Pn n −(θ+1) n
P
−
→
i=1 ln(1+yi ) and v( y ) = 1.
i=1 ln(1 + yi ), u(T, λ) = θ e Thus, by the Factorization Theorem
Pn
T = i=1 ln(1 + Yi ) is a sufficient statistic for θ.
Problem 5.5.6.
1− 1 −
E(θ̂) = E( Y ) = E(Y )
r r
1
= EY = θ.
r
−
Thus, θ̂ = 1r Y is an unbiased estimator for θ.
−
(b) We will show that θ̂ = 1r Y is a minimum variance estimator for θ (assuming you can obtain V ar(Y ) = rθ2 ).
First, we will find the Fisher Information
∂ 2 ln f(y; θ)
I(θ) = −E .
∂θ2
Note that
1 y
ln fY (y; θ) = ln yr−1 e−y/θ = − ln((r − 1)!) − r ln θ + (r − 1) ln y − .
(r − 1)!θr θ
Thus
∂ ln f(y; θ) r y
=− + 2
∂θ θ θ
5
and
∂ 2 ln f(y; θ) r 2y
2
= 2 − 3.
∂θ θ θ
Therefore (assuming you know how to obtain E(Y ) = θ)
r 2Y r 2 r 2rθ r
I(θ) = −E − 3 =− − E(Y ) = − 2 + 3 = 2.
θ2 θ θ2 θ3 θ θ θ
Thus,
1 1 θ2
CRLB = = r = .
nI(θ) n θ2 nr
Now, (assuming you know how to obtain V ar(Y ) = rθ2 )
1− 1 − 1 θ2
V ar(θ̂) = V ar( Y ) = 2 V ar(Y ) = 2 V ar(Y )/n = = CRLB.
r r r rn
So, since CRLB = V ar(θ̂) then indeed θ̂ is a minimum variance estimator for θ.