Prof. Dr.

Enno Mammen
Statistic II

Winterterm 2015/2016

5. Problem Sheet
Exercise 15 (4 Marks).

Consider for g(x) = 12 exp(|x|) the location family g(x ). Show that
tiable in quadratic mean.
Hint: Apply
Z

s+h (x) s (x) =

0

p
g(x ) is dieren-

p
1
h sign(x uh) g(x uh)du
2

and proceed as in the proof of Lemma 6.3.

Dene as usual s (x) = g (x). Then s (x) is for every x everywhere dierentiable
3
in except in = x. We dene s (x) := 2 2 sign(x )exp( |x|
) as the derivative of s (x)
2
where s (x) is dierentiable and for = x we set sx (x) := 0. Now dene as in the proof of
Theorem 6.3 for a sequence (n ) R with n 0 and for a sequence (hn ) R with |hn | = 1
and hn h (i.e. hn is eventually constant)
p

Solution.

fn :=

s+n hn s
and f := hs .
n

It is sucient to prove (fn f )2 dx 0 because

R

(fn (x) f (x) + (hn h)s (x))2 dx

Z
Z
2
2 (fn (x) f (x)) dx + 2 ((hn h)s (x))2 dx

(fn (x) hn s (x)) dx =

and ((hn h)s (x))2 dx converges to zero because hn h is eventually equal to zero. We know
that fn (x) converges to f (x) for any x 6= . This convergence almost everywhere does not aect
the proof of Theorem 6.3. With the hint and the Cauchy-Schwartz Inequality in the last step
we obtain
R

1
(s+n hn s )2
n2
2
Z 1
p
1
1
= 2
n hn sign(x un hn ) g(x un hn )du
n
0 2
Z
1 1

g(x un hn )du
4 0

fn2 =

and we conclude by Fubini

Z

fn2 (x)dx

Z Z

Z
=
0

1
g(x un hn )dudx
0 4
Z
1
g(x un hn )dx du
4
{z
}
|
=1

1
=
.
4
R
It is straight forward to compute f 2 (x)dx = 41 . Thus we have shown
Z
Z
2
fn (x) dx f (x)2 dx.

Using this result in the third step and Fatou's Lemma (which also works for convergence almost
everywhere) in the second we get
Z

lim inf 2f 2 (x) + 2fn2 (x) (f (x) fn (x))2 dx

n
Z
lim inf 2f 2 (x) + 2fn2 (x) (f (x) fn (x))2 dx
n
Z
Z
2
4 f (x)dx lim sup (f (x) fn (x))2 dx.
n
R
But this means lim supn (f (x) fn (x))2 dx = 0 which implies the convergence we were
4

f (x)dx =

after.

Exercise 16 (4 Marks).

Consider the family of densities

p f (x) = 1x[,+1] , i.e., the densities of the uniform distribution
on the interval [, + 1]. Is f (x) dierentiable in quadratic mean?
Solution. The p
family is not dierentiable in quadratic mean. We prove this via contradiction.
So assume that f (x) is dierentiable in quadratic mean, then there is a function such that
Z p

f+h (x)

(x)
f (x) h
2

2

dx = o |h|2 .

Note that this implies that (x)2 dx < , this follows from the fact that f (x) is square
integrable and the triangle inequality for the L2 norm. So we conclude by factoring out
p

Z p
Z p
Z
2

p
p
h2
f+h (x) f (x) dx h
f+h f (x) (x)dx +
(x)2 dx = o(|h|2 ).
4

(1)
Since we are only interested in the case h 0 it is sucient if we restrict to the case |h| < 1.
Now assume that 0 h < 1, then by Triangle- and Cauchy-Schwarz Inequality we obtain
Z 


p
p




f

f
(x)
(x)dx
+h

+h

+h+1

(x)dx +

+1

Z +h





1 (x)dx

{z
}
|
1 R
1
R +h
+1+h
(
12 ) 2 ( +1
(x)2 ) 2

2| |||2 h



(x)dx
Z +h+1



+ 
1 (x)dx
+1


1

where ||||2 := (x)2 dx 2 is the L2 norm. The same result can be obtained analogously if
h < 0. Thus equation (1) implies
R

Z p
2
 3
p
f+h (x) f (x) dx = O h 2 .

But this is a contradiction as we can compute

Z p
2
p
f+h (x) f (x) dx = 2|h|

which is not O(|h| 2 ).

3

Exercise 17 (4 Marks).

Consider again example four from the lecture (the linear index model with binary response),
i.e., we have two random variables (X, Y ) where Y takes values in {0, 1}. X is distributed with
continuous density fX and
P(Y = 1|X) = F (X T ).

where F is a known increasing function with continuous derivative f . Assume that F has the
following properties:
(i) 0 < F (xT ) < 1 for each x in the support of fX and each in a xed neighbourhood of
0 .
f (y)
<
(ii) M := supyR F (y)(1F
(y))
2

Moreover assume that E(XX T ) < . You can use that the joint density of (X, Y ) is given by
f (x, y) = fX (x)(yF (xT ) + (1 y)(1 F (xT )))

with respect to the measure = (0 + 1 ) where is the Lebesgue measure and a is the
c
measure with
p a ({a}) = 1 and a ({a} ) = 0.
Show that f (x, y) is dierentiable in quadratic mean at = 0 .
Remark: Integrals with respect to may be computed as follows:
Z

Z
G(x, y)d(x, y) =

G(x, 0) + G(x, 1)dx

R{0,1}

Note also that a sequence (xn , yn ) R {0, 1} converges if and only if xn x and (yn ) is
constant for n large enough.
We apply Lemma 6.3 to prove that
p f (z) with z = (x, y) is dierentiable in quadratic
mean. Hence we need to check that s (z) = f (z) is continuously dierentiable in . By noting
that y {0, 1} we get the following for the derivative
Solution.

p
f xT
s (z) = fX (x)
2

y
1

F 2 (xT )

1y
1

(1 F (xT )) 2

x.

Continuity of s (z) is now clear since all involved functions are continuous. Next we need to
check existence and continuity of I . By denition and the Remark
Z

fX (x)f xT

I =

2
F

R{0,1}

1
2

(xT )

(1

1
1
=
fX (x)f x
+
F (xT ) 1 F (xT )
R

2
Z
f xT
=
fX (x)
xxT dx
T
T
F (x )(1 F (x ))
R
|
{z
}
T

2

!2

1y
1

F (xT )) 2


xxT d(x, y)

xxT dx

and the existence follows by the assumption that E(XX T ) is nite. Continuity of I is now a
consequence of the dominated convergence Theorem.
Exercise 18 (4 Marks).

Consider the family of exponential densities (f )>0 , i.e., f (x) = ex if x 0 and f (x) = 0
for x < 0. Show that this family is dierentiable in quadratic
mean. Show that the ML estimator

of 0 is asymptotically normal in the sense that n(n 0 ) converges in distribution to a
normal distribution. Determine the variance of this normal distribution.
n is consistent.
Remark: You know already from the problem class that
We show that f (x)
p is dierentiable in quadratic mean by showing the assertions
of Lemma 6.3. Dene s (x) := f (x), it is dierentiable in (!) with derivative
Solution.

1x
e 2 x
2

s (x) =

, x0
.
, x<0

This function is also continuous in as we consider only > 0. Thus the rst assumption of
Lemma 6.3 is fullled. For the second and third we compute by using integration by parts
Z
4

s 2 (x)dx

Z
=
0

1
(1 x)2 x
e dx = 2

which is nite and continuous.

Hence assumptions (ii) and (iii) of Lemma 6.3 are also fullled
p
and we conclude that f (x) is dierentiable in quadratic mean with l (x) = 1x
if x 0
2

and l (x) = 0 if x < 0.

To prove the asymptotic normality we employ Theorem 6.2. The consistency has been shown in
the problem class (compute the MLE and apply the law of large numbers). Condition (ii) is easy
to check if we use log(x) x 1: Fix a a neighbourhood (0 , 00 ) of 0 and let 1 , 2 (0 , 00 )
with 1 > 2
1
+ |2 1 | |x|
2
|1 2 |

+ |2 1 | |x|
0
= k(x)|1 2 |

| log f1 (x) log f2 (x)| log

with k(x) = 10 + |x|. If 1 < 2 we get the same result. Using integration by parts we see
that E(k(X)2 ) < and thus assumption (ii) of Theorem 6.2 is fullled. Assumption (iii) (a) is
4

dierentiability in quadratic mean which we have just shown. So it remains to prove that the
Fisher information is non-singular (that is (iii) (b)):

I(0 ) = E

So Theorem 6.2 tells us that

and variance 420 .

(1 x0 )2
420


=

1
>0
420

n 0 ) converges to a normal distribution with mean zero

n(

You may hand in your solutions in groups of two. Deadline for submission is
Thursday, 19th November 2015, before the lecture.

Submission:
Homepage:

http://www.math.uni-heidelberg.de/stat/studinfo/teaching_stat/stat2/index.html