CSE 569 Homework #2

Total 3 points
Due on the Blackboard on Friday, October 6 by 11:59pm.

Note:
1. Submission of homework must be electronic. Most of problems can be solved by hand. The
plots of Problem 1 can be approximately done by hand or by a computer program. You can
either type your work or take pictures of your hand-written sheets and the upload your work
onto the Blackboard.
2. If you have any question on the homework problems, you should post your question on the
Blackboard discussion board (under Homework 2 Q & A), instead of sending emails to the
instructor or TAs. They will answer your question there. This will help to avoid repeated
questions, and also help the entire class to stay on the same page whenever any
clarification/correction is made.

Problem 1. Let X have an exponential density

e x , if x 0
p( x | )
0, otherwise
(1) Plot p(x|) versus x for =1 (i.e., p(x|) is viewed as a function of x).
(2) Plot p(x|) versus , 0 5, for x =2 ((i.e., p(x|) is viewed as a function of ).
(3) Given a training set of n samples D ={x1, x2, , xn}(i.i.d. samples drawn from the above
distribution), find the MLE for .

Problem 2. You are given a probability density function (pdf) p(x) for a random variable X as
p ( x) ce |2 x | , where |2x-| is the absolute value of (2x-), and is a parameter of the
density.
(a) What is the value for the constant c ?
(b) Assume that we have the following training data set D ={x1, x2, x3}. Find the Maximum
Likelihood Estimate (MLE) of given D.

Problem 3. A random variable X can take values 0, 1, 2, ., and it has the following
distribution:
n
P( X n | ) e
n!
where >0 is the parameter of the distribution. We are also given a training set D ={X1}, i.e.,
only one sample drawn from the above distribution.
(1) Find the MLE of .
(2) In this part, we further assume has the following prior distribution,
1
 e , if 0
f ( )
0, otherwise
Use the Bayesian Estimation technique to find an estimate for by finding the mean of the
posterior density f (|D).

Problem 4. Assume that we have N observation sequences drawn from a hidden Markov model
(HMM). Further assume that we have somehow obtained the corresponding state sequences
that have generated these observation sequences. Outline a way to estimate all the model
parameters for the underlying HMM.

Problem 5. (From Problem 3 of Chapter 4 in the textbook)

Problem 6. Consider a one-dimensional two-class classification problem, with the following

training data points for each class: {0, 0.4, 1, 3.7} for Class 1 and {4.8, 5, 5.5, 6} for Class 2.
Classify (by hand) a given test point x = 4, using the following two schemes respectively.
(1) The nearest-neighbor rule;
(2) The 3-nearest-neighbor rule.