Sarthak Jain (5481941) HW1 EE 5239, September 14, 2018

1:

(a) f (x, y) = x2 + y 2 + Bxy + x + 2y

∂f (x, y)
= 2x + By + 1 = 0 (1)
∂x
∂f (x, y)
= 2y + Bx + 2 = 0 (2)
∂y
Solving equation (1) and (2), we get
2 − 2B 4−B
(x = ,y = 2 ) (3)
B2 − 4 B −4
Equation (3) gives us stationary points. Note that for B=+2 or -2, both lines (1) and (2) are parallel and
stationary point does not exist.
Second order necessary condition for local minima:
 
2 2 B
∇ f (x, y) =
B 2

Eigen values of ∇2 f (x, y) = 2 − B, 2 + B. Both these eigenvalues should be non-negative for the matrix
to be positive semi-definite (necessary condition). Therefore −2 < B < 2.
Note that for B = +2 and -2, stationary points do not exist. Therefore they havent been included.
Since (i) necessary conditions are also sufficient conditions for quadratic uncontrained problems
AND (ii) all local minima are global minima for Quadratic problems,
Therefore, points (x = 2−2B
B 2 −4
, y = B4−B
2 −4 ), with −2 < B < 2 gives us global minima

2:

(a) f (x, y) = (x2 − 4)2 + y 2

For finding stationary points, we evaluate first derivative to zero
∂f (x,y)
∂y = 4x(x2 − 4) = 0
Therefore, x=0,+2,-2
∂f (x,y)
∂y = 2y = 0
Therefore, y=0
So, the stationary points are (0,0), (2,0), (-2,0)
But f(2,0) = f(-2,0) = 0; which is the minimum possible value of f(x,y).
Therefore points (2,0) and (-2,0) are global minimums.
Now for point (0,0), we find the Hessian:
 
2 −16 0
∇ f (x, y)|(x,y)=(0,0) =
0 2

Eigenvalues are 2,-16. Since one eigenvalue is positive and

one is negative, this point is neither a local maxima nor a local minima.

1
Sarthak Jain (5481941) HW1 EE 5239, September 14, 2018

Hence, we have proved that 2 stationary points are global minima and 1 is neither local minima nor
local maxima.

(b) f (x, y) = (1/2)x2 + x cos y

We first find stationary points:
∂f (x, y)
= x + cos y = 0 (4)
∂x
∂f (x, y)
= −x sin y = 0 (5)
∂y

Solving equations (4) and (5), we get: x = −cos(nπ/2) and y = nπ/2, where n is an integer
Now we compute the Hessian Matrix:
 
2 1 − sin y
∇ f (x, y)|(x,y)=(−cos(nπ/2),nπ/2) =
− sin y −x cos y

On computing the Hessian we find that:

(i) The even multiples of π/2 have positive definite Hessian as both their eigenvalues are positive.
(ii) The odd multiples of π/2 are neither local maxima nor minima because one of their eigenvalues is
positive and other is negative.
Set of local minima = (−cos(nπ), nπ) for all integers n

(c)f (x, y) = sin x + sin y + sin(x + y)

First we find the stationary points,

∂f (x, y)
= cos x + cos (x + y) (6)
∂x
∂f (x, y)
= cos y + cos (x + y) (7)
∂y

From equations (6) and (7), we see that cos x = cos y, therefore:
x = y or x = 2π − y
Putting these values in equation (1), we get:
cos(x) + cos(2x) = 0
cos x + 2 cos2 x − 1 = 0
cos x = (1/2)or − 1
Therefore stationary points are: (x, y) = (π/3, π/3), (5π/3, 5π/3), (π, π)
Now we check the Hessian for these points:
 
2 − sin x − sin (x + y) − sin (x + y)
∇ f (x, y) =
− sin (x + y) − sin x − sin (x + y)

2
(i) At (x, y)√ √ ∇ f (x, y) is negative definite because both eigen values are
= (π/3, π/3),
negative (− 3/2, −3 3/2). Therefore, (x, y) = (π/3, π/3) is a local maxima.
2
√ 5π/3), ∇ f (x, y) is positive definite because both eigen values are
(ii) At (x,√y) = (5π/3,
positive ( 3/2, 3 3/2). Therefore, (x, y) = (5π/3, 5π/3) is a local minima.
(iii) At (x, y) = (π, π), ∇2 f (x, y) is neither negative nor positive definite because both eigen values are
zero . Therefore, (x, y) = (π, π) is a saddle point.

2
Sarthak Jain (5481941) HW1 EE 5239, September 14, 2018

(d) f (x, y) = (y − x2 )2 − x2
First we find the stationary points:

∂f (x, y)
= 2(y − x2 )(−2x) − 2x = 0 (8)
∂x
∂f (x, y)
= 2(y − x2 ) = 0 (9)
∂y
From equations (8) and (9), we find that the only stationary point is (x, y) = (0, 0)
Next we find the Hessian at (x, y) = (0, 0):
 
2 −2 0
∇ f (x, y)|(x,y)=(0,0) =
0 2

Since eigen values (−2, 2) of Hessian is both negative and positive, (x,y)=(0,0) is neither a local minima
nor a local maxima.

(e) We have shown in part (d) that there are no local minima or local maxima for f(x,y), therefore it is a
monotonically increasing or decreasing function. Therefore global minima will lie at one of the boundary
points y=1 or -1.
Let y=1: f (x, 1) = (1 − x2 )2 − x2
Now this is a function of single variable. To find the stationary point, we find the first derivative:
∂f (x,1)
∂x = 2(1 − x2 )(−2x) − 2x = 0
p p
which gives x = + 3/2, − 3/2
Now, p p
3/2, 1) = f (− 3/2, 1) = −5/4
f( (10)
p p
If we
p do the similar thing
p for y=-1, we find that x = (1/ (2), −1/ (2)) and objective value at
(1/ (2), −1) or (−1/ (2), −1) is 1/4 which is larger p than the objective
p value in equation (10).
Therefore, the global minima occurs at (x, y) = ( 3/2, 1) and (− 3/2, 1)

3:

(a) Since α = 0 is a local minima for every direction d,

∂f (x ∗ +αd)
|α=0 = ∇T f (x∗)d = 0 (11)
∂α
Since d can be any direction, we choose d = ∇f (x∗). Now equation (4) becomes:
||∇f (x∗)||2 = 0
This means ∇f (x∗) = 0. Hence proved
(b) Lets take the function f (y, z) = (z − py 2 )(z − qy 2 ). First we show that for every line passing through
(y, z) = (0, 0), point (0, 0) is the minimum.
To show this, lets consider a function g(α) = f ((y, z) + (d1 , d2 )α), where (d1 , d2 ) represents any
direction.
Now we will show that g(α) is minimum at α = 0 for (y,z)=(0,0).
First order condition:

3
Sarthak Jain (5481941) HW1 EE 5239, September 14, 2018

g 0 (0) = h∇f (y, z)|(y,z)=(0,0) , (d1 , d2 )i (12)

But, ∇f (y, z)|(0,0) = [(z − p2 )(−2qy) + (z − q 2 )(−2py) (z − qy 2 ) + (z − py 2 )]T
= [0 0]T
Second order condition:
 
00 2 T 0 0
g (α)|α=0 = [d1 d2 ]∇ f (y, z)[d1 d2 ] = [d1 d2 ] [d1 d2 ]T
0 2

The second order condition will give us g 00 (0) = 2d22 which is always positive except when d2 = 0.
Therefore f (x, y) is local minimum in all directions except when d2 = 0 (and d1 not equal to zero)
Now lets check for the case when d2 = 0 i.e. we have to check the function on the Y axis (Z=0):
On the Y axis f (y, z = 0) = (pq)y 4 and we know that this function has a global and local minima at y = 0.
Hence, we have proved that function f(y,z) has a local minima at (0,0) for all directions.
Now lets take p < m < q, then f (y, my 2 ) < 0 if y 6= 0. Now we can take y as close to 0 as we want but
we wont find a region in which (0,0) is a local minima. Hence proved that (0,0) is not a local minima
of f(x,y).

4:

(a) Proof that if f 00 (x) is non-negative, function is convex

By Mean Value Theorem, we know that:
f (y) = f (x) + f 0 (x)(y − x) + f 00 (ymid )((y − x)2 )/2
But f 00 (ymid ) is non-negative. Therefore,

f (y) ≥ f (x) + f 0 (x)(y − x) (13)

Let x = µx1 + (1 − µ)x2 where 0 ≤ µ ≤ 1
Putting y = x1 and y = x2 in the inequality (13) gives us inequalities (14) and (15)

f (x1 ) ≥ f (x) + f 0 (x)(x1 − (µx1 + (1 − µ)x2 )) (14)

f (x2 ) ≥ f (x) + f 0 (x)(x2 − (µx1 + (1 − µ)x2 )) (15)

Multiplying inequality (14) by µ and inequality (15) by 1 − µ and after adding and simplying, we get
µf (x1 ) + (1 − µ)f (x2 ) ≥ f (x)
But x = µx1 + (1 − µ)x2 . Therefore
µf (x1 ) + (1 − µ)f (x2 ) ≥ f (µx1 + (1 − µ)x2 ) and hence the function is convex.

(b) Prove that if the function is convex, f”(x) is non-negative

First, lets derive the expression for f”(x)
0 0
f 00 (x) = Limh→0 f (x)−fh (x−h)
f 00 (x) = Limh→0 (f (x+h)−f (x))/h−(f
h
(x)−f (x−h))/h

f 00 (x) = Limh→0 (f (x+h)+fh(x−h)−2f

2
(x)

Now, since the function is convex, we can write

f 00 (x) = Limh→0 (2(1/2f (x+h)+1/2f
h2
(x−h))−2f (x)
≥ Limh→0 (2(f ((x+h)/2+(x−h)/2))−2f
h2
(x)

4
Sarthak Jain (5481941) HW1 EE 5239, September 14, 2018

Therefore, f 00 (x) ≥ Limh→0 (2(f (x))−2f

h2
(x)

f 00 (x) ≥ 0
Hence proved

5:

(a) For part (a), I will prove by a counter-example that the function is not convex.
Let assume f (x, y) = ||x0 y0T − xy T || is convex
Then for all 0 ≤ µ ≤ 1, µf (x, y) + (1 − µ)f (x, y) ≥ f (µ(x, y) + (1 − µ)(x, y))
Lets take (x1 , y1 ) = (2x0 , y0 ) and (x2 , y2 ) = (x0 , 2y0 ) and µ = 1/2
Now, f (x1 , y1 ) = ||x0 y0T − 2x0 y0T || = || − x0 y0T || = ||x0 y0T ||
Similarly, f (x2 , y2 ) = ||x0 y0T − x0 2y0T || = ||x0 y0T ||

µf (x1 , y1 ) + (1 − µ)f (x1 , y1 ) = (1/2)||x0 y0T || + (1/2)||x0 y0T || = ||x0 y0T || (16)

Now, f (µ(x1 , y1 ) + (1 − µ)(x2 , y2 )) = f ((3/2)x0 , (3/2)y0 )

= ||x0 y0T − (3/2)x0 (3/2)y0T || = ||x0 y0T − (9/4)x0 y0T ||
= (5/4)||x0 y0T || Therefore,

f (µ(x1 , y1 ) + (1 − µ)(x2 , y2 )) = (5/4)||x0 y0T || (17)

Since equations (16) < (17), we can see that our assumption of convexity is contradicted. Hence the
problem is not convex.

(b) For this problem we expand the given function,

Let A = x0 y0T and let us represent vectors x0 , y0 , x, y as:
x0 = [x01 x02 ...x0n ]T y0 = [y01 y02 ...y0n ]T x = [x1 x2 ...xn ]T y = [y1 y2 ...yn ]T
Therefore, we can write:
f (x, y) = (x01 y01 − x1 y1 )2 + (x01 y02 − x1 y2 )2 + ...(x01 y0n − x1 yn )2
+(x02 y01 − x2 y1 )2 + (x02 y02 − x2 y2 )2 + ...(x02 y0n − x2 yn )2
.
.
+(x0n y01 − xn y1 )2 + (x0n y02 − xn y2 )2 + ...(x0n y0n − xn yn )2
Differentiating with respect to x1 , we get:
∂f (x,y)
∂x1 = (x01 y01 − x1 y1 )y1 + (x01 y02 − x1 y2 )y2 + ...(x01 y0n − x1 yn )yn = 0
Simplifying we get

x01 (y1y01 + y2y02 + ...y1y01 ) = x1 (y12 + y22 + ... + yn2 ) (18)

Similarly differentiating with respect to x2 will give:

x02 (y1y01 + y2y02 + ...y1y01 ) = x2 (y12 + y22 + ... + yn2 ) (19)

Dividing equation (18) by (19) we get:

x1 x01
x2 = x02
If we do this for all xi s and yi s we get the following relations:

5
Sarthak Jain (5481941) HW1 EE 5239, September 14, 2018

x1 x01 x2 x02 x(n−1) x0(n−1)

= ; = ... ; = (20)
x2 x02 x3 x03 xn x0n
and y(n−1) y0(n−1)
y1 y01 y2 y02
= ; = ... ; = (21)
y2 y02 y3 y03 yn y0n

From equations (20) and (21) we can easily see that xi = px0i and yi = qy0i where p and q are real
numbers.
Putting this result in equation (18) we get

px01 (qy1y1 + qy2y2 + ...qy1y01 ) = x1 (y12 + y22 + ... + yn2 ) (22)

pq = 1 (23)
Thus the set of stationary points of f (x, y) are x = px0 and y = qy0 , such that pq = 1 (where
A = x0 y0T )
(c) Since frobenius norm is always non-negative, the minimum value it can attain is 0.
We show that at all the stationary points, f (x, y) = 0 and this will prove that all stationary points are global
minimum.
If A = x0 y0T , from part(b) we know that stationary points are (x, y) = (px0 , qy0 ) such that pq = 1
Now at stationary points, value of f (x, y) will be

f (x, y) = ||x0 y0T − xy T || = ||x0 y0T − px0 qy0T || = ||x0 y0T − (pq)x0 y0T || = 0 because pq = 1 (24)

Since all stationary points are global minima of f(x,y) and there are no constraints on x and y we can
say that all local optimal points are global optimal points