Tutorial ZQLuo

Convex Optimization Techniques for Signal
Processing and Communication

Zhi-Quan (Tom) Luo
Department of Electrical and Computer Engineering
University of Minnesota
Minneapolis, MN 55455
luozq@ece.umn.edu
Zhi-Quan (Tom) Luo
Content
1. Overview
2. Basic concepts, theory and algorithms
3. Applications
• Digital signal processing
• communication system design
4. Softwares
5. Bibliography
January 10, 2006 1

Overview Zhi-Quan (Tom) Luo
Part I: Overview
January 10, 2006 2

The Role of Optimization in Engineering

• A link/bridge between “science” and “engineering”, with many
applications
– signal processing, digital communication, fibre optic communication
– financial engineering
– control systems, mechanical engineering
– transportation
• What fraction of “real” problems are convex?

– By no means all
– Many go unrecognized
– Convex optimization plays important roles in nonconvex optimization
• Exploiting convexity in engineering context
January 10, 2006 3

What is Convex Optimization?

• Convex sets:
S = {(x, y)|x2 + y 2 ≤ 1}, ...
• Convex functions:
f (x) = |x|, ex, x2, − ln x, ...

f (x, y) = x2 + y 2, −x + y 4, ...
• Convex optimization problems:
minimize f0(x), subject to x ∈ X, fi(x) ≤ 0, i = 1, 2, ..., m,
where X is a closed convex set, each fi is convex function (0 ≤ i ≤ m).
January 10, 2006 4

+ - .

%& ' ( & ) ( * & , )% / 0, 1 )2' (

Overview
Zhi-Quan (Tom) Luo
!"#
Examples '()*Sets/Functions
$%&Convex
of +,-./01-2
Sets: 3456()*4 76894:;
56 789: < 76; 56 789:
=73>=:?7@5@A7=3@>=:?7@=5@ =:@
+,-+1./' ? +,-./0 3 4
and functions:
B B B
<=
CDEFGH
>" ?@ A" >B
C
CDECIFG EGJKLGM
NO -Q2/R-S,-T UV>W
,2/P X
January 10, 2006 5
YZ[KEDKJ\GEKJCI]^_à

Overview
!"#$%!&$'(%)*) +,+)+-./+,.!%("01(2.%3(/45.6%(,7* Zhi-Quan (Tom) Luo
Examples of Convex + + + Optimization

9 Problems
),)-. 8 :
#;"0.&$$(:<7
minimize f (x), subject to f (x) ≤ 0, i = 1, 2, ..., m.
0 i
BC
= >?@A
CD>EFGHIJGI
• eachK#$
f! 6!a%6linear
i,
is L'(%) function, ⇒ linear programming (LP)
• f0 is quadratic and fi (i) +≥,+)
1)+-is 89: ⇒ quadratic programming (QP)
. linear,
#;"0.(QCQP)
• all fi are convex quadratic &$$(M:NO
:PQ
January 10,R
S +6./4;#.6+,TU/+$.%!$;%.V#('$S!%.W
2006 6
2!%+!$+(,#XYZ[ZX
A Well Known Example: Minimum Least Squares
2
minimize kAx − bk , subject to Cx = d
• Well known and widely (wildly?) used in engineering and statistics

• The objective function f0(x) = kAx − bk2 is convex; the constraint set Cx = d is
an affine space, ⇒ convex.
• Much of the current practice in engineering design is based on Least Squares.
• It is time to use some new and more powerful optimization tools and models.
• This allows efficient solution of some previously considered intractable engineering
problems.
Variations:
• Note that we can take extra bound constraints ì ≤ xi ≤ ui. Easy!
• However, constraints of the form |xi − ai| ≥ 1 are difficult to handle (nonconvex).
Difficult
Easy problems can appear very similar to difficult ones!
January 10, 2006 7

Semidefinite Programming (SDP)

• SDP is a broad class of conic convex optimization problems:
n
X
minimize cixi
i=1
subject to A0 + x1A1 + x2A2 + · · · + xnAn ≥ 0,
where each Ai is a m × m symmetric matrix. The constraint is called linear matrix

inequality constraint (LMI).
• The set of symmetric positive semidefinite matrices forms a convex cone. LMI is a
nonlinear constraint on x.
• SDP includes as special cases: LP, convex QP, QCQP, etc.
• Very efficient algorithms and softwares: primal/dual interior point algorithms and
efficient matlab implementation (e.g., SeDuMi).
January 10, 2006 8

An Example of SDP
Covariance Matrix Approximation: For a zero mean stationary stochastic process, the
covariance matrix is positive semidefinite and Toeplitz. However, the so called finite sample
n
covariance matrix 1X T m
Rxx(n) = xixi , xi ∈ <
n i=1
is in general not Toeplitz.
m−1
X m
Method 1: minimizer kRxx(n) − riEik, subject to r ∈ < , where Ei is a
i=0
Toeplitz matrix whose i and n − i diagonals are equal to 1 and equal to zero elsewhere.
m−1
X
However, the resulting Toeplitz approximation matrix riEi may not be positive
i=0
semidefinite.
m−1
X m−1
X
Method 2: minimizer kRxx(n) − riEik, subject to riEi ≥ 0. This is a
i=0 i=0
SDP.
January 10, 2006 9

What’s the big deal about convex optimization?
Convex optimization formulation
• always achieves global minimum, no local traps
• certificate for infeasibility
• can be solved by polynomial time complexity algorithms (e.g., interior point algorithms)
• highly efficient software available
• the dividing line between “easy” and “difficult” problems (compare with solving linear
equations)
⇒ Whenever possible, always go for convex formulation.
January 10, 2006 10

Basic concepts, theory & algorithms Zhi-Quan (Tom) Luo
Part II: Basic Concepts, Theory & Algorithms

• Convex sets
• Convex functions
• Convex optimization problems: LP, QP, SOCP, SDP
• Interior Point Algorithms
January 10, 2006 11

Convex Set
$
!"#
S ⊆ <n is convex if
+ - .
%& ' ( & ) ( * & , )% / 0, 1 )2' (
x, y ∈ S, λ, µ ≥ 0, λ + µ = 1 ⇒ λx + µy ∈ S.
Geometrically, x, y ∈ S implies the line segment [x, y] ⊆ S . S is called a convex cone if
x, y ∈ S, λ, µ ≥ 0, ⇒ λx + µy ∈ S.
56 789: 76; 56 789:
3 4
January 10, 2006 12

Examples of Convex Set
• Linear subspace: S = {x | Ax = 0} is a convex cone

• Affine subspace: S = {x | Ax = b} is a convex set
• Polyhedral set: S = {x | Ax ≤ b} is a convex set
• PSD matrix cone: S = {A | A is symmetric, A ≥ 0} is convex
• Second order cone: S = {(t, x) | t ≥ kxk} is convex
Important property:
   
linear subspaces linear subspace
 affine subspaces   affine subspace 

Intersection of   is also a  
convex cones   convex cone 
convex sets convex set
January 10, 2006 13

Recognizing Convex Sets
PSD ³matrix cone: ´ S = {A | A is symmetric, positive semidefinite} is convex since

T T T¡ ¢
S= i6=j Sij x∈<n Sx , where Sij and Sx are linear subspaces and half spaces
respectively
T
Sij = {A | Aij = Aji}, Sx = {A | x Ax ≥ 0}.
3.5
2.5
2
a12
1.5
0.5
0
3
−0.5 2
−1 1
−0.5
0
0.5
1 0
1.5
2
2.5 −1
3
a22
a11
January 10, 2006 14

Recognizing Convex Sets
Second order cone: S = {(x, t) | kxk ≤ t} is a convex cone since

\ T
S= Sa , Sa = {(x, t) | a x ≤ t}.
kak=1
Here we note that kxk ≤ t iff aT x ≤ t for all kak = 1.
60
50
40
30
t
20
10
0
100
100
50
50
0
0
−50
−50
x2 −100 −100 x1
January 10, 2006 15

Robust Linear Constraint

• Linear constraint: S = {x | aT x ≥ b} represents a half space.
• Robust linear constraint:
T
S̄ = {x | (a + ∆a) x ≥ b + ∆b, ∀ k(∆a, ∆b)k ≤ ²}
is seen as the intersection of infinitely many half spaces
• In fact, robust feasible region can be characterized as

q
T
S̄ = {x | a x − b − ² kxk2 + 1 ≥ 0}
January 10, 2006 16

MVDR Beamforming
H H
mininmize w Ri+nw, subject to w a(θ0) = 1 ⇐ Least squares!
• uniform linear array; steering vector: a(θ) = (1, ejαθ , ej2αθ , ..., ej(M −1)αθ )H
• can handle sidelope bound constraints |wH a(θ)| ≤ ², ∀θ ∈ Θs.
• sidelope constraints can also be represented via an LMI (more later):
A0 + w1A1 + w2A2 + · · · + wM AM ≥ 0
• robust MVDR (more later) via robust linear constraint (more later)
H H
mininmize w Ri+nw, subject to |w (a(θ0) + ∆a)| ≥ 1, ∀k∆ak ≤ δ
• However, constraints like “only half of wi can be nonzero”are nonconvex.
January 10, 2006 17

Convex Functions

f (x) : <n → < is convex if

λf+ − ∀ y
∈<.
n
f (λx + (1 − λ)y) ≤ (x) (1 λ)f (y), x,
!"#$%$f&'(x)
When f (x) is differentiable,
"()*is$(&!+,iff&-$.
convex
f (x) ≥ / f0(x1 ) + ∆f (x6 )78

23245 2 (x −62x47),9for
0 0
T
0 :all62x47, ;x62
0 .
<247
?@=A
=> =
?@=>ABC?@=>AD@=E=>A
FGHIJKJIHLHMNG
January 10, 2006
OPQ',%!$%R(S*,%(TT%U"Q(VWXYZWWX[\]YX^_`,& 18
OQaTT,%'"&bcST$%T*(&$',def g
Recognizing Convex Functions

If f (x) is twice continuously differentiable, then
f is convex ⇔ ∇2f (x) ≥ 0 for all x ∈ <n.
Examples:
Non-convex Non-differentiable convex Differentiable convex
1 − kxk kxk xT Ax, A ≥ 0

2
cos(eT x) kxk1 e−kxk
xT Ax, A 6≥ 0 max{kxk2, eT x} − log(t2 − kxk2)
kxk3 kxk21 − log det(X)

... ... ...
January 10, 2006 19

Properties of Convex Functions

• Convexity over all lines:
f (x) is convex ⇔ f (x0 + th) is convex in t for all x0 and h
• Positive multiple:
f (x) is convex ⇔ αf (x) is convex, for all α ≥ 0
• Sum of convex functions:
f1(x), f2(x) convex ⇒ f1(x) + f2(x) is convex
• Pointwise maximum:
f1(x), f2(x) convex ⇒ max{f1(x), f2(x)} is convex
• Affine transformation of domain:
f (x) is convex ⇒ f (Ax + b) is convex
January 10, 2006 20

Some Commonly Used Convex Functions

• Piecewise-linear functions: maxi{aTi x + bi} is convex in x
• Quadratic functions: f (x) = xT Qx + 2q T x + c is convex iff Q ≥ 0
• Piecewise-quadratic functions: maxi{xT Qix + qiT x + ci} is convex in x if Qi ≥ 0

Ã n !1/k
X k
• Norm functions: kxkk = |xi| , where k ∈ [1, ∞)
i=1
• Convex functions over matrices: Tr(X), λmax(X) are convex on X = X T ;

and − log det(X) is convex on the set {X | X = X T , X ≥ 0}
m
X T −1
• Logarithmic barrier functions: f (x) = log(bi − ai x) is convex over
i=1
P = {x | aTi x ≤ bi, 1 ≤ i ≤ m}
January 10, 2006 21

Convex Optimization Problems

• Abstract form: problem
minimize f (x), subject to x ∈ C
is convex if C and f are convex (set, function)

Note: maximizing concave f over convex C is a convex optimization problem
• Standard form: problem
minimize f0(x)
subject to fi(x)≤0, i = 1, ..., m,
gi(x) = 0, i = 1, ..., p
is convex if f0, f1, ..., fm are convex, g1, ..., gp affine often written as
minimize f0(x)
subject to fi(x) ≤ 0; i = 1, ..., m,
Ax = b.
January 10, 2006 22

Recognizing Convex Optimization Problems
• Consider

minimize
x1 + x2
!−x
subject to 1 ≤ 0, −x2 ≤ 0
"
#$% ' & ( ()
* +, " * +, " * -
1 − x1x2 ≤ 0
It is convex in the abstract form since the feasible set is convex, but nonconvex in
. /0#%1 #( 2 # 301/4 3567%) 1) 8
standard form. . ) 6( 01 901$ # : ; < =
• We can replace 1 − x1x2. )≤ 15 )0 by6)a(convex

6( 01 # ; < >-, -constraint
?
− log x1 − log x2 ≤ 0.
BC
2
0
0 1 2
@A 3 4 5
January 10, 2006 23

Recognizing Convex Optimization Problems
• Linear program: fi all affine yields linear program

cT0 x + d0
minimize

cTi x + di≤ 0, i = 1, ..., m
subject to
Ax = b

• Constrained minimax problem: !"
!"#$%!&$'(%)*) +,+)+-./+,.!%("01(2.%3(/45.6%(,7*
minimize max T
i {ci9x + di } minimize# $%$#$&t' ()*,-.
+/01+2
+ + +
),)-. 8 : ⇔ 3456'788
+ <
subject to #; "0.&$$(:<7
Ax ≤ b subject to 9:
Ax /≤;b, cTi x + di ≤ t, ∀i
CDE@F>?@=AB@G
BC
= >?@A
>?@=AB@
CD>EFGHIJGI =
K#$!,6!%6L'(%)
January 10, 2006
) +,+)+-. 89: 24
'HIJ'33K3
#;"0.&$$(M:NO # $%$#$&' L
:PQ 3456'7889-.+/01+ ML
R 6/ 6 TU/ V W : /;<

Basic concepts, theory & algorithms

Zhi-Quan (Tom) Luo
Blind Channel Equalization via Linear Programming

• LTI channel; impulse response h: complex, unknown; QAM modulation

!"# $%&#'%()*+,-./
• Goal: design equalizer G(ω) such that H(ω)G(ω) = e−jDω , where D is delay.
0 123/.4356,7.89:.43./;<:.=>?/=.@
• Let xi be the channel output, and yk be the equalizer output. It is known that
01 ABC 28.=,:.89:.43./;<:.=>?/=.@
8.=,+/2DEF.4356,7.:@G=?HI5HJ 1L K1GM1ABC
minimize max |Re(y )| k k
⇒ channel equalization.
subject to Re(h1) + Im(h1) = 1
0;?NN ?/;I?,;.O 1 ABCPQRLSTUV 28.65<@
where Re(h1) + Im(h WXY1XZ).4
=356,175isH,?a/,normalizing 23>H?8.65<@
=8.;?/-?63H,?/constraint.
0;
X5 /5 8;?/=H:5,/H=?/GZYX[XZ 6,N,H=?/\]^\?:
N 8
• Note that Re(yk ) = _ à\Re GP(x
QR\k−i)Re(hi)− Im(xk−i)Im(hi) is linear in h.
i=−N
January 10, 2006 25

Blind Channel Equalization via Linear Programming

• Note that maxk {|Re(yk )|} is nonsmooth in h.
• Smoothing approach: approximating the ∞-norm by p-norm, with p large,
Ã !1/p
X p
max{|Re(yk )|} ≈ |Re(yk )|
k
k
and then remove the exponent p1 to obtain a smooth objective. ⇒ Gradient descent.
• Linear programming formulation:
minimize t
subject to Re(h1) + Im(h1) = 1
XN
−t ≤ Re(xk−i)Re(hi) − Im(xk−i)Im(hi) ≤ t, ∀ k
i=−N
• The above formulation requires fewer samples, and enjoys faster convergence.
January 10, 2006 26

5
0.5
0
−5
Basic concepts, theory & algorithms −10 Zhi-Quan (Tom) Luo−0.5

−10 −5 0 5 10 0 5
Blind Channel Equalization via Linear Programming (a)
10 0.5
10 11
55 4 1
0.5
0.5
00 02 0.5
00
−5
−5 0 0
−10
−10 −0.5
−0.5 −2
−0.5 −0.5
−10
−10 −5
−5 00 5 10
10 00 55 10
10 1515 2020 0 5 10 15 20
(a)
(a) (b)
(b) (c)
−4 −1
0.5
0.5 (a) Channel output (b) Real part −4
(c) Imaginary
−2 0
(a)
part2 4 0 10
4 4 1 1
1
2 2 0.50.5
00 0.5
0 0 0 0
0
−0.5
−2−2 −0.5 −0.5
0 5 10 15 20 −0.5
0 5 10 15 20 −0.5
(c)
(c)
−4−4 −1−1 −1
−4−4 −2−2 0 0 2 2 4 4 0 0 1010 20 20 30 30 0 10 20 30
(a)(a) (b)(b) (c)
1 1 (d) Equalized output (e) Real part (f) Imaginary part
0.50.5
January 10, 2006 27

0 0
−0.5
−0.5
−1−1
Quadratic Optimization Problems

• Convex quadratically constrained quadratic program (QCQP): each Qi ≥ 0

T T
minimize x Q

x
0 + p
0 x +
0!
c
T T
subject to x Qix + pi x + ci ≤ 0, i = 1, ..., m
Ax = b.
"# $% &$'(
)* +,-)'(.-/0(1-QP.
If Qi = 0, i ≥ 1, then it is a convex $&(1-"#If$0(some
'(-2 Q 6≥ 0, ⇒ nonconvex and hard!
i
• Optimal norm approximation with bound constraints:
3 (1(3 (4- 567589:658;
minimize 2#+,-kAx )''*< −=
5 bk>
subject to ì ≤ xi ≤ ui, i = 1, ..., n.
)*1.-@*A'(3(4$'(*1A&*+0-3(B7CD
January 10, 2006 EFGHIJGKLGMNOFP(B7C Q D 28
Convex Conic Optimization Problems
• Second order cone program (SOCP):
minimize cT x
subject to kAix + bik ≤ dTi x + ei
includes LP, convex QP, and convex quadratically constrained QP as special cases.
• Semi-definite program (SDP):
minimize Tr(CX)
subject to Tr(AiX) = bi, i = 1, .., m,
X ≥ 0.
includes LP (if C , Ai are diagonal) and SOCP as special cases.

• Model generality: LP < QP < QCQP < SOCP < SDP.
• Solution efficiency: LP > QP > QCQP > SOCP > SDP. For example, it is
much easier to exploit sparsity in LP than in SDP.
January 10, 2006 29

Robust Linear Programming via SOCP
T
minimize max (c + ∆c) x
k∆ck≤²
subject to (ai + ∆ai)T x ≤ bi + ∆bi, ∀ k(∆ai, ∆bi)k ≤ ²i
Robust linear constraint is equivalent to a SOC constraint

q
T T
(a + ∆a) x ≤ b + ∆b, ∀ k(∆a, ∆b)k ≤ ² ⇔ a x + b + ² kxk2 + 1 ≤ 0
So the robust linear program is equivalent to a SOCP
minimize t
p
subject to aTi x
+ bi + ²i kxk2 + 1 ≤ 0
²kxk + cT x ≤ t
January 10, 2006 30

Schur Complement
Let A > 0 and
· ¸
T A B T −1
X =X = , S =C−B A B.
BT C
Then S is called Schur complement of A in X , and
X ≥0 ⇔ S ≥ 0.
Schur complement is useful to represent nonlinear convex constraints as LMIs, e.g.,

· ¸
T T I (Ax + b)
(Ax + b) (Ax + b) − c x − d ≤ 0 ⇔ ≥ 0.
(Ax + b)T cT x + d
and · ¸
x 1
x ≥ 0, y ≥ 0, 1 − xy ≤ 0 ⇔ ≥0
1 y
January 10, 2006 31

Some Nonconvex Problems

‘Slight’ modification of convex problems can be very hard.
• Convex maximization, concave minimization, e.g.,
maximize kAx − bk2

subject to kxk ≤ 1.
• Nonlinear equality constraints, e.g.,
minimize cT x
subject to xT Qix + qiT x + ci = 0, 1 ≤ i ≤ K,
where Qi ≥ 0.
• Minimizing over nonconvex sets, e.g., integer constraints
find x such that Ax ≤ b, xi is integer.
January 10, 2006 32

Lagrangian Duality Theory

• Primal problem: Let p∗ be the optimal value of convex optimization problem
minimize f0(x)
subject to f1(x) ≤ 0, · · · , fm(x) ≤ 0.
• (Lagrangian) dual function:

g(λ) = inf L(x, λ) = inf {f0(x) + λ1f1(x) + · · · + λmfm(x)}
x x
where L(x, λ) = f0(x) + λ1f1(x) + · · · + λmfm(x) is the Lagrangian function

and λi’s are Lagrangian multipliers
• Weak duality: if λ ≥ 0 (dual feasible) and x is primal feasible then f (x) ≥ g(λ).
• Dual problem: Let d∗ be the optimal value of
maximize g(λ)
subject to λ ≥ 0.
• Duality gap: p∗ ≥ d∗. For convex problems and under mild conditions, p∗ = d∗.
January 10, 2006 33

KKT Optimality Condition
• Suppose each fi is differentiable. Then (x∗, λ∗) is primal and dual optimal iff
fi(x∗) ≤ 0
λ∗ ≥ 0
P
∇f0(x∗) + i λ∗i ∇fi(x∗) = 0
λ∗i fi(x∗) = 0
Here λ∗ serves as a certificate of optimality.
• Theorem of alternatives:
– there exists x such that fi(x) < 0, for all i
– there exists a λ ≥ 0 and λ 6= 0 such that g(λ) ≥ 0.
Exactly one of the above is true. In practice, λ serves as a certificate of infeasibility.
January 10, 2006 34

Interior Point Algorithms for SDP
minimize cT x
subject to A0 + x1A1 + . . . + xnAn ≥ 0
• Logarithmic barrier function:
φ(x) = − log det(A0 + x1A1 + . . . + xnAn)
• follow the Central path:

³ ´
∗ T
x (t) = argmin tc x + φ(x) , t ∈ [0, ∞)
• for any t ≥ 0, it is known f0(x∗(t)) ≥ f ∗ ≥ f0(x∗(t)) − m t

• works very well in practice; theoretical worst case polynomial complexity
• other polynomial interior point algorithms: primal-dual, predictor-corrector, etc
January 10, 2006 35

Applications Zhi-Quan (Tom) Luo
Part III: Applications

1. Robust beamforming
2. Optimal linear decentralized estimation
3. Pulse shaping filter design
4. MMSE precoder design for multi-access communication
5. Quasi-maximum likelihood detection via SDP relaxation
6. Transmit beamforming
January 10, 2006 36

Applications: robust beamforming Zhi-Quan (Tom) Luo
Robust beamforming
January 10, 2006 37

Robust Beamforming Application
x1 x2 xM−1 xM
w1 w2 w M−1 wM
• Widely used in wireless communications, microphone array speech processing, radar,

sonar, medical imaging, radio astronomy.
• The output of a narrowband beamformer is given by
H
y(k) = w x(k), where k is the sample index. (1)
January 10, 2006 38

Robust Beamforming
• The observation vector is given by
x(k) = s(k) + i(k) + n(k) = s(k)a + i(k) + n(k) (2)
where s(k), i(k), and n(k) are the desired signal, interference, and noise components,
respectively. Here, s(k) is the signal waveform, and a is the signal steering vector.
• The robustness of a beamformer to a mismatch between the nominal (presumed) and

real signal steering vectors becomes the main issue.
• Such mismatches can occur in practical situations as a consequence of look direction and
signal pointing errors, imperfect array calibration and distorted antenna shape, array
manifold mismodeling due to source wavefront distortions caused by environmental
inhomogeneities, near-far problem, source spreading and local scattering.
January 10, 2006 39

Robust Beamforming
• The optimal weight vector can be obtained through the maximization of the Signal-to-
Interference-plus-Noise Ratio (SINR)
σs2 |wH a|2

SINR = H (3)
w Ri+nw
n o
H
where Ri+n = E (i(t) + n(t)) (i(t) + n(t)) .
• The maximization of (3) is equivalent to
H H
min w Ri+nw subject to w a=1
w
¡ ¢−1
• The optimal weight vector is wopt = α R−1
a, where α =
i+n
H
a −1
Ri+na is the
normalization constant (to be omitted for brevity).
• In practice, Ri+n is replaced by the sample convariance matrix.
January 10, 2006 40

Robust Beamforming
• In practical applications, the steering vector distortions e can be bounded:
kek ≤ ².
• Then, the actual signal steering vector belongs to the set
A(²) = {c | c = a + e , kek ≤ ²}
• We impose a constraint that for all vectors in A(²), the array response should not be
smaller than one, i.e.
H
|w c| ≥ 1 for all c ∈ A(²)
January 10, 2006 41

Formulation
• The robust formulation of adaptive beamformer is
minimizew wH R̂w
subject to |wH (a + e)| ≥ 1, for all kek ≤ ².
• For each choice of e, the condition |wH (a + e)| ≥ 1 represents a nonlinear and
nonconvex constraint on w.
• Since there are an infinite number of vectors e with kek ≤ ², the robust beamforming
problem is a semi-infinite nonconvex quadratic program.
• It is well known that the general nonconvex quadratically constrained quadratic
programming problem is NP-hard and thus intractable.
January 10, 2006 42

Convex Reformulation
• However, due to the special structure of the objective function and the constraints,
the robust beamforming problem can be reformulated, surprisingly, as a convex second
order cone program:
minimizew wH R̂w © H ª
H
subject to w a ≥ ²kwk + 1 , Im w a = 0.
[Vorobyov-Gershman-Luo, 2001]
• The reformulation is based on simple 4- and Cauchy-Schwartz inequalities and the

homogeneous nature of the objective function.
• The above SOCP can be efficiently and easily solved via interior point method.
January 10, 2006 43

Mathematical Reformulation
H
|w (a + e)| ≥ 1, for all kek ≤ ²
m (4-inequality)
H H
|w a| − |w e| ≥ 1, for all kek ≤ ²
m (Cauchy-Schwartz)
H
|w a| − ²kwk ≥ 1
Therefore, the original robust beamforming formulation is equivalent to
minimizew wH R̂w
⇐ still non-convex!
subject to |wH a| ≥ ²kwk + 1.
January 10, 2006 44

Mathematical Reformulation
minimizew wH R̂w
subject to |wH a| ≥ ²kwk + 1.
m Phase rotation
minimizew wH R̂w © ª
subject to wH a ≥ ²kwk + 1 , Im wH a = 0.
January 10, 2006 45

Simulation Examples
Example 1: Steering vector mismatch due to local scattering
• In this example, the presumed signal steering vector is a plane wave impinging on the
array from 3◦.
• The real steering vector is formed by five signal paths and is given by
4
X jψi
ã = a + e b(θi) (4)
i=1
where a corresponds to the direct path, whereas b(θi) (i = 1, 2, 3, 4) correspond to

the coherently scattered paths, with θi, i = 1, 2, 3, 4 independently drawn.
• The phases ψi, i = 1, 2, 3, 4 are independently and uniformly drawn from the interval
[0, 2π].
January 10, 2006 46

Performance Comparison
15
0
−1 10
−2
5
OUTPUT SINR (DB)

−3
OUTPUT SINR (DB)
−4 0
−5
−5
−6
−7 −10
OPTIMAL SINR OPTIMAL SINR
−8 PROPOSED ROBUST BEAMFORMER PROPOSED ROBUST BEAMFORMER
SMI BEAMFORMER −15 SMI BEAMFORMER
−9 LSMI BEAMFORMER LSMI BEAMFORMER
EIGENSPACE−BASED BEAMFORMER EIGENSPACE−BASED BEAMFORMER
−10 −20
0 20 40 60 80 100 −20 −15 −10 −5 0 5
NUMBER OF SNAPSHOTS SNR (DB)
(a) output SINR v.s. sample size N ; 1st (b) output SINR versus SNR; 1st example
example
January 10, 2006 47

Simulation Examples
Example 2: Near-far steering vector mismatch
• In this example, we model the so-called near-far steering vector mismatch of the desired
signal, whereby the presume steering vector of the signal is a plane wave impinging on the
array from the normal direction 0◦, whereas the real steering vector corresponds to the
source located in the near field of the antenna at the distance D 2/λ = (M − 1)2λ/4
from the geometrical center of the array, where D = (M − 1)λ/2 is the length of
array aperture.
• The performance of the methods tested versus the number of training snapshots N for
the fixed SNR = −10 dB is shown in Fig. (c). Fig. (d) shows the performance of
these techniques versus SNR for the fixed training data size N = 30.
January 10, 2006 48

Performance Comparison
15
0
−1 10
−2
5
OUTPUT SINR (DB)

OUTPUT SINR (DB)
−3
−4 0
−5
−5
−6
−7 −10
OPTIMAL SINR OPTIMAL SINR
−8 PROPOSED ROBUST BEAMFORMER PROPOSED ROBUST BEAMFORMER
SMI BEAMFORMER −15 SMI BEAMFORMER
−9 LSMI BEAMFORMER LSMI BEAMFORMER
EIGENSPACE−BASED BEAMFORMER EIGENSPACE−BASED BEAMFORMER
−10 −20
0 20 40 60 80 100 −20 −15 −10 −5 0 5
NUMBER OF SNAPSHOTS SNR (DB)
(a) output SINR v.s. sample size N ; 2nd (b) output SINR versus SNR; 2nd example
example
January 10, 2006 49

Applications: optimal linear DES Zhi-Quan (Tom) Luo
Optimal linear decentralized estimation scheme
January 10, 2006 50

A Generic Decentralized Estimation Problem

• Our goal is to design a DES that minimizes the mean squared error (MSE) E(s − ŝ)2.
• Linear message functions: mn(xn) = QTn xn, where Qn is a tall matrix with fixed
dimension.
• Orthogonal AWGN channels: m̂n(xn) = mn(xn) + un = QTn xn + un,where un
is additive Gaussian noise with a known variance Tn.
• Linear fusion function: f (m̂1(x1), m̂2(x2), . . . , m̂N (xN )) = P1m̂1 + · · · +
PN m̂N , where Pn are matrices to be designed.
m1 (x1) m̂1(x1)
x1 = H1s + v1 S1 +
u1
m2 (x2) m̂2(x2)
x2 = H2s + v2 S2 +
u2
FC ŝ = f (m̂1, . . . , m̂N )
.. ..
.. ..
mN (xN ) m̂N (xN )

xN = HN s + vN SN +
uN
January 10, 2006 51

Single Sensor Case

• The mean squared error at the fusion center can be easily calculated as follows:
³ ³ ´´2
2 T
E(s − ŝ) = E s−P Q x+u
³ ³ ´´2
T
= E s − P Q (Hs + v) + u
³³ ´ ³ ´´2
T T
= E I − PQ H s − P Q v + u
° °2 ³ ³ ´ ´
° T ° T T
= °I − PQ H° + Tr P Q Q + T P ,
F
• This leads to the following formulation:
° °2 ³ ´
° T ° T T
minimize P,Q °I − PQ H° + Tr (P Q Q + T P )
F
subject to Tr (Q Q) ≤ p, P ∈ R`×k , Q ∈ R`×k ,
T
where p > 0 is the power budget.

• As a fourth order polynomial, the objective function is nonconvex.
January 10, 2006 52

Single Sensor Case

• The objective function
° °2 ³ ´
° T ° T T
g(Q, P) = °I − PQ H° + Tr (P Q Q + T P )
F
is a fourth order nonconvex polynomial in Q, P.

• g(Q, P) is convex in P, and P is unconstrained ⇒ g(Q, P) can be minimized first
£ ¤−1
with respect to P to obtain P = HT Q QT (I + HHT )Q + T implying
µ h i−1 ¶
T T T T
g(P) = n − Tr H Q Q (I + HH )Q + T Q H
• Thus, the original formulation becomes

Ãµ ¶−1!
³ ´−1
T T T
minimize P Tr I+H Q Q Q+T Q H
subject to Tr (QT Q) ≤ p, Q ∈ R`×k ,
January 10, 2006 53

Optimal DES Design
• If T = 0, then the objective further simplifies to

Ãµ ¶−1!
³ ´−1
T T T
ĝ(Q) = Tr I+H Q Q Q Q H
• The blue matrix is a project matrix of rank k.

• The minimum is then attained by choosing the columns of Q as the k largest left
singular vectors of H.
• This gives the optimal design for the single sensor case T = 0.
• Questions:
– What if T 6= 0? ⇐ still open.
– What about the multiple sensor case? ⇐ partial answers.
January 10, 2006 54

Optimal DES Design: Multiple Sensor Case

• The mean squared error at the fusion center can be easily calculated as follows:
Ã !2
N
X ³ ´
2 T
E(s − ŝ) = E s− Pn Qn xn + un
n=1
° °2
° N
X ° N
X ³ ´
° T ° T T
= °I − PnQn Hn° + Tr (Pn Qn Qn + Tn Pn ),
° °
n=1 F n=1
• The optimal linear DES design problem under individual sensor power constraint:
° °2
° N
X ° N
X ³ ´
° T ° T T
minimize P n ,Q n °I − PnQn Hn° + Tr (Pn Qn Qn + Tn Pn )
° °
n=1 F n=1
subject to Tr (QTn Qn) ≤ pn, Pn ∈ R `×kn
, Qn ∈ R`n×kn , 1 ≤ n ≤ N,
where pn > 0 is the power budget for sensor n.
January 10, 2006 55

Optimal DES Design: Multiple Sensor Case

• Minimizing Pn first yields the equivalent formulation
Ã !−1
N
X ³ ´−1
T T T
minimize ĝ(Q) = Tr  I + Hn Q n Qn Qn + Tn Q n Hn 
n=1
subject to Tr (QnQTn ) ≤ pn, Qn ∈ R`n×kn , n = 1, 2, . . . , N,
• The formulation is still nonconvex in the design variables {Qn}, and thus remains
difficult to handle numerically.
• Theorem: The computational complexity of designing an optimal linear DES is
NP-hard, even in the case where Tn = 0 and `n = 1 for all n.
• In other words, if each sensor is required to send exactly one linear message to the FC,
then the problem of optimally designing these linear message functions (in the MMSE
sense) is NP-hard.
January 10, 2006 56

Two Special Polynomial Time Solvable Cases

Analytic solution for the case of distortionless channels:
Theorem 1. If each sensor sends at least min{`n, `} real-valued messages to the fusion
center (i.e., kn ≥ min{`n, `}, for all n) and if sensor channels are distortionless (i.e.,
Tn = 0, for all n), then the optimal linear message functions are given by
 · ¸

 √p P̂T I`
T n n , n ∈ N> ,
m(xn) = Qn xn, with Qn = 0`n−` (5)

 √ £ ¤
pn I`n 0`−`n , n ∈ N≤,
where P̂n ∈ R`n×`n is the Q-factor in the QR factorization of Hn. Moreover, in this case,
the minimum achievable MSE is given by
Ã !−1
N
X T
Tr  I + Hn Hn . (6)
n=1
January 10, 2006 57

Two Special Polynomial Time Solvable Cases

Semi-definite programming reformulation for the case kn ≥ min{`n, `} and Tn = tnI
Theorem 2. Assume sensor channel noises are white with covariance matrices Tn = tnI
for some tn > 0. If the number of transmitted messages from sensor n is at least
kn ≥ min{`n, `} for all n, then the optimal linear DES design can be obtained by first
solving the following SDP (in the matrix variables W, {Un, Xn : n ∈ N≤}, {Ûn, X̂n :
n ∈ N>}):
minimize Tr (W)
 
W X I X
 
subject to  I S− Un − Ûn  º 0,
n∈N≤ n∈N>
· ¸
Un HTn −1
º 0, Tr (Xn) ≤ tn pn, Xn º 0, ∀ n ∈ N≤,
Hn I + Xn
· ¸
Ûn R̂Tn
º 0, Tr (X̂n) ≤ t−1
n pn , X̂n º 0, ∀ n ∈ N> .
R̂n I + X̂n
January 10, 2006 58

and then setting

 √ h 1/2 i
 Qn = tn X 0kn−`n , n ∈ N≤ ,
√ h n i (7)
 Q̂n = tn X̂1/2 0kn−` , n ∈ N> .
n
January 10, 2006 59

Applications: Pulse shaping filter design Zhi-Quan (Tom) Luo
Pulse Shaping Filter Design
January 10, 2006 60

Pulse Shaping Filter Design
• Nonconvex constraints can be converted to semi-infinite convex constraints.
• Some semi-infinite linear constraints can be compactly represented using LMI.
• Design of orthogonal signalling waveforms (pulse shapes) for digital communications

can be formulated as an SDP
• Previously awkward design problems can now be efficiently solved, including:

– minimum transmission bandwidth for a given filter length
– minimum filter length for a given bandwidth or spectral mask
• Designs are effective—they produce waveforms with superior performance to those

chosen in recent standards for digital mobile telephony.
January 10, 2006 61

Analogue Baseband PAM

& '#
$%()#%% %

!"
%%% *
%%% %%
X
sc(t) = d[n]p(t − nT )
n
The ‘channel’ generally includes modulators and demodulators
Common Design Goal: Find a waveform p(t) which minimizes the ‘spectral occupation’
subject to the ‘orthogonality’ constraint:
Z
p(t)p(t − nT ) dt = δ[n],
where δ[n] is the Kronecker delta.
January 10, 2006 62

DSP-Based Baseband PAM

)&#$% 123 4 5

# %
*+ .
/. 00 6

6

& $
(

' 0000

7

#% &#$% !,#-$%
000000 -
!" $
Now sc(t) has the same form, with

L−1
X
p(t) = g[k]φs(t − kT /N ),
k=0
• waveform design reduces to the design of a finite impulse response (FIR) filter, g[k]
• DSP implementation removes many of the physical constraints on analogue waveform
design
January 10, 2006 63

DSP-Based PAM II
• For most “reasonable” converters, the design goal reduces to finding a filter g[k]
which minimizes the spectral occupation subject to the orthogonality constraint:
X
g[k]g[k − N n] = δ[n].
k
• The spectral occupation is specified by a spectral mask:

jω jω jω
L(e ) ≤ |G(e )| ≤ U (e ), for all ω ∈ [0, π]
for some specified M`(f ) and Mu(f ).
We now demonstrate the principles of our design technique by studying several formulations
of a simple feasibility problem for the filter g[k]
January 10, 2006 64

A Simple Feasibility Problem
For a given N , L and spectral mask L(ejω ), U (ejω ), either find an orthogonal
filter g[k] of length at most L satisfying the spectral mask, or show that none
exists.
Formulation 1. Given N and L, either find a filter g[k], 0 ≤ k ≤ L − 1 such that
L−1
X
g[k]g[k − N `] = δ[`], ` = 0, 1, . . . , b(L − 1)/N c, (8a)
k=`N
jω jω jω
L(e ) ≤ |G(e )| ≤ U (e ), ∀ω ∈ [0, π] (8b)
or show that none exists.
Unfortunately, both constraints are non-convex in g[k].
January 10, 2006 65

Spectral Mask Constraints
Spectral mask on polynomial G(·):

jω jω jω
L(e ) ≤ |G(e )| ≤ U (e ) ∀ω ∈ [0, 2π).
• L(·) and U (·) are given. Typically piece-wise constant.
Pn
• G(ejω ) = k=0 g[k]e
−jkω
is a complex trigonometric polynomial
• Coefficients g[0], . . . , g[n] to be designed
0
dB
−10
Relative Magnitude,
−20
ylabel
−30
−40
−50
−60
0 0.1 0.2 0.3 0.4 0.5
f , cycles-per-sample
xlabel
January 10, 2006 66

Non-convexity
Each g[k] satisfies

jω jω jω
L(e ) ≤ |G(e )| ≤ U (e ) ∀ω ∈ [0, 2π).
(Due to the first ‘≤’ part.)
Observe: if g is real then

Ã n !2 Ã n !2
jω 2
X X
|G(e )| = g[k] cos kω + g[k] sin kω .
k=0 k=0
January 10, 2006 67

Define autocorrelation coefficients

X
rg [k] := g[k]g[k − i].
i
Note in the frequency domain (i.e., under Fourier transform)

n
X n
X
jω 2 jkω −jkω jω
|G(e )| = rg [k]e = r0 + 2 e rg [k] =: Rg (e ).
k=−n k=1
The two are equivalent.
January 10, 2006 68

Convex Reformulation II
Recall the two nonconvex constraints:
L−1
X
orthogonality constraint: g[k]g[k − N `] = δ[`],
k=`N
jω jω jω
spectral mask constraint: L(e ) ≤ |G(e )| ≤ U (e ), ∀ω ∈ [0, π]
Under the new variables rg [k]:
orthogonality constraint ⇒ rg [`N ] = δ[`],
jω 2 jω jω 2
spectral mask constraint ⇒ L(e ) ≤ Rg (e ) ≤ U (e ) , ∀ω ∈ [0, 2π).
January 10, 2006 69

A Hidden Condition
• Question: how to ensure the existence of {g[k]} such that

X
rg [k] = g[k]g[k − i].
i
• Answer: spectral factorization (Riesz-Fejer)
jω jω 2
∃G(·) : Rg (e ) = |G(e )| ∀ω ∈ [0, 2π)
jω
Rg (e ) ≥ 0 ∀ω ∈ [0, 2π).
January 10, 2006 70

A Semi-infinite LP formulation
P jω
For the autocorrelation of the filter rg [m] = k g[k]g[k + m], we have R g (e )=
|G(ejω )|2. Hence an equivalent formulation is:
Formulation 2. Given N , L and the spectral mask L(ejω ), U (ejω ), either find an
autocorrelation sequence rg [m], 1 − L ≤ m ≤ L − 1, with rg [−m] = rg [m], such
that
rg [`N ] = δ[`], for ` = 0, 1, . . . , b(L − 1)/N c, (9a)

jω 2 jω jω 2
L(e ) ≤ Rg (e ) ≤ U (e ) , ∀ω ∈ [0, 2π). (9b)
jω
Rg (e ) ≥ 0, for all ω ∈ [0, π], (9c)
or show that none exists.

PL−1
Note: Rg (ejω ) = rg [0] + 2 m=1 rg [m] cos(mω).
Important: the conditions (9b), (9c) can be transformed to LMI!
January 10, 2006 71

Example: Chip Waveform Design for IS95

• Design a ‘chip’ waveform to compete with that specified in the IS95 standard for Code
Division Multiple Access (CDMA) mobile telephony.
• Design specifications from IS95: relative spectral mask in dB, N = 4
• IS95 recommended filter: L = 48 with symmetric coefficients (linear phase), but does
not satisfy the orthogonality constraints
• Design problem: Find the minimum length filter which satisfies the spectral mask and
the orthogonality constraints
• Results in a filter of length L = 51
• This provides a substantial performance improvement for a marginal increase in

implementation cost
January 10, 2006 72

Spectra of the filters
0 0
dB
dB
−10 −10
Relative Magnitude,
Relative Magnitude,
−20 −20
ylabel
ylabel
−30 −30
−40 −40
−50 −50
−60 −60
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
xlabel
xlabel
(a) Designed filter, L = 51 (b) IS95 filter, L = 48
Power spectra of the filters in the example with the magnitude bounds from the IS95
standard. Note: L = 60 is required to obtain orthogonality and the mask achieved by
the IS95 filter
January 10, 2006 73

Spectral Mask Constraint: A Beamforming Example
• A linear array of equi-spaced antennas behaves like a spatial filter.
• Spatial frequency mask cosntraint: explicitly suppresses sidelobes
−10
ylabel, dB
−20
2
|W̃ (φ)|
−30
−40
−50
−60
−80 −60 −40 −20 0 20 40 60 80
φ, degrees
xlabel
January 10, 2006 74

Applications: MMSE multi-access transceiver design Zhi-Quan (Tom) Luo
MMSE Multi-Access Transceiver Design
January 10, 2006 75

Optimal Transceiver Design: Two-User Multi-Access

Transmitters Channel Matrices Receivers
s1 F1 H1 G1 s1
Noise n
x
+ received
signal
s2 F2 H2 G2 s2
Diagram of Two−user Communication System
Mathematical model: x = H1F1s1 + H2F2s2 + ρn, ρ > 0.

Detection: si = sign (Gix) , i = 1, 2.
Goal: Given the channel matrices, H1, H2, design transceivers F1, F2, G1, G2.
January 10, 2006 76

Mean Square Error
• The detection with receiver (equalizer) Gi: ŝi = sign (Gix) .

• Let ei denote the error vector (before making the hard decision) for user i, i = 1, 2.
Then
e1 = G1x − s1 = G1(H1F1s1 + H2F2s2 + ρn) − s1

= (G1H1F1 − I) s1 + G1H2F2s2 + ρG1n.
• This further implies

† † † 2 †
E(e1e1) = (G1H1F1 − I) (G1H1F1 − I) + (G1H2F2) (G1H2F2) + ρ G1G1
• Similarly, we have
† † † 2 †
E(e2e2) = (G2H2F2 − I) (G2H2F2 − I) + (G2H1F1) (G2H1F1) + ρ G2G2.
January 10, 2006 77

Formulation: MMSE Transceiver Design
• Our goal is to design a set of transmitting matrix filters Fi and a set of matrix equalizers
Gi such that the total mean squared error
† †
MSE = Tr(E(e1e1)) + Tr(E(e2e2))
is minimized.
• As is always the case in practice, there are power constraints on the transmitting matrix
filters:
† †
Tr(F1F1) ≤ p1, Tr(F2F2) ≤ p2
• The above is nonconvex.
• We first eliminate the variables G1, G2: the MMSE equalizers.
January 10, 2006 78

• By minimizing E(e1e†1) with respect to G1, we obtain the following MMSE equalizer
for user 1: G1 = F†1H†1W, where
³ ´−1
† † † † 2
W = H1F1F1H1 + H2F2F2H2 + ρ I .
• Substituting this into E(e1e†1) gives:

† † †
E(e1e1) = −F1H1WH1F1 + I.
• Similarly, the MMSE equalizer G2 for user 2 is given by G2 = F†2H†2W and resulting
minimized (with respect to G2) mean square error for user 2 is given by:
† † †
E(e2e2) = −F2H2WH2F2 + I.
January 10, 2006 79

Total MSE
Substituting into the above expression gives rise to

† †
MSE = Tr(E(e1e1)) + Tr(E(e2e2))
³ ´ ³ ´
† † † †
= −Tr F1H1WH1F1 − Tr F2H2WH2F2 + 2n
³ ´ ³ ´
† † † †
= −Tr WH1F1F1H1 − Tr WH2F2F2H2 + 2n
³ ´
† † † †
= −Tr W(H1F1F1H1 + H2F2F2H2) + 2n
2
= ρ Tr (W) + n,
where the last step follows from the definition of W.
January 10, 2006 80

• Introduce matrix variables: U1 = F1F†1, U2 = F2F†2.

• Then the MMSE transceiver design problem becomes
³ ´
minimizeU1,U2 Tr (H1U1H†1 + H2U2H†2 2
+ ρ I) −1
subject to Tr(U1) ≤ p1, Tr(U2) ≤ p2,

U1 ≥ 0, U2 ≥ 0.
• Reformulate using the auxiliary matrix variable W:
minimizeW,U1,U2 Tr (W)
W ≥ (H1U1H†1 + H2U2H†2 + ρ2I)−1
U1 ≥ 0, U2 ≥ 0.
January 10, 2006 81

SDP Formulation
• The constraint W ≥ (H1U1H†1 + H2U2H†2 + ρ2I)−1 is equivalent to LMI:

· ¸
W I
≥ 0. (3)
I H1U1H†1 + H2U2H†2 + ρ2I
• We obtain an SDP formulation:
minimizeW,U1,U2 Tr (W)
W satisfies (3),
U1 ≥ 0, U2 ≥ 0.
• Interior point method with arithmetic complexity O(n6.5 log(1/²)), ² > 0 is the
solution accuracy.
January 10, 2006 82

OFDM: Diagonal Designs are Optimal!

Result
If H1 and H2 are diagonal, as in the OFDM systems,

then the optimal transmitters are also diagonal.
Implication
The MMSE transceivers for an multi-user OFDM system

can be implemented by optimally setting the data rates and
allocating power to each subcarrier for all the users.
January 10, 2006 83

Linearly Precoded/Power Loaded OFDM
received signal
x
h(1) +
h(2) +
F IFFT FFT G
h(3) +
h(4) +
Transmitter Filter Channel (FIR) Noise n Receiver Filter
General Linearly Precoded OFDM System
received signal
x
f(1) h(1) + g(1)
f(2) h(2) + g(2)

IFFT FFT
f(3) h(3) + g(3)
f(4) h(4) + g(4)
Transmitter Filter Channel (FIR) Noise n Receiver Filter
Power−Loaded OFDM System
January 10, 2006 84

From SDP to SOCP Formulation
• Restricting to diagonal designs, the SDP becomes SOC:
n
X
minimizew,u1,u2 wi
i=1
Xn n
X
subject to u1(i) ≤ p1, u2(i) ≤ p2,
i=1¡ i=1 ¢
2 2 2
wi |h1(i)| u1(i) + |h2(i)| u2(i) + ρ ≥ 1,
u1(i) ≥ 0, u2(i) ≥ 0, i = 1, 2, ..., n.
• There exist highly efficient (general purpose) interior point methods to solve the above
second order cone program.
• Arithmetic complexity O(n3.5 log(1/²)), ² > 0 is the accuracy.
January 10, 2006 85

Properties of Optimal MMSE Transceiver
• Let u∗1 ≥ 0, u∗2 ≥ 0 be the optimal transceivers. Define:

½
I1 = {i | u∗1 (i) > 0, u∗2 (i) = 0}, I2 = {i | u∗1 (i) = 0, u∗2 (i) > 0},
Is = {i | u∗1 (i) > 0, u∗2 (i) > 0}, Iu = {i | u∗1 (i) = 0, u∗2 (i) = 0}.
• I1, I2: subcarriers allocated to user 1 and user 2;

Is and Iu: subcarriers shared and unused;
data rates: (|I1| + |Is|)/n, (|I2| + |Is|)/n
|h1 (i)|2 |h1 (j)|2

• – For each i ∈ I1 and j ∈ I2, we have |h2 (i)|2
≥ |h2 (j)|2
.
|h1 (i)|2 |h1 (j)|2

– For all i, j ∈ Is, we have |h2 (i)|2
= |h2 (j)|2
.
– For any i ∈ Iu and any j ∈ I1 ∪ Is, we have |h1(i)|2 < |h1(j)|2. Similarly, for
any i ∈ Iu and any j ∈ I2 ∪ Is, we have |h2(i)|2 < |h2(j)|2.
January 10, 2006 86

Intuitive Interpretation
• x = H1F1s1 + H2F2s2 + ρn, with Hi, Fi diagonal; x(i) = h1(i)f1(i)s1(i) +

h2(i)f2(i)s2(i) + ρ2n(i).
• In a fading environment, the path gains |h1(i)|2, |h2(i)|2 are random,
⇒ the probability of having two equal path gains is zero.
⇒ Is is singleton: at most one subcarrier should be shared by the two users.
• The remaining subcarriers are allocated to the two users according to the path gain
ratios: subcarrier i to user 1 and subcarrier j to user 2 only if
|h1(i)|2 |h1(j)|2
≥ .
|h2(i)|2 |h2(j)|2
• The subcarriers in Iu have small path gains for both users (i.e., both |h1(i)|2 and
|h2(i)|2 are small), and they should not be used by either user, i.e., they are useless
subcarriers!
January 10, 2006 87

Applications: Optimal rate allocation Zhi-Quan (Tom) Luo
Optimal Rate Allocation for Multi-Terminal Source Coding
January 10, 2006 88

Distributed Vector Source Estimation

• Assume a common vector source s ∈ R`
• Sensor observations: xi = His + vi, with
• Communication channels: orthogonal AWGN, differing noise variances
• Given a power/bandwidth/MSE requirement, design:
? local message functions mi(xi),
? final fusion function ŝ = f (m̂1, . . . , m̂L)
m1 (x1) m̂1(x1)
x1 = H1s + v1 S1 +
u1
m2 (x2) m̂2(x2)
x2 = H2s + v2 S2 +
u2
FC ŝ = f (m̂1, . . . , m̂L )
.. ..
.. ..
mL (xL ) m̂L (xL)

xL = HL s + vL SL +
uL
January 10, 2006 89

The Vector Gaussian CEO Problem

• Source s ∼ N (0, I`); Linear observation model with (deterministic and known) Hi,
and noise vi ∼ N (0, Ωvi ).
• Local observations are quantized to digital messages with finite rates, and are
transmitted to the FC without error; FC reconstructs s based on received finite-rate
messages.
• Assuming orthogonal links, (the separation theorem holds). [Xiao-Luo, 2005]
⇒ optimal joint design = optimal source coding + optimal channel coding.
• Rate distortion region IR(D): the set of all rates that allow the reconstruction of s
at the FC within a given distortion D .
nR1 bits
xn1 = H1sn + v1n f1
nR2 bits
xn2 = H2sn + v2n f2 ŝn
sn g
..
..
nRL bits
xnL = HL sn + vLn fL
January 10, 2006 90

Existing Work
• Berger-Tung [Berger-Tung’78] proposed an achievable region for the vector CEO

problem.
• The tightness of the Berger-Tung achievable region has not been proved in general.
• For the vector Gaussian CEO problem,
– [Gastpar et al’05] computed the sum-rate in the Berger-Tung achievable region
and interpreted it as a distributed Karhunen-Loève transform. They showed that the
problem is nonconvex.
– [Schizas-Giannakis-Jindal’05] also analyzed the achievable sum-rate in the Berger-
Tung region in a general framework of EC or CE schemes.
– They both proposed iterative algorithms to compute the optimal rate allocations
which minimizes sum-rate in the B-T region.
• Our work: the optimal rate allocation problem can be reformulated as a convex
Maxdet problem.
January 10, 2006 91

Berger-Tung Region
• Let us denote IL = {1, 2, ..., L} and define
 ¯ 
 ¯X 
def ¯
R(D0 ; Ωw1 , Ωw2 , . . . , ΩwL ) = ¯
(R , . . . , RL ) ¯ Ri ≥ I(XA ; ZA |ZAc ); ∀ A ⊆ IL .
 1 ¯ i∈A 
• The computation of B-T region requires specification of the conditional probability

distribution P (zi | xi).
• Berger-Tung considered the following Gaussian test channels
zi = xi + wi, i = 1, 2, . . . ,
where wi ∼ N (0, Ωwi ) are independent of xi. The resulting achievable region is
BT
[
R (D0) = R(D0; Ωw1 , Ωw2 , . . . , ΩwL ).
{Ωwi }
January 10, 2006 92

Berger-Tung Region for the Vector Gaussian CEO

Problem
• The sum-rate achieved through the Gaussian test channels for any given {Ωwi } is
L
X
def
RΣ = Ri = I(x1 . . . , xL; z1, . . . , zL)
i=1
( Ã L
! L
)
1 X T −1
Y −1
= log det I` + Hi (Ωvi + Ωwi ) Hi det(Iì + Ωw Ωvi ) .
2 i=1 i=1
i
• The specification of a Gaussian test channel also determines the final distortion
covariance matrix of estimating s from {zi}:
Ã L
!−1
X T −1
D= I` + Hi (Ωvi + Ωwi ) Hi .
i=1
January 10, 2006 93

Optimal Rate Allocation

• Minimizing the sum-rate subject to a distortion constraint leads to
min RΣ = R1 + . . . + RL
s.t. Tr(D) ≤ D0.
• The optimal rate allocation that is equivalent to determining the optimal noise
covariance matrices Ωwi for L parallel Gaussian test channels.
• In terms of Ωwi , the optimal rate allocation problem can be formulated as
Ã L
! L
1 X T −1 1X −1
min log det I` + Hi (Ωvi + Ωwi ) Hi + det(Iì + Ωw Ωvi )
Ωwi 2 i=1
2 i=1 i
Ã L
!−1
X T −1
s.t. Tr I` + Hi (Ωvi + Ωwi ) Hi ≤ D0 ,
i=1
Ωwi º 0.
January 10, 2006 94

Centralized Case (L = 1)
• If L = 1, or full cooperation is allowed among encoders, then there is analytical

solution for the previous problem.
• For the centralized case, the optimal rate allocation can be constructed as follows
[Schizas-Giannakis-Jindal’05]:
– obtain an MMSE estimator of s based on x;
– perform a classical vector Gaussian compression of the MMSE estimator (inverse
water-filling).
January 10, 2006 95

General Multi-sensor Case: L > 1
• When L > 1, the optimal rate allocation problem is not convex in terms of the matrix
variables {Ωwi : i = 1, 2, . . . , L}.
• For the general case of L > 1, Ωw must have block-diagonal structure
 
Ωw1 0 ... 0
 0 Ωw2 ... 0 
Ωw =  ... ... ... ... 
.
0 0 . . . ΩwL
This condition is needed to represent requirement that encoders must operate

independently.
• [Gastpar et al’05] [Schizas-Giannakis-Jindal’05] proposed Gauss-Seidel type
optimization procedures to choose {Ωwi }.
– local traps
– initialization, stepsize selections, termination
January 10, 2006 96

• We can reformulate it as an equivalent Maxdet problem that is convex.
• Step 1: introduce a distortion covariance matrix D, then we obtain
X1L
1 −1
min − log det D + log det(Iì + Ωw Ωvi )
D;Ωwi 2 i=1
2 i
s.t. Tr(D) ≤ D0,

Ωwi º 0, D º 0,
Ã L
!−1
X T −1
I` + Hi (Ωvi + Ωwi ) Hi ¹ D.
i=1
• Step 2: use the Schur complement property to obtain the following equivalent matrix
inequality for the third constraint
· PL T −1
¸
I` + i=1 Hi (Ωvi + Ωwi ) Hi I
º 0.
I D
January 10, 2006 97

• Step 3: Introduce new matrix variables Qi = (Ωvi + Ωwi )−1 for each i. Then,
−1 −1
Iì + Ωw Ωvi = (Iì − QiΩvi ) .
i
Replacing Ωwi by Qi, we can get the following equivalent problem:
X1 L
1
min − log det D − log det(Iì − QiΩvi )
D; Qi 2 i=1
2
s.t. Tr(D) ≤ D0,
D º 0,
· PL ¸
I` + i=1 HTi QiHi I
º 0.
I D
• The above Maxdet problem is now convex in the new variables {D, Qi : i =
1, 2, . . . , L}.
January 10, 2006 98

Numerical Plots of the Sum-Rate Distortion Function

• In the simulation, we take L = 2, and
Ωs = I2 ; 14
Lower bound with full encoder cooperation
Upper bound from distributed EC scheme
Optimal sum rate distortion function
Ωv1 = Ωv2 = I2; 12
· ¸ 10
1 0
H1 = ,
0 2
Rate
8
· ¸ 6
1 0
H2 = U UT ,
0 2 4
where 2
√
· ¸
2 1 −1
U= 2 1 1 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
Distortion (D 0)
January 10, 2006 99

Applications: Quasi-ML detection Zhi-Quan (Tom) Luo
Quasi-ML Detection Via SDP Relaxation
January 10, 2006 100

MIMO Channel
s1 h12 y1 • n - number of transmit antennas;

h22
• m - number of receive antennas;
s2 y2
hn2 • S n - constellation of signals with
dimension n;
sn ym • s ∈ S n - vector of transmitted signals
r drawn from C n;
ρ X
yi = hik sk + vi • y ∈ Cm - vector of received signals;
n k
• H ∈ Cm×n - the matrix of fading
Written in the vector form: coefficients, hik ∼ N (0, 1), ∀i, k;
• ρ - expected value of SNR at each receiver
r antenna;
ρ
y= Hs+v • v ∈ Cm - i.i.d. noise, vi ∼ N (0, 1), ∀i.
n
• Consider n = m in this talk.
January 10, 2006 101

Maximum-likelihood detection
• For a memoryless channel with equiprobable input signals (typical in practice), the
maximum-likelihood (ML) detection achieves the minimum error probability Pe.
• ML Detector solves the following optimization problem:
sM L = arg max
n
p(y|s, H)
s∈C
• For the Gaussian noise ML detection can be written in the form:

° q °2
° °
sM L = arg min ° y − ρ/n H s°
n°
s∈C °
• The problem is NP-hard in general due to the discrete nature of the constellation set.
January 10, 2006 102

Challenge of ML detection
• MIMO systems:
– Exhaustive search can be applied for small n.
– For reasonably large n (e.g., n = 200), exhaustive search is prohibitively expensive
• Existing approaches:
– either has polynomial complexity (O(n3)) but sacrifices performance,
– or ensures good decoding performance with exponential complexity
January 10, 2006 103

Existing suboptimal approaches

• Solve an unconstrained problem with penalty function:
q γ
2 2
s̄ = arg min ky − ρ/n Hs k + k sk
s∈Rn 2
• Assuming: invertibility of HT H + γ I; BPSK signalling, the estimate of ML solution

can be expressed as
Ãr µ ¶−1 !
ρ ρ T T
ŝ = sign H H + γI H y
n n
• Choosing different values for penalty factor γ we obtain
γ = 0, for decorrelator
γ → ∞, for matched filter
γ = 1, for LMMSE detector.
• All with O(n3) complexity, but sacrifices performance.
January 10, 2006 104

Performance degradation of suboptimal schemes

0
10
−1
10
−2
10
BER
−3
10
ML Detector
SDP Detector
LMMSE Detector
−4
Matched Filter
10 Decorrelator
Nulling and Cancelling
−5
10
2 4 6 8 10 12 14 16 18
SNR, dB
January 10, 2006 105

Another suboptimal scheme: Sphere Decoder

Intuition:
• The objective function is convex over Rn.
• Integer solution must be close to the
unconstrained minimum: 16
q
2 14
ŝ = arg min n
k y − ρ/n Hs k
s∈R
q 12
T −1 T
= n/ρ (H H) H y 10
8
• Idea: localize the exhaustive search to the
6
sphere around zero-forcing solution ŝ.
• Two questions: 4
– How to choose sphere radius r ? 2

2
– How to tell which points are inside the 0
0
1
−1
sphere? 3 2 1 0 −1 −2 −3
−2
January 10, 2006 106

Another suboptimal scheme: Sphere Decoder
• Sphere Decoder [Fincke-Pohst, 1985] answers the second question:

p
– Let ρ/n H = VR (QR-decomposition), then we have
q ρ
2 T T T T
ky − ρ/n Hsk = (s − ŝ) H H(s − ŝ) = (s − ŝ) R R(s − ŝ)
n
Xn Xn
2 rij 2 2
= rii(si − ŝi + (xj − x̂j )) ≤ r
i=1
r
j=i+1 ii
2
– A necessary condition for s inside the sphere is rnn (sn − ŝn)2 ≤ r 2. The remaining
coordinates sn−1, . . . , s1 can be searched iteratively. Back-track if necessary.
• The expected complexity is exponential [Jalden-Ottersten, 2004], despite good empirical
performance for small values of n and large ρ.
January 10, 2006 107

Sphere Decoder Performance

SD has excellent performance when it works, but can fail in low SNR region or for
large systems 0
10
Sphere Decoder
SDP Detector
−1
10
BER
−2
10
−3
10
30 35 40 45 50 55 60 65 70
Dimension, n
Figure 1: BER degradation due to the limit on detection time. Simulation parameters: BPSK
modulation, SNR = 10 dB and time limit per bit = 6.3 ms.
January 10, 2006 108
Dilemma
Existing approaches for MIMO detection
• either has a worst case polynomial complexity O(n3), at a substantial performance

loss
• or finds the ML solution via heuristic search, but with an exponential average
complexity
• Questions:
– is there a polynomial approximation algorithm which offers a constant factor
performance guarantee?
– How about SDP relaxation approach?
January 10, 2006 109

A close look at ML detection problem

q
ρ
• Data model: H ∼ N (0, I); y = n He + v; v ∼ N (0, I).
• Define matrix Q and new variable x:
" p # · ¸
T T
(ρ/n)
p H H − ρ/n H y s
Q= , x=
− ρ/n yT H kyk2 1
• The objective function (negative log likelihood function) can be written as

° q °2
° °
°y − ρ/n Hs° = xT Qx, ⇐ homogenization
° °
and the ML detection problem becomes
fM L = minimize xH Qx
⇐ BQP: NP-hard
subject to x ∈ {−1, 1}n
January 10, 2006 110

SDP Relaxation of BQP
• Introducing matrix variable X = xxH º 0, then X is rank 1 and

H H 2
x Qx = Tr(Qxx ) = Tr(QX), and Xi,i = xi = 1.
• ML detection problem is equivalent to the problem
fM L := min Tr(QX)
s.t. X º 0, X is rank-1 ⇐ BQP reformulation
Xi,i = 1, i = 1, . . . , n + 1.
fM L = minimize xH Qx
⇐ BQP: NP-hard
subject to x ∈ {−1, 1}n
January 10, 2006 111

SDP Detector, Relaxation
• SDP Detector drops the only non-convex constraint rank(X) = 1 and solves
fSDP := min Tr(QX)

s.t. X º 0,
Xi,i = 1, i = 1, . . . , n + 1.
• The output Xopt of the SDP problem is no longer rank-1. (But almost rank-1)
• Different randomized rounding procedures can be employed to generate an estimate of
transmitted signals ŝ, given Xopt.
• The complexity of the problem is O(n3.5) (interior point methods).
January 10, 2006 112

SDP Detector, Randomized Rounding
1. Take a spectral decomposition:

n+1
X T
Xopt = λiuiui .
i=1
2. Pick eigenvector ū that corresponds to the largest eigenvalue λ̄.

3. Generate a fixed number of vectors x randomly according to the distribution:
√ √
1 + λ̄ ūi 1 − λ̄ ūi
Pr{xi = +1} = , Pr{xi = −1} = .
2 2
√
4. Note: E{xi} = λ̄ ūi.
5. Output x̂ that achieves the smallest objective value fSDR = xT Qx.
January 10, 2006 113

SDP Detector, BER vs SNR, n = 10

0
10
SDP Detector
ML Detector
−1
10
−2
10
BER
−3
10
−4
10
−5
10
2 4 6 8 10 12 14 16 18
SNR, dB
Key property: for all ρ, performance gap bounded for all and n!
January 10, 2006 114

Softwares Zhi-Quan (Tom) Luo
Part VI: Softwares∗
∗
This list is somewhat outdated now. Consult www for the latest versions of software available.
January 10, 2006 115

SDPA
• Authors: Fujisawa, Kojima, Nakata
• Version; available: 5.02, 9/2000; yes (binary only)
• Key paper: [43]

The software manual and the SDPA source codes can be downloaded from
http://is-mj.archi.kyoto-u.ac.jp/~fujisawa/research.html
• Features: primal-dual method, users Meschbach library
• Language, Input format: C++, SDPA
• Solves: SDP
• Remarks: numerically robust, good for small size SDPs; parallel versions avaliable.
January 10, 2006 116

SeDuMi
• Authors: Sturm
• Version; available: 1.04, 9/2000; yes
• Key paper: [22]
• Features: self-dual embedding, dense column handling
• Language, Input format: Matlab + C, Matlab, SDPA, SDPpack
• Solves: SDP, convex quadratic program, SOCP, linear program
• Remarks: benchmark SDP/SOCP solver; now being maintained at McMaster

University, Canada; various Matlab interfaces have been built for SeDuMi, including
“convex” from Stanford University.
January 10, 2006 117

CSDP
• Authors: Borchers
• Version; available: 3.2, 12/15/2000; yes
• Key paper: [44]
• Features: infesaible predictor-corrector path following interior method

• Language, Input format: C; SDPA
• Solves: SDP
• Remarks: small to medium sized SDP’s. No support for SOCP constraints
January 10, 2006 118

DSPA
• Authors: Benson and Ye
• Version; available: 3.2, 11/2000; yes
• Key paper: [45]
• Features: Dual scaling potential reduction method, Matlab interface, generates primal
solution only when requested
• Language, Input format: C; SDPA
• Solves: SDP
• Remarks: good for solving large sparse problems arising from combinatorial
optimization
January 10, 2006 119

SDPT3
• Authors: Toh, Todd, Tutuncu

• Version; available: 3.0; yes
• Key paper: [46]
• Features: infeasible primal-dual and homogeneous self-dual methods, Meschbach
library, Lanczos steplength control
• Language, Input format: Matlab+C, SDPA
• Solves: SDP and SOCP
• Remarks: allows Hermitian matrix variables, good for small/medium size problems
(with up to around a thousand constraints involving matrices of order up to around
a thousand; can also solve some large sparse problems (up to 20,000 constraints and
50,000 variables)
January 10, 2006 120

MOSEK
• Authors: Andersen
• Version; available: 1.4-31; yes (binary)
• Key paper: [47]
• Features: special SOCP algorithm
• Language, Input format: C; QPS, AMPL
• Solves: SOCP
• Remarks: similar to SeDuMi except is 100% C code for speed; callable from Matlab;
exploits sparsity and handles fixed and upper bounded variables; does not handle
semidefinite matrix cone.
January 10, 2006 121

References Zhi-Quan (Tom) Luo
Part V: References∗
∗
Somewhat outdated.
January 10, 2006 122

References
[1] Nesterov, Y. and Nemirovskii, A., Interior-Point Polynomial Algorithms in Convex Programming, SIAM
Studies in Applied Mathematics, SIAM, Philadelphia, 1994.
[2] Ye, Y., Interior Point Algorithms: Theory and Analysis, Wiley-Interscience Series in Discrete
Mathematics and Optimization, John Wiley & Sons, 1997.
[3] Boyd, S., El Ghaoui, L., Feron, E. and Balakrishnan, V., Linear Matrix Inequalities in System and
Control Theory, SIAM Studies in Applied Mathematics, SIAM, Philadelphia, 1994.
[4] Boyd, S. and Vandenberghe, L., Convex Optimization, Cambridge University Press, 2003. [Some of the
figures and examples in this tutorial were modelled from this book.]
[5] Mittelmann, H.D., “An Independent Benchmarking of SDP and SOCP Solvers,” preprint, Department
of Mathematics, Arizona State University, 2001.
[6] Wang, X., Lu, W.S. and Antoniou, A., “Constrained Minimum-BER Multiuser Detection,” IEEE
Transactions on Signal Processing, Vol. 48, No. 10, Oct. 2000, pp. 2903–2909.
[7] Vandenberghe, L. and Balakrishnan, V., “Algorithms And Software Tools For LMI Problems In Control:
An Overview,” Proceedings of the 1996 IEEE International Symposium on Computer-Aided Control
System Design, 1996, pp. 229–234.
[8] Vandenberghe, L. and Balakrishnan, V., “Algorithms And Software For LMI Problems In Control,”
IEEE Control Systems Magazine, Vol. 17, No. 5, Oct. 1997, pp. 89–95.
January 10, 2006 123

[9] Wu, S.-P., Boyd, S. and Vandenberghe, L., “FIR Filter Design Via Semidefinite Programming And
Spectral Factorization,” Proceedings of the 35th IEEE Conference on Decision and Control, Vol. 1,
1996, pp. 271–276.
[10] Alkire, B. and Vandenberghe, L., “Handling Nonnegative Constraints In Spectral Estimation,”
Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers,
Vol. 1 , 2000, pp. 202–206.
[11] Balakrishnan, V., Wang, F. and Vandenberghe, L., “Applications Of Semidefinite Programming In
Process Control,” Proceedings of the 2000 American Control Conference, Vol. 5, 2000, pp. 3219–3223.
[12] Vandenberghe, L. and Balakrishnan, V., “Semidefinite Programming Duality And Linear System Theory:
Connections And Implications For Computation,” Proceedings of the 38th IEEE Conference on Decision
and Control, Vol. 1, 1999, pp. 989–994.
[13] Xiao, L., Johansson, M., Hindi, H., Boyd, S. and Goldsmith, A., “Joint Optimization Of Communication
Rates And Linear Systems”, Proceedings 2001 Conference on Decision and Control, December 2001,
[14] Kosut, R., Chung, W., Johnson, C. Boyd, S., “On Achieving Reduced Error Propagation Sensitivity
In DFE Design Via Convex Optimization” Proceedings of the 39th IEEE Conference on Decision and
Control, Vol. 5, pp. 4320 - 4323 , December 2000.
[15] Wu, S.-P. and Boyd, S., “SDPSOL: A Parser/Solver For Semidefinite Programs With Matrix Structure”,
in Recent Advances in LMI Methods for Control, L. El Ghaoui and S.-I. Niculescu, editors, SIAM,
chapter 4, pp. 79–91, 2000.
[16] Lebret, H. and Boyd, S., “Antenna Array Pattern Synthesis Via Convex Optimization,” IEEE
Transactions on Signal Processing, Vol. 45, No. 3, March 1997, pp. 526–532.
January 10, 2006 124

[17] Lobo, M., Vandenberghe, L., Boyd, S. and Lebret, H., “Applications Of Second-Order Cone
Programming,” Linear Algebra and its Applications, Vol. 284, November 1998, pp. 193–228.
[18] Vandenberghe, L. Boyd, S. and El Gamal, A., “Optimizing Dominant Time Constant In RC Circuits”
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 17, No. 2, Feb.
1998, pp. 110–125.
[19] Alkire, B. and Vandenberghe, L., “Interior-Point Methods For Magnitude Filter Design,” Proceedings
of 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 6, 2001, pp.
3821–3824.
[20] Dumitrescu, B., Tabus, I. and Stoica, P., “On The Parameterization Of Positive Real Sequences And
Ma Parameter Estimation,” IEEE Transactions on Signal Processing, Vol. 49, No. 11, Nov. 2001, pp.
2630 -2639.
[21] Stoica, P.; McKelvey, T.; Mari, J., “MA Estimation In Polynomial Time,” IEEE Transactions on Signal
Processing, Vol. 48, No. 7, July 2000, pp. 1999 -2012.
[22] Sturm, J. F., “Using Sedumi 1.02, A Matlab Toolbox For Optimization Over
Symmetric Cones,” Optim. Meth. Software, vol. 11–12, pp. 625–653, 1999. See
http://fewcal.kub.nl/sturm/software/sedumi.html for updates.
[23] Vandenberghe, L. and Boyd, S., “Semidefinite Programming”, SIAM Review, vol. 31, pp. 49–95, 1996.
[24] Yu, W., Rhee, W. and Cioffi, J., “Multiuser Transmitter Optimization for Vector Multiple Access
Channels”, manuscript, STAR laboratory, Department of Electrical Engineering, Stanford University,
CA 94305-9515.
January 10, 2006 125

[25] Pesavento, M., Gershman, A. and Luo, Z.-Q., “Robust Array Interpolation Using Second-Order Cone
Programming,” Accepted for publication in IEEE Signal Processing Letters.
[26] Ma, W.-K., Davidson, T.N., Wong, K.M., Luo, Z.-Q. and Ching, P.-C., “Quasi-maximum-likelihood
Multiuser Detection Using Semi-definite Relaxation,” Accepted for publication in IEEE Transactions
on Signal Processing.
[27] Luo, Z.-Q., Meng, M., Wong, K.M. and Zhang, J.-K., “A Fractionally Spaced Blind Equalizer Based
on Linear Programming,” Accepted for publication in IEEE Transaction on Signal Processing.
[28] Li, L., Luo, Z.-Q., Wong, K.M. and Bossé, E., “Semidefinite Programming Solutions to the Robust
State Estimation Problem”, Accepted for publication in SIAM Journal on Optimization.
[29] Afkhamie, K.H., Luo, Z.-Q. and Wong, K.M., “MMSE Decision-Feedback Equalization with Short
Training Sequences: An Application of Interior Point Least Squares,” Accepted for publication in IEEE
Transactions on Signal Processing.
[30] Li, L., Luo, Z.-Q., Wong, K.M. and Bossé, E. “Convex Optimization Approach to Identity Fusion
For Multi-Sensor Target Tracking,” IEEE Transaction on Systems, Man and Cybernetics, Part A:
Systems and Humans, Vol. 31, No. 3, pp. 172–178, 2001.
[31] Fu, M., Souza, C. and Luo, Z.-Q., “Finite Horizon Robust Kalman Filter Design,” Accepted for
publication in IEEE Transactions on Signal Processing.
[32] Ding, Z. and Luo, Z.-Q., “A Fast Linear Programming Algorithm for Blind Equalization,” IEEE
Transactions on Communications, September, 2000.
[33] Afkhamie, K.H. and Luo, Z.-Q., “Blind Identification of FIR Systems Driven by Markov-Like Input
Signals,” IEEE Transactions on Signal Processing, Vol. 48, No. 6, pp. 1726–1736, 2000.
January 10, 2006 126

[34] Davidson, T.N., Luo, Z.-Q. and Wong, K.M., “Spectrally-Efficient Orthogonal Pulse Shape Design via
Semidefinite Programming,” IEEE Transactions on Signal Processing, Vol. 48, No. 5, pp. 1433–1445,
2000.
[35] Afkhamie, K., Luo, Z.-Q. and Wong, K.M., “Adaptive Linear Filtering Using Interior Point Optimization
Techniques,” IEEE Tran. on Signal Processing, Vol. 48, No. 6, pp. 1637–1648, 2000.
[36] Li, X.-L., Luo, Z.-Q., Wong, K.M. and Bosse, E., “An Interior Point Linear Programming Approach
to Two-Scan Data Association,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 35,
No. 2, pp. 474–490, 1999.
[37] Shahbazpanahi, S., Gershman, A., Luo, Z.-Q. and Wong, K.M., “Robust Adaptive Beamforming
For General-Rank Signal Models,” manuscript, Department of Electrical and Computer Engineering,
McMaster University, February 2002.
[38] Maricic, B., Luo, Z.-Q. and Davidson, T.N., “Blind Constant Modulus Equalization via Convex
Optimization,” manuscript, Department of Electrical and Computer Engineering, McMaster University,
April 2001.
[39] Luo, Z.-Q., Davidson, T.N., Giannakis, G.B. and Wong, K.M., “Transceiver Optimization for Multiple
Access through ISI Channels”, manuscript, Department of Electrical and Computer Engineering,
McMaster University, September 2001.
[40] Davidson, T.N., Luo, Z.-Q. and Sturm, J., “Linear Matrix Inequality Formulation of Spectral Mask
Constraints,” manuscript, Department of Electrical and Computer Engineering, McMaster University,
2001.
[41] Cui, S., Luo, Z.-Q. and Ding, Z., “Robust Blind Multiuser Detection against Signature Waveform
January 10, 2006 127

Mismatch,” manuscript, Department of Electrical and Computer Engineering, McMaster University,
2001.
[42] Vorobyov, S., Gershman, A. and Luo, Z.-Q., “Robust Adaptive Beamforming Using Worst-Case
Performance Optimization,” Proceedings of 2002 ICASSP, Orlando, Florida, 2002.
[43] Fujisawa, K., Kojima, M. and Nakata, K., “SDPA (Semidefinite Programming Algorithm) – User’s
Manual –,” Technical Report B-308, Tokyo Institute of Technology, Revised on August 2000.
[44] Borchers, B., “CSDP, A C Library for Semidefinite Programming,” Optimization Methods and Software,
Vol. 11, 1999, pp. 613–623.
[45] Benson, S.J. and Ye, Y., “DSDP3: Dual Scaling Algorithm for General Positive Semidefinite
Programming,” preprint ANL/MCS-P851-1000, Argonne National Labs.
[46] Toh, K.C., Todd, M.J., and Tutuncu, R.H., “SDPT3 – a Matlab Software Packgage for Semidefinite
Programming, version 3.0,” Technical Report, National University of Singapore, June 2001.
[47] Andersen, E.D., Roos, C. and Terlaky, T., “On Implementing a Primal-Dual Interior-Point Method
for Conic Quadratic Optimization,” Report W-274, Helsinki School of Economics and Business
Administration, Department of Economics and Management Science, Helsinki, Finland, December
2000.
January 10, 2006 128

Acknowledgement
• Graduate students: Afkhamie, Cui, Dam, Meng, Li, Liu, Lu, Maricic, Saad, Tan, Yu,
Kisialiou, Xiao,...
• Postdocs: Davidson, Gao, Jin, Li, Sturm, Wu, Zhang, Vorobyov...
• Colleagues: Wong, Davidson, Gershman
• Industrial sponsors: Nortel/Mitel/Ontario Hydro/...
• Federal/provincial support: NSF/ARO/DREV/DREO/NSERC/CITO...
Thank You!
http://www.ece.umn.edu/users/luozq
January 10, 2006 129

Tutorial ZQLuo

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Tutorial ZQLuo

Încărcat de

Drepturi de autor:

Formate disponibile

Convex Optimization Techniques for Signal

Processing and Communication

2. Basic concepts, theory and algorithms

January 10, 2006 1

January 10, 2006 2

The Role of Optimization in Engineering

• What fraction of “real” problems are convex?

• Exploiting convexity in engineering context

January 10, 2006 3

What is Convex Optimization?

f (x) = |x|, ex, x2, − ln x, ...

• Convex optimization problems:

minimize f0(x), subject to x ∈ X, fi(x) ≤ 0, i = 1, 2, ..., m,

where X is a closed convex set, each fi is convex function (0 ≤ i ≤ m).

January 10, 2006 4

 +,-+1./' ? +,-./0 3 4

Examples of Convex + + + Optimization

A Well Known Example: Minimum Least Squares

• Well known and widely (wildly?) used in engineering and statistics

January 10, 2006 7

Semidefinite Programming (SDP)

where each Ai is a m × m symmetric matrix. The constraint is called linear matrix

• SDP includes as special cases: LP, convex QP, QCQP, etc.

January 10, 2006 8

January 10, 2006 9

What’s the big deal about convex optimization?

Convex optimization formulation

• always achieves global minimum, no local traps

• certificate for infeasibility

• highly efficient software available

⇒ Whenever possible, always go for convex formulation.

January 10, 2006 10

Part II: Basic Concepts, Theory & Algorithms

• Convex optimization problems: LP, QP, SOCP, SDP

• Interior Point Algorithms

January 10, 2006 11

Geometrically, x, y ∈ S implies the line segment [x, y] ⊆ S . S is called a convex cone if

56 789: 76; 56 789:

January 10, 2006 12

Examples of Convex Set

• Linear subspace: S = {x | Ax = 0} is a convex cone

January 10, 2006 13

Recognizing Convex Sets

PSD ³matrix cone: ´ S = {A | A is symmetric, positive semidefinite} is convex since

January 10, 2006 14

Recognizing Convex Sets

Second order cone: S = {(x, t) | kxk ≤ t} is a convex cone since

Here we note that kxk ≤ t iff aT x ≤ t for all kak = 1.

January 10, 2006 15

Robust Linear Constraint

is seen as the intersection of infinitely many half spaces

• In fact, robust feasible region can be characterized as

January 10, 2006 16

• However, constraints like “only half of wi can be nonzero”are nonconvex.

January 10, 2006 17

f (x) ≥ / f0(x1 ) + ∆f (x6 )78

Recognizing Convex Functions

f is convex ⇔ ∇2f (x) ≥ 0 for all x ∈ <n.

1 − kxk kxk xT Ax, A ≥ 0

xT Ax, A 6≥ 0 max{kxk2, eT x} − log(t2 − kxk2)

kxk3 kxk21 − log det(X)

January 10, 2006 19

Properties of Convex Functions

f (x) is convex ⇔ f (x0 + th) is convex in t for all x0 and h

+,-+1./' ? +,-./0 3 4

• Consider

• We can replace 1 − x1x2. )≤ 15 )0 by6)a(convex