Documente Academic
Documente Profesional
Documente Cultură
• y = Hx + e
• Minimize
4
J(x) = (y − Hx)T W (y − Hx) = ||y − Hx||2W (1)
Solution:
x̂ = (H T W H)−1 H T W y (2)
Say L = L − L0 + L0 . Here LH = I, L0 H = I, so (L − L0 )H = 0
LV LT = L0 V LT0 + (L − L0 )V (L − L0 )T + 2L0 V (L − L0 )T
= L0 V LT0 + (L − L0 )V (L − L0 )T + (H T V −1 H)−1 H T (L − L0 )T
= L0 V LT0 + (L − L0 )V (L − L0 )T ≥ L0 V LT0
• Minimize
J(x) = (x − x0 )T Π−1 T
0 (x − x0 ) + (y − Hx) W (y − Hx) (3)
0 4 0 4
x = x − x0 , y = y − Hx0
J(x) = x0T Π−1
0 x0
+ y 0T
W y 0
= zM −1z
4 0 I
z = − x0
y0 H
4 Π−1 0
M = 0
0 W
Get:
x̂ = x0 + (Π−1 T
0 + H W H)
−1 T
H W (y − Hx0 )
HiT Hi T
= Hi−1 Hi−1 + hTi hi
x̂i = (Π−1
0 + H T
i H i )−1 T
Hi Yi
= (Π−1
0 + H T
i−1 H i−1 + h T
i h i ) −1
(H T
i−1 Y i−1 + h T
i yi )
Define
Pi = (Π−1 T
0 + Hi Hi )
−1
, P−1 = Π0
So Pi−1 −1
= Pi−1 + hTi hi
1/2 1/2
If Wi 6= I, this modifies to (replace yi by wi yi & hi by wi hi ):
x̂i = x̂i−1 + Pi−1 hTi (wi −1 + hi Pi−1 hTi )−1 (yi − hi x̂i−1 )
The above is also the Kalman filter estimate of the state for the following
system model:
xi = xi−1
yi = hi xi + vi , vi ∼ N (0, Ri ), Ri = wi−1 (4)
RLS was for static data: estimate the signal x better and better as more and
more data comes in, e.g. estimating the mean intensity of an object from a
video sequence
RLS with forgetting factor assumes slowly time varying x
Kalman filter: if the signal is time varying, and we know (statistically) the
dynamical model followed by the signal: e.g. tracking a moving object
x0 ∼ N (0, Π0 )
xi = Fi xi−1 + vx,i , vx,i ∼ N (0, Qi )
yi = hi xi + vi , vi ∼ N (0, Ri )
This is also the MAP estimate, i.e. x̂i also maximizes p(xi |Yi )
At i = 0, x̂0 = 0, P0 = Π0 .
Using formula for the conditional distribution of Z1 |Z2 when Z1 and Z2 are
jointly Gaussian,
E[xi |Yi ] = x̂i|i−1 + Pi|i−1 hTi (Ri + hi Pi|i−1 hTi )−1 (yi − hi x̂i|i−1 )
V ar(xi |Yi ) = Pi|i−1 − Pi|i−1 hTi hi Pi|i−1 (Ri + hi Pi|i−1 hTi )−1
x̂i|i−1 = Fi x̂i−1
Pi|i−1 = Fi Pi−1 FiT + Qi
x̂i = x̂i|i−1 + Pi|i−1 hTi (Ri + hi Pi|i−1 hTi )−1 (yi − hi x̂i|i−1 )
Pi = Pi|i−1 − Pi|i−1 hTi hi Pi|i−1 (Ri + hi Pi|i−1 hTi )−1
• RLS:
– adaptive noise cancelation, given a noisy signal dn assumed to be
given by dn = uTn w + vn , get the best estimate of the weight w.
Here yn = dn , hn = un , x = w
– channel equalization using a training sequence
– Object intensity estimation: x = intensity, yi = vector of intensities of
object region in frame i, hi = 1m (column vector of m ones),
• Kalman filter: Track a moving object (estimate its location and velocity
at each time)