Sunteți pe pagina 1din 31

FUNDAMENTAL TECHNIQUES

FOR 3D COMPUTER VISION


A Tutorial for Entry Level Researcher

version AUTHOR YEAR Apart from any fair dealing for the purposes of research or private study, or

1.0
criticism or review, this publication may only be reproduced, stored or
transmitted, in any form or by any means, with the prior permission in
SUNU WIBIRAMA 2011 writing of the author. Citing is expected when part or whole content of
this material is used in academic publication

FUNDAMENTAL TECHNIQUES FOR 3D COMPUTER VISION


A Tutorial for Entry Level Researcher

Written by:

Sunu Wibirama, M.Eng


Department of Electrical Engineering and Information Technology,
Faculty of Engineering,
Gadjah Mada University
INDONESIA
 

Copyright Statement

This tutorial is taken from a research report entitled Real-time 3D Eye Movements Tracking and
Visualization using Dual Cameras Acquisition funded by AUN/Seed-Net-JICA Programme
in 2008-2010. Apart from any fair dealing for the purposes of research or private study, or criticism or
review, this publication may only be reproduced, stored or transmitted, in any form or by any means, with
the prior permission in writing of the author. Please cite the following paper when part or whole
content of this tutorial is used in academic publication:

S. Wibirama, S. Tungjitkusolmun, C. Pintavirooj, and K. Hamamoto, “Real Time Eye Tracking using Initial
Centroid and Gradient Analysis,” in Proceeding of The 2009 6th International Conference of Electrical
Engineering/Electronics, Computer, Telecommunications, and Information Technology (ECTI). ISBN:
978-1-4244-3388-9. Pattaya, Thailand. 6-8 May 2009, pp. 1058-1061.



 

 




                   

 ρ3 ρ2

               









XS YS
 xi = f  yi = f  
ZS ZS



 

               
  
 v
XS 
 v =  YS   
 Z S 

 v  v


XS 
Y 
 v= S 
 ZS 

 
1 


f 0 0 0
 P =  0 f 0 0   
 0 0 1 0 


 w = Pv  


 fX S 
 w =  fYS   
 Z S 

             zs 

 XS 
f ZS 
 w=  
 YS 
f 
 ZS 



 

          w       




                  
                 
                

 p ( x , y ) 
0 0







                

 ( x p ix , y p ix ) 




 

x pix = k x xi + syi + x0
 XS Y  
= kx f + sf S + x0
ZS ZS

y pix = k y yi + y0
 YS  
= ky f + y0
ZS

 k  k  s


x y

 ( x , y )  u 


p ix p ix

 u = [u v w] , where x pix =
T u
and y pix = 
v

w w


X 
u  α x s x0 0  S 
  v  = G.v =  0 αy y0
 Y
0  S   
   Z 
 w   0   S 
0 0 1
 1 

 α = k f  α = k f        
x x y y
f     

α x s x0 0 α x s x0  1 0 0 0
   
 G = 0 α y y0 0 = 0 α y y0  0 1 0 0 = K [ I3 | O3 ]  
0 0 1 0 0 0 1  0 0 1 0
   







 









              




   . 



 


                  






 w = Pv  
 v  w 
 P 

                

 w = PTG v  

1 0 0 −XG 
 
0 1 0 −YG 
 TG =   
0 0 1 − ZG 
 
0 0 0 1 


 w = PRTG v  
 R = R β Rα



 

 cos β 0 − sin β 0
0 0 
 Rβ = 
1 0
 
sin β 0 cos β 0
 
0 0 0 1


1 0 0 0
0 cosα sin α 0
 Rα = 
0 − sin α cosα 0
 
 
0 0 0 1


 w = PTC RTG v  

1 0 0 −XO 
 
1 0 −YO 
 TC = 
0
 
0 0 1 −Z O 
 
0 0 0 1 


XS 
u 
 v  = K I | O R −T   YS 
   [ 3 3 ] T   
O3 1   ZS 
 w   
 1 




R −T 
  T  
O3 1



 


          X    v = [ X Y Z 1]  
W S S S
T

 X 
W

 X 
 XW =  W   
1 

 u = [u v w] 
T

 u = K R ( X W − t )  
 u 
X  X 
 u = [ KR | − KRt ]  W  = M  W  = M 
XW  
 1   1 

  X 
W

             α  
   α M    α ≠ 0           

 r = [ x pix y pix ]T = [ p q]
T
 

 X  n

 r 
n

X 
 α n rn = M  n   
 1 

ρ3 to 
  ρ2        





 

 
               
                
               

              X = [ X Y Z ]   T

 r = [ p q]  m 


T
ij


X 
α p   m11 m12 m13 m14   
 α q  =  m m22 m23
 Y
m24     
   21 Z 
 α   m31 m32 m33 m34   
1

α p   m11 X + m12Y + m13 Z + m14 


 α q  = m X + m Y + m Z + m   
   21 22 23 24 

 α   m31 X + m32Y + m33 Z + m34 

 α  ( m 31 X + m32Y + m33 Z + m34 )  [α p αq α ] 


p ( m31 X + m32Y + m33 Z + m34 ) = m11 X + m12Y + m13 Z + m14
  
q ( m31 X + m32Y + m33 Z + m34 ) = m21 X + m22Y + m23 Z + m24

 m ,....., m 


11 34


m 
X Y Z 1 0 0 0 0 − pX − pY − pZ − p   11 
 
0 0 0 0 X Y Z 1 − qX −qY −qZ
m
− q   12  = 0  
  
    
 m34 

 AM = 0  
               




 


               

 Amn = U mm DmnVnnT  




 M = [K R | − K R t ] = [ A | b ]  
                  
 A = KR  b = −KRt 
 t = − A−1b  









 


                     



 ( X , Y , Z ) 








   
    


   
    


   
    


   
    



 






              



 
 










 

 



                

                



 k  k 
1 2

 k 
3

  





 




              


       p  p      
1 2

 ( x pix , y pix ) 


 ( X , Y , Z ) 
S S S

x pix = k x xi + sy i + x0
 XS Y  
= kx f + sf S + x0
ZS ZS

y pix = k y yi + y0
 YS  
= ky f + y0
ZS

 ( x , y )  f  k  k 


0 0 x y

 s ( x , y )  ( x , y ) 


d d pix pix

r  ( x , y ) 
d d

 r= (xd − x0 ) + ( yd − y0 )
2 2
 





 

 x = xd − x0 ; y = yd − y0  

 (
cx = x k1r 2 + k2 r 4 + k3r 6 ) 

 (
cy = y k1r 2 + k2r 4 + k3r 6 ) 


 (
d x = 2 p1 x y + p2 r 2 + 2 x
2
) 

 ( )
d y = p1 r 2 + 2 y + 2 p2 x y 
2


 ( x pix , y pix ) 

 x pix = x d + ( c x + d x )  

 y pix = y d + ( c y + d y )  




( )
x pix = xd + x k1r 2 + k2 r 4 + k3 r 6 + 2 p1 x y + p2 r 2 + 2 x ( 2
) 
( ) (
y pix = yd + y k1r 2 + k2 r 4 + k3 r 6 + p1 r 2 + 2 y + 2 p2 x y
2
)

 r 2 + xk
x pix = xd + xk1
 r 4 + xk
2
 r 6 + 2 p x y + p r 2 + 2 x 2
3 1 2 ( ) 
(
y pix = yd + yk1r 2 + yk2 r 4 + yk3 r 6 + p1 r 2 + 2 y + 2 p2 x y
2
)



(x pix − xd ) − xk
 r 2 − xk
1
 r 4 − xk
2
 r 6 − 2 p x y − p r 2 + 2 x 2 = 0
3 1 2 ( )  
(
( y pix − yd ) − yk1r 2 − yk2r 4 − yk3r 6 − p1 r 2 + 2 y − 2 p2 x y = 0
2
)
 AX = 0  X 




 


 A 
 X 
1
 ( pix d )
 x −x  2
− xr  4
− xr  6
− xr −2 x y ( )
− r 2 + 2 x   k1 
2

 
   k2 
 ( y pix − yd ) − yr 2

− yr 4 − yr 6 (
− r 2 + 2 y
2
) −2 x y    = 0 
  k3 

   p 
   
1

 p2 

 AX = 0  

 Amn = Umm DmnVnnT  



 ( x pix , y pix ) 

                


 


             


            





 

             

              

                






 C  C 
1 2

 W 
        R '   T '     
 R ''  T '' 
   u '  u ''            
 X 



 

              

            m ' , m ' , m '   
1
T
2
T
3
T

 m '' , m '' , m '' 


1
T
2
T
3
T


 X '
 u '   m '11 m '12 m '13 m '14     m '1T 
    
u ' =  v '  =  m '21 m '24    = M ' X =  m '2T  X
Y'
m '22 m '23
 Z '
 w '  m '31 m '32 m '33 m '34     m '3T 
 
 
1


 X ''
 u ''   m ''11 m ''12 m ''13 m ''14     m ''1T 
 u '' =  v ''  =  m ''    
m ''24    = M '' X =  m ''2T  X
Y ''
   21 m ''22 m ''23
 Z '' 
 w ''  m ''31 m ''32 m ''33 m ''34     m ''3T 
 
 1

 u '  u ''             
 X          


u ' : v ' : w ' = m '1T X : m '2T X : m '3T X
  
u '': v '': w '' = m ''1T X : m ''2T X : m ''3T X


u ' m '2T X = v ' m '1T X u '' m ''2T X = v '' m ''1T X
 u ' m '3T X = w ' m '1T X u '' m ''3T X = w '' m ''1T X  
v ' m ' X = w' m ' X
3
T
2
T
v '' m '' X = w '' m '' X
3
T
2
T


 u ' m '2T − v ' m '1T   u '' m ''2T − v '' m '1T 
 T  T
 u ' m '3 − w ' m '1  X = 0
T
u '' m ''3 − w '' m ''1  X = 0 
T

v ' m '3T − w ' m '2T  v '' m ''3T − w '' m ''2T 
   



 

 w'  w'' 
 −v'  −v' 


(u ' w ' m ' − v ' w ' m ' − u ' v ' m ' + v ' w ' m ' ) X = (u ' w ' m ' − u ' v ' m ' ) X = 0
2
T
1
T
3
T
1
T
2
T

 3
T


( u '' w '' m '' − v '' w '' m '' − u '' v '' m '' + v '' w '' m '' ) X = ( u '' w '' m '' − u '' v '' m '' ) X = 0
2
T
1
T
3
T
1
T
2
T
3
T


 ( −w ' m '2
T
)
+ v ' m '3T X = 0 ( −w '' m '' 2
T
+ v '' m ''3T X = 0  ) 





( u ' w ' m ' − u ' v ' m ' ) X = −u ' ( − w ' m ' + v ' m ' ) X = 0 
2
T
3
T
2
T
3
T


( u '' w '' m '' − u '' v '' m '' ) X = −u '' ( − w '' m '' + v '' m '' ) X = 0
2
T
3
T
2
T
3
T




(u 'm ' 3
T
)
− w ' m '1T X = 0 ( u '' m '' 3
T
− w '' m ''1T X = 0 )  
(v 'm ' 3
T
− w ' m '2 T )X =0 ( v '' m '' 3
T
− w '' m ''2T )X =0

 u ' m '3T − w ' m '1T 
 T 
 v ' m '3 − w ' m '2  X = AX = 0 
T

  u '' m ''3T − w '' m ''1T 



 
v '' m ''3 − w '' m ''2 
T T

 ( X ≠ 0 ) 
 det( A) = 0 
 Amn = U mm DmnVnnT  





 






 u ' T3 −w ' T1 


 
 v '  3
T
− w ' T2 
 u " T3 −w " T1 
   =  = 0
  v " 3
T
− w " T2   
u ''' T −w ''' 1T 

 3

v '''  3
T
− w ''' T2 

 



 

MATLAB IMPLEMENTATION





             

%---------------------------------------------%
% Matlab implementation of 3D Reconstruction %
% Written by: Chuchart Pintavirooj and %
% Sorapong Aoothaphao %
% Modified by: Sunu Wibirama %
% Please use for educational purpose only %
%---------------------------------------------%

%read input
clear

for j=1:3

switch (j)

case 1

a=imread('pic1.jpg','jpg');

aa=a(:,:,1);
figure(1);imagesc(aa);colormap('gray');

case 2

a=imread('pic2.jpg','jpg');
aa=a(:,:,1);

figure(2);imagesc(aa);colormap('gray');

case 3

a=imread('pic3.jpg','jpg');

aa=a(:,:,1);

figure(3);imagesc(aa);colormap('gray');
end

hold on;



 

%define 2D coordinate of each point
for i=1:8

[u(i,1),u(i,2)]=ginput(1);

plot(u(i,1),u(i,2),'r+');
end

% define world coordinate of the box

xx=[0 0 0 1

10.5 0 0 1

10.5 10.5 0 1

0 10.5 0 1
0 0 10.5 1

10.5 0 10.5 1

10.5 10.5 10.5 1


0 10.5 10.5 1];

%compute matrix M

for i=1:8

G(2*i,1)=0;

G(2*i,2)=0;

G(2*i,3)=0;
G(2*i,4)=0;

G(2*i,5)=xx(i,1);

G(2*i,6)=xx(i,2);
G(2*i,7)=xx(i,3);

G(2*i,8)=1;

G(2*i,9)=-u(i,2)*xx(i,1);

G(2*i,10)=-u(i,2)*xx(i,2);

G(2*i,11)=-u(i,2)*xx(i,3);

G(2*i,12)=-u(i,2);

G((2*i)-1,1)=xx(i,1);

G((2*i)-1,2)=xx(i,2);
G((2*i)-1,3)=xx(i,3);



 

G((2*i)-1,4)=1;
G((2*i)-1,5)=0;

G((2*i)-1,6)=0;

G((2*i)-1,7)=0;
G((2*i)-1,8)=0;

G((2*i)-1,9)=-u(i,1)*xx(i,1);

G((2*i)-1,10)=-u(i,1)*xx(i,2);

G((2*i)-1,11)=-u(i,1)*xx(i,3);

G((2*i)-1,12)=-u(i,1);

end
[U, D, V]=svd(G);

VT=V;

M=VT(:,end); %last columm


%M = transpose(reshape(M_vector, 4, 3));

m(1,1)=M(1);

m(1,2)=M(2);

m(1,3)=M(3);

m(1,4)=M(4);

m(2,1)=M(5);
m(2,2)=M(6);

m(2,3)=M(7);

m(2,4)=M(8);
m(3,1)=M(9);

m(3,2)=M(10);

m(3,3)=M(11);

m(3,4)=M(12);

switch (j)

case 1
u1=u;

m1=m;

case 2
u2=u;



 

m2=m;
case 3

u3=u;

m3=m;
end

vv=m*xx';

vv=vv';

vv(:,1)=vv(:,1)./vv(:,3);

vv(:,2)=vv(:,2)./vv(:,3);

title('Re project')
for i=1:8

plot(vv(i,1),vv(i,2),'go');

end
end%for j=1:3

size(u);m=ans(1);

%3D Reconstruction using Direct Linear Transformation

for i=1:8

A(1,1:4)=u1(i,1)*m1(3,1:4)-m1(1,1:4);

A(2,1:4)=u1(i,2)*m1(3,1:4)-m1(2,1:4);
A(3,1:4)=u2(i,1)*m2(3,1:4)-m2(1,1:4);

A(4,1:4)=u2(i,2)*m2(3,1:4)-m2(2,1:4);

A(5,1:4)=u3(i,1)*m3(3,1:4)-m3(1,1:4);
A(6,1:4)=u3(i,2)*m3(3,1:4)-m3(2,1:4);

[U, D, V]=svd(A);

zz(:,i)=V(:,4);

end

ake=zz;

zz=zz';
zz(:,1)=zz(:,1)./zz(:,4);

zz(:,2)=zz(:,2)./zz(:,4);

zz(:,3)=zz(:,3)./zz(:,4);
% Plot image



 

figure(4);
cla reset; hold on

d = [1 2 3 4 1 5 6 7 8 5 6 2 3 7 8 4];

plot3(zz(d,1),zz(d,2),zz(d,3),'b:');
plot3(zz(:,1),zz(:,2),zz(:,3),'b.','markersize',20)

A=m1(:,2:4);

%scaling = sqrt(m1(3,2)^2 + m1(3,3)^2 + m1(3,4)^2);

%m1 = m1 ./ scaling;

A=m1(:,2:4);

[Q R]=qr(A^-1);
R1=R^-1;

R1=R1/R1(3,3)

A=m2(:,2:4);

%scaling = sqrt(m2(3,2)^2 + m2(3,3)^2 + m2(3,4)^2);

%m2 = m2 ./ scaling;

A=m2(:,2:4);

[Q R]=qr(A^-1);

R2=R^-1;
R2=R2/R2(3,3)

A=m3(:,1:3);

%scaling = sqrt(m3(3,2)^2 + m3(3,3)^2 + m3(3,4)^2);


%m3 = m3 ./ scaling;

A=m3(:,2:4);

[Q R]=qr(A^-1);

R3=R^-1;

R3=R3/R3(3,3)





 



 
 

 
 






 



 

 

 

 

 

 


 

 

 

 

 

 

 

 



 

 



 


 



 


 

 


 




S-ar putea să vă placă și