Mathematical Foundations of

Photogrammetry,
Metrology
– The Second Part of Photogrammetry

Monday, September 09, 2019 Close Range Photogrammetry, Computer Vision and 3D Modeling 97
Analytical Photogrammetry

 Based on camera parameters, measured photo

coordinates and ground control

 Accounts for any tilts that exist in photos

 Solves complex systems of redundant equations by

implementing least squares method

Monday, March 24, 2008 EE 5358 Computer Vision 98

Review
L y
Collinearity Condition Z
f xa

a
Tilted photo ya
plane x
O
The exposure station
of a photograph, an object Y

ZL
point and its photo image A
Za

all lie along a straight Xa

line. XL
Ya

YL

Monday, March 24, 2008 EE 5358 Computer Vision 99

Image Coordinate System
z’
y’
• Ground Coordinate System - X, Y, Z L
Z x’
za’
wrt Ground Coordinate System
ya’
• Exposure Station Coordinates – XL, YL, ZL Xa’

• Object Point (A) Coordinates – Xa, Ya, Za

• Rotated coordinate system parallel to ground

ZL
coordinate system (XYZ) – x’, y’, z’
Y
A
wrt Rotated Coordinate System Za
Xa
 Rotated image coordinates – xa’, ya’, za’

 xa’ , ya’ and za’ are related to the measured

Ya
photo coordinates xa, ya, focal length (f) and the XL
three rotation angles omega, phi and kappa. YL

Monday, March 24, 2008 EE 5358 Computer Vision 100

Rotation Formulas
 Developed in a sequence of three independent two-dimensional rotations.
• ω rotation about x’ axis
x1 = x’
y1 = y’Cosω + z’Sinω
z1 = -y’Sinω + z’Cosω
• φ rotation about y’ axis
x2 = -z1Sinφ + x1Cosφ
y2 = y1
z2 = z1Cosφ + x1Sinφ
• κ rotation about z’ axis
x = x2Cosκ + y2Sin κ
y = -x2Sinκ + y2Cosκ
z = z2

Monday, March 24, 2008 EE 5358 Computer Vision 101

 Rotation Matrix
x = x’(CosφCosκ) + y’(SinωSinφCosκ + CosωSinκ) + z’(-CosωSinφCosκ + SinωSinκ)
y = x’(-CosφSinκ) + y’(-SinωSinφSinκ + CosωCosκ) + z’(CosωSinφSinκ + SinωCosκ )
z = x’(Sinφ) + y’(-SinωCosφ) + z’(CosωCosφ)

X = MX’ x = m11x’ + m12y’ + m13z’

 x  m11 m12 m13   x'  Rotation Matrix y = m21x’ + m22y’ + m23z’
 y   m 21 m 22 m 23  y ' z = m31x’ + m32y’ + m33z’
    
 z  m31 m32 m33   z ' 

The sum of the squares of the three “direction cosines” in any row or in any
column is unity.
M -1 = MT
X’ = MTX

Monday, March 24, 2008 EE 5358 Computer Vision 102

 Rotation Matrix
 Matrix Rotasi
 x   r11 r12 r13   X 
 y   r r r Y 
   21 22 23   
 z   r31 r32 r33   Z 
 Uraikan setiap element Matrix Rotasi tersebut!
z z z z x x
 A  A 
y A


o o x y
y  o 
Tahap 1: Rotasi  thd sumbu x x  y 
Tahap 2: Rotasi  thd sumbu y
Tahap 3: Rotasi  thd sumbu z
3D Similarity Transformation
z A
 y
Z x

XA x


X0
Y

  1 t  X
XA  X0   R x

X A X0  r11 r21 r31   x 

 Y    Y   1  r r22 r32   y 
 A  0  11
 Z A   Z 0  r13 r23 r33   z 
Collinearity Condition Equations
y
a x
r y

p xa x Sistem Koordinat Citra
z
c O P

Bidang Proyeksi Z  S
X0

XA
Y A (XA,YA,ZA)

Sistem
Koordinat
Ruang (Object) X
  
XA  X0  S  X A   X 0   r11 r21 r31   xa 
  t
 Y    Y    r r22  
r32   ya  
X A  X 0  R x a  A   0   12
 Z A   Z 0   r13 r23 r33   c 
Collinearity Condition Equations
 Reverse
 Transformation
 
x a   1R (X 0  X A )
 xa   r11 r12 r13   X 0  X A   1
 y    1 r r22 r23   Y0  YA 
 a  21
 c   r31 r32 r33   Z 0  Z A 

 Persamaan baris ke-3 bisa diubah secara eksplisit

dalam
 Substitusikan ke persamaan dalam baris pertama dan
kedua
Collinearity Condition Equations
 c   1r31( X 0  X A )   1r32 ( Y0  YA )   1r33 ( Z 0  Z A )
1 c
 
r31( X 0  X A )  r32 ( Y0  YA )  r33 ( Z 0  Z A )

 c[r11 ( X 0  X A )  r12 (Y0  YA )  r13 ( Z 0  Z A )]

xa 
[r31 ( X 0  X A )  r32 (Y0  YA )  r33 ( Z 0  Z A )]

 c [ r21( X 0  X A )  r22 ( Y0  YA )  r23 ( Z 0  Z A )]

ya 
[ r31( X 0  X A )  r32 ( Y0  YA )  r33 ( Z 0  Z A )]
 Collinearity Condition Equations
Collinearity condition equations developed from similar triangles
x'a y 'a  z 'a x = m11x’ + m12y’ + m13z’
 
XA  XL YA  YL ZL  ZA y = m21x’ + m22y’ + m23z’
z = m31x’ + m32y’ + m33z’

XA XL 
  YA YL 
  
 ZA  ZL 

xa  m 11 z ' a  m12  z ' a  m13  z' a
ZA ZL 


 ZA  ZL 
 

 Z A  Z L

 * Dividing xa and ya by za

 XA XL   YA YL 
  
 Z A  ZL 

* Substitute –f for za
ya  m 21 z ' a  m 22  z ' a  m 23  z' a

 ZA ZL   ZA  ZL 
 

 Z A  Z L

 * Correcting the offset of Principal

 XA XL   YA YL 
   
 ZA  Z L  point (xo, yo)
za  m 31 z ' a  m32  z ' a  m 33  z' a


 ZA ZL   ZA  ZL 
 
 ZA  Z L 




 m11( XA XL ) m12(YA  YL ) m13( ZA  ZL ) 
x a  xo  f  
 m31( XA  XL )  m32(YA YL )  m33( ZA ZL ) 
 m 21( XA XL)  m 22(YA  YL)  m 23( ZA  ZL) 
y a  yo  f  
 m31( XA  XL )  m32(YA YL )  m33( ZA  ZL ) 

Monday, March 24, 2008 EE 5358 Computer Vision 108

 Collinearity Equations
• Nonlinear  m11( XA  XL)  m12(YA  YL )  m13( ZA  ZL ) 
x a  xo  f  
• Nine unknowns  m31( XA  XL )  m32(YA YL ) m33( ZA  ZL ) 
• ω, φ, κ  m 21( XA  XL )  m 22(YA  YL )  m 23( ZA  ZL ) 
y a  yo  f  
 m31( XA  XL )  m32(YA YL )  m33( ZA  ZL ) 
• XA, YA and ZA
• XL, YL and ZL
Taylor’s Theorem is used to linearize the nonlinear
equations
 substituting
f ( n ) (a)

n0 n!
( x a) n q  m31 ( X A  X L )  m32 (YA  YL )  m33 ( Z A  Z L )
r  m11 ( X A  X L )  m12 (YA  YL )  m13 ( Z A  Z L )
s  m21 ( X A  X L )  m22 (YA  YL )  m23 ( Z A  Z L )

Monday, March 24, 2008 EE 5358 Computer Vision 109

Linearizing Collinearity Equations
Rewriting the Collinearity Equations
r  F 
Fo 
 F 
 d 
 F 
 d 
 F 
 d 
 F 
 dXL 
 F 
 dYL   dZL
F  xo  f  xa    0    0    0  XL  0  YL  0  ZL  0
q Taylor’s Theorem  F 

 F 
 dXA 
 F 
 dYA  dZA  xa
 XA  0  YA  0  ZA  0
s  G   G   G   G   G   G 
G  yo  f  ya Go   d 
   0
 d 
   0
 d 
   0
 dXL 
 XL  0
 dYL 
 YL  0
 dZL
 ZL  0
q  G   G   G 
  dXA   dYA  dZA  ya
 XA  0  YA  0  ZA  0

• F0 and G0 are functions F and G evaluated at the initial

approximations for the nine unknowns
• dω, dφ, dκ are the unknown corrections to be applied to the
initial approximations
• The rest of the terms are the partial derivatives of F and G wrt
to their respective unknowns at the initial approximations
Monday, March 24, 2008 EE 5358 Computer Vision 110
Applying LLSM to Collinearity Equations
• Residual terms must be included in order to make the
equations consistent
b11d  b12 d  b13d  b14 dX L  b15 dYL  b16 dZ L  b14 dX A  b15 dYA  b16 dZ A  J  Vxa
b21d  b22 d  b23 d  b24 dX L  b25 dYL  b26 dZ L  b24 dX A  b25 dYA  b26 dZ A  K  V ya

J = xa – Fo ; K = ya – Go
b terms are coefficients equal to the partial derivatives
Numerical values for these coefficient terms are obtained by using
initial approximations for the unknowns.
The terms must be solved iteratively (computed corrections are added
to the initial approximations to obtain revised approximations) until
the magnitudes of corrections to initial approximations become
negligible.

Monday, March 24, 2008 EE 5358 Computer Vision 111

Analytical Stereomodel

 Mathematical calculation of three-dimensional ground

coordinates of points in the stereomodel by analytical
photogrammetric techniques

 Three steps involved in forming an Analytical

Stereomodel:
• Interior Orientation
• Relative Orientation
• Absolute Orientation

Monday, March 24, 2008 EE 5358 Computer Vision 112

Analytical Interior Orientation
 Requires camera calibration information and
quantification of the effects of atmospheric refraction.
 2D coordinate transformation is used to relate the
comparator coordinates to the fiducial coordinate system
to correct film distortion.
 Lens distortion and principal-point information from
camera calibration are used to refine the coordinates so
that they are correctly related to the principal point and
free from lens distortion.
 Atmospheric refraction corrections are applied.

Monday, March 24, 2008 EE 5358 Computer Vision 113

Analytical Relative Orientation
 Process of determining the elements of exterior
orientation
 Fix the exterior orientation elements of the left photo of
the stereopair to zero values
 Common method in use to find these elements is through
Space Resection by Collinearity(see slide below)
 Each object point in the stereomodel contributes 4
equations
 5 unknown orientation elements + 3 unknowns(X, Y & Z)

Monday, March 24, 2008 EE 5358 Computer Vision 114

 Space Resection by Collinearity
 Formulate the collinearity equations for a number of control points whose X,
Y and Z ground coordinates are known and whose images appear in the
tilted photo. The equations are then solved for the six unknown elements of
exterior orientation which appear in them.
Space Resection collinearity equations for a point A
b11d b12 d  b13 d b14 dX L  b15 dYL  b16 dZ L  J  Vxa
b21d b22 d b23 d  b24 dX L  b25 dYL  b26 dZ L  K  V ya
A two dimensional conformal coordinate transformation is used
X = ax’ – by’ + Tx X, Y – ground control coordinates for the point
Y = ay’ + bx’ + Ty x’, y’ – ground coordinates from a vertical photograph

a, b, Tx, Ty – transformation parameters

Monday, March 24, 2008 EE 5358 Computer Vision 115

Analytical Absolute Orientation

 Utilizes a 3D conformal coordinate transformation

 Requires a min. of 2 horizontal and 3 vertical control
points
 Stereomodel coordinates of control points are related to
their 3D coordinates in a cartesian coordinate system
 Coordinates of all stereomodel points in the ground
system can be computed by applying the transformation
parameters

Monday, March 24, 2008 EE 5358 Computer Vision 116

Bundle Adjustment

Adjust all photogrammetric measurements to ground

control values in a single solution

Unknown quantities
• X, Y and Z object space coordinates of all object points
• Exterior orientation parameters of all photographs
Measurements
• x and y photo coordinates of images of object points
• X, Y and/or Z coordinates of ground control points
• Direct observations of the exterior orientation parameters of the photographs

Monday, March 24, 2008 EE 5358 Computer Vision 117

Bundle Adjustment-Observations
 Photo Coordinates - Fundamental Photogrammetric Measurements made with a
comparator or analytical plotter. According to accuracy and precision the coordinates
are weighed
 Control Points – determined through field survey
 Exterior Orientation Parameters – especially helpful in understanding the angular
attitude of a photograph
Regardless of whether exterior orientation parameters were observed, a least squares
solution is possible since the number of observations is always greater than the
number of unknowns.
 m11i ( X j  X Li )  m12i (Y j  YLi )  m13i ( Z j  Z Li )  xij, yij – measured photo coordinates of the image
xij  xo  f  
 m31i ( X j  X Li )  m32i (Y j  YLi )  m33i ( Z j  Z Li )  of point j on photo i related to fiducial axis system

 m ( X  X Li )  m22i (Y j  YLi )  m23i ( Z j  Z Li )  xo, yo – coordinates of principal points in fiducial axis

yij  yo  f  21i j 
 m31i ( X j  X Li )  m32i (Y j  YLi )  m33i ( Z j  Z Li )  system

f - focal length/principal distance Xj, Yj, Zj – coordinates of point j in object space

m11i, m12i, …….,m33i – rotation parameters for photo i XLi, YLi, ZLi – coordinates of incident nodal point
of camera lens in object space

Monday, March 24, 2008 EE 5358 Computer Vision 118

Bundle Adjustment - Weights
 Photo coordinates 1
 1
 2 0 

 2
x y 
σ02 – reference variance Wij   o2 
xij

ij ij
Wij   ij
x

 y x  y2   0 1 
σxij2 , σyij2 – variances in xij and yij resp. ij ij ij
  y2ij 

σxijyij = σyijxij – covariance of xij and yij
• Ground Control coordinates
1
• σXj2, σYj2, σZj2 – variances in Xj00, Yj00, Zj00 resp.   X2 X Y X Z 
2 
j j j j j
..
• σXjYj = σYjXj – covariance of Xj00 with Yj00 W   0  Y j X j  Y2 Y Z
j j j 
• σXjZj = σZjXj – covariance of Xj00 with Zj00  Z Y  Z2 
 ZjX j j j j 
• σYjZj = σZjYj – covariance of Yj00 with Zj00
1
  2i     X  Y  Z 
• Exterior Orientation Parameters 
  ii
i i

 2
i i

 
i

 X
Li i Li

 Y  Z
i Li

i i i i Li i Li i Li 

.       2  X  Y  Z 
Wi   i i i i i i Li i Li i Li

 X L ii X 
Li i
X 
Li i
 X2 Li
X Y
Li Li
X ZLi Li 
 Y  Y  Y X  Y2 Y Z 
 YLii Li i Li i Li Li Li Li Li

 Z L ii Z 
Li i
Z 
Li i
Z X
Li Li
Z Y
Li Li
 Z2 Li


Monday, March 24, 2008 EE 5358 Computer Vision 119

Direct Linear Transformation (DLT)
 This method does not require fiducial marks and can be solved without
supplying initial approximations for the parameters
Collinearity equations along with the correction for lens distortion
 m11 ( X A  X L )  m12 (YA  YL )  m13 ( Z A  Z L )  δx, δy – lens distortion
x a   x  x0  f x  
 m31 ( X A  X L )  m32 (YA  YL )  m33 ( Z A  Z L )  fx – pd in the x direction
 m ( X  X L )  m22 (YA  YL )  m23 ( Z A  Z L ) 
ya   y  x0  f y  21 A  fy – pd in the y direction
 m31 ( X A  X L )  m32 (YA  YL )  m33 ( Z A  Z L ) 
L1  ( x0 m31  f x m11 ) / L
Rearranging the above two equations L2  ( x0 m32  f x m12 ) / L
L3  ( x0 m33  f x m13 ) / L
L X  L2Y  L3 Z  L4
xa   x  1 L4  x0  f x (m11 X L  m12YL  m13 Z L ) / L
L9 X  L10Y  L11Z  1 L5  ( y0 m31  f y m21 ) / L
L5 X  L6Y  L7 Z  L8 L6  ( y0 m32  f y m22 ) / L
ya   y 
L9 X  L10Y  L11Z  1 L7  ( y0 m33  f y m23 ) / L

Monday, March 24, 2008 EE 5358 Computer Vision 120

DLT(contd.)
The resulting equations are solved
L8  y0  f y (m21 X L  m22YL  m23 Z L ) / L
iteratively using LSM
L9  m31 / L
1
 X L   L1 L2 L3   L4  L10  m32 / L
 Y   L L6 L7  L  L11  m33 / L
 L  5  8 L  (m31 X L  m32YL  m33 Z L )
 Z L   L9 L10 L11   1 

 Advantages
- No initial approximations are required
for the unknowns.
 Limitations
 - Requirement of atleast six 3D object space control points
- Lower accuracy of the solution as compared with a rigorous bundle
adjustment

Monday, March 24, 2008 EE 5358 Computer Vision 121

Kalibrasi Kamera
 Parameter Internal Orientasi Kamera
 Principal Distance c (Jarak Utama Foto)
 Principal Point Offset , x0 dan y0 (Vektor pergeseran
Principal Point/ Titik Utama)
Kalibrasi Kamera
 The pixel coordinate system origin is at the centre of the pixel at the top
left of the sensor with the positive X-axis directed towards the right and
the positive Y-axis directed downwards.
 The image coordinate system is centered on the sensor with the
positive X-axis again directed towards the right and the positive Y-axis
directed up.
the solid outer border
represents the physical
sensor extents, while the
dashed interior border is
meant as an aid in
visualizing the position of
the pixel coordinate
system.
Kalibrasi

Kamera
The location of an artificial target or natural feature measured either
manually or by the centered technique is recorded to sub pixel accuracy
in the pixel coordinate system as shown in previous Figure.
 A half pixel is subtracted from both x and y to account for the fact that
the pixel coordinate system is centered at the top left pixel. Before use
in any adjustment, the measured coordinates are then converted to the
image coordinate system as follows.

The centre of the sensor in the pixel coordinate system is

determined by

The measured image coordinates are then

Kalibrasi Kamera
 c k [r11k ( X 0k  X i )  r12k (Y0k  Yi )  r13k ( Z 0k  Z i )] U
xi  x 0    c
[r31k ( X 0k  X i )  r32k (Y0k  Yi )  r33k ( Z 0k  Z i )] W

 c k [r21k ( X 0k  X i )  r22k (Y0k  Yi )  r23k ( Z 0k  Z i )] V

yi  y0    c
[r31k ( X 0k  X i )  r32k (Y0k  Yi )  r33k ( Z 0k  Z i )] W

U V
x  x 0  c  x y  y 0  c  y
W W
x   x0  x r 2 K1  x r 4 K 2  x r 6 K 3  (2 x 2  r 2 ) P1  2 P2 x y
y   y 0  yr 2 K 1  yr 4 K 2  yr 6 K 3  (2 y 2  r 2 ) P2  2 P1 x y
y  y  y0
Analytical Self Calibration
The equations take into account adjustment of the
calibrated focal length, principal-point offsets and
symmetric radial and decentering lens distortion.
r
xa  x0  xa (k1ra2  k 2 ra4  k3 ra6 )  (1  p32 ra2 )[ p1 (3xa2  ya2 )  2 p2 xa ya ]  f
q
s
ya  y0  ya (k1ra2  k 2 ra4  k3 ra6 )  (1  p32 ra2 )[ p2 ( xa2  3 ya2 )  2 p1 xa ya ]  f
q
xa, ya – measured photo coordinates related to fiducials
xo, yo – coordinates of the principal point
xa = xa – xo where q  m31 ( X A  X L )  m32 (YA  YL )  m33 ( Z A  Z L )
ya = ya - yo r  m11 ( X A  X L )  m12 (YA  YL )  m13 ( Z A  Z L )
s  m21 ( X A  X L )  m22 (YA  YL )  m23 ( Z A  Z L )

Monday, March 24, 2008 EE 5358 Computer Vision 126

Analytical Self Calibration(contd.)
ra2  xa2  ya2
k1, k2, k3 = symmetric radial lens distortion coefficients
p1, p2, p3 = decentering distortion coefficients
f = calibrated focal length
r, s, q = collinearity equation terms
Provides a calibration of the camera under original conditions which existed
when the photographs were taken.
Geometric Requirements
- Numerous redundant photographs from multiple locations are required, with
sufficient roll diversity
- Many well-distributed image points be measured over the entire format to
determine lens distortion parameters
The numerical stability of analytical self calibration is of serious concern.

Monday, March 24, 2008 EE 5358 Computer Vision 127

Applications
 Automobile Construction
 Machine Construction, Metalworking, Quality Control
 Mining Engineering
 Objects in Motion
 Shipbuilding
 Structures and Buildings
 Traffic Engineering
 Biostereometrics

Monday, March 24, 2008 EE 5358 Computer Vision 128

Biomedical Applications
 Linear tape and caliper measurements of inherently
irregular three-dimensional biological structures are
inadequate for many purposes.
 Subtle movements produced by breathing, pulsation of
blood, and reflex correction for control of postural
stability.
 Short patient involvement times, avoids contact with the
patient and thereby avoiding risk of deforming the area of
interest and spreading infection.
 All medical photogrammetric measurements require
further interpretation and analysis to allow meaningful
information to be given to the end-user.
Monday, March 24, 2008 EE 5358 Computer Vision 129
Bibliography
• Kamara, H.M. (1979). Handbook of Non-Topographic
Photogrammetry, American Society of Photogrammetry.
• Wolf , Paul R., Dewitt, Bon A. (2000). Elements of
Photogrammetry, McGraw Hill.
• Devarajan, Venkat and Chauhan, Kriti (Spring 2008). Lecture
Notes: Mathematical Foundation of Photogrammetry, EE 5358
University of Texas at Arlington.
• Karara, H.M. (1989). Non-Topographic Photogrammetry,
American Society for Photogrammetry and Remote Sensing.
• Mitchell, H.L. and Newton, I. (2002). Medical photogrammetric
measurement: overview and prospects. ISPRS Journal of
Photogrammetry & Remote Sensing, 56, 286-294.
Monday, March 24, 2008 EE 5358 Computer Vision 130
Acknowledgments

 Dr. Venkat Devarajan

 Kriti Chauhan

Monday, March 24, 2008 EE 5358 Computer Vision 131