ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246

Contents lists available at ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing

journal homepage: www.elsevier.com/locate/isprsjprs

Aerial multi-camera systems: Accuracy and block triangulation issues

Ewelina Rupnik ⇑, Francesco Nex, Isabella Toschi, Fabio Remondino
3D Optical Metrology Unit (3DOM), Bruno Kessler Foundation (FBK), Trento, Italy

a r t i c l e i n f o a b s t r a c t

Article history: Oblique photography has reached its maturity and has now been adopted for several applications. The
Received 28 August 2014 number and variety of multi-camera oblique platforms available on the market is continuously growing.
Received in revised form 22 December 2014 So far, few attempts have been made to study the influence of the additional cameras on the behaviour of
Accepted 23 December 2014
the image block and comprehensive revisions to existing flight patterns are yet to be formulated. This
Available online 16 January 2015
paper looks into the precision and accuracy of 3D points triangulated from diverse multi-camera oblique
platforms. Its coverage is divided into simulated and real case studies. Within the simulations, different
Keywords:
imaging platform parameters and flight patterns are varied, reflecting both current market offerings and
Oblique images
Simulation
common flight practices. Attention is paid to the aspect of completeness in terms of dense matching algo-
Flight planning rithms and 3D city modelling – the most promising application of such systems. The experimental part
Orientation demonstrates the behaviour of two oblique imaging platforms in real-world conditions. A number of
Bundle adjustment Ground Control Point (GCP) configurations are adopted in order to point out the sensitivity of tested
Accuracy imaging networks and arising block deformations. To stress the contribution of slanted views, all scenar-
Automation ios are compared against a scenario in which exclusively nadir images are used for evaluation.
Ó 2014 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier
B.V. All rights reserved.

1. Introduction Thanks to the rapid progress in military photography from air-

Oblique aerial images, alone or in combination with vertical
aerial images, have always been seen as a very attractive data
source thanks to their intuitive nature. Documented use of oblique
imagery dates back to the nineteenth century. Gaspard Félix Tour-
nachon acquired the first aerial photograph in 1858 by raising a
balloon over the village of Petit Bicêtre in France (Colwell, 1997).
Two years later, the oldest surviving oblique aerial image was
taken by J.W. Black and S. King in Boston, USA, from a balloon. In
1887 a balloon again was employed by a German forester to
acquire aerial photos over forests in order to identify and measure
stands of trees (Colwell, 1997). The same year also saw the birth of
aerial photography from a kite, thanks to a British meteorologist,
E.D. Archibald (Colwell, 1997). At about the same time in France,
the Tissandier brothers conducted other experiments of kite and
balloon aerial photography. Thereafter this practice of acquiring
images from kites moved across the Atlantic and advanced rapidly.
In 1906 G.R. Lawrence, using between nine and seventeen large
kites to lift a huge camera, took some oblique aerial photographs
of San Francisco (USA) after an enormous earthquake in the area.
⇑ Corresponding author.
E-mail addresses: rupnik@fbk.eu (E. Rupnik), franex@fbk.eu (F. Nex), toschi@fbk.eu
(I. Toschi), remondino@fbk.eu (F. Remondino).
planes, kites were then abandoned in favour of powered flight,
which gained importance for military reconnaissance during
World War I. Simultaneously, single and multiple lens cameras
were produced for oblique-only or combined configurations in
the UK, Germany, France, Italy, Switzerland and USA (Manual of
Photogrammetry, 1952). In the 1930s for instance, the U.S. Geolog-
ical Survey and the U.S. Army used a Fairchild T-3A five-lens film
camera for mapping, surveillance and reconnaissance purposes.
Finally, during World War II, oblique aerial photography was
employed for reconnaissance purposes before and after bombing
missions (Nocerino et al., 2012), thus further stimulating the rapid
development of new cameras, lenses, films and camera mounting
systems (Aber et al., 2010).
The concept of fitting several imaging sensors into a unique
camera housing re-emerged for civil applications with the intro-
duction of digital imaging technology (Petrie, 2010). However,
the true revival of oblique imaging systems for geospatial applica-
tions occurred in 2000 when Pictometry International introduced
their five-lens camera system that incorporated vertical and slant
views (Patent Ser. No. 60/425,275, filed November 8, 2002). Pic-
tometry imagery has also, more recently, been disseminated
through the Bird’s Eye view function of current Bing Maps. Over
the past decade, the geospatial industry has enthusiastically taken
up oblique imaging technology. New businesses have been born

http://dx.doi.org/10.1016/j.isprsjprs.2014.12.020
0924-2716/Ó 2014 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.
234 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246
(e.g. Icaros, Simplex Mapping Ltd., VisionMap) and a growing num-
ber of existing companies (e.g. IGI, Leica/Hexagon, Track’Air/
MIDAS, Vexcel/Microsoft, BSF Swissphoto) now include oblique
camera systems in their portfolios. As competition has grown,
the market has evolved and expanded its offerings. To date, most
prominent cities in the world have been covered with georefer-
enced oblique images and these datasets are usually updated every
few years to ensure that they are current. This also gives rise to
valuable multi-temporal databases of image data (Karbo and
Simmons, 2007).
Today, the oblique camera systems are available in a variety of
configurations, the differences between designs being mainly in
the imaging sensors (number, arrangement, format and sensitivity)
and the mode of acquisition, which include:
– maltese-cross systems comprising five different imaging sensors,
one pointing vertically and the other four pointing symmetri-
cally in the four cardinal directions. Camera formats range from
small to large depending upon the intended applications, and
the tilt angles normally lie between 30° and 45°. These oblique
camera systems are suitable for large areas mapping of 3D city
modelling.
– fan systems, which mainly come as twin cameras, with the sen-
sors positioned linearly along- or across- track, making them
very suitable for corridor mapping. A quite innovative fan solu-
tion is offered by VisionMap with the A3 Edge sensor, a stepping
frame system that captures up to 64 images per sweep, corre-
sponding to a field of view of 109°.
Most of today’s oblique camera systems acquire RGB images,
though acquisitions in the NIR band are becoming standard in
the newer commercial systems.
On the application side, the actual interest in oblique photogra-
phy for mapping purposes is due to its primary quality: the imag-
ing of entire building façades and, normally, their footprints as
well. Possible applications based on oblique aerial views are multi-
ple: dense point cloud extraction for 3D city modelling (Wang
et al., 2008; Fritsch et al., 2012), monitoring situations during mass
events and environmental accidents (Kurz et al., 2007;
Grenzdörffer et al., 2008; Petrie, 2008), building detection and
reconstruction (Nex et al., 2013), urban area classification (Gerke
and Xiao, 2013), identification of structural damage to buildings
(Nyaruhuma et al., 2012), updating of road databases (Mishra
et al., 2008) and administration services (Lemmens et al., 2008).
From a practical point of view, the extracted geometric information
should be complete, reliable, generated with minimal interaction,
all while keeping the survey cost at the optimal level. However,
although the abundance and practical utility of this oblique image
data is clear, both dealing with the resulting very large datasets
and automated processing remain a challenge. This is primarily
because complex image configurations are imposed by the oblique
image acquisitions and because of shortcomings in on-board direct
orientation sensors that do not satisfy the strict requirements of
geomatics applications.
The prospect of information extraction from oblique imagery
has also given impetus to scientific initiatives (e.g. EuroSDR Bench-
mark on Image Matching; ISPRS Scientific Initiatives) aimed at
evaluating the current status of available tools and at boosting fur-
ther research into automated orientation (Wiedemann and More,
2012; Rupnik et al., 2013), dense image matching (Gerke, 2009;
Rupnik et al., 2014), object detection and 3D reconstruction (Xiao
et al., 2012). These efforts target developers and users in delivering
comparative performance analyses on the provided datasets, in
particular for dense matching and 3D reconstruction (Cavegn
et al., 2014). However, in spite of new data processing develop-
ments and comprehensive evaluation of block triangulation
results, what has been largely overlooked until now is any revision
of traditionally adopted flight patterns. Open questions include: (i)
How do different overlap scenarios influence the precision of com-
puted object points? (ii) What is the right balance between aerial
survey precision and productivity? (iii) How should a flight be
planned in terms of overlap and camera configuration parameters
(e.g. tilt) to achieve most complete results? (iv) What, after all, is
the influence of the oblique views on the image block triangulation
from the point of view of obtainable accuracy?
In the following sections of the paper, the authors address the
above problems related to oblique multi-camera systems in terms
of image triangulation (self-calibration is not the primary scope of
the paper). Section 2 reports on existing flight planning routines
and simulation studies of various block configurations. The influ-
ences of varying sensor size, tilt angle and overlap (in nadir and
oblique cameras) are considered regarding the obtainable accu-
racy, with attention being given to the issue of completeness of
point cloud retrieval in urban environments. Section 3 briefly
describes the image orientation task related to oblique imagery.
Lastly, Section 4 reports the results of two real-case studies. The
behaviour of image blocks is evaluated for different Ground Con-
trol Point (GCP) configurations, different image overlaps, and GNSS
assisted image triangulation.
2. Flight planning – simulated case studies
Flight planning refers to the initial determination of flight
geometry, given the area of interest and the required end-product,
and thus the desired accuracy. In the case of traditional nadir imag-
ery, the parameters that are optimized are the flying height, the
camera with its optics and the overlap pattern. Depending upon
the type of photogrammetric application, adopted lenses can vary
from wide to narrow angles. As a general rule, long focal lengths
are used when large height variations are present (cities, moun-
tainous areas), whereas short focal lengths are preferred in rural
areas, for overview flights or when better height accuracy is tar-
geted. Extensive studies on optimization of flight pattern scenarios
were carried out during the second half of the 20th century
(Förstner, 1985; Ackermann, 1992; Kraus, 1997, 2007). At that
time, digital photogrammetry was still an unrealised concept and
manual stereoscopic observations were commonplace. Hence, the
attention remained around vertical imaging while combined verti-
cal and oblique photography was conceived merely for pictorial
representation. The prevailing rule-of-thumb suggested that flying
scenarios with 60% forward overlap would ensure a 3-fold cover-
age for points down each side of the image, providing for strip for-
mation. Furthermore this also afforded good stereoscopic viewing
with a good base length for 3D measurement. At the same time,
the 20–40% side overlap provided a sufficiently strong geometry
to tie the neighbouring strips of photography together. Reduced
processing times and compilation costs were a top priority; there-
fore the fewer images covering the area of interest the better the
economy of a project.
The costs of mapping with photogrammetry started to signifi-
cantly reduce as the technology matured and the digital era was
embarked upon. This brought about automated image measure-
ment, which raised the compilation efficiency and reduced human
operator interaction. Automation gave rise to a new tendency to fly
denser patterns, in particular in the in-flight direction. Also, think-
ing in terms of single stereo models gave way to thinking in terms
of multi-image bundles of rays (McGlone, 2004; Aber et al., 2010).
Within this context of ‘‘digital evolution’’, oblique multi-camera
systems started again to be developed and employed for different
application fields and market niches. They began to be thought
as instruments for cartographic mapping, rather than only for
 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246
visualization purposes or reconnaissance, as was the case in the
past. The imaging geometry of oblique camera systems is analo-
gous to close-range applications, with all their characteristics,
e.g. greatly varying scale within the images, illumination changes,
multiple viewing directions and wide baselines and hence large
perspective differences between the views. To fully exploit the
oblique cameras on-board, be it for texture mapping or image
dense matching, the flight planning must thus more cautiously
adapt to the situation on the ground, and the overlaps must be sep-
arately calculated for nadir, backward/forward, and side cameras.
Apart from the flying height and the camera optics, the effective
overlaps are conditioned on the camera tilt angles. As will be
shown later, only a just combination of all the parameters will pro-
duce images that can serve for precise, and complete 3D scene
reconstruction (Fritsch and Rothermel, 2013; Rupnik et al., 2013).
In the following paragraphs, the authors evaluate the quality of
3D point triangulation under several imaging configurations. Sim-
ilar investigations using multi-view nadir and stereo oblique
geometry have been presented in Förstner (1998) and Gerke
(2009), respectively.
2.1. Simulation procedure
The purpose of simulation is to provide a priori variances of tri-
angulated points from the inversion of the normal equation matrix
of the bundle adjustment, without performing a real measurement.
It is a way of optimizing object point accuracy when minimum
information about the scene is available, a process known within
close-range photogrammetry as network design. The design prob-
lem (Fraser, 1984, 1996; Mason, 1995) typically divides into Zero-
order (the datum definition problem), First-order (the network
configuration problem), Second-order (the weight problem) and
Third-order (the densification problem) design. In the following
we consider only First-order design, i.e. the optimization of obser-
vation configurations by (i) simulating a flight, (ii) generating
image observations by back-projecting 3D points to all cameras/
images and (iii) performing a bundle adjustment to retrieve the
standard deviations from the diagonal elements of the symmetric
covariance matrix. Object points, camera positions and orienta-
tions are regarded as free parameters. The estimated precision of
object points is based on the following standard errors – image
coordinate 1 pixel, camera position 0.1 m, camera orientation
0.001°. Collinearity equations constitute the basis for the 3D
restitution.
The simulated scenarios are imaged with a maltese-cross system
and vary according to the following parameters (Table 1 and
Fig. 1): sensor size, forward and side overlap, camera focal length
and field of view. The flying height is adapted in order to keep
the Ground Sampling Distance (GSD) computed in the nadir
images constant. The choice and values of particular parameters
reflect the current market offerings and the most common flight
practices. To distinguish between the sensors, they are referred
to as small, medium and large although the size of the latter would
normally be classified as medium. The unconstrained scenarios
Table 1
Sensor specifications for the three simulated case studies, A, B, and C, which correspond, respectively, to small, medium and large format cameras of maltese-cross design. Field of
view (FOV) of oblique cameras in along direction refers to forward/backward looking views while the across-track direction refers to side-looking views. 30° and 45° oblique
angles (indicated as ‘‘tilt’’ in the table) are tested. The tilt refers to the viewing along cameras’ line-of-sight. The flying heights selected for the case studies are: 1200 m (A), 1000 m
(B) and 1600 m (C).
Case A
Nadir Oblique
Sensor size (mm/pix) 36  24/7360  4912
Focal length (mm) 50 80
FOV (°) along/across 26/39 tilt ± 8.5/8.5
235
reflect the situation where observations from all cameras are
accepted, while in the constrained scenarios only observations in
the nadir cameras and oblique cameras of same look direction
are considered, as discussed in Section 3. In order to account for
possible occlusions that influence point observability, normally
prevailing in urban environments, a complex of buildings
(Fig. 4a, street width of 5 m and 10 m, building height 20 m) was
embedded in the simulated scene. Every point back-projected to
a camera is verified whether obstructing with the scene or not. Fur-
ther constraint is imposed on the oblique cameras, specifying that
object points can be observed only from cameras that look the
same direction (see also Section 3). This tolerance is full-fledged
given the large perspective distortion between slanted views that
hamper the matching of homologous points (Rupnik et al., 2013).
2.2. Precision, productivity, completeness
2.2.1. Precision
According to the data contained in Table 1, presented case stud-
ies involve the maltese-cross system with small (A), medium (B)
and large format (C) cameras. Results presented in Fig. 3 show
the relationship between object point precision (computed from
the covariance matrix) and factors such as overlap, camera tilt
angle, and imaging ray redundancy. Due to the fact that the scale
of the nadir images is kept constant for all case studies, the figures
of precision do not vary significantly between the cases.
Within the different imaging configurations, the overlap and tilt
angle of the oblique cameras, and the imposed constraint, largely
influence the results. It can be noted (Fig. 3) that the increased tilt
angle contributes to:
– a larger point redundancy (observations),
– improved precision, i.e. a better standard deviation of 3D points,
– a homogenous standard deviation in all three dimensions.
However, imposition of the look direction constraint causes:
– degradation in precision along the Z axis through a decrease in
both the number of observations and the intersection angles,
– inhomogeneous planimetric precision (especially noticeable for
large tilt angles) due to the different intersecting angles in the
XZ and YZ planes.
Compared with the classical scenario of exclusively nadir
images (Fig. 3), it can be observed that while the planimetric pre-
cision is only slightly improved, the standard deviation in the Z
direction (height) is 2–3 or 4–5 times better, depending whether
the look direction constraint is enabled.
2.2.2. Productivity
Increasing overlap between subsequent acquisitions and strips
is costly in the image collection time and post-processing but
causes substantial gain in precision, as proves Fig. 2. The figure dis-
plays the precision differences between the overlap 60/40 and 80/
Case B Case C
Nadir Oblique Nadir Oblique
49  36/8176  6132 70  45/11,674  7514 53  40/8900  6650
50 80 80 120
39/52 tilt ± 12.5/12.5 31/47 tilt ± 9.5/(+9.5–15.7)
236 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246

Fig. 1. Schematic representation of the simulated camera systems in cases A, B and C for 45° tilt angle (a–c). Overlapped FOVs for the different systems (d). Top row
represents the along-track direction, the bottom row is the across-track configuration. The angles reported for every camera system show the maximum achievable incidence
angle.

Fig. 2. Differences in precision counted in percent for scenarios A, B and C in different tilt angle configuration (a–c). The plain line refers tothe comparison of 60/40 and 80/60
flight pattern. The crossed line refers to the comparison of 80/60 and 80/80 flight pattern. The percentage is computed as a ratio e.g. r60=40 x  rx80=60 =r60=40
x . Number of
acquisitions necessary to cover a 4 km  5 km area of interest in various overlap scenarios, with the camera systems A, B, C (d).

60 (plain line) as well as between 80/60 and 80/80 (crossed line).
Throughout all the scenarios, flying with a 80/60 overlap pattern,
rather than 60/40, delivers standard deviations better by ca. 50%
along X, Y, Z directions. On the other hand, when comparing the
standard deviations between 80/60 and 80/80 overlap, a precision
increase of only about 20–30% is observed. To give more meaning
to the figures, they should be ‘‘weighted’’ by the additional number
of acquisitions needed to densify the data collection, especially
that the number grows exponentially (Fig. 2d). Confronting the
two, the rationality suggests that the 80% along- and 60% across-
track patterns would be the golden mean as the return in precision
for the cost of extra labour is substantial. Having said this, it must
 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246 237

be underlined that dense matching in oblique datasets is feasible

and camera tilt angle. The numbers within the diagram boxes correspond to point redundancy according to the different overlap. See Section 3 for explanation of constrained and unconstrained cases. The flight trajectory is along
Fig. 3. For the three case studies A, B, C (small, medium and large format, respectively), standard deviations (m) of computed object coordinates in X (red line), Y (green line), Z (blue line) are reported with respect to image overlap
from images of the same perspective. This is analogous to the
automatic tie point extraction step where SIFT-like features are
matched across views of similar look direction (see Section 3).
It implies that both across- and along-track overlaps are equiva-
45° Constrained

lent and favourably equal to 80%.

With the GSD size equal for all cases, the variable sensor size
affects only the survey productivity. In Fig. 3 the redundancy fig-
ures of the C camera system are in the majority of cases 20–30%
smaller than in A and B. Notably, the measures of precisions do
not drop accordingly (see also Fig. 2).

2.2.3. Completeness
Besides the precision and productivity aspects, sensor size and
tilt angle are also important from the standpoint of the complete-
ness of the reconstructed scene. Increasing the value of both
parameters enlarges the camera’s field of view (FOV) and pro-
vides for a bigger GSD, respectively (assuming fixed focal length).
But, larger FOVs contribute to more severe occlusions – an unde-
45°

sirable artefact in image-based 3D city modelling (Haala and

Kada, 2010). Because of this, as a part of the project planning,
attention should be paid to careful adaptation of the camera plat-
form parameters to the situation in the field, i.e. to the topogra-
phy of the city to be flown over.
Fig. 4b–e exemplifies the effect of the tilt angle on the visibility
of two selected façades. Both façades are oriented parallel to the
flight direction, and the width of the street in front of them are
the Y axis. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

10 m (building E10) and 5 m (building E5). The simulated example

30 ° Constrained

was derived from the flight with 80/60 overlap, surveyed with the
camera system of type A. Due to the occlusion effects, the tilt
angle of 45° introduces inhomogeneous distribution of redun-
dancy along the façades. The 30° tilt angle results in fewer but
uniform redundancies. The width of the street and the building
height play a major role in planning a successful urban survey
campaign. The taller the city architecture the lower the camera
incidence angle should be. Therefore, a compromise ought to be
found between the camera tilt setting, given focal length, sensor
size, overlap and the geometry of the surveyed area.

3. Image block orientation

The processing workflow for oblique imagery can be divided in

30°

sequential steps (Rupnik et al., 2014), namely image orientation,

dense matching and point cloud filtering, feature extraction,
orthoimage generation and monoplotting (Murtiyoso et al.,
2014). While this follows the traditional photogrammetric pro-
cessing pipeline, the algorithms for the orientation and matching
phases need to be slightly adjusted in order to cope with oblique
image datasets. The orientation task is recapped below. Dense
matching and feature extraction issues are discussed in Nex
et al. (2013).
In the image orientation process, the interior and exterior
parameters are nowadays often known a priori, as retrieved with
a prior calibration procedure or measured directly with on-board
Nadir

sensors (GNSS/IMU), respectively. Nonetheless, these parameters

are generally regarded as approximate if one has metric and auto-
matic applications in mind. An adjustment in a least squares
sense is therefore mandatory, requiring linearization of the equa-
tions and good approximate values of unknown parameters. On
the basis of image data (homologous points), these parameters
are (i) retrieved with direct methods (e.g. resection, perspec-
tive-n-point problem) individually for each image, (ii) concate-
A

C
B

nated to a common reference system with the help of the

fundamental/essential matrix or trifocal tensor, and eventually
(iii) refined inside the bundle adjustment. The homologous points
238 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246

E10 , 45° E10, 30° E5, 45° E5, 30°

(a) (b) (c) (d) (e)

Fig. 4. The simulated complex of buildings (20 m high) (a). Given the two building façades (E5, E10), figures (b–e) show a color coded map with the number of images where a
point on the façade is observed. E10 (10 m width street) and E5 (5 m width street) are observed with the A camera system in 80% along-, 60% across-track overlap. Oblique
cameras are tilted by 30° (c and e) and 45° (b and d). The flight direction is along the Y (green) axis. (For interpretation of the references to color in this figure legend, the
reader is referred to the web version of this article.)

Fig. 5. Camera network of the Milan dataset (left) and five different configurations of the ground control information. The Y axis is parallel to the flight direction (Y runs
vertically).

Fig. 6. Camera network of the Graz dataset (left) and four different configurations of the ground control information. The Y axis is orthogonal to the flight direction (Y runs
horizontally).

are found automatically across multiple aerial views with standard
area-based image matching techniques (such as Normalized Cross
Correlation and Least Squares Matching) or feature-based methods
(SIFT, SURF, etc.) (Apollonio et al., 2014) coupled with robust esti-
mators to remove possible wrong correspondences (RANSAC, Least
Median of the Squares, etc.).
Oblique imagery is richer in content compared to nadir imag-
ery. Objects (building, roads, etc.) are recorded at different scales,
and object features are back-project under different projective
transformations. Also, due to different viewing directions grave
occlusions are exhibited. Such configurations, are characteristic
of unordered terrestrial image networks (Barazzetti et al., 2010;
Pierrot-Deseilligny and Clery, 2011) and the generation of putative
correspondences between images is therefore prone to a greater
number of mismatches. Indeed, the literature reports that tradi-
tional photogrammetric software is optimized for nadir image
acquisition and generally performs poorly for other imaging geom-
etries (Jacobsen, 2008; Gerke and Nyaruhuma, 2009).
In the developed processing pipeline, the computation of
approximate orientation parameters is guided with the help of a
connectivity graph (Rupnik et al., 2013). The connectivity between
images refers to a graph with nodes and edges being representa-
tions of images and their relationships. Two images are linked with
an edge if and only if they are spatially compatible. GNSS/IMU data
 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246 239

Table 2
Dataset specifications (N and O stand for nadir and oblique). The given overlap is calculated on the nadir images.

Data Camera Sensor size (mm) Focal N/O FOV along/ Image GSD N # images Y/X Area
(mm) across N (°) scale (m) N/O overlap (%) (km)
Milan Midas 5 36  24 80 17/25 15,000 0.10 125/375 70/30 35
100
Graz Vexcel Osprey I 70  45 (nadir); 23.5  36 (left/right) 71.5  23.5 51 48/70 21,000 0.12 20/160 75/65 3  1.5
(for-/backward)
80

Table 3
A summary of all performed tests. ID refers to a name given to a particular test. The columns indicate the adjustment parameters used as constants, unknowns, observed
unknowns, and observations. FX means fixed IOR, FR refers to self-calibrating bundle adjustment (i.e. free IOR).

ID Input images Constants (C) unknowns (U) Exterior orientation Interior orientation GCPs, observed in GCPs, observed in Tie
observations (O) EORa IOR + AP nadir oblique pointsb
1FX Nadir C U
U U U U
O U U U
2FX Nadir + oblique C U
U U U U
O U U U
3FX Nadir + oblique C U
U U U U U
O U U U U
1FR Nadir C
U U U U U
O U U U
2FR Nadir + oblique C
U U U U U
O U U U
3FR Nadir + oblique C
U U U U U U
O U U U U
a
GNSS observations are used inside the bundle block adjustment as observed unknowns only in the GNSS-assisted scenario.
b
Tie points in image space treated as observed and in object space as unknown.

MIDAS 5 UltraCam Osprey I

Forward and backward oblique views Forward and backward oblique views

along-track overlap across-track overlap along-track overlap across-track overlap

Side oblique views Side oblique views

along-track overlap across-track overlap along-track overlap across-track overlap

Fig. 7. Overlaps in forward/backward and side oblique cameras for Midas 5 (left) and UltraCam Osprey I (right) systems corresponding to 70/30 and 75/65 overlaps in nadir
views. The overlap in the oblique views is given as a ratio of the complete length of the median of a trapezoid (forward/backward camera in cross-track overlap and side
camera in along-track overlap) or the axis of symmetry (forward/backward camera in along-track overlap and side camera in cross-track overlap), to its length between
subsequent acquisitions (red line). It is computed in along- (horizontal red line) and cross-track (vertical red line). (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

beneficially provides the input information for establishing the reducing the number of possible outliers. There are three condi-
image associations. Traversing across the connectivity graph helps tions (constraints) to be fulfilled for an image pair to be regarded
in speeding up the extraction of image correspondences and in as compatible:
240 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246
Table 4
The computed a posteriori standard error of unit weight r0 (pixel), for the different
configurations of ground control information reported in Figs. 5 and 6. Note that the
sigma values are the same for both, with and w/o the GNSS support.
Data GCPs config. 1FX 2FX 3FX 1FR
Milan 0–4 0.49 0.47 0.55 0.47
Graz 1–4 0.38 0.36 0.39 0.31
a Their ground footprints coincide by a given percent.
b Look directions of the images are similar or one is nadir-looking
(similarity can be then violated).
c The number of extracted homologous points for the pair is
above a given threshold.
Given a graph complying with conditions a and b, feature
extraction is allowed between images with at least two edges.
Next, the edges of the graph are enriched with another attribute,
i.e. the number of extracted tie points, and all images are concate-
nated. Concatenating images for the initial solution is understood
to (i) find triplets of images within the connectivity graph to be
used for computing approximate orientations, and (ii) providing
all the triplets a sequence in the concatenation order. A bundle
adjustment controlled in this way minimizes the risk of divergence
by maximizing the similarity of images within particular triplets,
hence maximizing the ratio of good to bad matches and ensuring
the block’s cohesion.
As a last step, the control information in form of GNSS/IMU data
and/or GCPs is included in the adjustment computations. With this,
the data is transformed to the coordinate system of the control
information. The quality of the resulting orientation is strongly
coupled to the spatial distribution of the control information, its
‘‘observability’’ in the images as well as the importance given in
the adjustment. It should be remarked that if the surveyed area
surpasses a certain size, the Earth curvature effects must be
considered.
4. Real case studies
4.1. Datasets
The first dataset (Milan) comprised 550 images and was
acquired with a Midas 5 multi-camera system (one nadir and four
oblique looking cameras tilted at 45°). The image block covered an
area of ca. 3 km by 5 km that was flown with 70/30 (Y/X) overlap
related to nadir images (Fig. 5). In order to evaluate the quality
of the block orientation results, 25 GCPs were GNSS-surveyed with
a mean positional accuracy less than 5 cm.
The second dataset (Graz) comprised 180 images and was
acquired using an UltraCam Osprey I system (one nadir camera
with six oblique-looking cameras tilted at 45° along the four car-
dinal directions – see Gruber and Wolfgang (2014)). The area
covered ca. 3 km by 1.5 km and was flown with an overlap of
75/65 (Y/X) related to nadir images (Fig. 6). In a similar manner
to the Milan case study, a metric evaluation of the achieved
results was performed using ground truth information in form
of GCPs and check points (CP) GNSS-surveyed to a mean accu-
racy of 5 cm.
In both datasets the approximate exterior orientation param-
eters (EOR) were available as GNSS/IMU observations. The
dataset specifications are reported in Table 2. The structure of
the flights resemble the comprehensive block specification
of Honkavaara et al. (2006a), without crossing flight lines. Corre-
sponding overlaps computed for oblique images are presented in
Fig. 7.
4.2. Tests
Two pieces of control information (GCPs and GNSS observations)
were incorporated in the computations, and regarded as soft con-
2FR 3FR straints, i.e. treated as unknowns and observations (so-called
0.45 0.52 observed unknowns), therefore assigned corrections within the
0.29 0.31 bundle adjustment. The blocks were oriented using different con-
figurations of GCPs (yellow circles in Figs. 5 and 6) and independent
CPs (red markers in Figs. 5 and 6).1 For both datasets, configuration 1
shows a good GCP distribution along the borders of the block,
whereas configurations 2, 3 and 4 are examples of unfavourable con-
trol point distributions. Although the latter may be seen as unrealistic
cases, they are shown for the sake of completeness and to stress the
role of ground control information in the bundle solution. Thanks to
the large number of points surveyed in the Milan test area, an addi-
tional configuration was added (configuration 0). It comprises a good
GCPs distribution across the whole area of interest. A summary of the
performed tests is provided in Tabs. 3 and 5, where abbreviated
descriptions of the different processing scenarios are listed as well.
The initial values of internal orientation parameters (IOR) and
additional parameters (AP) were taken from a calibration certifi-
cate provided by the company executing the flight. These input
data can be either held fixed within the bundle adjustment process
or included as unknowns in a self-calibration procedure. In our
tests, the scene contains significant height differences – leading
to variable scale across images, hence both possibilities were
tested. A version of the Fraser calibration model (Fraser, 1997),
involving three coefficients of radial distortion and two coefficients
of decentering distortion, was used in the experiments. While for
middle-format cameras this could not be sufficient and more
sophisticated approaches are available (Cramer, 2009), the camera
calibration issue was not the primary scope of this article.
In the following, two measures of adjustment quality are
reported, the sigma naught (r0) and the root mean square values
(RMS), defined as the square root of the mean squared difference
between nominal values and corresponding adjusted observations.
Moreover, to give a more global impression of the possible deforma-
tions within the image block, the resulting sparse point clouds are
compared to a reference point cloud, which was chosen to be the
one exhibiting the optimal measures of accuracy against CPs. Net-
work 3FR in GCPs configurations 0 (Milan) and 1 (Graz), i.e. self-cal-
ibrating condition and good distribution of GCPs was selected. The
imaging geometry corresponds to the constrained case of Section
3, i.e. tie point observations were allowed from all nadir images
and those oblique images with the same viewing directions.
The purpose of the performed tests is to investigate how the
accuracy of triangulated 3D points is affected by the following fac-
tors: (i) introduction of oblique imagery in the image block, (ii)
self-calibration, (iii) spatial distribution of GCPs and (iv) GNSS sup-
port. All tests were conducted using MicMac, an open-source soft-
ware suite for bundle block adjustment and dense image matching
(Pierrot-Deseilligny and Paparoditis, 2006; Pierrot-Deseilligny and
Clery, 2011).
4.3. Bundle adjustment results
MicMac solves the orientation task within a bundle adjustment
by means of weighted least mean squares. Classically, the imaging
equation f (see Eq. (1)), i.e. the functional model of the adjustment,
is defined in terms of the following parameters: rotations R, per-
spective centers C, camera interior parameters I, and object coordi-
nates of the tie points P. The optimal values of the parameters are
1
For interpretation of color in Figs. 5 and 6, the reader is referred to the web
version of this article.
 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246 241

Table 5
RMS errors computed at CPs. The rows represent 5 different GCPs configurations; the column labels are explained in Table 3. The values correspond to the geo-referencing and
block adjustment based on GCP information while the values in bracket correspond to the GNSS-assisted triangulation.

GCP RMS No. No. MILAN dataset No. No. GRAZ dataset
config. (m) GCPs CPs GCPs CPs
1FR 2FR 3FR 1FX 2FX 3FX 1FR 2FR 3FR 1FX 2FX 3FX
x 0.12 0.12 0.15 0.10 0.07 0.13
(0.16) (0.14) (0.15) (0.16) (0.13) (0.16)
0 y 7 12 0.12 0.10 0.15 0.18 0.28 0.30 – – – –
(0.18) (0.12) (0.14) (0.17) (0.13) (0.14)
z 0.42 0.30 0.13 0.45 0.55 0.20
(0.32) (0.30) (0.20) (0.34) (0.31) (0.20)
x 0.23 0.22 0.18 0.36 0.16 0.21 0.04 0.04 0.05 0.04 0.04 0.03
(0.26) (0.21) (0.16) (0.26) (0.20) (0.17) (0.04) (0.04) (0.05) (0.05) (0.06) (0.05)
1 y 4 15 0.22 0.14 0.16 0.27 0.48 0.45 4 3 0.03 0.06 0.04 0.04 0.06 0.05
(0.19) (0.13) (0.14) (0.18) (0.14) (0.15) (0.03) (0.04) (0.03) (0.03) (0.04) (0.04)
z 6.85 1.39 0.26 10.68 3.68 0.64 0.08 0.08 0.02 0.17 0.13 0.13
(0.76) (0.54) (0.22) (0.77) (0.56) (0.20) (0.04) (0.10) (0.04) (0.20) (0.18) (0.16)
x 0.55 1.01 0.33 1.07 1.28 0.42 0.03 0.05 0.06 0.03 0.04 0.06
(0.26) (0.21) (0.22) (0.31) (0.20) (0.24) (0.03) (0.05) (0.05) (0.03) (0.09) (0.05)
2 y 3 16 0.29 0.35 0.21 0.68 1.81 0.92 3 4 0.03 0.03 0.03 0.04 0.03 0.04
(0.23) (0.15) (0.17) (0.30) (0.15) (0.19) (0.03) (0.03) (0.03) (0.03) (0.05) (0.07)
z 12.22 11.92 0.80 19.75 15.40 1.63 0.34 0.27 0.21 0.49 0.61 0.62
(0.77) (0.56) (0.29) (2.47) (0.58) (0.27) (0.08) (0.09) (0.04) (0.33) (0.34) (0.41)
x 0.31 0.80 0.75 1.79 3.29 3.33 0.04 0.14 0.13 0.11 1.36 1.35
(0.12) (0.13) (0.14) (0.17) (0.10) (0.13) (0.04) (0.14) (0.13) (0.11) (0.53) (0.52)
3 y 3 16 0.78 2.14 1.81 4.46 7.28 6.84 3 4 0.09 0.11 0.05 0.07 0.49 0.49
(0.38) (0.35) (0.35) (0.80) (0.35) (0.39) (0.09) (0.05) (0.05) (0.08) (0.19) (0.20)
z 6.67 2.29 1.77 11.97 3.81 2.10 2.85 2.54 2.76 4.14 2.75 2.90
(0.36) (0.34) (0.24) (0.79) (0.40) (0.54) (0.09) (0.24) (0.17) (0.24) (0.14) (0.26)
x 0.28 0.20 0.19 0.63 1.16 1.46 0.05 0.04 0.04 0.04 0.05 0.04
(0.28) (0.24) (0.23) (0.27) (0.22) (0.25) (0.04) (0.06) (0.06) (0.06) (0.15) (0.13)
4 y 3 16 0.21 0.23 0.28 0.72 2.02 2.50 4 3 0.05 0.05 0.06 0.06 0.05 0.06
(0.19) (0.09) (0.13) (0.18) (0.10) (0.11) (0.04) (0.04) (0.03) (0.03) (0.03) (0.06)
z 4.30 2.24 1.59 7.62 4.75 2.30 1.07 2.47 2.86 2.22 2.86 3.32
(0.72) (0.52) (0.22) (0.70) (0.52) (0.24) (0.07) (0.13) (0.18) (0.21) (0.28) (0.42)

found by minimizing an objective function (Eq. (3)) which is the Here, the hat variables (e.g. ^l) indicate the estimates whereas the
sum of squared discrepancies multiplied by a weighting function bar variables (e.g. ^l) indicate the original observations. Also, r is
q(v), see Eq. (4). The weight function is composed of three terms: the number of degrees of freedom (redundancy), Nobs the number
1
r 2 that weights observations given their a priori accuracy,
of observations within a given group of observations (e.g. GCPs)
obs
Nmax and Nmax a threshold that limits the influence of a group of observa-
Nmax þN obs
that limits the influence of a particular group of observa-
tion below the specified threshold value.
tions on the solution, and pffiffiffiffiffiffiffiffiffiffi
1
v 2
that ‘‘penalizes’’ observations In our evaluations, the tie points are measured with the SIFT
1þðlÞ
operator and the a priori image measurement accuracy (robs) is
with large residuals making it a robust estimation. The normalizing
set to 0.5 pixel. The rejection parameter B is set to be ten times
factor l is dataset-dependent and should be set according to the
estimated observation accuracy. As a general rule, its value will
robs. Estimated accuracies of the remaining observations are cho-
sen in the range of 0.05–0.1 m for GCPs and 0.5–1.0 m for GNSS
change from iteration to iteration. The normalizing factor should
observations. The Nmax parameters are chosen to balance the influ-
be high when unknowns are poorly approximated and recede
ence of all groups of observations within the adjustment, and to
along the iterations. If a residual exceeds the user-defined B
avoid the situation where only the abundant tie points or GNSS
parameter, the weight takes on the value zero, hence eliminating
data have a dominant influence upon the solution. The quality
the influence of that particular observation in the adjustment. An
and quantity of all groups of observations must be taken into
in-depth explanation of the adopted weighting scheme can be
account when assigning values to these parameters (e.g. 550 GNSS
found in (Pierrot-Deseilligny, 2014).
observations at 1.0 m and 7 GCPs at 0.1 m). In our tests, the Nmax
^l ¼ f ð R; b bI; PÞ
b C; b ¼ l þ v ð1Þ were assigned the value of 50 for the group of tie points and GNSS
data. The GCPs were not weighted due to its scarcity compared
V ¼ l  f ð R; b bI; PÞ
b C; b ð2Þ with other groups of observations. In other words, the second term
of Eq. (4) is equal to 1 for each GCP observation.
X
qðv Þv 2 ! minimum ð3Þ Table 4 reports the a posteriori standard error of unit weight of
the bundle adjustment (r0 in Eq. (5)) computed in the aforemen-
tioned tests. Introducing oblique images into the network, but
1 Nmax 1
qðv Þ ¼ if ðv > BÞ 0 else   qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð4Þ allowing the image measurements of GCPs only in nadir images
r2obs Nmax þ Nobs 1 þ ðlv Þ2 (2FX, 2FR), brings about a slight improvement in the re-projection
error. On the contrary, when GCPs are measured both in nadir and
P oblique views (3FR, 3FX), the re-projection error grows. It is an
qðv Þv 2
r20 ¼ ð5Þ expected outcome given that point identification and precise mea-
r
surement in slanted views are more difficult. Also, the ground sam-
242 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246

MILAN dataset
GCPs
conf. 1FR 2FR 3FR 1FX 2FX 3FX

. GRAZ dataset

Fig. 8. Discrepancy vectors at GCPs (red triangles) and CPs (open circles). Above: Milan dataset, GCP configurations 0 and 3, with GNSS. Below: Graz dataset, GCP
configurations 1 and 3, with GNSS. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

pling distance can be larger than that for the nadir looking camera. nario to scenario, r0 remains nearly constant and a plausible expla-
Furthermore, although freeing camera parameters can help to nation for this is that even if the GCPs introduce ‘‘strain’’ into the
compensate for systematic lens- and sensor-related errors, the block, resulting in locally larger residuals, the robust weight func-
improvement in terms of a reduced r0 is minor. Finally, from sce- tion (Eq. (4)) diminishes their effect on the sigma naught value.
 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246 243

1FX - 3FX 1FR - 3FR 1FX - 1FR 3FX - 3FR

MILAN dataset

no-GNSS

μ = 0.89 m μ = 0.53 m μ = 0.68 m μ = 0.23 m

σ = 0.76 m σ = 0.48 m σ = 0.46 m σ = 0.15 m

GNSS

μ = 0.35 m μ = 0.36 m μ = 0.07 m μ = 0.10 m

σ = 0.26 m σ = 0.28 σ =0.06 σ =0.09

GRAZ dataset

no-GNSS

μ = 0.08 m μ = 0.06 m μ = 0.18 m μ = 0.25 m

σ = 0.11 m σ = 0.06 m σ = 0.11 m σ = 0.21 m

GNSS

μ = 0.10 m μ = 0.06 m μ = 0.15 m μ = 0.22 m

σ = 0.08 m σ = 0.05 m σ = 0.09 m σ = 0.18 m

Fig. 9. l – arithmetic mean Euclidean distance, and r – standard deviation between sparse point clouds computed in GCP configurations 0 (Milan) and 1 (Graz). E.g. 1FX – 3FX
refers to the difference between sparse point clouds resulting from 1FX and 3FX scenarios. Red dots mark the GCPs, the black dots are the CP’s. Note that the legends should be
decoded separately for each dataset. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

4.3.1. Influence of oblique views
It is evident from the RMS values in Table 5 that for the Milan
dataset, observations in oblique views improve object point accu-
racy in height (compare 1FR and 1FX with 2FR/3FR and 2FX/3FX),
which is consistent with expectations from the simulations pre-
sented in Section 2. For the Graz dataset the advantage of oblique
views is less obvious but nevertheless visible especially in the FR
examples of GCP configurations 1 and 2, where it leads to better
accuracies in the Z direction. What is symptomatic is the fact that
in GCP configurations 3 and 4, i.e. the unfavourable point distribu-
tions, nadir images (1FR) deliver either better or the same Z-accu-
racy results as 2FR or 3FR with the oblique views included. The
exclusively nadir block seems to be more stable in the event of
deficient ground control information, which is opposite to the
results in the Milan dataset. The different behaviour could be due
to (i) different parameters of the nadir camera models, i.e. large
sensor and shorter focal length of the Osprey I system provide
for better imaging geometry than the small sensor and long focal
length of Midas 5 (see Table 2), and (ii) different overlap of the
two image blocks (75/65 versus 70/30 related to nadir cameras).
4.3.2. Influence of self-calibration
For the Milan dataset, the bundle solutions without GNSS gen-
erally show a gain in accuracy when IO parameters are treated as
unknowns (self-calibration). In the GNSS-assisted solutions this
evidence is less pronounced and the RMS errors at independent
CPs are comparable, suggesting that (i) the available camera cali-
bration is valid and self-calibration is superfluous, and (ii) the
newly adjusted camera calibration parameters in the GCP-only
cases do not correct for systematic errors stemming from physical
deficiencies in the additional parameters model but, due to projec-
tive coupling with the EOR, do compensate for the suboptimal esti-
mates of camera rotations and positions (see Section 4.4 for more
evidence). For the Graz dataset, freeing self-calibration parameters
in the bundle adjustment improved results in the Z coordinates for
all GCPs configurations (compare all FX versus FR). The provided
244 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246

1 2 3 4 2 3 4
3FX (GCPs config. 1(2)-4) - 3FX (GCPs config 0(1))

no- no-
GNSS GNSS

μ = 0.34 m μ = 0.83 m μ = 2.09 m μ = 1.56 μ = 0.33 m μ = 1.43 m μ = 1.15 m

σ = 0.18 m σ = 0.47 m σ = 1.18 m σ = 0.86 m σ = 0.23 m σ = 0.92 m σ = 0.83 m

GNSS GNSS

μ = 0.08 m μ = 0.19 m μ = 0.45 m μ = 0.18 m μ = 0.16 m μ = 0.35 m μ = 0.19 m

σ = 0.13 m σ = 0.21 m σ = 0.15 m
σ =0.03 m σ = 0.09 m σ = 0.25 m σ = 0.08 m
3FR (GCPs config. 1(2)-4) - 3FR (GCPs config 0(1))

no-
no-
GNSS
GNSS

μ = 0.07 m μ = 1.28 m μ = 1.07 m

μ = 0.18 m μ = 0.50 m μ = 1.10 m μ = 0.98 m σ = 0.05 m σ = 0.94 m σ = 0.75 m
σ = 0.09 m σ = 0.27 m σ = 0.51 m σ = 0.63 m

GNSS GNSS

μ = 0.09 m μ = 0.18 m μ = 0.28 m μ = 0.14 m μ = 0.01 m μ = 0.11 m μ = 0.07 m

σ = 0.01 m σ = 0.07 m σ = 0.06 m
σ = 0.04 m σ = 0.09 m σ = 0.15 m σ = 0.07 m

Fig. 10. l – arithmetic mean Euclidean distance, and r – standard deviation between point clouds in selected scenarios and the reference cloud. The reference is unique for all
comparisons – 3FR in GCP configuration 0 in Milan, and GCP configuration 1 in Graz. Left: Milan, right: Graz datasets. Note that the legends should be decoded separately for
each dataset.

camera calibration, unlike that of the Milan dataset, was not suffi- weights for the GNSS data should be used within the bundle
cient to achieve the most optimal result and self-calibration affor- adjustment.
ded superior results. As seen in the discrepancy plots in Fig. 8, the
vectors in Milan dataset do not change between the cases of fixed 4.4. Evaluation of derived sparse point clouds
or freed. Their lengths in the Graz dataset, however, clearly recede
if camera calibration parameters are re-adjusted. No ground truth in the form of DSM data was available for
either the Milan or Graz networks. Possible deformations and the
4.3.3. Influence of GCPs distribution and GNSS observations level of noise in the derived 3D object coordinates (i.e. sparse point
The GCP distribution plays a crucial role in determining CP cloud) were evaluated for the cases of with and without oblique
accuracy. So long as the distribution of control information is even images, GNSS/IMU data and when GCP configurations changed.
and appropriately dense, the constraint on CPs is satisfactory (GCP Conclusions were drawn from comparisons in Figs. 9 and 10.
configuration 0 and 1 in Table 5). As soon as the GCP configuration In Fig. 9, in order to ascertain the influence of the inclusion of
gets sparser, extrapolation effects emerge and are reflected in lar- oblique images, scenarios 1FX and 1FR were compared with sce-
ger RMS error values (GCPs configuration 2, 3 and 4 in Table 5 and narios 3FX and 3FR. To free the analyses from the impact of GCP
Figs. 8 and 10). The inclusion of GNSS constraints largely mitigates locations inasmuch as possible, results obtained in GCPs configura-
the problem (GCPs configuration 2, 3 and 4 in Table 5, values in tion 0 (Milan) and 1 (Graz) were used. As a side-product, also the
brackets). Under poor GCP distributions, the Z coordinate suffers influence of self-calibration on the triangulated 3D points was
the most, and this is the case for both datasets. In GCP configura- devised from the differences 1FX with 1FR and 3FX with 3FR.
tion 3 the blocks ‘‘wobble’’ in counter directions (compare the In Fig. 10, in order to determine the influence of the GCPs distri-
RMS in X and Y in Table 5). Nonetheless, although the Milan dataset bution, the oblique image blocks 3FR and 3FX, in all GCP configu-
appears highly sensitive to GCP variations, the sensitivity of the rations, were compared against a reference result. The reference
Graz network is minor, especially when GNSS constraints are is unique for all comparisons and was chosen to be the one report-
included. Again, the imaging geometry and extent of image overlap ing the optimal RMS error values at CPs, namely 3FR in GCPs con-
are the most probable factors influencing this sensitivity. In the figuration 0 (Milan) and 1 (Graz). The comparisons quantified the
Milan case study, switching on the GNSS constraints in one of mean differences and standard deviations. Also color coded maps
the ‘‘good’’ scenarios (3FR, GCPs configuration 1) leads to larger of the geometric differences (Euclidean distance) were produced.
coordinate discrepancies at CPs (0.20 m against 0.13 m). This The distance values were color coded such that they ranged from
behaviour could be a sign of ‘‘tension’’ between ground control 0 to 6 times (Milan dataset) or 3 times (Graz dataset) the standard
information and perspective centers. As a remedy, less optimistic deviation determined in each test.
 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246
4.4.1. Influence of oblique views and self-calibration
The sparse point differences confirm and complement the con-
clusions of the previous subsection. Adopting oblique cameras
reduces the magnitude of noise in 3D object space and homogenis-
es the distribution of discrepancies within the block. Blocks con-
taining the full set of 5 or 7 imaging cones are less sensitive to
the GCPs arrangement and shorten the growth of discrepancies
outside the region containing the GCPs. Bundle adjustment with
self-calibration brings about further profits with regard to noise
and deformations (see the patterns in 1FX – 1FR and 3FX – 3FR).
The ‘‘dooming’’ effect that is characteristic for un-modelled radial
distortion error in 1FX – 1FR of the Graz dataset (Fig. 9, bottom),
provides further evidence of an outdated camera calibration and
justifies the freeing of additional parameters in the bundle adjust-
ment. James and Robson (2014) found that such systematic errors
can be mitigated if oblique images are acquired in the block. Based
on real and simulated studies, they showed that in the presence of
radial distortion, with an assumed error-free camera model, (i) a
nadir block will produce the ‘‘dome’’ in the DEM (see 1FX – 1FR)
and (ii) an oblique block will decrease the magnitude of discrepan-
cies (no ‘‘dome’’ effect in 3FX – 3 FR). It can be validly claimed that
under conditions of self-calibration, the systematic vertical DEM
error is eliminated, for all practical purposes, which explains the
small discrepancies obtained in 1FR – 3FR. Similar conclusions
are reported in Wackrow and Chandler (2011), Nocerino et al.
(2014). Besides the radial distortion, another cause of the
systematic block deformations in Figs. 9 and 10 is possibly the
un-modelled out-of-plane distortion, i.e. unflatness. Fraser (1997)
warns that systematic errors caused by the sensor unflatness are
critical in that the image space perturbations might be concealed
while their effect on photogrammetric triangulation precision is
severe. The induced radial displacement depends on the value of
the incidence angle thus wide-angle lenses are especially subject
to manifest the errors. The standard 8-parameter and the extended
10-parameter models are insufficient to compensate for these
misclosures. Yet, they are present in today’s photogrammetric sys-
tems. In theory higher degree polynomial functions could remove
these systematisms, but in practise their application is not advised
for the arising correlations between the parameters. Development
of new mathematical models is therefore encouraged (Honkavaara
et al., 2006b; Tang et al., 2012).
4.4.2. Influence of GCPs distribution and GNSS observations
In both datasets the ‘‘strain’’ that the GCPs induce on the entire
block is striking in all the block adjustment solutions that do not
include GNSS constraints. The discrepancies are kept small within
the general region enclosed by GCPs (marked with red dots in
Fig. 9) whereas extrapolation errors increased away from this
region. When GNSS constraints are introduced into the bundle
adjustment, the magnitude of these discrepancies is substantially
reduced, as is the extrapolation error. GNSS observational con-
straints stabilize the bundle adjustment solution, due to projection
centers behaving like control points. The visible elongated defor-
mation pattern (Fig. 9, top) arising in the comparison of the nadir
image blocks (1FX – 1FR) with GNSS constraints was most likely
be due to dependencies between camera calibration parameters
and exterior orientation parameters.
5. Conclusions
This paper has presented an analysis of oblique multi-camera
systems employed for mapping applications. The simulated and
real case studies demonstrate that increasing the overlap, and thus
point redundancy, improves 3D precision and accuracy. The tradi-
tional 60/40 flight pattern applied to an oblique image block pro-
245
duces substantial enhancement in precision over the
corresponding block with nadir images only. The precision in
height determination is increased some two to five times com-
pared to the nadir image case. While increasing the overlap to an
80/60 scenario can bring about further improvements, the change
in precision between 80/60, 80/80 and 90/90 is less dramatic. To
grasp the actual value of the increased overlap, the gain in preci-
sion ought to be confronted with the extra labour imposed by den-
sified data collection. It has been shown that the latter grows
exponentially. On the other hand, having in mind the application
of 3D city modelling from imagery and knowing that the content
of urban scenes is complex, with visibility restricted due to occlu-
sions, the overlap should be kept large. It is advised that as a part of
the project planning camera system and flight parameters are
adapted to the topography of the city. The building height to street
width ratio is an important factor in the planning phase.
If productivity is considered as the number of images needed to
conduct a survey over an area, evidently larger sensors and greater
tilt angles will outperform smaller sensors and tilt angles (pro-
vided that there is no change in focal lengths). Nonetheless, it
has been shown that although the 45° tilt angle gives best results
from the perspective of object point precision, it is more prone to
occlusions.
The empirical part of the paper evaluated two datasets, one
from a Midas 5 camera system with small sensors (Milan dataset)
and the other recorded with an UltraCam Osprey I with large sen-
sor sizes (Graz dataset). The UltraCam Osprey I dataset outper-
forms the Midas 5 dataset in accuracy as a consequence of (i)
larger along- and cross-strip overlap, (ii) different imaging
geometry that stems from different camera internal parameters,
i.e. large sensor and short focal length versus small sensor and long
focal length, and (iii) the camera being designed for metrically
accurate mapping purposes rather than comprising integrated
cameras designed more for the consumer market. The advantage
of the UltraCam system is noticeable when considering CP accu-
racy, which was in the same range as the GCPs measurement accu-
racy. Furthermore, the magnitude of the noise in the sparse point
cloud comparisons was smaller. The tests conducted, however,
do not constitute a basis for direct comparison of the two oblique
camera systems due to (i) the different flight patterns, (ii) different
extent of the regions covered, and (iii) it was not the aim of this
paper to comment upon which of the two systems might be
superior.
Both evaluations indicate that inclusion of oblique imagery into
the bundle adjustment brings a significant gain in height accuracy,
as well as improved stability of the image block when (i) the over-
lap is small and (ii) no GNSS or well distributed GCPs are available.
Likewise, the height accuracy is improved in the self-calibrating
bundle adjustment. The characteristic deformation patterns arising
in point cloud comparisons revealed the remaining sensor-related
errors. The applied 8-parameter camera model failed to correct all
remaining systematic errors. For this reason, future work will
involve the study of other additional parameter models able to
remove all inaccuracies and thus more suitable for the employed
sensors.
Acknowledgements
This work is partially funded by the RAPIDMAP project, a Euro-
pean Union (EU) funded project in the International Cooperation
Activities under the Capacities Programme of the EU 7th Frame-
work Programme for Research and Technology Development
(CONCERT-Japan).
The authors are thankful to Blom-CGR S.p.A. (Parma, Italy) and
Microsoft UltraCam (Graz, Austria) for providing the oblique imag-
ery datasets processed in the presented work.
246 E. Rupnik et al. / ISPRS Journal of Photogrammetry and Remote Sensing 101 (2015) 233–246

References Kraus, K., 1997. Photogrammetry – Advanced Methods and Applications. Dümmler
Verlag, 466pages.
Kraus, K., 2007. Photogrammetry: Geometry from Images and Laser Scans. Walter
Aber, J.S., Marzolff, I., Ries, J., 2010. Small-Format Aerial Photography: Principles,
de Gruyter, 459 pages.
Techniques and Geoscience Applications. Elsevier, 268pages.
Kurz, F., Müller, R., Stephani, M., Reinartz, P., Schröder, M., 2007. Calibration of
Ackermann, F., 1992. Kinematic GPS control for photogrammetry. Photogramm.
wide-angle digital camera system for near real time scenarios. International
Record 14 (80), 261–276.
Archives of Photogrammetry and Remote Sensing 36(1/W51), Hannover,
Apollonio, F., Ballabeni, A., Gaiani, M., Remondino, F., 2014. Evaluation of feature-
Germany.
based methods for automated network orientation. Int. Arch. Photogramm.
Lemmens, M., Lemmen, C., Wubbe, M., 2008. Pictometry: Potentials for land
Remote Sens. Spatial Inform. Sci. 40 (5), 47–54.
administration. In: International Federation of Surveyors, 6th FIG Regional
Barazzetti, L., Scaioni, M., Remondino, F., 2010. Orientation and 3D modeling from
Conference, San Jose, Costa Rica.
markerless terrestrial images: combining accuracy with automation.
Manual of Photogrammetry, second ed., Chapter II, Part1, Aerial Cameras and
Photogramm. Rec. 25 (132), 356–381.
Accessories, ASPRS, 1952.
Cavegn, S., Haala, N., Nebiker, S., Rothermel, M., Tutzauer, P., 2014. Benchmarking
Mason, S.O., 1995. Conceptual model of the convergent multistation network
high density image matching for oblique airborne imagery. Int. Arch.
configuration task. Photogram. Rec. 15 (86), 277–299.
Photogramm. Remote Sens. Spatial Inform. Sci. 40 (3), 45–52.
McGlone, J.C., 2004. Manual of Photogrammetry. American Society for
Colwell, R.N., 1997. History and place of photographic interpretation. In: Philipson,
Photogrammetry and Remote Sensing, 1151pages.
W.R. (Ed.), Manual of Photographic Interpretation, second ed. American Society
Mishra, P., Ofek, E., Kimchi, G., 2008. Validation of vector data using oblique images.
for Photogrammetry and Remote Sensing, Bethesda, Maryland, United States,
In: Advances in Geographic Information Systems, 16th ACM SIGSPATIAL
pp. 3–47.
International Conference, Irvine, CA.
Cramer, M., 2009. Digital Camera Calibration – Report of EuroSDR Project. 262
Murtiyoso, A., Remondino, F., Rupnik, E., Nex, F., 2014. Oblique aerial photography
pages. Frankfurt a.M. 2009.
for building façade inspection and damage assessment. Int. Arch. Photogram.
Förstner, W., 1985. The reliability of block triangulation. Photogramm. Eng. Remote
Remote Sens. Spatial Inform. Sci., 40–41
Sens. 51, 1137–1149.
Nex, F., Rupnik, E., Remondino, F., 2013. Building Footprints Extraction from Oblique
Förstner, W., 1998. On the theoretical accuracy of multi image matching,
Imagery. ISPRS Ann. Photogramm. Remote Sens. Spatial Inform. Sci. II-3/W3, pp.
restoration and triangulation. In: Festschrift zum 65. Geburtstag von Prof. Dr.-
61–66.
Ing. mult. G. Konecny. Institut für Photogrammetrie, Universität Hannover.
Nocerino, E., Menna, F., Remondino, F., 2012. Multi-temporal analysis of landscapes
Fraser, C.S., 1984. Network design considerations for non-topographic
and urban areas. Int. Arch. Photogram. Remote Sens. Spatial Inform. Sci. 39 (4),
photogrammetry. Photogramm. Eng. Remote Sens. 50 (8), 1115–1126.
85–90.
Fraser, C.S., 1996. Network Design. In: Atkinson, K.B. (Ed.), Close Range
Nocerino, E., Menna, F., Remondino, F., 2014. Accuracy of typical photogrammetric
Photogrammetry and Machine Vision, Whittles, Scotland, 371p., pp. 256–281.
networks in cultural heritage 3D modeling projects. Int. Arch. Photogram.
Fraser, C.S., 1997. Digital camera self-calibration. ISPRS J. Photogramm. Remote
Remote Sens. Spatial Inform. Sci. 40-5, 465–472.
Sens. 52 (4), 149–159.
Nyaruhuma, J., Gerke, M., Vosselman, G., 2012. Verification of 3D building models
Fritsch, D., Rothermel, M., 2013. Oblique image data processing: potential,
using mutual information in airborne oblique images. ISPRS Ann. Photogram.
experiences and recommendations. Proc. 54th Photogrammetric Week, pp.
Remote Sens. Spatial Inform. Sci., I-3.
73–88.
Petrie, G., 2008. Systematic aerial oblique photography using multiple digital
Fritsch, D., Kremer, J., Grimm, A., 2012. Towards all-in-one photogrammetry. GIM
cameras. In: VIII International Scientific & Technical Conference. Porec, Croatia.
Int. 26 (4), 18–23.
Petrie, G., 2010. Current developments in airborne digital frame cameras: as
Gerke, M., 2009. Dense matching in high resolution oblique airborne images. Int.
displayed in the Intergeo 2010 exhibition. GeoInformatics 13 (8), 34–40.
Arch. Photogram. Remote Sens. Spatial Inform. Sci. 38 (3/W4), 77–82.
Pierrot-Deseilligny, M., 2014. MicMac, Apero, Pastis and Other Beverages in a
Gerke, M., Nyaruhuma, A., 2009. Incorporating scene constraints into the
Nutshell. Manual http://logiciels.ign.fr/?Telechargement,20 (Last access
triangulation of airborne oblique images. International Archives of
October, 2014).
Photogrammetry, Remote Sensing and Spatial Information Sciences 38(1-4-7/
Pierrot-Deseilligny, M., Clery, I., 2011. APERO – open source bundle adjustment
W5), Hannover, Germany.
software for automatic calibration and orientation of a set of images. Int. Arch.
Gerke, M., Xiao, J., 2013. Supervised and unsupervised MRF based 3D scene
Photogramm. Remote Sens. Spatial Inform. Sci. 38 (5/W16), 269–277.
classification in multiple view airborne oblique views. ISPRS Ann. Photogramm.,
Pierrot-Deseilligny, M., Paparoditis, N., 2006. A multiresolution and optimization –
Remote Sens. Spatial Inform. Sci., II-3/W3, pp. 25–30.
based image matching approach: an application to surface reconstruction from
Grenzdörffer, G., Guretzki, M., Friedlander, I., 2008. Photogrammetric image
SPOT 5 – HRS stereo imagery. Int. Arch. Photogramm., Remote Sens. Spatial
acquisition and image analysis of oblique imagery. Photogram. Rec. 23 (124),
Inform. Sci. 36 (1/W41).
372-368.
Rupnik, E., Nex, F., Remondino, F., 2013. Automatic orientation of large blocks of
Gruber, M., Wolfgang, W., 2014. Calibrating the new ultracam osprey oblique aerial
oblique images. Int. Arch. Photogram. Remote Sens. Spatial Inform. Sci. 40 (1/
sensor. Int. Arch. Photogram. Remote Sens. Spatial Inform. Sci. 40–3 (W1), 47–
W1), 299–304.
52.
Rupnik, E., Nex, F., Remondino, F., 2014. Oblique multi-camera systems –
Haala, N., Kada, M., 2010. An update on automatic 3D building reconstruction. ISPRS
orientation and dense matching issues. Int. Arch. Photogramm. Remote Sens.
J. Photogram. Remote Sens. 65 (6), 570–580.
Spatial Inform. Sci. 50 (3/W1), 107–114.
Honkavaara, E., Ahokas, E., Hyyppä, J., Jaakkola, J., Kaartinen, H., Kuittinen, R.,
Tang, R., Fritsch, D., Cramer, M., 2012. New rigorous and flexible Fourier self-
Markelin, L., Nurminen, K., 2006a. Geometric test field calibration of digital
calibration models for airborne camera calibration. ISPRS J. Photogram. Remote
photogrammetric sensors. ISPRS J. Photogram. Remote Sens. 60 (6), 387–399.
Sens. 71, 76–85.
Honkavaara, E., Jaakkola, J., Markelin, L., Nurminen, K., Ahokas, E., 2006b.
Wackrow, R., Chandler, J.H., 2011. Minimising systematic error surfaces in digital
Theoretical and empirical evaluation of geometric performance of multi-head
elevation models using oblique convergent imagery. Photogram. Rec. 26 (133),
large format photogrammetric sensors. Int. Arch. Photogram. Remote Sens.
16–31.
Spatial Inform. Sci. 36 (1), 56–61.
Wang, Y., Schultz, S., Giuffrida, F., 2008. Pictometry’s proprietary airborne digital
Jacobsen, K., 2008. Geometry of vertical and oblique image combination. In: Remote
imaging system and its application in 3D city modelling. Int. Arch. Photogram.
Sensing for a Changing Europe: Proceedings of the 28th Symposium of the
Remote Sens. Spatial Inform. Sci. 10 (B1), 1065–1070.
European Association of Remote Sensing Laboratories, Istanbul, Turkey.
Wiedemann, A., More, J., 2012. Orientation strategies for aerial oblique images. Int.
James, M.R., Robson, S., 2014. Mitigating systematic error in topographic models
Arch. Photogram. Remote Sens. Spatial Inform. Sci. 39 (B1), 185–189.
derived from UAV and ground-based image networks. Earth Surf. Proc. Land. 39
Xiao, J., Gerke, M., Vosselman, G., 2012. Building extraction from oblique airborne
(10), 1413–1420.
imagery based on robust facade detection. ISPRS J. Photogram. Remote Sens. 68,
Karbo, N., Simmons, G., 2007. Aerial imagery from different angles. Prof. Surveyor
56–68.
Mag. 27, 8–11.