S CIEntIfIC ProgrAmmIng
Editors: Konstantin Läufer, laufer@cs.luc.edu
Parallel Programming
with SkeletonS
By Joel Falcou
Parallel programming is bound to become the main concern of software developers in the coming decades.
Various models aim to solve this tension, trading efficiency for abstraction or vice versa, but how about
getting both?
B ack at the dawn of scientific
computing, parallel machines
were revered as titans that
few could approach and even fewer
could tame. Today, after decades of
progress in both hardware and soft-
ware development, parallel comput-
ing is a mainstream technique for
getting things done. However, as
this progress spawned increasingly
powerful machines, it spelled doom
for many developers.
As Herb Sutter1 stated, the free
lunch is over for sequential pro-
gramming: designers must take
concurrency and parallel program-
ming into account at every level of
software design to avoid the dreaded
“this application doesn’t scale” re-
sult. For nonspecialists, this means
that writing efficient code for such
machines or groups of machines will
become less trivial, as it usually in-
volves dealing with low-level APIs
such as OpenMP, Posix threads,
and the message-passing interface
(MPI). However, years of experience
have shown that using such frame-
works is difficult and error-prone;
deadlocks and other undesired be-
haviors make parallel software de-
velopment very slow compared to the
classic, sequential approach. As an
alternative, software design patterns
offer an easy way to build reusable,
structured sequential software com-
ponents. Similarly, various attempts
58 Copublished by the IEEE CS and the AIP
Konrad Hinsen, hinsen@cnrs-orleans.fr
have emerged to provide a structured
framework to design and implement
nontrivial parallel applications and
deliver high performance.
In this article, I present such a pro-
gramming model—Parallel Algorith-
mic Skeletons—along with a library
called Quaff that implements it in
C++ and makes parallel application
development easier.
Parallel Skeletons
in a Nutshell
The concept of parallel skeletons2,3 is
based on the observation that many
applications express parallelism in the
form of a few recurring patterns of
computation and communication. An
example of such a pattern is a pipeline,
which is a model of parallel function
composition. Another such pattern is
a data-parallel structure, in which a
master process slices data along some
dimension and distributes them to a
pool of slave processing units.
Essentially, a skeleton is a pattern
that occurs in a significant number of
parallel applications; some skeletons
are general, whereas others are specific
to an application domain. Although
design patterns are rather informal,
skeletons have been formalized to the
extent that we can express them as con-
crete language constructs or templates.
Recently, commercial users have start-
ed to show interest in these methods.
Google’s MapReduce model for dis-
1521-9615/09/$25.00 © 2009 IEEE
tributed data mining is, in fact, noth-
ing but a specific parallel skeleton.
Every community or application
domain has its own specific skeletons.
In computer vision—the example
that I use in this article—parallel
skeletons mostly involve slicing and
distributing regular data. Parallel ex-
ploration of a tree-like structure or a
randomized walk through some itera-
tion space are common in operational
research applications. The main ad-
vantage of this parameterization of
parallelism is that all low-level, ar-
chitecture, or framework-dependent
code is hidden from the user, who
only has to write sequential code
fragments and instantiate skeletons.
The skeleton approach thus provides
a decent level of organization.
Another interesting feature of skel-
etons is their ability to be nested. If
we look at a skeleton as a function
that takes functions as arguments and
produces parallel code, then any in-
stantiated skeleton is eligible to be an-
other skeleton’s argument. Skeletons
are thus higher-order functions in the
sense of functional programming—
that is, functions whose parameters
are themselves functions. To be able
to perform the transformation from a
combination of skeletons to a working
parallelized program, we need an ap-
plication model capable of represent-
ing an arbitrarily complex program as
a set of functions.4
Computing in SCienCe & engineering

As an example, consider an image-
processing application for detecting
edges in a video stream. We would
build this application from four
functions:
• load, which retrieves an image from
the video stream;
• thresh, which applies a binary
threshold to an image;
• edge, which extracts edges as lists of
lines; and
• save, which saves the result to a file
on disk.
The sequential version of our applica-
tion is defined as the sequential com-
position of these four functions:
As = sequence [load, thresh, edge, save],
where sequence represents the se-
quential composition skeleton.
We can introduce the first level of
parallelism by noticing that we can
run the four functions in parallel if we
apply them to different elements of the
data stream. Thus, while load is load-
ing the ith image, we can apply thresh
to the (i − 1)th image. In general, we
can apply the thresh function on imag-
es it, it–1, it–2, and it–3 in parallel. This
parallelization scheme—a pipeline—is
a classic yet useful form of parallelism
that we can choose as a skeleton. A first
parallel version of A is thus
A1 = pipeline [load, thresh, edge, save].
We can express another level of par-
allelism by noticing that we can run
the thresh function on different im-
age slices in parallel. So, if we define
two functions for slicing and merg-
ing images (slice and merge, respec-
tively), we can express As by using a
new parallel construction—map—
which slices an image, distributes the
may/June 2009
PIPE
Skeleton tree Process network φ φ2 φ
generation
C++
φ1 FARM3 φ3
φ2
figure 1. Quaff code generation process. the compile-time system analyzes C++
code using skeleton constructors to build a process network. this network is then
used to generate the message-passing interface (mPI) code for compilation.
slices over a pool of slave processors,
and merges the results of a function’s
parallel application (which is thresh in
our example):
A2 = pipeline [map [slice, thresh,
merge], edge, save].
We thus define the final paral-
lel application in terms of skeleton
nesting—map being nested inside a
pipeline—and the list of sequential
functions. As far as the developer is
concerned, the parallelization is done
because the skeleton implementation
will handle all the low-level com-
munication and marshaling details.
Building parallel applications is sim-
plified because the developer only
needs to know the skeletons’ opera-
tional semantics. Another advantage
is that developers can reuse existing
sequential functions directly because
they don’t need to know about the
parallelization process.
The Quaff Library
Quaff is a skeleton-based parallel pro-
gramming library whose main task
is to rely on C++ template metapro-
gramming to reduce the overhead
traditionally associated with object-
oriented implementations of such li-
braries. The basic idea is to use the
C++ template mechanism so that
skeleton-based programs expand at
compile time and generate a new C++
MPI code to be compiled and execut-
ed at runtime. This code generation
totally removes the overhead associ-
ated with runtime polymorphism
and function forwarding. To do this,
developers transform the skeleton
C+MPI code
2 2
production generation C
MPI
φ1 f φ3
structure extracted from the applica-
tion definition into a list of executable
instructions for a given parallel archi-
tecture, as Figure 1 shows.
Other research describes this pro-
cess and its associated specific lan-
guage.5 Whereas languages such as
metaOCaml or Template Haskell
natively support such constructions,
C++ requires the use (and abuse) of
template metaprogramming and op-
erator overloading.
Let’s examine the Quaff interface
and which skeletons it supports.
The Quaff Programming Model
Figure 2 presents a simple algorithm
parallelized with Quaff. In this ex-
ample, we want to apply the function
comp to a vector in a parallel way.
The actual code is split into four
parts, starting with the definition of
the user functions. The only limita-
tion is the argument ordering (input
first, output last), which is a require-
ment to enable Quaff to determine
how data should be transferred be-
tween processes. The library pro-
vides communication support for
all standard C and C++ types and
some standard template library con-
tainers—such as vector or list—thus
limiting the marshaling code we
must write to support custom types.
Next, the user initializes the parallel
execution environment at line 10 via
the initialize function. From this
point on, we can evaluate and run
skeleton expressions on the underly-
ing MPI-enabled parallel machine.
The application is a combination
of skeleton constructors on lines 12
through 14.
59
 SCIEntIfIC ProgrAmmIng
# include <quaff / quaff .hpp >
using namespace quaff ;
void load ( vector <float >& d );
void comp ( vector <float > const & d, vector <float >& r);
void save ( vector <float > const & r );
int main ( int argc , const char * argv [])
{ description of parallel applications
initialize (arg , argv );
run ( ( seq ( load )
, map <16 >( seq( comp ))
, seq ( save )
) );
finalize ();
} Application to
figure 2. Quaff sample code. In this example, we want to apply the function comp
to a vector in a parallel way.
In this example, we first load data
from a file, distribute this data over
processors using the map skeleton,
perform the computation, and gather
the results, which are then saved back
on disk. This sample code shows the
explicit call to the MAP and SEQ
skeleton constructors and the use of
the comma operator as the CHAIN
skeleton constructor. Note that we
can parameterize skeleton construc-
tors via additional information—for
instance, map takes an additional
template parameter that describes
over how many processors the data
will be distributed. Finally, the fi-
nalize function shuts down the
parallel execution environment in
line 17. Again, note that the process
of structuring the communication
and generating MPI primitive calls is
mostly done at compile time thanks
to metaprogramming, leading to a
very small overhead (a few percent)
compared to a manual implementa-
tion of the same application.
Supported Skeletons
Because we usually define skeletons by
generalizing the parallelization pat-
terns that arise from the implementa-
tion of specific application classes, no
single standard list of skeletons exists.
Quaff supports a small subset of skel-
60
etons that the community agrees on
so far:
• The SEQ skeleton encapsulates
user-defined sequential functions
to use them as parameters of other
skeletons.
• The CHAIN skeleton calls other
skeletons in sequence.
• The PARDO skeleton supports ad
hoc parallelism; it simply spawns
parallel processes with no defined
communication scheme.
• The PIPELINE skeleton is func-
tionally equivalent to parallel func-
tion composition.
• The FARM skeleton models irreg-
ular, asynchronous data parallelism
in which a master process dynami-
cally distributes inputs to a pool of
slave processors using some param-
eterizable heuristic.
• The MAP skeleton models regular
data parallelism that divides the
input data into subsets on which
a given function is applied. The
MAP skeleton then merges back
the subset results to produce the
total output.
All these skeletons are directly us-
able in Quaff via the correspond-
ing function. Some of them are also
mapped to operators to make the
easier—for instance, the CHAIN,
PIPE, and PARDO skeletons are re-
spectively mapped to the comma op-
erator, the bitwise or, and the bitwise
and operators.
Computer Vision
To demonstrate Quaff’s expressive-
ness and efficiency, we can parallel-
ize a realistic application from the
domain of computer vision. Com-
puter vision features various com-
plex, time-consuming algorithms
and is a field of choice for paralleliza-
tion. The application I’ve chosen
performs object detection and track-
ing in a stereoscopic video stream us-
ing a probabilistic algorithm. In this
approach, detecting and tracking is
done by computing a posteriori the
probability density of a state vec-
tor representing the tracked object
from a series of observations follow-
ing a standard Bayesian inference
procedure. To solve such a problem,
we can either analytically solve the
Chapman-Kolmogorov prediction
equation or use a Monte Carlo meth-
od. The particle filter algorithm
models such a probabilistic proce-
dure with a Markov process by esti-
mating the probability density by a
weighted, discrete set of observations
inside the observation universe.6
In our case, we want to track a pe-
destrian in a 3D world. To do so, we
try to estimate the probability density
of a set of particles containing the pe-
destrian’s 3D position and 3D velocity
vector. Those particles also have an
evolution model that represents how
we can model a pedestrian particle
between frames. No pedestrian can
run faster than the speed of sound,
thus capping the velocity vector’s
norm, and no pedestrian can teleport
Computing in SCienCe & engineering
 # include <quaff / quaff .hpp >
using namespace quaff ;

# define NPROC 16
typedef std :: vector < particles > data_t ;

void gui ();

between frames, thus providing a
continuity equation on position.
Our implementation of the particle
filter algorithm uses four functions:
void generate ( data_t & );
void measure ( image const &, data_t const &, data_t & );
void sample ( data_t const &, data_t & );
particles estimate ( data_t const & );
void update_gui ( particles const & );

• generate builds the particle dis- int main ( int argc , const char * argv [])
{
tribution from the last iteration
initialize (arg , argv );
results,
• measure extracts features from the run ( seq ( gui )
video stream to evaluate each par- & ( map <NPROC >( ( seq ( generate )
ticle’s interest score by using an im- , seq ( measure )
age descriptor, , seq ( sample )
• sample resamples the particle set )
by replicating particles with large )
weight and trimming particles with , seq ( estimate )
, seq ( update_gui )
small weight, and
)
• estimate computes the particle );
set’s average to get the current finalize ();
frame estimation. }

This algorithm’s parallelization is

based on distributing the particle set figure 3. Quaff code for the particle filter application. After starting up the parallel
over the available processors and then environment, the application is split into two parts: the gUI handling and the
computing the estimated position of actual computation code that follows the algorithm.
the object on the root node, which
also runs the application’s GUI parts.
Figure 3 presents the Quaff listing
for this application, and Figure 4 shows
a sample execution of our 3D tracking Left Right
image image
application. The upper part of the fig-
ure shows the two video streams and
the projection of the particle distribu-
tion, whereas the lower part shows the
estimated pedestrian’s 3D path pro-
jected on the ground plane.

Estimated

P arallel programming is bound to

become an everyday problem for
a large part of the scientific comput-
Particles
distribution
projection
3D path

ing community. These developers

will have to struggle with increasingly
complex architectures and will need
tools to overcome these difficulties.
Parallel skeletons could offer a solu- figure 4. Sample tracking session. the top of the screen shows the right and left
tion to several problems. Because it’s image provided by the cameras; a subset of the particle set and the estimated
an easy-to-use and efficient way of position are projected (the yellow and red boxes). the lower part shows a bird’s-
building parallel applications, this eye view of the reconstructed pedestrian path.

may/June 2009 61
 SCIEntIfIC ProgrAmmIng

Other Parallel main idea is to generate a process topology from the con-

SkeletOn librarieS struction of various skeleton classes and use a distributed

container to handle data transmission. this polymorphic

S everal implementations of the parallel skeleton para-

digm are available for various programming languages.
they show that the skeleton paradigm is indeed easy to
integrate into mainstream languages.
the Eskel Library by murray Cole represents a concrete
attempt to embed the skeleton-based parallel program-
ming method into the mainstream of parallel program-
ming. It offers various skeletal parallel programming
constructs that resemble mPI primitives and are directly
usable in C code. However, Eskel’s low-level API requires
the programmer to take care of internal implementa-
tion details. Eskel and a lot of seminal information about
skeletons are available at http://homepages.inf.ed.ac.uk/
abenoit1/eSkel/.
Herbert Kuchen based mUESLI, the C++ münster Skel-
eton Library, on a platform-independent structure. the
C++ skeleton library offers a high level of abstraction but
stays close to a language that’s familiar to many developers.
moreover, the C++ binding for higher-order functions and
polymorphic calls ensures that the library is type safe. the
main problem is that the overhead due to dynamic poly-
morphism is rather high (between 20 and 110 percent for
rather simple applications). A prototype is available at www.
wi.uni-muenster.de/PI/forschung/Skeletons/index.php.
finally, Lithium is a Java skeleton library for clusters and
grid-like networks of machines whose underlying execu-
tion model is a macro data-flow one. the choice of Java
is motivated by the fact that it provides an easy-to-use,
platform-independent, object-oriented language. Lithium
also spawned ASSISt, a skeleton-based framework for
grid computing. Lithium is available at www.di.unipi.it/
marcod/Lithium/.

Table 1. Timing results for various particle sets.

Np = 1,000 Np = 2,000 Np = 5,000 Np = 10,000
P=1
time 672.30 ms 1,986.19 ms 11,783.58 ms 21,142.71 ms
P=2
time 409.94 ms 1,115.84 ms 6,074.01 ms 10,898.30 ms
Speedup 1.64 1.78 1.87 1.94
Efficiency 82% 89% 93.5% 97%
P=4
time 263.65 ms 711.90 ms 3,527.45 ms 6,110.61 ms

Speedup 2.55 2.79 3.22 3.46

Efficiency 63.7% 79.5% 80.5% 96.5%
P=6
time 187.79 ms 467.34 ms 1,975.37 ms 3,645.29 ms
Speedup 3.58 4.25 5.75 5.80
Efficiency 59.7% 70.8% 95.8% 96.7%
P = 12
time 162.00 ms 291.23 ms 1,046.86 ms 1,828.95 ms
Speedup 4.15 6.82 10.85 11.56
Efficiency 34.6% 56.8% 90.4% 96.3%
P = 28
time 108.61 ms 170.05 ms 512.79 ms 786.27 ms
Speedup 6.19 11.68 22.15 26.89
Efficiency 22.1% 41.7% 79.1% 96.1%

model can solve many of the com- munications and synchronization. they’re applicable, they provide a con-
mon issues associated with parallel Parallel skeletons won’t be the uni- venient way to describe computational
programming, such as handling com- versal solution for everyone, but when problems and solve them efficiently.

62 Computing in SCienCe & engineering

 Lower nonmember rate
of $32 for S&P magazine!
The Quaff library shows that users IEEE Security & .NET SEcuriTy • iNTErviEw wiTh MEliSSa haThaway

can operate such a model in a main- Privacy is THE

stream language with a high level of premier magazine NovEMbEr/DEcEMbEr 2008
voluME 6, NuMbEr 6

abstraction and without efficiency for security

loss. Currently, Quaff targets clus-
ters, multicore machines, and the
professionals.
Cell processor, with only small differ-
ences in the interface. The goal is to
develop skeleton-based applications Top security
on heterogeneous platforms with a professionals
single source code—architectures
GooGle’s Android PlAtform • risk Assessment for norwAy’s infrAstructure

in the field share

such as the IBM RoadRunner cluster
of Cell-enhanced multicore cluster information JAnuAry/februAry 2009

on which
Volume 7, number 1

nodes, for example.

you can rely:

References
1. H. Sutter and J. Larus, “the free Lunch Is • Silver Bullet podcasts
over: A fundamental turn toward Concur-
rency in Software,” Dr. Dobb’s J., vol. 30, no. and interviews
3, 2005; www.ddj.com/web-development/ • Intellectual Barack’s BlackBerry Bind • education via second life

184405990. Property
2. m. Cole, Algorithmic Skeletons: Structured Protection
Management of Parallel Computation, mIt March/april 2009

Press, 1989.
and Piracy volume 7, numBer 2

3. D.B. Skillicorn, “Architecture-Independent

• Designing for
Parallel Computation,” Computer, vol. 23, no. Infrastructure Security
12, 1990, pp. 38–50. • Privacy Issues
4. m. Aldinucci and m. Danelutto, “Stream Par- • Legal Issues and
allel Skeleton optimization,” Proc. 11th Int’l
Cybercrime
Conf. Parallel and Distributed Computing and
Systems, ACtA Press, 1999, pp. 955–962 • Digital Rights
5. J. Sérot and J. falcou, “functional meta- Management
Programming for Parallel Skeletons,” Proc. • The Security Profession
Int’l Conf. Computational Science, Springer-
Verlag, 2008, pp. 154–163.
6. J. falcou et al., “real time Parallel Imple-
mentation of a Particle filter Based Visual Visit our Web site at
tracking,” Workshop on Computation In-
tensive Methods for Computer Vision, 2006,
www.computer.org/security/
CD-rom.

Joel Falcou is an assistant professor at the

University Paris-Sud and researcher at the Subscribe now!
Laboratoire de recherche d’Informatique
in orsay, france. His work focuses on in-
vestigating high-level programming mod-
els for parallel architectures (present and
www.computer.org/services/
future) and providing efficient implemen- nonmem/spbnr
tation of such models using high-perfor-
mance language features. Contact him at
joel.falcou@lri.fr.

may/June 2009 63