Contents

Artistic Research 5
SUBJECT : Structural Time and Spatial Composition 5
Context 7
parametric composition 9
tessellation, resolution and scale 11
dimensions and projections 15
cinematic, simulation and dynamic 17
Implementation 21
choice, generative series 21
workow 22
Bibliography 29
Artistic Research
SUBJECT : Structural Time and Spatial Composition
Mathematics, geometry and computation are possible inspiration sources and tools leading to an intellectual
piece of work. This art piece does not need to be accurate or cartesian at all. Formalized ideas are never-
theless often used for constructing forms (gestalt) and patterns
1
which can physically be materialized in
1
James Tenney and Larry Polansky.
Temporal gestalt perception in music.
1980
architectural design. Using another medium, such materialization can also result to musical scores or purely
sonic pieces.
Each medium brings a different reading and comprehension of a sin-
gle piece; the correlation between them supports the apprehension of
the work. A piece of music using other media can be apprehended in
the same way an architectural project could be presented
2
. Beyond the
2
Kas Oosterhuis. What exactly is
nonstandard architecture? 2009
nal sound itself, the work passes thru several contexts, inspirations
and its conception. For instance, the question of contextualization con-
cerns installations, theatre and dance pieces. But it also concerns vari-
ous ways of representing the work using data visualizations
3
or even
3
Julie Steele and Noah Lliinsky. Beauti-
ful visualization. 2010
writing books for example.
Music scores are an interesting case of study particularly for pure elec-
tronic music. Both prescriptive and descriptive notations exist for this
kind of music too
4
. But their conception and use are fairly different
4
John Grayson. Sound sculpture. 1975
from scores for acoustic instrumental pieces. Beyond the fact it is also
a piece of art, the use of an electronic music score has no particular
order of reading. The simple traditional order starts from the concep-
tion, its interpretation, and the nal listening. There are no such steps
because there is no need of instrumentalist interpretation
5
. Also all
5
Olivier Messiaen. The technique of my
musical language. 1956
levels, happening in the same "box", can occur at any time of the cre-
ative and listening process. Such scores can then be the cause and the
consequence of the music piece. We have seen many examples in the
60s with graphic scores. Also today, audio-visual interactive pieces us-
ing interfaces set the player both as a listener and a composer. A huge
amount of work has been done in this direction in the past decades
and many synesthesic pieces have risen from it
6
.
6
Carsten Nicolai. Auto pilot. 2002
The project of this thesis concentrates in a slightly different direction.
The aim is to apply compositional processes both to space and time.
6
The score then becomes one entire part of the piece; it is the spatial
composition of the piece
7
. It can be presented in many forms and
7
Michel Jacobs. The art composition.
1926; and Graham Nerlich. The shape
of space, second edition. 1929
context: mostly a piece of music but also a sculpture, video, 3d print-
ing, paper... Although they are the piece of art, they still are called
score because they can be read in any direction of space, in the same
way one would read text or an evolution of patterns
8
. It is not here
8
James Gips. Shape grammars and
their uses. 1975; and Stephen Wolfram.
A new kind of science. 2002
question to recreate another generalized writing for electronic music.
The link between sound and audio does not have to be proven in a
cartesian way
9
. The reader or listener discovers a world by himself
9
Richard E Cytowic. Synesthesia. 2002
while listening or reading evolutions. His perception and interpreta-
tion would maybe have more signicance than the actual audio-visual
connections of the piece
10
.
10
Jean-Claude Risset. Rhythmic para-
doxes and illusions. 1997
The text has the tendency to be didactic and describe a true theory
in physics, psychoacoustics, musicology or aesthetics. It is rather one
contextualization of my work in the same way architects would present
one project with technical, social, historical and economic contexts. My
personal thoughts could possibly transport the public into a purely
abstract and unreal universe. This world is part of the art and would
eventually give one continuity of my work.
Olivier Pasquet - Berlin - July 2014
7
Context
Below are mutual researches and consequently aesthetic orientations found in pieces presented in this doc-
ument.
Tours Creations
Pieces TTF AHI ALI NBP STV ASS JAM RIB HR8 R13
Pure sonic composition X X X X X X X X X X
Spatialization (pan, wfs, ambi) X X X X X X
Spatialization (env) X X X
Generative X X X
Visualization X X
Visual composition X X
Sculpture X X
Nano-composition
Staged composition X X
TTF: Tu Tiens sur tous les Fronts, AHI: Aucun Homme nest une Isle, ALI: Aliados, NBP: Nuits Blanches Paris 2013,
STV: Steve V (King Different), ASS: A Script for Synthesis, JAM: Joyeux Animaux de la Misre, RIB: Rib, HR8: hr
8798, R13: r136a1
Research subjects in the table are set in a specic order; from the starting point with a traditional approach
to composition, to the nal point of the research where composition is at its widest denition. "X" should
have the tendency to go down the table in a near future. That would mean I am reaching the radicality and
formal depth I am looking for. One difculty and one question is beyond technical aspects; should the "X"
disappear from the top of the table ? Should I give up my current work or nd a clever way to link both
aesthetics.
Pure sonic composition is the most traditional technique consisting on
writing for all compositional dimensions such as timbre, rhythm, dy-
namics etc. There is no focus on only one of them but they all have an
important role. For example, Philippe Manoury uses grammar or rules
to compose elements by elements
11
. The approach is one extension of
11
Philippe Manoury. Les grammaires
musicales gnratives. 2012
integral serialism.
Spatialization (pan, wfs, ambi) adds spatialization to the writing. Sound
displacement are composed as any other compositional dimension as
John Chowning did in a simple but efcient way in Stria in 1977
12
.
12
Matteo Meneghini. Stria, by john
chowning. 2003
There are many tools for composing movements of sounds. There are
also several techniques for properly perceiving sounds sources and
their existence in a specic space.
8
Spatialization (env) makes use of envelope spatialization in which sounds
are reduced to quantic entities close to the limits of perception and
understanding. For example, Emmanuel Nunes made brilliant use of
envelope spatialization although he was not really consider extending
it to its limits
13
.
13
Franois Xavier Fron, Julien
Boissinot, and Catherine Guastavino.
Upper limits of auditory rotational
motion perception. 2009
Generative shows pieces where some formal and symbolic generation
is used. The result tends to be an evolution of short quantic rhythmic
patterns
14
.
14
Tony Myatt. Processes and systems in
computer music; from meta to matter.
2011
Visualization is the line for pieces which are in some extend connected
to a visual part from analysis. I am not planning to work on this part
already explored by many other artists.
15
15
Pierre Boulez. Das fruchtfeld. paul
klee. 1989
Sculpture shows pieces which are not only sonic but also visual. Both
media are being read or listen as if the score was the piece itself. An
important part of the composition processes is being done before the
choice of medium. The nal piece is an audio/visual object. This is the
true architectural part
16
.
16
Valentina Croci. Techniques and
technologies in morphogenetic design.
2010
Nano-composition shows pieces also using quantic rhythmic patterns
often close to the limits of perception and understanding. But they
are not always sound pieces. They are composed using synthetic biol-
ogy and/or nanotechnologies
17
. Such pieces are composed integrally:
17
Eshel Ben Jacob. Bacteria harnessing
complexity. 2010; and IBM. A boy and
his atom, the worlds smallest movie.
2013
from a microscopic to a macroscopic scale. Time scale for sonic parti-
cles and pitches, space scale for quantic structures. These works can
be presented in the form of installations with or without sound.
Staged composition are pieces using approaches described previously
but, this time, with a confrontation to reality. For instance, the ques-
tion of constraints from the stage, other people, and how to perform
an electronic piece rises again. Interrogations about audience, social
positions, interfaces between thought and physicality are back. Those
confrontations can then be tools for more societal subjects as it is often
the case
18
.
18
Stamatia Neofytou-Georgiou. The
semiotics of images in romeo castel-
luccis theatre. 2010
9
parametric composition
Figure 1: jtol.bach.humanize
randomizes rhythmic trees on a
grid
I have written a Max library for rhythmic trees with the help of JT Rinker. I started this collection of tools
before starting the doctorate but it is still under progress. It is a library dedicated to real-time pattern gen-
eration and can be applied to architectural design, music, dance; everything requiring constructed evolution
in space and time. It deals with multi-scaling and multi-dimensions where rhythm is considered to be a
skeleton onto everything else is attached (pitches, params...).
Figure 2: Meter induction
A pulse stream is deducted from another stream of discrete events. Pul-
sation (felt or not felt) induces a metrical structure which can be or-
ganized then represented using a particular data structure. Each ratio
before a new parenthesis level is the total duration for what is inside
this parenthesis. For instance, simple musical tuplets would be ex-
pressed in the following way:
( 1 ( 1 1 1 ) ) => ( 1/3 1/3 1/3 ) => ( 0.333 0.333 0.333 ) ( 1)
Rhythms are often represented with rationals because they can easily
be apprehended by humans; this is only one reason. This representa-
tion has its limits from a mathematical or computational points of view
when formal problems occur. Also, the quantic and fractal disconti-
nuity of meter can sometimes generate very complicated situations for
improvised, non western or oral transmitted music
19
but this is not
19
Gilbert Nouno. Suivi de tempo
appliqu aux musiques improvises.
2008
the case when music is played by a machine. An algorithm would have
problems dening and nding onsets in a pulse stream but it would
be much easier to accurately position events, accents etc.
If time is segmented using bars, each bar and its contained discrete
events can be set on a circle [3], a spiral or hyper-spiral. Rotation rep-
resents time. The duration of a bar can be either set by the sum of
10
what it contains. It can also be independent and scale all inside dura-
tions according to a xed duration. This second approach is used for
rhythmic tree representations and can sometimes be very useful when
durations are xed by factors external to the inside of the bar.
Figure 3: The duration of a bar
can be represented by the cir-
cumference of a circle for in-
stance
We are here talking about limits, continuity, integral calculus and
moreover differential equations for complex situations like composi-
tion using several parameters. For instance, problems like homorhythm
can make use of such mathematical, geometrical tools. A function is
considered as an innity of points innitely close to each-other at a mi-
croscopic scale. It is indeed relatively easier to nd one common onset
from several rhythms using a function rather discrete points connected
to each other. In the same way quantication or swing techniques can
use similar intellectual tools
20
; either a musical and empirical ap-
20
Carlos Agon, Gerard Assayag, Joshua
Fineberg, and Camilo Rueda. Kant: a
critique of pure quantication. 1994
proach or a more formal approach using discrete or continuous rep-
resentations. The question to go from discrete to continuous and vise
versa involves musical decisions according to musical needs. It is a
question of mapping; in a similar way we connect physical sensors to
musical parameters.
Instead of composing using pulses, it is then possible to compose using
"curves". The general term "curves" is here used as a generic geomet-
ric or articulative term. These "curves" could for example be functions
(bijective or not) or parametric equations. They are described with
equations and parameters
21
. I call this parametric composition because
21
Dmitri Tymoczko. A geometry
of music. 2011; and Iannis Xenakis.
Formalized music. 1992 it can have deep esthetic musical implications as well as the revolu-
tionary piece Metastaseis from Iannis Xenakis.
Connections with structuralism in mathematics can be seen. The ap-
proach to jtol is more orientated towards a discrete "new complexity"
rather than continuity
22
. A tree representation is indeed similar to
22
The concept is closed to the ENP
notation front end in PWGL or om:tree
in Open Music
theoretic rhytmic techniques from Brian Ferneyhough or the music
from Richard Devine who both make intense use of nested irregular tu-
plets and combinatorial processes
23
. The idea of complexity interests me
23
Mika Kuuskankare and Mikael
Laurson. Expressive notation package.
2004
more from the point of view of perception rather than its direct techni-
cal construction. Only the result count, so random or stochastic processes
11
are part of it. Also, the notion of continuum from set theory concep-
tually and perceptively makes links between discrete and continuous
rhythm
24
. Visual rhythmic layers in the piece ??hr 8798, described ear-
24
Gyrgy Ligetis sound piece Contin-
uum (1968) or Casey Reas visual piece
pfft (2014) are good examples here.
lier, are so dense one could not literally understand anything; only the
overall and perceptive understandings are still consistent. This consis-
tency maintains a continuity. In other words, discrete time onsets are
so close they perceptively build a continuous evolution of texture.
tessellation, resolution and scale
Figure 4: A deep zoom inside
my piece Kaspar V (2012) made
with deconstructive caustics pat-
terns using super high resolu-
tion imagery
Discrete visual information can be seen as raster graphics. Continuous information, based on mathematical
expressions, can be seen as vector graphics. There is here a close connection with computer music: para-
metric music would be vectorized and pulse music rasterized. Vectorization leads to the generalization and
organization of information; it eventually leads to data compression. Many artists used connections between
raster (rhythm, moirs, lines etc) and vector (interpolations etc) to create synesthetic connections; Rijochi
Ikeda or Ryoichi Kurokawa are interesting examples in that eld.
In typography, a point is the smallest whole unit of measure, being
a subdivision of the larger pica
25
. These points can be positioned or
25
Sinan Bokesoy and Jean Baptiste
Thiebaut. An approach to visualization
of complex event data for generating
sonic structures. 2008; and Martin
Solomon. The art of typography. 1986
classied onto a plane but they do not need to touch one another. It is
also the case with traditional music scores with a note, or a group of
notes. Such symbols are not always connected together. Those symbols
can sometimes take the form of nested or tiled data structures; tuplets
12
for instance. It is nevertheless different with signal, sound. Events
are positioned onto a continuous and unswerving timeline. Sound
must always t into a continuous timeline because time is entropic and
cannot physically be stopped. There cannot be "holes" into a signal;
only connected silences or discontinuities. It is different with symbolic
writing in musical elds or architectural design where it is possible to
have "jumps"
26
. Tectonics would be interesting to endeavor.
26
Sigrid Block Philippe Veenendaal
Diederik Williams Chris Adriaenssens.
Shell structures for architecture. 2014;
and Johann Rafelski and Berndt Mller.
The structured vacuum. 1985
Figure 5: Vectorized Penrose
Tiling
Tessellation is consequently necessary for signals such as sound. On
the other side, it is not an obligation for symbols such as scores. But
everything depends how signal and symbol are distinguished. Also,
the reader of symbols has the freedom to read in any direction and
dimension; graphs are therefore useful for scores. Only staffs can give
a direction for time in the case of traditional music notation.
A score describes how will eventually be the content of the nal piece.
But it is also a piece of work by itself. It is then a composition for
another composition. It is then possible to conceive a graph, with its
representation of information, as a piece by itself. It is then possible
to consider the evolution of a graph into a dened space rather simply
considering it as changes of states.
We saw there is an apparent continuum between "raster" (granular)
and "vector" (continuous) symbols when perception plays with reso-
lution
27
. For instance, zooming inside a Penrose tiling [5] allows the
27
Lindsay MacDonald. The limits of
resolution. 2010; Bob Sheil. Ad - high
denition. 2014; and Reginald Bain.
Algorithmic composition quantum
mechanics & the musical domain. 1970
reader to decrypt the information from another perspective with dif-
ferent details or a different temporality. That zooming can be discrete
or continuous. But once again here, continuity comes from the limits
of a quantized (granular) universe with minimal elements acknowl-
edged as being beyond human understanding or perception. This can
be called a quantic continuum.
Many music theories make use of a quantic approach
28
. Sound is
28
Douglas JE Nunn, Alan Purvis, and
Peter D Manning. Acoustic quanta.
1996; and Curtis Roads. Microsound.
2001
considered as granular even at its smallest scale. Granular syntheses
utilize sound grains which can be considered as quanta. It is similar
with grain from visuals (pixels, grain of picture etc). Sonic grains can
be composed onto a timeline organized in pitch, dynamics or many
other sound descriptors. Textures are created using tiling techniques
like concatenation of chosen grains together, overlapping, quantizing
onto a time grid, random positioning etc. The technique was widely
used in the 1990s
29
. It quickly becomes very musical because time is
29
The architect Michael Hansmeyer or
photographer Andreas Gursky play
with scale and multiplicity. Each micro-
element is a single system connected
to others and create something on a
macro-scale.
involved. It is now largely employed for music information retrieval or
gabor signal processing. Pure compositional techniques are used when
grains are considered as being long. The technique becomes a true
synthesis when they are short enough and reach limits of perception,
or understanding. Some composers use set theories to organize grains
13
30
. Some others create patterns in which the nal texture is made of
30
Forte Allen. The structure of atonal
music. 1977; and Paul Riker. The
serialism of milton babbitt. 2010
both grains and their carrier, envelope or grid for instance. Alike for
clothes which prints are also sometimes dependent on textile itself
31
.
31
Yohji Yamamoto often concentrates on
the feel and touch of the clothing. The
early digital artist Otto Beckmann uses
tv pixels or laser dots as visible carrier.
On one hand, synthesis techniques can be involved in signal processing.
On the other hand, compositional techniques can rather be part of sym-
bolic processing. I explained earlier there can then be a quantic con-
tinuum between both processes. This continuum can be represented
by a graph with a nested structure. Quanta are joined together by
rules of inference. These rules can for instance be a bayesian network
or a grammar network
32
. The overall is an organization containing
32
Edoardo Acotto and Moreno An-
dreatta. Between mind and mathemat-
ics. different kinds of computational
representations of music. 2012 smaller sets of other organizations [6]. Each set, or cell, smaller or
with the same size, is considered as a scale.
A change of scale is an iteration in the compositional process
33
. We
33
Przemyslaw Prusinkiewicz and
Aristid Lindenmayer. The algorithmic
beauty of plants. 1990
talk about computation where time is unreeled from calculation.
Figure 6: multi-scalar and multi-
order organization of informa-
tion. Hidden markov models
are an exception because it only
utilizes probabilities. In our
case, pattern cells can be gener-
ated by anything and are related
with others with rules. Mini-
mal patterns when zoomed at
the very bottom of the scale are
quanta(e).
For instance, fractal elements occur when rules are all the same ac-
cross scales. So fractals are exceptions because they always have self-
similarity or scale invariance between scales. Stephen Wolframs cellu-
lar automata or formal grammar Lindenmayer system are interesting
procedural generators within cells but they rarely totally fulll the
conditions to generate an entire piece [8].
Scale Relativity is a theory of space-time initially developed by Lau-
rent Nottale
34
. Erwin Schrdingers work is also involved. It is an ex-
34
Laurent Nottale. The theory of scale
relativity. 2009
tension of the special and general relativity applied to physical scales
(time, length etc) and aims to unify both relativities; a little like string
theory. It is used to predict the location of exoplanets for example.
The Planck scale is applied to mass, time and length. It is the small-
est observable possible scale not only because it maybe is the actual
14
Figure 7: The grey section corre-
sponds to what is understood by
humans. Physics breaks down
beyond the lines because the
model is still not accurate. There
is a suspect comparability with
the understanding of audio (dif-
ferent from limits of perception).
The reason comes from nature
and the way perception is hard-
coded in the human brain.
smallest scale but also because current laws of physics cannot con-
ceive further. This limit will probably be pushed away with another
"understanding" of the world. There is an alikeness with audio; sound
or music [7]. Beyond perception limits, there are boundaries for the
brain to understand the structure or characteristics of an audio signal.
It takes time to apprehend a intensity as much as it needs size to t
mass. There is some kind of "incompressibility of energy". If the Plank
scale were maybe applied to audio grains, the quantic continuum would
then be a closed space. The reason of similitude is not actually linked
with physics. It is rather linked with something more general; it comes
from nature and the way perception is hardcoded in the human brain
35
.
35
Noam Chomsky. Syntactic structure.
1957
Figure 8: My piece 8799c (2013)
is not made with cellular au-
tomata but it involves an evolu-
tion of patterns using computa-
tion. Zooming in or out creates
interferences (moir with pixels
from the screen). Density of dots
creates grey levels with a zoom-
out.
15
Figure 9: Mesh of a minimal
surface made with the rhino 3d
software.
dimensions and projections
As described earlier, a graph is both a score and a visual piece itself. It is then not only a visualization
but also an evolutive form dened by a coordinate system and parameters. It is a parametric composition
as I dened earlier. The notion of parametric equation is generalized to surfaces, manifolds and algebraic
varieties of higher dimension. For example, a strait line in a 2d space is equivalent to a continuous change
of one dimension compared to the other. This phenomenon is similar to signals like frequency versus
time. It is different in the case of symbols where frequency and time are naturally de-correlated. They
are articially linked together by an ordering structure build by the composer. The piece ends up being
this organization. This structure is then a multidimensional object in a space which cartesian coordinates
correspond to compositional parameters
36
. The objects are made of form, or gestalt, which can also change
36
Linda Dalrymple Henderson. The
fourth dimension and non-euclidean
geometry in modern art. 1983
over time. Several versions of a same object would just make a series of objects that could then be visualized
in a movie. It could also become a series of several music pieces. I am very interested by series because it
raises questions in relation with industrialization of music.
These forms can be constructed using various kinds of generative al-
gorithms one can nd in music composition or architectural design
37
.
37
George Stiny and James Gips. Shape
grammars and the generative spec-
ication of painting and sculpture.
1969
Morphogenesis (biological process that causes an organism to develop
its shape) is for instance a great source of inspiration. John Frazer for-
malized reptilian structures as a tiling system. Also, the famous Karl
Sims was a pioneer on articial life and composed evolving creatures
using genetic algorithms
38
.
38
John Frazer. An evolutionary archi-
tecture. 1995; and Karl Sims. Evolving
virtual creatures. 1994
Unfortunately, very few composers use such inspiration and instead
mostly use basic generators of form. One of the reason comes from
cultural background and lack of open-mindedness of institutions. An-
other reason comes from the very strong difculty to associate gen-
erative concepts with a musical interest. Strangely enough, using a
16
singular generative system ends with a result either too complex or
too simple to apprehend. It is a question of "good" ratio; harmony
between complexity and simplicity. It is also a main issue concerning
synesthesia; artistic (constrained) choices and concessions have to be
made between the triangle "generator" & "audio" & "visual". This sub-
ject is about mapping and remains one of the main artistic question
raised in this thesis.
Forms can then be projected onto a coordinate system. They can also
be projected to another form like a plane or a sphere for instance. A
projection can be seen as a reduction of dimension which is a reduc-
tion of parameters for music. The object does not have to belong to
an euclidian space; space deformation could deform the entire object
hence its projections onto the coordinate system, hence its parameters.
Projection also happens when a reader watches a visual score. In that
case, the projection, point of view, can be set to a plane in the same
way 3d softwares make use of virtual cameras. Positioning a camera
plane depends on what has to be shown from the object. Here also, it is
an articial correlation between the abstract understanding of reading
and the pure visual aesthetics. It is an artistic choice.
Figure 10: Dots in 3d are pro-
jected onto a plane, the viewer.
The more perpendicular is the
perspective angle of aligned
dots, the closer they are from
each other. They eventually
draw a continuous line. A part
of my piece Kaspar I (2010)
is voluntary shown here in a
small size. But it is vectorized
and can be zoomed.
We have seen one piece is one object. But since it is generative, there
can be series. Each piece of a same series do not need to have the same
duration or tessiture. The object is transformed and only morphology
remains. Here again, geometric and morphologic computations can
be used for transformations
39
. Parameters can be used for changing
39
Robert Alan Dorgan. Music in archi-
tecture / architecture in music. 1990;
and Godfried T Toussaint. Compu-
tational geometry and morphology.
1986 forms so we are still talking about parametric composition. For instance,
we saw that symbols are naturally de-correlated from signals and that
only an articial organization, composition, is connecting them. It is
easily possible to scramble coordinate systems and totally get another
result from a same form [11]. Algebraic geometry concerns the de-
scription of properties of geometric structures using algebraic expres-
sions. It involves the study of "position" with topos and the study of
form with topology
40
.
40
Christof Migone. Volume of conne-
ment and innity. 2003; and Georges
Legendre. Ad - mathematics of space.
2011
Figure 11: simple orthogonal ro-
tation of graph
17
Figure 12: parabola and cubic
curve in a projective space. In
this example, the initial space
has innite ends. The spheri-
cal one on which curves are pro-
jected is a limited space. Per-
mutation is normally a discreet
process: folding generates dis-
continuities. A projection onto a
sphere can rather be seen as con-
tinuous permutations.
More generally, music visualizations and processing often use forms
and geometry to better apprehend information in a visual way. Angle
of views, projections and dimensions changes are used a lot to trans-
pose that information into a better usable state and then use more
adapted tools. A simple example are chords or rhythms represented
or processed onto a circle [13]. A circle has been chosen because of the
relative cyclicality of musical scale or rhythms (modulo). This is also a
closed shape allowing to build and compare polygons.
Figure 13: simple permuta-
tions, shiftings, of one constant
rhythm in an eight beat bar
cinematic, simulation and dynamic
During the nineteenth century, Helmholtz studied the relationship between musical harmony and the human
perceptual apparatus
41
. His discoveries can guide an investigation into the relationship between music and
41
Hermann Helmholtz. On the sensa-
tions of tone as a physiological basis for
the theory of music. 1885
visual art. Helmholtz concluded that we appreciate the geometric progression in sound frequencies because
our ears seem to produce these overtones even in the absence of their physical presence. Time factor is
undoubtedly critical when reading, watching or listening information. Order (direction or linearity) and
speed are an important factor in the deduction of incoming signals.
The reader of those signals has a belief of understanding. This faith
drives him along what he thinks being a particular process. That pro-
cess does not have to be the one that actually generated the piece;
the reader can eventually consider whatever he wants in the way he
18
Figure 14: snapshots of a rhyth-
mic score of mine (2011). Note
the composed position of small
balls and their densities. The
piece can be read onto paper or
as a movie. Notice the impor-
tance of denition and scaling in
this piece.
wants. Temporality of reading is nevertheless forced with moving im-
ages pieces, traditional scores or even music. A timeline drives de-
duction in these special cases. The notion of memories then plays an
important role in inference. This is strongly the case with music even
though temporality can sometimes be considered differently; installa-
tion works for instance.
Figure 15: bubble chamber;
electrically charged particles
leave traces after collision.
If the reader is free to interpret any incoming information, he can be-
lieve in actual scores but also in any other traces lefts by natural and
unartistic processes; a "score" from nature [15]. He can also consider
data visualization or any other synesthetic object as a score although it
is not always the case. Level of complexity also has an important role
in the predictability of events or patterns. This is one of the reason why
randomness is widely used for composing seemingly complex struc-
tures. Complexity is again here an important area for experimentation.
Figure 16: 8799d a+b+c=a*b*c
(2014) triptych piece on pvc. It is
basically a 2d matrix of random
sets. Minimalism is on one hand
at its simplest and most possible
atness. On another hand, max-
imalism is at its best; the most
incompressible pattern. One can
only compare between the three
canvas; the medium and its con-
guration becomes then as im-
portant as the content itself.
Time can then be unrolled when someone reads a score. The score
can then become a music using another media but using the same in-
ferences. The score can also be dynamic and change over time. Time
19
becomes then just another dimension on which it is possible to project
parameters. Its structure then also becomes a tree structure with vari-
ous scales and rules as I described earlier.
Figure 17: This piece is based
on a single repetitive loop pat-
tern translated in a plane by
an autonomous multi-agent
system. It is again another min-
imalistic piece because only the
path is read by visitors. It is
also a "complex" piece because
of its many and seemingly
random iterations.
Another time is consequently possible; a time eventually describing
physical time. Then another time describes the time that describes
physical time etc. There is no difference between time and space in that
case. The amount of times is the order of time structure. For instance,
a movie, already working with time, can see its timing changing with
various versions. Realtime music pieces always change performance
after performance because of their input and possible internal genera-
tive elements. Working on the time of a dynamic piece (time-based) is
working on a "meta piece". Some people call it generativity
42
. There
42
For his piece Solo, Karlheinz Stock-
hausen not only dictates the interpre-
tive processes but also rules the order
of the parts in a descriptive rather than
xed way.
is nevertheless a difference between time and space at the very end of
the creative chain; when time becomes real with its constraints in the
physical world. This nal step toward reality is actually the one that
confront a work to reality, to other people judgement. It is also useful
a feedback that can also sometimes be considered for corrections of the
abstraction. This feedback loop is then another element in the entire
generative system and eventually give it life and stability.
It is also useful and inspiring to chose to separate time from space.
Not only at the end of the creative process for physical movements
like sound spatialization. I described earlier the possibility to com-
pose using topological spaces
43
or form themselves with topology
43
Guerino Mazzola. The topos of music.
2002; and Georges Legendre. Ad -
mathematics of space. 2011
44
. It is also possible to use the missing link between symbolic form
44
Robert Cogan. Imaging sonic struc-
ture. 1986
and its surroundings by adding timed elements, by adding movements
and traces (again). This has very close links with iteration. Again here,
cellular automata, self-organizing systems or other computational ma-
chines are involved. Spatial computing takes time into account in the
sense computation is space oriented
45
. 45
Louis Bigo. Reprsentations symbol-
iques musicales et calcul spatial. 2013;
and Konrad Zuse. Calculating space.
1969
Besides computational theory of mind, genomics and algorithmic trad-
ing, one of the ultimate work with computational complexity is the
simulation of the universe. N-body simulations simulate a dynamical
system of particles. It studies processes of non-linear structure forma-
tion such as galaxy laments and galaxy halos from the inuence of
dark matter. A simulation imitates a real-world process or system over
time. It becomes composition when rules that govern it are initially
set by humans. In that sense, such multi-agent system is a rich spa-
tial composition. It would be interesting to change the rules without
caring about actual physics but rather artistic results. Douglas Hofs-
tadters Gdel, Escher, Bach is not about mathematics, art, and music
but rather about how cognition and thinking emerge from well-hidden
neurological mechanisms. Those mechanisms can nevertheless spring
a totally imaginary universe materializing to series of "art" pieces.
Figure 18: Rather old but still
beautiful simulation of the
Universe from the Max-Planck-
Institut fr Astrophysik in 2005.
Implementation
choice, generative series
Media Art is a generic term for art involving media technologies. The
subject is then often about the kind of media used by the piece itself.
This is the case here since the above poetical and theoretic explanation
is part of the piece of art and has an important role. I would rather
call this conceptual art because the concept is the personal unnished
universe in which the next pieces should be taking place.
All of the above poetical and theoretic explanation is then applied to
rhythm onto score and sound. Igor Stravinsky said "There can be
no music without rhythm". I focus this work on rhythm because it
concentrates pieces to the purity of a structural skeleton. This does
not mean there is no sound at all but rather a minimalistic approach.
But sound somehow has to be part of rhythmic evolutions; little bit
as for traditional piano music. Rhythms on a nanoscopic scale can
generate macroscopic forms in the same way of nanotechnology.
There is not the intention to build a software that could be used by
anybody. It is rather the construction and organization of a workow
allowing experimentation and composition inside the vision I describe
in the previous sections of this document. Also, there is not the wish to
generalize generative music nor the writing for electronic music. This
is more the work for musicologists and it is a huge risk that should
be avoided as much as possible. Such trap exists because of the re-
lation between computers and art. Choices have then to be done in
order to make the work the opposite of a generalization. Generaliza-
tion nevertheless exists for pieces inside a series and uniqueness only
applies to series as a single piece. I have the intention to build series
because they better relate their context. For instance, Pierre Soulages
is an interesting artist piece by piece but also for the work of his life-
time. Composing series brings the opportunity to present the piece
with a richer context. I want the context, and concept, to possibly be
shown as part of the piece itself. Far from being didactic or pedagogic,
they can just be inspiration sources or something completely fake but
22
allowing the audience to "travel" and read the piece. I believe it is one
strength for architects who rarely only show a building without its so-
cial, historical and technical context. This strength must be used on
pure music for social and political reasons.
workow
Figure 19: summarize of how
information should be orga-
nized and possibly transformed
according to the vision de-
scribed in the previous parts of
this document.
[19] summarizes how information should be organized and possibly
transformed according to the vision described in the previous parts of
this document. Three main directions describe the workow the best:
Scales are constructed using different rules. These scales are in relation
one to another using possibly different rules.
Nested tree data structures are needed with the possibility to only gen-
erate or transform one layer at a time.
Form generators independently, or not, applied to each scale.
It is legitimate in that case to use Music composition tools as well
as Architecture composition tools. Tools are more or less adapted
23
to the variety of scales; dened as "microscopic" or "macroscopic" for
convenience.
Figure 20: Grasshopper "def-
inition" (external) i wrote to
export to python (written in
C#...)
Figure 21: Jtol swing engine
using Max
! " #$ %& '% &! &" ($ $& )% "! ""
*+,+-./0- 2.-.3+/+-456+7/8 7.A=B/
! " #$ %& '% &! &" ($ $& )% "! ""
*+,+-./0- 2.-.3+/+-456+7/8 09+-./0-:;</=9<>
! " #$ %& '% &! &" ($ $& )% "! ""
*+,+-./0- 2.-.3+/+-456+7/8 ?.<;+@=+?+
! " #$ %& '% &! &" ($ $& )% "! ""
Figure 22: generated hacked
AthenaCL functions.
Figure 23: Python is controlling
the entire workow.
I use Rhinoceros and Grasshopper as the main system [20]. Rhinoceros
already has a tree data structure. It generates the "macroscopic" form
of the piece by placing onsets using a rich variety of advanced genera-
tive tools. On another side, Jtol, a homemade Max library, build nested
tree rhythms on a "microscopic" (sec) or "nanoscopic" scale (samples)
[21]. This inuences both synthesis and the structure of the piece.
Those "microscopic" events are then hooked to the macroscopic onsets
built by Rhino. All resulting events are linked to a series of "articu-
lations" at any layer of the process. These "articulations" are kind of
multidimensional low-frequency oscillators or breakpoint functions. A
profound deep hack of AthenaCL is used for that. All that symbolic
data is then applied to sound engines and other systems for arrange-
ment and "improvement". CSound scores are generated and make a
very minimalistic music using simple waveforms and noises. It is the
same with SuperCollider. Ableton Live reads midi les containing
notes and controllers. Its API is also used to better controls virtual
and real synthesizers. It is then possible to compose with a less mini-
malistic and more conventional approach. The score part is built using
Processing for a minimalistic view; it cooks vertor graphics. A more
crafted version is drawn using Rhinoceros, Autodesk 3dsMax and v-
ray, a very powerful ray tracing plugin I know pretty well.
Csound, AthenaCL and Rhino now all use Python [23]. This has be-
come the main language for this workow. The choice of language is
also dependent from perennity and inter-software use. I have now be-
come very careful with programming becoming obsolete after several
years. I use Max because part of this workow could be used for live
performance thru MaxforLive devices or just patches. The situation of a
live performance with this ofine system will undoubtedly come.
The workow now seems to work properly, it is more than time to
make sounds and constraint the system even more in order to make it
a true musical instrument. Already interesting questions have risen :
The complex relation between the actual sound and its score. This is
an ancestral and pure synesthesic question. How to make something
visually readable, interesting and audible and understanding at the
same time. The easy and pretty way is to stay minimalistic. I have the
intention to use scaling to make it readable, minimalist and rich and
complex at the same time. The most obvious direction is the construc-
tion of spectrograms. They are a visual representation of the spectrum
of frequencies in a sound as they vary with time. Only 3 parameters
are visible on the score. This technique is the most efcient for the ap-
prehension of a piece in which rhythmic structure is important. Other
24
parameters than pitch, time and amplitude could be used; parameters
from a parametric composition could easily be seen 3 by 3. The next
step would be nding other ways to add and reduce (by projection)
dimensions.
Figure 24: One polyphonic
quanta generated with Python
inside Csound.
In the case of a simple spectrogram, events from Rhino are in a space
and not onto a timeline. We have seen a timeline could not have dis-
continuities and a symbolic space could have. Events can be placed
anywhere with Rhino... This rises a very deep question about hori-
zontality and the continuity of musical time. It is also a huge question
about verticality: how events are vertically tting together.
Another question is the way such piece has to be performed. The
fact a visual score can sometimes be provided for reading opens the
performance to a variety of formats such as installations etc. I have
the wish to compose series with two versions each. A-side would be
the "rich" version coming from Live. B-side would be the synchronized
skeleton coming from CSound. The audience could then compare both
pieces as for a diptych. Comes then the question of performance of
two pieces to be compared ! Again here, it is media art in the sense
it is about the medium itself, how a piece is performed and its social
reception.
There are two versions: one "matrix" which is only the structure and
another one. The "matrix" sounds like typical computer music from the
1970-1980s when academic computer music was not yet taken over by
mixed music in institutions. Pieces (pitched based...) with formalized
structures were fashionable. I am heading to this direction with nowa-
days technology: mostly calculation power, synthesis techniques and
speakers quality. I could add artifacts, a grain from the medium, such
as various kinds of background noises and short dynamics.
Figure 25: Extended phyllotaxis
spiral with Grasshopper and
Rhino. This example does not
have any micro-rhythms hooked
to events; so only one scale (or-
der one).
Figure 26: Out-coming synthe-
sized spectrogram with simple
sine-waves from CSound (the
buzz opcode is actually used).
25
Figure 27: Other synthesized
spectrograms. One with only
one pitch all along the piece
but many changes of metric in
a microscopic scale. Another
one with a visible ternary form.
Again two others with phases.
None have any micro-rhythms
hooked to events yet. The other
version, coming from midi les,
and the API from Live, allow the
control of a wide variety of syn-
theses like virtual but also gear
synthesizers. Those gears could
be an opening to the question of
relation with a live control and
performance.
List of Figures
1 jtol.bach.humanize randomizes rhythmic trees on a grid 9
2 Meter induction 9
3 The duration of a bar can be represented by the circumference of
a circle for instance 10
4 A deep zoom inside my piece Kaspar V (2012) made with decon-
structive caustics patterns using super high resolution imagery 11
5 Vectorized Penrose Tiling 12
6 multi-scalar and multi-order organization of information. Hidden
markov models are an exception because it only utilizes proba-
bilities. In our case, pattern cells can be generated by anything
and are related with others with rules. Minimal patterns when
zoomed at the very bottom of the scale are quanta(e). 13
7 The grey section corresponds to what is understood by humans.
Physics breaks down beyond the lines because the model is still
not accurate. There is a suspect comparability with the under-
standing of audio (different from limits of perception). The rea-
son comes from nature and the way perception is hardcoded in
the human brain. 14
8 My piece 8799c (2013) is not made with cellular automata but it
involves an evolution of patterns using computation. Zooming in
or out creates interferences (moir with pixels from the screen).
Density of dots creates grey levels with a zoom-out. 14
9 Mesh of a minimal surface made with the rhino 3d software. 15
10 Dots in 3d are projected onto a plane, the viewer. The more per-
pendicular is the perspective angle of aligned dots, the closer they
are from each other. They eventually draw a continuous line. A
part of my piece Kaspar I (2010) is voluntary shown here in a small
size. But it is vectorized and can be zoomed. 16
11 simple orthogonal rotation of graph 16
28
12 parabola and cubic curve in a projective space. In this example,
the initial space has innite ends. The spherical one on which
curves are projected is a limited space. Permutation is normally
a discreet process: folding generates discontinuities. A projec-
tion onto a sphere can rather be seen as continuous permuta-
tions. 17
13 simple permutations, shiftings, of one constant rhythm in an eight
beat bar 17
14 snapshots of a rhythmic score of mine (2011). Note the composed
position of small balls and their densities. The piece can be read
onto paper or as a movie. Notice the importance of denition and
scaling in this piece. 18
15 bubble chamber; electrically charged particles leave traces after
collision. 18
16 8799d a+b+c=a*b*c (2014) triptych piece on pvc. It is basically a
2d matrix of random sets. Minimalism is on one hand at its sim-
plest and most possible atness. On another hand, maximalism
is at its best; the most incompressible pattern. One can only com-
pare between the three canvas; the medium and its conguration
becomes then as important as the content itself. 18
17 This piece is based on a single repetitive loop pattern translated in
a plane by an autonomous multi-agent system. It is again another
minimalistic piece because only the path is read by visitors. It is
also a "complex" piece because of its many and seemingly random
iterations. 19
18 Rather old but still beautiful simulation of the Universe from the
Max-Planck-Institut fr Astrophysik in 2005. 19
19 summarize of how information should be organized and possi-
bly transformed according to the vision described in the previous
parts of this document. 22
20 Grasshopper "denition" (external) i wrote to export to python
(written in C#...) 23
21 Jtol swing engine using Max 23
22 generated hacked AthenaCL functions. 23
23 Python is controlling the entire workow. 23
24 One polyphonic quanta generated with Python inside Csound. 24
25 Extended phyllotaxis spiral with Grasshopper and Rhino. This
example does not have any micro-rhythms hooked to events; so
only one scale (order one). 24
26 Out-coming synthesized spectrogramwith simple sine-waves from
CSound (the buzz opcode is actually used). 24
29
27 Other synthesized spectrograms. One with only one pitch all
along the piece but many changes of metric in a microscopic scale.
Another one with a visible ternary form. Again two others with
phases. None have any micro-rhythms hooked to events yet. The
other version, coming from midi les, and the API from Live, al-
low the control of a wide variety of syntheses like virtual but also
gear synthesizers. Those gears could be an opening to the ques-
tion of relation with a live control and performance. 25
Bibliography
Edoardo Acotto and Moreno Andreatta. Between mind and math-
ematics. different kinds of computational representations of music.
2012.
Sigrid Block Philippe Veenendaal Diederik Williams Chris Adri-
aenssens. Shell structures for architecture. 2014.
Carlos Agon, Gerard Assayag, Joshua Fineberg, and Camilo Rueda.
Kant: a critique of pure quantication. 1994.
Forte Allen. The structure of atonal music. 1977.
Reginald Bain. Algorithmic composition quantum mechanics & the
musical domain. 1970.
Louis Bigo. Reprsentations symboliques musicales et calcul spatial.
2013.
Sinan Bokesoy and Jean Baptiste Thiebaut. An approach to visualiza-
tion of complex event data for generating sonic structures. 2008.
Pierre Boulez. Das fruchtfeld. paul klee. 1989.
Noam Chomsky. Syntactic structure. 1957.
Robert Cogan. Imaging sonic structure. 1986.
Valentina Croci. Techniques and technologies in morphogenetic de-
sign. 2010.
Richard E Cytowic. Synesthesia. 2002.
Linda Dalrymple Henderson. The fourth dimension and non-
euclidean geometry in modern art. 1983.
Robert Alan Dorgan. Music in architecture / architecture in music.
1990.
Franois Xavier Fron, Julien Boissinot, and Catherine Guastavino.
Upper limits of auditory rotational motion perception. 2009.
32
John Frazer. An evolutionary architecture. 1995.
James Gips. Shape grammars and their uses. 1975.
John Grayson. Sound sculpture. 1975.
Hermann Helmholtz. On the sensations of tone as a physiological
basis for the theory of music. 1885.
IBM. A boy and his atom, the worlds smallest movie. 2013.
Eshel Ben Jacob. Bacteria harnessing complexity. 2010.
Michel Jacobs. The art composition. 1926.
Mika Kuuskankare and Mikael Laurson. Expressive notation pack-
age. 2004.
Georges Legendre. Ad - mathematics of space. 2011.
Lindsay MacDonald. The limits of resolution. 2010.
Philippe Manoury. Les grammaires musicales gnratives. 2012.
Guerino Mazzola. The topos of music. 2002.
Matteo Meneghini. Stria, by john chowning. 2003.
Olivier Messiaen. The technique of my musical language. 1956.
Christof Migone. Volume of connement and innity. 2003.
Tony Myatt. Processes and systems in computer music; from meta to
matter. 2011.
Stamatia Neofytou-Georgiou. The semiotics of images in romeo
castelluccis theatre. 2010.
Graham Nerlich. The shape of space, second edition. 1929.
Carsten Nicolai. Auto pilot. 2002.
Laurent Nottale. The theory of scale relativity. 2009.
Gilbert Nouno. Suivi de tempo appliqu aux musiques improvises.
2008.
Douglas JE Nunn, Alan Purvis, and Peter D Manning. Acoustic
quanta. 1996.
Kas Oosterhuis. What exactly is nonstandard architecture? 2009.
Przemyslaw Prusinkiewicz and Aristid Lindenmayer. The algorith-
mic beauty of plants. 1990.
33
Johann Rafelski and Berndt Mller. The structured vacuum. 1985.
Paul Riker. The serialism of milton babbitt. 2010.
Jean-Claude Risset. Rhythmic paradoxes and illusions. 1997.
Curtis Roads. Microsound. 2001.
Bob Sheil. Ad - high denition. 2014.
Karl Sims. Evolving virtual creatures. 1994.
Martin Solomon. The art of typography. 1986.
Julie Steele and Noah Lliinsky. Beautiful visualization. 2010.
George Stiny and James Gips. Shape grammars and the generative
specication of painting and sculpture. 1969.
James Tenney and Larry Polansky. Temporal gestalt perception in
music. 1980.
Godfried T Toussaint. Computational geometry and morphology.
1986.
Dmitri Tymoczko. A geometry of music. 2011.
Stephen Wolfram. A new kind of science. 2002.
Iannis Xenakis. Formalized music. 1992.
Konrad Zuse. Calculating space. 1969.