Documente Academic
Documente Profesional
Documente Cultură
BIBLIOGRAPHY 199
FIGURES 209
Introduction
The dissertation presents a history of modern ideas about vision. I believe that vision is
not a timeless concept; rather, each period understands vision differently depending on how it is
used. In the twentieth century, vision acquired new roles as the medium of mass communication
and the instrument of labor, and, as any other productive tool, it was subjected to engineering,
rationalization and automation. Such new disciplines as applied experimental psychology and
cognitive science, communication engineering and film, robotics, and advertising design
continue to search for ways to utilize vision productively. In the process, they generate new
knowledge about vision, at the same time reducing it to a few disjoined and limited models. The
dissertation chapters follow the development of four such models: vision as a code, vision as a
means of logical reasoning, vision as a way to capture spatial information, and vision as
information processing.
Let us consider a few definitions of vision that are representative of entire research paradigms
human perception carried out at the MIT Artificial Intelligence Laboratory. This book has been
the most influential account of the computational approach to vision, shared by computer
What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what
is where by looking. In other words, vision is the process of discovering from images what is
present in the world, and where it is.1
1 David Marr, Vision (New York: W.H. Freeman and Company, 1982), 3.
There is nothing "plain" about this definition of vision; it is functional and pragmatic. Visual
information about the world: the identity of objects and their positions. In fact, this is the only
kind of information which may be required for a robot or an automatic missile to perform its
task. Marr projects these goals of machine vision onto human vision. Here, vision is reduced to
the common denominator shared by humans and low level organisms: to detect an obstacle, a
predator, a prey.
In a special 1984 issue of Cognition (the leading periodical of cognitive science) devoted
to visual representations, Steven Pinker outlined the understanding of vision commonly held by
cognitive psychologists:
Certain abstract problems could be best solved by translating their entities into imagined objects,
transforming them using available image transformations, detecting the resulting spatial relations
and properties, and translating those relations and properties back to the problem domain.2
In this definition, vision is valued merely for its topological properties -- the ability to represent
such relations as inclusion, proximity, and relative positions. According to many cognitive
scientists, these properties make it a more efficient medium for problem-solving and abstract
thinking than language. And since the mind itself is imagined as an exemplary model of
computational efficiency that evolved through evolution, scientists postulate the centrality of
visual (read: topological) representations for various mental processes. To think productively and
without waste, we might dispense with language and instead think through images, submerging
Here is another model of vision that narrows its definition to a particular productive
property. In a 1951 overview of applied experimental psychology, the field which is today
What is the purpose of this research into the relative properties of the senses? In contrast to a
cockpit, radar display or an automated production line, is primarily perceptual in nature. His or
her work is to monitor the displays and to detect those signals that require intervention. Starting
in World War II, experimental psychologists have collaborated with engineers in the design of
displays and perceptual strategies for their operators. One of the important questions in this
research has been the relative advantages of visual, auditory and tactile displays for the detection
and analysis of different kinds of signals. Consequently, Fitts describes vision as a sense which is
most reliable and efficient for the constant surveillance of a single source -- it is "basically
selective and intermittent." According to this model, in order to use vision most productively,
In fact, there is no single meaning of "vision" which is shared among them -- "vision" as such
does not exist. The only thing that unites these modern approaches to vision is the quest for the
efficient, reliable, and effective instruments of labor. In this respect, vision is just a set of
separate tools which can be employed by either a human or a machine to get work done.
It is this emphasis on the productive uses of vision that distinguishes my project from a
number of recent histories of vision. In 1988 Hal Foster edited the anthology Vision and
historicising modern vision, that is, rejecting it as a natural or a cultural constant.4 In the last few
years, each of the five contributors -- Martin Jay, Jonathan Crary, Rosalind Krauss, Norman
Bryson, and Jacqueline Rose -- have developed their essays into books which today largely
At the center of this discourse is the experience of a seeing subject: how is what one sees
influenced by psychic structures, by gender and power positions, and by the constraints of the
body; how is one's subjectivity determined by the experience of seeing and being seen? At its
boundaries are works of art and other cultural representations, which are interpreted as the reified
The accounts of these writers greatly expanded the horizons of our knowledge about
modern vision. Yet, what remained unexamined is the employment of vision for work. Indeed, it
is remarkable that the grounding philosophical and psychoanalytic models of vision, influential
in this discourse, are theorized from the experience of a "vacationer." For Sartre, it was the
person wandering in the deserted Parisian park; for Lacan, it was the young man (Lacan
himself), "being observed" by a shining sardine can, while he was in a boat with some fishermen
near a small port; for Freud, it was a young woman confined to isolation in a Viennese apartment
or sent to a resort. Not only are these subjects far from a modern workplace, but they are also
away from the prototypical perceptual spaces of modernity: the city street, the movie theater, the
shopping arcade.
4 Hal Foster, ed., Vision and Visuality (Seattle: Bay Press, 1988).
5 Martin Jay, Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought
(Berkeley: The University of California Press, 1993); Jonathan Crary, Techniques of the
Observer: on Vision and Modernity in the Nineteenth Century (Cambridge: The MIT Press,
1990); Rosalind Krauss, The Optical Unconscious (Cambridge: The MIT Press, 1993); Norman
Bryson, Looking at the Overlooked: Four Essays on Still Life Painting (Cambridge: Harvard
University Press, 1990); Jacqueline Rose, Sexuality in the Field of Vision (London: Verso,
1986).
It is precisely the experience of these spaces that gave rise to a very different kind of
theorizing of vision in the writings of Walter Benjamin. Scrutinizing these new spaces of
modernity, Benjamin was among the first to notice the contiguity between the perceptual
Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless, today's
pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus technology has
subjected the human sensorium to a complex kind of training. There came a day when a new and
urgent need for stimuli was met by the film. In a film, perception in the form of shocks was
established as a formal principle. That which determines the rhythm of production on a conveyer
belt is the basis of the rhythm of reception in the film.6
Here Benjamin already notices that vision became a functional and pragmatic activity rather than
a disinterested or contemplative one. The eye is trained to keep pace with the rhythm of
industrial production at the factory and to navigate through the complex visual semiosphere
Among the contributors to Vision and Visuality, Jonathan Crary is the most sensitive to
the primacy of work for the history of vision in the modern period. In Techniques of the
psychophysics, and such techniques as optical apparatuses and illusions prepared vision for its
later productive deployment. This far-reaching archeology is extremely revealing, yet, because
the account ends in the middle of the nineteenth century, it can outline the future deployment of
vision only in very broad terms: as participating in the process of the rationalization of labor and
as adapting to the new flux of dematerialized images and signs. In order to understand what
really happened to vision in the twentieth century, how it was actually deployed and investigated,
Crary's Techniques of the Observer is necessary but not sufficient. This century has seen the
emergence of disciplines and techniques of vision that simply cannot be traced to the period
6 Walter Benjamin, "On Some Motives in Baudelaire," in Illuminations, ed. Hannah Arendt
(New York: Schochen Books, 1969), 175.
analyzed by Crary. In order to understand their current directions, we need to consider social
phenomena of more recent origin such as automation, computerization, and "the information
revolution." Most importantly, what could not have been predicted from Crary's account is the
radical shift in the nature of labor itself. Benjamin described the rhythm of perceptual shocks of
the assembly line worker of the industrial era, who was nevertheless paid to perform physical
work: lifting, hammering, filing, and molding. Today, however, in the post-industrial era, the
same worker is paid for "perceiving" -- watching, detecting, scanning, and monitoring.
of vision -- computer vision and cognitive science, virtual reality and other new human-computer
interfaces. In this archeology, I try to uncover the recently accumulated layers -- 1920s, 1940s,
1960s. Rather than postulating a single paradigm shift, I describe a number of distinct
The first chapter is concerned with the model of vision as code of communication.
Investigations of the psychological effects of basic colors and elemental forms, conducted in
experimental psychology since the second half of the nineteenth century, made possible the idea
of a rational visual language composed from simple elements. This idea acquired new
importance in the 1920s, when artists found themselves in the role of the designers of mass
communication. In Soviet Russia, artists collaborated with psychologists in order to create a fully
rationalized visual code of mass communication where each visual element is capable of
Visual reasoning is the topic of the second chapter. While classical philosophy
considered vision unsuitable for reasoning, in the modern period this attitude is reversed. Freud's
intellectual montage, and the adaptation of graphic diagrams in logic and in popular visual
culture are all symptoms of this emerging understanding of vision: as the most efficient medium
for logical thinking. With the centrality of cognition for the post-industrial workplace, the notion
of visual reasoning acquired new importance. Consequently, the work on the models, techniques,
and technologies of visual reasoning, which was previously pursued by a few individuals, now
became a matter of systematic research on the industrial scale in such fields as cognitive
The third chapter addresses the role of vision as a way to capture spatial information.
Historically, this role has been played by the techniques of perspectival representation. Between
1940 and 1960, these techniques were automated (computer vision, computer graphics) and also
expanded beyond the realm of the visible with radar and other remote sensing instruments. The
automation of these techniques necessitated a new paradigm of vision research, which defined
space.
The last chapter takes up the understanding of vision in terms of information processing,
which emerged after World War II. In the post-industrial society, vision became the principal
order to monitor and improve the system's productivity, engineers and experimental
psychologists have adopted a common framework of information theory for both human and
machine components.
Today, the number of new disciplines which study vision continues to expand: image
interfaces and so on. Why? Modern society relies on vision not only as a means for the
production of subjectivity but first of all as a means of economic production. If the first use of
vision has been extensively theorized, the second so far remained largely unexamined. I hope
that this dissertation will begin filling this missing gap in histories of vision.
Chapter 1. Visual Atomism
1. Introduction
Let us compare two statements both related to the question of visual effectiveness but made in
different periods.
Charles Henry was a French writer who is mainly remembered today for his theory of
scientific aesthetics which greatly influenced Georges Seurat and Paul Signac.7 Writing in the
1880s, he advocated that aesthetic responses to simple perceptual elements (which for him were
Art pursues the expression of the physiognomy of things, and aesthetics studies the conditions in
which these things are satisfying; that is, when they are represented gay or sad, agreeable or
disagreeable, beautiful or ugly...aesthetic things for us are reduced to forms, to colors, and to
sounds.8
Half a century later, in 1927, in the article Photography in Advertisement, L‡szl— Moholy-Nagy
states:
A modern engineer, if his goal and the functional purpose of his work are clear, can without any
great effort make a product that is formally adequate and perfect in its economic construction. But
the photographic advertisements of our time are not so easy to define. They don't come with
"user's instructions." Research into the physiological and psychological laws of visual
effectiveness is still far behind the times, compared to the study of the physical laws.9
Both Henry and Moholy-Nagy argue that the creation of visual artifacts should be rationalized
and grounded in scientific psychological knowledge. However, the impetus for this
rationalization is fundamentally different. For Henry, the function of visual artifacts is artistic
7 See JosŽ ArgŸelles, Charles Henry and the Formation of a Psychophysical Aesthetics
(Chicago: University of Chicago Press, 1972).
8 Qtd. in Paul C. Vitz and Arnold B. Glimcher, Modern Art and Modern Science. The Parallel
Analysis of Vision (New York: Praeger, 1984), 87. My analysis of experimental psychology and
scientific aesthetics in this chapter is indebted to this groundbreaking book.
9 L‡szl— Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era, ed.
Christopher Phillips (New York: Aperture, 1989), 87.
expression; for Moholy-Nagy, it is to induce effects in the viewer. For Henry, the artifacts are
situated in the realm of the aesthetic, producing pleasure or displeasure, divorced from practical
life; Moholy-Nagy is concerned with everything practical -- changing beliefs, attitudes, behavior.
Henry calls for the rationalization of beauty while Moholy-Nagy wants to rationalize the process
of communication. He believes that modern designers have to grasp "the laws of visual
effectiveness" to be able to control the viewer's response -- precisely and predictably. And
finally, while Henry's "scientific aesthetics" deals with the artistic expression and aesthetic
In this chapter I will discuss one solution to the problem of the rationalization of vision
advanced by the 1920s artistic avant-garde. Investigations of the psychological effects of basic
colors and elemental forms, conducted by psychologists since the second half of the nineteenth
century, made possible the idea of a rational visual language composed from simple elements --
"atoms" of visual communication. This idea was already pursued in the nineteenth century by
such artists as Seurat and such theoreticians as Henry. However, in the 1920s, when artists found
themselves in the role of the designers of mass communication, the idea of an atomistic visual
language acquired new urgency and importance. The comparison between the statements of
Henry and Moholy-Nagy makes it clear that it was no longer a question of scientific aesthetics;
of a work of art producing an aesthetic effect in an individual spectator. Now it became the
importance. Therefore, Moholy-Nagy writes about the need to precisely "engineer" photographic
advertisements, and Bauhaus welcomes experimental psychologists. The trend reaches its
extreme in Soviet Russia, where artists thought that they could control not just the consumer
habits of segments of society (as in the West), but the consciousness of the whole country. Here
the program of research into the "atoms" of visual communication became most systematic,
translating into the establishment of a number of psychological laboratories at various art
institutions.10 Their goal was to create a fully rationalized visual language of mass
As other nineteenth century sciences, experimental psychology approached its subject, the
human mind, by postulating the existence of further indivisible elements, the combination of
which would account for perceptual or mental experience. If chemistry and physics postulated
the levels of molecules and atoms and so forth, and if biology saw the emergence of the concepts
of cell and chromosome, experimental psychology applied the same reductive logic to the human
mind. Psychologists divided sensory consciousness into different modalities: vision, hearing, and
The first psychological laboratory was founded by Wilhelm Wundt at the University of
Leipzig in 1879. The general program of Wundt's laboratory was to define the fundamental
elements of each modality and to formulate the laws according to which these elements are
combined. In 1896 former student of Wundt, E.B. Titchener, who brought experimental
psychology to the U.S., proposed that there are 32,800 visual sensations and 11,600 auditory
sensory elements, each just slightly distinct from the rest. Titchener summarized his research
program as follows: "Give me my elements, and let me bring them together under the
psychophysical conditions of mentality at large, and I will guarantee to show you the adult mind,
This analysis of visual experience into its elements seemed to have no practical utility. In
fact, the new discipline of experimental psychology developed within philosophy, both
intellectually and institutionally. In German universities of the 1870s, more lecture courses were
devoted to psychology than any other branch of philosophy except logic.13 Wundt, formerly a
Yet, even before the discipline of experimental psychology was recognized officially, it
had already found its first eager users -- artists and aestheticians. This was not simply a question
of creating works of art which directly illustrated the new scientific theory, in this case,
Neoimpressionists which represented reality with dots and dashes, the painterly equivalents of
sensory elements. Rather than illustrating an atomistic theory of perception, some art theorists
and artists realized that they could adopt its approach as a basis of a truly scientific aesthetic and
art.
What made this idea possible? As experimental psychologists split visual experience into
separate aspects (color, form, depth, motion) and subjected these aspects to a systematic
investigation, from the outset they had been asking two questions. First, what physiological and
psychological mechanisms are responsible for the perception of each aspect of vision? Secondly,
what is the psychological effect on the subject of the stimuli, such as elemental colors or forms?
It is this second question which was of great interest to aestheticians and artists. By relying on
elemental colors or forms with known psychological effect, artists could now predict and
anticipate viewers' emotions. Conversely, aestheticians could now develop an objective aesthetic,
"visual configuration considered as distinct from such things as color and pictorial content and
from the representation of objects."15 While some psychologists were analyzing human
perception of simple forms such as squares, circles, and straight lines of different orientations,
14 Ibid., 39.
15 Vitz and Glimcher, Modern Art and Modern Science, 144.
others were investigating their psychological effects. In the 1870s a French professor of art,
Charles Blanc, developed a theory of the intrinsic psychological significance of vertical, straight,
and oblique lines.16 Blanc popularized the ideas of the Dutch painter and theorist Humbert de
Superville who around 1830 first claimed that simple lines conveyed emotions by having an
Blanc argued that the straight line is a "symbol of unity" and the curved line of "variety." Of the
straight lines, the most important is the vertical -- since man stands perpendicular to the horizon
and since he is bilaterally symmetric. The horizontal line is next in importance and then the two
obliques. Blanc discusses three types of human faces characterized by three orientations and by
the associated effects of happiness, rest and sadness.17
In the 1880s, Charles Henry further advanced Blanc's ideas, developing the "aesthetic
protractor" to measure the harmony of line angles (fig. 2). Henry's device was designed to
measure "whether the angles between lines radiating in different directions from a single point
are harmonious."18
It is significant that the first laboratory research on the aesthetics of simple forms was
conducted in 1876 by Gustav Fechner -- the founder of mathematical methods for the
measurement of sensations which became the basis for experimental psychology. Later, at the
turn of the century, many proto-Gestalt psychologists started to investigate the aesthetics of
simple forms, isolating preferences for squares, rectangles, ellipses, and triangles as well as
different lines.19 Their publications became filled with "abstract" pictures consisting of simple
forms.
This psychological research into the effects of simple forms influenced Seurat, Signac,
Wassily Kandinsky, Paul Klee, and Piet Mondrian, among others. For instance, Seurat, who was
16 Ibid., 170.
17 Ibid., 169.
18 Ibid., 170.
19 Ibid., 155.
familiar with the works of Blanc and Henry, advanced a similar theory of the intrinsic
psychological effects of lines of different orientation (fig 3).20 In another example, Kandinsky,
in Point and Line to Plane, advocated "microscopic" analysis of three basic elements of form
(point, line, and plane) claiming that there exists reliable emotional responses to simple visual
configurations.21 Equally telling of Kandinsky's program are the titles of the articles he
published in 1919: "Small Articles About Big Questions. I. About Point," and "II. About
Line."22
At first, artists would embed simple lines or forms in their otherwise representational
compositions. A typical case is Seurat who based the orientations of lines in his major paintings,
such as Le Cirque (1890-91) and La Parade (1887-88), on the theory which he derived from
Blanc: "Gaiety...of line, lines above the horizontal; calmness...the horizontal...Sadness...of line,
downward directions."23 Gradually, however, artists gave up representation altogether and began
to compose works which would consist solely of the simple elements already studied by
psychologists. It is, therefore, not accidental that the paintings of Mondrian, Klee, and Kandinsky
look remarkably similar to the visual stimuli already widely used by psychologists in previous
decades. They are also experiments, a result of a systematic investigation into what Kandinsky
called "the science of art," the science which would allow the reliable communication of any
emotional experience.
20 Ibid., 170-71.
21 Wassily Kandinsky, (1926), Point and Line to Plane (New York: Solomon R. Guggenheim
Foundation, 1947).
22 Yu. A. Molok, "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel
Florensky's 'dictionary of symbols.' A few margins), Sovetskoe Iskusstvoznanie 26 (1990): 328.
23 Qtd. in Vitz and Glimcher, 170.
That which science can and must do is to expand the agreeable within us and outside of us, and
from this point of view its social function is immense in this time of oppression and blind
conflicts. It ought to spare the artist hesitations and useless attempts in assigning or indicating the
way in which he can find ever more rich aesthetic elements; it ought to furnish the critic a rapid
means of discerning the ugly, so often informulable, however much it is felt.24
Henry equates an artist with a scientist whose vocation is to provide the viewers with the means
of escaping the pressures and the unpleasant experiences of everyday life through the beauty of
art. The artist is the therapist of a society, and because of the importance of this role, he ought to
follow a scientific method in order to make his works most effective in helping the viewers to
Moholy-Nagy, in the already quoted statement from the 1920s, also compares the artist to
an applied scientist -- an engineer. The artist, however, is no longer removed from the realm of
everyday life. Just as the modern engineer shapes every aspect of material reality -- the machines
for living (architecture), the machines for transportation (airplanes, cars, trains), the machines to
produce other machines (the tools of industrial production) -- an artist has the crucial role of
shaping the psychical reality, controlling people's emotions and ideas through advertisements,
Moholy-Nagy's appeal to designers and photographers brings forth the new purpose of
the research into the elemental units of vision -- the need to optimize the process of mass
communication. This new purpose became crystallized at the moment when modernist artists
claimed the position of designers of mass propaganda in Soviet Russia in the 1920s. At this
moment the two lines of inquiry -- artistic exploration of the visual elements and research in
of Soviet art institutes of the 1920s such as INKhUK (Institute of Artistic Culture, Moscow
1920-24), GAKhN (State Academy of Artistic Sciences, Moscow 1921-30), and VKhUTEMAS
(State High Art-Technical Studios), El Lissitsky, Alexander Rodchenko, Osip Brik, and others
The paradigm of research into the elements of visual communication and their effects can be
conveniently traced through the history of INKhUK, which during the four years of its existence
united the key groups of avant-garde artists and theoreticians most committed to this research.
In the first year of INKhUK's existence the group led by Kandinsky was the most active.
His program called for the cataloging of basic elements of art and to describe their emotional
laboratory in the Fall of 1920 but Kandinsky and his supporters were forced to leave a few
months later.27 Kandinsky then became vice-president of the newly formed GAKhN and the
head of its largest physical-psychological division. The method of Kandinsky's division was
characterized as "the observation of and experimentation with the content and processes of
artistic creation and reception as well the work of art in all its complexity, understood as a
25 Other Soviet research institutes of the 1920s whose programs included the development of a
scientific approach to art and art history include: The State Institute of Artistic Culture (headed
by Kasimir Malevich, Leningrad 1923-27); The Institute of Literature, Art and Language, a part
of Communist Academy (1918-36); The State Institute for the History of the Arts (Leningrad
1921-31); The Institute of Art History, a part of Moscow University; The Russian Association of
Research Institutes of Social Sciences (Moscow 1924-31).
26 Wassily Kandinsky, (1920), "Programma raboty Instituta khudozhestvennoy kultury (The
Program of the Institute of Artistic Culture)," in Sovetskoe iskusstvo za 15 let. Materialy i
dokumentatsiia (Fifteen years of Soviet art: materials and documentation), ed I. Matsa (Moscow,
IZOGIZ: 1933), 126-139.
27 The work of INKhUK during first year of its existence has been documented in S.O. Khan-
Magomedov, "INKhUK: vozniknovenie, formirovanie i pervyy period raboty. 1920" (INKhUK:
appearance, formation and first period of its work, 1920), Sovetskoe Iskusstvoznanie (1981):
332-368.
3. Visual Atomism: a Code for Mass Communication
psycho-physical object."28 The members of the academy included art historians, practicing
and his supporters formulated a new program in total opposition to Kandinsky. Against
Kandinsky's "subjective" psychologism they emphasized the materiality of objects. They also
making of objects rather than the creation of art.29 However, the project of a science of visual
elements was not abandoned in practice, only the emphasis shifted from the "subjective" analysis
material objects. INKhUK members, who started to teach at the newly formed VKhUTEMAS
(State High Art-Technical Studios), had the opportunity to put these ideas into practice.30 For
instance, in his course on graphics Rodchenko would start his students with systematic exercises
to combine simple forms (circle, triangle, and square) within a given format in order to test the
In its third stage INKhUK became the center of activity of the promoters of the idea of
production art -- Osip Brik, Boris Arvatov, Nikolai Tarabukin, and others.32 In 1923, this group
founded the journal LEF (The Journal of the Left Front of Art, Moscow 1923-25) the goal of
which was to unite artists of the Left dedicated to radical revolutionary activity in the face of
what they perceived as the new flourishing of "bourgeois" art made possible by the NEP.
28 P.C. Kogan, "O zadachakh akademii i yeyo zhurnala" (About the goals of the academy and its
journal), Iskusstvo 1, no. 1 (1923): 9.
29 A number of early constructivist manifestos are now available in English translation in
Nicholas H. Allison, ed., Art Into Life: Russian Constructivism 1914-1932 (New York: Rizzoli
International Publications, 1990), 61-82.
30 It was not accidental that the artists referred to their activities at INKhUK as "laboratory
work." "Laboratory" connoted scientific labor, rather than artistic experimentation. "Institut
Khudozestvennoy Kultury" (The Institute of Artistic Culture), Russkoe Iskusstvo 2-3 (1923): 85.
31 A.H. Lavrent'ev, "Propedevticheskaya distsiplina 'Grafika.' Vkhutemas. 1920-22 gody" (The
discipline 'Graphics.' VKhUTEMAS. 1920-22), Tekhnicheskaya Estetika 7 (1984): 19.
32 "Institut Khudozestvennoy Kultury," 86.
What should this radical revolutionary activity be? Western art historians, critics, and
artists glorified the constructivists' desire to give up the traditional roles of artists and instead
submerge themselves in industry, to leave the artist's studio for the factory -- in other words, to
become industrial designers.33 What received practically no attention was another role which
constructivists and many other Soviet artists were claiming for themselves in the 1920s -- the
advertisement. And it was in this role, particularly as advertising designers, that VKhUTEMAS
artists, as well as their counterparts from the Bauhaus, were able to accomplish the transition
"from an easel to a machine" (to quote the title of a book by Tarabukin, one of the theoreticians
of production art).34 These machines were not a lathe or an engine but a printing press, a film
camera, a light projector, a radio transmitter -- the machines of mass communication rather than
mass production.
Left artists were united in rejecting art in favor of mass propaganda. In the first issue of
LEF Sergei Tretyakov writes: "Revolution put forward new practical tasks [for artists] --
affecting mass psyche, organizing the will of the class"; "The work of futurism is parallel to the
work of communism -- futurism is fighting for the same dynamic organization of personality
without which the movement toward communism is impossible."35 For LEF, this task of the
revolutionary education of the masses required a scientific method and in the same article
Tretyakov called for the creation of a "science of art as a means of emotional and organizing
33 In 1923, Lunacharsky wrote: "They [constructivists] play at being engineers, but they don't
know as much of the essence of machinery as a savage." Qtd. in Brandon Taylor, Art and
Literature under the Bolsheviks (London: Pluto Press, 1991), 1: 177.
34 Between 1923 and 1925 Rodchenko designed advertising for many Soviet government
agencies; Lissitsky, who was sent to Germany to publicize the art of the first Socialist state,
created brilliant designs for many German firms; so did Moholy-Nagy and other Bauhaus
faculty.
35 Sergei Tretyakov, "Otkuda i kuda? (Perspektivu futurizma)" [Where from and where to?
(Perspectives of futurism)], LEF 1 (1923): 197, 203.
influence on psyche, in connection with the task of class struggle."36 In fact, artists should
become as scientific in their work as Marxists are in their political activity. To do so, they should
rationalize the creative process and put art education on a scientific basis.37 This idea was most
Culture). Thus, Tarabukin organized the artist's workshop aiming to bring the principles of
Taylorism and Fordism into artistic practice: "We are analyzing processes of work, experiment
with compressing work periods, aim to rationalize work and leisure. In its practice the workshop
tries to get away from bohemian habits so common to artists."38 Artists should also rationalize
viewers' reactions by developing a science which would enable them to produce predictable and
measurable effects in the viewer. Thus, the author of a 1927 article which summarized the
This dual goal of rationalization was symbolized by the popular metaphor of artist as
36 Ibid., 199.
37 LEF required the same from poets and writers who could make their work scientific by
relying on the advances made by Russian Formalists in studying the devices of literature. For
LEF, these devices became the tools of verbal propaganda. In the first issue of LEF Osip Brik
writes: "OPOYAZ studies the laws of poetic production... OPOYAZ will help the art of the
proletariat, but not with vague conversations about the 'proletariat's spirit' and 'communist
consciousness,' but with precise technical knowledge of the devices of contemporary poetic
work." Osip Brik, "T.n. 'formalnyi metod'" (So called 'formal method'), LEF 1 (1923): 214-215.
The meaningless experimental poetry of Futurists which originally made Formalists recognize
the "independent value of the poetic sound" now became full of meaning; the techniques of
"poetic instrumentation" were now seen as the techniques of organizing the consciousness of the
masses.
38 I. Matsa, Sovetskoe iskusstvo za 15 let, 279.
39 Ibid., 281.
LEF. Aleksey Gastev, a leading figure of Proletcult as well as of the Soviet Taylorist movement
who in 1921 organized the Central Institute of Labor coined the phrase "social engineering" to
describe the process of the radical reconstruction of the human psyche.40 Tretyakov applied the
metaphor to art: "Along with man of science the art worker should become psycho-engineer,
psycho-constructor."41
The crucial question then becomes how to rationalize visual effectivity. In principle,
different approaches are possible: to create particular conditions of reception; to limit the number
of available images in the public sphere (this was done later in the period of Socialist Realism);
and finally, to create a particular code, a new visual language guaranteed to communicate
effectively and efficiently to mass audiences (often, in the case of Soviet Russia of the 1920s,
illiterate). It is the last solution which was explored by Leftist artists such as Rodchenko or
Lissitsky.
Why did the notion of a visual language consisting of simple abstract forms become
attractive? First, abstract geometric forms were neither associated with bourgeois figurative
representation nor did they look like the "subjective" abstract improvisations of Kandinsky.
Secondly, as Igor Golomstock points out, abstraction was justified by emphasizing its stylistic
origin in truly popular decorative and religious art (for instance, icons).42 Thirdly, simple forms
"The most unambiguous and immediately recognizable forms are geometric forms. No one will
confuse a rectangle with a circle, or a circle with a triangle."43 Finally and most importantly,
such forms were imagined to be perfectly suited for the purpose of controlling the effect in the
viewer -- both the idea and the imagery being prepared by decades of investigations in the
40 Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 26.
41 Tretyakov, "Otkuda i kuda?," 202.
42 Golomstock, Totalitarian Art, 26.
43 Qtd. in Golomstock, Totalitarian Art, 24.
psychology of perception and in scientific aesthetics. The tradition of research into the
effectiveness of visual elements transformed into the dream of mass "psychophysical culture"
The most striking record of this dream is the paper "Engineerism" which was presented at
one of INKhUK's meetings. The paper starts with an analysis of modernism as a reduction of
visual experience to its elements: "In the final stage of its development art has renounced
representation and has moved toward visual sensations as such."44 The trend continues with
abstract art, the aim of which, according to the author, is "the organization of visual perceptions."
Bourgeois art cannot go beyond this stage of analysis, but in the first proletarian state, this
analytic work will become the basis of a new visual culture of social control: "Visual sensations
as such will concertedly shape the human being as an organized unit, with the tempo of
From this perspective, we also should not be surprised to find in the same period the
recurrent attempts to establish various graphic alphabets and dictionaries. The poet Velemir
Khlebnikov in his 1919 article "Artists of the World" postulated two types of a universal visual
code -- graphic signs and color equivalents. In the already mentioned INKhUK program,
Kandinsky proposed to create a full dictionary of colors, forms and their combinations with
descriptions of the psychological effect of each element. Pavel Florensky together with his
1922.46 This "international and ahistorical dictionary" should have contained all cultural
regretted that while whole academies were busy preparing and preserving the dictionaries of
the "universal language of mankind."47 Similar projects of a universal graphic alphabet were
advanced in the same years by Ivan Puni, Petr Miturich, and other Russian artists.48
To summarize: early Soviet attempts to theorize and put into practice a scientifically grounded
visual language of mass communication bring together artistic exploration of visual elements and
research in experimental psychology. While the desire of Soviet artists to emulate "scientific"
Marxism and to adopt Taylorism and Fordism were already sufficient reasons to pursue the
science of elements of art, the ultimate justification for such a science was provided by
experimental psychology and scientific aesthetics (Blanc, Henry, and others). Their
investigations of the psychological effects of basic colors and simple forms prepared the idea of
a language of mass communication composed from simple elements. In theory, this model called
to enumerate visual elements and to describe their effects in order to compose dictionaries of
"visual effectiveness." In practice, it translated into the creation of works composed from distinct
From 1921 to 1929 the psychological studies of viewers' responses to visual forms were
actively conducted in the central art institutes. In 1926 GAKhN created a psychophysiology
laboratory, a psychophysics department, and laboratories for experimental aesthetics and art
theory.49 In order to arrive at the general laws of the effectiveness of visual forms the
investigators have systematically analyzed the basic dimensions of volume, line, color, and
scientific basis. Jack Chen, a Communist British artist trained in the Soviet Union during the
1920s, wrote about this program of art production based on psychological research:
Art must be a science, an industry. Pictures and sculptures should be constructed according to
exact scientific principles after colors and forms had been classified according to their human
reaction values...A picture according to the Left was really nothing but a "machine" for generating
certain predetermined human reactions. Artists should be engineers of form and color.50
There does not appear to be a disagreement among the Left as to the function of art -- to
create predetermined viewer responses. But how was this to be achieved? In other words,
assuming that we have isolated the elements of a visual code, how should these elements affect
the viewer? Do they induce emotions, do they communicate ideas or do they control behavior in
To answer this question, Soviet artists of the 1920s relied heavily on the psychological
theories of the day.51 It is possible to discern two distinct trends in how psychology was used. In
the first trend, autonomy was granted to the human psyche. The second trend privileged
physiology, considering the mind a neurological organ not different from the rest of the
organism. This trend was represented by such influential scientists as Bekhterev and Pavlov,
The latter model was also prevalent in Soviet Marxist psychology in the 1920s. In 1924 it
was challenged by Lev Vygotsky, a young psychologist and a former literary critic. Recognized
today as the forerunner of cognitive psychology, he argued that the higher cognitive functions
are qualitatively different from the lower physiological processes and require different
50 Jack Chen, Soviet Art and Artists (London: The Pilot Press, Ltd., 1945), 58.
51 For the history of Soviet psychology in the 1920s, I have consulted David Joravsky, Russian
Psychology: A Critical History (Cambridge, Mass.: Basil Blackwell, 1989); Alex Kozulin,
Psychology in Utopia: Toward a Social History of Soviet Psychology (Cambridge: The MIT
Press, 1984).
investigative approaches. In his famous address to the Second Russian Psychoneurological
psychology of behavior, claiming that it cannot be understood through the study of reflexes.
Significantly, the turning point of Vygotsky's career from literary critic to psychologist was his
1924 dissertation Psychology of Art. Through the analysis of Hamlet Vygotsky attempted to
demonstrate that art is a product of the capacities of the specifically human mind -- imagination,
Crucial for Vygotsky's theory was the study of the formal organization of works of art:
...every work of art is considered to be a system of stimuli, which are organized consciously and
intentionally in order to cause an aesthetic reaction. This way we reconstruct the structure of the
reaction while analyzing the structure of the stimuli...The general direction of this method can be
summarized in the following formula: from the form of the art work through the functional
analysis of its elements and structure to the reconstruction of the aesthetic reaction and the
discovery of its general laws.52
In Vygotsky's formula the given was the existing works of art and the unknown was the laws of
aesthetic reaction. As we have already seen, for the artists, designers, and film directors of the
time the formula was reversed. If for the psychologist Vygotsky, visual works represented a
reservoir of knowledge about the human mind, artists, on the contrary, were hoping to utilize the
objective psychological knowledge in order to create visual works which would produce pre-
Contemporary psychologists -- the supporters of Pavlov and Bekhterev on the one hand,
and younger psychologists such as Vygotsky and A. Luria on the other hand, argued over
whether the mind could be reduced to the laws of physiology. Similarly, the artists in theory and
in practice drew on these two alternative psychological models. Sergei Eisenstein's attempts to
ground his filmmaking methods in different psychological theories epitomizes the two models.
In one of his latest written works Eisenstein summed up his work in film:
Although Eisenstein's goal remained the same, the means of achieving the desired effect were
On the one hand, Eisenstein developed the concept of "intellectual montage," privileging
a purely intellectual response. The goal of cinema, in this view, was thought to induce
dialectical reasoning. The structure of the film itself was conceived of as dialectical thinking in
visual form with montage being the means of representing the dialectical process through the
contrast and juxtaposition of images. This preoccupation of Eisenstein with cinema as the
visualization of the work of the intellect culminated in his project to create a screen adaptation of
Marx's Capital.54
On the other hand, Eisenstein's second important concept developed in the Montage of
Attractions takes the viewers' physiological reaction as a point of departure.55 Eisenstein based
his concept on the psychological theories of Ludwig Klages and William James.56 According to
Klages, in a human being emotional states are expressed through bodily movements. Klages also
insisted that human expressivity is characterized by a unique quality -- the muscular contractions
of one person are involuntarily repeated by the observer. James' theory was related to Klages' but
causally reversed. He postulated that emotions were the effect of muscular contractions -- one
does not cry because he is sad, but one becomes sad due to crying. Eisenstein combined the two
theories: the emotional state of the actor translates into his muscular movements; these
The important issue, then, was the training of actors in simulating precise gestures and facial
expressions in order to produce a desired emotional response in the viewer. Eisenstein proposes
"montage of attractions." The film becomes a script of the emotional responses of viewers.
The two concepts -- the intellectual montage and montage of attractions -- are the rare
summits in Eisenstein's intellectual career where the opposing tendencies of his thought
concerning the modes of cinematic influence on the viewer can be clearly distinguished. Most of
the time they seem to have been intertwined, appearing simultaneously in his changing
conception of cinematic effectivity. Similarly, in the cultural imagination of the 1920s, the two
tendencies appear to be running side by side. Is there a mind, a specifically human autonomous
cognitive apparatus, with operational laws not reducible to simple reflexes? Or can human
stimuli?
This second position received its extreme elaboration in the theory developed by
Emmanuel Enchmen, an early Socialist revolutionary.57 Enchmen's "theory of new biology" was
based on Pavlovian reflexology and proposed a purely proletarian culture based directly on
conditioned reflexes, without the necessity of conventional language or thought. The new
proletarian language would consist of conditioned grunts and gestures and replace the verbal
All in all, the co-existence of the two schools in Soviet psychology lead to the co-
communicating ideas and bypassing consciousness altogether. This dichotomy is reflected, for
instance, in the following statement from "New Artistic Program of Proletcult" published in
forms (poster, advertisement, agitational theater, cinema) and in its material-organizational forms
These different models of affectivity were also used to understand the roles of visual
elements which were being "isolated" and "purified" in the laboratories of the 1920s. The
elements could have affected the viewer in more than one way. No longer just catalysts of basic
emotional states, as in the tradition of experimental psychology and scientific aesthetics, they
were now re-interpreted as capable of carrying complex messages addressed to the mind, and as
I have discussed different models of affectivity which underlined the research into the elements
of a visual code. But what kind of code can be formed from such elements?
Semiotician Luis Prieto distinguished between codes with and without articulation.59 A
code with articulation consists of a number of elements combined to produce messages. The
examples are human language, playing cards, and maritime flag codes. The crucial feature of any
of these codes is that they allow for the generation of a large or even infinite number of messages
from a small set of elements. For instance, in the case of human language, its speaker can
generate an infinite number of sentences. In contrast, in codes without articulation there are as
many meanings as elements. For example, in the language of flowers, each flower signifies a
In view of this distinction, the research into visual elements could justify both a code with
articulation as well as a code without articulation. In other words, it is possible to understand the
particular behavioral response), or as just a part of the message, like a word in a sentence,
The artistic tradition of visual atomism, from Blanc to Kandinsky, appears to favor the
first interpretation. In their writings, Blanc, Henry, and Seurat correlated the orientation of
straight lines with different emotions such as gaiety or sadness. Kandinsky's famous
questionnaire, compiled in 1920 while he was active in INKhUK, also indicates that he
considered the elements to directly correspond to emotions or verbal meanings: "How do you see
a triangle -- do you feel that it is moving, where, do you see it as more witty than square; is the
sensation from a square similar to the sensation from a lemon; what is a canary's song more like -
- a triangle or a square."60
A code without an articulation requires as many signs as there are potential messages and
therefore is not efficient. Yet, it is this inefficiency which makes it ultimately suitable for new
communicative situations faced by artists and designers in the earlier decades of this century.
Traffic signs, the labels on instrument panels, trademarks had to be unambiguous and instantly
recognizable by an untrained eye. Not surprisingly, the designers in every field have adopted
simple abstract forms. Thus, in 1909, when Mondrian and Kandinsky were moving toward their
first abstractions, the first International Conference on the Regulation of Automobile Traffic took
For the Soviet "psycho-engineers," the notion of a visual code without articulation, which
correlates each message with a separate sign, had a particular attraction. The behaviorist
psychologies of Pavlov and Bekhterev popularized the notion of directly controlling behavior
through conditioned reflexes. Simple visual forms could have acted as such stimuli, controlling
"the proletarian zombie" as the lights and whistles already controlled a dog in Pavlov's
laboratory. In this interpretation, the dictionary of visual elements would contain not a set of
meanings or emotions but a set of commands -- previously learned responses to stimuli. This is
what the author of the "Engineerism" paper seems to have in mind when he writes of the future
where "visual sensations as such will concertedly shape the human being as an organized unit."
In other words, just as the motorists in the 1920s were beginning to memorize the international
traffic signs, learning how to respond to them automatically (today, this is the closest most of us
come to experience conditioning through a visual code), artists took this as a paradigm for
Yet, while suitable for communication of basic emotional states or a set of behavioral
stimuli, the code without articulation is not practical for communicating an arbitrary large
number of messages. If visual language is to compete with a verbal language, and even to replace
it, visual forms have to be interpreted as equivalents of letters or words. They also have to be
supplemented by grammar, the rules of how these visual "words" are combined to form
sentences.
Indeed, contemporary debates on whether such animals as dolphins or primates can use
human language center around the attempts to teach them to use visual signs in this way. It is
61 A.S. Sardarov, "5000 let evolutsii doroznogo znaka" (5000 years of the evolution of a traffic
sign), Tekhnicheskaya Estetika 9 (1984): 14-19.
easy to train animals to respond to visual forms as distinct messages: a square means "take food,"
a circle means "jump," and so on. The question is whether animals can string such symbols
together to form new messages, thus treating the individual symbols as words rather than as
complete messages. At least a few researchers have claimed that they have taught some animals
What about humans? The idea of replacing particular verbal language by a universal
visual language -- more efficient, capable of reaching new masses of immigrants (U.S.) or
"proletarians of the whole world" (U.S.S.R.) was often expressed in the 1920s. Film, in
particular, was seen as the prime candidate for a visual Esperanto. In a 1921 interview D.W.
Griffith said: "A picture is the universal symbol, and a picture that moves is a universal
language. Moving pictures, someone suggests, 'might have saved the situation when the Tower
of Babel was built.'"62 Miriam Hansen writes about the metaphor of film as visual Esperanto in
the U.S.:
Griffith participated in the widespread celebration of film as a new "universal language" which
accompanied its formation as an institution. Used by journalists, intellectuals, social workers,
clergy, producers, and industrial apologists alike, the metaphor of film as a universal language
drew on a variety of discourses (Enlightenment, nineteenth-century positivism, Protestant
millennialism, the Esperanto movement, and the growing advertising industry) and oscillated
accordingly between utopian and totalitarian implications.63
However, it was across the Atlantic, in the U.S.S.R., where the comparison between film and
verbal language was explored most systematically by Victor Shklovsky and other literary
theorists associated with the so-called formalist school as well as by film directors and theorists
such as Eisenstein, Vsevolod Pudovkin, and Lev Kuleshov. In their writings we encounter
62 Qtd. in Miriam Hansen, "The Hieroglyph and the Whore: D.W. Griffith's Intolerance," The
South Atlantic Quarterly 88, no. 2 (1989): 363.
63 Ibid., 362.
frequent comparisons of a shot and a word, or a sequence and a sentence as well as references to
a "grammar of film."64
Can we find similar kinds of analogies proposed for the elements of a single image? Just
as the artists were comfortable with different mutually exclusive models of affectivity, they were
equally at ease with two contradictory models of a code. If the visual elements were imagined to
be capable of carrying complete messages, they were also thought of as the elements of a
message. Thus, art historian S.O. Khan-Magomedov writes about the INKhUK program: "The
point was to discover certain primary elements of artistic expressiveness, in themselves without
What we don't find in the 1920s, however, is equally serious thinking about the grammar
of a still image. It was only in the 1960s, when the analogy between a verbal and a visual
language was taken with new rigor in structural semiotics, that coherent theorizing about the
language. This is reflected in the popularity of the term "visual language," this term which is so
familiar to us today that we tend not to ask about its historical specificity. What this term points
64 Paradoxically, the classical Hollywood film style, which matured by the 1920s, did become
the universal visual language of the twentieth century, while the films of Soviet avant-garde
directors, who were most systematic in theorizing film as a language, were rejected by mass
audiences and continue to play art houses, as they did in the 1920s.
65 Khan-Magomedov, "INKhUK," 343.
66 The best critical review of visual semiotics available in English is Gšran Sonesson, Pictorial
Concepts. Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual
World
6. (Lund, Sweden: Lund University Press, 1989).
Conclusion
to, in my view, is the new social role of vision as the medium of mass communication which it
The attempts to rationalize vision in this role, to conceive of it as a code which can
function independently of verbal language and which can be effective in the new communicative
situations created by modernization have centered around the quest for the elements, the "atoms"
of visual communication. This quest reaches its culmination in the work of a number of art
institutes in Soviet Russia in the 1920s, where artists collaborate with experimental psychologists
The search for the essential elements of each art which preoccupied early twentieth
century culture is usually seen in the context of artistic modernism. This search is interpreted in
the context of the rhetoric of the purity of medium. Indeed, especially in the 1910s-20s, artists
tried to reduce every medium to its unique qualities. To do so, they gave up representation and
concentrated on the material elements thought to be unique to each medium. Poets, such as
Russian futurists, were experimenting with sounds; filmmakers proposed that the essence of
cinema was movement and temporal rhythm (French film theory of the 1920s)67 or montage
(Kuleshov's group in Russia); and painters were exploring pure colors and geometric forms. The
following statement made in 1924 by Jean Epstein, a French avant-garde filmmaker and
For every art builds its forbidden city, its own exclusive domain, autonomous, specific and hostile
to anything that does not belong. Astonishing to relate, literature must first and foremost be
literary; the theater, theatrical; painting, pictorial; and the cinema, cinematic. Painting today is
freeing itself from many of its representational and narrative concerns...And any literature worthy
of the name turns its back on those twists and turns of plot which lead to the detective's discovery
67 See especially Jean Epstein, "On Certain Characteristics of PhotogŽnie," in French Film
Theory and Criticism, ed. Richard Abel (Princeton: University of Princeton Press, 1988), 1: 314-
318; Germaine Dulac, "Aesthetics, Obstacles, Integral CinŽgraphie," in French Film Theory and
Criticism, 1: 389-397.
of the lost treasure...The cinema must seek to become, gradually and in the end uniquely,
cinematic; to employ, in other words, only photogenic elements.68
Yet, as this chapter has suggested, more was at stake in the quest for the visual "table of
elements" than just the demand for purity of medium. While many artists searched for "ever
more rich aesthetic elements" (Henry), trying to make their art into a "forbidden city" (Epstein),
others welcomed the opportunity to take their art to the streets. For these artists, the science of
visual elements has provided hope for the creation of a fully rationalized visual language of mass
communication.
The dream to rationalize communication was not unique to this project -- the proposals
for a universal language go back to the seventeenth century. Descartes suggested that the lexicon
these elements, one can generate "an infinity of different words."69 In the early eighteenth
century Leibniz outlined a language in which grammatical and logical structure would coincide,
thus making possible the automation of thinking. The basic elements of his ideal language were
the inventory of these concepts "the alphabet of human thought." Through the algebra of thought
primitive concepts would be combined to form any complex idea. 70 The proposal of a universal
language composed from graphic symbols and therefore suitable only for writing followed soon
with the first pure pasigraphy ("writing for all") invented by Joseph de Maimieux in 1797.71
However, until the twentieth century these remained isolated philosophical ideas advanced by
single individuals. With the rise of mass communications, the fantasy of a universal rational
language attains new significance, extending into the realm of vision. Thus, what we now
68 Ibid., 314-15.
69 Qtd. in Winfried Nšth, Handbook of Semiotics (Bloomington and Indianapolis: Indiana
University Press, 1990), 272.
70 Ibid., 274.
71 Ibid., 269.
witness are systematic and extensive attempts to establish fully rationalized visual languages of
mass communication. At the same time, the studies of behavior and conditioning in physiology,
behaviorism and reflexology, lead to new interpretations of the functioning of the elements of a
visual Esperanto. Not only would they be capable of communicating ideas thus affecting the
intellect, but they would also directly control behavior bypassing the mind altogether, thus
naturally. Imagination, on the other hand, requires a special effort and is in no way necessary for
For example, when I imagine a triangle I not only conceive (intelligo) that it is a figure
comprehended by three lines, but at the same time also I look upon (intueor) these three lines as
present by the power and internal application of my mind (acie mentis), and this is what I call
imagining. But if I desire to think of a chiliagon, I indeed rightly conceive that it is a figure
composed of a thousand sides, as easily as I conceive that a triangle is a figure composed of only
three sides; but I cannot imagine the thousand sides of a chiliagon as I do the three sides of a
triangle, nor, so to speak, view them as present [with the eyes of my mind].72
Indeed, how can humans possibly reason through visual representations, if the latter are not
capable of representing general concepts to begin with? In Treatise Concerning the Principles of
Human Knowledge, George Berkeley insisted, for instance, that it is impossible to have a mental
image of an idea, such as "man," as a generality; it is only possible to visualize a tall or a short
man, but not man as such. Thinking deals with generalities and does not tolerate particular
things. If a philosopher tries to reason about the nature of "man," any image of a particular man
The philosophical arguments against visual reasoning still continue in this century, but
they are out of step with the modern productive use of vision: not only the concepts of "visual
72 RenŽ Descartes, "Meditations on the First Philosophy," in The Rationalists, trans. John Veitch
(Garden City, NY: Doubleday & Company, 1960), 160-161.
73 Rudolf Arnheim, "A Plea for Visual Thinking," in New Essays on the Psychology of Art (Los
Chapter
Angeles
1. 2. I See,
and
Introduction Therefore
London: I Think
University of California Press, 1986), 136.
thinking," "thinking through images," etc. have become respectable and commonplace,74 but
they are also put into practice daily through new techniques and technologies of visual
representation. While Descartes thought that it was impossible (and, at any rate, unwise) to
imagine the thousands of sides of a chiliagon, today scientists stare at computer monitors, where
computer imaging brings before their eyes not only chiliagons but geometric objects of n-
dimensions, objects composed not from thousands but from millions of sides, objects
representing weather systems, diffraction patterns, atomic surfaces, and other processes
inconceivable to the bare imagination (fig. 5).75 "Scientific visualization" is the name of the new
field which claims to transform the nature of scientific reasoning in biology, chemistry, and even
mathematics.
Likewise, Berkeley's insistence that images cannot convey "man" as a generality has been
superseded. Since the 1960s, numerous space stations sent to the Moon, Venus, Mars or simply
into the infinity of the Universe have carried images destined to communicate to intelligent
beings (should they happen to find the stations) the basics of human civilization and the "essence
of man" in a "universal language of images." Long after the sun explodes and man disappears,
space stations will continue to fly on their trajectories, carrying such images as the famous
drawing by Leonardo da Vinci of an "ideal man." Intelligent (or perhaps, not so intelligent)
beings will eventually discover these images, learning from them if not about human civilization
in general, then at least about the artistic tastes of administrators of space programs, assuring
The images of general or abstract concepts can be encountered not only on space
missions but on Earth, everyday and everywhere, in numerous logotypes and designs (for
74 As witnessed, for instance, by such titles as Thinking Eye (Paul Klee) or Visual Thinking
(Rudolf Arnheim).
75 Examples of scientific visualizations presented at the Showcase at SIGGRAPH 1992.
SIGGRAPH '92 Final Program (New York: The Association for Computing Machinery, 1992),
33-40.
instance, in symbols which distinguish men's from women's lavatories), or in the representation
of unimaginable data sets. However, these examples do not tell us anything about the specificity
of the new function of vision in modernity. After all, for centuries pictures have been employed
as visualizations of the holy stories. Michael Baxandall cites a late thirteenth-century text which
Know that there were three reasons for the institution of images in churches. First, for the
instruction of simple people, because they are instructed by them as if by books. Second, so that
the mystery of the incarnation and the examples of the Saints may be more active in our memory
through being presented daily to our eyes. Third, to excite feelings of devotion, these being
aroused more effectively by things seen than by things heard.76
Therefore, what is different about the modern function of vision is not simply the use of images
to represent abstract concepts concretely. What is different is the idea of reasoning through
images, the idea unthinkable for Descartes or Berkeley. Thus, the field of scientific visualization
is based on the assumption that visual observation of computer generated images of data sets and
processes leads to scientific breakthroughs otherwise impossible. Richard Mark Friedhoff and
chemistry might be able to deduce benzene's structure" (a reference to Friedrich KebulŽ's 1865
discovery of the molecular stucture of benzene supposedly made after he saw a dream involving
a snake biting its own tale).77 Similarly, Jaron Lanier, one of the most visible
researchers/promoters of virtual reality, maintains that virtual reality will lead to the age of "post-
symbolic communication," the communication of ideas and arguments solely through images.78
Finally, a prominent cognitive psychologist Philip Johnson-Laird claims that everyday logical
76 Michael Baxandall, Painting and Experience in Fifteenth Century Italy (Oxford: Oxford
University Press, 1972), 41.
77 Richard Mark Friedhoff and William Benson, The Second Computer Revolution:
Visualization (W.H. Freeman and Company, 1991), 13.
78 See, for instance, Timothy Druckrey, "Revenge of the Nerds. An Interview with Jaron
Lanier," Afterimage (May 1991): 5-9.
reasoning takes place through images: when we are engaged in reasoning, we construct a mental
model of a situation which represents its essential features in the form of topological relations.79
Before it became possible to send Leonardo's images to remote corners of the Universe to
dramatically speed up scientific reasoning, before psychologists could claim that reasoning in
humans is a matter of manipulating mental images, a profound change in the cultural attitude
toward vision had to take place. This change occurred between the 1870s and the 1920s.
Logical thinking entails the representation of abstract ideas and, also, of the logical
relations between them. During this period, Galton, Venn, Freud, Eisenstein, and others put
vision to use for both purposes. More importantly, for the first time, we can find in their work the
In 1877 Galton's composite portraits visualized universal human types; in 1880, John
Venn made public a method for solving problems in logic by using graphic diagrams (section 3).
theory of how abstract notions and logical forms can be visualized (section 5). In the late 1920s
Eisenstein proposed that film can be used "to encourage and direct the whole thought process"
(section 4).
These different models of how vision can be used in reasoning represent the first stage in
the reversal of attitude towards the inadequacy of vision. The second stage arrived when, in post-
industrial society, the concern with the efficiency of the mind overwhelmed the concern with the
efficiency of the body. Now, the questions of how information can be coded, stored, retrieved,
and processed more efficiently give direction to the study of the mind, including the use of visual
from this new perspective, and they postulate the centrality of visual representations for human
To be able to appreciate the novelty of the modern paradigm of visual reasoning, I will begin by
discussing the classic philosophical positions on the relationship between vision and reason.
Many philosophers thought vision to be unsuitable for reasoning, first of all because throughout
the history of philosophy, reason was closely associated with logic. If, following its
formalization by George Boole and Gottlob Frege in the nineteenth century, logic has developed
into a sophisticated branch of mathematics, today seemingly removed from our everyday
which rational debates should be conducted.80 Therefore, it was Aristotle who originated the
tradition of discussing human reasoning and logic interchangeably. Locke, for instance defines
reason as "that faculty whereby man is supposed to be distinguished from beasts, and wherein it
is evident he much surpasses them." Claiming that the highest order of reason consists in "the
Sense and intuition reach but a very little way. The greatest part of our knowledge depends upon
deductions and intermediate ideas: and in those cases where we are fain to substitute assent instead
of knowledge, and take propositions for true without being certain they are so, we have need to
find out, examine, and compare the grounds of their probability.81
80 Antony Flew, ed., A Dictionary of Philosophy (London: Pan Books Ltd., 1984), 208-212.
81 John Locke, An Essay Concerning Human Understanding, ed. A.S. Pringle-Pattison (Oxford:
Clarendon Press, 1924),
2. I See, Therefore I ThinkBook IV, Ch. 17.
The equation of reason with logic persisted late into the twentieth century. Piaget
assumed that the final stage in the development of human thought processes was a kind of formal
logic analysis. Equally, Levi-Strauss's binary oppositions and Greimas's semiotic square are
based on stripped down formal logic. This logic is postulated as the structure of the "native
mind" (Levi-Strauss) or the structure of all signification (Greimas).82 Only in the last decades
have cognitive psychologists discovered that human judgments do not follow statistical
probabilities and tend to ignore information about the prior probability of an event;83 that rather
than having some uniform "laws of thought," humans employ a variety of heterogeneous
heuristics; and that in general the normative rules of logic do not describe human thinking.84
However, the idea of a separation between logic as a normative or prescriptive discipline and
human reasoning, which can be described and studied empirically and which does not follow any
strict rules of logic, is quite recent, and would be unimaginable for Aristotle or Locke.85
Therefore, in classical philosophy to reason is to carry out logical operations. How then
can one possibly reason through images? How can one represent visually "all," "some," "any" or
82 What is different about reason as imagined by Levi-Strauss and Greimas in contrast to Locke
is its new industrial efficiency: the concept of binary oppositions was brought by Jakobson from
MIT, the birthplace of information theory which demonstrated that binary code is the most
economical of all. According to Geoffrey Sampson, "what led Jakobson to the hypothesis that all
parameters are 'binary' seems to have been the mathematical notion that a transmission-code is
more efficient when it uses only independent binary choices." Geoffrey Sampson, Schools of
Linguistics (Stanford, CA: Stanford University Press, 1980), 250.
83 Gillian Cohen, The Psychology of Cognition, 2nd ed. (London and New York: Academic
Press, 1983), 190.
84 Michael I. Posner, and Gordon L. Shulman, "Cognitive Science," in The First Century of
Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 393.
85 A recent dictionary of philosophy still equates thinking with logic: "The mental activity of (a)
theoretical contemplation directed toward some object with a view to reaching a propositional
conclusion; or (b) practical deliberation directed toward some object with a view of reaching a
decision to act." Flew, A Dictionary of Philosophy, 352. Emphasis mine - L.M. An encyclopedia
of philosophy is equally categorical: "particular thoughts have some kind of logical form; they
may be categorical, hypothetical, disjunctive, universal, particular, and the like." Paul Edwards,
ed., Encyclopedia of Philosophy (New York: Macmillan, 1967), 8: 101.
other logical constants? How to represent truth and falsity? How to distinguish particular from
general?
Reason, of course, cannot proceed without the raw materials, collected for it by vision.
However, the role of vision is limited to the passive and unintelligent collection of data
empiricist philosophy in the nineteenth century, gave additional support to this conceptual
hierarchy by providing a topological scheme: the outside world enters through the filters of the
eyes and then is carried through visual pathways inside the head, to the brain, the site of "higher
mental faculties." Even today, psychological textbooks are organized around the same hierarchy,
inherited from empiricist philosophy: they start with chapters on sensation (vision and other
senses) and end with cognition (thinking, memory, text comprehension, etc.)
These are some of the ways in which philosophy has demoted vision in relation to reason: it can't
represent logical forms; it can't generalize; its only job is to collect the raw material reason
needs.86 However, this philosophical tradition is counterbalanced and upset by other traditions.
For, while subordinating vision to reason, philosophers also often claimed that reason in fact was
born of vision. Moreover, it can be said that they could only conceive of reason in visual terms.
For instance, while in terms of the temporality of everyday mental functioning, vision
serves reason, its task being simply the collection of data for reason to work with, in terms of a
much longer temporality of phylogenetic development of the human race, vision is often
postulated as reason's origin. In particular, Locke, Herder, Vico, Nietzsche, and others claimed
that abstract categories of language evolved from concrete visual (and also bodily) experience.
Locke: "I doubt not but, if we could trace them [words] to their sources, we should find, in all
86 In his Downcast Eyes, unavailable to me at the time of this writing, Martin Jay discusses the
role of vision in modern French philosophy. Martin Jay, Downcast Eyes: the Denigration of
Vision in Twentieth-century French Thought (Berkeley: The University of California Press,
1993).
languages, the names which stand for things that fall under our senses to have had their first rise
from sensible ideas."87 Nietzsche: "Everything which makes man stand out in relief against the
animal depends on this faculty of volatilizing the concrete metaphors into a schema and therefore
resolving perception into an idea."88 Vico: "It is noteworthy that in all languages the greater part
of the expressions relating to inanimate things are formed by metaphor from the human body and
Vico's ideas, in particular, have been recently revived in the work of the linguist George
Lakoff. Lakoff argues that most, if not all, abstract semantic concepts and abstract human
reasoning are based on metaphorical mapping of spatial concepts (such as inside-outside, part-
whole, and others) which are inherently meaningful to human beings since they have bodies.
Like Vico before him, Lakoff challenges three traditional hierarchies -- between concrete
and abstract thought, between vision and reasoning, and between literal and metaphorical
claims that in thinking we project the structures of our sensory and bodily experience to abstract
87 Qtd. in Tzvetan Todorov, Theories of the Symbol (Ithaca: Cornell University Press, 1984),
237.
88 Friedrich Nietzsche, "On Truth and Falsity in their Ultramoral Sense," in Early Greek
Philosophy and Other Essays (New York: The Macmillan Company, 1924), 181.
89 Thomas Goddard Bergin and Max Harold Fisch, The New Science of Giambattista Vico
(Ithaca: Cornell University Press, 1968), 129. Discussing the metaphorical origin of abstract
concepts, Vico goes on to claim that the common assumption "that prose speech is proper
speech, and poetic speech improper" reverses the true history of human mind: "first nations"
have spoken and thought in poetry, i.e. through the metaphorical tropes (131).
situations.90 The semantics of natural language is built upon the meaningful "image-schemes"
which are the generalizations of what Lakoff calls our "basic human experience." We live inside
our bodies; our bodies consist of parts which are linked. Therefore, we conceptualize all
situations in terms of containers (for instance, an enormous number of expressions use in and
out), part-whole relations (divorce is splitting up) and links (we make connections, break
relations). For Lakoff, metaphors such as these are not the exception but the norm in language,
concepts. Because spatial (more precisely, topological) relations are an important part of our
experience, spatial metaphors are at the core of the semantics of human languages; reaching
goals, moving out, walking over an enemy, etc. Not only does the semantics of natural languages
turn out to be based on spatial metaphors but abstract reason itself is the manipulation of
spatially represented information: "natural language reasoning makes use of at least some
them, focusing on part of them, etc."91 In fact, reason is a special case of more general mental
operations which are needed for perception: "Abstract reasoning is a special case of image-based
With this claim, Lakoff transcends the earlier views of Locke, Vico, and Nietzsche
according to whom abstract reason phylogenetically evolved from vision. It is one thing to
suggest that abstract concepts have evolved from "sensible ideas" (Locke) or perceptions
(Nietzsche). It is quite different to start with the assumption, as Lakoff does, that abstract ideas
are nothing but a limited set of visual (more precisely, topological) forms (such as inside-outside,
Here the modern obsession with the visualization of reasoning leads to the most dramatic
break with the traditional conceptualization of the relation between reason and vision. Modern
writers such as Lakoff or Johnson-Laird, claim is that there is only one kind of cognitive
operations, which reason and vision share. These operations are fundamental, original to vision:
reason borrows these operations; it does not invent any of its own.
91 Ibid., 149.
92 George Lakoff, "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?"
Cognitive Linguistics 1, no. 1 (1990), 65. Perhaps even this very brief presentation of Lakoff's
ideas already makes apparent a substantial problem of his theory -- the rather unquestioned
notion of "basic human experience."
Johnson-Laird has proposed that when we are engaged in reasoning, we construct a
mental model of a situation which represents its essential features in the form of topological
relations.93 The conclusion then simply can be "read off" from the representation. For instance,
how can we see the truth of a modus ponens: "If X is an A, and if all A's are B's, then X is a B"?
the right conclusion is invariably arrived at. According to Johnson-Laird, we don't have to
But do these views of Lakoff and Johnson-Laird really represent a break? Perhaps this
revolt against the condemnation of vision in the philosophical tradition was prepared by this
tradition itself? For, while declaring vision unsuitable for reason, philosophers conceived of
reason in visual terms all along: witness the persistence of the metaphors of eye, light, mirror,
revelation and unveiling, from Plato to Descartes, Hegel, Heideger, and Lacan as standing in for
"Idea" comes from the Greek verb "to see" and is frequently linked to ancient optics and
theories of perception. According to Aristotle, in thinking ideas function to substitute for the
absent objects as their images: "The reasoning mind thinks its ideas in the form of images; and as
the mind determines the objects it should pursue or avoid in terms of these images, even in the
Hume thinking involved a sequential series of images; these images are tied to certain habits of
the mind to move from one image to another. Reason was equated with an eye which compared
all objects of thought laid out before it. As summarized by Jonathan Crary,
But whether it is Berkeley's divine signs of God arrayed on a diaphanous plane, Locke's sensations
'imprinted' on a white page, or Leibniz's elastic screen, the eighteenth century observer confronts a
unified space of order...on which contents of the world can be studied and compared, known in
terms of a multitude of relationships. In Rorty's words, "It is as if the tabula rasa were perpetually
under the gaze of the unblinking Eye of the Mind..."96
95 Qtd. in Jean Matter Mandler and George Mandler, eds., Thinking: From Association to
Gestalt (New York: John Wiley & Sons, Inc., 1964), 9.
96 Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth
Century (Cambridge: The MIT Press, 1990), 55.
It was Heidegger who, in What is Called Thinking?, insisted on the visual basis of
Western philosophy (both epistemology and ontology) from Plato to Hegel most strongly.97
presence and permanence, 'seeing' is especially apt to serve as an explanation for the grasping of
1968).
what is present and what is permanent."98 For Plato, Being is idea, present as outward
appearance. However, already in Plato the seeds of the split between object and subject are
implanted: Being is presence but at the same time is what man brings before his eyes. From this
moment on, idea is gradually transformed into perception and representation, that, according to
Heidegger, characterizes the Cartesian cogito. Descartes laid a new foundation in privileging the
thinking subject, but the certitude of the cogito derives from its visibility: as Heidegger explains,
Indeed, Cartesian rationality is based on the notion of a subject which is outside himself,
observing himself contemplating the subject. This is the price Descartes pays for his
condemnation of images and imagination in relation to reason: he can only conceive of reason in
visual terms. Before "I think therefore I am" comes "I see therefore I think."
53.
99 Qtd. in Ibid., 54.
Like Siamese twins, vision and reason are inseparable in classical philosophy, being connected
in a complex network of dependencies, only a few of which have been discussed above. On the
one hand, vision is but a servant of reason, unable to represent logical relations and abstract
concepts and therefore, clearly unsuitable for reasoning. On the other hand, vision achieves its
revenge, consistently slipping in as the phylogenetic origin of reason and as the basis of
represents a break with this classical tradition or is still a part of it, he clearly lives in a different
age. Vision and reason are no longer thought of as antagonists; on the contrary, "visual
reasoning" is now taken for granted and even promoted as a new highly efficient means of
decision making and/or propaganda, suitable for the age of mass mobilization, mass education,
with Stalin's plans to industrialize Russia overnight, dreams of turning workers into dialectical
thinkers by passing them through the darkness of special incubators of cognitive skills -- movie
houses, thus shortening the duration of the learning process to hours. Heidegger is also a
contemporary of Benjamin who wants to teach German workers dialectical thinking even
without any external aids (such as film) -- by simply juxtaposing images, in montage-like
fashion, in the mind. Finally, Heidegger's younger compatriot, Rudolf Arnheim, becomes the
most outspoken champion of thinking in images, arguing that artistic skills should become the
The signs of the arrival of the new age of visual reasoning start to appear in the last
decades of the nineeteenth century. In 1880 English logician John Venn published an article "On
Thinking."
101 Martin Gardner, Logical Machines and Diagrams, 2nd ed. (Chicago: The University of
Chicago Press, 1982), 32.
article presented a method to use graphic diagrams to represent complex logical relations and it
soon became the standard for logicians. Venn's method not only made it possible to illustrate the
rules of inferential logic in visual forms; but much more importantly, now one could reason
automatically by simply drawing pictures (fig. 6). Consider, for instance, the following
inference: "If all A is contained in B, and if C is contained in A, then C must also be contained in
B." Its validity becomes obvious once the terms are represented in a diagrammatic form.
Likewise, such inferences as "All men are mortal. Socrates is a man. Therefore, Socrates is
mortal" can be arrived at through similar diagrams. Images thought to be unsuitable for
representing logical relations and thus unsuitable for reasoning turn out to be its perfect indeed
its natural medium since they can represent topological relations with sufficient precision and
complexity.
If Venn's diagrams showed that vision can be used to arrive at logical relations, Galton's
composite photographs proved that abstract concepts can be represented visually. Sir Francis
Galton, a statistician and a cousin of Charles Darwin, a founder of eugenics (a project of social
betterment through planned breeding), and the author of highly influential psychological texts,
the next three decades.102 Fabricated by a process of successive registration and exposure of
portraits onto a single plate, Galton's composites were thought to constitute true statistic
averages, representing human types -- a criminal, a prostitute, an Englishman, a Jew, and others
(fig. 7). Galton wrote about his composite pictures that they are "much more than averages; they
are rather the equivalents of those large statistical tables whose totals, divided by the number of
classes and entered on the bottom line, are the averages. They are real generalizations, because
102 Allan Sekula, "The Body and the Archive," October 39 (1987): 40.
103 Qtd. in Ibid., 47.
Galton not only claimed that "the ideal faces obtained by the method of composite
portraiture appear to have a great deal in common with...so-called abstract ideas" but in fact he
proposed to rename abstract ideas "cumulative ideas." In contrast to the human mind, "a most
imperfect apparatus for the elaboration of general ideas," Galton championed his composite
photographs, which, being mechanical and precise, were much more reliable for arriving at
abstract representations.104
With his photographs, Galton not only proposed that universals may be represented
through generic images; he actually objectified and materialized them. Plato's ideas were given
concrete form: they could now be touched, copied, fabricated, multiplied, distributed, etc.
If Venn's and Galton's inventions overturned the traditional claims about the impossibility
of visual reasoning, they did that not through new philosophical arguments about vision but first
of all by inventing new ways to use it. The latter invented a new visual technology: representing
universals through composite photographs. The former introduced a new visual technique, a new
way to employ visual representations, better yet, a new graphic sign system -- diagrams which
In effect, Venn's technique and Galton's technology, which gave logical forms and
abstract ideas material tangible form, externalized reasoning. What before was a mental process,
a uniquely individual state, now became part of a public sphere. Unobservable and interior
processes and representations were taken out of individual heads and put outside -- as drawings,
teaching and propaganda, standardized, and mass-distributed. What was private became public.
What was unique became mass-produced. What was hidden in an individual's mind became
shared.
We should not be surprised, then, that the revolutionary new medium, the medium of mass
society par excellence -- cinema -- was immediately proclaimed to be the machine for the
externalization of private mental functions and states. In 1916 Hugo MŸnsterberg, a Professor of
Psychology at Harvard University and one of the founders of the fields of industrial and applied
psychology, published The Film: A Psychological Study, today canonized as one of the earliest
lies in its ability to reproduce, or "objectify" various mental functions on the screen: "The
photoplay obeys the laws of the mind rather than those of the outer world."107 In contrast to the
theater, where the action is constrained by the limitations of physical reality, film is free to shape
arbitrarily its material, closely approximating flashes of memory, the flights of imagination, and
to the progression of time, in film the action can suddenly jump back and forth, just as in an act
of imagination.
MŸnsterberg was not content to point out the analogy between film and mental life; in an
astounding analysis, he correlated the main cinematic techniques to different mental functions
such as attention and memory, one-to-one. For example, in the close-up, "everything which our
mind wants to disregard has been suddenly banished from our sight and has disappeared,"
analogous to how our attention selects a particular object from the environment. Similarly, the
"In both cases," MŸnsterberg wrote, "the act which in the ordinary theater would go on
in our mind alone is here in the photography projected into the pictures themselves."108 The
psychological laboratory became indistinguishable from the movie house; the textbook of
experimental psychology -- from the cinematographer's manual. The mind was projected on the
The next logical step was taken by a German psychologist Kurt Levin who, in 1924-25, was the
first to use film in his experiments. He wrote that "fiction film attempts to objectify certain
psychological processes for the viewer. Psychological (scientific) film studies to what extent
these psychological processes can be objectified."109 Soviet psychologist A.P. Luria, who
planned to establish a psychological laboratory in Moscow in cooperation with the State Film
Academy, acquainted Levin with Eisenstein, who attended the shooting of one of Levin's films
connection between the interest in visual reasoning and the rise of mass communication, of
which film was a major vehicle. The emergence of new mass societies in the earlier part of this
century dictated the necessity to communicate ideological concepts to mass populations which
were often illiterate.111 In the 1920s Eisenstein boldly conceived a screen adaptation of Marx's
Capital as a way to efficiently bring about the political enlightenment of Russian audiences,
especially the peasants who would not sit through a political lecture but, attracted by the
"novelty" of a movie projector, would come to see movies, regardless of what was shown.112
Unprecedented as his project was, its radicalism lay not only in the decision to visualize the
abstract notions and logic of Capital but in the method employed, which, according to Eisenstein,
would directly provoke dialectical thinking in audiences.113 Jacques Aumont concludes that for
Eisenstein, "the object privileged in Marx's work is not a theoretical one, like any of the key
concepts from Capital. It is at another level entirely that Eisenstein selects his true object -- the
Marxist method itself."114 Thus it was not simply a matter of the modern redeployment of the
directions of the 1492 sermon: "...Images of the Virgin and the Saints were introduced...on
account of the ignorance of simple people, so that those who are not able to read the scriptures
can yet learn by seeing the sacraments of our salvation and faith in pictures."115 The viewers of
Capital were not only to learn the scriptures of the new atheistic religion; they were to learn the
process of reasoning.
111 For instance, according to the 1926 census, out of every 1,000 citizens of the U.S.S.R., only
445 were literate. Peter Kenez, The Birth of the Propaganda State. Soviet Methods of Mass
Mobilization, 1917-1929 (Cambridge: Cambridge University Press, 1985), 157.
112 Kenez, The Birth of the Propaganda State, 220.
113 The pioneering work of Annette Michelson was important in bringing my attention to
Eisestein's Capital project. Annette Michelson, "Reading Eisenstein Reading 'Capital'," October
2 (1976): 27-38; October 3 (1977): 82-88.
114 Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and
Indiana University Press, 1987), 163.
115 Qtd. in Baxandall, Painting and Experience in Fifteenth Century Italy, 41.
It is significant that the most categorical statement by Eisenstein on the possibility of
"filmic reasoning," reasoning through images, appears in the context of his discussion of the
Maintaining the denotation of "God," the images increasingly disagree with our concept of God,
inevitably leading to individual conclusions about the nature of all deities...a chain of images
attempted to achieve a purely intellectual resolution, resulting from a conflict between a
preconception and a gradual discrediting of its purposeful steps.
Step by step...power is accumulated behind a process that can be formally identified with that of a
logical deduction...The conventional descriptive form for film leads to the formal possibility of a
kind of filmic reasoning. While conventional film directs emotions, this suggests an opportunity to
encourage and direct the whole thought process as well.116
Far from simply representing God or deities, as they did for centuries, here images serve a totally
new function -- to provoke and direct reasoning, reasoning of a particular kind -- "Marxist
dialectics." In accordance with its principles, as canonized by the official Soviet philosophy,
Eisenstein wants to present the viewer with the visual equivalents of thesis and anti-thesis so that
the viewer can then proceed to arrive at synthesis, i.e. the correct conclusion, pre-programmed by
Eisenstein.
"The content of CAPITAL (its aim) is now formulated: to teach the worker to think
viewers would become self-sufficient thinkers, learning the skill of "Communist decoding of the
world," each a walking camera, snapping pictures of visual thesis and anti-thesis, the brain
and effectively.
116 Sergei Eisenstein, "A Dialectical Approach to Film Form," in Film Form: Essays in Film
Theory, ed. Jay Leyda (New York: Harcourt Brace and World, 1949), 62. Emphasis mine -- L.M.
117 Sergei Eisenstein, "Notes for a Film of 'Capital,'" trans. Maciej Sliwowski, Jay Leuda, and
Annette Michelson, October 2 (1976): 10.
The proclamation that I'm going to make a movie of Marx's Das Kapital is not a publicity stunt. I
believe that the films of the future will be found going in this direction (or else they'll be filming
things like The Idea of Christianity from the bourgeois point of view!) In any case, they will have
to do with philosophy...the field is absolutely untouched. Tabula rasa.118
Yet, Einstein's theory was not an isolated development. Many in the artistic left of the 1920s
shared a similar belief in the cognitive power of new visual forms such as montage. In the late
1920s Alexander Rodchenko promoted the use of montage sequences in graphic design and, like
individual image corresponded to a single concept, and thinking was thought to be provoked
"dialectical seeing," central in his unfinished Passagen-Werk project, also depends on montage,
but within a single frame, so to speak. "Dialectical seeing" was conceived by Benjamin as a way
to grasp the forms of the present by looking to the past and to the future in the same instant,
devices of close-up, cut-back, and montage. Different, in effect, as they are, these developments
are symptoms of a single social imaginary at work: to make reasoning more efficient and at the
same time more controllable by externalizing it and rendering it visible. This imaginary can be
glimpsed in the new techniques of visual representation, such as Venn's diagrams. It also governs
the development and the promotion of new visual technologies, such as Galton's composites or
Eisenstein's dialectical montage. Finally, this imagination can be also read in theoretical
speculations about how abstract concepts and logical forms can be represented as images in
Is there a single person whose work brings into focus the modern preoccupation with
visual reasoning -- whether in the form of new theoretical models, new techniques of visual
person must be Sigmund Freud. He is not only more explicit in his defense of the possibility of
visual reasoning; he develops a systematic theory of how abstract notions and logical forms can
Seeing: Walter Benjamin and the Arcades Project (Cambridge: The MIT Press, 1989).
be visualized. This theory becomes one of the cornerstones of a new science, devoted, we should
note, to externalization, bringing into the open, into public view the most intimate, private, and
this, the new science goes well beyond MŸnsterberg's notion of the externalization of memory
and attention by the techniques of film -- after all, these mental functions can be voluntarily
controlled and commanded by consciousness. Indeed, MŸnsterberg and his fellow experimental
psychologists at the turn of the century scrutinized the mind through the technique of
introspection -- which, by default, limited them to the study of only those mental acts and
functions that could be voluntarily controlled and consciously observed. Instead, the new science
postulates the existence of unobservable, uncontrollable psychic forces, which, lying deep below
The latter hypothesis, it is often said, constitutes the most dramatic attack on Cartesian
rationality launched in this century. However, it should be noted for the record that the new
science also gives rationality and reason even more power than before -- and not only because it
tamed the unconscious for the consciousness but also because it gained another ground for
reason -- vision. Having postulated the existence of subterranean unconscious ideas and
phenomena can be brought to the surface, and rendered visible and harmless. It claims that
unconscious ideas, groups of ideas, and even arguments are transformed into the form of dream-
images. By analyzing these images, an analyst can recover these ideas and arguments.
rise of visual reasoning.121 This work exemplifies a new understanding of vision -- as not
inferior to reason but, on the contrary, as its equal, as a medium quite capable of representing
abstract ideas and logical arguments. According to Freud, dreams can translate into visual form
such concepts as "a high-placed official" (Freud 378) or "superfluous" (441). They can also
represent logical connection (349), causality, negation (361, 372) or contradiction (470). In short,
121 All subsequent refererences in this chapter to Freud, unless overwize indicated, are to
According to Freud, the mind's contents, ideas and logical arguments, are securely hidden
from its owner, from the psychoanalyst and from the public, lurking, invisible and unreachable,
in the archaic cave of the unconscious. They travel through the darkness of association networks,
like newly formed urban masses, riding to and from work in the underground subways, which in
the end of the nineteenth century were rapidly constructed in major metropolitan centers. Like
these urban masses, always a danger, potentially lurking in the darkness of the labyrinths of city
streets at night, ideas are formed, dissolved and rearranged in the total darkness, which the
This, of course, cannot be tolerated. Streets are streamlined and, even more importantly,
well lit. Indeed, the turn of the century is often called "the age of electricity" due to its obsession
with electrical light. But does not this light provide the best way of fighting invisibility and
assuring externalization? Similarly, ideas and reasoning need to be externalized, brought into the
open, rendered visible. Luckily, Freud assures us, the psyche already possesses a process which
transforms the invisible contents of the unconscious into images within our reach. All the
psychoanalyst needs to do is to properly interpret these dream images by tracing them back to
representability in the peculiar psychical material of which dreams make use -- for the most part,
that is, representability in visual images," (379) Freud provides a number of examples of how
dreams transform abstract concepts and thoughts into pictures.122 Here are a few of them:
122 Mieke Bal's seminar on "Interpreting Culture" at the University of Rochester brought my
attention to this point. For the original discussion of considerations of representability see
Samuel Weber, The Legend of Freud (Minneapolis: University of Minnesota Press, 1982).
"A high-placed official" is represented by a high tower (378).
"Working my way through" is represented by "a long knife under a cake, as though to lift
(441).
Legs (part of the body) are represented through pillars or columns (382).
Is there a single logic these examples follow? Yes, and we have already encountered it in Lakoff.
Lakoff, as we have seen, claims that abstract concepts and abstract reasoning take shape in the
process of extending the structures of human sensory and bodily experience to abstract
situations; and that even the most abstract categories are metaphors, based on such topological
relations as part-whole and inside-outside. To use Freud's examples, such concepts as "a high-
placed official" or "working my may through" are meaningful in the first place because they
Lakoff postulates that behind all seemingly abstract concepts of natural language lie
concrete "image-schemes." If we accept Lakoff's theory for a moment and read Freud's examples
through its terms, it appears that in dreams abstract concepts and ideas turn back into the "image-
schemes" from which they evolved in the first place. Thus, "a high-placed official" becomes
Of course, the point is not that Lakoff's theory "explains" Freud or that Freud
"anticipates" Lakoff; rather, it is that both implicitly rely on the notion which has always figured
centrally in the speculations about the origin of language and reason (including, as we have seen,
of such thinkers as Locke, Herder, Vico, and Nietzsche): that abstract concepts have evolved
from concrete ones, the latter directly connected with perceptual experience. Freud, however,
adds a new and important notion: regression. If in dreams abstract concepts and abstract thinking
itself return to their "origin" as images and "image-based reasoning" (Lakoff's term) this
The notion of regression is the key for understanding Freud's theory of visual thinking.
Not only does it explain why in dreams abstract concepts return to their visual origins, it also
partly clarifies why dreams invariably rely on visual means of representation at all: "what is
older in time is more primitive in form and in psychical topography lies nearer to the perceptual
end" (587). Regression to the visual mode in dreams, then, operates in three ways: regression to
an older representational mode (formal regression); topographic regression (to the perceptual
system); and temporal regression to the older (in the evolutionist sense) psychical structures.
There is also another, fourth meaning of regression to the visual mode: regression to the visual
scenes witnessed in childhood ("primal scene"). Therefore, the visual means of representation
and expression" (587). Vision, in Freud's diagnosis, is seen as the primordial, original language
of mankind.
Should we be surprised that, according to Freud, Vienna's inhabitants circa 1900, modern
and respectable by day, at night totally lose their modernity, shed away progress, and instead
immerse themselves in archaic visual illusions? No, if we recall the view, frequently voiced in
the earlier part of the twentieth century, that the new visual forms of mass culture represent a
return, indeed, a regression to humanity's archaic stage. Most frequently, it was cinema which
was seen (for instance, by Nabokov) as a time machine, where the metropolitan masses, stripped
of their individuality, subjectivity, or even simply basic interiority, in total darkness (like that of
prehistoric caves, even darker than the cave described by Plato), gave up the light of
Enlightenment, Civilization and Reason, and instead were chanting in unison...a collective body,
mesmerized by visual forms, moving before them on the screen, made possible through the
represents the regression to the original language of mankind -- the language of pictures;
however, he does not think that this entails the loss of rationality, logic, and reason. Freud not
only claims, as we have just seen, that the dream-work has the means to represent singular ideas,
even abstract ones, but he also confidently describes the mechanisms by which the dream-work
"What representation do dreams provide for 'if,' 'because,' 'just as,' 'although,' 'either-or,'
and all the other conjunctions without which we cannot understand sentences or speeches?," asks
Freud. His initial answer is negative: "In the first resort our answer must be that dreams have no
means at their disposal for representing these logical relations between the dream-thoughts"
(347). But astonishingly, contradicting his own statement, Freud then proceeds to elaborate the
mechanisms by which the whole range of logical relations can be represented in images:
Spatial proximity is used to represent logical relation: "Whenever they [dreams] show us
two elements close together, this guarantees that there is some specially intimate connection
An alternative ("either-or") may be represented by dividing the dream into two pieces of
Negation is usually discarded (353) -- but later Freud gives examples to the contrary
(361, 372).
Similarity ("just as") is the "favorite" of dreams -- "this relation, unlike any other, is
capable of being represented in dreams in a variety of ways" (354), including "composition" -- the
process of constructing composite images in dreams is evidently the same as when we imagine or
Freud does not explicitly connect his descriptions of the dream-work's mechanisms of the
visualization of logical relations with his view of the dream-work as regression to archaic mental
structures. However, his characterization of these mechanisms only makes sense if we assume
that, like everything connected with the dream-work, they owe their character to regression. The
connection is provided in the work of another writer, Lucien LŽvy-Bruhl, a leading French
anthropologist and Freud's contemporary. Like Freud, LŽvy-Bruhl was concerned with the
nature of archaic mental structures. But while Freud located these structures in the unconscious
of every modern civilized individual, LŽvy-Bruhl claimed to have found them in the mental
enjoyed wide popularity in the first decades of the twentieth century. According to LŽvy-Bruhl,
the thinking of primitive people follows a logic quite different from our own "normal" logic.
First of all, primitive people confuse temporal succession with causality. If they witness the two
events following each other, they assume that the two are causally related. Thus, LŽvy-Bruhl
recounts that if one of them sees a snake fall down from the tree in front of him and learns soon
thereafter that his son is dead, he will relate these two facts.123 Second, primitive people believe
that if two objects have been physically connected, they continue to influence each other when
separated. For instance, some primitive people are careful to hide a babies' milk-teeth -- for if
somebody finds them, then the new teeth will grow improperly. Summing up this feature of
primitive logic, LŽvy-Bruhl writes: "Primitive people confuse the antecedent with the cause.
This would be a very common error in reasoning that is known as post hoc, ergo propter hoc
sophism."124
These "laws" of "primitive thinking," as described by LŽvy-Bruhl, are the same ones that
Freud claims govern the representation of the relations between dream-thoughts. Savages think
that two events following each other are causally related (LŽvy-Bruhl) -- dreams represent
causality by temporal sequence (Freud). For savages, two objects physically connected are also
logically related -- dreams signify a logical connection by the spatial closeness of elements or by
the simultaneity of time (Freud, 349). Moreover, as "savage mind," the dream-work finds
similarity everywhere, seeing logical relations where there are none: "One and one only of these
logical relations is very highly favored by the mechanism of dream-formation; namely, the
relation of similarity, consonance or approximation -- the relation of 'just as.' This relation,
unlike any other, is capable of being represented in dreams in a variety of ways" (354). The
visual language of the dream-work -- the archaic language of the psyche, as described by Freud -
- is the same as the primordial "mental language" still at work in modern savages -- as claimed
by LŽvy-Bruhl.
123 Lucien LŽvy-Bruhl, Les functions mentales dans les sociŽtŽs primitives (Paris: 1910), 72.
124
6. Ibid.,
The Rise73.
of the Diagram
Freud does not just insist that abstract ideas and logic can be visualized or provide a theoretical
model of how this can be done in principle. He practices what he preaches. What makes The
Interpretation of Dreams a truly modern text, compatible with the age of illustrated magazines,
color printing, scientific documentaries, dialectical montage, and other techniques and
technologies of visual communication is not the abundance of illustrations or figures (in fact,
there are only a couple). Freud does not have to include illustrations, because many of his
theories, and indeed his overall theory of the psyche (the famous topographic model), are based
on simple visual models, easy to grasp and easy to remember. And this is what makes his text
modern. The reader is invited to disregard the complexities of real anatomical data and instead to
something of the kind" (574). According to Freud, this apparatus is made up of just a few
components (called by Freud agencies or systems), which are arranged in spatial order (574-
583). This "spatial analogy" (653) allows Freud to visualize any psychic process as a simple
It is as though Freud senses that the construction of such models is very effective in
pushing the reader to "see" the self-evidence of the appropriate conclusions. Thus, after
presenting the typographic model of the psyche, Freud uses it to explain the newly introduced
concept of regression: "I believe that the name 'regression' is of help to us in so far as it connects
a fact that was already known to us with our schematic picture, in which the mental apparatus
was given a sense or direction. And it is at this point that picture begins to repay us for having
constructed it. For an examination of it, without any further reflection, reveals a further
To clarify the logical relation between the conscious and the unconscious, Freud again
relies on the same technique of constructing a visual diagram. From the point of view of the
ontogenetic and phylogenetic explanation of the formation of the psychical apparatus, primary
processes, as their name implies, "are present in the mental apparatus from the first" (642) while
secondary processes appear only later and never achieve full domination. Thus, concludes Freud,
the primary processes (and corresponding to them the system of the unconscious) constitute the
essence of psychical life: "The unconscious is the true psychical reality" (651). To emphasize
this point and to make sure the reader remembers it, Freud "visualizes" it in the form of a simple
image of a sphere within a larger sphere: "The unconscious is the larger sphere, which includes
within it the smaller sphere of the conscious" (651). He then "reads off" his conclusion from this
Freud's skillful use of visual models, analogous to Venn's diagrams, to represent the
logical relations between the terms in his theories is unusual in that Freud painstakingly
describes a visual model verbally, rather than directly presenting the reader (or should we say,
the viewer?) of his text with a picture. It is as though Freud fully realized the power of diagrams
to represent complex concepts, models and arguments but, circa 1900, he was still hesitant to
In 1933 Freud finally includes a real diagram in The New Introductory Lectures on
Psychoanalysis to illustrate the relation between two triads of concepts: id, ego, and superego,
and unconscious, preconscious, and conscious. Freud's diagram, which effectively signifies a
complex set of relations by just a few lines, is representative of a whole family of new techniques
of visual representation, new to the modern period (fig. 8). These techniques have risen to fill
what can be called the cognitive needs of modernization: to quickly communicate more and more
complicated relations between various entities, to manipulate huge sets of data (whether in
science -- which became more and more the problem of managing collected data -- or in the
relations -- in short, to represent reasoning visually and to reason with the help of vision. These
techniques of visual representation comprise what semioticians call the graphic sign systems --
signifying systems, employing a particular set of graphic conventions to represent particular
kinds of objects.125
We have already encountered one of these new graphic systems -- Venn's diagrams.
These diagrams rely on the unique property of vision, which, along with touch, as Rudolf
Arnheim points out, is "the only sensory medium that conveys such spatial properties as
inclusion, overlap, parallelism, size, etc., with some precision."126 This property makes possible
the precise representation of the logical relations between classes in graphic form. But, while in
principle visual images are capable of representing spatial properties, not every image can
effectively do so. For instance, continuously varying tonal gradients are useless, whereas discrete
elements are effective. And these elements, to be easily readable, must have enough visual
difference among them. Therefore, Venn's diagrams are typically drawn using simple lines. The
classes are represented through circles (a representation in the form of rectangles is possible, but
it would be ambiguous, leading the viewer to believe that the shapes of figures code some
additional logical relation). If circles overlap, the common area (representing exclusive "and") is
In short, we are dealing with a distinct graphic system, employing particular kinds of
graphic marks, optimized for the representation of particular references -- a logic of classes.
This graphic system accentuates one property of images (the ability to represent spatial
properties such as inclusion) at the expense of other properties, for instance, the ability to
Venn's diagrams is an example of a specialized graphic sign system. Their use is limited
to a narrow professional domain of logic. In the same period, another graphic sign system for
visualization of logical relations comes into use (fig. 9). In contrast to Venn's diagrams, the
125 The most comprehensive study of graphic sign systems is Jacques Bertin, Semiology of
Graphics (Madison, WI: The University of Wisconsin Press, 1983).
126 Arnheim, "A Plea for Visual Thinking," 143.
graphic marks of this sign system are now everywhere, seemingly the very air of which the
What I have in mind is the gradual emergence of greatly simplified graphic marks which
came to form a notation, a sort of an erector construction set, whose parts can be used to quickly
dependencies. This construction set includes boxes, arrows, vectors, solid and dotted lines, and
so on. Arrows can denote such relations as "belongs to," "relates to," "is a member of a class of,"
"causes," etc. Boxes, on the other hand (or simply any area separated by lines) can represent a
physical object, the state of a system, a stage in a process or any abstract entity. Freud's drawing
circa 1933, where a few encircled areas together with dotted and solid lines represent a complex
relationship between psychical agencies, illustrates the power of this graphic system, which I
accidental. A construction set includes some parts with more or less fixed functions (for instance,
wheels for moving models), while others (bars with holes) can be used for a variety of purposes -
- to hold other parts, to represent parts of a model, to be stationary or to move. Similarly, while
some elements of the diagrammatic system have relatively fixed meanings (for instance, wavy
brackets), others, such as arrows or dotted lines, can have numerous meanings, depending on
This leads us to the second feature. In diagrammatic language, the same few symbols can
represent anything: a box, for instance, may stand for an idea, a theory, a building, 200 buildings,
a citizen, an army, or a state of a physical system. Unlike the users of verbal language who learn
particular semantics, the users of the diagrammatic system learn the general conventions of
syntax. It is enough to know that a box represents an entity -- regardless of what the entity may
be. In this, diagrammatic language, used to represent a set of entities and the relations between
them, functions exactly like the symbolic language of logic: a properly executed logic diagram is
isomorphic with a logical statement.127 But this power is achieved through the standardization
ideas or objects, but rather the abstract relations evoked in any process of reasoning: "a part of;"
"leads to;" "associated with." It is not accidental, therefore, that this language of arrows and
boxes is what recent generations of computer programs, aiming to help reasoning or to imitate it
(outliners, hypertext, project management tools, idea generators, and the like) provide their users
with. The mental construction set became the graphic toolbox, always just a mouse click away.
argument, popularized since the adaptation of Macintosh user interface, that it helps the user to
recognize familiar symbols and commands regardless of the program she is using (fig. 10).
simplification. While drawings were used for centuries to represent schematic information rather
accompanied by the simplification of graphic symbols -- boxes, arrows, and the like. (This was
also accompanied by the streamlining of typography.) Now, it becomes much easier to "read off"
logical relations represented by the diagrams, since there are fewer graphical marks to scan.
Obviously, we would not find a sharp "break" in graphic paradigms, with a new graphic system
appearing overnight. According to Ernst Gombrich, the arrow does not acquire its meaning as
pointer or vector until the eighteenth century. And even then, he could only find it in a couple of
Similarly, we can find other abstract graphic marks employed for centuries in graphic systems
used in particular professions -- for instance, engineering drawing. Gradually, however, these
symbols leave these professional contexts and enter visual culture at large.
Finally, what is the relation between the development of the diagrammatic graphic
system and the overall movement in modern visual culture towards the simplification,
In the opening chapter of Techniques of the Observer, Jonathan Crary points out an
inherent contradiction present in most standard accounts of modernism. On the one hand, these
accounts postulate the emergence of new radical models of perception and representation,
summed up in the notion of abstraction, whose first signs appear with Manet, impressionism or
post-impressionism and which fully blossoms with Kandinsky, Kupka, Mondrian, and others. On
the other hand, it is also postulated that in the same historical period new popular visual forms
developed (first photography and then cinema), further solidifying realism. As Crary notes,
128 Ernst Gombrich, "Pictorial Instructions," in Images and Understanding, ed. Barlow Horace,
Colin Blakemore and Miranda Weston-Smith (Cambridge: Cambridge University Press,
1990) , 28.
"Classical space is overturned, so it seems, on one hand, but it persists on the other. This
conceptual division leads to the erroneous notion that something called realism dominated
popular representational practices, while experiments and innovations occurred in a distinct (if
Crary's way to solve this contradiction is to claim, throughout his book, that the so-called
realistic forms of popular visual culture, based on photography, and abstract art are both
grounded in the emergence of "subjective vision" in the early nineteenth century, in which the
link between representational signs and external referents is severed. "I argue," writes Crary,
"that some of the most pervasive means of producing 'realistic' effects in mass visual culture,
such as stereoscope, were in fact based on a radical abstraction and reconstruction of optical
century."130 What his anti-realist argument leaves out, however, is any consideration of the
between the shapes of objects as definitely located in space and their pictorial representations"
(William Ivins) became greatly amplified. In Ivins' words, photography is "a form of
picturemaking that is not only precisely duplicable but one in which geometrical perspective is
so inherent that today the camera is used as a surveying and measuring instrument."131 Rather
than being anti-realistic, photography became the perfect technology of instrumental realism
(Allan Sekula's term), literally allowing its users to have physical power over the represented
subjects and objects. As Sekula points out in his important account of the instrumental uses of
photography at the turn of the century, criminal identification photographs, for instance, "are
reconnaissance photographs make possible identification and destruction of the objects they
capture.132
A different way to solve the contradiction, noted by Crary, between the opposition
abstraction/high art versus realism/popular visual culture is to point out that in fact the
development of abstract visual forms was not limited to modernist art but equally took over the
realm of the popular -- first of all, in the adaptation of what I have called here diagrammatic
visual language. But in order to see this it is necessary to include in the discussion more than
images from the realms of art, or entertainment or leisure -- the realms to which the problematic
of high-low still limits itself, and whose borders in Crary's argument are represented by
modernist painting, on the one side, and photographs, stereoscopes, phenakistiscopes and other
popular nineteenth century optical toys, on the other. Rather, it is necessary to consider the whole
range of images employed in modern culture -- images employed not only for leisure but also for
work; images through which, literally speaking, modern work gets done: aerial surveillance
photographs and criminal identification shots, engineering and architectural drawings and plans,
maps and typography, instrumentation displays, and scientific imagery. And, last but not least,
we should consider the abstract graphic marks of diagrammatic visual language, designed to
facilitate cognitive and communicative workloads of modernity, in the office, at school, in the
control room, in the boardroom, and on the assembly line -- which is today replaced by equally
monotonous rows of computer terminals, outputting not Model T, but bits of information; this
steel and coal of the information -- or shall we better say, cognitive age.
Information age, knowledge economy, post-industrial society. All these terms come to designate
the changing nature of society after World War II: the gradual displacement of manual labor by
visual reasoning acquired new importance. Consequently, the work on the models, techniques,
and technologies of visual reasoning, which was previously pursued by a few isolated individuals
(Galton, Freud, Eisenstein), has now become a matter of systematic research on the industrial
scale in such fields as cognitive psychology, scientific visualization, and virtual reality. I will
What is the relation between the psychological theories of the mind and their contemporary
visual technologies? The recurrent claims that new visual technologies externalize and objectify
reasoning, and that they can be used to augment or control it, are based on the assumption of the
isomorphism of mental representations and operations with external visual effects such as
dissolves, composite images, and edited sequences. This assumption, which I so far have not
representations and operations really are, the mind surely does not contain pictures, photographs
or film clips which some mental homunculus looks at. The external images presented to the mind
are not magically transplanted inside it as ready-made ideas and arguments. Regardless of what
visual forms can be presented before the eye -- diagrams, photographs, film images -- as any
other visual input, they are subjected to the complicated processing by the nervous system which
about vision, ignited by every new round of visualization technology: photography, film,
computer animation, and virtual reality. Consider the claims which surround the new field of
scientific visualization -- visualization of data sets, their relationships and their dynamic behavior
revolution because they act as the direct "extension of preconscious visual processes."133 They
assume that the images on a computer screen do not simply function as an aid for reasoning but
that they are equivalent to the mental representations the mind may construct while thinking --
Or consider the technology, which, even more so than scientific visualization, is seen as
capable of completely objectifying, better yet, transparently merging with mental processes --
virtual reality (VR). Again, the descriptions of its capabilities do not distinguish between internal
mental functions, events and processes, and externally presented images. This is how, according
to Jaron Lanier, VR can take over human memory: "You can play back your memory through
time and classify your memories in various ways. You'd be able to run back through the
experiential places you've been in order to be able to find people, tools."134 Lanier also claims
language or any other symbols. Indeed, why should there be any need for linguistic symbols, if
everybody, rather than being locked into a "prison-house of language" (Jameson), will happily
live in the ultimate nightmare of democracy -- the single mental space which is shared by
everybody, and where every communicative act is always ideal (Habermas). This is Lanier's
example of how post-symbolic communication will function: "you can make a cup that someone
else can pick when there wasn't a cup before, without having to use a picture of the word
"cup."135 Here, as with the earlier technology of film, the fantasy of objectifying and
augmenting consciousness, extending the powers of reason, goes hand in hand with the desire to
133 Friedhoff and Benson, The Second Computer Revolution: Visualization, 13.
134 Druckrey, "Revenge of the Nerds," 9.
135 Ibid., 6.
Locked in virtual reality caves, with language taken away, we will communicate through
What can one make of this apparently unsound, yet irresistible, assumption of
isomorphism between the mental process of reasoning and external, technologically generated
visual forms, haunting us at least since the end of the nineteenth century? The conflation of
outside and inside is, of course, symptomatic of the desire to project the inside onto the outside,
to make it objective and public. But is this all? To really understand the persistance of this
assumption, we should turn to the history of ideas about the nature of mental processes.
It is well known that technologies have historically provided and continue to provide the
models according to which people imagine the mind. In the seventeenth century it was the clock,
in the nineteenth -- the motor, in the second half of the twentieth -- digital computers. More
precisely, the paradigms were provided not by the technologies themselves, but by theories
which made them possible. To take the last paradigm as an example, cognitive psychology, born
in the 1950s and gaining prominence ever since, approaches the mind as an information-
processing system, as software which runs on the hardware of the brain. But what gave cognitive
psychology its epistemological basis was not the new technology itself (computers), but the
information theory accompanying it. It was this paradigm which substituted the discussions of
mind and brain by the notion of "human information processing." And before information
136 At SIGGRAPH 1992, the premier annual conference on Computer Graphics and Interactive
Techniques in the U.S., attended by nearly 30,000 people, about a dozen VR exhibits always had
long lines of visitors. However, the lines to two of these exhibits were much longer than any of
the others. One was called Dome, the other -- Virtual Reality Cave; in both cases, to see the
show, the visitors had to go inside some cave-line structures. It did not matter that one of the
exhibits featured a scientific visualization display, of interest only to specialists. Clearly, the fact
that in order to see the spectacle one had to go inside a dark, cave-like space, different from
normal space, provided enough attraction at the end of this century, just as it did at its beginning,
when millions flocked into the dark caves of movie theaters.
theory, theories of the mind were influenced by thermodynamics (as in Freud) and mechanics
(Hobbes).137
It can be also claimed, however, that human imagination about the mind's operations is
limited by the current visual technologies. Lakoff's assertion that "natural reasoning makes use of
at least some unconscious and automatic image-based processes such as superimposing images,
scanning them, focusing on part of them" (1986: 149) as well as Johnson-Laird's proposal that
logical reasoning is a matter of scanning visual models, would have been impossible before
television and later, computer graphics. These visual technologies made operations on images
such as scanning, focusing, and superimposition seem natural. Even more telling are the models
of cognitive psychologists who, in the last two decades, have systematically scrutinized the role
played by mental images in reasoning. These models, which define mental images in terms of
such characteristics as spatial resolution, speed of access, basic graphic operations (rotation,
translation, copy), seem to be describing first of all computer imaging systems. Psychologists
argue among themselves which imaging systems better resemble mental processes, but they do
not doubt the basic metaphor. As Paul Virilio notes, "now the virtual images of the computer
screen seem to confirm not only the existence of certain forms of representation but, more
137 While this theory is well known and widely accepted, other facts suggest that, at least,
sometimes, the influence runs in the opposite direction -- biological and psychological theories
of body and mind providing paradigms for theories of mechanisms. For instance, it appears that
Norbert Wiener's cybernetics was inspired by the concept of homeostasis developed in biology:
"Physiologist Walter B. Cannon viewed the animal body as a self-regulating machine. Building
on the work done by Claude Bernard in the nineteenth century, Cannon developed the concept of
'homeostasis' -- the process by which the body maintains itself in a state of internal equilibrium.
Cannon's ideas were well known to Norbert Wiener. In fact, Cannon's Wisdom of the Body
(1932) may be read as sort of an introduction to Wiener's Cybernetics (1948)." Charles Eames
and Ray Eames, A Computer Perspective: Background to the Computer Age (Cambridge:
Harvard University Press, 1990), 99. Another example is provided by the turn of Artificial
Intelligence (AI) in the 1980s from trying to simulate the disembodied mind to the simulation of
a collective of primitive organisms, having the functionality of insects. Drawing directly on
research in biology, the researchers in AI hope that intelligence will emerge as a product of the
collective behavior of machines simulating simple biological organisms.
138 Paul Virilio, Lost Dimension (New York: Semiotext(e), 1991), 114.
Similarly, in the earlier period, when Freud, in The Interpretation of Dreams, described
the mechanisms by which dream-thoughts and the logical relations between them are represented
in dreams, he and his fellow psychologists relied on available visual technology for their
understanding of the mind. Not surprisingly, Galton's composites, the earliest form of image
processing before digital computers, provided a particularly attractive model. Freud compared
the process of condensation with one of Francis Galton's procedures which became especially
famous: making family portraits by overlaying a different negative image for each member of the
family and then making a single print.139 Writing in the same decade, the American
psychologist Edward Titchener opened the discussion of the nature of abstract ideas in his
textbook of psychology by noting that "the suggestion has been made that an abstract idea is a
sort of composite photograph, a mental picture which results from the superimposition of many
particular perceptions or ideas, and which therefore shows the common elements distinct and the
individual elements blurred."140 He then proceeds to consider the pros and cons of this view. We
should not wonder why Titchener, Freud and other psychologists take the comparison for granted
not question why their models of the mind are so similar to the computer workstations on which
Thus, not only reason has always been conceived by philosophers in visual terms; as
psychologists begin to furiously take on the questions philosophers only wondered about,
subjecting mental processes to controlled, scientific, laboratory study, their models begin to
reflect, more and more, the external visual forms made possible by whatever visual technology
dominates the period. As before, vision is the mirror reason looks into.
139 Sigmund Freud, Standard Edition of the Complete Psychological Works (London: Hogarth
Press, 1953), 4: 293.
140 Edward Bradford Titchener, A Beginner's Psychology (New York: The Macmillan
Company, 1915), 114.
Given the reliance of psychological theories of the mind on contemporary visual
technologies, is there any "progress" between the turn of the century and today, except that the
isomorphism between the mental and the objective became even more prominent; and
externalization of reasoning has been taken much further, both technologically and theoretically.
On the one hand, the refinement of various medical imagining techniques in the 1980s
made possible an increasingly precise imaging of brain activity, including the visualization of
reasoning -- in a literal sense. It is now possible to ask the experimental subject to concentrate
on solving a problem and to see which parts of the cerebral cortex are active. The question of
whether reasoning in fact depends on the operations normally involved in perception becomes
more and more the question which, according to a number of researchers, can be answered
through experimentation -- it is enough to show that the part of a cortex normally dedicated to
More importantly, in their theories, many cognitive psychologists have accepted as given
the equivalence between internal mental processes and the operations which can be done with
external, objectively existing visual representations and objects. Consider the debates about the
nature and role of mental imagery, which have constituted one of the most active areas of
research in cognitive psychology in the last two decades.142 On the one hand, there are those
(such as Zenon Pylyshyn) who argue that mental imagery simply consists of the use of general
thought processes to simulate physical perceptual events. In this view, if the subjects report the
presence of mental imagery during reasoning and problem solving, this is simply a side effect, a
141 Martha Farah, "Is Visual Imagery Really Visual? Overlooked Evidence from
Neuropsychology," Psychological Review 95, no. 3 (1988): 307-317.
142 For a summary of different positions, see Ronald A. Finke, Principles of Mental Imagery
(Cambridge: The MIT Press, 1989).
by-product of real mental computations which do not involve visual representations. On the other
hand, there are those psychologists and neurophysiologists who, through experiments and direct
imaging of brain activity, want to prove that reasoning takes place through the construction and
manipulation of mental images (Alan Pavio, Roger Shepard, Stephen Kosslyn, Martha Farah).
One of the most well-known experiments in defense of the latter view has been done by
Roger Shepard and Jacqueline Metzler of Harvard University.143 They presented subjects with
703.
pairs of perspective line drawings of three-dimensional forms constructed from small cubes. The
subjects' task was to determine whether or not the forms were identical in shape, despite the
difference in orientation. Shepard and Metzler have found that the reaction time was proportional
to the degree of rotation which is required to bring the two objects into a similar position. These
results were taken as proof that in solving the problem, the subjects mentally rotated
corresponding dynamic characteristics and are governed by the same laws of motion."144 Thus, a
mental process was equated with an operation one would perform with real, objectively existing
objects.
Other experiments in defense of the position that many kinds of reasoning involve
manipulation of mental imagery entail the comparison of abstract qualities. When subjects were
asked to recall two animals and to judge which one was larger, the reaction time decreased
proportionally to the difference in estimated size. In another experiment, one group of subjects
was asked to rate animals in intelligence on a scale from one to ten, while another group had to
compare the intelligence of pairs of animals. Again, the reaction time decreased as the distance
in rated intelligence increased. It was concluded that when the subjects tried to discriminate
between two objects, reaction time was shorter the greater the difference between two objects,
regardless of whether the objects were really presented (for instance, two lines of different
length), or were imagined (size of animals), or whether the qualities to be compared were
We shall leave to psychologists the debates whether these and numerous related
experiments indeed prove that internal mental processes involve the manipulation of pictures
imagining what mental processes are like, contemporary psychologists have assumed, without
any reservations, an equivalence between the internal and the external, between the mental
was first equated with the visual. Once this was accomplished, it was simple and only logical to
equate the visual inside with the visual outside, the objectivity and standardization of images
drawn on a classroom blackboard, on the screen of a movie theater, or, most recently, on the
computer terminal.
In this chapter I am concerned with the answer to the problem of how to represent, communicate,
induce or perform reasoning more efficiently. This problem first became important with the
emergence of mass society which required an efficient means to communicate abstract ideas to
large audiences. It became even more important with the shift to post-industrial society when
logical reasoning and other cognitive skills attained central economic importance.
The answer was to adopt vision as a more efficient medium. It can be perceived in
Freud's theoretical speculations of how ideas and logical arguments can be visualized, in the
graphic system), and in the claims made about new visual technologies (Galton, MŸnsterberg,
Eisenstein).
Vision, traditionally considered to be inferior to reason, is now seen as the proper, indeed,
as the ideal medium for it. And this is not only a question of the efficiency
8. Analog Engine
and effectiveness of communication, although, with visual reasoning, these two demands are
four frames a second. And, with pictorial statistics and diagrammatic visual language, reasoning
becomes more effective: complex conceptual hierarchies are reduced to boxes and arrows, huge
sets of data become marks on paper which can be easily manipulated. But what is equally
important is that visual reasoning fits perfectly with the demand of mass society for
standardization. The subjects have to be standardized, and the means by which they are
standardized need to be standardized as well. Hence the objectification of internal, private mental
processes, including ideas and logical connections, and their equation with external visual forms
which can be easily manipulated, mass produced, and standardized on its own. Whether we are
dealing with the conventions of Venn's diagrams, of signs of diagrammatic sign system, or of
cinematic techniques, the private and individual is translated into the public and becomes
regulated.
generalizations and unsuitable for representing logical forms, now this is all forgotten. Vision is
accepted as being fully capable of representing logical relations (Freud), abstract ideas (Galton),
This equation of vision with logic, accomplished by the beginning of the twentieth
century, is the first stage in vision's rationalization. When industrial society turned into the
society of information, computers became the dominant technology, both economically and
intellectually, and behaviorism (efficiency of the body) was replaced by cognitive psychology
(efficiency of the mind), this rationalization entered into a second stage. Now, with the questions
of how information can be coded, stored, retrieved, and processed more efficiently becoming a
matter of economic survival, cognitive psychologists, together with computer scientists, began to
scrutinize vision from this new perspective. In the process, they came up with very precise
descriptions of what vision is. However, what they describe is not some ahistorical, essential
vision, but vision already conceived in a historically and socially specific way -- as an efficient
computational system.
Why is it that when cognitive psychologists started to wonder about the properties of data
structures involved in human reasoning, vision began to take a central place in their models?
Because every mental process is conceived as a computation, and it is assumed that this
computation has to be efficient. The criterion of efficiency is a very general and pervasive notion
in contemporary cognitive psychology. This criterion is the main factor in deciding which of the
evolution -- it is said that in the process of evolution the most efficient method, or the most
efficient representational code had to be adopted. However, the real justification of this criterion,
in my view, is economic, coming from the close connection between cognitive psychology and
its master discipline, to which cognitive psychology ultimately serves -- computer science, and
behind it, the computer industry. Just as computer scientists would not want to build an
Once psychologists realized that images possess some unique properties, which make
them a very efficient medium for computation, it was only natural that they entered into their
theories. How does Johnson-Laird justify his theory of mental models, according to which
logical reasoning involves the construction and manipulation of spatial models, analogous to
or scanned in any direction, while propositional representations can only be processed in one
Why do some cognitive psychologists argue that human long-term memory stores not
only knowledge in the form of propositions, but also as images? Because if we store appearances
of objects then later we can compute those properties that could not be anticipated when we saw
the objects initially.146 In contrast to propositional (linguistic) representation, images turn out to
be capital which can be used over and over in the mind's economy.
Finally, why all the debates in cognitive psychology about the role of mental imagery in
reasoning in the last two decades? Because many problems can be solved more efficiently by
analog rather than digital computer, and vision has unique properties that enable it to function as
such a computer:
If images are representations in a medium with certain fixed properties, and can be subjected to
transformations such as rotation and scaling, then imagery could be used as an analog computer, to
solve problems whose entities and relations are isomorphic to objects and spatial relations. That is,
certain abstract problems could be best solved by translating their entities into imagined objects,
transforming them using available image transformations, detecting the resulting spatial relations
and properties, and translating those relations and properties back to the problem domain.147
What are these "certain properties" which make vision into an analog engine? On the one hand,
images can represent continuously varying information. On the other hand, images can represent
topological relations such as inclusion, "next to," and others. Taken together, these two
properties make vision a very powerful computer, better suited for solving many problems than
language. As Steven Pinker explains, in a visual medium, "it is impossible to represent the fact
that one object is next to another without also committing oneself to which is to the left. (This is
not true for abstract propositional representations, where the two-place predicate 'next to' can be
representations but this simply would not be as efficient. So efficiency becomes the final reason
why vision, in spite of the arguments of Descartes, Locke or Leibniz, is today equated with
by it -- it became an analog code, a "data structure" more proficient for reasoning than language.
What remained the same, although recast in new terms, is the idea that vision came
before verbal language. If for Freud vision was the original, primordial language of humankind,
today cognitive psychologists speculate that the continuous, analog representational system of
mental images emerged first, followed by the system for propositional representation (language).
long awaited publication in English of one of the most influential essays of modern art history --
Chapter
1. 3. Mapping Space
Visual Nominalism
Erwin Panofsky's Die Perspektive als 'symbolische Form'.149 The interest generated around the
re-emergence of this legendary essay, written in 1924-1925,150 demonstrates that the problem of
1991) .
perspective is still felt to be relevant to contemporary culture. Joseph Leo Koerner starts his
review of Perspective as Symbolic Form with the following claim: "Perspective is again the
rage."151 The second event was the Gulf War, the outcome of which was largely predetermined
that modern warfare has become a matter of the "logistics of perception."152 True, broadcasts
have included more traditional views of soldiers, planes, and tanks as seen from the outside, by a
reporter's video camera. But what we also saw were not just images of the war, but endless
images through which the war was carried out: video images from an infrared camera mounted
on a plane; video images from a camera installed on a weapon guided by a laser sensor; video in
its role as "battle damage assessment" where a weapon equipped with an imaging device follows
a weapon of destruction and records details of the damage. This was no longer a traditional
reporter's view of a battle. We saw what the soldiers themselves saw: the images that were their
only information about their targets. More often, in a strange case of identification, we witnessed
Television reports became catalogs of various vision technologies used to plan a battle, to
fight it, and to assess the results. They included satellite imagery of Iraqi territory; diagrams of
Iraqi radar activity over the course of a week; 3-D visualizations of weapons; a 3-D animation
computed from satellite data of a mountain region of Iraqi thought to contain a secret sight of
as U.S. pilots and soldiers training in flight and battle simulators. We also saw military advisors
scrutinizing and interpreting aerial photographs and satellite imagery. We were also shown Iraq's
weak, antiquated resistance against the superior Western technologies of vision, image making,
and image analysis. Iraq fought back with camouflage weapons, decoys, and finally, according to
Western journalists, with oil fires set to confuse U.S. surveillance satellites and planes.
The Gulf War was the combat of surveillance against camouflage, visibility against
invisibility, human eye against computer eye. This warfare was indeed based on the "logistics of
perception," but we can describe its visual techniques even more precisely. Vision was employed
in a limited way as an instrument to capture and represent information about shapes of objects
The productive employment of this function of vision can be seen most dramatically in
the example of radar. A radar screen shows nothing but distances between the radar and other
objects in its field of vision. The colors and textures are not important. The guiding laser sensor
of a "smart" missile tracks the same information -- the distance between the missile and the
target. Similarly, it is the layout of a scene, distances between points, dimensions of objects
The multitude of vision technologies which only "see" distances and dimensions are
paralleled by the reduction of human vision to the same single function. Thus, the images from
traditional or infrared video cameras, which contain information about various features of the
visual world, are used by a soldier or a pilot in a much more limited way -- as a source of
information about distances and shapes in a scene. What is the shape of an object on the horizon
Finally, the technologies (such as radar) and techniques (using human sight in a function
of a radar) of obtaining information about shapes and distances in the real world are
complemented by 3-D computer graphics that simulate the real world on the basis of such
information. In their turn, simulated images generated with 3-D computer graphics are used in
flight and battle simulators, where the military eye undergoes its training to learn how to reduce
The use of technologies which capture and visualize information about shapes and
distances today extends beyond warfare into all spheres of industry and science. The pilots of
commercial airlines utilize the same radar screens and are trained in the same flight simulators as
Air Force pilots.153 Engineers and architects use 3-D graphics to visualize their designs. Oil-rich
153 As I write this in early 1993, many companies which yesterday supplied very expensive
simulators to the military are busy converting them into entertainment arcade-based systems. In
fact, one of the first such systems already commercially operating in a number of major cities,
including Chicago and Tokyo -- Battletech Center from Virtual World Entertainment, Inc. -- is
directly modeled on SIMNET (Simulation Network) developed by DARPA (Defense Advanced
Research Projects Agency). SIMNET can be thought of as the first model of cyberspace, the very
first collaborative VR environment. SIMNET consists of a number of individual simulators,
networked together, each containing a copy of the world database and the virtual representation
of all other participants in the conflict such as the Kuwaiti theater of operations. Similarly, a
Battletech Center comprises a networked collection of futuristic cockpit models with VR gear.
For $7 each, seven players can fight each other in a simulated environment. In another example,
in 1992 Lucas Arts has teamed up with Hughes Aircraft, combining the expertise in computer
games of the former with the expertise in building actual flight simulators of the latter, in a joint
venture aimed at theme-park type rides. On the connection between SIMNET and Battletech
oil fields. And, in automated factories, tireless robotic arms move parts under the control of
In this chapter I will discuss twentieth century automation of the use of vision to record and
visualize geometric and topographic information -- to chart the distance and shape of real and
imaginary scenes and objects. This function of vision complements another function which was
the subject of the previous chapter: the visual representation of abstract ideas and their logical
relations. Here we will deal with the opposite problem -- how to represent a single, concrete
object or concrete scene. How, for instance, to represent not an idea of a chair but a particular
chair with a unique shape and dimensions. These two kinds of referents -- abstract ideas and
arguments, on the one hand, and concrete three-dimensional shapes, on the other hand, can be
related to two long-standing philosophical positions on what constitutes reality. For Plato,
sensible particulars were but pale reflections of Ideas or Forms. Aristotle criticized Plato,
declaring the primary substances were not Ideas but the individual things such as particular men
or animals. These opposing views have continued to be debated in scholastic philosophy, with
context of location-based entertainment -- arcades and theme parks -- see Richard Cook,
"Serious Entertainment," Computer Graphics World (May 1992): 40-48.
154 Allan Sekula relies on this distinction between the philosophical positions of realism and
nominalism in his discussion of different systems of police photography in the end of the
In scholastic philosophy the terms "realism" and "nominalism" refer to two opposing
ontological positions: realism claiming the primacy of Ideas and nominalism -- the primacy of
"nominalism" are somewhat different, the original meanings are helpful in distinguishing two
I have suggested in the previous chapter that the desire to visualize abstract ideas and
logical relations was new to modernity. But it would not be reasonable to propose that the
function of vision which can be called visual nominalism -- to capture the identity of individual
objects and spaces by recording distances and shapes -- originates in the twentieth century as
well. After all, maps were used for many centuries; and the sophisticated graphic language of
engineering drawings was well developed by the nineteenth century. Yet, as this chapter will try
to demonstrate, in this century the automation of visual nominalism, which began with the
Renaissance perspective, entered a new stage. The sign of this automation is the multitude of
new technologies used to capture and visualize three-dimensional reality that have emerged since
magnetic resonance imagining, 3-D computer graphics, and computer holography. Finally, since
the early 1960s work has been under way to automate vision completely, to create computer
vision systems that will recognize objects and interpret scenes automatically.
The development of these technologies has been accompanied by massive research into
the general problems of visual nominalism in computer science, experimental psychology, and
source of depth information and, vice versa, to transform this information into realistic images.
The work on the automation of visual nominalism has also lead to new attention to particular
aspects of human perception. A new paradigm for the study of human vision emerged during the
1970s at MIT associated with the name of David Marr.156 Within this paradigm, the goal of
algorithms by which the brain "computes" shapes of objects from retinal input in the hope that
This chapter brings together seemingly unrelated visual technologies and techniques:
radar, 3-D computer graphics, and computer vision. I will suggest that they are ultimately related
by the function they all serve -- visual nominalism. Until this century, this function has been
geometry and, later, photography. Rather than thinking of them as artistic techniques, such
writers as William Ivins and Bruno Latour emphasize their significance for the development of
science, modern technology, and the economy. What these techniques allow best is to represent
quantitatively and efficiently the knowledge of the surrounding space. New twentieth century
techniques further extend the usefulness of these kinds of representations. Radar allows to obtain
156 David Marr, Vision (New York: W.H. Freeman and Company, 1982).
spatial information without the limitations of visibility (section 3); 3-D computer graphics add
the dimension of interactivity (section 4); and computer vision automates visual nominalism
While there is no history of the burgeoning techniques of visual nominalism in the twentieth
century, the earlier history of these techniques has been studied in detail by art historians. The
problem of representing three-dimensional space upon a two dimensional plane has been a
subject of art historical research from at least the beginning of the century. It is quite remarkable
that the people who are today recognized as the founders of modern art history (Alois Riegl,
Heinrich Wšlfflin, and Erwin Panofsky), all defined the field as the history of the representation
of space. Working within the paradigms of cyclic cultural development and racial topology, they
related the representation of space in art to the spirit of entire epochs, civilizations and races. In
development as the oscillation between two extreme poles, two ways to understand space, which
he called "haptic" and "optic." Haptic perception isolates the object in the field as a discrete
entity, while optic perception unifies objects in a spatial continuum.157 Riegl's contemporary,
Heinrich Wšlfflin, similarly proposed that the temperament of a period or a nation expresses
itself in a particular mode of seeing and representing space. Wšlfflin's Principles of Art History
(1913) plots the difference between Renaissance and Baroque on five dimensions: linear --
painterly, plane -- recession, closed form -- open form, multiplicity -- unity, and clearness --
unclearness. Finally, another founder of modern art history, Erwin Panofsky, contrasted the
157 Michael Ann Holly, Panofsky and the Foundations of Art History (Ithaca: Cornell University
Press,
2. "The1984), 73.
most important event of the Renaissance."
"aggregate" space of the Greeks with the "systematic" space of the Italian Renaissance in the
the history of spatial representation and the evolution of abstract thought. The former moves
from the space of individual objects in antiquity to the representation of space as continuous and
space. Correspondingly, the evolution of abstract thought progresses from ancient philosophy's
"systematic."
Panofsky's 1924-25 essay has been recognized as the first in what became an ever-
every known characteristic of the modern period: economic, social, and philosophical. Some of
these interpretations acquired the character of dogmas, self-evident truths. For instance, the
perspective in visual art has been presented as one of the central metaphors in interpreting
Most interpreters have singled out an aspect of perspective, which then become for them
the core of its symbolic function. For instance, Pierre Francastel, and following him, John
Berger, and many film theorists of the 1960s and 1970s, focused on the mechanical character of
perspective and its capacity to organize the representation of reality around a single point of
view. They theorized a connection between Renaissance perspective and the development of
which is concerned with solidity and extension, with numbers and equivalents -- the
space were replaced by the pictures of cultivated, appropriated landscapes or of rooms cluttered
by precisely drawn objects signifying wealth. Perspective presented the world as ready to be
mastered, consumed, colonized -- the world originating in the eye of the spectator.161
interpretations. For instance, for Rudolf Arnheim, a pyramidal world portrayed by perspective
signifies a hierarchical conception of human existence.162 On the other hand, for Fernande
Saint-Martin, perspectival representation which can only show objects at a distance, corresponds
to the "conceptual rather than sensorial, 'ideological' rather than concretely experienced" relation
Panofsky's own account of perspective is more subtle than many of the consequent
philosophical system, but he is careful not to read one as an expression of the other. His
conclusion is open: perspective can be equally interpreted as subjective and objective, since it
presents space as an objective phenomenon but inevitably perceived from a subjective point of
Perspective subjects the artistic phenomenon to stable and even mathematically exact rules, but on
the other hand, makes the phenomenon contingent upon human beings, indeed upon the
individual: for these rules refer to the psychological and physical conditions of the visual
impression, and the way they take effect is determined by the freely chosen position of a
subjective "point of view."164
162 Rudolf Arnheim, Art and Visual Perception. The New Version (Berkeley: University of
California Press, 1974), 295.
163 Fernande Saint-Martin, Semiotics of Visual Language (Bloomington and Indianapolis:
1990), 141.
164 And while modern perspective, as a coherent unity, is contrasted by Panofsky to the modes
of representation in antiquity and the Middle Ages, he also suggests that within the perspective
mode there exist many distinct variations. Working within the same mode of perspective,
different cultures and epochs take its meaning to be different: the Dutch represent "near space,"
Germans favor "oblique space," Italians explore "high space."
While Panofsky's essay had to wait almost six decades before its publication in English,
Perspective) still remains untranslated and unknown to Western art historians.165 In his article
Joseph Leo Koerner briefly refers to the essay and regrets its unavailability in English (this is the
only reference I am aware of).166 The essay was written by the Russian philosopher, linguist,
mathematician and art historian Father Pavel Florensky, who headed the department of the
essay can be justly considered as the counterpart to Panofsky's work, completed in 1922, two
years before Panofsky's work. Both Panofsky and Florensky interpret any perspectival system as
convention and oppose the view that any particular system is natural.168 At the same time, their
arguments tend to favor antithetical perspectival systems and world views: linear perspective and
Renaissance individualism in the case of Panofsky; inverted perspective and medieval religious
progression towards the "systematic" yet subjective space of the Renaissance; thus, the
Renaissance perspective is given a privileged status, its final triumph as inevitable as the
progress of history itself. The discussion of medieval representation of space becomes for
presented as a linear course from Antiquity to Renaissance: "When work on certain artistic
problems has advanced so far that further work in the same direction, proceeding from the same
premises, appears to bear fruit, the result is often a great recoil, or perhaps better, a reversal of
Panofsky claims, while it may appear that medieval representation gave up the advance of
bodies and space to surface, in the same way and with the same decisiveness, by these very
means it also managed for the first time to confirm and establish the homogeneity of bodies and
space." 170
Florensky, on the other hand, is mostly interested in the medieval representation of space
as manifested by the spatial constructions of Russian icons between the fourteenth and sixteenth
neither a result of the lack of artistic skill nor a step towards the Renaissance perspective.171
corresponds to a coherent world view. It is not hard to see that for the orthodox and slavophile
artists, like the artists of ancient Egypt and China, were aware of perspective but consciously
refused its power, choosing instead "religious objectivity and super-personal metaphysics." For
paid little attention to anything in the way of systematic spatial relationship between objects."
stability of the world view begins to disintegrate, and the sacred metaphysics of collective
individual point of view, and only in a particular moment of time -- only then perspectivalism,
Florensky's Slavophilism strikes us today as being politically biased; yet, this bias makes
us realize that Panofsky's seemingly more objective account is also biased in its privileging of
what Florensky labeled "bourgeois individualism." While Florensky offered a theoretical critique
of linear perspective, for his compatriots at the time this critique became a practical problem:
what new visual forms shall be suitable for the "first people's state"? Similarly identifying linear
perspective with "bourgeois individualism," El Lissitsky and Kazimir Malevich relied in their
designs on other types of perspectival construction such as parallel projection. In this type of
projection, all projection rays are parallel to each other, instead of originating from a single point
of view. Parallel projection, with its absence of a single point of view, was presented as a more
suitable way to symbolize the vision and knowledge of a collective. Paradoxically, this
"objective" point of view of a collective was expressed in the image (commonly encountered in
172 Florensky, "Obratnaya Perspektiva," 125-126. Justifying the value of reverse perspective as a
symbol of "super-personal metaphysics," Florensky offered a critique of the arguments later to
be found in Panofsky's "Die Perspektive als 'symbolische Form.'" For instance, he precisely
articulates the notion of "systematic" space, which, according to Panofsky, represented the
triumph of the Renaissance and was still absent in Medieval thought: "If we summarize
everything which is said in a formal sense against Medieval art, they are reduced to one
criticism: 'There is no understanding of space.' This criticism means that there is no spatial unity,
no scheme of Euclid's and Kant's space, the latter reduced, within the framework of painting, to
linear perspective and proportionality."
Malevich's writings) of the individual artist looking at the Earth from an infinitely far away point
173
one episode in what by now became a widely accepted narrative -- the shattering and negation of
perspectival space by modernist artists. According to this narrative, perspective was already
dead by the time art historians such as Panofsky had begun writing its history. Such narrative is
announced, for instance, in the very title of Pierre Francastel's Painting and Society. Birth and
Destruction of Plastic Space from the Renaissance to Cubism (1952). The opening section of
The fact is that around 1910 a certain space was shattered. It was a space of common sense, of
knowledge (savoir), of social practice, or political power...a space, too, of classical perspective and
geometry, developed from the Renaissance onwards on the basis of the Greek tradition (Euclid,
logic) and bodied forth in Western art and philosophy, as in the form of the city and town.174
Yet, if perspective disappeared from modern art, it survived as one of techniques of the
visual nominalism, a method for precisely representing the three-dimensional world on a two-
dimensional surface. In this role, it extended into many new domains and became the foundation
another interpreter of perspective -- William Ivins and his short 1939 essay On the
Rationalization of Sight.175 If Panofsky connects the development of perspective with the idea
of infinite space, abstract space existing prior to objects, Ivins on the contrary emphasizes that
perspective allows the creation of precise maps of three-dimensional reality, to record the shapes
of concrete objects and the layout of concrete spaces. It is the tool of a businessman and a
In Ivins' definition, perspective is "a practical means for securing a rigorous two-way, or
reciprocal, metrical relationship between the shapes of objects as definitely located in space and
174 Henri Lefebvre, The Production of Space (Oxford: Blackwell Publishers, 1991), 25.
175 William M. Ivins, On the Rationalization of Sight (New York: Da Capo Press, Inc., 1975).
their representations."176 This definition sums up a number of unique characteristics of
perspectival projection. First, the size of graphic marks in a perspectival image is proportionally
related to the dimensions of represented objects. For instance, a foot-long object would be
represented by a line of three inches, while an object twice as long would become a line of six
inches, and so on. Second, the ratio of sizes between these lines is proportional to the ratio of
sizes between the corresponding objects. Third, the perspectival image does not only record the
shapes and dimensions of objects but also contains information about their distance from one
another (in contrast to parallel projection, for instance). Finally, all these relationships between
the perspectival symbols on the one hand, and between the symbols and the corresponding
objects on the other hand, are controlled by a single rule or, to use the language which would
become common within a decade after the publication of On the Rationalization of Sight, by a
single algorithm. Given the coordinates of the point of view and the projection surface, the
steps.
The most important quality of perspective foregrounded in Ivins' definition is the precise
and reciprocal relationship it establishes between objects and their symbols. We can go from
objects to symbols (two-dimensional representations); but we can also go from such symbols to
three-dimensional objects. Using the rules of perspective, an architect could create a visual
representation of an already existing building, change this representation, and the workers would
execute the specified changes in the building itself. Better yet, our architect can design a brand
new building -- not by painstakingly constructing a small model, where each change takes time
to implement -- but by creating a perspectival drawing, where each form is just a line and change
requires only a single movement of an eraser (or a single press of a key on a computer keyboard,
176 Ibid., 9.
Bruno Latour extended Ivins' idea by pointing out that a reciprocal relationship made
possible by perspective allows us not only to represent reality but also to control it.177 Latour
sees perspectival representations as the "most powerful instrument of power," defined as the
ability to mobilize resources across space and time, to manipulate these resources at a distance.
For instance, we cannot measure the sun in space directly, but we only need a small ruler to
around the sun, we would still be better off studying the sun through its representations which we
can bring back from the trip -- because now we have unlimited time to measure, analyze, and
catalog them. We can also represent absent things and plan our movement through space by
working on representations: "One cannot smell or hear or touch Sakhalin Island, but you can
look at the map and determine at which bearing you will see the land when you send the next
fleet."179 We can move objects from one place to another by simply moving their
representations: "You can see a church in Rome, and carry it with you in London in such a way
as to reconstruct it in London, or you can go back to Rome and amend the picture." Finally, as
Latour points out, "the two ways become a four-lane freeway! Impossible palaces can be drawn
realistically, but it is also possible to draw possible objects as if they were utopian ones." Real
and imagined objects can meet on a flat space of perspectival representation. All in all,
perspective is more than just a sign system, reflecting reality -- it makes possible the
Latour proposes that the history of science and technology should be understood as a cascade of
such innovations in representation. Yet, his argument is already anticipated by Ivins. Ivins
concludes his essay by stating that the beginning of the rationalization of sight through the
discovery and the development of perspective "was the most important event of the
Renaissance."180 It was more important than the fall of Constantinople or the Reformation or the
Counter Reformation because it made all other discoveries possible by launching a constantly
science, for instance biology, which could now represent forms of nature with mathematical
precision. It also stimulated the rise of modern engineering and manufacturing by making
Ivins' approach stands in sharp contrast to the more traditional art historical analyses of
perspective by Panofsky and Francastel. They are concerned with perspective as an artistic form
and do not look beyond its history in art. Ivins, on the contrary, is concerned with visual culture -
moment and the fundamental role they play in shaping every aspect of society. Because he sees
these techniques and technologies183 as ultimately determining social development, his approach
is deeply materialistic. Perspective, for Ivins, is not an artistic form reflecting social or
philosophical modernity; it is the very condition which makes modernity and modernization
possible.184 For instance, he writes: "Many reasons are assigned for the mechanization of life
absolutely prerequisite to it."185 Ivins is also a materialist in the most extreme Marxist sense
because he suggests that the available forms of representation, such as perspective, determine the
perspective was for picture making, "it is doubtless even more important to general thought,
because the premises on which it is based are implicit in every statement made with its aid."186
The constant extensions of the fields of usefulness of the pictorial symbol that is precisely
duplicable [through printing] and of the grammars of its use [development of perspective and
descriptive geometry] have had a most astonishing effect not only upon knowledge but upon
thought and its basic assumptions or intuitions...Relativity, which now in one form or another runs
throughout contemporary thought and practice, is in large measure a development of ideas that
were evolved through the study and use of projective transformations.187
It is Ivins' general focus on visual culture, rather than art, that makes him attentive to the
current use of perspective and its continuing development and expansion. While Panofsky or
Francastel write about perspective in the past tense aware of its demise in modern art, Ivins on
the contrary notes "that today there are few sciences and technologies that are not predicated in
one way or another upon this power of invariant pictorial symbolization."188 Modern designers,
scientists or engineers, of course, do not simply use perspective as it was formulated by Alberti
in the fifteenth century; they use much more sophisticated techniques. According to Ivins, the
rationalization of perspectival sight proceeded in two directions. On the one hand, perspective
became the foundation for the development of the techniques of descriptive and perspective
geometry which became the standard visual language of modern engineers and architects (fig.
11). On the other hand, the photographic technologies automated the creation of perspectival
more or less simultaneously. Indeed, as Ivins points out, Ni pce and Talbot, the founders of
photography, were contemporaries of Monge and Poncelet, decisive figures in the development
Writing On Rationalization of Sight between 1936 and 1938, Ivins mentions such examples of
field of archeology, and criminal detection.189 However, all these applications of perspectival
techniques already existed in the nineteenth century and, by the 1930s, did not represent the
"latest" developments.
While photo reconnaissance was first employed systematically on a mass scale during
World War I, the interest in using photography for aerial surveillance existed since its invention.
FŽlix Tournachon Nadar, one of the most eminent photographers of the nineteenth century,
famous for his portraits, succeeded in exposing a photographic plate at 262 feet over Bi vre,
France in 1858. He was soon approached by the French Army to attempt photo reconnaissance
but rejected the offer. In 1882, unmanned photo balloons were already in the air; a little later,
they were joined by photo rockets both in France and in Germany. The only innovation of World
War I was to combine aerial cameras with a superior flying platform: the airplane.190
published a proposal to use photographs for scale measurement. His proposal was based on the
existence of a geometrical relationship between the photographic image and the object being
photographed. Why, for instance, climb a facade of a cathedral in order to measure it (as
Meydenbauer had to do, nearly once getting killed) when it is much safer to measure a
photograph? Additionally, wrote Meydenbauer, "some may find it hard to believe, but
1991) .
experience has proven than one can see, not everything, but many things, better in scale
measurement than on the spot." In 1885 the Royal Prussian Institute for Scale Measurement was
practice.191
employed decades before the time of Ivins' writing. In his 1844 Pencil of Nature William Fox
shelves of china: "should a thief afterwards purloin the treasures -- if the mute testimony of the
picture were to be produced against him in court -- it would certainly be evidence of a novel
kind."192 In 1883, Alphonse Bertillon, director of the Identification Bureau of the Paris
Prefecture of Police, developed a system for classification and identification of criminals that
relied on photographs. By 1893, Bertillon's system was already employed in the United States,
Belgium, Switzerland, Russia, much of South America, Tunisia, the British West Indies, and
Romania.193
Modern perspective and descriptive geometry were fully developed in the first half of the
measurement, identification, and classification by the end of the century. What was the next step
In fact, while Ivins was writing his essay on perspective, across the Atlantic, in England,
work was already underway to install twenty radar stations on the east and southeast coasts to
provide surveillance of these air approaches. These radar installations turned out to be absolutely
essential in the coming war, allowing for the severely outnumbered Royal Air Force to defeat the
191 Harun Farocki, "Reality Would Have to Begin," Documents 1/2 (1992): 136-146.
192 Qtd. in Sekula, "The Body and the Archive," 6.
193 Sekula, "The Body and the Archive," 25, 35.
Luftwaffe in the Battle of Britain. Radar, the latest technology of visual nominalism, became
The following anecdote about the "discovery" of radar, regardless of its historical
accuracy, is useful in understanding how radar works.195 In 1922, two civilian scientists
employed by the Navy set out to conduct a routine communication experiment along the arm of
the Potomac river. As usual, they set a short-wave radio receiver and transmitter along the two
banks of the river. This time, however, a passing steamer interrupted the signal, creating an
electronic echo. The scientists immediately realized that such echoes could be used to deduce the
This anecdote illustrates the main principle of radar operation. Radar is an acronym for
Radio Detection and Ranging. Like sound waves, radio waves create echoes when they are
interrupted by objects in their path. Radar transmits a radio wave in a particular direction. The
signal reflected back from the objects is picked up by an antenna. The time between the
transmission and the reception of the echo indicates the distance to the object; the direction the
antenna is pointing when the echo is received reveals the object's position in relation to the radar.
Radar exemplifies the further rationalization of visual nominalism. All it sees and all it
shows are the positions of objects, 3-D coordinates of points in space, points which correspond
to submarines, aircraft, birds or missiles. Color, texture, even shape are disregarded. Instead of
Alberti's window, opening onto the full richness of the visible world, a radar operator sees a
194 For the history and technology of radar, I mainly relied on two sources: Echoes of War
(Boston: WGBH Boston, n.d.), videotape; McGraw-Hill Encyclopedia of Science & Technology:
an International Reference Work in Twenty Volumes Including Index (New York: McGraw-Hill,
1992.)
195 Echoes of War (Boston: WGBH Boston, n.d.), videotape.
196 Numerous variations of basic radar technology exist. For instance, in addition to active
radars which send a signal and detect energy reflected by objects there are also passive radars
which do not send a signal themselves. However, all radars have in common the use of
electromagnetic radiation (radio waves) to detect and measure objects in their vicinity.
screen, a dark field with a few bright spots. Here, the function of visual nominalism, which
perspectival image performed along with many other functions (representation of light, color,
Radar serves a single function -- but it performs it more efficiently than any previous
longer limited by conditions of visibility. Vision is no longer confined by the distance resolvable
by the human or camera eye, since radar can detect objects hundreds of miles away, and at any
time of day or night. Second, this recording takes place instantly. Previously, military
commanders had to wait until pilots come back from surveillance missions and the film was
developed. The inevitable delay between the time of the observation and the delivery of the
finished image limited its usefulness: since by the time the image was produced, enemy positions
With radar, its vision not limited by visibility, its image literally tracking the referent,
rationalization of visual nominalism reaches a new stage. The usefulness of vision and of
and imaging technologies. While radar came to supplement perspectival representation and
photography in terms of its function, it also represents a fundamental break with older
technologies. Like the photographic camera, radar captures the positions of objects and displays
them on a flat surface. But how it sees has little resemblance to what was commonly meant by
Instead of "looking" in a particular direction, radar sees all around, its antenna scanning
the area in a circular sweep. Instead of passively receiving light reflected by objects, radar sends
energy into the environment. Its vision is active; it can be compared to exploration of the
environment by a blind person.197 Instead of relying, like photography, on the small region of
the electromagnetic spectrum to which our eyes are sensitive, it uses other regions, sending and
receiving waves of different lengths. Vision is no longer limited by the spectral capacity of the
human or camera eye; it is extended to include the whole of electromagnetic spectrum. The
visible becomes a small part of a larger field of sensory exploration of the environment.
Images produced by radar are also fundamentally different than those of previous
imaging technologies, from drawing paper to photographic film. The photographic image is an
imprint corresponding to a single referent or to a limited time of observation. With the invention
of the radar screen, the image surface is no longer static. For the first time it becomes constantly
updatable in real time. Here, with radar, we see for the first time the mass employment of a
fundamentally new kind of display, which soon comes to dominate modern visual culture, found
today in video or computer screen. What is new about such a display is that its image can change
in real time reflecting changes in the referent, be it the position of an object in space (radar), any
change in visible reality (live video) or changing data in computer memory (computer screen).
The radar screen changes, tracking the referent. But while it appears that the element of
time delay always previously present in imaging is eliminated, in fact time enters radar image in
a new way. With older photographic technologies, all parts of an image are exposed
simultaneously. Now the image is produced through sequential scanning, which means that its
contiguous parts in fact correspond to different moments in time. Like in an audio record,
Along with radar, many other technologies of visual nominalism came into existence following
the advances in electronics and computers during World War II: ultrasonic imaging,
197 There also exist passive radars which have no transmitters but are equipped to measure
signals from targets themselves. The active radar can be thought as a historical continuation of
projectors previously used by the military to artificially light up the scene of a battle to increase
visibility.
multispectral photography, multispectral imaging, infrared, sonar, magnetic resonance imaging,
and so on.198 As radar, these technologies are effectively used to record distance, position,
layout, shape, and volume. Sonar, for instance, detects objects in the water by using sound
waves. Ultrasonic computer tomography uses sound waves and computer graphics to construct
images of body tissues. Multispectral photography isolates energy reflected from surfaces in a
visual nominalism are more efficient but also more specialized. If older optical technologies
indiscriminately capture a number of dimensions of visual reality (color, shape, texture, tonality),
now a single dimension can be isolated. More precisely, a single dimension, in this case 3-D
information, is broken into distinct components recorded by different imaging technologies. The
technique of perspective allows one to record both the shape of objects and their relative
positions in relation to each other. Radar only records positions, it is blind to shape. On the other
hand, a laser or ultrasound range finder produces a depth map, treating as shapes both the actual
This specialization can be seen as a side effect of the automation of visual nominalism.
This automation requires a priori assumptions about the nature of reality which become
embedded in different technologies. In the case of radar, reality is reduced to a set of geometric
points which have no extension. In the case of a range finder, reality is treated as if a skin, a
membrane, a continuous 3-D form. We may recall here the far-seeing prediction of the effects of
photography published in 1859 by Oliver Wendell Holmes: "Every conceivable object of Nature
and Art will soon scale off its surface for us. Men will hunt all curious, beautiful, grand objects,
Holmes' words seem to come true not only metaphorically (modern culture which values only
Along with the separation between technologies that record positions and technologies
that record shapes, another specialization emerges. Depending upon whether the position of the
observer in relation to objects in space is important or not, the technologies of visual nominalism
specialize in creating two different types of representations which can be called viewer-oriented
and viewer-independent.200 The first kind of representation is exemplified by the radar screen, a
record of distances between enemy targets and the subject -- the point in the center of the screen.
Here, even more so than in the traditional perspective, the world is represented in relation to the
observer who becomes the center of the coordinate system. In the second type of representation,
the distance between the observer and the object is unimportant and in fact, the position of the
observer cannot even be deduced from the image. These are images which employ types of
projection other than perspective: maps, engineering and architectural plans, and aerial and
A third specialization, a third division of labor brought about by new technologies is that
between vision and imaging, which now become distinct processes. Engineering textbooks and
encyclopedias group many new technologies of visual nominalism under the term "remote
sensing," defined as the gathering and imaging of information without actual physical contact
with the object or area being investigated.201 This definition clearly separates the two
199 Oliver Wendel Holmes, "The Stereoscope and the Stereograph," in Photography, Essays and
Images: Illustrated Readings in the History of Photography, ed. Beaumont Newhall (New York:
Museum of Modern Art, 1980), 60.
200 This distinction is related to but not synonymous with the distinction between "viewer-
centered" and "object-centered" representations, common since David Marr's Vision (New York:
W.H. Freeman and Company, 1982). It is also related, but again, not equivalent to, the difference
between perspectival and orthographic projections. The first records reality from a single point of
view, the second does not have a single point of view.
201 McGraw-Hill Encyclopedia of Science & Technology, 15: 311.
operations: the gathering of information (vision) and its presentation (imaging). The first
operation may have nothing to do with what is visible to the human eye, but in the second
operation the eye eventually comes into play since the gathered information has to be presented
biological senses (for instance, we can think of radar, which sends waves in order to detect
objects, as an augmented sense of touch) into information perceivable by the human eye. They
map other parts of the electromagnetic spectrum into a small visible part. The invisible (sound
waves, microwaves, ultraviolet waves) is converted into the visible. This is why these
technologies are also often called imaging technologies -- multispectral imaging, ultrasound
I have emphasized the continuity of function between perspectival drawing, photography, and
newer technologies such as radar and ultrasound imaging. Yet, not only these newer technologies
of visual nominalism serve the same function as the old -- to capture the identity of individual
objects and spaces by recording distances and shapes -- but they rely on the same principle of
In his seminar "Of the Gaze as Object Petit a" Jacques Lacan argued that this is how
something we do not have to rely on light or to operate in the domain of the visible. Nor do we
have to limit images to 2-D representations of 3-D reality. We can represent an object by another
202 Jacques Lacan, "On the Gaze as Objet Petit a," in The Four Fundamental Concepts of
Psycho-Analysis, ed. Jacques-Alain Miller (New York: W.W. Norton & Company, 1981), 67-
122.
object or represent a 2-D form by another form. All that is required is a rule to establish the
correspondence between the points of the object being imaged and the points on the image.
Similarly, says Lacan, "what is an issue in geometric perspective is simply the mapping
of space, not sight."203 Perspective is one such rule, a particular method to establish a
correspondence between the object and its image. The method of perspective consists of
connecting a single point in space (usually referred to as subject's point of view) with a number
of points on the object by straight lines; the intersection of these lines with a plane creates an
image. It is coincidental that perspective, whether as a part of the human sight apparatus or as a
part of the photographic apparatus, works through light. Light travels in straight lines, therefore
it can be used to create perspectival images. But one can construct such images without light: "In
Descartes, dioptrics, the action of the eyes, is represented as the conjugated action of two
sticks."204 As Lacan points out further on in the seminar, this idea that perspective is not limited
to sight alone but functions in other senses as well defines the classical discourse on perception:
"The whole trick, the key presto!, of the classic dialectic around perception, derives from the fact
that it deals with geometric vision, that is to say, with vision in so far as it is situated in a space
Lacan's clarification that the principle of perspective is not limited to the visible helps us
understand that the technologies of remote sensing function on the principle of perspective.
Regardless of their lengths, all waves travel in straight lines, and therefore points in space are
connected by straight lines to a point of reception (such as radar antenna) or recording (such as a
photographic camera). Radar, infrared imaging, sonar, and ultrasound are all part of what Lacan
of visibility. In the next section I will discuss the technology of 3-D computer graphics, which
From the moment of adaptation of perspective, artists and draftsmen have attempted to aid the
laborious manual process of creating perspectival images.206 Between the sixteenth and the
nineteenth century various "perspectival machines" were constructed. They were used to
to help students learn how to draw in perspective, to impress artists' clients, or to serve as
intellectual toys. Already in the first decades of the sixteenth century, DŸrer described a number
of such machines.207 One device is a net in the form of a rectangular grid, stretched between the
artist and the subject. Another uses a string representing a line of sight (fig. 12). The string is
fixed on one end, while the other end is moved successively to key points on the subject. The
point where the string crosses the projection plane, defined by a wooden frame, is recorded by
two crossed strings. For each position, a hinged board attached to the frame is moved and the
point of intersection is marked on its surface. It is hard to claim that such a device, which gave
rise to many variations, made the creation of perspectival images more efficient, however the
images it helped to produce had reassuring mechanical precision. Other major types of
206 For a survey of perspectival instruments, see Martin Kemp, The Science of Art (New Haven:
Yale University Press, 1990), 167-220.
207
4. Ibid.,
3-D 171-172.
Computer Graphics: Interactive Perspectivalism
Why manually move the string imitating the ray of light from point to point? Along with
perspectival machines a whole range of optical apparatuses was in use, particularly for depicting
landscapes and in conducting topographic surveys. They included versions of camera obscura
from large tents to smaller, easily transportable boxes. After 1800, the artist's ammunition was
strengthened by camera lucida, patented in 1806.208 Camera lucida utilized a prism with two
reflecting surfaces at 135û. The draftsman carefully positioned his eye to see both the image and
the drawing surface below and traced the outline of the image with a pencil.
Optical apparatuses came closer than previous perspectival devices to the automation of
perspectival imaging. However, the images produced by camera obscura or camera lucida were
only ephemeral and considerable effort was still required to fix these images. A draftsman had to
meticulously trace the image to transform it into the permanent form of a drawing.
With photography, this time-consuming process was finally eliminated. The process of
imaging physical reality, the creation of perspectival representations of real objects was now
mechanized. However, this mechanization did not affect other uses of perspectival
representation. According to Latour, the greatest advantage of perspective over other kinds of
representations is that it establishes a "four-lane freeway" between physical reality and its
representation. We can combine real and imagined objects in a single geometric model and go
back and forth between reality and the model. By the twentieth century, the creation of a
geometric model of both existing and imagined reality still remained a time consuming manual
process, requiring the techniques of perspectival and analytical geometry, pencil, ruler, and
eraser. Similarly, if one wanted to visualize the model in perspective, hours of drafting were
required. And to view the model from another angle, one had to start all over again. The
mechanization and automation of geometrical modeling and display were yet to come.
by Henry Ford in U.S. factories in 1913. The assembly line relied on two crucial principles. The
first was the standardization of parts, already employed in the production of military uniforms in
the nineteenth century. The second, newer principle, was the separation of the production process
into a set of repetitive, sequential, and simple activities that could be executed by workers who
did not have to master the entire process and could be easily replaced.
It seemed that mechanical modernity was at its peak. Yet, in the same year the Spanish
inventor Leonardo Torres y Quevedo had already advocated the industrial use of programmed
machines.209 He pointed out that although automatons existed before, they were never used to
The ancient automatons...imitate the appearance and movement of living beings, but this has not
much practical interest, and what is wanted is a class of apparatus which leaves out the mere
visible gestures of man and attempts to accomplish the results which a living person obtains, thus
replacing a man by a machine.210
With mechanization, work is performed by a human but his or her physical labor is augmented
by a machine. Automation takes mechanization one step further: the machine is programmed to
The term "automation" was coined in 1947; and in 1949 Ford began the construction of
the first automated factories. Mass automation was made possible by the development of digital
computers during World War II and thus became synonymous with computerization. A decade
later, the automation of the process of constructing perspectival images of both existent and non-
existent objects and scenes was well underway.211 By the early 1960s Boeing designers already
209 Charles Eames and Ray Eames, A Computer Perspective: Background to the Computer Age
(Cambridge: Harvard University Press, 1990), 65-67.
210 Qtd. in ibid., 67.
211 I am not aiming here by any means to provide a full account of the history of 3-D computer
graphics or its various uses. I am concerned with computer graphics as one development, among
others, in the general move toward the rationalization of perspectival imaging. For a more
relied on 3-D computer graphics for the simulation of landings on the runway and of pilot
began in the Renaissance. This automation became possible because perspectival drawing has
always been a step-by-step procedure, an algorithm involving a series of steps required to project
coordinates of points in 3-D space onto a plane. Before computers the steps of the algorithm
were executed by human draftsmen and artists. With a computer, these steps can be executed
computer were published in 1963 by Lawrence G. Roberts, then a graduate student at MIT.213
perspectival geometry from the early 1800s to get the mathematics of perspective.214 The
algorithm reduces reality to solid objects, and the objects are further reduced to planes defined by
straight lines. The coordinates of the endpoint of each line are stored in a computer. Also stored
are the parameters of a virtual camera -- the coordinates of a point of view, the direction of sight,
and the position of a projection plane. Given this information, the algorithm generates a
comprehensive account of 3-D computer graphics techniques, see J. William Mitchell, The
Reconfigured Eye: Visual Truth in the Post-Photographic Era (Cambridge, The MIT Press,
1992), 117-162.
212 Jasia Reichardt, The Computer in Art (London and New York: Studio Vista and Van
Nostrand Reinhold Company, 1971), 15.
213 L.G. Roberts, Machine Perception of Three-Dimensional Solids, MIT Lincoln Laboratory
TR 315, 1963; L.G. Roberts, Homogeneous Matrix Representations and Manipulation of N-
Dimensional Constructs, MIT Lincoln Laboratory MS 1405, 1965.
214 "Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and
Harvard," in SIGGRAPH '89 Panel Proceedings (New York: The Association for Computing
Machinery, 1989), 72.
The subsequent development of computer graphics can be seen as the struggle to
automate other operations involved in producing perspectival stills and moving images. The
perspectival image of a geometric model as seen from an arbitrary point of view -- a picture of a
virtual world recorded by a virtual camera. But, just like with the early perspectival machines
described by DŸrer, early computer graphics systems did not really save much time over
computer technology with manual labor. As in traditional animation, twenty-four plots were
required for each second of film. These plots were computer-generated and consisted of simple
lines. Each plot was then hand-colored by an artist. Finished plots were filmed, again manually,
on an animation stand.215 Gradually, throughout the 1970s and the 1980s, the coloring stage
was automated as well. Many algorithms were developed to add the full set of depth cues to a
synthetic image -- hidden line and hidden surface removal, shading, texture, atmospheric
Today, these algorithms make possible the simulation of almost any object in such a way
that its computer image is indistinguishable from the photograph. Yet, this by itself does not
represent a radical break with older techniques of visual nominalism. A painter can paint the
215 This mixture of automated and pre-industrial labor is characteristic of the early uses of
computers for the production of images. In 1955 the psychologist Attneave was the first to
construct an image which was to become one of the icons of the age of digital visuality --
random squares pattern. A pattern consisted of a grid made from small squares colored black or
white. A computer generated table of random numbers has been used to determine the colors of
the square -- odd number for one color, even number for another. Using this procedure, two
research assistants manually filled in 19,600 squares of the pattern. Paul Vitz and Arnold B.
Glimcher, Modern Art and Modern Science (New York: Praeger Publishers, 1984), 234. Later,
many artists, such as Harold Cohen, used computers to generate line drawings which they then
colored by hand, transferred to canvas to serve as a foundation for painting, etc.
216 For further discussion of the problem of realism in computer graphics, see Lev Manovich,
"'Real' Wars: Esthetics and Professionalism in Computer Animation," Design Issues 6, no. 1
(Fall 1991): 18-25; Lev Manovich, "Assembling Reality: Myths of Computer Graphics,"
Afterimage 20, no. 2 (September 1992): 12-14.
same image, although it would take much longer. What does represent a radical break is
In 1962 Ivan Sutherland designed his now legendary Sketchpad program. With
Sketchpad, a human operator could create graphics directly on the computer screen by touching
the screen with a light pen. In the same year, ITEK began marketing its Electronic Drafting
Machine, an apparatus similar to Sketchpad.217 Although both programs dealt only with 2-D
graphics, they introduced a new paradigm of interactive graphics: by changing something on the
screen, the operator changed the data in the computer's memory. 218
When this paradigm of interactive editing was combined with the algorithms of 3-D
graphics, a radically new way of using perspectival images emerged. This development was
more revolutionary than the automation of perspective construction per se. Indeed, traditional
draftsman could have accomplished what the computer at Boeing was doing -- generating plots
in perspective given 3-D database -- only more slowly. But now it became possible to change the
point of view of a virtual camera and see the corresponding changes in the perspectival image
after a short delay. It also became possible to build and modify 3-D models interactively and
The emergence of interactive 3-D computer graphics started the race to eliminate the time
delay between the action of an operator and the displayed results. To be able to move through
virtual world in real time a computer has to generate 30 frames a second. The amount of
217 "Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and
Harvard," 51.
218 In fact, interactive computer graphics technology appeared earlier, although it was not
publicized. Already in the 1950s the Air Force used interactive CRT displays and light pens in
order to more efficiently process information obtained by radar. Both CRT displays and light
pens were designed at Lincoln Laboratory as part of the SAGE project. Using this technology,
Lincoln researchers created a number of computer graphics programs. They included programs
which made posible the display of brain waves (1957), of simulated planet and gravitational
activity (1960), as well as the creation of 2-D drawings (1958). "Retrospectives II: The Early
Years in Computer Graphics at MIT, Lincoln Lab, and Harvard," 42-54.
calculations involved increases proportionally as the model becomes more complex and as more
In this race for speed, which accelerated in the 1970s as synthetic images began to be
utilized in flight simulators, the algorithms of 3-D graphics were gradually transported from
software into hardware, each algorithm becoming a special computer chip. In 1966 Ivan
Sutherland and his colleagues started research on the first head-mounted display, a prototype of
Virtual Reality -- the ultimate interface for 3-D computer graphics. The research was
cosponsored by ARPA (Advanced Research Projects Agency) and the Office of Naval
The fundamental idea behind the three-dimensional display is to present the user with a
perspective image which changes as he moves. The retinal image of the real objects which we see
is, after all, only two-dimensional. Thus if we place suitable two-dimensional images on the
observer's retinas, we can create the illusion that he is seeing a three-dimensional object...The
image presented by the three-dimensional display must change in exactly the way that the image
of a real object would change for similar motions of the user's head.220
The images seen in 1970 by the first users of Sutherland's system, such as a wireframe
cube or a square room, looked remarkably similar to those utilized by Renaissance artists in their
geometrical objects because they most clearly showed the achievement of perspective -- the
creation of a 2-D image of a 3-D object with foreshortening that matches our visual
experience.221 The difference was that now the perspectival image of a cube was not fixed on a
sheet of paper; it appeared to an observer as floating in the air in front of her, changing its view
219 Howard Rheingold, Virtual Reality (New York: Simon & Schuster, Inc., 1991), 105.
220 Qtd. in ibid., 104.
221 Later, Renaissance artists would deliberately insert checkerboard floors in perfect
perspective in their paintings to impress their clients -- a move repeated in computer graphics,
where such checkerboard background receding towards infinity became a favorite motif.
in correspondence with the slightest movement of the observer's head. Perspectival imaging
became interactive.
In order to achieve this interactivity, special hardware was built. To speed up the
calculations necessary to update the synthetic image 30 times a second, Sutherland's group broke
down the process of 3-D image synthesis into a series of discrete steps. Each step was delegated
to a special electronic circuit: a clipping divider, a matrix multiplier, a vector generator. Later on,
such circuits became specialized computer chips, connected together to achieve real-time, high
resolution, photorealistic 3-D graphics. Silicon Graphics, one of the major manufacturers of
The term appropriately symbolizes the second stage of the automation of perspectival
imaging. At the first stage, the photographic camera, with perspective physically built into its
lens, mechanized the process of creating perspectival images of existing objects. Now, with the
perspectival algorithm and other necessary geometric operations embedded in silicon, it becomes
In the next section, I will discuss the final stage in the automation of perspectivalism --
In his papers, published between 1963 and 1965, Roberts formalized the mathematics necessary
for generating and modifying perspective views of geometric models on the computer. This,
writes William J. Mitchell, was "an event as momentous, in its way, as Brunelleschi's perspective
mind not the automation of perspectival imaging but another, much more daring goal -- "to have
two fields were born simultaneously: 3-D computer graphics and computer vision, automation of
The field of computer vision can be seen as the culmination of at least centuries-long
histories. The first is the history of mechanical devices designed to aid human perception, such
as Renaissance perspectival machines. This history reaches its final stage with computer vision,
Harvard," 57.
which aims to replace human sight altogether. The second is the history of automata, whose
construction was especially popular in the seventeenth and eighteenth centuries. Yet, despite
which imitated human's or animal's bodily functions and the modern day robots equipped with
computer vision systems, artificial legs, arms, etc. As noted by Leonardo Torres, old automata,
while successfully copying the appearance and movement of living beings, had no economic
value. Indeed, such voice synthesis machines as Wolgang von Kempelen's 1778 device which
directly imitated the functioning of the oral cavity or AbbŽ Mical's T tes Parlantes (1783)
operated by a technician hiding offstage and pressing a key on a keyboard were used only for
entertainment.224 When in 1913 Torres called for automata that would "accomplish the results
which a living person obtains, thus replacing a man by a machine" he was expressing a
fundamentally new idea of using automata for productive labor. A few years later, the brother of
the Czech writer Karel Capek coined the word robot from the Czech word robota, which means
"forced labor."225 Capek's play R.U.R. (1921) and Fritz Lang's Metropolis (1927) clearly
century technology simply added the sense of sight to eighteenth century mechanical statues. But
even to see computer vision as the continuation of Torres', Capek's or Lang's ideas about
industrial automation which replaces manual labor would not be fully accurate. The idea of
224 Remko Scha, "Virtual Voices," MediaMatic 7, no. 1 (1992): 33. Scha describes two
fundamental approaches taken by the developers of voice imitating machines: the genetic method
which imitates the physiological processes that generate speech sounds in the human body and
the gennematic method which is based on the analysis and reconstruction of speech sounds
themselves without considering the way in which the human body produces them. While the
field of computer vision, and other fields of artificial intelligence, first clearly followed
gennematic method, in the 1980s, with the growing popularity of neural networks, there was a
shift towards the genetic method -- direct imitation of the physiology of the visual system. In a
number of laboratories, scientists begin to build artificial eyes which move, focus, and analyze
information exactly like human eyes.
225 Eames and Eames, A Computer Perspective, 100.
computer vision became possible and the economic means to realize this idea became available
only when automation entered its second stage after World War II. The attention turned from the
automation of the body to the automation of the mind, from physical to mental labor. This shift
to the automation of mental functions such as vision, hearing, reasoning, problem solving is
exemplified by the very names of the two new fields that emerged during the 1950s and 1960s --
artificial intelligence and cognitive psychology. The latter gradually replacing behaviorism, the
dominant psychology of the "Fordism" era. The emergence of the field of computer vision is a
part of this cognitive revolution, a revolution which was financed by the military escalation of
the Cold War.226 This connection is solidified in the very term "artificial intelligence" which
may refer simultaneously to two meanings of "intelligence": reason, the ability to learn or
In the 1950s, faced with the enormous task of gathering and analyzing written,
photographic, and radar information about the enemy, the CIA and the NSA (National Security
Agency) began to fund the first artificial intelligence projects. One of the earliest projects was a
Program for Mechanical Translation, initiated in the early 1950s in the attempt to automate the
monitoring of Soviet communications and media.227 The work on mechanical translation was
probably the major cause of many subsequent developments in modern linguistics, its move
postulating the existence of language universals in the domain of grammar, implied that
translation between arbitrary human languages could be automated. The same work on
mechanical translation was also one of the catalysts in the development of the field of pattern
typical example is character recognition, the first stage in the process of automating translation.
Pattern recognition was also used in the U.S. for the monitoring of Soviet radio and telephone
the space race began, image processing became another major new field of research. Image
1964, the space program for the first time used image processing to correct distortions in the
National Photographic Interpretation Center (NPIC) was created to produce photoanalysis for the
rest of the U.S. intelligence community and, as Manual De Landa points out, by the end of the
next decade computers "were routinely used to correct for distortions made by satellite's imaging
sensors and by atmospheric effects, sharpen out-of-focus images, bring multicolored single
images out of several pictures taken in different spectral bands, extract particular features while
diminishing or eliminating their backgrounds altogether..." De Landa also notes that computer
analysis of photographic imagery also became the only way to deal with the pure volume of
intelligence being gathered: "It became apparent during the 1970s that there is no hope of
keeping up with the millions of images that poured into NPIC...by simply looking at them the
way they had been looked at in World War II. The computers therefore also had to be taught to
compare new imagery of a given scene with old imagery, ignoring what had not changed and
contrast, remove the effects of blur, extract edges, record differences between two images, and so
on, greatly eased the job of human photoanalysts. And the combining of image processing with
pattern recognition made it possible in some cases to delegate the analysis of photographs to a
computer. For instance, the technique of pattern matching used to recognize printed characters
can also be used to recognize objects in a satellite photograph. In both cases, the image is treated
as consisting of two-dimensional forms. The contours of the forms are extracted from the image
are then compared to templates stored in computer memory. If a contour found in the image
matches a particular template, the computer signals that a corresponding object is present in a
photograph.
A general computer vision program has to be able to recognize not just two-dimensional
but three-dimensional objects in a scene taken from an arbitrary angle.231 Only then it can be
used to recognize an enemy's tank, to guide an automatic missile towards its target or to control a
robotic arm on the factory floor. The problem with using simple template matching is that "a
that makes the representation ambiguous with respect to the object."232 While pattern
recognition was working for images of two-dimensional objects, such as letters or chromosomes,
be the first attempt at solving the general task of automatically recognizing three-dimensional
231 Within the field of computer vision, a scene is defined as a collection of three-dimensional
objects depicted in an input picture. David McArthur, "Computer Vision and Perceptual
Psychology," Psychological Bulletin 92, no. 2 (1982): 284.
232 Paul R. Cohen and Edward A. Feigenbaum, eds., The Handbook of Artificial Intelligence
(Los Atlos, CA: William Kaufmann, Inc., 1982), 3: 139.
objects.233 His program was designed to understand the artificial world composed solely of
polyhedral blocks -- a reduction of reality to geometry that would have pleased CŽzanne. Using
image processing techniques, a photograph of a scene was first converted into a line drawing.
Roberts' program had access to three-dimensional models of objects: a cube, a rectangular solid, a
wedge, and a hexagonal prism. They were represented by coordinates (x, y, z) of their vertices. A
Was this enough to completely automate human vision? This depends upon how we
define vision. The chapter on computer vision in The Handbook of Artificial Intelligence (1982)
opens with the following definition: "Vision is the information-processing task of understanding
a scene from its projected images."235 But what does "understanding a scene" mean? With
understanding becomes highly pragmatic. In the best tradition of the pragmatism of James and
Pierce, cognition is equated with action. The computer can be said to "understand" a scene if it
can act on it -- move objects, assemble details, destroy targets. Thus, in the field of computer
vision "understanding a scene" implies two goals. First, it means the identification of various
from the image. A robot, for instance, need not only recognize particular objects, but it has to
missile not only has to identify a target but also to determine the position of this target in three-
dimensional space.
It can be seen that Roberts' program simultaneously fulfilled both goals. His program
exemplified the approach taken by most computer vision researchers in the following two
decades. A typical program first reconstructs the three-dimensional scene from the input image
and then matches the reconstructed objects to the models stored in memory. If the match is good,
the program can be said to recognize the object, while simultaneously recording its position (fig.
14).
identifies them) by reading their shapes through touch. As for a blind person, understanding the
world and the recognition of shapes are locked together; they cannot be accomplished
independently of one another. With computer vision, visual nominalism, which, after all, is just a
particular use of vision, attains unprecedented importance. Now, to see means to see shapes and
I have presented the new twentieth century technologies of visual nominalism as building upon
perspective, extending its powers in space and time. Technologies of remote sensing, such as
radar, extended perspective beyond the realm of the visible. 3-D computer graphics speeded up
and automated design and perspectival display of the models of both real and imagined objects.
With the emergence of the field of computer vision, perspectival sight reaches its
apotheosis and at the same time begins its retreat. At first computer vision researchers believed
that they could invert the perspective and reconstruct the represented scene from a single
perspectival image. Eventually, they realized that it is often easier to bypass perspectival images
Latour points out that with the invention of perspective it became possible to represent
absent things and plan our movement through space by working on representations. To quote
him again, "one cannot smell or hear or touch Sakhalin island, but you can look at the map and
determine at which bearing you will see the land when you send the next fleet."236 Roberts'
program extended these abilities even further. Now the computer could acquire full knowledge
of the three-dimensional world from a single perspectival image! And because the program
determined the exact position and orientation of objects in a scene, it became possible to see the
reconstructed scene from another viewpoint. It also became possible to predict how the scene
Roberts' program worked using only a single photograph -- but only because it was
presented with a highly artificial scene and because it "knew" what it could expect to see.
Roberts limited the world which his program could recognize to simple polyhedral blocks. The
shapes of possible blocks were stored in a computer. Others simplified the task even further by
However, given an arbitrary scene, composed from arbitrary surfaces of arbitrary color
and lighted in an arbitrary way, it is very difficult to reconstruct the scene correctly from a single
perspectival image. The image is "underdetermined." First, numerous spatial layouts can give
rise to the same two-dimensional image. Second, "the appearance of an object is influenced by
its surface material, the atmospheric conditions, the angle of the light source, the ambient light,
the camera angle and characteristics, and so on," and all of these different factors are collapsed
together in the image.238 Third, perspective, as any other type of projection, does not preserve
many geometric properties of a scene. Parallel lines turn into convergent lines; all angles change;
equal lines appear unequal. All this makes it very difficult for a computer to determine which
Thus, perspective, which until now served as a foundation of new technologies of visual
nominalism, becomes the drawback which needs to be overcome. Perspective, this first step
towards the rationalization of sight (Ivins) has eventually become a limit to its total
The realization of the ambiguities inherent in a perspectival image itself came after years
of vision research. In trying to compensate for these ambiguities, laboratories began to scrutinize
237 Cohen and and Feigenbaum, The Handbook of Artificial Intelligence, 3: 141.
238 Ibid., 128.
the formal structure of a perspectival image with a degree of attention unprecedented in the
history of perspective. In 1968 Adolpho Guzman classified the types of junctions that appear in
line representations after he realized that a junction type can be used to deduce whether regions
of either side of a junction line were part of the same object. 239 In 1986 David Lowe presented
a method to calculate the probability that a particular regularity in an image (for instance, parallel
lines) reflects the physical layout of the scene rather than being an accident due to a particular
viewpoint.240 All other sources of depth information such as shading, shadows or texture
Despite these advances, a single perspectival image remained too ambiguous a source of
information for practical computer vision systems. An alternative has been to use more than one
image at a time. Computer stereo systems employ two cameras which, like human eyes, are
positioned a distance apart. If the common feature can be identified in both images, then the
position of an object can be simply determined through geometric calculations. Other systems
But why struggle with the ambiguity of perspectival images at all? Instead of inferring
depth directly by employing various remote sensing technologies. In addition to video cameras,
modern vision systems also utilize a whole range of different range finders such as lasers or
ultrasound.241 Range finders are devices which can directly produce a three-dimensional map of
an object. The same basic principle employed in radar is used: the time required for an electro-
magnetic wave to reach an object and reflect back is proportional to the distance to the object.
But while radar reduces an object to a single point and in fact is blind to close-by objects, a range
produces a "depth map," a record of an object's shape which can be then matched to geometric
models stored in computer memory thus bypassing the perspectival image altogether (fig. 15).
information, the generation of perspectival views, and the identification of objects -- eventually
converged. Remote sensing, 3-D computer graphics, and computer vision form a closed circle.
As we have seen, algorithms of 3-D computer graphics were first developed by Roberts in order
to solve the general problem of computer vision. From that point on, the two fields developed in
parallel. The field of computer graphics was gradually learning to simulate more and more
"realistic" images of reality, adding shading, shadows and texture to initial wireframe drawings.
Meanwhile, the field of computer vision was learning to deal with shading, shadows, and texture
In effect, the goal of computer vision is to undo what 3-D computer graphics aims to
achieve. The former tries to reconstruct a scene from its photographic image, to deduce viewer
independent information about the objects: their shapes and dimensions. The goal is to "produce
an object centered representation that is independent of the particular details of the viewing angle
or photographic process."242 The aim of 3-D computer graphics is exactly the opposite: given
the objective information about the scene (shapes and positions of objects, their materials,
direction of light) to produce its image which is virtually indistinguishable from a photograph.
At the foundation of this mutual dependence is the shared geometrical model of reality as
a set of surfaces having a particular orientation, position, and curvature. The circle is completed
by the technologies of remote sensing that, by producing depth maps of real scenes, can
242 David Peat, Artificial Intelligence: How Machines Think (New York: Baen Enterprises,
1985), 230.
automatically reduce them to geometric models. Portable shape acquisition cameras are already
being developed that, instead of taking snapshots of a scene, will automatically generate
geometric models which then can be used for computer vision or 3-D computer graphics.243
Technologies of remote sensing can automatically create a depth map of reality. Using
such depth map as input, 3-D computer graphics can be used to generate an image which, in its
turn, can act as input to a computer vision system. Computer vision then can reconstruct
"objective" information from an image, a model which can act as a source for further 2-D
computer graphics synthesis. A series of mirrors, constructions and reconstructions are triggered
by each other, with speed only limited by the processing power of "geometry engines."
In this chapter I discussed the twentieth century automation of the function of vision which I
have called visual nominalism -- capturing the information about shapes and distances and
remote sensing extend the capacities of human vision beyond the limits of visibility, making it
possible to track positions of objects in real time. Techniques of 3-D computer graphics allow the
interactive display and construction of geometric models of both real and non-existent objects.
Finally, computer vision automates the recognition of objects and the reconstruction of three-
In this discussion, the concept of perspective played a special role for a number of
reasons. I relied on the familiarity of this concept to explain the principles of operations of newer
twentieth century technologies of visual nominalism. I have also relied on Ivins' account of
perspective and its extension by Latour to suggest that the Renaissance's adaptation of
243
6. Rheingold, Virtual Reality, 229, 252.
Conclusion
perspective represented the first step in the automation of visual nominalism. While other
cultures used sophisticated methods of space mapping, the importance of perspective lies not in
its representational superiority but in its algorithmic character. This algorithmic character
enabled the gradual development of visual languages of perspective and descriptive geometry
and, in parallel, of perspectival machines and technologies, from a simple net described by
DŸrer to photography and radar. And when digital computers made possible mass automation in
general, automation of perspectival vision and imaging followed soon. It is this automation of
vision and imaging which gave the West superiority in the "logistics of perception," and
topography allows one to control reality across space and time. The advantage of perspective is
that it makes it possible to present this information in an easily readable form within a single
image.
In the West, until this century, perspective formed the foundation of techniques and
extend perspective, utilizing to the extreme its inherent qualities such as the algorithmic
character and the reciprocal relationship it establishes between reality and representation. For
instance, the perspective algorithm, a foundation of both computer graphics and computer vision,
is used to generate perspectival views given a geometric model and to deduce the model given a
perspectival view. In another example, technologies of remote sensing, such as radar, extend
perspectival vision beyond the visible, relying on the fact that "what is an issue in geometric
perspective is simply the mapping of space, not sight" (Lacan). Yet, while giving rise to new
technologies of "geometric vision," perspective also becomes a limit to the final automation of
sight -- recognition of objects by a computer. Finally, it is displaced from its privileged role,
becoming just one among other techniques of space mapping and visualization.
In order to automate sight, to replace human vision with machine vision, a new
understanding of vision became necessary. David Marr's Vision (1980) has been the most
influential account of this paradigm, shared by computer scientists and psychologists. The book
What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what
is where by looking. In other words, vision is the process of discovering from images what is
present in the world, and where it is.244
There is nothing "plain" about this definition of vision. Marr projects the goals of computer
vision onto human vision: the identification of objects represented in an image and the
reconstruction of their positions in three-dimensional space. The larger part of the book is
devoted to the discussion of algorithms by which the human nervous system may accomplish this
reconstruction.
In this way, nature was redefined as the most useful instrument in its own replacement.
What David Marr, and many others have assumed, is that the key to successful computer vision
systems lies in emulating the algorithms hidden in human nervous system. If only these
algorithms, created by nature during millions of years of evolution, could be understood, the way
The automation of vision is a part of the overall process of industrial automation after
World War II. Automation affects the modern understanding of vision in two crucial ways. On
the one hand, automation entails the replacement of human cognitive functions by a computer,
such as the substitution of vision by computer vision. Here, visual nominalism becomes the
dominant paradigm for the study of vision. On the other hand, the automation also involves the
integration of human and machine in new man-machine systems, such as radar installations or
pilot cockpits. The need to describe the performance of human and machine components in the
A modern engineer, if his goal and the functional purpose of his work are clear, can without any
great effort make a product that is formally adequate and perfect in its economic construction. But
the photographic advertisements of our time are not so easy to define. They don't come with
"user's instructions." Research into the physiological and psychological laws of visual
effectiveness is still far behind the times, compared to the study of the physical laws.245
In the opening chapter, where this quotation already appeared, I have traced one line of research
into "the physiological and psychological laws of visual effectiveness," suggesting that
investigations of the psychological effects of basic colors and elemental forms, conducted in
experimental psychology since the second half of the nineteenth century made possible the idea
of a rational visual language composed from simple elements. In theory, this model called for the
enumeration of visual elements and the description of their effects in order to compose
dictionaries of "visual effectiveness." In practice, it translated into visual works which at first
only contained and then gradually became fully composed of simple abstract forms. Such theory
and practice were already pursued in the nineteenth and early twentieth centuries by a number of
artists (Seurat, Signac, Kandinsky, Klee, Mondrian) and critics interested in "scientific
aesthetics" (Blanc, Henry). However, in the 1920s, when artists found themselves in the roles of
the designers of mass communication, the idea of a scientifically grounded visual language
acquired new urgency and importance. It was no longer a question of scientific aesthetics, of a
work of art producing an aesthetic effect in an individual spectator. Now it became a question of
design of mass communication was too important to be left to an artist's intuition. Therefore,
Bauhaus welcomed experimental psychologists. The trend reached its extreme in Soviet Russia,
where artists thought that they could control not just the consumer habits of segments of society
(as in the West), but the consciousness of the whole country --"affecting mass psyche, organizing
the will of the class" (Tretyakov). Here the discourse of engineering visual communication
laboratories at various art institutions, such as VKhUTEMAS, GAKhN, and the State Institute of
Artistic Culture. Their goal was to create a fully rationalized visual language of mass
presented at INKhUK: "visual sensations as such will concertedly shape the human being as an
organized unit."246
This is one development, one trajectory -- from Blanc's theories and Fechner's
the masses through perception was systematically pursued. This trajectory leads from Wundt's
from the laboratory to the sites of mass communication -- a movie hall, a printed page, an
architectural space, a stadium, a city street with posters, advertisements, and placards.
In this concluding chapter I will follow the next part of this trajectory, the next stage in
interfaces. The visual language advocated by Rodchenko or Lissitsky -- elemental abstract forms,
each having a predictable effect -- seemed the most effective solution to the problem of
246 Nicholas H. Allison, ed., Art Into Life: Russian Constructivism 1914-1932 (New York:
Rizzoli International Publications, 1990), 81.
rationalizing vision. But after World War II, a new kind of rationalization became necessary. A
new paradigm was now needed to encompass not only human communication but also
communication between a human and a machine, the two linked in a human-machine system. It
The new paradigm, called information theory, emerged in the 1920s in response to the
order to measure the efficiency of communication systems, such as telephone, radio, and
television. However, in the late 1940s, the theory left its engineering context and was embraced
by aestheticians, linguists, social scientists, and semioticians. Most importantly, it was taken up
by psychologists faced with the practical task of designing human-machine systems which
increasingly came to dominate both the battlefield and the workplace after World War II (radar
screen, aircraft controls, computer terminals of the automated factory). As human vision became
the main channel of communication between human and machine, it also came to be understood
in terms of information theory which now provided a new quantitative model of a human
observer. Thus the dreams of the avant-garde for an engineering approach to visual
The avant-garde was concern with rationalizing vision, with creating visual communication
engineers, as can be seen from Moholy-Nagy's statement. But what is a modern engineer?
The engineering profession emerged in the 1870s as the liaison between science and
industry. The job of an engineer was to put any industrial or corporate process on a scientific
basis in order to achieve the maximum output with a minimum investment of time, materials,
and energy. In other words, the job of an engineer was to make a process efficient, regardless of
what industry s/he worked in. A 1933 book on engineering defines it in this way: "Engineering is
the science and art of efficient dealing with materials and forces...it involves the most
economical design and execution of a vast number of important undertakings, assuring, when
properly performed, the most advantageous combination of accuracy, safety, durability, speed,
simplicity, efficiency and economy possible for the design and service."247 The avant-garde
artists aimed to apply these principles of engineering to the field of mass visual communication.
In 1921, during a heated discussion at INKhUK (Institute of Artistic Culture) over how to
turn their fellow artists into "constructivists," who would utilize modern engineering methods for
mass propaganda -- participants expressed doubts about whether traditional artists could
accomplish this transition from "art into life." Boris Arvatov suggested that accomplished artists
should become political activists, and the young ones would have to go to the polytechnic.248
More categorical was the opinion of Sternberg, who, after asking the same question, "What are
artists to do?," answered it in these words, "They're good for nothing, they should be dealt with
the way the Cheka deals with counterrevolutionaries."249 A few minutes later in the discussion
he suggested a more humane solution, the solution which was repeatedly put into practice in
regard to Soviet non-conformist artists decades later -- their confinement to a mental institution.
"All the artists who conduct youth groups, and who are going to continue their work and perhaps
teaching the way they have in the past, we should really stuff a hospital with these artists."250
Arvatov's request that artists should get their training in an engineering school, and
Sternberg's warning that those who refuse to comply would be shot, illustrate the grave
seriousness with which many Soviet avant-garde artists of the 1920s took the ideal of
engineering as a model for their own practice. Consider the term constructivism itself. In
247 Qtd. in Theodore Hoover and Charles Fish, The Engineering Profession (Stanford: Stanford
University Press, 1941), 416. Emphasis mine -- L.M.
248 Allison, Art Into Life, 74.
249 Ibid.
250 Ibid., 77.
Russian, konstruktor (the one who constructs) is a synonym for engineer. This allowed a number
of artists to call themselves konstruktors and to refer to their movement as constructivism. The
term conveniently emphasized that, like engineers, they would construct utilitarian objects or
artists who were said to rely on inspiration. Translated into English, the word constructivism
loses its close association with engineering, leading to a somewhat different interpretation of the
entire movement.
In the early 1920s, constructivist artists competed to be the best at embodying the ideal of
an engineer. Vertov announces his desire to become "kinok-engineer, controlling cameras over
distance."251 Eisenstein, who, like Sternberg, graduated from an engineering school before the
1917 Revolution, compares the theatrical apparatus (actors' performances, sound effects,
lighting) to the machinery (orudie obrabotki) which a director should use to form the viewer -- as
a milling machine forms machinery parts.252 Later, he likens the effect of a film to the work of
an engine -- a significant analogy since engineering was historically identified with the design of
engines.253 Tatlin photographs himself next to his already legendary design for the monument to
the Third International. The monument, with its two intertwining lattice spirals 400 meters high,
looks like an Eiffel Tower put at an angle, eternally moving into the sky, a symbol appropriate to
signify the idea of permanent revolution, the (future) industrial might of the new state and, at the
same time, the engineering skills of its designer. Finally, Lissitsky and his students design
251 Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 143.
252 Sergei Eisenstein, "Montazh atraktsionov (Montage of attractions)," LEF 3 (1923): 71.
253 A book on the engineering profession published in 1941 quotes these dictionary definitions
of engineering: "engineering: originally, the art of making engines; in its modern and extended
sense, the art and science by which mechanical properties of matter are made useful to man in
structures and machines;" "engineering: the art and science of making, building, or using engines
and machines, or of designing and constructing public works or the like requiring special
knowledge of materials, machinery, and the laws of mechanics." Hoover and Fish, The
Engineering Profession, 416.
Here is another device: similar to traditional portraits where the identity of a person was
established by depicting the instruments of his or her craft, constructivists' portraits and self-
portraits prominently feature engineering instruments. Lissitsky's self portrait: on the background
of graphed paper -- his face and through it -- a hand holding a compass, its leg touching a
perfectly drawn square. The title of this photomontage is Konstruktor. Even more dramatic is
another Lissitsky's montage, a portrait of Tatlin (fig. 16). Tatlin is standing on a chair, holding a
long pole -- a ruler. By his feet are mathematical symbols (an integral, a square root), a curve
crossing axis lines. Behind him is a blackboard with more symbols. And, a monstrosity: a huge
compass, widely open, growing straight from Tatlin's eye. A man-compass -- measuring,
calculating, constructing.
positive," declared architect Vesnin during one INKhUK discussion, "but an artist must look
after his own affairs, and his affair is the [psycho-physiological] effect of form on [human]
consciousness." In other words, Vesnin is saying: let us leave the engineering of machines and
mass-produced objects to real engineers. Our business, the business of designers is the
architecture, mass spectacles, posters, designs, photomontages, which would produce calculated
bodily responses, emotions, and meanings in the viewers. "The modern engineer has created
brilliant objects: the bridge, the steam engine, the airplane, the crane,...the modern artist must
create objects that are equal to them in power, intensity, and potential in the context of their
artist is to engineer vision, and to engineer vision means to affect the viewer with engineering
254 Qtd. in Anatole Senkevitch, Jr., "The Sources and Ideals of Constructivism in Soviet
Architecture," in Art Into Life: Russian Constructivism 1914-1932, ed. Nicholas H. Allison
(New York: Rizzoli International Publications, 1990), 173-174.
In this statement from the early 1920s, Vesnin envisions an engineer as the designer of
physical objects -- the bridge, the steam engine, the airplane, the crane. But already in the same
decade a new kind of engineer appears, whose job is to design not objects but communication
systems -- telephone, radio, television. A new paradigm appears as well -- information theory
which becomes the scientific basis of communication engineering. In the next section, I will
In 1922 Moholy-Nagy created his famous "Telephone Pictures." Here is his account:
In 1922 I ordered by telephone from a sign painter five paintings in porcelain enamel. I had the
factory's color chart before me and I sketched my paintings on graph paper. At the other end of the
telephone the factory supervisor had the same kind of paper, divided into squares. He took the
dictated shapes in correct position. (It was like playing chess by correspondence.)255
There is no conclusive proof that this event actually took place. At any rate, the story helped
Moholy-Nagy get a teaching job at the Bauhaus the next year, where he could fully pursue his
vision of using the latest communication systems as artistic tools. His colleague wrote of the
Bauhaus atmosphere: "There is incessant talk of cinema, optics, mechanics, projection and
continuous motion...Is this the atmosphere in which painters like Klee and some others of us can
go on developing? Klee was quite depressed yesterday when talking about Moholy."256
The story of "Telephone Pictures" became one of the myths of modern art, or rather, of
the modern struggle against the Romantic notion of art: to refuse subjectivity, the unique touch
255 L‡szl— Moholy-Nagy, The New Vision (New York: Wittenborn, 1947), 79. The three
existing accounts of this episode are discussed in Eduardo Kac, "Aspects of the Aesthetics of
Telecommunications," in SIGGRAPH '92 Visual Proceedings, ed. John Grimes and Gray Lorig
(New York: The Association for Computing Machinery, 1992), 52-53.
256 Qtd. in Otto Stelzer, "Moholy-Nagy and His Vision," in L‡szl— Moholy-Nagy, Painting,
Photography,
2. Information Film (1925;
Theory: reprint, Cambridge:
an Engineer The MIT Press, 1973), 146.
Analyzes Communication
of the artist's hand and the artist's eye (exemplified in Bauhaus by Klee) and to embrace the
mass-produced, the industrial, the mechanical, electrical, electronic, and digital -- from
Duchamp's ready-made to Op Art, Kinetic Art, copy art, telecommunication art, fax art,
computer art -- in short, the tradition of industrial art, throughout the twentieth century feeding
intended the story this way, as a myth, a beginning more important than Duchamp's ready-
mades. Indeed, not just art as chess, as a logical game (as with Duchamp), but -- art as "playing
chess by correspondence," a communication over distance between a sender and a receiver who
Let us analyze Moholy-Nagy's story from the point of view of telecommunication -- the
communication over distance. It mentions three different communication systems. First, the most
ancient: communication with the help of signs, such as traffic or street signs (Moholy-Nagy
orders his pictures at the sign factory). Second, more modern: the postal service. Third, still a
recent development for the 1920s: the telephone. As communication systems developed, they
made possible the transmission of more information over longer distances in a shorter period of
time. The effectiveness of road signs was limited by the distance at which they were visible, and
the amount of information they carried was minimal. With the postal service, this distance limit
was overcome, but a significant time delay remained. This delay was finally eliminated by the
telephone.
In the story, Moholy-Nagy communicated his design to the factory supervisor over the
telephone, dictating coordinates and colors of squares. He was able to do this only by fitting his
design to a pre-existing factory grid, treating the grid's squares as minimal units of his
composition. (Moholy-Nagy, in other words, fully embraced digital visuality and computer art
long before digital computers.) But could he, in 1922, transmit this design as a picture -- and in
real time? And what if the picture was an arbitrary image, for instance a photograph of a real
The rapid transmission of photographic pictures was just becoming possible, utilizing the
photographs began to be regularly sent between London and New York over a submarine cable
in less than three hours. On one end, a picture was segmented using a grid, and the tone of each
cell was coded; on another end, the received code would drive a special printer fitted with
A few years later, Moholy-Nagy included the examples of this "wireless telegraphed
photography" in his Painting, Photography, Film, proudly putting them at the very end of the
photographic section of the book to represent the latest achievement in mechanized tele-seeing
(fig. 17). Yet, this system was not fast enough for the age of "the film; the electric sign,
was limited by the channel capacity of an older communication system -- telegraphy. Images
contained much more information than telegraphic cable was originally designed to carry, so
speed had to be sacrificed. Throughout the 1920s, engineers in Russia, the U.S., England, and
Germany were working on a new system to transmit images in real time called television --
radio, and television, along with their growing economic importance -- urged the creation of a
general communication theory.259 With telephone, radio, and television becoming the
foundations of whole new industries in the earlier decades of the century, the question of
257 M. D. McFarlane, "Digital Pictures Fifty Years Ago," Proc. IEEE 60, no. 7 (1972).
258 Moholy-Nagy, Painting, Photography, Film, 39.
259 For a history of the developments leading to the creation of mathematical theory of
communication, see Colin Cherry, On Human Communication, 2nd ed. (Cambridge: The MIT
Press, 1968), 41-52.
efficiency came to the foreground. Engineers needed a quantitative theory which could define
and measure the commodity that was communicated over telegraph, telephone, and radio
networks. Quantitative measures were also required in order to compare the capacities of various
communication systems (for instance, wire versus radio transmission) and to evaluate their
performance.
The distant precursor to modern communication theory can be found in the centuries-old
interest in inventing secret codes for military and diplomatic communication. But with the
deciphering of messages more difficult for the enemy was joined by a new problem, that of
designing codes which could make everyday communication more efficient. Already by 1825,
telegraphists designated certain commonly used messages, such as commercial expressions and
greetings, by single numbers, thus making coded messages shorter. The famous dot-dash code,
introduced by S.F.B. Morse in 1832, associated the most commonly used letters with the shortest
dot-dash symbols, thus making coded messages use even fewer symbols.260
The problem of how to devise the most efficient methods of encoding, transmitting, and
decoding signals was greatly intensified by the development of television in the 1920s.
Television, the "Far Seer," required the simultaneous transmission of a much greater amount of
off at high speed at the camera end, transmitted, and reassembled in the receiver."261 The great
channel capacity required for television spurred new theoretical studies, and by the end of the
1920s, the fundamental ideas of the modern mathematical theory of communication (which also
(Significantly, it was the great promise and also the great difficulty in the development of a
The essential ideas of information theory were presented in 1927 at the International
Congress of Telegraphy and Telephony by R.V.L. Hartley of the Bell Telephone Laboratories in
measure whereby the capacities of various systems to transmit information can be compared"
and to "discuss its application to systems of telegraphy, telephony, picture transmission and
television over both wire and radio paths."262 Hartley's theory conceptualized the transmission
of a message as the successive selection of signs from a set, known to both the sender and the
receiver. If the receiver knows beforehand which of the symbols will be transmitted, no new
uncertainty about which sign can be received. If a particular message is made up from N signs
which can be chosen from a total set of S signs, then there are SN distinct possibilities. Hartley
proposed that the "quantity of information" (H) of such a message is H=N log 2S. Hartley also
established one of the basic laws of information theory: the total amount of information a system
can transmit "is proportional to the product of the frequency range which it transmits by the time
available for transmission."263 In other words, the more signals a system transmits
simultaneously (frequency range, or bandwidth) and the longer the time available for
With Hartley's paper the two fundamental tools of information theory were now in place:
communication system. It may seem that the theory defined information in an abstract and
counter-intuitive way by associating it with the freedom to make a choice (on the part of the
262 R.V.L. Hartley, "Transmission of Information," Bell System Tech. Journal 7 (1928): 535.
263 Ibid., 554.
sender) and with uncertainty (on the part of the receiver). Yet, the definition of information is
quite pragmatic from the point of view of an engineer concerned with making a communication
system efficient. What the measure of information (H) defines is the minimum number of binary
choices which enables the receiver to reconstruct the message, or conversely, which is required
to transmit it.264 In other words, "it is a measure of the minimal effort by which message can be
transmitted."265
The development of information theory was further motivated by the work of scientists
and engineers on communication and electronic systems during World War II, the most
important of which was radar. In its modern form, the theory was formulated by Claude
Shannon; in 1949, his papers, together with a non-technical overview by Warren Weaver, were
published in what became the bible of the coming information age -- Mathematical Theory of
Communication.266
more general model of any communication situation (fig. 18). To emphasize this generality, his
human communication:
The information source selects a desired message out of a set of possible messages...The
selected message may consist of written or spoken words, or of pictures, music, etc.
The transmitter changes this message into the signal which is actually sent over a
communication channel from the transmitter to the receiver. In the case of telephony, the channel
264 Each binary choice constitutes one bit of information. Weaver writes: "if one has available
say 16 alternative messages among which he is equally free to choose, then since 16=24 so that
log 216=4, one says that this situation is characterized by 4 bits of information." Claude E.
Shannon and Warren Weaver, The Mathematical Theory of Communication (Urbana: The
University of Illinois Press, 1949), 100-101.
265 D. Gabor, "A Summary of Communication Theory," in Proceedings of a Symposium on
Applications of Communication Theory, London, 1952, ed. Willis Jackson (London: Butterworth
Scientific Publications, 1953), 2.
266 Shannon and Weaver, The Mathematical Theory of Communication.
is a wire, the signal a varying electrical current on this wire; the transmitter is the set of devices
(telephone transmitter, etc.) which change the sound pressure of the voice into the varying
electrical current. In telegraphy, the transmitter codes written words into sequences of interrupted
currents of varying lengths (dots, dashes, spaces). In oral speech, the information source is the
brain, the transmitter is the voice mechanism producing the varying sound pressure (the signal)
which is transmitted through the air (the channel). In radio, the channel is simply space (or the
aether, if any one still prefers that antiquated and misleading word), and the signal is the
electromagnetic wave which is transmitted.
The receiver is a sort of inverse transmitter, changing the transmitted signal back into a
message, and handing this message on to the destination. When I talk to you, my brain is the
information source, yours the destination; my vocal system is the transmitter, and your ear and the
associated eighth nerve is the receiver.
In the process of being transmitted, it is unfortunately characteristic that certain things are
added to the signal which were not intended by the information source. These unwanted additions
may be distortions of sound (in telephony, for example), or static (in radio), or distortions in shape
or shading of picture (television), or errors in transmission (telegraphy or facsimile) etc. All of
these unwanted changes in the transmitted signal are called noise.267
It is not accidental that the principal examples in this description -- telegraphy, telephony, and
oral speech -- all have to do with written or verbal communication. By the middle of the century,
both the engineer and the public still associated electronic communication with telephone and
radio, and this is why the example of human communication which naturally comes to Weaver is
speech. It would be few decades before electronic communication would become synonymous
ISDN, cyberspace); before the terms of information theory, such as channel, bit, bandwidth will
enter the speech of journalists -- telling us everyday about yet another development in visual
telecommunication -- as well as the speech of policymakers, endlessly debating who will control
The Mathematical Theory of Communication appeared in 1949 and within years the theory was
taken up by linguists, psychologists, social scientists, and even art historians. Thus, Roman
267
3. Ibid.,
The 98-99.ofEmphasis
Influence in the
Information original.
Theory or the Ideology of the Code
Jakobson, who as a professor at MIT was close to the source of the paradigm, adopted the model
for linguistics and then semiotics.268 Even quicker to embrace the paradigm were experimental
psychologists.269
As information theory was transformed from a tool of the communication engineer into a
broad intellectual paradigm, three crucial developments took place. First, the theory was
extended to understand not just communication within an electronic system, but also
communication between humans as well as between humans and machines. Second, the theory
was adopted to describe the meaning and effect of communication -- its semantic and pragmatic
aspects. Third, the basic assumption of the theory, that communication is a one-way process, was
extended to theorize social communication. In this section, I will first consider these three
developments in turn.
The first development is connected with the figure of Norbert Wiener, a professor of
mathematics at MIT who supervised Shannon's research. In 1948 Wiener, published Cybernetics
or Control and Communication in the Animal and the Machine which proposed that seemingly
control, thus opening a way to think of social, political, and economic problems in information-
When I communicate with another person, I impart a message to him, and when he communicates
back with me he returns a related message which contains information primarily accessible to him
In this explication Wiener talks about human communication, but his realization that the
during the war when he worked on anti-aircraft artillery systems. Wiener realized that the key to
successful control lies in feedback -- the modification of behavior of a system (be it human or a
machine) by taking into account the results of the actual (rather than predicted) performance. The
results of the performance are communicated back to the control mechanism, making it adjust
"When we desire a motion to follow a given pattern, the difference between this pattern and the
actually performed motion is used as a new input to cause the part regulated to move in such a
The authors of information theory always insisted that the theory was only concerned
with the conditions for the correct and efficient transmission of signals constituting the message,
and not with its meaning, interpretation, or effect. Hartley points out that "in estimating the
capacity of the physical system to transmit we should ignore the question of interpretation."272
Shannon also begins by insisting that "semantic aspects of communication are irrelevant to the
engineering problem."273 Finally, Weaver starts his essay included in The Mathematical Theory
technical (how accurately the signs can be transmitted), semantic (how accurately signs convey
270 Norbert Wiener, (1950), The Human Use of Human Beings. Cybernetics and Society (New
York: Avon Books, 1967), 24-25.
271 Norbert Wiener, Cybernetics. Of Control and Communication in the Animal and the
Machine (New York: John Wiley & Sons, Inc., 1948), 13.
272 Hartley, "Transmission of Information," 538.
273 Shannon and Weaver, The Mathematical Theory of Communication, 3.
the desired meaning), and pragmatic (how effective the message is in affecting the behavior of
the receiver). Weaver refers to these levels as A, B, and C.274 He next emphasizes that the
mathematical theory of communication as developed by Shannon applies only to the first level,
yet also immediately suggests that it is deeply relevant for the second and third. Impressed by the
fact that the technical aspect of a communication process is finally controlled by a precise
mathematical theory, Weaver clearly hopes that the theory can somehow be extended to handle
the elusive questions of semantics and pragmatics with the same quantitative precision.
Throughout the essay he promises a detailed discussion of how this can be done. Weaver never
keeps his promise; instead, in a number of places he suddenly announces that perhaps no
modifications will be necessary. The announcements appear abruptly at the end of sentences,
explosions in what is otherwise the dry rhetoric of a scientific discourse. "It is the purpose of this
concluding section to review the situation, and to see to what extent and in what terms the
original section was justified in indicating that the progress made at Level A is capable of
contributing to levels B and C, was justified in indicating that the interrelation of the three levels
is so considerable that one's final conclusion may be that the separation into the three levels is
really artificial and undesirable." "It is almost certainly true that a consideration of
communication on levels B and C will require additions to the schematic diagram on page 97
[fig. 18], but it seems equally likely that what is required are minor additions, and no real
revision."275
The temptation to extend information theory to handle the meaning and effect of
communication proved too great to resist. This was understandable given the historical context of
the Cold War during which the theory matured and attracted public attention. This context is not
example, the meaning to a Russian of a U.S. newsreel picture." "The problem of effectiveness
involves...all the psychological and emotional aspects of propaganda theory."276 Even more
revealing is the opening paragraph of the essay, where within a few lines we are transported from
artistic communication, painting, theater, and ballet, to the theater of the new cybernetic warfare,
which emerged during World War II and now protected the U.S. against a Soviet attack:
The word communication will be used in a very broad sense to include all of the procedures by
which one mind may affect another. This, of course, involves not only written and oral speech, but
also music, the pictorial arts, the theatre, the ballet, and in fact all human behavior. In some
connections it may be desirable to use a still broader definition of communication, namely, one
which would include the procedures by means of which one mechanism (say automatic equipment
to track an airplane and to compute its probable future positions) affects another mechanism (say a
guided missile chasing this airplane).277
In 1952 Y. Bar-Hillel, who was working in the field of automatic language translation (which,
as I mentioned in the previous chapter, also developed during Cold War with the promise to
automate the surveillance and analysis of Soviet military communication and media), extended
language system. In the same years, the Shannon-Weaver model (fig. 18) was adopted in the
U.S. by the growing field of mass communication studies as the theoretical foundation of the
field. Weaver's hope that the model, originally designed to deal solely with the technical aspects
of telecommunication, could be applied to describe semantic and pragmatic levels as well, was
fully realized: the model was adopted to describe the process of mass communication: sender
(film studio, TV station, publisher) -- message (a film, a television program, a newspaper story) -
Shannon begins his exposition of information theory by stating that "the fundamental problem of
selected at another point."279 American scholars of mass communication applied the same
postulate to the semantic level. Even a textbook on mass communication published in the 1970s,
still states: "if the meaning of the destination is isomorphic with the meaning of the source which
originated the act, then communication can be said to have taken place."280 With such a
definition, any discrepancy between the codes of a sender and a receiver becomes undesirable
"noise." As Stuart Hall pointed out, the ideology lies in assuming that encoding and decoding
codes are or should be the same: "What are called 'distortions' or 'misunderstandings' arise
precisely from the lack of equivalence between the two sides in the communication
exchange."281 Thus, while American social scientists contrasted American democracy with
Soviet totalitarianism, they simultaneously adopted the theoretical model according to which
mass communication was synonymous with following the prescribed meaning. In short,
Despite the fact that modern mass communication crucially depends on the visual channel,
American scholars of mass communication never seriously confronted its visual aspects. Others,
however, have tried to use information theory to understand visual communication. I will next
279 Shannon and Weaver, The Mathematical Theory of Communication, 3. Shannon includes
"approximately" because one of the main concerns of a communication engineer is to increase
the efficiency of a communication system by making it transmit only the information which is
absolutely essential for the correct identification of a message by the receiver. For instance, the
telephone does not carry the whole range of frequencies of the human voice but only those
sufficient for understanding what is said.
280 Melvin L. De Fleur and Sandra Ball-Rokeach, Theories of Mass Communication (New
York: Longman, 1975), 127.
281 Stuart Hall, "Encoding/Decoding," in Culture, Media, Language, ed. Stuart Hall et al.
(Hatchinson, 1980), 131.
discuss these attempts which took place in the late 1950s and in the 1960s in semiotics,
In the early 1960s Roland Barthes published The Photographic Message and Rhetoric of
the Image where he applied a semiotic apparatus to the analysis of mass media images, in his
case, publicity photographs.282 However, it may be surprising to discover that The Photographic
The press photograph is a message. Considered overall this message is formed by a source of
emission, a channel of transmission and a point of reception. The source of emission is the staff of
the newspaper, the group of technicians certain of whom take the photo, some of whom choose,
compose and treat it, while others, finally, give it a title, a caption and a commentary. The point of
reception is the public which reads the paper. As for the channel of transmission, this is the
newspaper itself...
Barthes, however, was not going to propose, as communication researchers did, that the
system of mass communication also contains a feedback mechanism -- the right of readers to
write letters to newspapers with complaints or suggestions -- thus assuring that mass
communication truly serves the purposes of Democracy and Free Speech, rather than
unquestioned ideology. In fact, Barthes' whole purpose in his articles on the semiotics of mass
media images was to reveal the mechanisms by which images communicate or subvert an
ideological meaning. Communicate or subvert: Barthes' position oscillates between these two
mutually exclusive positions. On the one hand, Barthes proposes in The Rhetoric of the Image
that the specificity of a photograph's codes is what allows it to naturalize the ideological message
in a particularly transparent manner; on the other hand, in the same article he also claims that an
image is essentially polysemous and therefore a caption is required to anchor a single meaning.
In a later article The Third Meaning Barthes makes an even stronger claim for the subversive
282 Roland Barthes, (1961), "The Photographic Message," in Image, Music, Text, ed. Stephen
Heath (New York: Hill and Wang, 1977); "Rhetoric of the Image," (1964), in Image, Music,
Text.
nature of the visual code in relation to both ideology and interpretation by suggesting that a film
still contains an excess, an "obtuse meaning" not reducible to its "informational" or "symbolic"
meanings.283
The code. More than any other, it was this particular concept of information theory that
entered semiotics in the 1960s. And for semioticians concerned with visual signs, it became the
main preoccupation of their research: is there a unique visual code, an autonomous "language of
pictures," and if so, what is its specificity? Now, after we have followed the development of
information theory, we can see the historical specificity of approaching vision as a code by
Barthes, J.M. Floch, Felix ThŸrlemann, the members of the Groupe Mu, Fernande Saint-Martin,
and other semioticians.284 The notion of the code, as information theory itself, has emerged in
response to the practical problem faced by engineers: how to make telecommunication systems
more efficient and how to measure this efficiency. Encoding is one of the principal ways to
achieve this efficiency. Morse code condensed transmitted messages by associating the shortest
combination of dots and dashes with the most frequently used letters. Later, communication
engineers were able to achieve even greater economy by encoding only as much information as
necessary for the receiver to decode the message with sufficient accuracy.285 When carried over
to visual semiotics, the notion of the code retained its original engineering meaning, its
association with the issue of efficiency. Indeed, to ask what kind of information can be obtained
through seeing and/or represented as images; how visual codes differ from other kinds of codes;
whether visual representations have any unique qualities in terms of their effect on the viewer --
in short, to propose the uniqueness of the visual code or to deny this uniqueness is already to
283 Roland Barthes, (1970), "The Third Meaning," in Heath, Image, Music, Text.
284 For a critical discussion of visual semiotics, see Gšran Sonesson, Pictorial Concepts.
Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World
(Lund, Sweden: Lund University Press, 1989).
285 A principal contribution of Shannon was his famous Sampling Theorem which specified the
absolute minimum of the signal which needs to be retained before information is irreversibly
lost.
conceive of vision as productive, as potentially a special way of transmitting information, to
The semioticians did not go as far as to apply the mathematical apparatus of information
theory to the analysis of visual signs. However, the field of experimental aesthetics, which, from
its origins in the nineteenth century always tried to reduce the effect of an image to a
mathematical formula, welcomed the quantitative aspect of the theory. But how to apply a theory
developed for the purpose of engineering communication to a very different question of aesthetic
information in an image could represent its aesthetic effect. In the 1960s the German aesthetician
Max Bense developed a theory of "information aesthetics" and tried to measure the
"information" contained in such classical images as Rembrandt's etchings. It was not at all
researchers went on to construct simple abstract patterns where "information" was easier to
quantify. They reasoned that since the amount of information in a message is directly
proportional to its unpredictability and inversely proportional to its redundancy, a pattern made
from small squares, where the tone of each square is determined randomly, would be the most
unpredictable and therefore would contain the most information. On the contrary, in a pattern
like Mondrian's or Malevich's painting there is much more redundancy (more predictability) and
consequently, the amount of information is low. In a number of experiments, subjects rated their
"aesthetic pleasure" when looking at different patterns (fig. 19); psychologists then tried to
286 This carrying over of assumptions of information theory into semiotics reaches its extreme in
Jurij Lotman's Structure of the Artistic Text (1970). Lotman was strongly influenced by
cybernetics which in the 1960s was more popular in the Soviet Union than it ever was in the
West and which became as important for Soviet semioticians as Saussure or Peirce. (In part, it
was used to justify semiotics in the eyes of the authorities by presenting it as a part of
cybernetics, concerned with the study of different artificial languages, from computer languages
to the languages of art.) Lotman claimed that the artistic text is a message coded in a particular
way and that art is the most economical way to transmit information because of the unique
features of the artistic code. Jurij Lotman, (1970), The Structure of the Artistic Text (Ann
Arbour: Department of Slavic Languages and Literatures, The University of Michigan, 1977).
determine the level of visual organization that was biologically most pleasurable.287
Finally, one of the most influential texts of post World War II art history is also heavily
indebted to information theory.288 In Art and Illusion (1960) Ernst Gombrich provides a
revealing analysis of Greek vases and mosaics as precursors to modern digital codes; he
compares the reading of pictures to the decipherment of codes; and he uses various concepts of
information theory throughout the book.289 However, Gombrich does more than simply pay
tribute to a fashionable theory. During the war, Gombrich was employed for six years by the
British Broadcasting Corporation to listen for radio transmissions.290 He was confronted daily
with the central problem of modern information theory -- the identification of signal in the
presence of noise. As he himself admits, it was this experience which led to his deep interest in
Rather than being accidental, the paradigm of information theory plays a central role in
Art and Illusion. One of Gombrich's key concepts is "the beholder's share": the importance of
287 See, for instance, D.E. Berlyne, "Novelty, Complexity, and Hedonic Value," Perception and
Psychophysics 8 (1970): 279-86; D. Dorfman and H. McKenna, "Pattern Preference as a
Function of Pattern Uncertainty," Canadian Journal of Psychology 20 (1966): 143-53; P.C. Vitz,
"Preference for Different Amounts of Visual Complexity," Behavioral Science 2 (1966): 105-14.
It is interesting to compare this work in experimental aesthetics with Lotman's view of art as
communication (see footnote 278). Lotman proposed that a unique feature of art is that it carries
more information than any other kind of communication, while experimental aestheticians
proceeded under the assumption that there is some optimal level between the complete lack of
information and the complete saturation by it which characterizes the most pleasurable works of
art.
288 I am grateful to Michael Ann Holly and Norman Bryson who brought to my attention the
relevance of Art and Illusion to the problematic of this chapter.
289 E.H. Gombrich, Art and Illusion (Princeton: Princeton University Press, 1960), 39-40, 88,
205.
290 Ibid., 204.
291 Ibid., 208, 211.
image is always ambiguous, potentially corresponding to numerous spatial configurations. It is
equally ambiguous as a representation of other aspects of visual reality, substituting a few brush
strokes for a field of grass, a few spots of color for a human form. The job of a beholder is to
supplement the partial information in the image with her or his own projection. In other words,
a signal in the presence of noise. According to information theory, this decoding is always
probabilistic: there can never be the absolute certainty than an original signal is correctly
however, interprets it as the source of our aesthetic pleasure. We derive pleasure from art
precisely because in trying to "identify a signal" we have to use our imagination. There is an
unexpected agreement here between Gombrich and experimental aestheticians of the 1960s, such
as Bense. Both start from the assumption that the aesthetic pleasure is proportional to the amount
of information in a message, i.e. its unpredictability. As Gombrich explains it, "the greater the
probability of a symbol's occurrence in any given situation, the smaller will be its information
content."292 Because of their ambiguity, representational images in Western art carry significant
Mass communications, visual semiotics, and aesthetics are just some of the fields that adopted
the 1920s, which dreamed of engineering visual communication, would welcome all these
engineer in response to the growth of television, was now used to rationalize the idea that mass
meaning (mass communication studies). The theory, designed to measure only the efficiency of
telecommunications networks, was now adopted to describe all other levels of communication as
well. Visual semiotics and experimental aesthetics extended information theory to the levels of
meaning and effect, the former addressing the specificity of visual signification, the latter
However, it was the field of experimental psychology that was most transformed by
information theory. After World War II, two crucial developments took place. The first is that
who approach human vision as a communication system, studying its channel capacity,
bandwidth, and other engineering characteristics. The second is that work, more and more,
became a matter of the mental processing of visually presented information. And this made
experimental psychology, for the first time since the discipline emerged, practically important; in
fact, as important as communication engineering. In the next three sections we will see how
these developments came about and what their impact was on modern understanding of vision.
Why was it that even though the essential ideas of information theory were articulated in the
1920s, it was only in the late 1940s, after the publication of The Mathematical Theory of
Communication, that the theory was suddenly widely accepted in so many fields? Is there a
relationship between this acceptance and the social and economic changes which were taking
place in modern society after World War II? The best way to answer these questions is to
compare the role of information theory in the second half of the twentieth century to the role
First, a historical parallel. Hermann von Helmholtz formulated the universal law of the
conservation of energy in 1847. Exactly a century later, the theory of information in its modern
form was developed by Shannon and Wiener. Helmholtz was instrumental in turning the theory
of thermodynamics into a more general paradigm which treated the work of nature, the work of
machines, and human labor on the same terms. Similarly, Wiener extrapolated the concepts of
information theory and the techniques of control engineering into the science of cybernetics
which described the organization and behavior of human, machine, and society using a single set
theory. In 1929 the physicist Leo Szilard identified entropy with information, and the measure of
information with the negative of the measure of entropy.293 Entropy is the central notion of
differentiated, and less probable state to a chaotic, undifferentiated, and more probable state.
Since entropy is the measure of randomness, i.e. the unpredictability of a thermodynamic system,
and since information similarly measures the unpredictability of a message, the equations for
entropy and information are the same. As Weaver explains, "for a communication source one can
say, just as he would also say it of a thermodynamic ensemble, 'This situation is highly
But beyond these connections, there is a more crucial parallel. Just as thermodynamics
provided the model for understanding human work in the latter nineteenth and first half of the
293 Howard Resnikov, The Illusion of Reality (New York: Springer-Verlag New York Inc.,
1989), 5.
294Shannon and Weaver, The Mathematical Theory of Communication, 103.
twentieth century, information theory has been crucial for a new post-industrial model of work,
In The Human Motor: Energy, Fatigue and the Origins of Modernity Anson Rabinbach
demonstrated how the scientific ideas of thermodynamics, formulated in the middle of the
nineteenth century, became central for the conception of work in modernity. Helmholtz, who
discovered the law of the conservation of energy, promoted this law as the universal principle
which equally applies to nature, machines, and humans. Helmholtz "portrayed the movements of
the planets, the forces of nature, the productive force of machines, and of course, human labor
power as examples of the principle of conservation of energy."295 All work was understood as
the expenditure of energy, with a crucial consequence of redefining human labor as labor power,
the expenditure of the energy of a body. Thus a worker was redefined as a "human motor." This,
in turn, lead to the emergence, towards the end of the century, of the movement which
Rabinbach calls the European science of work, "the search for the precise laws of muscles,
nerves, and the efficient expenditure of energy centered on the physiology of labor."296 In
manual labor, the energy stored in the body where it was accumulated through the intake of food,
sleep, and rest is transferred into muscular force -- hammerer striking a blow, filer filing a
machine part, and so on. Therefore, psychologists, physiologists and industrial experts searched
for methods to maximize both the accumulation of a worker's energy (through proper nutrition,
shorter working hours, appropriate breaks) and its expenditure in labor. Just as an engineer
designing an engine was concerned with the most efficient transfer of fuel energy into
movement, European work experts aimed to maximize worker efficiency and to eliminate
possible waste. Central to the quest for the efficiency of the human motor was the struggle
295 Anson Rabinbach, The Human Motor: Energy, Fatigue, and the Origins of Modernity (Basic
Books, Inc., 1990), 3.
296 Ibid., 10.
against fatigue, understood as the equivalent of entropy. "As entropy revealed the loss of energy
involved in any transfer of force, so fatigue revealed the loss of energy in the conservation of
Kraft to socially useful production. As energy was the transcendental, 'objective' force in nature,
The European science of work may appear to be very similar to the American scientific
management consultant. As a part of his program, Taylor aimed to minimize and standardize the
time required by a worker to perform each operation. He employed the method of time studies
whereby the best workers were timed and the results became the norm to be followed by the
rest.298 Later, Frank and Lilian Gilberts (he -- an engineer, she -- a psychologist) popularized
another method of motion study.299 They argued that maximizing worker productivity is best
achieved by the elimination of unnecessary movements and making the necessary more efficient.
Although both time and motion studies and the European science of work were concerned with
the efficiency of manual work, there was a fundamental difference between the two
approaches.300 Taylorism aimed for maximum productivity, and had no concern for the
exhaustion and deterioration of the human motor. In contrast, European scientists aimed for
optimum productivity, and therefore were concerned not only with the rationalization of the
workplace, but also with the workers' health, nutrition, safety, and the optimal length of a
workday. In short, Taylorism had no reservations about replacing one exhausted human motor
with another -- the philosophy which in the U.S. seems to go hand in hand with the emerging
ethics of the consumer society and with immigration policies which assured the constant supply
the human motor. The two paradigms converged after World War I, when European
industrialists partly adopted the more brutal, but ultimately more effective Taylorist methods,
while U.S. managment experts became more sensitive to workers' physiology and psychology.
Taylorism reduced the worker's body to a mechanical machine and had no concern for
her or his mind. Indeed, as Marta Braun points out, Taylorism aimed to systematically rob the
worker of any degree of independence or even understanding of the overall work process by
"separating responsibility for the execution of work from its planning or conception."301 This
disdain for the mind was shared by behaviorism, which matured at the same time as the
European science of work and Taylorism, and which equally well characterizes the imaginary of
hard-edged social engineering of the first half of the twentieth century. In 1913, J.B. Watson, the
founder of behaviorism, explicitly defined it as the science of social control: "Psychology as the
behaviorist views it is a purely objective experimental branch of natural science. Its theoretical
goal is the prediction and control of behavior."302 Behaviorism approached the human subject as
Concerned with controlling the body, it almost completely suppressed any studies of perceptual
or mental processes between 1920 and 1950 in the U.S. It was a psychology well suited for
301 Marta Braun, Picturing Time: the Work of Etienne-Jules Marey (1830-1904) (Chicago: The
University of Chicago Press, 1992), 337.
302 Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century
of Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers,
5. 1979), Engineer
Communication 27. Analyzes Human Vision
After World War II, the references to time and motion studies disappear. And by 1960, the
In this shift, the concepts of manual labor, production of goods, and fatigue were replaced
by new concepts of cognitive labor, information processing, and noise. Taylorism, Gilberts'
motion studies, and behaviorism gave way to engineering psychology, "human information
processing," and cognitive science. In short, with the transformation of industrial society into
post-industrial society, the disciplines of the efficiency of the body were replaced by the
disciplines concerned with the efficiency of the new instrument of labor -- the mind.
One obvious and perhaps the earliest sign of this transition was the speed with which
information theory was disseminated into so many fields by the end of the 1940s and early
1950s. But, along with being a symptom of post-industrial society which was yet to fully
manifest itself, information theory also provided actual techniques for defining and studying the
human subject as an "information processing unit," and its visual apparatus as a crucial part of
"human information processing." Information theory therefore is central for understanding the
new ideas and techniques of vision that emerged after the World War II.
The significance of the adaptation of information theory as a new model of the human
observer, which took place in the 1950s, can be best seen through the following analogy:
1. Manual versus mental labor. The point is not whether corporeal labor was indeed
universally displaced by mental labor: this is different from country to country, from industry to
industry. What is important is that the obsession with the rationalization of corporeal work
with the rationalization of the mind (cognitive psychology, artificial intelligence, cognitive
engineering). Regardless of the percentage of the work force that still may be engaged in manual
labor, society is no longer concerned with spending more intellectual resources to perfect
workers' movements. What comes under scrutiny since the 1950s, when cognitive psychology
begins to displace then dominant behaviorism, are mental functions: perception, attention, text
Already in 1961, in an influential study of automation in French industry, Pierre Naville and his
fellow sociologists had described the transition from the "work of the laborer to the work of
this study Rabinbach writes, "The appearance of the cerebral worker whose material and product
is 'information' is emblematic of the vast distance traversed between the worker who surveys
that automation does not lead to the replacement of human by machine. Rather, the worker's role
becomes one of monitoring and regulation: watching displays, analyzing incoming information,
making decisions, and operating controls. And it is the corresponding human functions of
perception, attention, memory, and problem solving which become the subject of research by
What Taylor's scientific management was for the age of industrialization, cognitive
sciences are for the age of automation. In the 1940s, Herbert Simon worked on theories of
management, the field of research originated by Taylor. Having recognized the increasing
importance of mental skills in the corporate workplace, Simon became one of the pioneers of
cognitive science with his work on automatic reasoning by computer. In 1964 he wrote that "the
cognitive science was Jerome Bruner. Reflecting back on his work in the 1950s, he noted in
1983: "It seems plain to me now that the 'cognitive revolution'...was a response to the
2. Just as thermodynamics in the nineteenth century provided a model for the laboring
body, information theory provided a new model for the mental processes, becoming particularly
Communication, which was the first attempt to understand human sensory-motor performance as
information processing, or, as he put it, to describe it "in terms originally developed for
telephone engineering."307 This statement may at first remind us of the comparison between
nerves and telegraphy, frequently evoked by nineteenth century physiologists and psychologists,
including Helmholtz.308 Yet, the approach of Broadbent and other psychologists who followed
him was fundamentally new. It was less concerned with human physiology than with the limits
telephone engineer is concerned with the capacity of the telephone system, a psychologist is
concerned with the amount of visual or auditory information that can be adequately processed by
a human system; sensed, recognized, stored in memory, recalled, acted upon. What for
305 Qtd. in Douglas Noble, "Mental Materiel: The Militarization of Learning and Intelligence in
U.S. Education," in Cyborg Worlds: the Military Information Society, ed. Les Levidov and
Kevin Robins (London: Free Association Books, 1989), 34.
306 Qtd. in Ibid., 34-35.
307 Broadbent, Perception and Communication, 36.
308 Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth
Century (Cambridge: The MIT Press, 1990), 93-94.
For a communication engineer, information theory provides both a standard set of terms
which specify the limits of this performance. For instance, the capacity of a system depends on
the code used. Morse code makes transmitted messages shorter by associating the most
frequently used letters with the shortest dot--dash symbols. The time saved in transmitting,
however, is offset by the additional time now required to encode and decode the message.
Another example is the principle, already established by Hartley, that the maximum amount of
information a system can transmit is the product of its bandwidth and the time available for
transmission. In the 1920s, it was possible to transmit images over telegraph wires -- a small
information capacity, and filter, psychologists also applied its principles to understanding the
limits of human performance. One of Broadbent's contributions was the idea that the nervous
system cannot process all of the sensory stimulation which may be available and that there is "a
filter at the entrance to the nervous system which will pass some classes of stimuli but not
others"309 (fig. 20). In other words, because of its limited bandwidth, a nervous system can only
process a certain amount of information in a given unit of time. How much can be processed
depends on the nature of this information since different human senses such as vision and
capacities.
Once filtered through the senses, the information is further processed through a series of
stages. Each stage is understood as a specific communication system with its own capacities
determined by its codes and bandwidth. Figure 21, taken from 1986 Handbook of Perception and
So just as the productivity of a laboring body was limited by the available energy, it is
now assumed that the productivity of the mind is limited by its information-processing capacity.
In fact, the notion of a limited reservoir of the energy in the body has found a direct correlatation
in the notion of there being a limited amount of human information-processing resources which
can be used in different ways to deal with the incoming information. For instance, the subject
may concentrate all his/her resources on processing visual information, thus sacrificing his or her
3. Fatigue versus noise. Just as fatigue is the opposite of energy, the limit to the body's
the limit to perfect communication and, consequently, the limit to faultless human information
processing. Noise is the source of error. And error is the worst enemy of the human-machine
The late nineteenth century science of work came about as a struggle against fatigue, this
"subversive element in the human motor, the body's fifth column...fatigue was the physiological
limit of even the most perfectly executed work, the horizon of the metaphor of the human
motor."310 As physiologists and psychologists were preoccupied with fighting the body's
fatigue, cognitive psychologists became preoccupied with preventing human cognitive and
asking: "Are errors a basic part of human nature that cannot be changed, or is there some way to
eliminate or reduce error and the damage errors cause?" The experts on human factors classified
reliability, and compiled tables of Human Error Probability (HEP) for different tasks.311
What are the sources of human error? In contrast to a manual worker of the industrial
which present information in real time about the changing status of a system or an environment,
real or virtual: a radar screen tracking a surrounding space (see chapter 3); a computer screen
updating the prices of stocks; a video screen of a computer game presenting an imaginary
battlefield; a control panel of an automobile showing its speed, etc.312 From time to time, some
information causes an operator to make a decision and to intervene in the system's operation: tell
the computer to track an enemy bomber noticed on the radar screen; buy or sell a stock; press a
joystick; change gears. It is not essential that in some situations these interventions may be
required every second (a pilot engaged with an enemy, a computer game player, a financial
analysis monitoring stock prices), while in others they are needed very rarely (a technician
monitoring an automated plant, power station, a nuclear reactor; a radar operator monitoring a
radar screen, waiting for potential enemy planes). This is why a human-machine system is
defined as "an equipment system, in which at least one of the components is a human being who
interacts with or intervenes in the operation of the machine components of the system from time
to time."313
periodic rhythm of perceptual shocks; the experience shared by an assembly line worker, by a
311 Barry H. Kantowitz and Robert D. Sorkin, Human Factors: Understanding People-System
Relationships (John Wiley & Sons: 1983), 30-57.
312 A 1965 textbook on human-machine systems calls an automobile "a first rate example of a
true man-machine system...a highly complex system in which the operator plays a commanding
role or actively intervenes in the system from time to time." Alphonse Chapanis, Man-Machine
Engineering (Bemont, CA: Wadsworth Publishing Company, Inc., 1965), 16.
313 Chapanis, Man-Machine Engineering, 16. Emphasis mine -- L.M.
pedestrian, and by a film viewer. 314 This experience, characteristic of modernization, finds a
direct continuation in one type of cybernetic workplace: the constant overwhelming amount of
information; the constant cascade of cognitive shocks which require immediate interventions (a
pilot engaged with an enemy, a player of a computer game). Now, however, these shocks arrive
exclusively through the visual channel (dials, computer screen, head-mounted display). Thus of
the roles mentioned by Benjamin, it is the film viewer rather than the assembly line worker who
However, there is also a second type of work experience, new to post-industrial society: work as
waiting for something to happen. A radar operator waiting for a tiny dot to appear on the screen;
a technician monitoring an automated plant, power station, or nuclear reactor, knowing that a
software bug will eventually manifest itself, making a pointer on one of numerous dials shoot
situations where nothing may happen for hours eventually loses vigilance. In terms of the
information processing approach, when a signal occurs infrequently and randomly over time, an
operator is more likely to miss the signal.315 This is one source of human error. Another source
314 "Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless,
today's pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus
technology has subjected the human sensorium to a complex kind of training. There came a day
when a new and urgent need for stimuli was met by the film. In film, perception in the form of
shocks was established as a formal principle. That which determines the rhythm of production on
a conveyer belt is the basis of the rhythm of reception of film." Walter Benjamin, "On Some
Motives in Baudelaire," in Illuminations, ed. Hannah Arendt (New York: Schochen Books,
1969), 175.
315 In the extreme case of sensory deprivation the subject becomes totally disoriented,
experiences illusions and even faints. It is interesting that in the 1950s, when automation was
making this new work-as-waiting-for-signal commonplace, experimental psychologists began to
conduct experiments in artificial sensory deprivation. In 1953 L.A. Riggs and his colleagues
discovered that if the image is stabilized on the retina, thus eliminating miniature eye movements
normally accompanying vision, the image vanishes. In 1954-57, W. Heron constructed an
experimental cubicle to study the effects of total perceptual isolation. Subjects, deprived of all
visual, auditory, and tactile stimuli, experienced extreme boredom, restlessness and, eventually,
visual hallucinations. L.A. Riggs, et al., "The Disappearance of Steadily Fixated Visual Tests
is visual illusions, limits to the reliability of human vision, which preoccupied experimental
psychologists from the early days of the discipline. Still another source is the inherent limitations
of a human as a decision maker. When cognitive psychologists began to study human reasoning,
they made a grave discovery: human judgments do not follow statistical probabilities and ignore
information about the prior probability of an event.316 So even if the operator detects a signal in
Along with providing a model that allowed psychologists to conceive of human vision as a
theory also provided a general concept to specify an inherent limit to this performance -- noise.
The incorporation of the concept of noise is what distinguishes modern information theory, as
developed by Shannon, Wiener, Kolmogoroff, and others, from the original formulations by
Hartley. In any real telecommunication system noise is always present due to the random motion
of electrons in all electrical conductors. Therefore, in practice, the signal at the receiving end is
always to some degree different from the signal originally sent. One of Shannon's principal
channel to transfer information in the presence of noise. As Colin Cherry notes, "This channel
limit which no practical system can exceed, and its principal value is that it provides a standard
Just as entropy limits the performance of any thermodynamic system (such as a motor),
leading to some of the energy being lost unproductively, noise sets the limit on the efficiency of
Objects," J. Optical Soc. America 43 (1953): 495; W. Heron, "The Pathology of Boredom,"
Scientific American 196 (1957): 52-56.
316 Gillian Cohen, The Psychology of Cognition, 2nd ed. (London and New York: Academic
Press, 1983), 190.
317 Cherry, On Human Communication, 52.
communication. And just as attempts to rationalize manual performance stumbled across the
problem of fatigue, the "physiological limit" to the efficiency of work, psychologists recognized
noise as the inherent limit in the performance of a human information processing system. If the
source of noise in a telecommunication apparatus is the random motion of electrons, the source
of noise in the nervous system is the random firing of neurons. Always present, these random
firings are assumed to generate a small but constantly present level of neural activity which
interferes with human information processing. The observer may miss a signal (for instance, a
flash of light in a psychophysical experiment or a dot on a radar screen in a real situation) or, on
the contrary, detect a signal where none was present; or s/he may make a mistake in estimating
In fact, the noise in man-made components and biological noise become combined. Due
to the noise in its electronic parts, radar may display a dot when nothing is present or the location
of the dot may be inaccurate. Additional noise is added by the imperfections of a display, such as
glare, inadequate contrast, or placement. Finally, neural noise may cause the radar operator to
miss the dot, to misjudge its size and, consequently, either to miss an enemy plane or to send an
intercepting weapon towards a friendly aircraft. In this way, the overall performance of a human-
machine system, such as a radar installation, is equally limited by the performance of its
A radar operator? A human-machine system? Was not I talking about information theory as a
practitioners, according to William Estes, "developed the tradition and the reputation of being
6. Human Engineering
among the purest of pure scientists" during its first decades?318 Practitioners who for decades
never left the seclusion of the university laboratory while their colleagues -- physiologists,
fatigue experts, behaviorists -- seemed to hold the knowledge necessary to maximize the
The fact that after 1950 experimental psychologists displaced their colleagues in
importance, of course, fits in with the larger shift from industrial to post-industrial society and
the new image of work: visual and mental processing of information rather than corporeal labor.
The problem with such global explanations is that they explain things too well. It is easy to
connect the interest of experimental psychologists in human information processing to the shift
Jonathan Crary writes that Fechner's equations were "a means of rendering a perceiver
manageable, predictable, productive and above all consonant with other areas of
standard in experimental psychology by the 1890s, Didier Deleule points out that "the measured
time is only the sign of the living machine's capacity for increasing its productivity."320
Both Crary and Deleule are right in seeing the social utility of experimental psychology.
It should be also noted, however, that this social utility remained unfulfilled for many decades. In
fact, the enormous amount of knowledge accumulated by psychologists did not find its
a precise map of the new occupations and new conditions of modern experience which call for
perceptual skills. During World War I, England, Germany, and France utilized experimental
psychologists to design and administer tests for aviation pilots, aeronautical, and airplane
of psychologists published papers on the readability of written text and of highway signs and on
the visibility of lights at sea.322 However, in the industrial world which conceived of the worker
as a human motor and was largely concerned with the productivity of manual rather than
perceptual labor, these studies were an exception rather than the mainstream rule.
It was World War II which finally put to use the expertise of experimental psychologists.
Why did this happen? The first textbook on applied experimental psychology (1949) opens by
For years experimental psychologists have worked diligently in academic laboratories studying
man's capacities to perceive, to work, and to learn. Only very slowly, however, have the facts and
methods which they have assembled been put to use in everyday life. A particularly glaring gap in
modern technology, both industrial and military, is the lack of human engineering -- engineering
of machines for human use and engineering of human tasks for operating machines. Motion-and-
time engineers have been at work on many of these problems, but the experimental psychologist is
also needed for his fundamental knowledge of human capacities and his methods of measuring
human performance.
The recent war put the spotlight on this gap. The war needed, and produced, many
complex machines, and it taxed the resources of both the designer and operator in making them
practical for human use. The war also brought together psychologists, physiologists, physicists,
design engineers, and motion-and-time engineers to solve some of these problems. Though much
of their work began too late to do any real good, it has continued on a rather large scale into the
piece.
Today, there are many groups busy with research on man-machine problems. They use
different names to describe the work in its various aspects: biotechnology, biomechanics,
psychoacoustics, human engineering, and systems research. Other names may be appropriate and
may appear in the future. In casting about for a title for this book, we tried to select one that would
describe the subject matter without the restrictive connotations attaching to some of the names
321 Morris Viteles, Industrial Psychology (New York: W.W. Norton & Company, Inc., 1932),
43.
322 Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental
Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1287-
1340.
mentioned above. Applied Experimental Psychology seems best to fill these requirements, because
the traditional data and subject of experimental psychology are fundamental to this field.323
Already before the war, experimental psychologists assisted in selecting military personnel for
such jobs as pilot or airplane observer by administering special aptitude tests. During the war, a
much greater number of pilots, radar operators and other similar personnel became needed. The
emphasis was shifted, therefore, from selecting personnel with particularly good perceptual and
motor skills to designing the equipment (controls, radar screens, dials, warning lights) to match
the sensory capacities of an average person.324 And it was the field of experimental psychology
that possessed the knowledge about the sensory capacities of an average, statistical person: how
visibility and acuity vary between day and night; how the ability to distinguish colors and
brightness vary with illumination or distance; what the smallest amount of light is which can be
reliably noticed; and so on.325 All this data was now utilized for designing better displays and
controls of the first modern human-machine systems such as high-speed aircrafts or radar
installations.
The development of these new human-machine systems during the war pushed human
perceptual and mental performance to the limit and this was the second reason why experimental
psychologists were called in. The performance of a human-machine system was limited by
human information capacity to process information. In the words of the authors of Applied
Experimental Psychology,
We can make a machine that will do almost anything, given enough time and enough engineers.
But man has limits to his developments, at least as far as we can see it. When we think how much
a single radar can do in a small fraction of a second, and then realize by comparison that even the
simplest form of reaction for a human being requires about a fifth of a second, we realize what we
are up against... The full potential of radar, for example, lagged far behind physical developments
323 Alphonse Chapanis, Wendell R. Garner, and Clifford T. Morgan, Applied Experimental
Psychology (New York: John Wiley & Sons, Inc., 1949), v.
324 Ibid., 8.
325 Estes, "Experimental Psychology: an Overview," 630.
because human operators could not master the complex operation of this machine system. We had
to worry about such things as a new kind of visual signal -- very small and not very bright.326
Considering that the authors described the work of time-and-motion engineers as directly leading
to applied experimental psychology, this rhetoric can be expected. Taylor was impatient with the
limitations of the body; now there was a similar impatience with the limitations of human
information processing. With Taylor, it was the question of the speed of muscular movements;
now, it became the question of reaction time: the minimum time in milliseconds required for an
always put subjects in, so to speak, boundary conditions. They measured sensory thresholds,
such as the least amount of light which can be detected. They also measured just noticeable
differences (j.n.d.), the smallest difference between two stimuli which can be detected. Finally,
they measured reaction times, the measure which became the main tool to deduce the time taken
experiments were designed, and they remained largely unchanged from the times of Weber,
Fechner, and Wundt. In a detection experiment, the task of an observer is to detect the presence
of barely visible stimuli, for instance a tiny light briefly flashed in the dark (did I see
something?). In an identification experiment, the task is to identify which of possible stimuli was
presented, for instance which of two colors (which one did I see?). In a recognition experiment,
the task is to not only detect something, but to recognize what it is, for instance: what was the
During World War II, the radar operator, the anti-aircraft gunner, the aircraft pilot found
themselves in the same situations in which nineteenth century psychologists put their
experimental subjects. The setups of psychophysical experiments became, in all details, the
the actual tasks faced by the operators of human-machine systems. Like the subject of a detection
experiment, a radar operator scans the radar screen for a barely noticeable dot of light.327 Like
the subject of an identification experiment, he has to try to guess whether this dot is the same or
different from another dot which from his previous experience he knows to correspond to a
identify a plane by its shape. And all of them, especially the pilot, are engaged in a sort of
Thus, nineteenth century psychophysical setups became the military, and soon, civilian
workplaces of post-industrial society; from there, they traveled back into laboratories, leading to
such close interrelations between basic research in experimental psychology and its practical
applications that they were no longer separable.328 The terms "applied experimental
term standard today -- "human factors." The radar operator who in the 1940s and 1950s was the
prototypical figure, the computer user. Thus, references to "human-machine systems" became
references to "human-computer systems." The same amount of intellectual energy and research
which in the middle of the century went into theorizing the performance of a radar operator and
adapting him and radar display to each other, today goes into the work on computer interfaces. In
327 As Paul Fitts notes in his 1951 overview of engineering psychology, "radar operators are
often forced to search for weak signals at near-threshold levels." Fitts, "Engineering Psychology
and Equipment Design," 1290.
328 For example, a 1947 article in American Psychologist describes the work of Naval Research
Laboratory as following these three directions: "the design of gun fire control and missile control
instruments from the point of view of ease and efficiency of operation; the design and evaluation
of synthetic gunnery and missile control trainers; and basic psychological research." But what is
meant here by "basic research"? We read that "at present, all basic research studies are aimed at
the eye-hand coordination problem involved in target tracking." "Target tracking" is just one
example of a military task which traveled into a psychological laboratory, and gradually become
a standard psychophysical experiment. Franklin Taylor, "Psychology at the Naval Research
Laboratory," American Psychologist 2, no. 3 (1947): 87, 91.
retrospect then, we should recognize the radar operator as the central figure standing at the
origins of post-industrial society, the figure which put directly into motion the new disciplines of
the efficiency of the mind: engineering psychology, human information processing, and
cognitive science.
We are now in a position to see that the ready acceptance of information theory by
psychologists in the 1950s was not simply a part of a "Zeitgeist" of the coming information age.
Rather it was a practical necessity. The 1986 Handbook of Perception and Human Performance
describes the rationale for adopting the information processing approach in psychology in this
way:
Because the language of information processing provides an objective and quantitative way of
describing the basis of human performance, it has proven useful in applications. Indeed, much of
the impetus for the development of this kind of empirical study stemmed from the desire to
integrate description of the human within the overall systems. In the design of cockpits, control
systems for weapons, and other military applications, it has been useful to have descriptions of the
speed, accuracy, and reliability of the human in terms comparable to those used for other
mechanical parts of the system.329
to characterize human performance quantitatively. But not just any language. If the human is
conceived of as part of a human-machine system (for instance, in the role of a radar operator), it
is convenient to use the same language to describe the performance, capacities, and errors of
both the electronic and human components. Put differently, if the human-machine system is
same terms need to be applied to its human components if the performance of the overall
component as well.
Modernization brought with it a special discipline concerned with efficiency -- engineering. The
job of an engineer was to ensure maximum performance with a minimum investment of energy,
systems (communication engineering) or human bodies (scientific management, time and motion
studies). Inspired by modern engineering, the avant-garde of the 1920s tried to systematically
To engineer vision meant to eliminate waste, to use minimal material resources. Thus,
of block letters consisting of straight lines; it also eliminated illustrations and "wasteful"
decorations by making type itself the main element of design. The goal: maximum impact with
To engineer vision also meant to minimize the psycho-physical resources required of the viewer.
Vertov writes in his famous 1923 manifesto: "The least advantageous, the least economical
communication of a scene is theatrical communication."330 In contrast, montage forces the eye to see the
right thing at the right time, thus eliminating the visual waste of theater, ballet, painting, and other
traditional forms. In montage, "camera drags the eyes of a film viewer from hands to legs, from legs to
well, the avant-garde promoted montage as an example of possible economy, in this case
economy of time. Maud Lavin describes the 1930 manifesto of the group of leading German
designers headed by Kurt Schwitters: "Walter Dexel writes that modern man has the right to
expect communications in the shortest possible time. Willi Baumeister points out that
photomontage is efficient, allowing for the quick grasp of several images at once...Similarities
between photomontage and film are often emphasized, with photomontage being considered a
Finally, to engineer vision also meant to be able to measure its efficiency, or, to use the language
school, invented his first theory of artistic communication, the famous "montage of attractions":
"...Laboratory analyses and diagrams...Mendeleev's Periodic Table and the laws of Gay-Lussac and Boyle
Mariotte in the realm of art!...Let us therefore search for the unit which will measure the influence exerted
by art! Science has its "ions," its "electrons," its neutrons." Art will have -- attractions!"333
In its desire to engineer vision, the avant-garde was ahead of its time. The systematic engineering
of vision took place only after World War II when vision become the major instrument of labor,
the most productive organ of a worker in a human-machine system. To ensure the maximum
performance of such a system, it became necessary to engineer it around the capacities and the
limitations of human vision. It also became necessary to understand vision in a new way: as
information processing.
332 Maud Lavin, "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of
New Advertising Designers," in Montage and Modern Life: 1919-1942, ed. Matthew Teitelbaum
(Cambridge: The MIT Press, 1992), 54.
333 Qtd. in Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and
Indiana University Press, 1987), 41. Emphasis mine -- L.M.
This is the history of the engineering of vision from INKhUK to MIT: from the Soviet art
institute of the 1920s which gathered the avant-garde artists united by the idea of engineering
vision -- to the premier engineering school where much of this engineering later occurred
through the work on information theory, on radar and computer graphics, on new human-
early 1920s. It ends with the recent research by cognitive scientists and engineers at MIT. What
justifies this temporal continuity, on the one hand, and disciplinary shift, on the other hand --
from the Soviet artistic avant-garde to one of the premier science institutions in the world, which
interfaces? What can be in common between Soviet constructivism and American cognitive
science?
Both are the product of the age of engineering. Engineering, the quintessentially modern
profession, appeared in the 1870s. The job of an engineer was to ensure maximum performance -
materials, and time. The engineer was the new specialist in the efficiency of any productive
process, regardless of its nature. And this ideology of "engineerism" became the modern myth,
the religion of a society aspiring to total rationalization.334 In order to employ vision efficiently
in its new roles, whether as a medium of propaganda and advertising or as the channel of
communication between human and machine, artists, designers, and scientists adopted the
engineering approach as well. It is not surprising that Soviet constructivists, the pioneers of
modern propaganda design, insisted on calling themselves engineers rather than artists.
It is also not surprising that the archeology of twentieth century vision would uncover
numerous similar solutions to the problem of its rationalization in such seemingly distant fields
as film and robotics, graphic design and experimental psychology. For instance, let us compare
two statements: one by a designer, the other by a psychologist. El Lissitsky wrote in the 1920s
regarding his new geometric style of poster design: "The most unambiguous and immediately
334 Max Weber proposed rationalization as an umbrella concept to describe the social processes
of modernity. See, for instance, From Max Weber: Essays in Sociology (New York: Oxford
University Press, 1958).
Conclusion
Max Weber, General Economic History (New York, 1927).
recognizable forms are geometric forms. No one will confuse a rectangle with a circle, or a circle
with a triangle."335 In 1951, the already mentioned experimental psychologist Paul Fitts echoed
practically the same words in his summary of research on the use of graphic symbols for human-
machine interfaces: "Geometrical figures differ considerably in legibility. Straight lines have
been found to be more legible than curved ones. The triangle, the rectangle, and the square have
been reported to be more easily recognized under conditions of low illumination and in
peripheral vision than circular and hexagonal forms."336 Both Lissitsky and Fitts were concerned
with the same problem -- reliability and speed in the recognition of visual symbols -- and,
professionals have understood vision as a medium of communication and have attempted to put
its use on a scientific basis. If in the early part of the century this research focused on human
communication (the human as the subject of mass propaganda and mass entertainment), after
World War II, the focus shifted to human-machine interface. Human vision became the key
machine.
This history can be summarized by three images. The first image: a portrait of Tatlin by
Lissitsky (fig. 16). A compass, extending straight from Tatlin's eye, a metaphor of vision for
work.
The second image, a popular icon in the late 1980s and early 1990s, promoted virtual
reality interface designed at NASA/Ames Human Factors Research Center (fig. 22).337 Instead
335 Qtd. in Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 24.
336 Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental
Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1298.
337 On NASA/Ames virtual reality research in the 1980s, see Scott S. Fisher, "Virtual Interface
Environments," in The Art of Human-Computer Interface Design, ed. Brenda Laurel (Reading,
Mass.: Addison-Wesley Publishing Company, 1990): 423-438.
of the metaphor of the eye-compass, a reality: video monitors strapped to the eyes. The notion of
vision as work is now fully realized: the operator wearing the gear works by mentaly processing
visually presented information. The gear is designed using all the available knowledge
accumulated by experimental psychology about human vision. In the photograph we see the last
leftover from the age of manual labor -- an arm in a DataGlove. It will soon disappear since
through gaze tracking the operator can control the system by merely looking at different points in
virtual space.
What is the third image which stands between these two? Let me quote from the
description of the history of engineering psychology found in an 1965 overview of the field:
The primary emphasis in time-and-motion engineering has been on man as a worker; that is, as a
source of mechanical power. It was not until World War II that a new category of machines
appeared -- machines that made demands not upon the operator's muscular power, but upon his
sensory, perceptual, judgmental, and decision-making abilities. The job of a radar operator, for
example, requires virtually no muscular effort, but makes severe demands on sensory capacity,
vigilance, and decision-making ability. This new class of machines raised some intricate and
unusual questions about human abilities: How much information can a man absorb from a radar
screen?338
The World War II radar operator is this third image, a historical intermediary between Lissitsky's
man-compass (1920s) and NASA's virtual reality user (1980s). The figure of a radar operator
stands at the gates to the post-industrial society of perceptual labor, at the origins of research on
human-machine interface, and the understanding of human vision as information processing (fig.
23).
Anderson, John Robert. Cognitive Psychology and Its Implications. W.H. Freeman and
Company, 1980.
ArgŸelles, JosŽ. Charles Henry and the Formation of a Psychophysical Aesthetics. Chicago:
University of Chicago Press, 1972.
Arnheim, Rudolf. "A Plea for Visual Thinking." In New Essays on the Psychology of Art. Los
Angeles and London: University of California Press, 1986.
Arnheim, Rudolf. Art and Visual Perception. The New Version. Berkeley: University of
California Press, 1974.
Aumont, Jacques. Montage Eisenstein. London and Bloomington: BFI Publishing and Indiana
University Press, 1987.
Barthes, Roland. "Rhetoric of the Image." In Image, Music, Text, edited by Stephen Heath. New
York: Hill and Wang, 1977.
Barthes, Roland. "The Photographic Message." In Image, Music, Text, edited by Stephen Heath.
New York: Hill and Wang, 1977.
Barthes, Roland. "The Third Meaning." In Image, Music, Text, edited by Stephen Heath. New
York: Hill and Wang, 1977.
Baxandall, Michael. Painting and Experience in Fifteenth Century Italy. Oxford: Oxford
University Press, 1972.
Bergin, Thomas Goddard and Max Harold Fisch. The New Science of Giambattista Vico. Ithaca:
Cornell University Press, 1968.
Berlyne, D.E. "Novelty, Complexity, and Hedonic Value." Perception and Psychophysics 8
(1970): 279-86.
Braun, Marta. Picturing Time: the Work of Etienne-Jules Marey (1830-1904). Chicago: The
University of Chicago Press, 1992.
Bibliography
Brik, Osip. "T.n. 'formalnyi metod'" (So called 'formal method'). LEF 1 (1923): 214-215.
Bryson, Norman. Looking at the Overlooked: Four Essays on Still Life Painting. Cambridge:
Harvard University Press, 1990.
Buck-Morss, Susan. The Dialectics of Seeing: Walter Benjamin and the Arcades Project.
Cambridge: The MIT Press, 1989.
Chen, Jack. Soviet Art and Artists. London: The Pilot Press, Ltd., 1945.
Cherry, Colin. On Human Communication. 2nd ed. Cambridge: The MIT Press, 1968.
Cohen, Gillian. The Psychology of Cognition. 2nd ed. (London and New York: Academic Press,
1983).
Cohen, Paul R. and Edward A. Feigenbaum, eds. The Handbook of Artificial Intelligence. Los
Atlos, CA: William Kaufmann, Inc., 1982.
Cook, Richard. "Serious Entertainment." Computer Graphics World (May 1992): 40-48.
Crary, Jonathan. Techniques of the Observer: on Vision and Modernity in the Nineteenth
Century. Cambridge: The MIT Press, 1990.
De Fleur, Melvin L. and Sandra Ball-Rokeach. Theories of Mass Communication. New York:
Longman, 1975.
De Landa, Manuel. War in the Age of Intelligent Machines. New York: Zone Books, 1991.
Descartes, RenŽ. "Meditations on the First Philosophy," In The Rationalists. Garden City, NY:
Doubleday & Company, 1960.
Dudley, Andrew. Concepts in Film Theory. Oxford: Oxford University Press, 1984.
Dulac, Germaine. "Aesthetics, Obstacles, Integral CinŽgraphie." In vol. 1, French Film Theory
and Criticism, edited by Richard Abel. Princeton: University of Princeton Press, 1988.
Eames, Charles and Ray Eames. A Computer Perspective: Background to the Computer Age.
Cambridge: Harvard University Press, 1990.
Edgerton, Samuel. The Renaissance Rediscovery of Linear Perspective. New York: Basic Books,
1975.
Eisenstein, Sergei. "A Dialectical Approach to Film Form." In Film Form: Essays in Film
Theory, edited by Jay Leyda. New York: Harcourt Brace and World, 1949.
Epstein, Jean. "On Certain Characteristics of PhotogŽnie." In vol. 1, French Film Theory and
Criticism, edited by Richard Abel. Princeton: University of Princeton Press, 1988).
Farah, Martha. "Is Visual Imagery Really Visual? Overlooked Evidence from
Neuropsychology." Psychological Review 95, no. 3 (1988): 307-317.
Farocki, Harun. "Reality Would Have to Begin." Documents 1/2 (1992): 136-146.
Finke, Ronald A. Principles of Mental Imagery. Cambridge: The MIT Press, 1989.
Flew, Antony, ed. A Dictionary of Philosophy. London: Pan Books Ltd., 1984.
Foster, Hal, ed., Vision and Visuality. Seattle: Bay Press, 1988.
Freud, Sigmund. The Interpretation of Dreams. New York: Avon Books, 1965.
Freud, Sigmund. Standard Edition of the Complete Psychological Works. London: Hogarth
Press, 1953.
Friedhoff, Richard Mark and William Benson. The Second Computer Revolution: Visualization.
W.H. Freeman and Company, 1991.
Gardner, Martin. Logical Machines and Diagrams. 2nd ed. Chicago: The University of Chicago
Press, 1982.
Gombrich, E.H. "Pictorial Instructions." In Images and Understanding, ed. Barlow Horace, Colin
Blakemore and Miranda Weston-Smith. Cambridge: Cambridge University Press, 1990.
Gombrich, E.H. Art and Illusion. Princeton: Princeton University Press, 1960.
Hall, Stuart. "Encoding/Decoding." In Culture, Media, Language, ed. Stuart Hall et al.
Hatchinson, 1980.
Hansen, Miriam. "The Hieroglyph and the Whore: D.W. Griffith's Intolerance." The South
Atlantic Quarterly 88, no. 2 (1989): 361-392.
Hartley, R.V.L. "Transmission of Information." Bell System Tech. Journal 7 (1928): 535-563.
Hearst, Eliot. "One Hundred Years: Themes and Perspectives." In The First Century of
Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates,
1979.
Heidegger, Martin. What is Called Thinking? New York: Harper & Row, Publishers, 1968.
Hochberg, Julian. "Sensation and Perception." In The First Century of Experimental Psychology,
edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, 1979.
Holly, Michael Ann. Panofsky and the Foundations of Art History. Ithaca: Cornell University
Press, 1984.
Holmes, Oliver Wendel. "The Stereoscope and the Stereograph." In Photography, Essays and
Images: Illustrated Readings in the History of Photography, edited by Beaumont Newhall. New
York: Museum of Modern Art, 1980.
Hoover, Theodore and Charles Fish. The Engineering Profession. Stanford: Stanford University
Press, 1941.
"Institut Khudozestvennoy Kultury" (The Institute of Artistic Culture). Russkoe Iskusstvo 2-3
(1923): 85-88.
Ivins, William M. On the Rationalization of Sight. New York: Da Capo Press, Inc., 1975.
Jay, Martin. Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought.
Berkeley: The University of California Press, 1993.
Jay, Martin. "Scopic Regimes of Modernity." In Vision and Visuality, ed. Hal Foster. Seattle:
Bay Press, 1988.
Joravsky, David. Russian Psychology: A Critical History. Cambridge, Mass.: Basil Blackwell,
1989.
Kandinsky, Wassily. Point and Line to Plane. New York: Solomon R. Guggenheim Foundation,
1947.
Keller, Evelyn Fox and Christine R. Grontkowski. "The Mind's Eye." In Discovering Reality:
Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy of
Science, edited by Sandra Harding and Merrill B. Hintikka. London: D. Reidel Publishing
Company.
Kemp, Martin. The Science of Art. New Haven: Yale University Press, 1990.
Kenez, Peter. The Birth of the Propaganda State. Soviet Methods of Mass Mobilization, 1917-
1929. Cambridge: Cambridge University Press, 1985.
Khan-Magomedov, S.O. "INKhUK: vozniknovenie, formirovanie i pervyy period raboty. 1920"
(INKhUK: appearance, formation and first period of its work, 1920). Sovetskoe Iskusstvoznanie
(1981): 332-368.
Koerner, Joseph Leo. "The Shock of the View." The New Republic (April 26, 1993): 32-38.
Kogan, P.C. "O zadachakh akademii i yeyo zhurnala" (About the goals of the academy and its
journal). Iskusstvo 1, no. 1 (1923).
Kovasznay, L.S.G. and H.M. Joseph. "Image Processing." Proceedings of IRE 43 (1955): 560-
570.
Krauss, Rosalind. The Optical Unconscious. Cambridge: The MIT Press, 1993.
Lacan, Jacques. "On the Gaze as Objet Petit a." In The Four Fundamental Concepts of Psycho-
Analysis, edited by Jacques-Alain Miller. New York: W.W. Norton & Company, 1981.
Latour, Bruno. "Visualization and Cognition: Thinking with Eyes and Hands." Knowledge and
Society: Studies in the Sociology of Culture Past and Present 6 (1986): 1-40.
Lavin, Maud. "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of New
Advertising Designers." In Montage and Modern Life: 1919-1942, edited by Matthew
Teitelbaum. Cambridge: The MIT Press, 1992.
LŽvy-Bruhl, Lucien. Les functions mentales dans les sociŽtŽs primitives. Paris: 1910.
Lotman, Jurij. The Structure of the Artistic Text. Ann Arbour: Department of Slavic Languages
and Literatures, The University of Michigan, 1977.
Mandler, Jean Matter and George Mandler, eds. Thinking: From Association to Gestalt. New
York: John Wiley & Sons, Inc., 1964.
Manovich, Lev. "Assembling Reality: Myths of Computer Graphics." Afterimage 20, no. 2
(September 1992): 12-14.
Manovich, Lev. "'Real' Wars: Esthetics and Professionalism in Computer Animation." Design
Issues 6, no. 1 (Fall 1991): 18-25.
Marr, David. Vision. New York: W.H. Freeman and Company, 1982.
Matsa, I., ed. Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia (Fifteen years of Soviet
art: materials and documentation). Moscow, IZOGIZ: 1933.
McArthur, David. "Computer Vision and Perceptual Psychology." Psychological Bulletin 92, no.
2 (1982): 283-309.
McFarlane, M. D. "Digital Pictures Fifty Years Ago." Proc. IEEE 60, no. 7 (1972).
Michelson, Annette. "Reading Eisenstein Reading 'Capital'." October 2 (1976): 27-38; October 3
(1977): 82-88.
Miller, G.A. Language and Communication. New York: McGraw-Hill Book Co., Inc., 1951.
Mitchell, J. William. The Reconfigured Eye: Visual Truth in the Post-Photographic Era.
Cambridge, The MIT Press, 1992.
Mitchell, W.J.T. Iconology. Image, Text, Ideology. Chicago: The University of Chicago Press,
1985.
L‡szl— Moholy-Nagy. Painting, Photography, Film. 1925. Reprint. Cambridge: The MIT Press,
1973.
Molok, Yu. A. "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel Florensky's
'dictionary of symbols.' A few margins). Sovetskoe Iskusstvoznanie 26 (1990): 322-343.
MŸnsterberg, Hugo. The Photoplay: A Psychological Study. New York: D. Aplleton & Co.,
1916.
Newhall, Beaumont. Airborne Camera. New York: Hastings House, Publishers, 1969.
Nietzsche, Friedrich. "On Truth and Falsity in their Ultramoral Sense." In Early Greek
Philosophy and Other Essays. New York: The Macmillan Company, 1924.
Noble, Douglas. "Mental Materiel: The Militarization of Learning and Intelligence in U.S.
Education." In Cyborg Worlds: the Military Information Society, edited by Les Levidov and
Kevin Robins. London: Free Association Books, 1989.
Nšth, Winfried. Handbook of Semiotics. Bloomington and Indianapolis: Indiana University
Press, 1990.
Panofsky, Erwin. Perspective as Symbolic Form. New York: Zone Books, 1991.
Peat, David. Artificial Intelligence: How Machines Think. New York: Baen Enterprises, 1985.
Posner, Michael I. and Gordon L. Shulman. "Cognitive Science." In The First Century of
Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979.
Quastler, H., ed. Information Theory in Psychology: Problems and Methods. Glencoe, IL.: Free
Press, 1955.
Rabinbach, Anson. The Human Motor: Energy, Fatigue, and the Origins of Modernity. Basic
Books, Inc., 1990.
Rappoport, A.G. "El Lissitsky i ego 'Pun-geometiya'" (El Lissitsky and his 'Pan-geometry').
Sovetskoe Iskusstvoznanie 25 (1989): 113-130.
Reichardt, Jasia. The Computer in Art. London and New York: Studio Vista and Van Nostrand
Reinhold Company, 1971.
Resnikov, Howard. The Illusion of Reality. New York: Springer-Verlag New York Inc., 1989.
Reveaux, Tony. "Virtual Reality Gets Real." New Media (January 1993): 36-41.
Rheingold, Howard. Virtual Reality. New York: Simon & Schuster, Inc., 1991.
Riggs, L.A. et al. "The Disappearance of Steadily Fixated Visual Tests Objects." J. Optical Soc.
America 43 (1953): 495.
Roberts, L.G. "Machine perception of three-dimensional solids." In Optical and Electo Optical
Information Processing, edited by J.T. Tippett. Cambridge: The MIT Press, 1965.
Sampson, Geoffrey. Schools of Linguistics. Stanford, CA: Stanford University Press, 1980.
Sardarov, A.S. "5000 let evolutsii doroznogo znaka" (5000 years of the evolution of a traffic
sign). Tekhnicheskaya Estetika 9 (1984): 14-19.
Schramm, Wilbur. "How Communication Works." In The Process and Effects of Mass
Communication. Urbana: University of Illinois Press: 1954.
Sekula, Allan. "The Instrumental Image: Steichen at War." In Photography against the Grain:
Essays and Photo Works, 1973-1983. Halifax: The Press of the Nova Scotia College of Art and
Design, 1984.
Sekula, Allan. "The Body and the Archive." October 39 (1987): 3-64.
Shannon, Claude E. and Warren Weaver. The Mathematical Theory of Communication. Urbana:
The University of Illinois Press, 1949.
SIGGRAPH '89 Panel Proceedings. New York: The Association for Computing Machinery,
1989.
SIGGRAPH '92 Final Program. New York: The Association for Computing Machinery, 1992.
Sonesson, Gšran. Pictorial Concepts. Inquiries into the Semiotic Heritage and its Relevance for
the Analysis of the Visual World. Lund, Sweden: Lund University Press, 1989.
Spriegel, William R. and Clark E. Myers, eds. The Writings of the Gilbreths. Homewood, IL.,
1953.
Stelzer, Otto. "Moholy-Nagy and His Vision." In L‡szl— Moholy-Nagy, Painting, Photography,
Film. 1925. Reprint, Cambridge: The MIT Press, 1973.
Taylor, Brandon. Art and Literature under the Bolsheviks. London: Pluto Press, 1991.
Taylor, Franklin. "Psychology at the Naval Research Laboratory." American Psychologist 2, no.
3 (1947): 87-92.
Taylor, Frederick Winslow. The Principles of Scientific Managment. New York, 1967.
Titchener, Edward Bradford. A Beginner's Psychology. New York: The Macmillan Company,
1915.
Todorov, Tzvetan. Theories of the Symbol. Ithaca: Cornell University Press, 1984.
Tretyakov, Sergei. "Otkuda i kuda? (Perspektivu futurizma)" [Where from and where to?
(Perspectives of futurism)]. LEF 1 (1923): 192-203.
Virilio, Paul. War and Cinema: the Logistics of Perception. London: Verso, 1989.
Viteles, Morris. Industrial Psychology. New York: W.W. Norton & Company, Inc., 1932.
Vitz, P.C. "Preference for Different Amounts of Visual Complexity." Behavioral Science 2
(1966): 105-14.
Vitz, Paul C. and Arnold B. Glimcher. Modern Art and Modern Science. The Parallel Analysis
of Vision. New York: Praeger, 1984.
Weber, Max. From Max Weber: Essays in Sociology. New York: Oxford University Press, 1958.
Weber, Samuel. The Legend of Freud. Minneapolis: University of Minnesota Press, 1982.
Wiener, Norbert. The Human Use of Human Beings. Cybernetics and Society. New York: Avon
Books, 1967.
Wiener, Norbert. Cybernetics. Of Control and Communication in the Animal and the Machine.
New York: John Wiley & Sons, Inc., 1948.
FIGURES