Interactive art - a sensorial aesthetic

STUDENT NAME: Benjamin Low Teck Hui

An essay submitted in partial fulfillment of the requirements for the Diploma of Media Arts

(Interactive Art)

Date of submission: 26 Nov 2010

LASALLE College of the Arts

© Benjamin Low Teck Hui

Introduction

This paper posits that interactive art is sensory in nature in that it invokes and engages

the sensorium, and that through this richness of sensory experience, the socio-cultural context

of the artwork can be effectively communicated. This position is based on critical theories of

how phenomenological perception is tied to consciousness and experience.

2
A unique art form

Unlike almost any other art forms, interactive art has a non-linear narrativei. This

means that the audience can observe, explore and modify the narrative of the art work. The

non-linearity comes from the fact that there are no fixed outcomes since the audience’s

experience of the art work depends on his or her actions. While interactivity has existed in

other art forms, such as some forms of ancient Greek theatre, contemporary forum theatre and

Marcel Duchamp’s mobiles in the 1920s, it is only in recent decades, with the advancement

of digital and media technology, that the genre of interactive art has arrived in its own right

as a mainstream art form, and is rapidly evolving through the connectivity of the World Wide

Web.

Interactive art, with its accruements of technology, creates open possibilities on the

form in which the artwork takes. It can take the form of wearable art, be incorporated into

theatre technical design for the performing arts, exist purely on a virtual screen, as an

installation in a gallery space or incorporated into the architecture of a public space.

For example, the installation works of David Rokeby, a pioneer in this field, often

incorporate sophisticated imaging, sensor and mechanical components that react and respond

to human touch and presence. His work, Very Nervous System (1982-1990), which uses body

movement to create sound, has undergone three successive variations, each time using better

technology.

3
 Fig 1. Very Nervous System. David Rokeby. 1990

Hence, interactive art, as defined in this paper, is art that offers a non-linear narrative,

is independent of form for its presentation and engages the space between technology and art

making. For these reasons, the experience of interactive art occupies a unique position among

other art forms.

Consciousness and experience

The phenomenal world is experienced as an ensemble of sensations – the texture of

our clothing, the background noise of traffic, the warmth or coolness of the air, our visual

surroundings and so on. In Phenomenology of Perception (1945), Maurice Merleau-Ponty

unveils the phenomenology of perception, a thesis he calls “the primacy of perception”. We

are embodied subjects, we perceive phenomena through our bodies first before we can reflect

or philosophise them via this mediation which is instantaneous and synonymous with our

being and perception in, as, and with body, i.e. embodiment1.

Our bodily sensations of the phenonemal world, combined with our thoughts,

emotions and mental impressions co-interacting and co-referencing one another in

simultaneity, constitute what Bary Dainton calls co-consciousnessii. In co-consciousness, our

senses do not operate in isolation. The senses are all engaged at once and operate in

reciprocal co-dependence - taste is mediated by touch and smell, a human presence is

experienced via bodily warmth, sight as well as speech. Co-consciousness describes the state

of our consciousness at any one point of time. Over time, these states form a stream of

consciousnessiii, which consitute our common notion of what experience is.

If co-consciousness describes how we experience the world, then co-consciousness

also describes how we experience art. How we experience interactive art - with its non-linear
1
Merleau-Ponty, Maurice. Trans: Colin Smith. Phenomenology of Perception. London: Routledge, 2005.
4
narrative, two-way exchange of meaning, exploration and open outcomes, and the way in

which it engages the sensorium (to be elaborated) - closely mirrors how we experience the

real world, that is, in a non-linear co-conscious sort of way.

A sensorial aesthetic

The word ‘aesthetic’ comes from the ancient Greek word for ‘sense perception’ or ‘I

feel’. Aesthetics is related to the idea of beauty, that is, an object of beauty appeals to the

senses and causes the viewer to arrive at a perception of beauty. Interactive art appeals to the

sensoriumiv. It moves away from the emphasis on the aesthetic of form towards the aesthetic

of experience. The non-linear co-conscious experience of interactive art that engages the

sensorium is what I call the sensorial aesthetic.

The sound artist Stephen Vitiello often features ambient sounds in his installations,

such as creaking noises in buildings, clicking frogs, falling rain or buzzing insects. A Bell For

Every Minute2, installed in a park, features recorded bells all over New York City and

beyond. Through the sounds of the bellsv, the listener experiences New York City aurally as

the sounds have corresponding locations on a mapped journey. Consciously or not, the

listener would form mental images of these locations which may then inspire various

thoughts and feelings, such as nostalgia, curiosity, puzzlement, awe, annoyance and so on.

2
14th Street Passage, between West 13th Street and West 14th Street, New York. June 2010. Presented in
partnership with Creative Time, and the New York City Department of Parks & Recreation.
5
 Fig 2. A Bell For Every Minute. Stephen Vitiello. 2010.

Lovers Leap3, by Miroslaw Rogala, involves two screens on opposite sides of a room,

showing altered perspective scenes of a busy downtown street at Michigan Avenue, Chicago,

that shifts and warps with the participants’ movements in the room. However, if the person

stays still for a while, he will experience a “leap” into a normal random animated scene of the

same street, a beach or of daily life in Jamaica. The participant thus interacts kinaesthetically

with the artwork which engages his spatial awareness of the room. The speed of his

movement as well as his position in the room are factors that modify the artwork. The

participant would be analysing in the manner in which the artwork responds to him and his

train of thought emotions would probably be hijacked as he experiences the “leap”, that is, if

he stays still long enough.

3
Interactive installation environment. ZKM Centre for Media Art, Karlsruhe, Germany. 1995.
6
 Fig 3. Lover’s Leap. Miroslaw Rogala. 1995.

Interactive artwork can also offer a sensory experience akin to synaesthesia. Sounds

invoke a sense of space in the Music for Sound Joined Rooms4 series by Maryanne Amacher,

sonic theatre in large expansive structures, in which “architecture magnifies the sensorial

presence of experience - rooms, walls and corridors that sing5.” Sound sculpture invoking a

corporeal presence of an organism’s internal organs can be found in Uberorgan6 by Tim

Hawkinson, a massive musical instrument installed in several rooms that visually resembles

the chest cavity and internal organs of a large living organism.

In a kind of haptic visualityvi in The Tijuana Projection, Krzysztof Wodiczko

projected the faces of women working in the maquiladoras7 onto the dome of the El Centro

Cultural as they spoke emotionally of incest, police abuse, and work place discrimination.

The audience was immersed within the intimate territory of another person’s personal and

emotional space, their senses enveloped by the decorporealised presence of the narrator. They

4
Installation and performances. Bern, Switzerland. 1998
5
Lovejoy, Margot. Digital Currents: art in the electronic age, p205. Routledge, 2004.
6
Sound installation. Ace Gallery of Los Angeles. 2000
7
Assembly plants in Mexico near the border with the United States.
7
sensually feel the other person through empathy in this field of haptic vision, such that their

tactile perception is triggered by the visual representation of the narrator’s face.

Fig 4. The Tijuana Projection. Krzysztof Wodiczko. 2001.

Telemetic dreaming8 explores the implications of user interaction with a projected

virtual presence of a real woman on a bed, with profound results that show that the emotional

responses of the participants as well as the woman are just as real as if she were there

physically, as testified by the performer9. This shows that physical touch does not have to be

present as part of the sensorial aesthetic for it to work. The sensorial aesthetic works based on

the principle of stream of consciousness, whereby the all senses work co-dependently and in

non-linear simultaneity. The resulting amalgamation of sensations, even without physical

touch, is enough for the reflective part of co-consciousness (thought, emotion and mental

image) to form the perception of touch.

8
Commissioned for the annual summer exhibition curated by the Finnish Ministry of Culture in Kajaani, with
support from Telecom Finland, in June 1992.
9
Kozel,Susan. Spacemaking: Experiences of a Virtual Body. Dance Theatre Journal, vol 11 no 3. Autumn 1994.
8
 Fig 5. Telemetic Dreaming. Paul Sermon. 1992.

These examples reflect that interactive artworks are rich in sensations, which

combined with the interactive narrative, offer a non-linear co-conscious kind of experience -

the sensorial aesthetic. The sensorial aesthetic is able to influence our sensation of space, as

in artworks that invoke a sense of space, as well as our sense of touch via a decorporealised

human presence.

From sensation to conception

As mentioned in the earlier discussion on experience and consicousness, human

beings are symbolical creatures and attach meaning and interpretations to the embodied

knowledge attained through their senses. Our sensations are mediated by feeling and thought

for us to arrive at a provisional kind of knowledge - a notion of experience, an idea or a

concept. “The given cannot be known in itself. What can be known is a construct of

experience, a creation of feeling and thought10.”

Over time, a baby learns through touch, sight and hearing via his interaction with his

parents, that these particular objects in his world of fleeting impressions are associated with

feelings of warmth and comfort and notions of safety and providence. Yi Fu Tuan describes

how people arrive at a concept of an object or place via sensation, thought and emotion - “An
10
Yi Fu Tuan. Space and Place – The Perspective of Experience, p9. U of Minnesota Press. 2001
9
object or place achieves concrete reality when our experience of it is total, that is, through all

the senses as well as with the active and reflective mind.”

Coming back to the examples shown earlier, Very Nervous System prompts the

participant to explore the relationship between man and machine, by engaging the body in a

joyful and intimate sound-making experience with the computer in a human-scaled space. A

Bell for Every Minute invites the listener to relate to the urban landscape through the sounds

captured in that environment, and reflect on the site’s connection to the city around it.

Lover’s Leap provides a sensory experience whose context is to explore the notions of

rationality and control over technology. The Tijuana Projection raises the social awareness of

the plight of a marginalized group of people within the local community through the haptic

visuality of its narrator. Telemetic Dreaming informs us that a virtual human presence is

enough to invoke a sense of touch and a real emotional response, hinting at the socio-political

implications of virtual reality.

Art, with its utopian ideals, and functioning as a cultural barometer of sorts, aims to

communicate and effect meaningful socio-cultural change. Interactive art, using new media

technology, can create powerful experiences for the purpose of education and raising social

awareness. An example of such is the Italian artist collective known as Molleindustria, whose

mission is to reappropriate the medium of electronic games for educational, political and

social change. The McDonald’s videogame11 is one of their projects to expose the political,

environmental and social implications of running a worldwide fast food chain.

The sensorial aesthetic, together with the specific socio-cultural context of the

interactive artwork, guides the process from sensation to conception. Due to the open

structure of the interactive narrative, it can be difficult to control or predict the outcome of

11
Flash game downloadable at www.mcvideogame.com
10
the artwork. What is more certain is that the aura12 of the artwork, together with the sensorial

aesthetic, engages the audience with a unique kind of experience.

12
an artwork’s presence in time and space, that gives it its unique identity, a term coined by the media theorist
Sir Walter Benjamin.
11
Summary

The sensorium cannot be denied as part of our stream of consciousness. Interactive

art, with its non-linear narrative and richness of sensory forms, has a sensorial aesthetic that

mirrors the nature of our daily experiences.

The sensorial aesthetic, together with the socio-cultural context of the artwork,

engages our sensorium for a “true-to-life” experience, a simulacrum of reality that provokes

and engages our perception of reality.

Conclusion

Interactive art is uniquely positioned as an art form for socio-cultural communication through

the sensorial aesthetic. Artists could do well to consider this genre in their practice.

12
i
Endnotes
E

An interactive narrative is a time-based representation of character and action in which a reader can affect, choose, or,
change the plot. The first-, second-, or third-person characters may actually be the reader. Opinion and perspective are
inherent. Image is not necessary, but likely. (Mark Stephen Meadows, Pause and Effect – the art of interactive
narrative, p62, New Riders, 2002)
ii
As you start to manipulate the object (to figure out what object is given to you while you are blindfolded and seated on
a chair, as part of a game) you have tactile sensations in your hands and fingers. These do not occur by themselves, but
are continuous with the rest of your bodily experience (e.g. your body-image:sitting hunched in a chair). You are also
having some thoughts – ‘What is this damned thing? – emotional feelings (mounting frustration), and mental images
(you are trying to find an image to fit the feel). These thoughts and images do not occur in isolation from one another,
they are experienced together – they are co-conscious – both with one another (thought + emotional feeling + mental
image) and your various bodily experiences. (Barry Dainton, Stream of Consciousness – Unity and Continuity in
Conscious Experience, Introduction p3, Taylor & Francis, 2006)
iii
As examples of items possessing phenomenal character I have referred to particular experiences, but experiences do
not typically occur in isolation from one another. A stream of consciousness is an ensemble of experiences that is
unified both at and over time, both synchronically and diachronically. (Barry Dainton, Stream of Consciousness – Unity
and Continuity in Conscious Experience, Introduction p2, Taylor & Francis, 2006)
iv
Sensorium is the “resulting set of experiences ... from the visual, auditory, olfactory, and the tactile ... as sites of
embodied knowledge ... and is the subject’s way of coordinating all of the body’s perceptual and proprioceptive signals
as well as the changing sensory envelope of the self (Caroline A. Jones, Sensorium - Embodied Experience,
Technology, and Contemporary Art, p8, The MIT Press, 2006)
v
Sounds range from the iconic rings of the New York Stock Exchange bell, the historic Dreamland bell (recorded days
after it was discovered in the water off Coney Island), the United Nation's Peace Bell, and more everyday and personal
sounds of bike bells, diner bells, and neighbourhood church bells. During park hours an individual bell will ring each
minute from speakers placed throughout the tunnel space where it will be installed, the overtones fading out as the next
bell begins. A chorus of the selected bells will play at the top of each hour, filling the space. The sounds will be
represented on a physical sound map that identifies the location of each bell, allowing the listener to follow the
geographic journey of the recordings. (http://www.thehighline.org/about/public-art/vitiello)
vi
‘Haptic Visuality’ is an expression associated with the experience of sensual memorizing within visual
representation... to indicate an exploration into redefining notions of the perceptible, which extends the argument on
representational expressions towards an understanding of the embodied experience intercultural cinema conveys for a
postcolonial and transnational world view. (Jaeckel, Monica. Approach via the ‘Smooth Space’ - Laura U.Marks’
Research on Intercultural New Media Practice. October 2005.)