SENSORY ETHNOGRAPHY IN A DIGITAL-MATERIAL WORLD

Sarah Pink, RMIT University, Australia and Halmstad University, Sweden

Introduction
In this chapter I argue for an approach to sensory research that accounts for the
‘digital materiality’ of everyday worlds. The chapter undertakes two tasks: first it
outlines how new self-tracking technologies might be understood from the
perspective of researchers interested in sensory experience, activity and
environments; second it presents the foundations for using such technologies in
sensory ethnography research practice.
I combine three themes developed in recent work to discuss sensory
ethnography1 in relation to digital ethnography2 and as part of everyday environments
of digital materiality3. I draw on research into how people use and experience self-
tracking technologies undertaken in Melbourne, Australia to demonstrate two
elements of the potential of self-tracking technologies in sensory ethnography: the use
of sensory reenactments of technology use to enable research participants to invite
researchers to empathetically imagine what their experience of using and living with
technologies is like; and how self-tracking technologies can simultaneously become
sensory elicitation technologies, through which we can learn more about participants
everyday embodied experiences.
In 2015 I discussed how ‘body monitoring technologies are … becoming
increasingly ubiquitous in everyday life. These include technologies such as sleep
monitors, platforms where calorific intakes can be calculated, digital pedometers and
more, and they create digital data which can be reflected on, shared and more. Often
these are accessible through smartphone apps thus linking the qualities and affor-
dances of the smartphone discussed above to the use of body data’. I proposed these
technologies were interesting for sensory ethnography practice ‘because they lead us
potentially to the ways people experience their bodies’, in that in measuring the body
‘they are quantifying … a sensory embodied experience – for instance of walking,
running or eating’. While ‘Such technologies do not express the sensoriality of such
experiences, … they make the body visible in new ways’ in that they make
representational precisely the elements of sensory embodied experience we would
argue are non-representational. I suggested that ‘working with participants via their
uses of monitoring technologies, and data-visualisations of their embodied sensory
experiences offers us another route towards understanding how they experience the
body–environment relationship’ and are ‘emplaced in the world’4.

Introducing the sensory ethnographic-theoretical dialogue

Sensory ethnography is an approach to ethnographic practice developed in dialogue
with anthropological theories of sensory perception, learning and knowing. I my own
practice it is an interventional, rather than observational ethnography, orientated
towards design and provocation and attends to what we know, learn and do as

1
Sarah Pink: Doing Sensory Ethnography. Second Edition. London: Sage. 2015
2
Sarah Pink: Heather Horst, John Postill, Larissa Hjorth, Tania Lewis and Jo Tacchi:
Digital Ethnography: Principles and Practice. London: Sage. 2016.
3
Sarah Pink, Elisenda Ardevol and Debora Lanzeni (eds): Digital Materialities.
Oxford: Bloomsbury. 2016.
4
Pink: Doing Sensory Ethnography. P136-7.

1
ethnographers as open and incrementally developed across a career, rather than within
closed projects.
Much work on the senses in culture and society has concentrated on the
modern western five-sense-sensorium, whether to engage it as a set of categories
through which to form research agendas,5 demonstrate that it is not universal6,
critically engage with its status as an object of anthropological study7 or to seek ways
its categories might be used while maintaining a phenomenological appreciation of
their constructedness.8 My interest in sensory experience and perception focuses on
how we know, learn, improvise and move on through and in everyday worlds. I am
concerned with how these elements of everyday life, knowing and action can be
knowable to others, how researchers might learn about them from research
participants, and particularly with how visual, digital technologies as well as verbal
discussions contribute to those learning processes. Here my interests in sensory
experience and perception and in digital technologies intersect in the process of
ethnographic knowing.
In a world where digital technologies are increasingly ubiquitous sensory
ethnography needs to be re-thought as an approach that can encounter everyday
environments where the digital is entangled.9 This means considering how sensory
experience, perception and ‘cultures’ might be interpreted to account for theoretical
understandings of the roles of the digital in the constitution of everyday environments
and trajectories. This means understanding ‘media presence’ beyond the conventional
fields of the study of media content and media as communication10 to undertake
sensory ethnography in a world where digital technologies and the qualities,
affordances and potentialities they offer are always present.

Sensory ethnography in a digital-material world

From the perspective of sensory ethnography, by bringing together theories of sensory
experience and perception with digital media theory we can understand how digital
technologies and media form part of our sensory embodied engagements with the
world11. From the viewpoint of digital ethnography, the issue concerns how a focus
on sensory experience can produce understandings of our relationships with digital
technologies.12 As both perspectives indicate, we need to account for how digital
technologies are part of human activity, (sensory) experience and of the environments
in which this occurs.
The concept of digital materiality seeks to define and discuss how things and
processes occur in a contemporary environment where digital technologies are

5
David Howes (ed): The Empire of the Senses. Oxford: Berg. 2005.
6
Kathryn L. Geurts: Culture and the Senses: bodily ways of knowing in an African
community. Berkely, Los Angeles, London: University of California Press. 2003.
7
Tim Ingold: The Perception of the Environment. London: Routledge. 2000.
8
Vaike Fors, Asa. Backstrom and Sarah Pink: Multisensory emplaced learning:
resituating situated learning in a moving world. In Mind, Culture, and Activity: An
International Journal, 20(2): 170-183. See also Sarah Pink: Doing Sensory
Ethnography. 2013.
9
Pink: Doing Sensory Ethnography
10
Sarah Pink and Kerstin Leder Mackley: Saturated and Situated: rethinking media in
everyday life. In Media, Culture and Society 35(6): 677-691. 2013.
11
Pink: Doing Sensory Ethnography. Chapter 6.
12
Pink et al: Digital Ethnography. Chapter 2.

2
entangled in everyday life. The notion of the digital material does not start with ‘an a
priori definition about what is digital and what is material’ but is understood as ‘a
process, and as emergent, not as an end product or finished object’ which challenges
‘the boundaries that are assumed when asking what is digital and what is not’13. The
notion of digital materiality is also a methodological proposal, since ‘in order to
centre our attention in the process of making in the everyday life mess, we need a
processual approach to digital materiality. This creates a prism through which to
examine the complex interfaces at which we engage with technologies, architectures
and narratives that constitute the materiality of the everyday’14. Here I use this
understanding to consider how self-tracking technologies, their use and their
relationships with sensory perception are similarly entangled in this messiness of the
everyday.
When viewed as processual, a context of digital materiality is characterized by
ongoing potentiality. This is to a certain extent related to the ongoing presence or ‘on-
ness’ of digital technologies15 whereby on/off are no longer binary states or statuses
for media, and instead we encounter and engage with digital technologies, platforms,
social media and more, as being ‘there’ and having the potential, possibility or
expectation of becoming active or apparent. This absence of the ‘off’ state means the
digital cannot be thought of as separate from the world it is part of. Self tracking
technologies are interesting for exploring sensory ethnography in a digital-material
world, because they have a particular status within such an environment. They might
be ongoingly automatically in process, users might expect them at some moment to
send an alert, or to check data or to input an instruction that a particular activity needs
to be recorded. They are thus part of a flow or process of digital materiality.
However, because such technologies create ‘data assemblages’16 and bring
together human experience and data about physical activity, they are also ambiguous
technologies in their relationship to the body and to data, since their relationship to
what a quantitative and measured world is not always what it might seem. My
ethnographic encounters with self trackers show that for some users, self tracking
technologies did not produce what research participants believed are accurate or fixed
representations of their physical activity, but rather serve them in a more qualitative
sense.17
Therefore, to undertake sensory ethnography in a digital material world
involves acknowledging that sensory experience, perception and knowing do not
necessarily happen in relation to distinct ‘digital’ and ‘material’ entities or processes
but emerge as part of context of digital materiality.

Self-tracking
There is a growing literature about self-tracking in fields of digital sociology18 and
sociology of consumption19. Much research has focused on the Quantified Self (QS)

13
Pink, et al: ‘Digital Materiality’. P10.
14
Pink etal: ‘Digital Materiality’ P11.
15
Pink and Leder Mackley. ‘Saturated and Situated’
16
Deborah Lupton: Digital companion species and eating data: Implications for
theorising digital data–human assemblages. Big Data & Society
2016, 3 (1) DOI: 10.1177/2053951715619947
17
This strand of analysis will be discussed in future publications.
18
Deborah Lupton: The Quantified Self. Oxford: Polity. 2016.

3
movement, which provides a range of forums and meet-ups, predominantly in the
global North, through which self-trackers form a ‘community’ (http://quantified-
self.meetup.com/). Research in this field has begun to examine how the body is
represented and experienced through self-tracking technologies. However there has
been little attention to the implications in relation to what scholars in the social
sciences and humanities have called the sensory,20 visceral21 elements of everyday
experience. Self-tracking technologies, particularly body-monitoring technologies
offer an ideal example through which to engage with situated examples of sensory
embodied experience, as proposed in the opening section of this chapter. I next
introduce self-tracking technologies as everyday technologies and as research tools. I
do not refer to particular brands, or cover all eventualities, but focus on two
variations: smart-phone apps and wearable wristbands.
Wearables and apps overlap in everyday experience, since data collected by a
wearable will often be displayed (visualized) though a smartphone app and both can
usually be used with an online platform accessed through other devices. In common
the algorithmic calculations they make and report on to users are also based on
additional data that is input, such as age, weight, height. Yet, for the purposes of
sensory ethnography, wearables and smart phone apps as data collector/producers
bear a different relationship to the human body, and this proximity is significant in
terms of not only the data that they report, but also in terms of what it feels like for
people to use them. Apps that track elements of body activity can be downloaded onto
smart phones – including those for tracking movement automatically through GPS
mapping, and which count steps, distances covered, types of exercise or transport
used, and calories used. This automatically produced data can also be combined with
additional data input by users, including information about calories or other aspects of
foods consumed, self-reported data on sleep times and quality and more. Some apps
might depend on particular technologies that connect them to the body, for example,
as detailed below one research participant used a meditation app that, via an ear-clip
would measure his levels of stress and anxiety. More conventional sports-training
heart-rate monitors would also be strapped on the chests of users and smart phones
are strapped onto the bodies of runners and during routine exercise. Wristband
wearables differ in that they perform some automated data-collection/production
modes but are worn on the body, whereas a smartphone tends to be carried on the
body or in something attached to the body. A wearable has a different ongoing
proximity to the body, feels different when carried on the body and senses a different
range of body activity. Typically contemporary wearable wristbands track movement,
distance covered, calculate calories burned, sleep quality and monitor heart rates. For
some activities users need to change the band’s mode, for instance when going to bed,
so it will monitor sleep, wakeful periods and sleep quality, or if they begin particular
modes of exercise.

19
Mika Pantzar and Minna Ruckenstein. The heart of everyday analytics: emotional,
material and practical extensions in self-tracking market. In Consumption Markets &
Culture 18 (1): 92-109. 2015.
20
Eg, Howes: Empire of the Senses
21
Hayes-Conroy, A. and Hayes-Conroy, J. (2010) Visceral Geographies: Mattering,
Relating, and Defying. In Geography Compass 4/9 (2010): 1273–1283.

4
 Both technologies provide on-screen data visualisations (and sensory alerts)
which can be accessed across different devices. While not discussed here, the ‘small
data’ they produce also, when harvested by third parties, contributes to big data.22

Sensing, Shaping, Sharing: sensory ethnography in practice

The examples discussed here derive from the Sensing, Shaping, Sharing project
focusing on the past, present and future of self-tracking users and technologies in a
mediated world. I reflect on the ethnographic strand here discuss how self-tracking
technologies can connect the digital and the sensory both for research participants and
as part of how researchers can learn about other people’s experiential worlds. I
explore this by reflecting on the performative and verbal ways participants invited me
into the environments in which they self-tracked and encountered their data.
A methodological challenge in researching self-tracking is how to encounter
other people’s experiences, particularly when they self-track either automatically in
the ‘background’ and intermittently check data, or during activities during which they
cannot be interrupted (eg training, sleeping or meditating). Video reenactment
methods (discussed in depth elsewhere, relating to everyday life routines in the
home23 and digital technology use24) resolve some of these issues regarding
researching 1) how people use and experience these technologies and 2) their sensory
experience of a digital-material environment when self-tracking technologies are part
of it.
The performance of self-tracking as a sensory embodied and emotional
experience with and in relation to technology, implies how self-tracking is implicated
in mind-body configurations. For example Christof, a man in his 30s, described in our
interview how he used his meditation app to seek inner balance, and track his moods.
He also demonstrated to me how he used the app, which led him to discuss further
with me how as much as tracking and collecting data, the app also played another role
in his sensory embodied experience of meditation. Christof picked up one of the three
devices he had brought to the interview, telling me that ‘This is actually my
meditation app, its just an iPod, but it has this contraption which I'll set up’. He
picked up a cable with a clip on one end and which was plugged at its other end into
the iPod and continued: ‘so what I'll do here is clip that to my earlobe’ putting on the
earlobe clip, noting how ‘it just takes my pulse, and by using the tool …’. He then
touched the screen of the iPod to enter the app and turned the iPod to the camera to
show me the data visualization, explaining that ‘its got a breathometer down below
that guides my breathing so now I breathe in ... and out’. As he did so a dot on the
breathometer visualization moved back and forth and made sounds, while the graphic
visualisations above on the screen are compiled with the data. I asked him how long
he would normally do this for and he told me that 'It varies, sometimes 10 minutes,
sometimes half an hour, I just use that in the midst of my normal meditation'. He went
on to describe how he would meditate at the beginning or the end of the day, and how

22
Deborah Lupton: Personal Data Practices in the Age of Lively Data (July 28, 2015).
Available at
SSRN:http://ssrn.com/abstract=2636709 or http://dx.doi.org/10.2139/ssrn.2636709
23
Eg: Sarah Pink and Kerstin Leder Mackley: Reenactment Methodologies for
Everyday Life Research: Art Therapy Insights for Video Ethnography. In Visual
Studies 29(2), pp.146-154. 2014.
24
Sarah Pink, Jolynna Sinanan, Larissa Hjorth and Heather Horst. Tactile Digital
Ethnography. In Mobile Media and Communication. 2016.

5
this would help him to 'get centred and sometimes I use the app, sometimes I don't,
sometimes I find its better to use music, but I do like, I've got quite an active mind, so
for me, its actually, using these sorts of tools helps me focus on the task at hand
actually whether its music, or a guided meditation, or using the app ... if I was just to
sit there with stillness my mind would take over and take me somewhere else'.
Christof’s performance of his use of the app in a physical and material sense,
and showed how his sensory experience and the visualisations on the app came
together to enable him to understand his state of relaxation in a particular way. It also
served as a prompt to bring to the fore the personal experiential elements of
meditation that were part of his own everyday practices. In doing so it took him away
from the discussion of the technology itself to reflect further on the experience of
meditation as always for him being a mediated activity, with music or guidance. Thus
these research encounters and techniques offer us ways to understand not only how
people use self-tracking technologies to understand their bodies, but also the provide
us with routes into exploring the nature and sensory embodied experiences of such
activities with participants in relation to their data.
However, research that uses self-tracking technologies is not necessarily
simply about the experience of the technologies or data themselves and can also focus
on participants’ experiences of a particular environment or activity. While the
technology and their experience of it will always be implicated in their experiences,
self-tracking enables us to ask people to record aspects of their lives and use the
technology as a memory-jogging or provocative device. An analogue analogy to this
is use of photo-elicitation methods in visual sociology and anthropology25. In one
variation of photo-elicitation research participants are asked to photograph their lives,
environments or an activity, and to discuss the photographs with the researcher. In
other versions the photographs might be taken by the researcher or might be found
photographs. In the case of self-tracking the relationship of the participant to the
materials produced is slightly different, in that in some cases it might be self-input
and in others it would be automatically collected through the self-tracking device
without the participant needing to intervene at all or only in minimal ways.
For example, in one interview David, a serious sports person who participated
in competitive running in the past, showed me how he used a Fitbit wristband
wearable that automatically recorded his daily steps, heart rate and sleep, that he
combined with other technologies and manual data input. Our discussion of his
experience of using the technology brought to the fore his embodied experience of
playing tennis, in a way that emphasized his sensory experiences and simultaneously
showed how his use of the technology enabled alternative interpretations of his
experiences to be drawn into his analysis.
When we initially discussed the technology David told me ‘it monitors your
pulse when you’re sleeping, and its like for fitness, if you see something spike up then
that means you’re over training and you should cut back, so this would work really
well on that’. As he was no longer training he didn’t usually worry about this, but he
noted that something curious had recently happened:

… actually I looked at it last night because tennis was brutal yesterday, it was
a three set one and I thought it would be up more, but the sleep was pretty
much the same. I didn’t wake up cramping or anything, which I was really

25
Eg, John Collier and Malcolm. Collier. Visual Anthropology: Photography as a
Research Method. Albuquerque: University of New Mexico Press. 1986.

6
 happy about, but its like I slept pretty much the same amount of time with the
same amount of restlessness that I would in any other night, I thought it would
be worse just because I sort of knew I was going to bed sore, the …
[painkiller] worked [laughs]

When we looked at the data associated with the wearable on his desktop computer
later in our meeting, David returned to this example, to elaborate on how the data had
enabled him to bring additional understanding to his interpretation.

… here’s the heart rate stuff, yesterday … because I played tennis, it does
come back and it says the daily average is 4400 … so it measures everything,
the one that I … do track is looking at sleep stuff and you can see like last
night I had seven hours and 36 minutes sleep, woke up for five minutes at one
point, 43 minutes I was tossing around, and its, I woke up around four
[o’clock] it looks like … [he puzzles over the data briefly], but I’ve just been
watching it out of interest, … you see the rem stuff … but I haven’t really
looked at this that carefully to see if there are any trends

David was interested in his sleep data, but not sufficiently to analyze it quantitatively.
Instead, like other participants he cast a more qualitative glance over it, to assess what
it was telling him. He also considered this in relation to his conscious memories of his
sensory experience of aspects of the events he discussed:

… today I woke up late but I was actually exhausted, I was actually surprised
that, the sleep pattern was pretty much the same as every other night [Sarah:
due to the tennis?] yeah – yesterday was a long three-setter, so I thought I
would wake up more often but I didn’t… its just like looking over time here, it
looks like pretty much … a trend, I do sleep about the same amount, its like
somewhere seven, seven and half hours, six or seven, yeah I sleep about not
the same time every night ...

This example has a role in my wider research, where, along with research encounters
with other participants it will contribute to my analysis of how people experience data
in relation to their embodied sensory experiences. However as a single example it also
shows how we might use self-tracking technologies as entry routes into how people
remember, understand and articulate their sensory embodied experiences of human
activity in the world.

Conclusion
As I have shown here, self-tracking has great potential as a sensory ethnography
research technology, and for a focus on a core issue for sensory ethnography – that is
the relationship between the mind-body-environment. It should form part of a
reflexive agenda that accounts for how self-tracking research methods will only ever
produce knowledge with self-trackers. Yet, this applies to any study since
ethnography is only ever done with research participants.
The further potential for self-tracking technologies as sensory ethnography
technologies remains to be explored, and will emerge as the technologies and their
possibilities also develop. I end this article with a call to researchers in this field to
experiment and explore these possibilities. As self-tracking becomes increasingly part

7
of everyday life, it offers us a growing number of opportunities to do sensory research
in new ways.

Acknowledgments
The research discussed here is part of ‘Sensing, shaping, sharing: measuring and
imagining the body in a mediatized world’ funded by the Riksbankens Jubileumsfond,
Sweden and in collaboration with Vaike Fors, Martin Berg and Tom O’Dell, at
SCACA, Halmstad University, Sweden.