Proceedings of the 2009 13th International Conference on Computer Supported Cooperative Work in Design

Collaborative Ethnography: An approach to the elicitation of cognitive

requirements of teams
Luiz Carlos L. Silva Junior!, Marcos R. S. Borges!, Paulo Victor R. de Carvalho!
!Programa de P6s-Graduac;ao em Informatica, Universidade Federal do Rio de Janeiro, Brasil.
{luizcarlos, mborges}@nce. ufrj. br
paulov@ien.gov.br

Abstract
The paper presents a collaborative ethnography
approach for cognitive requirements elicitation ofwork
teams in complex environments. It discusses the
concepts of cognitive systems and their requirements,
and presents a review ofmethods commonly used in the
elicitation of requirements both in the case of
traditional systems and complex systems. Then, it
points to some advantages ofa collaborative approach
in comparison to other approaches. An evaluation plan
of the approach based on experimentation and the
development of a groupware to support the proposed
methodology is also presented. The groupware aims to
stimulate collaboration and an organization in the
elicitation process ofcognitive requirements.
Keywords: Ethnography,
Cognitive systems.

CSCW,

Requirements,

1. Introduction
The human interactions in group work activities are
important objects in the study of human problem
solving and decision-making processes, and give
important clues to investigate the tacit knowledge that
teams uses during their work activity. Behind these
interactions we can find important mechanisms of
reasoning that will drive people, according to their
experience, to select and manage their actions in
context-dependent work situations. The analysis of
these interactions can lead to the discovery of basic
requirements for the construction of artifacts that can
efficiently support the process of team members
decision-making.
The aim of this paper is to show how collaborative
ethnography (i.e. an ethnography performed by many
agents who can interact with each other) can be used to
elicit the knowledge that shape team members
interactions. We also show in which way the
knowledge
obtained using
the
collaborative
ethnography can be very important to elicit useful
cognitive requirements for the implementation of
information support systems that will give adequate
support to people tasks.
We begin the paper with cognItIve systems
definitions and its requirements through some examples

of these systems in real life. Then we present some

cognitive task analysis (CTA) methods used in
knowledge elicitation process describing their strong
and weak points. Later, the collaborative ethnography
approach for cognitive requirements elicitation will be
presented and compared with the other methods
normally used. Finally, we show the requirements of a
groupware to support the collaborative ethnography
approach to improve some weak points of the method,
enabling its use in an effective way.

2. Cognitive systems design requirements

A cognitive system is a self regulated and adaptable
system that functions using knowledge about itself and
the environment to planning and modify their actions
[9]. An adequate design of a cognitive system depends
basically on: The existence of a common vocabulary
between their parts; the study of the agents' cognition
during their actual activity, analyzing the mantechnology interactions in real environments [10] rather
than studies that analyze each system component apart
(i.e. man, machine and interface) in more controlled
settings. Our research focus on cognitive systems
design because their ever growing sociotechnical
complexity due new technologies (automation,
communication, information) and new system
structures (more regulation levels, consumer feedback
and control) claims to a change of the traditional
(linear) models and old paradigms of human cognition
in system design to they take in consideration the actual
human-system interactions.
Nowadays we can observe the technology
facilitating the execution of tasks and operations with
high degree of automation. Cognitive systems are
present in energy power plants, air traffic management
systems, vehicles, control of emergencies, and in work
environments that demand high degree of coordination.
To produce effective cognitive systems, i.e. cognitive
systems that can be adequately handled by the human
agents, there is a need to consider actual cognitive
requirements in the system design phase.
However, the increase use of technology results in
an increase of the complexity of cognitive systems. The
digital control rooms with large video display units,
computers with ever growing processing capacity, and
other technological devices to support cognitive tasks

978-1-4244-3535-7/09/$25.00 2009 IEEE

167

generate another problems, as the search for high

efficiency, making that the systems operate closer to
their maximum capacity and safety boundaries, an
increasing performance dependence among the various
tasks, and a significant growth of the amount of data
that have to be processed by the human being [10].
Such situations can generate a considerable
degradation in these systems. To cope with complexity,
human agents adapt their activities generating
variability in the tasks execution, which may lead to
systems malfunction (the drift to failure mechanism). It
is important to note that in the last few decades serious
accidents have happened in sociotechnical complex
systems, indicating the need to change the paradigm on
how human cognition is treated in complex systems
design [18].
Many of these system failures and accidents emerge
in situations, in which operators did not understand the
system actual situation, i.e., the information systems do
not provide the right information in the right moment
to the operators who were not able to adapt their
behaviors according to the actual situation demands and
make their decisions in a safe way. To deal with this
problem is necessary to emphasize design processes
that facilitate the adaptation of the human cognition to
the system functionality.
The cognitive requirements, in the scope of this
work, can be defined as the functional requirements of
the system that will give critical support to the cognitive
activities of the operator throughout the execution of
his/her work. This means that these requirements will
be paramount for the construction of information
systems capable to enable people to achieve adequate
situation awareness (enhance perception possibilities,
decision-making and action planning support).
These cognitive requirements must be incorporated
in the design of displays, man-machine interfaces, and
are present in the collaborative characteristics of many
systems. We can observe some examples in: control
rooms [5], [19]; military decision-making systems [3];
air traffic control [2], [8].
To elicit knowledge for complex system design, we
argue that is necessary to carry out cognitive task
analysis in actual work situations, understanding how
and why operators make their actions, and the intrinsic
human-system relations (human-organization, humantechnology, and human-human actions). Therefore, we
claim that collaborative ethnography is one of the most
adequate approach to elicit cognitive requirements. In
the next section, we will describe some methods
traditionally used to elicit knowledge for information
systems design, in order to compare them to the
collaborative ethnography approach we propose in this
paper.

3. Knowledge elicitation approaches for

cognitive task analysis

In this section, some CTA methods already used for

in information system design in several environments
will be described. Some of these methods, like
interviews and direct observation are often used in the
elicitation of requirements for information system
design and business-oriented processes. Ethnographic
approaches are not so used in information systems
design, but they are widely used in the elicitation of
experts' knowledge and social factors.

3.1. Interviews
Interview is one of the most common methods for
requirements elicitation, however nor always does this
technique capture all the information needed to a
complete requirements definition. To a large extent, the
data generated from interviews are field notes, meeting
reports, which are difficult to analyze later on, and can
be complemented with the existing documentation in
the organization.
In many situations, the information elicited from
interviews was incomplete, due the existence of
difficult questions to be answered, i.e., questions related
to activities where tacit knowledge is used. Bell [1]
noted that is not rare the situation in which people tell
the idealized information, i.e., people say how they
should do their tasks rather than how they actually do
them.
There are research to refine the interview technique
[4], helping the organization and integration of the data
using argument structures or domain models, as the
Joint Application Development (JAD) approach that
provide a way to requirement elicitation allowing the
identification of different points of views, and means to
solve conflicts and achieve consensuses.

3.2. Observation
The observation of people actual work activities in
the organizations is a useful way to understand the
interactions, practical skills, and tacit knowledge
developed by the work teams, as well as the
organization culture. According to Bell [1], we do not
have to ask people what they do. Rather, we must
observe how people carry out their activities, because
from observation findings it is possible to discover the
reasons and motivations behind systems use, as well as
the problems or difficulties related with system
usability.
The use of observation appears in some domains for
knowledge elicitation and system improvement. In
ergonomics, direct observations are the most used
method to understand the actual work conditions, the
operators' activities and their influence on the people
health and in the system production [6].

168

The observation can be participative or nonparticipative according to the behavior of the observers
in the observation process [17]. In the participative
observation, observers interact with the workers
observed to achieve a better understanding of the work
activities. The non-participative observations can be
direct or indirect. In the direct observations the people
are observed individually, the behaviors are written
down and registered (field notes), whereas in the
indirect observations there is the use of technology to
support the register process (audio and video
equipment).
However, observations techniques are not efficient
for the capture of cognitive skills in many cases. In
emergence situations where the observer cannot
intervene directly or even be present during the activity,
it is easy to catch the tacit knowledge and the cognitive
skills using only non-participative observations.
Therefore, the experiments, simulations, interviews or
tell histories techniques may support the capture tacit
knowledge in these situations [3].

the difficulties, and the respective adaptations to face

work problems.
Using the ethnography for the elicitation of
requirements, we are able to get more details and a
more complete description of these requirements,
because the activities are observed instead of described
by the users. These factors are especially important for
the analysis and design of complex systems.
One of the ethnographic techniques we will use
ahead in the Collaborative Ethnography we propose in
this research is the Fast Ethnography [11] (see figure 1)
that recognizes the time and resource limitations to
carry out detailed and intensive ethnographic studies,
and uses short term observations together with the
debrief meetings feedback.

3.3. Ethnography
The ethnography is a methodology that comes from
the Social Anthropology, which consists of studying an
object living the same reality of this object. This
approach aims to understand and to describe, a nation,
its people or culture, using natural or participative
observation for long periods.
Using ethnography, the observed group or culture
issues are understood by living in the same
environment, being present, and making the things that
the people make and as they make [1]. It is possible to
understand how and, mainly, why the activities are
done in one determined way, because the phenomenon
is studied inside the social, cultural and organizational
context [16].
It is important to note that the ethnography contains
traces of the two previous techniques, because the
information collected are complemented with data from
observations and from informal interviews. However,
the difference in the ethnography is that all individuals
(observers and observed) are situated in the same
context. Therefore, the method has many possible
advantages [13]: Bigger familiarization with the domain
of the organization; discovery of the informal activities,
issues related to cognitive and collaborative
mechanisms; better understanding of the actions,
decisions, strategies, behaviors and interactions of the
agents in the context where they occur; perception on
how the interaction with the resources or devices
occurs; and more visibility about usability problems of
the existing technologies.
Moreover, the ethnography shows how the physical
disposal of the work environment and the layout of the
devices and equipment have influence, positive or
negative, on the efficiency of the activities as well as in

Figure 1 - Processo de etnografia rapida

The ethnographic approach presents also some

problems. Normally, its execution is complex, due the
impossibility of one person alone to capture and to
analyze all the information of the social environment.
The method demands extensive planning and
coordination issues if the ethnography will be carried
out by teams, because people may have different
perceptions and points of view about the observed
activities. Another limitation is the need of a long time
to capture and analyze what occurs in the field, to get
rich and detailed information [15]. In some cases, is
necessary a familiarization with the domain to be
studied. Moreover, it is difficult the use of the
technique in large work settings, complex and
distributed, due the huge number of variables to be
observed.
The technique also can present risks for the
researchers, or to be impracticable if the presence of the
researches/observers jeopardize the work activities
observed. In some cases, there is the need of getting the
permission for entrance in the work settings and for
registering the information, and also the acceptance and
the assent from those who will be observed [6].
In the scope of software development process the
ethnographic technique also presents some difficulties.
One of them is absence of formalism and objectivity
required for information systems designers, because it
presents the results in a descriptive and textual format
[20], making difficult their transformation in software
requirements.

169

4. The collaborative ethnography approach

for cognitive requirements elicitation
In this section we will discuss how the
collaborative approach can address the elicitation of
cognitive requirements elicitation of work teams. We
first stress the importance of the combination of
collaboration and ethnography, particularly how this
combination can help the study of complex activities.
Then, we present a succinct review of previous work,
which have used this combined approach to elicitate of
interactions and the social dimension of teamwork.

groups and their interaction being performed by a team

of ethnographers is a very challenging goal. The huge
amount of information collected that needs to be
organized and processed requires not only an organized
process but also an appropriate supporting tool. The
complexity of this alternative is a consequence of
combined circumstances originated by the multiple, and
perhaps conflicting, perspectives from the observers'
part and the potentially high number of interactions
among the members of the observed group. We will try
to avoid the loss of relevant information intrinsic to
scenario B.

4.1. Why collaborative ethnography?

4.2. Applications of collaborative ethnography

First, it is pertinent to clarify why to add the

collaboration to the ethnography method. A
straightforward combination between observers and
observed subjects is presented in Table 1. We can have
a single person observing another person or a group,
and a team observing a single person or a group.

The collaborative ethnography is a technique that

has been used extensively in other domains for the
social analysis of work activities. May and PattilloMcCoy [14] reported a work reproducing the way the
ethnographers discuss their findings in a collaborative
way generating a single document with the information
collected and their different viewpoints. They reported
how different the ethnographers' perceptions about the
environment are. They claim that variety is due to
several cognitive factors that also help the reach of
consensus in the discussion.
It is also important to note the value of the
collaborative ethnography to complement the details of
the object of study. The inconsistencies and ambiguities
of observations are more easily identified and solved. In
other words, the ethnographers playa positive influence
on the results of the process.
On the other hand, it becomes clear that the
collaborative ethnography has some drawbacks. First, it
shares, or perhaps increases, the time spent to collect,
analyze and conclude about the collected information.
The need to join, to compare and to discuss the
information collected requires time. The collaboration
required needs extra effort and is time consuming. The
field notes needs to be combined in order to reflect the
different perspectives of observers.
According to May and Patillo-McCoy [14] the
observation itself influences the results of the
ethnography. Thus, the use of several observers opens
the possibility of analyzing this impact in the process
and, at the same time, enriching the information
collected. We expect that the approach contributes to
the expose of tacit information, specialized knowledge
and the decision rationale, mostly due to the interaction
between the ethnographer with the task, the
environment and especially with other ethnographers.
Machado [13] also carried out a collaborative
ethnography study aimed to elicitate software
requirements. The study reports an experience with
field observations and with the use of a tool that
facilitates the consolidation of data from multiple
sources into a shared repository. In comparison with
traditional techniques based on interviews, the

Ta bl e 1 Com bOInarIon 0f eth nograp hIy st UdOles

Observer/ Observed
Individual
Teams

Individual
A
C

Teams
B
D

Scenario A is the most common case: An

individual observing another individual carrying out
some task. This situation also refers to a number of
observer/observed pairs being done in parallel. If the
observation refers to the same task some variation may
occur between different observers/observed. The results
are quite dependent on the background and on the
previous experience of the observer. There may also be
some information loss during the observation. In this
scenario, the aggregation of information from different
sources is not an easy task and may show some
inconsistencies.
Scenario B is possible but not common. In this
scenario the loss of information is potentially high. It is
very difficult to a single observer to capture and to
understand all tasks and interactions, particularly if the
activities are complex. A way to overcome these
difficulties is to do the observation in several sessions.
The scenario C is normally used when the
observation requires multiple perspectives over the
same observer or task. In this scenario it is expected
that the information gathered and processed would be
richer that that collect by a single observer. This can be
considered a particular case of scenario D, assuming
that the observers would meet and discuss about their
findings before, during and after the ethnography. This
is not easy to realize without the support of an
appropriate process and a supporting tool.
Finally, the scenario D is the circumstance that
will be dealt with in this work. The observation of

170

observation approach have shown more efficient and

generated more complete information.
The experiments carried out in that study has
shown that a groupware plays an important role in the
coordinating of activities and on the organization of the
data (videos, photos, comments, notes, etc.) reducing
the burden generated by the collaborative ethnography.

5. Groupware requirements to support

collaborative ethnography
According to Guerlain [7], the evaluation of
activities from a coordinated team involves the
independent observation of multiples individuals. The
analysis of these independent observations requires they
should be coordinated, codified and correlated before a
subjective evaluation can be performed.
Thus, similar to other works [7] [12] [13], the
support of a groupware tool aimed to assist the
information exchange and the discussion among
ethnographers is desired. The groupware tool that
supports the collaborative ethnography should facilitate
collaboration and interaction after the gathering of field
data in order to identify the cognitive requirements with
less effort.
The basic functions for the groupware to support
the collaborative ethnography are:
register, update and closing of ethnography
projects;
record profiles of observer and observed persons;
assign of observers to observed activities and
persons;
record of notes and maintenance of historical data;
document creation and its association with
problems, comments, ideas and hypotheses raised
by ethnographers;
support the discussion and negotiation about the
documents and the requirements generated;
support awareness about the level of participation
and contribution;
support multimedia elements;
support awareness mechanisms for both
synchronopus and asynchronous interaction,
informing them about contributions (new elements,
comments, etc.) added by other members.
The groupware tool will be used after the
information capture in the field. The team members will
be able to individually insert all type of information
they believe are important to the requirements
elicitation. Then, the tool will support the discussion,
the consensus information, and also how the former led
to the latter. It will also allow members to group their
notes using categories (actor, setting, process, etc.). The
notes can be associated with problems, cognitive
requirements and possible solutions
to these
requirements. See Figure 2 for a schematic view of the
data managed by the groupware.

Figure 2 - Products and their relationships of

Collaborative Ethnography

The tool will provide means to coordinate the

ethnography activities in both asynchronous and
distributed way. It should help analysts to reach
consensus, besides inducing the reflexive thinking. It
will also help the planning of further studies and
observations on specific parts of the work, which
demand a deeper analysis. The tool should be designed
using usability principles that should be evaluated
during the study. However, the main goal of the study
and the tool, which is to contribute to the efficiency and
efficacy of the process, will be pursued. A set of
questionnaires will try to verify the conformity of the
approach and the tool to these goals.

6. Conclusions and future work

In this work we reviewed some important concepts
in the area of cognitive systems. We also revised the
main approaches for the knowledge elicitation:
interviews, observation and ethnography. We
confirmed that these approaches, though extensively
adopted, still show some inadequacies and inefficiency
in the requirements elicitation process, particularly in
relation to cognitive requirements in complex
environments.
To deal with these problems, we proposed a
collaborative ethnography approach as a more
appropriate alternative for the cognitive requirements
elicitation. We claim that the ethnography approach has
many advantages, such as a detailed analysis of the
environment where the operator performs his/her task.
When ethnography is combined with a collaborative
approach that supports different viewpoints as well as
interaction and discussion among the observers, we
believe we have an approach that produces a more
efficient and consequential results.
Part of the tuning of our approach is dependent on
the application domain. Thus the groupware tool has to
be highly parameterized. The goal is to facilitate the
identification of cognitive requirements from the mass
of information gathered in the process and the reaching
of consensus over the main points to address.

171

For the experiments we will conduct two types of

techniques for the elicitation of cognitive requirements:
the first based on interviews and the second based on
the approach proposed in this paper. We intend to carry
out a qualitative and quantitative comparison of the
results. We aim to evaluate not only the method itself
but also the support of the groupware tool. Our purpose
is to check if our hypothesis about the efficiency and
efficacy of our approach hold true in a variety of
situations.

References
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

Bell, G., (2004). Insights into Asia: 19 Cities, 7

Countries, 2 Years - What people really want from
technology. Technology@lntel Magazine.
Bentley, R., Hughes, J., Randall, D., Rodden, T.,
Sawyer, P., Shapiro, D., Sommerville, I., (1992).
Ethnographically-informed systems design for air traffic
control, In Proc. Computer-Supported Coooperative
Work Conference, ACM Press, New York, 123-129.
Crandall, B., Klein, G. Hoffman R.R., (2006). Working
minds: a practitioner's guide to cognitive task analysis Cambridge, Mass.: MIT Press.
Christel, M. G.; Kang, K. C., (1992). Issues in
Requirements Elicitation, Technical Report, CMU/SEI92-TR-012, ESC-TR-92-012, September.
dos Santos, I.1.A.L., Teixeira, D.V., Ferraz, F.T.,
Carvalho, P.V.R. (2007). The use of a simulator to
include human factors issues in the interface design of a
nuclear power plant control room. Journal of Loss
Prevention in the Process Industries 21(3), 227-238.
Guerin, F., Laville, A., Daniellou, F., Duraffourg, 1.,
Kerguelen, A., (2004). Compreender 0 trabalho para
transforma-Io: a pratica da ergonomia. Edgard Blucher:
Funda~ao Vanzolini, Sao Paulo.
Guerlain S., Shin T., Guo H., Calland 1.F., (2002). A
team performance data collection and analysis system.
In: Proc. of the human factors and ergonomics society
46th annual meeting, Baltimore, MD, 30 September-4
October.
Harper, R., Hughes, J.A. and Shapiro, D., (1990).
Harmonious
Working
and
CSCW:
Computer
Technology and Air Traffic Control, In: Studies in
Computer Supported Cooperative Work: Theory,
Practice and Design, ed. 1. Bowers and S. Benford,
Amsterdam, North- Holland.
Hollnagel, E. & Woods, D. D., (1999). Cognitive
Systems Engineering: New wine in new bottles.
International Journal of Human-Computer Studies,
Volume 51, Number 2, August 1999 ,pp. 339-356(18)
Hollnagel, E. & Woods, D. D., (2005). Joint cognitive
systems: Foundations of cognitive systems engineering.
CRC Press, Inc., Boca Raton, FL, 2005.
Hughes, J., King, V., Rodden, T., and Andersen, H.,
(1994). Moving out from the control room: ethnography
in system design. Proc. of the ACM conference on
Computer supported cooperative work, Chapel Hill,
North Carolina, October 1994, p. 429-439.
Hughes, 1., O'Brien, 1., Rodden, T., Rouncefield, M.
(1997). Ethnography, Communication and Support for
Design. Technical Report Ref. CSEG/24/1997, Lancaster
University, Lancaster, UK, 1997. Available at:
http://www.comp.lancs.ac.uk/computing/research/cseg/9
7_rep.html
Machado, R. G., (2008). Developing of a Collaborative
Ethnography Method for Software Requirements
Elicitation. Rio de Janeiro, 2008. M.Sc. Dissertation,

[14]

[15]

[16]

[17]

[18]
[19]

[20]

172

Graduate Program in Informatics, Federal University of

Rio de Janeiro, Brazil (In Portuguese).
May, R. A. B., Pattillo-McCoy, M., (2000). Do You See
What I See? Examining a Collaborative Ethnography.
Qualitative Inquiry, Vol. 6, No.1, 65-87.
Millen, D., (2000). Rapid ethnography: time deepening
strategies for HCI field. Proc. ACM Symposium on
Designing Interactive Systems, ACM Press, New York,
280-286.
Myers, M. D., (1999). Investigating information systems
with etnographic research. Communications of the
Association for Information Systems, Vol. 2, Article 23,
December.
Nardi, B., (1997). The use of ethnographic methods in
design and evaluation. In: Helander, M.G., Landauer,
T.K., Prabhu, P. (Eds), Handbook of Human-Computer
Interaction. Elsevier Science, North-Holland, Holanda,
361-366.
Reason, 1. T., (1990). Human Error. Cambridge, U.K.:
Cambridg University Press.
Roth, E. M., Christian, C.K., Gustafson, M., Sheridan,
T.B., Dwyer, K., Gandhi, T.K., Zinner, M.1., Dierks,
M.M., (2004). Using field observations as a tool for
discovery: analysing cognitive and collaborative
demands in the operating room. Cognition, Technology
and Work 6, 148-157.
Sommerville, I., Rodden, T., Sawyer, P., Bentley, R.,
Twidale, M., (1993). Integrating ethnography into the
requirements engineering process. Proc. of IEEE
International Symposium on Requirements Engineering,
San Diego, CA, 165-1 73.