TooI-Mediated Coordination of VirtuaI Teams in CompIex

Systems









ABSTRACT
Coordination in online spaces, speciIically in peer
production systems, has Irequently been an aIterthought,
with participants and observers largely agreeing with the
phenomenon that the wisdomofthecrowdswillcreatean
emergent order that will drive project goals and ensure that
the most equitable possible path will be taken due to this
charitable and democratic process. The reality oI this
situation is that oIten, the tools available Ior coordination,
evaluation, and work articulation are not suitable to the task
at hand. The goal oI the present study will be to Iirst,
review and expand upon our knowledge oI how tool-
mediated coordination currently Iunctions within a peer
production systems; second, identiIy and analyze how
members within that system perceive the coordinative
aIIordances at a project and global level; and third, given
the tools that are currently used, the research on them, and
the desires oI the active contributors to that system, identiIy
potential vacuums where a new intervention would have the
most impact.
Author Keywords
Coordination; bots; tool-mediated coordination; CSCW
ACM CIassification Keywords
H.5.3. Group and Organization InterIaces Computer-
supported cooperative work.
GeneraI Terms
Theory; Measurement.
INTRODUCTION
Group coordination in online spaces poses some unique
challenges. Among them, it is typical that group members
are not collocated, potentially with individual team
members not even being located in the same time zone.
Accordingly, communication among team members can be
diIIicult and Irequently only occurs in asynchronous
Iashion. Related, knowledge oI other group members, their
talents, their desires, and their schedules and oIIline
interests are Irequently unknown. Policies and social norms
are Irequently only gleaned through immersion in that
group - a process that can only be accomplished through
time, increasing the barrier to entry Ior new participants
who may otherwise have been willing contributors. Rules
and governance structures Irequently exist, but in a similar
vein to the prior issue, they are oIten impenetrable to new
members, ensuring an entrenched minority controls the
rules Ior the majority. Given these myriad challenges and
diIIiculties, it is indeed remarkable that these types oI
projects exist, let alone succeed. However some have
succeeded; chieI among them is Wikipedia.
In what must now be a Iamiliar reIrain to researchers oI
online systems - Wikipedia is an online encyclopedia Iirst
launched in January oI 2001 |22|. Running on the open-
source MediaWiki platIorm, the encyclopedia introduced
users to a novel new Iorm oI collaboration and
communication; an online encyclopedia that anyone can
edit, created with the goal oI making Ireely available the
sum oI all human knowledge
1
. Since that time, Wikipedia
has grown to be one oI the most visited sites on the web,
currently ranking number six in both global and US visits
2
,
with over 10.9 million articles, over 21 million registered
users, comprised oI over 428 million edits by registered
users and over 126 million edits by anonymous users since
its inception in the English Wikipedia alone
3
.
Given this, it is a Iair estimation to label Wikipedia as one
oI the most active and successIul commons-based peer
production communities existing today. These types oI peer
production communities typically exhibit three types oI
primary structural attributes |2|: Iirst, the objects being
acted upon should be modular, allowing Ior a more clean
division oI potential eIIort; second, those objects should be

1
https://en.wikiquote.org/wiki/JimmyWales
2

http://www.alexa.com/siteinIo/http3A2F2Fen.wikipe
dia.org, as oI April, 2014
3
Numbers collected Irom the Wikipedia Toolserver,
http://toolserver.org/, accurate as oI April, 2014

Permission to make digital or hard copies oI all or part oI this work Ior
personal or classroom use is granted without Iee provided that copies are
not made or distributed Ior proIit or commercial advantage and that copies
bear this notice and the Iull citation on the Iirst page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
speciIic permission and/or a Iee.
CSCW14,February 1519, 2014, Baltimore, Maryland, USA.
Copyright 2014 ACM xxx-x-xxxx-xxxx-x/xx/xx...$15.00.

- 1 -
Confidential Submission - Under Review - CSCW 2015
Sentence too
long, please
consider
separating it.
Generalizati
on/please
change or
cite.
always/
usually
unknown to
whom?
Researchers or
within the
community?
Cite?
Maybe new
sentence?
Nice
paragraph!
No clear logical connection between
paragraphs. Maybe start paragraph with
something like: peer production
communities like wikipedia....
First sentence should go to the previous
paragraph.

relatively granular, meaning that the prior division results in
work units that are manageable Ior the intended population
(individual, small group, or large collective) to operate on
them eIIectively; and third, the mechanisms by which
changes to the above objects are modiIied can be integrated
back into the whole community in a simple, low-cost
manner. From the perspective oI Wikipedia, one aspect oI
this modular Iorm oI development could be the
WikiProject.
WikipediadefinesaWikiProjectasagroupofcontributors
who want to work together as a team to improve
Wikipedia.
4
Rather than distinguish the atomic elements oI
Wikipedia !"# peer production community as the
encyclopedia article, I suggest that the modular objects as
described above may be at the level oI the WikiProject,
these collections oI editors working together to improve
Wikipedia, adopting tasks less constrained than editing an
individual article yet still providing boundaries upon the
spectrum oI activity an editor may expect to execute
through the stated scope and goals oI the WikiProject itselI.
The scope oI these projects can vary widely - Irom cats to
technology to military history - WikiProjects exist Ior a
variety oI subject matter and task centered areas. Beyond
these topic-Iocused areas prior research has shown that
task-oriented projects also exist, and while editing activity
to topic-oriented projects (such as the beIore mentioned
cats, technology, and military history) has seen Iairly steady
decline, these alternative task-Iocused projects have seen a
Iairly consistent increase in activity since 2005 |19|,
indicating an increase in the complexity oI coordination
given an ever expanding project.
To this end, the present study entails, Iirst, a quantitative
and qualitative analysis oI coordination practices in
WikiProjects. This includes identiIying and classiIying the
myriad ways in which automated tools and manual eIIorts
are used to Iacilitate project work - including talk pages,
user scripts, and bots, as well as tools external to the
MediaWiki platIorm. Once identiIied, these tools will be
manually classiIied to more accurately determine the
spectrum oI work activities currently supported by existing
tools, to deIine the types oI structured coordination
behavior that could be more eIIectively supported with the
introduction oI a tool created expressly Ior that purpose.
Analysis oI existing tool-mediated practices will be
completed using Benklers characteristics of commons-
based peer production communities and Malone and
Crowstons coordination mechanisms for small group
maintenance |6, 18|. To discover which oI the challenges
associated with group coordination can be addressed, we
identiIied means oI coordination suggested by the prior
models that are currently not present in existing practice, as
well as classiIied active Wikipedian's current desires
gleaned Irom a Wants & Needs analysis |5|. Ultimately,

4
https://en.wikipedia.org/wiki/Wikipedia:WikiProject
this study aims to enumerate the types oI coordination
currently at work within WikiProjects through the analysis
oI the tools used and the desires oI those active within those
groups, towards the goal oI identiIying means oI better
supporting those distributed workers to maintain the Iuture
health oI the encyclopedia and nurture the thriving
community behind it.
BACKGROUND AND RELATED WORK
The Iocus oI this literature review will be on the
coordination oI small groups Irom the perspective oI
commons-based peer production communities. This will
include, Iirst, a brieI overview oI commons-based peer
production, including its provenance and utility in
interpreting activities in online communities; second, a look
at prior research into coordination in small groups,
including methods to analyze and interpret coordination
activities within small groups; and third, I will introduce
and explore recent research in analyzing mediating tools
used Ior organization and coordination in online settings.
These three sections represent the work that will Iacilitate
understanding the structure oI small group work, the
processes those groups enact to support their work, and the
speciIic tools utilized to enable more complex collaborative
processes, respectively. Each oI these will be called upon in
more detail in the sections that Iollow, and their connection
to the current study will be made salient throughout this
text.
Commons-Based Peer Production Communities
(Structure)
InCoasesPenguin,or,Linux and The Nature oI the Firm,
Benkler |1| introduces the idea oI a commons-based peer
production community as an entity distinct Irom existing
notions oI Iirms or markets, instead relying on the
creativity, intelligence, and eIIorts oI distributed individuals
to work towards a common goal. This stipulation was
extended Irom prior work done by Eben Moglen, who
introduced Moglens Metaphorical Corollary to Faradays
Lawasa stance whichstates that if you wrapthe internet
around every person on the planet and spin the planet,
soItware Ilows in the network. Itsanemergentpropertyof
connected human minds that they create things Ior one
anothers pleasure and to conquer their uneasy sense of
being too alone,
5
. Benklers twist on Moglens
metaphorical corollary posited that these peer production
communities are typiIied by two primary characteristics |2|.
First, that production is decentralized, that action within the
community is dictated by the desires, talents, and attributes
oI the members oI that community rather than a central
organizer, allowing Ior emergent goals and activities to
drive production rather than a single coordinating vision.
Second, that the community will rely on social cues and
motivations rather than explicit market prices to coordinate
community member activity. Further, and as mentioned

5
In (Benkler, 2002), p. 380.

- 2 -
Confidential Submission - Under Review - CSCW 2015
Please
make it
uniformal
Expand further on both of them
before mentioning them. Show
how this work is related and what
was missing
Explain before
mentioning
mechanisms?
Since our focus is wikipedia, why should
the literature focus on small groups?
Great!
Mention
earlier.

above, successIul commons-based peer production
communities Irequently share three core structural
attributes: Iirst, that the objects oI production are modular,
allowing Ior individual work tasks to be created Irom the
distinct, atomic needs oI a community. Second, that those
modular objects oI production are granular enough to allow
individual eIIorts to make a meaningIul impact on
production, or to allow Iurther modularization to enable
small groups to complete tasks in an eIIicient manner. And
third, successIul peer production communities will
Irequently require a low-cost means oI integrating work
objects back into the collective whole. In other words,
while the Iirst and second properties are regarding the
disaggregation oI work objects into manageable and
meaningIul tasks to be completed by distributed
contributors, the third property ensures that work can be
simply and easily aggregated back into the Iinal product.
As an example, Viegas, et al |21|, adopted Benklers
Iramework to explore the process behind which articles are
promoted to Featured Article (FA) status, a promotion that
typically involves multiple rounds oI peer review, iteration
on article improvement, and Iinally culminates in a
community decision on whether to promote the article.
Through the course oI their research they Iound that this FA
process does indeed Iollow the primary evolutionary phases
oI peer production communities laid out by Benkler,
namely, disaggregation and aggregation, but that the
mechanism that it did was somewhat novel in that both
steps are taken simultaneously - aprocessthatcoordinates
individuals efforts around quality assurance
(|aggregation|), while doing so in a distributed manner that
relies on independent modules - five minutes of human
attention([disaggregation]).
A more direct translation of Benklers framework onto a
peer production community may be possible given the work
suggested in this study in brieI, coordination activity in
WikiProjects. From this lens we can identiIy WikiProjects
as the modular work elements allowing editors to more
eIIectively locate and deIine their eIIorts with regard to
their stated membership in projects (Attribute 1). Beneath
the project level are tasks, oIten codiIied within
WikiProjects as Task Forces |9|, allowing groups to more
eIIectively distribute and organize eIIorts in a granular
Iashion (Attribute 2)
6
. Finally, the eIIorts oI these atomic
work units can be recorded in the larger encyclopedia and
the results oI that eIIort can be recorded on the project page
to aid coordination and task tracking Ior Iuture eIIorts
(Attribute 3), an integration and Ieedback mechanism that is
currently not ideally implemented, I would argue a stance

6
See also,
http://en.wikipedia.org/wiki/Wikipedia:WikiProjectCounci
l/Guide/TaskIorces, Ior a more detailed description oI
WikiProject Task Forces
that will be Iurther expanded upon and addressed in later
sections oI this study.
Coordination in SmaII Group Settings (Process)
Crowstons coordination mechanisms for small group
!"#$%&$"$'&
Coordination Theory |6, 18| deIines coordination succinctly
as managing dependencies between activities. From this
deIinition two key tenets can be immediately identiIied:
Iirst, that iI there are no dependencies in an activity, there is
nothing to coordinate. Any activity that exists in isolation
(iI such a proposition were possible), with no portion oI that
activity depending on any other to successIully complete
would not require any coordinating power, or articulation.
Second, due to the broadness oI the deIinition it is also clear
that coordination can be a part oI many types oI systems, be
they social, psychological, educational, biological, or
computational. Further, as coordination is deIined as
managing dependencies, by classifying and describing
those dependencies it is possible to more clearly delineate
the types oI coordination mechanisms that might be
possible to ameliorate those dependencies. For example, iI
one activity is dependent on the successIul completion oI
another activity, coordination mechanisms that Iacilitate
sequential tracking, notiIications, or peer review may be
useIul in driving an overall process to completion. A
summary of Malone and Crowstons dependencies and
alternative coordination mechanisms is in Table 1, above.
While a great deal oI literature within the CSCW research
community cites Coordination Theory as a necessary
advancement to more eIIectively enable group work
7
, there
have been very Iew studies that more actively tie design
processes and activity outcomes directly to the Iramework
that Malone and Crowston explicated. Through the lens oI
Coordination Theory, I suggest it will be possible to more
eIIectively deIine and explore dependencies in project
activities and, through doing so, to Iirst allow us to more
eIIectively label coordination activities within projects with
the language that has emerged Irom the theory, and second
suggest potential interventions or improvements that could
be made to those processes. Particularly relevant to the
CSCW community, while there have been many calls to
have design processes that are grounded in both
organizational, psychological, and social science theories,
there have been very Iew examples oI that type oI research
that use these theoretical Ioundations in an applied and
dynamic manner |10|.

7
For example, (Ackerman, 2000; Grudin, 1994). For a
more complete list, see http://scholar.google.com/scholar?
qcscw&btnG&hlen&assdt52C48&sciodt02C48
&cites15059460397161337063&scipsc1

- 3 -
Confidential Submission - Under Review - CSCW 2015
Why
mention
again? Be
strategic
and
mention it
where it
matters
the most!
Maybe
organizational
graph of
wikipedia
would help

Mediating TooIs in Peer Production Communities
(TooIs)
II you have a Twitter or Facebook account, you have
probably interacted with a bot. One recent study has shown
that as much as 7 oI Twitter users are non-human, or
bots
8
while another study places that estimate as high as
13.8 |3|. From a design or tools perspective, these bots
are enablers, allowing those who control them to easily
distribute thousands oI messages to Iollowers almost
instantly, potentially with the goal oI diluting a message,
spreading propaganda, or possibly even destabilizing the
election process |23|.
But these tools are not inherently malicious, or even social.
A popular user-submitted content aggregator site, Reddit, is
host to a number oI bots that perIorm cleanup,
administrative, or minor editing tasks
9
. And there are a
plethora oI bots on the previously mentioned Twitter that
exist Ior less neIarious purposes, albeit some are more
humorous than useIul
10
. In Iact, an increasingly large
amount oI the experience in many online environments is
not the result oI the primary platIorm, but a combination oI
the bots, tools, and scripts that shape our interactions with

8
http://oursocialtimes.com/7-oI-twitter-users-are-not-
human/
9
http://www.reddit.com/r/botwatch, showing news,
updates, and content related to Reddit bot operation
10
http://www.businessinsider.com/twitter-bots-2011-
6?op1, which describes, Ior instance, a bot that will send
you a tweet whenever a FedEx package is scanned or
updated, a calendar scheduling bot, or even a bot that, iI
youmentionthewordSeinfeldinapostwillreplywitha
random quote Irom the show.
those platIorms - an aggregation that Geiger has termed
bespokecode|13|.
In a platIorm such as Wikipedia, this bespoke code
comprises some estimated six million lines oI code an
order oI magnitude larger than the some 600,000 lines that
make up the core MediaWiki platIorm in the Iorm oI the
bots that patrol Ior vandalism, grammatical errors, stylistic
omissions, and the scripts that support the ongoing
maintenance oI the site, the customizable interIaces, and
even those that manage automated greetings Ior newcomers
to the community, all among many, many others. In |13| as
well as |12|, Geiger argues that these bots and bespoke
tools are increasingly ingrained in the systems that we
interact within on a daily basis, and that our experience oI
those systems is now increasingly shaped, expanded, or
constrained by those tools, and as these tools impact our
perception oI and means oI Iuture interactions with that
system they require greater and careIul consideration. More
speciIically, in |12| it is shown how pivotal these tools
actually are to the healthy Iunctioning oI the community,
that while robust and reliable quality control mechanisms
exist within Wikipedia to control and combat the problems
oI spam and vandalism, the proper Iunctioning oI these
tools is what enables the community to continually grow, as
well as maintaining the status and quality expected by users
oI the encyclopedia. Beyond that, however, Geiger argues
that these tools are no mere Iorce multipliers, simply
duplicating human eIIorts at greater speed and eIIiciency,
but through their implementation and maniIestation are in
Iact integral to the core experience users have when they
visit the site.
It is clear that these bots, these tools which shape our
perceptions oI the online spaces we occupy, are worthy oI
greater attention and exploration. But through this
exploration oI research and practice it is also evident that
!"#"$%"$&' ()*"+$,*-." &00+%-$,*-0$ 1"&2,$-31
Shared resources Firstcome,firstserve,priorityorder,budgets,managerialdecision
Task assignments (sameasforSharedresources)
Producer/Consumer
relationships

Prerequisite constraints NotiIication, sequencing, tracking
TransIer Inventorymanagement(e.g.justintime,Economicorderquantity)
Usability Standardization, ask users, participatory design
Design Ior manuIacturability Concurrent engineering
Simultaneitv constraints Scheduling, synchronization
Task/Subtask Goal selection, task decomposition
Table1:MaloneandCrowstonscommondependenciesandexamplealternativecoordinationmechanismsfordealing
with them. Originally from (Malone & Crowston, 1994).


- 4 -
Confidential Submission - Under Review - CSCW 2015

there has been a dearth oI attention paid to a section oI
CSCW literature which could show a great deal oI promise
Ior the Iuture that oI automated systems to Iacilitate
coordination in these online spaces. The current study aims
to Iill this vacuum, to chart the aIIordances and
dependencies oI an existing and active system Irom both an
applied and theoretical perspective, to more Iully explicate
the operation and nature oI that system Irom the lens oI the
tools that enable such interactions, and to ultimately
introduce the potential Ior new means oI coordination
within the system that can be measurably, empirically
tested to Iurther our knowledge oI how these online
collaborative spaces may be more eIIectively designed in
the Iuture.
Current Study
My Iocus thus Iar, and the Iocus oI much oI this work
moving Iorward, is on how tools Ior coordination in online
spaces are currently being used and on how that use may be
Iirst, better understood within the context oI those
communities and, second, how these coordinative tools may
be improved to more eIIectively Iacilitate these myriad and
complex interactions. As stated above, Malone and
Crowston define coordination as managing dependencies
between activities |18|. Geiger Iurther clariIies that
deIinition by indicating that the coordinative power oI these
tools is beyond acting in the capacity oI a mere Iorce
multiplier, indicating that the Iunctions these tools IulIill
are not simply increasing the amount oI work that
individuals can complete in these online spaces, but
Iundamentally alter the ways that individuals can complete
work in those spaces. Finally, work by Gilbert et al |14|
explored the potential Ior passive situational awareness
tools
11
to direct attention and activity oI distributed group
members, speciIically through groups adoption oI the Hot
Articles tool a means Ior WikiProject members to quickly
see the articles that are under the scope oI their project
which have received the most edits within the last Iew days.
While these situational awareness mechanisms provide a
context Ior individual action, they do not serve a
coordinative Iunction per Malone & Crowston above,
namely, by managing dependencies through explicit
individual or tool-mediated activity. As such, these
situational awareness tools will be distinct Irom those we
deIine as serving a coordinative Iunction within
WikiProjects. Given the prior considerations, we suggest
that to serve a coordinative Iunction within the space oI an

11
See (Dourish & Bellotti, 1992) Ior a concise description
oI situational awareness - Information related to these
Iactors |ie, inIormation sharing, knowledge oI group and
individual activity, and coordination| contributes towards
what we reIer to as awareness. In these terms, awareness is
an understanding oI the activities oI others, which provides
a context Ior your own activities.
online team, a tool must meet the Iollowing two primary
criteria:
First, the tool must in some way manage dependencies oI
those attempting to participate in that group work. This
deIinition extends Irom the taxonomy oI dependencies as
outlined by Malone and Crowston |18|, including the
producer/consumer relationship (notiIication oI status,
transIer oI task along dependency chain), simultaneity
constraints (eIIectively manage multiple group members
working towards identical ends), and task/subtask
dependencies (hierarchy oI tasks required Ior successIul
completion). Notably, each oI these also indicate that any
tool-mediation that occurs will require trace evidence in the
system in which they are deployed to eIIectively
communicate the nature oI the dependency, successIully
coordinated or not, to the remaining team members. As
stated above, any activity completed in isolation would
require no coordinating power or articulation.
Second, the tool must mediate action in a manner that
extends beyond being a Iorce multiplier oI existing tasks.
While these types oI tools are plentiIul, useIul, and possibly
one oI the primary reasons that Wikipedia has continued to
thrive, the ability to ampliIy existing eIIort through
reducing the eIIort to complete arduous tasks does not
impact the coordination oI remaining group members.
Moving Iorward Irom this more speciIic notion oI tool-
mediated coordination work, we suggest answering the
Iollowing primary questions through the course oI this
study:
RQ1: How can tool-mediated coordination in distributed
voluntarv virtual teams be most effectivelv mapped,
modeled? It is commonly understood that the use oI
automated and semi-automated tools in Wikipedia play an
increasingly important and active role in Wikipedia |12;
13|, perIorming tasks Irom handling vandalism |12|, task
routing |4|, and task management |17|, among many others.
But to this date there has not been a Iocused analysis on the
use oI automated and semi-automated tools used to
Iacilitate coordination in WikiProjects. This Iirst research
question aims to address this vacuum by Iirst, enumerating
the steps to eIIectively describe the ecosystem oI
coordinative automated and semi-automated tools at use
within WikiProjects by the deIinition stipulated above, and
second, through completing the data collection and analysis
to present a description oI that ecosystem in a meaningIul
and actionable way.
RQ2: Through what means can this mediation inform the
design of future technological interventions? Beyond
simply mapping the ecosystem oI automated and semi-
automated tools at use within current WikiProjects, the goal
oI this study will be to analyze current work practices Irom
the perspectives oI the Irameworks described above to
enumerate the potential Ior new tools to have a positive
impact within these spaces. That is, given existing work

- 5 -
Confidential Submission - Under Review - CSCW 2015

practices within WikiProjects and the tools available to
those project members, how may Crowstonsprinciplesfor
selI-organizing groups be used to inIorm and recommend
new design directions to Iacilitate coordination within these
groups, all Irom the unique perspective and needs oI
WikiProject members?
METHOD
Data Ior the study were collected Irom the Wikimedia
Toolserver
12
, using custom Python scripts to aggregate and
cache all revisions to the English Wikipedia on a local
MySQL database. Data recorded include all edits to the
encyclopedia bucketed by week, as well as all WikiProjects
and all pages under the scope oI each project (the prior
identiIied using the category structure present on Wikipedia
pages), and the user groups oI all users in Wikipedia (Ior
instance, to diIIerentiate between bot and non-bot
accounts). Detail on how we identiIied tool-mediated
project edits is given in the relevant section below.
In addition to this quantitative data, qualitative data were
gathered Irom a Wants & Needs analysis |5|, a user-
centered design method allowing Ior structured
brainstorming that results in a prioritized list oI user
requirements. During this analysis, data were collected
Irom nine longstanding Wikipedians regarding the types oI
inIormation they would Iind most valuable to be readily
apparent within the scope oI WikiProjects and within the
larger ecosystem oI Wikipedia as a whole |20|.
Identifying Automated and Semi-Automated WikiProject
Edits
The landscape oI Wikipedia contributions has become
increasingly complex, with automated, semi-automated, and
otherwise tool-mediated edits playing an increasingly
important role in the maintenance and ongoing health oI the
encyclopedia |11;12;13|. But due to the nature oI these
myriad tools, the largely unstructured ecosystem oI bots,
user scripts, gadgets, and external applications available to
contributors, it is oIten diIIicult to identiIy exactly which
edits were tool-mediated
13
. As such, we take a three-
pronged approach to identiIying these tool-mediated edits
with the goal oI determining what coordination mechanisms
are at work within the scope oI WikiProjects.
1) Edits by users with the bot flag: To be an oIIicially
recognized bot in Wikipedia, a unique bot account must be
created and a request must be made with the Bot Approvals
Group where the tasks that the bot aims to complete and the
code that will enact those changes can be reviewed, vetted,
and ultimately approved or rejected. Once a request is
approved, the user account will be added to the bot user

12
http://toolserver.org/
13
Geiger, personal communication,
https://lists.wikimedia.org/mailman/listinIo/wiki-research-l,
Killthebotsthreadfrom5/18/2014
group, eIIectively Ilagging that account as an oIIicial
legitimate alternative account, capable oI editing the
encyclopedia in an automated or semi-automated Iashion.
Edits by these accounts are easily identiIiable and provide a
simple starting point to aggregating the types oI activities
that automated and semi-automated users complete within
the scope oI WikiProjects.
2) Edits with standard strings in the revision comments:
While it is oIIicial policy that bots go through the oIIicial
review and approval process beIore operation, there are still
accounts which operate bots which have not been approved
and therefore do not have the bot Ilag added to the user
account, as well as semi-automated tools which Iacilitate
edits on behalI oI individual users. To identiIy these tools,
and ultimately to identiIy the Iunction that they serve within
the project space, we additionally looked at all edits within
the project scope, identiIying strings which occured
Irequently within the revision comments and tags that are
added in known tool-mediated edits. For example, tags
added in revision comments such as WP:TW,
MWT|MWT, or AWB|AWB representing edits
completed with Twinkle, Mikes Wiki Tool, and the Auto
Wiki Browser, respectively
3) Edits bv individual user accounts made in quick
succession: Finally, to identiIy tool-mediated edits that
neither originate Irom a legitimate bot account nor contain
any Iormulaic revision comment string, we will analyze all
edits to pages within the scope oI WikiProject Irom
individual users that occur in rapid succession. As Geiger
and HalIaker showed |11|, the majority oI inter-edit time in
edit sessions extends Irom seconds up to an hour, with the
greatest Irequency being around one minute. To increase
the probability that we are retrieving only tool-mediated
edits, thereIore, we identiIied all accounts which made two
or more successive edits to any page within the scope oI a
WikiProject in Iive seconds or less. Manual inspection oI
the results oI this analysis removed any edits which
ultimately did not appear to stem Irom an automated or
semi-automated tool, and are simply the result oI a user
rapidly committing multiple changes to the project.
Characterizing Automated and Semi-Automated
Participation in the WikiProject Space
The prior section determines if a bot or tool contributes to a
project. Beyond that, we aim to classiIy how each oI them
contributes to coordination within the project space. Data
analysis oI automated and semi-automated edits to the
project space will be completed in two primary ways. First,
a mixed methods analysis oI bots that have edited
WikiProject pages, subpages, and pages transcluded within
each oI them was completed which aimed to both describe
the types oI tool-mediated edits that are done to project
pages and to show to what extent those edits may have an
impact on the coordination oI group members. Second, we
completed a content analysis oI these bots, their
Iunctionality, and the boundaries oI their utility to

- 6 -
Confidential Submission - Under Review - CSCW 2015
How were they
recruited?
What were the
criteria to
choose them?
How many
were invited in
total? What
may be the
limitations
imposed to the
study due to
the selection
criteria or the
number of
participants?
Why also use
qualitative
data(promote
your work)?
How long did you
collect the data?
Which are the
starting adnd
ending dates of
the data? How
many edits on
how many pages
Were collected?
How many
projects?
What does
longstanding in
this study
mean? It would
help people
building upon
your work to
know all these
simingly minor
details. For
instance,
Someone may
do research on
differences
between
newbies and
expert users.
Semi-
automated
edits are not
tools-
mediated?

WikiProject members with particular Iocus paid to their
potential, realized or not, to coordinate activity oI those
distributed project members.
In other words, we describe quantitatively the extent and
impact oI bot edits on WikiProject pages, as well as
describe qualitatively what the actual and intended purpose
oI the bots most active on project pages is. The content
analysis oI WikiProject contributing bots was completed
using a directed content analysis approach |15| utilizing
MaloneandCrowstonscoordinationmechanismsforsmall
group maintenance |18| as a starting Iramework Ior
analysis. This directed content analysis allows Ior a more
open inquiry into the speciIic mechanisms that these bots
play in WikiProject coordination and, given their current
usage within projects, Iacilitates the distinctions between
coordination mechanisms that may be suggested by
Crowstons Coordination Theory |6| and those that are
actually used in practice, highlighting the potential Ior a
novel technological intervention into this space.
Interpreting User Needs through Focus Group Sessions
Beyond the analysis oI bot utility and activity within
WikiProject spaces, we also completed a directed content
analysis oI data recorded during a Iocus group session
involving multiple senior Wikipedians involved in the
maintenance and Iunction oI a broad spectrum oI
WikiProjects. These Iocus group data, collected during
2009 |20|, comprise the recollections oI nine Wikipedians
discussing the means by which they complete their daily
work, the problems they encounter during those daily tasks,
and their suggestions and reIlections regarding the myriad
ways in which those tasks may be improved. Data were
collected using a Wants & Needs analysis |5|, a user-
centered design method allowing Ior structured
brainstorming that results in a prioritized list oI user
requirements. Starting Irom Malone and Crowstons
coordination mechanisms Ior small group maintenance
|18|, a directed approach to content analysis was completed
which Iacilitates a more nuanced understanding oI actual
Wikipedians desires and frustrations, as well as allowing
us to more eIIectively identiIy how those desires and
Irustrations align, or Iail to align, with existing theoretical
Irameworks identiIying coordination practices in these
online spaces.
FINDINGS
Our analysis shows that while it is true that bots play a very
active role in updates to WikiProject pages and subpages,
much oI this activity is not related to the coordination oI
those projects. In the sections that Iollow we describe the
results oI identiIying bots by each oI the above methods, as
well as the Iunction each oI those bots plays within the
operation and maintenance oI each project.
Automated and Semi-Automated TooIs at Work Within
the WikiProject space
Boteditsidentifiedthroughthebotflag:

Figure 1. Density plot of bot edits to WikiProjects (left) and non-WikiProjects (right). Red represents the Wikipedia
namespace (4) and blue represents the Wikipedia Talk namespace (5). X-axis is square root of edit counts.

- 7 -
Confidential Submission - Under Review - CSCW 2015
Where the data collected also from the
same period or before that? If not, why?
How has the environment changed
since?
numerous
I would suggest
presenting the
results rst to the
reader and then
commenting on
the. Let him
decide if what you
argue is in line
with the ndings

Looking at all bot edits to WikiProject pages Ior one year
starting May 1st, 2013, it is apparent that bot edits to
WikiProject pages Iollow the same power law distribution
that much human contributions to online communities
Iollow (Figure 1, showing the density plot oI bot edits to
WikiProject related pages as well as non-WikiProject pages
with the x axis showing the square root oI individual bot
edit counts, split by Wikipedia and Wikipedia Talk
namespaces the namespaces where WikiProject pages
reside). Table 2 shows a more detailed summary oI the bots
active on WikiProject related pages, split by both
namespace (between Wikipedia and Wikipedia Talk) and
whether the edit was to a project page directly or to another
page transcluded on the project page.
In all, bot edits to WikiProject pages comprised 23.8
(507,574 edits) oI the total edits to the Wikipedia and
Wikipedia Talk namespaces, whereas the number oI
WikiProject pages, subpages, and corresponding Talk pages
make up only 16.97 (218,611 WikiProject-related pages)
oI the total pages within those namespaces. This
discrepancy, evident in the density plots in Figure 1 above,
is largely the result oI a select few bots high level of
activity across multiple projects. The top Iive bots by edit
counts to WikiProject related pages, Ior instance, make up
22.32 oI the total bot edits, with all remaining edits by
bots making up the remaining 1.48 oI the total
14
.

14
Top Iive bots by total edit counts to WikiProject visible
pages are AAlertBot (233,545 edits), WP 1.0 bot (134,001),
COIBot (70,756), JL-Bot (18,723), and Cyberbot I
(15,606).
Initial analysis oI the quantitative data clearly indicates two
primary Ieatures. First, bot activity is higher among
WikiProject related pages than non-WikiProject pages,
notably in the Wikipedia namespace (as opposed to the
Wikipedia Talk namespace). Second, although there is a
greater amount oI proportional bot activity on WikiProject
related pages, the clear majority oI that activity is coming
Irom a very select Iew bots. The nature oI those edits, and
the Iunctions they aim to serve within the WikiProject
space, will be explored Iurther in the sections that Iollow.
Non-bot edits identified through revision comments:
While there are currently no absolute means oI identiIying
tool-mediated edits within Wikipedia (see Iootnote 13
above), many oI the more common tools used within
Wikipedia leave identiIiable traces through either user
names (Ior instance, AntiVandalBot, OrphanBot, or
PseudoBot) or in the revision comment leIt with an edit (Ior
instance, Ior rollbacks, Huggle, Twinkle). To identiIy
traces oI these tool-mediated edits, we again reIer to the
data collected oI all revisions to WikiProject pages,
subpages, corresponding Talk namespace pages, and all
pages transcluded on any oI the above Ior the year
beginning May 1
st
, 2013.
Wikipedia Namespace
name edits is_project_edit
1: AAlertBot 160357 1
2: WP 1.0 bot 133199 0
3: AAlertBot 73188 0
4: COIBot 70756 1
5: Cyberbot I 15587 0
6: JL-Bot 15509 1
7: Mr.Z-bot 5525 1
8: TedderBot 4784 0
9: JL-Bot 3135 0
10: HotArticlesBot 1748 1
11: HotArticlesBot 1654 0
12: Scsbot 1419 1
13: Mathbot 1339 1
14: SineBot 1083 1
15: WP 1.0 bot 802 0
16: Theo's Little Bot 772 1
17: WoodwardBot 491 1
18: Theo's Little Bot 478 0
19: Legobot 395 0
20: CommonsDelinker 292 1
Wikipedia Talk Namespace
name edits is_project_edit
1: MiszaBot II 8259 1
2: SineBot 1379 1
3: ClueBot III 1273 1
4: Legobot 1048 1
5: Mr.Z-bot 360 1
6: Scsbot 299 1
7: Theo's Little Bot 143 1
8: SporkBot 132 1
9: Yobot 85 1
10: AvicBot 77 1
11: Xqbot 47 1
12: EmausBot 33 1
13: RFC bot 23 1
14: BattyBot 7 1
15: Lowercase sigmabot 6 1
16: JL-Bot 4 1
17: Amalthea (bot) 3 1
18: DumbBOT 3 1
19: HasteurBot 3 1
20: OgreBot 3 1
Table 2: The top bot edits to WikiProject pages, subpages, corresponding Talk pages, and templates transcluded on any
of those pages for the period from 05/01/2013 to 05/01/2014. The primary Wikipedia namespace is on the left (showing
20 of 82 total bots who edited during that period) and the Wikipedia Talk namespace to the right (showing 20 of 25 total
bots who edited). Theis_project_editcolumnforeachtableindicatesthattheeditwasmadedirectlytoaWikiProject
related page (1) or that it was made to a template which was transcluded on that project page (0).


- 8 -
Confidential Submission - Under Review - CSCW 2015
Align columns

The resulting data set included 346,008 total revisions to
WikiProject related pages during the time period. Revision
comments and user names were compared against a list
reIerenced in the prior Iootnote
15
, Iacilitating a quantitative
analysis oI all automated and semi-automated (tool-
mediated) revisions present in the data under analysis. This
analysis yielded only 4,874 total tool-mediated edits within
the original set oI over 346,000. ForeIront among the tool-
mediated edits within the WikiProject space was
AutoWikiBrowser, a semi-automated editor that allows
users to make multiple edits more quickly and easily than iI
done by hand, simpliIying the process oI editing Ior many
more tedious tasks. Second among the top tool-mediated
editswasundo,abuilt-in Iunctionality that enables users
to easily, with one click, undo a prior revision on any
Wikipedia page, Iollowed by Twinkle, a tool that attempts
to automate many Irequent tasks such as listing a page Ior
speedy deletion, tagging a page Ior protection, or jumping
to the diII oI previous revisions oI the current page.
Combined, these tools represent 96.3 oI the tool-mediated
revisions identiIied within the WikiProject space Ior the
period under analysis (see Figure 2 Ior a breakdown oI tools
identiIied), however altogether these tool-mediated edits
still only represented 1.41 oI the total edits to pages
within the WikiProject space.
Non-bot edits identified through successive revisions:
A third means oI identiIying automated and semi-
automated contributions to the WikiProject space is by
limiting the edits under analysis to those which were

15
List identiIying Irequent tool and bot edits through trace
data at
https://wiki.toolserver.org/view/MySQLqueries#Automate
dtoolandbotedits
committed in under a threshold that would exceed the
committing habits oI most typical Wikipedians. For our
analysis, and inIormed by prior work by Geiger and
HalIaker |11|, we selected a threshold oI two or more
consecutive edits in under Iive seconds to be a reasonable
cutoII. Using the above dataset oI all non-bot revisions to
pages within the WikiProject space, including Talk pages
and pages transcluded within WikiProjects, we reduced the
initial set oI 346,008 revisions to 25,851 which met the
timing criteria. Through manual sorting and selection oI
Irequently occurring strings we identiIied 26 strings within
revision comments which accounted Ior 97 oI the total
revisions present in the reduced data set.
Among the top comments identiIied were
[[WP:FWDS|FWDS]] (the trace of a deletion sorting
tool), references to archiving, moved page, Updating
user statistics (a trace left by an administrator completing
batch updates), and AWB|AWB (the previously
introduced AutoWikiBrowser). Figure 3 shows a more
detailed view oI the top comment strings and the extent to
which each was Iound within the data set, representing the
top ~94 oI strings by edit count.

Figure 2. Chart showing the most frequent tool-mediated edits to
the WikiProject space, as identified through revision comments
and user names.

Figure 3. Chart showing relative distribution of comment strings
found in revisions which occurred > 5 seconds apart from one
another.

- 9 -
Confidential Submission - Under Review - CSCW 2015
Aforementioned

There are two notable Iindings Irom the above. First, is the
lack oI identiIiable tool markers Iound in the revision
comments. While FWDS and AWB were clearly
delineated, others were identiIied only by task (archiving or
updating user statistics), by project (WikiProject Deletion
Sorting) or not at all (447 oI the revisions which occurred
within Iive seconds oI each other had no revision comment
at all). Second, is the nature oI the tools and bots identiIied
in this and the prior analyses. Namely, while the tools
identiIied do !"#$%&" the work oI Wikipedians working
within the project space, they do not '(()#$*%&" that work
by the stipulation above. SpeciIically, while these tools can
ameliorate the hassle oI tedious and repetitive tasks they do
not manage dependencies between activities, |6|. While
they perIorm work that is valuable and necessary to the
continued health and operation oI the community, including
the longevity and sanity oI those who have taken on the
responsibility oI maintaining that community, they are
primarily acting as force multipliers rather than as a
means to Iacilitate coordination and work articulation
between group members. And although these concurrent
edits make up a much greater percentage oI the total
revisions to the WikiProject space compared to the tool-
mediated edits analyzed above (7.47 compared to 1.41),
neither oI the examinations excavated tools that serve in a
coordinative Iunction as previously described. The
Iollowing section will analyze with greater detail the
Iunction oI the bots at play within the WikiProject space as
well as the coordinative Iunction each serves.
RoIes of the Bots Active within the WikiProject Space
In our analysis oI bot edits to WikiProject pages, subpages,
Talk pages, and all pages included in the above Ior the year
starting May 1
st
, 2013, we identiIied a list oI 49 unique bots
that made at least one edit (min: 1, median: 94, mean:
10360, max: 233500, sd: 38996). And while the most active
oI these bots within the realm oI WikiProjects had a
signiIicant number oI revisions within the project space,
very Iew oI them actually serve in any coordinative Iashion
within that space.
Through a directed content analysis |15| starting with
Crowstons coordination mechanisms Ior small group
maintenance |6| as a means oI directing analysis, only one
of Crowstons dependencies was managed through the
operation oI the tool per the Iramework, the
Producer/Consumer relationship may be coordinated
through successIul management oI prerequisite constraints
by means including notiIication, sequencing, or tracking.
As such, bots which manage notiIication oI project or
article events requiring Iurther action, such as AAlertBot
which alerts when pages move in or out oI certain
workIlows such as Article Ior Deletion or Request Ior
Comments, successIully mediate activity between those
whochangeapagesstate(theProducer)andthosewhoare
in a position to take action given that new state (the
Consumer).
This must be diIIerentiated Irom bots which increase
visibility or situational awareness oI project activity without
serving a direct coordinative purpose. This may include
bots such as the HotArticlesBot which maintains lists oI the
top edited articles within the scope oI individual
WikiProjects or bots such as Mathbot, SvickBOT, or
WoodwardBot which serve more as an inIormation
resource through updating article lists and project statistics.
Figure 5 displays the spectrum oI bot activities as classiIied
by our analysis. Notably, the only dependency included in
Crowstons framework mediated by bot activity is the
Producer/Consumer relationship, and that only through the
explicit or implicit notiIication mechanisms at play when a
bot records a state change in an article or WikiProject status
with an expected action to occur as a result oI that change,
as in AAlertBot above. The other two groups,
Visibility/Awareness and Force Multiplier, would not be
included in Crowstons framework or be covered by our
stipulation oI coordination.
Tracking the Needs of Wikipedians
Given that so much oI the bot-driven work in Wikipedia is
not designed to support coordinative Iunctions, it appears
there may be an opportunity to develop such automated
agents Ior the collective eIIort oI Wikipedians. To ground
our thinking about this potential need in the community oI
Wikipedia editors, we conducted two extensive
requirements gathering group interviews with experienced
editors |20|. Those sessions generated rank-ordered lists oI
the kinds oI system improvements that these Wikipedians
most desired. In an analysis oI that data, motivated by the
desire to know how many system-Iocused desires were
related to means or tools Ior coordination, we discovered a
constellation oI community needs. In total, we Iound 24

Figure 4. The spectrum of coordinative functions
encompassed by bots active in the WikiProject space. Only
thefirst,Producer/Consumer,isreflectedinCrowstons
coordination framework.

- 10 -
Confidential Submission - Under Review - CSCW 2015

unique needs Ior coordination mechanisms (this excludes
an additional 13 unique needs Ior awareness mechanisms).
In our assessment oI these 24 coordination mechanisms
desired by the community, we noted that all could be
Iacilitated through new tools, either embedded in Wikipedia
itselI, ormore likelythrough supplemental (bespoke)
code. Many oI these desires could readily be addressed by
automated processes (e.g., Irequency oI being warned,
number oI compliments received) and many others could be
handled by semi-automated processes (e.g., work in
mentoring new editors).
DISCUSSION
Through the process oI the analysis above we have hoped to
conIront two primary research questions. First, how can
coordination in online spaces be most eIIectively mapped
and modeled, and second, how might the insight gained
Irom that analysis inIorm and direct the design oI tools
intended to Iacilitate continued coordination within those
online spaces in which the analysis took place? Throughout
we have argued that while myriad bots, tools, and scripts
exist to ameliorate the act oI contributing to these online
spaces, and WikiProjects in particular, the tools currently in
use within these spaces do not currently IulIill the spectrum
oI coordinative Iunctions shown by prior research to ensure
the optimal Iunctioning oI these types oI small groups.
Further, we have supported this stance Irom an empirical
analysis oI the automated and semi-automated tools used
within and beyond the scope oI WikiProjects, highlighting
the means in which the data was collected, the methods by
which that data was interpreted, and the theoretically based
justiIication Ior the suggestions that resulted namely, that
given the complex nature oI the systems many oI these
tools are created to support, there exists the potential Ior
more Iocused interventions to directly and positively impact
the ability oI these online groups to collaborate in a more
successIul and sustainable manner.
Returning for a moment to Benkler and Nissenbaums
primary structural attributes oI commons-based peer
production communities, speciIically the third attribute
stating that the mechanisms by which individual eIIorts are
integrated back into the whole should be eIIicient and low-
cost. From the lens oI WikiProjects, this can be seen as the
means by which individual editors are able to maintain
awareness oI project activities, project needs, project
members, and how they are able to act upon each oI these
needs. Currently there is a vacuum in tools aimed at
Iacilitating these project level interactions. The intervention
suggested in this work and the analysis leading to its
creation aims to Iill this vacuum, to Ioster a peer production
community that is resilient, adaptive, and transparent.
CONCLUSION
The primary contributions oI this work are two-Iold: Iirst,
while signiIicant research has been completed to understand
the role oI bots and tools in Wikipedia (see, Ior instance,
|12; 13|), there has been much less Iocused attention on the
use oI those bots and tools as a means oI structured
coordination within WikiProjects. This includes an
examination oI current practice with respect to the
theoretical and practical underpinnings oI what
coordination is, how it is enacted, and what potential there
is Ior Iuture interventions into that coordinative practice.
Second, given this more Iocused understanding oI
mediating tools Ior structured collaboration within
WikiProjects, the potential Ior design interventions to
Iacilitate project goals and activity has been more clearly
articulated, providing a roadmap Ior developers and
researchers to interrogate small group Iunctioning in online
spaces and suggest more eIIective and directed
interventions Ior those groups.
While the Iocus Ior this analysis oI coordination in online
teams has been on WikiProjects within Wikipedia, I expect
the knowledge gained Irom these investigations to be
immediately relevant to other similarly distributed teams
with regard to both the theory and practice oI coordination.
What these investigations aim to accomplish is not
inherently to improve coordination within WikiProjects, but
to more clearly deIine what coordination is within online
spaces, to expand on our knowledge oI its measurement, its
Iunction, and its evaluation.
ACKNOWLEDGMENTS
This work has been supported by
.
REFERENCES
1. Benkler,Y.(2002).CoasesPenguin,or,LinuxandThe
Nature oI the Firm. The Yale Law Journal, 112(3), 369
446. Retrieved Irom
http://www.utexas.edu/law/journals/tlr/sources/Volume
92/Issue 2/Hemel/DSB/Hemel.In211.01.SB.pdI
2. Benkler, Y., & Nissenbaum, H. (2006). Commonsbased
Peer Production and Virtue*. Journal oI Political
Philosophy, 14(4), 394419. Retrieved Irom
http://onlinelibrary.wiley.com/doi/10.1111/j.1467-
9760.2006.00235.x/Iull
3. Chu, Z., Gianvecchio, S., Wang, H., & Jajodia, S.
(2010). Who is tweeting on Twitter: human, bot, or
cyborg? Proceedings oI the 26th Annual Computer
Security Applications ConIerence, 2130. Retrieved
Irom http://dl.acm.org/citation.cIm?id1920265
4. Cosley, D., Frankowski, D., & Kiesler, S. (2005). How
oversight improves member-maintained communities.
Proceedings oI the SIGCHI ConIerence on Human
Factors in Computing Systems - CHI05Human
Factors in Computing Systems, 11.
doi:10.1145/1054972.1054975
5. Courage, C., Baxter, K. (2004). Understanding Your
Users: A Practical Guide to User Requirements
Methods, Tools, and Techniques. Morgan KauImann
Publishers Inc. San Francisco, CA, USA. 2004.

- 11 -
Confidential Submission - Under Review - CSCW 2015
(2006)

6. Crowston, K. (2003). A taxonomy oI organizational
dependencies and coordination mechanisms. In T.
Malone, K. Crowston, & G. Herman (Eds.), Organizing
Business Knowledge: The MIT Process Handbook (pp.
85108). Cambridge, MA: MIT Press. Retrieved Irom
http://ccs.mit.edu/papers/CCSWP174.html
7. Dourish, P., & Bellotti, V. (1992). Awareness and
coordination in shared workspaces. Proceedings oI the
1992ACMConferenceon,(November),107114.
Retrieved Irom
http://dl.acm.org/citation.cIm?id143468
8. Forte, A., & Bruckman, A. (2008). Scaling consensus:
Increasing decentralization in Wikipedia governance.
Hawaii International ConIerence on System Sciences,
Proceedings oI the 41st Annual ConIerence, 1517.
Retrieved Irom
http://ieeexplore.ieee.org/xpls/absall.jsp?arnumber44
38860
9. Forte, A., Larco, V., & Bruckman, A. (2009).
Decentralization in Wikipedia governance. Journal oI
Management InIormation Systems, 26, 4972. Retrieved
Irom
http://mesharpe.metapress.com/index/p44g2544m94715
14.pdI
10. Forte, a., & Lampe, C. (2013). DeIining, Understanding,
and Supporting Open Collaboration: Lessons From the
Literature. American Behavioral Scientist, 57(5), 535
547. doi:10.1177/0002764212469362
11. Geiger, R., & HalIaker, A. (2013). Using edit sessions to
measure participation in wikipedia. Proceedings oI the
2013 ConIerence on Computer Supported Cooperative
Work, 861869. Retrieved Irom
http://dl.acm.org/citation.cIm?id2441873
12. Geiger, R., & HalIaker, A. (2013). When the levee
breaks:withoutbots,whathappenstoWikipedias
qualitycontrolprocesses?WikiSym13Proceedingsof
the 9th International Symposium on Wikis and Open
Collaboration, 05. Retrieved Irom
http://dl.acm.org/citation.cIm?id2491061
13. Geiger, R. S. (2014). Bots, bespoke, code and the
materiality oI soItware platIorms. InIormation,
Communication & Society, 115.
doi:10.1080/1369118X.2013.873069
14. Gilbert, M., Morgan, J., McDonald, D. W., & Zachry,
M. (2013). Managing complexity: strategies Ior group
awareness and coordinated action in Wikipedia.
Proceedings oI the 9th International Symposium on
Open Collaboration, 18. Retrieved Irom
http://dl.acm.org/citation.cIm?id2491060
15. Hsieh, Hsiu-Fang, & Shannon, Sarah E. (2005). Three
approaches to qualitative content analysis. Qualitative
Health Research, 15(9), 127788.
doi:10.1177/1049732305276687
16. Kittur, A., Chi, E., Pendleton, B. A., Suh, B., &
Mytkowicz, T. (2007). Power oI the Iew vs. wisdom oI
the crowd: Wikipedia and the rise oI the bourgeoisie.
World Wide Web, 1(2), 19. Retrieved Irom
http://edouard-lopez.com/Iac/ICPS -
S7/Complexity/2008-Wikipedia-As-A-Complex-
System/Power oI the Few vs. Wisdom oI the Crowd:
Wikipedia and the Rise oI the Bourgeoisie.pdI
17. Krieger, M., Stark, E., & Klemmer, S. (2009).
Coordinating tasks on the commons: designing Ior
personalgoals,expertiseandserendipity.CHI09
Proceedings oI the SIGCHI ConIerence on Human
Factors in Computing Systems, 14851494. Retrieved
Irom http://dl.acm.org/citation.cIm?id1518927
18. Malone, T. W., & Crowston, K. (1994). The
interdisciplinary study oI coordination. ACM
Computing Surveys, 26(1), 87119.
doi:10.1145/174666.174668
19. Morgan, J., Gilbert, M., McDonald, D. W., & Zachry,
M. (2014). Editing beyond articles: diversity &
dynamics oI teamwork in open collaborations.
Proceedings oI the 17th ACM ConIerence on Computer
Supported Cooperative Work & Social Computing,
550563. Retrieved Irom
http://dl.acm.org/citation.cIm?id2531654
20. Searle, E., Derthick, K., Ferro, T., Morgan, J. T., &
Zachry, M. (2009). Using User-Centered Design to
Discover Motivation and Priorities in an Established
OnlineCommunity.SIGDOC09,313314. doi:ACM
978-1-60558-559-8/09/10
21. Viegas, F., Wattenberg, M., & McKeon, M. (2007). The
hidden order oI Wikipedia. Online Communities and
Social Computing, 09. Retrieved Irom
http://link.springer.com/chapter/10.1007/978-3-540-
73257-049
22. Voss, J. (2005). Measuring wikipedia, 2005. In
International ConIerence oI the International Society Ior
Scientometrics and InIormetrics: 10th, Stockholm
(Sweden) (pp. 2428). Retrieved Irom
http://eprints.rclis.org/handle/10760/6207
23. Waugh, B., Abdipanah, M., Hashemi, O., Rahman, S.
A., & Cook, D. M. (2013). The InIluence and Deception
oI Twitter: The Authenticity oI the Narrative and
Slacktivism in the Australian Electoral Process.
Australian InIormation WarIare and Security
ConIerence, 2838. Retrieved Irom
http://ro.ecu.edu.au/isw/54/



- 12 -
Confidential Submission - Under Review - CSCW 2015