See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/264123205

The Rise of Social Bots

Article in Communications of the ACM · July 2014

DOI: 10.1145/2818717 · Source: arXiv

CITATIONS READS

140 942

5 authors, including:

Emilio Ferrara Onur Varol

University of Southern California Northeastern University
113 PUBLICATIONS 1,848 CITATIONS 26 PUBLICATIONS 397 CITATIONS

SEE PROFILE SEE PROFILE

Filippo Menczer Alessandro Flammini

Indiana University Bloomington Indiana University Bloomington
205 PUBLICATIONS 7,652 CITATIONS 107 PUBLICATIONS 5,226 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

River Networks View project

DESPIC View project

All content following this page was uploaded by Onur Varol on 25 July 2014.

The user has requested enhancement of the downloaded file.

 XX
The Rise of Social Bots
EMILIO FERRARA, Indiana University
ONUR VAROL, Indiana University
CLAYTON DAVIS, Indiana University
FILIPPO MENCZER, Indiana University
ALESSANDRO FLAMMINI, Indiana University

The Turing test asked whether one could recognize the behavior of a human from that of a computer al-
gorithm. Today this question has suddenly become very relevant in the context of social media, where text
constraints limit the expressive power of humans, and real incentives abound to develop human-mimicking
software agents called social bots. These elusive entities wildly populate social media ecosystems, often go-
arXiv:1407.5225v1 [cs.SI] 19 Jul 2014

ing unnoticed among the population of real people. Bots can be benign or harmful, aiming at persuading,
smearing, or deceiving. Here we discuss the characteristics of modern, sophisticated social bots, and how
their presence can endanger online ecosystems and our society. We then discuss current efforts aimed at
detection of social bots in Twitter. Characteristics related to content, network, sentiment, and temporal pat-
terns of activity are imitated by bots but at the same time can help discriminate synthetic behaviors from
human ones, yielding signatures of engineered social tampering.

The rise of the machines

Bots (short for software robots) have been around since the early days of computers:
one compelling example is that of chatbots, algorithms designed to hold a conversa-
tion with a human, as envisioned by Alan Turing in the 1950s [Turing 1950]. The
dream of designing a computer algorithm that passes the Turing test has driven ar-
tificial intelligence research for decades, as witnessed by initiatives like the Loebner
Prize, awarding progress in natural language processing.1 Many things have changed
since the early days of AI, when bots like Joseph Weizenbaum’s ELIZA [Weizenbaum
1966] —that employed a Rogerian approach to fool the interlocutor— were developed
as proofs-of-concept or for delight.
Today, social media ecosystems populated by hundreds of millions of individuals
present real incentives —including economic and political ones— to design algorithms
that exhibit human-like behavior. Such ecosystems also raise the bar of the challenge,
as they introduce new dimensions to emulate in addition to content, including the so-
cial network, temporal activity, diffusion patterns and sentiment expression. A social
bot is a computer algorithm that automatically produces content and interacts with
humans on social media, trying to emulate and possibly alter their behavior. Social
bots have been circulating on social media platforms for a few years [Lee et al. 2011;
Boshmaf et al. 2011].

1 www.loebner.net/Prizef/loebner-prize.html

This work is supported by the National Science Foundation (grant CCF-1101743), by DARPA (grant
W911NF-12-1-0037), and by the James McDonnell Foundation. The funders had no role in study design,
data collection and analysis, decision to publish, or preparation of the manuscript.

Corresponding author: E. Ferrara (ferrarae@indiana.edu)

Author’s addresses: E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini.
Center for Complex Networks and System Research.
School of Informatics and Computing.
Indiana University Bloomington.
919 E. 10th Street, Bloomington (IN) 47408 - USA
XX:2 E. Ferrara et al.

Engineered social tampering

What are the intentions of social bots? Some of them are benign and, in principle, in-
nocuous or even helpful: this category includes, for example, social bots that automat-
ically aggregate content from various sources (like simple news feeds), and automatic
responders to inquiries, increasingly adopted by brands and companies for customer
care. Although this type of bots are designed to provide a useful service, they can some-
times be harmful, for example when they contribute to the spread of unverified infor-
mation or rumors. A notable example is that of the Boston marathon bombing [Cassa
et al. 2013], when false accusations spread widely on Twitter mostly due to bots auto-
matically retweeting posts without verifying the facts or checking the credibility of the
source.
With every new technology comes abuse, and social media are no exception. A second
category of social bots includes malicious entities designed specifically with the pur-
pose to harm. These bots mislead, exploit, and manipulate social media discourse with
rumors, spam, malware, misinformation, political astroturf, slander, or even just noise.
This results in several levels of societal harm. For example, bots used in political astro-
turf artificially inflate support for a candidate; their activity can endanger democracy
by influencing the outcome of elections. In fact, this might already have happened: dur-
ing the 2010 U.S. midterm elections, a small set of social bots started a smearing cam-
paign against one U.S. Senate candidate for Delaware, injecting thousands of tweets
pointing to a pernicious website reporting unverified information, aimed at slander-
ing this candidate and support his opponent [Ratkiewicz et al. 2011a]. A similar case
was reported for the Massachusetts special election of 2010 [Metaxas and Mustafaraj
2012]. In the latter case, the target candidate lost the elections, even if it is impossible
to determine whether and to what extent this could be attributed to these bots. So-
phisticated bots can generate personas that are more credible as fake followers, and
thus harder for simple filtering algorithms to detect. They make for valuable entities
on the follower market: recent allegations regarding the acquisition of fake followers
have involved Mitt Romney,2 Newt Gingrich,3 Barack Obama,4 and other exponents of
the international political scene.
More examples of the devastating effects of social bots are reported every day by
journalists, analysts and researchers from all over the world. A recent orchestrated
bot campaign successfully created the appearance of a sustained discussion about a
tech company. Automatic trading algorithms, now trained to use Twitter signals to
predict the stock market [Bollen et al. 2011], picked up this conversation and started
trading heavily the company’s stocks. This resulted in a 200-fold increase in market
value, bringing the company’s worth to 5 billion dollars.5 The analysts realized too
late the orchestration behind this operation. This event, however, may not be unprece-
dented: on May 6, 2010 a flash crash occurred in the U.S. stock market, when the
Dow Jones plunged over 1,000 points (about 9%) within minutes —the biggest one-day
point decline in history. Researchers speculated wildly about the causes of the disas-
ter, and trading interference due to social bots could not be ruled out [Hwang et al.
2012]. A similar effect on the stock market was recorded when, on April 23, 2013, the
Syrian Electronic Army hacked into the Associate Press Twitter account and posted a

2 Fake Twitter accounts may be driving up Mitt Romney’s follower number — www.theguardian.com/world/
2012/aug/09/fake-twitter-accounts-mitt-romney
3 Update: Only 92% of Newt Gingrich’s Twitter Followers Are Fake — gawker.com/5826960/
update-only-92-of-newt-gingrichs-twitter-followers-are-fake
4 Pols have a #fakefollower problem — www.politico.com/story/2014/06/twitter-politicians-107672.html
5 The Curious Case of Cynk, an Abandoned Tech Company Now Worth $5 Billion — mashable.com/2014/07/
10/cynk
The Rise of Social Bots XX:3

false rumor about a terror attack on the White House, in which President Obama was
allegedly injured.6

The bot effect

These anecdotes illustrate few harmful effects of social bots on our increasingly in-
terconnected society. In addition to potentially endangering democracy, causing panic
during emergencies, and affecting the stock market, social bots can harm our society
in even subtler ways. A recent study demonstrated the vulnerability of social media
users to a social botnet designed to expose private information, like phone numbers
and addresses [Boshmaf et al. 2011]. This kind of vulnerability can be exploited by
cybercrime and cause the erosion of trust in social media [Hwang et al. 2012]. Bots can
also hinder the advancement of public policy, by creating the impression of a grassroots
movement of contrarians [Ratkiewicz et al. 2011b], or polarize the political discussion
driving attention only on specific, biased sources [Conover et al. 2011]. They can al-
ter the perception of social media influence, artificially enlarging the audience of some
people [Edwards et al. 2014], or they can ruin the reputation of a company, for com-
mercial or political purposes [Messias et al. 2013]. A recent study demonstrated that
emotions are contagious on social media [Kramer et al. 2014]: elusive bots acting unno-
ticed in a population of unaware humans could easily manipulate them and alter their
perception of reality, with unpredictable results. Indirect effects of social bot activity
include the alteration of social media analytics, adopted for various purposes such as
TV ratings,7 expert finding [Wu et al. 2013], and scientific impact measurement.8
For all these reasons, we deem it crucial to design advanced methods to automati-
cally detect social bots (or to discriminate between humans and bots). The research in
this direction has just started: some groups are trying to reverse-engineer social bots
to understand their functioning [Freitas et al. 2014], while others are creating bots
themselves [Hwang et al. 2012; Briscoe et al. 2014] to study the susceptibility of people
to their influence [Boshmaf et al. 2011; Wagner et al. 2012]. The strategies currently
employed by social media services appear inadequate to contrast this phenomenon.

Act like a human, think like a bot

One of the greatest challenges of bot detection in social media is understanding what
modern social bots can do. Early bots mainly performed one type of activity: posting
content automatically. These bots were as naive as easy to spot by trivial detection
strategies. In 2011, James Caverlee’s team at Texas A&M University implemented a
honeypot trap that managed to detect thousands of social bots [Lee et al. 2011]. The
idea was simple and effective: the team created a few Twitter accounts whose role
was solely to create nonsensical tweets with gibberish content, in which no human
would ever be interested. However, these accounts attracted many followers. Further
inspection confirmed that the suspicious followers were indeed social bots trying to
grow their social circles.
In recent years Twitter bots have become increasingly sophisticated, making their
detection more difficult. Our replication of Caverlee’s honeypot strategy —and more
sophisticated variants— yielded only a handful of bots in 2013. The boundary between
human-like and bot-like behavior is now fuzzier. For example, social bots can search

6 Syrian hackers claim AP hack that tipped stock market by $136 bil-
lion. Is it terrorism? — www.washingtonpost.com/blogs/worldviews/wp/2013/04/23/
syrian-hackers-claim-ap-hack-that-tipped-stock-market-by-136-billion-is-it-terrorism/
7 Nielsen’s New Twitter TV Ratings Are a Total Scam. Here’s Why. — defamer.gawker.com/
nielsens-new-twitter-tv-ratings-are-a-total-scam-here-1442214842
8 altmetrics: a manifesto — altmetrics.org/manifesto/
XX:4 E. Ferrara et al.

Table I. Classes of features extracted by Bot or Not?

Class Number of features Description
Network 112 Network features capture various dimensions of information diffu-
sion patterns. We build networks based on retweets, mentions, and
hashtag co-occurrence, and extracts their statistical features. Exam-
ples include degree distribution, clustering coefficient, and centrality
measures.
User 56 User features are based on Twitter meta-data related to an account,
including language, geographic locations, and account creation time.
Friends 208 Friend features include descriptive statistics relative to an account’s
social contacts (followees), such as the median, moments, and entropy
of the distributions of their number of followers, followees, posts, and
so on.
Timing 24 Timing features capture temporal patterns of content generation
(tweets) and consumption (retweets); examples include the signal
similarity to a Poisson process [Ghosh et al. 2011], the average time
between two consecutive posts, and such.
Content 411 Content features, are based on linguistic cues computed through
natural language processing, especially part-of-speech tagging; ex-
amples include the frequency of verbs, nouns, and adverbs in the
phrases produced by the account.
Sentiment 339 Sentiment features are built using general-purpose and Twitter-
specific sentiment analysis algorithms, including happiness, arousal-
dominance-valence, and emotion scores [Golder and Macy 2011;
Bollen et al. 2011].

the Web for information and media to fill their profiles, and post collected material
at predetermined times, emulating the human temporal signature of content produc-
tion and consumption —including circadian patterns of daily activity and temporal
spikes of information generation [Golder and Macy 2011]. They can even engage in
more complex types of interactions, such as entertaining conversations with other peo-
ple, commenting on their posts, and answering their questions [Hwang et al. 2012].
Some bots specifically aim to achieve greater influence by gathering new followers and
expanding their social circles; they can search the social network for popular and influ-
ential people and follow them or capture their attention by sending them inquiries, in
the hope to be noticed [Aiello et al. 2012]. To acquire visibility, they can infiltrate popu-
lar discussions, generating topically-appropriate —and even potentially interesting—
content, by identifying relevant keywords and searching online for information fitting
that conversation [Freitas et al. 2014]. After the appropriate content is identified, the
bots can automatically produce responses through natural language algorithms, pos-
sibly including references to media or links pointing to external resources. Other bots
aim at tampering with the identities of legitimate people: some are identity thieves,
adopting slight variants of real usernames, and stealing personal information such as
pictures and links. Even more advanced mechanisms can be employed; some social
bots are able to “clone” the behavior of legitimate people, by interacting with their
friends and posting topically similar content with similar temporal patterns.
Bot or Not?
At the beginning of 2014, we embarked on the design of a social bot detection frame-
work for Twitter, called Bot or Not?. The main idea was to identify several classes of
features that help recognize and separate bot-like from human-like behavior. Previ-
ous familiarity with Twitter bots [Ratkiewicz et al. 2011b] allowed us to isolate six
classes of features, summarized in Table I. Overall, our system generates more than
one thousand features used to learn human and bot prototypes.
To classify an account as either social bot or human, the model must be trained with
instances of both classes. Finding and labeling many examples of bots is challenging.
The Rise of Social Bots XX:5

Fig. 1. Classification performance of Bot or Not? for four different classifiers. The classification accuracy is
computed by 10-fold cross validation and measured by the area under the receiver operating characteristic
curve (AUROC). The best score, obtained by Random Forest, is 95%.

As a proof of concept, we used the list of social bots identified by Caverlee’s team. We
used the Twitter Search API to collect up to 200 of their most recent tweets and up
to 100 of the most recent tweets mentioning them. This procedure yielded a dataset of
15 thousand manually verified social bot accounts and over 2.6 million tweets. Caver-
lee’s list also contains legitimate (human) accounts. The same procedure resulted in
a dataset of counterexamples with 16 thousand people and over 3 million tweets. We
used this dataset to train the social bot detection model and benchmark its perfor-
mance.
Bot or Not? achieves very promising detection accuracy (see Fig. 1). Some feature
classes, like the user meta-data, appear more revealing and they can be easily ex-
plained (see Fig. 2). Note that such performance evaluation is based on Caverlee’s
dataset from 2011; we are already aware of more recent social bots that cannot be re-
liably detected. Bots are continuously changing and evolving. Further work is needed
to identify newer annotated instances of social bots at scale.
Borderline cases also exist, such as accounts (sometimes referred as cyborgs) that
are a mix of humans and social bots (e.g., when humans lend accounts to bots), or
hacked accounts [Zangerle and Specht 2014]: detecting these anomalies is currently
impossible.
To make the detection system broadly accessible, we developed a Web-based applica-
tion that interfaces with the Twitter API and retrieves the most recent activity of any
account, to make a determination of whether that account exhibits bot-like or human-
like behavior. The Web interface, depicted in Fig. 3, allows one to inspect any active
Twitter account. Data about that account and its contacts are collected and processed
in real time. The classifier trained on all feature classes provides a likelihood score
that the account is a social bot. For instance, in Fig. 3, we see that one of the authors is
deemed less likely to be a bot compared to an algorithm developed by Sune Lehmann’s
XX:6 E. Ferrara et al.

Fig. 2. Subset of user features that best discriminate social bots from humans. Bots retweet more than
humans and have longer user names, while they produce fewer tweets, replies and mentions, and they are
retweeted less than humans. Bot accounts also tend to be more recent.

team.9 The system also presents disaggregate scores according to models trained on
each feature class independently.
Often, an account may be classified as a social bot according to some feature classes,
but not according to others. This is due to the large heterogeneity of features exhibited
by people —some may have bot-like features, for example their meta-data or friend
information.
In addition to the classification results, Bot or Not? provides a variety of visualiza-
tions that capture some insights about the features exploited by the system. Examples
are displayed in Fig. 4. We invite the reader to explore these interactive visualizations
directly at truthy.indiana.edu/botornot.

Master of puppets
If social bots are the puppets, additional efforts will have to be directed at finding
the “masters.” Governments10 and other entities with sufficient resources11 have been
alleged to use social bots to their advantage. Even assuming the availability of effective
detection technologies, it will be crucial to reverse-engineer the observed social bot
strategies: who they target, how they generate content, when they take action, and

9 You are here because of a robot — sunelehmann.com/2013/12/04/youre-here-because-of-a-robot/

10 Russian Twitter political protests ‘swamped by spam’ — www.bbc.com/news/technology-16108876
11 Fake Twitter accounts used to promote tar sands pipeline — http://www.theguardian.com/environment/
2011/aug/05/fake-twitter-tar-sands-pipeline
The Rise of Social Bots XX:7

Fig. 3. Web interface of Bot or Not? (truthy.indiana.edu/botornot). The panels show the likelihood that the
inspected accounts are social bots along with individual scores according to six features classes. Left: The
Twitter account of one of the authors is identified as likely operated by a human. Right: A popular social bot
is correctly assigned a high bot score.

what topics they talk about. The ability to extrapolate such information will allow us
to systematically infer who are the entities behind them.
Tools like Bot or Not? help shed light on the intricate world of social bots. Yet many
research questions remain open. For example, nobody knows exactly how many so-
cial bots populate social media, or what share of content can be attributed to bots
—estimates vary wildly and we might have observed only the tip of the iceberg. Bot
behaviors are already quite sophisticated: they can build realistic social networks and
produce credible content with human-like temporal patterns. As we build better de-
tection systems, we expect an arms race similar to that observed for spam in the past.
Static training instances are an intrinsic limitation of supervised learning in such a
scenario; machine learning techniques such as active learning might help respond to
newer threats. The race will be over only when the increased cost of deception will no
longer be justified due to the effectiveness of early detection.
The future of social media ecosystems might already point in the direction of envi-
ronments where machine-machine interaction is the norm, and humans will navigate
a world populated mostly by bots. We believe there is a need for bots and humans to
be able to recognize each other, to avoid bizarre, or even dangerous, situations based
on false assumptions of human interlocutors.12

12 That Time 2 Bots Were Talking, and Bank of America Butted In — www.theatlantic.com/technology/
archive/2014/07/that-time-2-bots-were-talking-and-bank-of-america-butted-in/374023/
XX:8 E. Ferrara et al.

A B

C D

Fig. 4. Visualizations provided by Bot or Not?. (A) Part-of-speech tag proportions. (B) Language distribution
of contacts. (C) Network of co-occurring hashtags. (D) Emotion, happiness and arousal-dominance-valence
sentiment scores. (E) Temporal patterns of content consumption and production.

ACKNOWLEDGMENTS
The authors are grateful to Mohsen JafariAsbagh, Prashant Shiralkar, Qiaozhu Mei, and Zhe Zhao.
 The Rise of Social Bots XX:9

REFERENCES
Luca Maria Aiello, Martina Deplano, Rossano Schifanella, and Giancarlo Ruffo. 2012. People are Strange
when you’re a Stranger: Impact and Influence of Bots on Social Networks. In ICWSM: 6th AAAI Inter-
national Conference on Weblogs and Social Media. AAAI, 10–17.
Johan Bollen, Huina Mao, and Xiaojun Zeng. 2011. Twitter mood predicts the stock market. Journal of
Computational Science 2, 1 (2011), 1–8.
Yazan Boshmaf, Ildar Muslukhov, Konstantin Beznosov, and Matei Ripeanu. 2011. The socialbot network:
when bots socialize for fame and money. In ACSAC: 27th Annual Computer Security Applications Con-
ference. ACM, 93–102.
Erica J Briscoe, D Scott Appling, and Heather Hayes. 2014. Cues to Deception in Social Media Communica-
tions. In HICSS: 47th Hawaii International Conference on System Sciences. IEEE, 1435–1443.
Christopher A Cassa, Rumi Chunara, Kenneth Mandl, and John S Brownstein. 2013. Twitter as a sentinel
in emergency situations: lessons from the Boston marathon explosions. PLoS currents 5 (2013).
Michael Conover, Jacob Ratkiewicz, Matthew Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro
Flammini. 2011. Political polarization on Twitter. In ICWSM: 5th International AAAI Conference on
Weblogs and Social Media.
Chad Edwards, Autumn Edwards, Patric R Spence, and Ashleigh K Shelton. 2014. Is that a bot running the
social media feed? Testing the differences in perceptions of communication quality for a human agent
and a bot agent on Twitter. Computers in Human Behavior 33 (2014), 372–376.
Carlos A Freitas, Fabrı́cio Benevenuto, Saptarshi Ghosh, and Adriano Veloso. 2014. Reverse Engineering
Socialbot Infiltration Strategies in Twitter. arXiv preprint arXiv:1405.4927 (2014).
Rumi Ghosh, Tawan Surachawala, and Kristina Lerman. 2011. Entropy-based Classification of “Retweeting”
Activity on Twitter. In SNA-KDD: Proceedings of the KDD workshop on Social Network Analysis.
Scott A Golder and Michael W Macy. 2011. Diurnal and seasonal mood vary with work, sleep, and daylength
across diverse cultures. Science 333, 6051 (2011), 1878–1881.
Tim Hwang, Ian Pearce, and Max Nanis. 2012. Socialbots: Voices from the fronts. Interactions 19, 2 (2012),
38–45.
Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale
emotional contagion through social networks. Proceedings of the National Academy of Sciences (2014),
201320040.
Kyumin Lee, Brian David Eoff, and James Caverlee. 2011. Seven Months with the Devils: A Long-Term
Study of Content Polluters on Twitter. In ICWSM: 5th International AAAI Conference on Weblogs and
Social Media.
Johnnatan Messias, Lucas Schmidt, Ricardo Oliveira, and Fabrı́cio Benevenuto. 2013. You followed my bot!
Transforming robots into influential users in Twitter. First Monday 18, 7 (2013).
Panagiotis T Metaxas and Eni Mustafaraj. 2012. Social media and the elections. Science 338, 6106 (2012),
472–473.
Jacob Ratkiewicz, Michael Conover, Mark Meiss, Bruno Gonçalves, Alessandro Flammini, and Filippo
Menczer. 2011a. Detecting and tracking political abuse in social media. In ICWSM: 5th International
AAAI Conference on Weblogs and Social Media. 297–304.
Jacob Ratkiewicz, Michael Conover, Mark Meiss, Bruno Gonçalves, Snehal Patil, Alessandro Flammini, and
Filippo Menczer. 2011b. Truthy: mapping the spread of astroturf in microblog streams. In WWW: 20th
International Conference on World Wide Web. 249–252.
Alan M Turing. 1950. Computing machinery and intelligence. Mind (1950), 433–460.
Claudia Wagner, Silvia Mitter, Christian Körner, and Markus Strohmaier. 2012. When social bots attack:
Modeling susceptibility of users in online social networks. In WWW: 21th International Conference on
World Wide Web. 41–48.
Joseph Weizenbaum. 1966. ELIZA – a computer program for the study of natural language communication
between man and machine. Commun. ACM 9, 1 (1966), 36–45.
Xian Wu, Ziming Feng, Wei Fan, Jing Gao, and Yong Yu. 2013. Detecting Marionette Microblog Users for Im-
proved Information Credibility. In Machine Learning and Knowledge Discovery in Databases. Springer,
483–498.
Eva Zangerle and Günther Specht. 2014. Sorry, I was hacked A Classification of Compromised Twitter
Accounts. In SAC: the 29th Symposium On Applied Computing.

View publication stats