2010 Ninth International Conference on Machine Learning and Applications

Semi-Automatic WordNet Based Emotion

Dictionary Construction
David B. Bracewell
General Electric Global Research
One Research Circle
Niskayuna, NY 12309
david.bracewell@ge.com

Abstract—This paper describes an algorithm for semi-
automatically creating an emotion dictionary using WordNet.
The algorithm takes as input a set of seed words that have
had emotion information assigned as well as WordNet sense
information. From this list an initial dictionary is automatically
created using the various relations found within WordNet. Then,
various correction stages are performed where parts of the
dictionary are shown to the user for verification and additional
information in the form of emotion polarity and probability
information are assigned. Using the proposed algorithm with a set
of 549 seed words, an emotion dictionary containing over 13,000
WordNet senses was created in just under 7 person-hours of time.
To evaluate the created dictionary its usefulness in improving the
performance of affect and sentiment classification was examined.
Classification was performed using support vector machines and
a baseline non-machine learning dictionary based algorithm. The
results showed that the error rate is reduced when using the
dictionary over when not using the dictionary.
I. I NTRODUCTION
Emotions play an important role in guiding the decisions
we make and determining the language we use. They also act
as the basis of human relationships [5]. Besides its importance
in human-computer interaction, emotion also plays a role in
language understanding. In a similar idea to that of Cowie et
al.[3], language is made up of an explicit and implicit message.
Understanding the explicit message is the goal of natural lan-
guage processing and semantic analysis. The implicit message
gives the intent of the explicit message and often is given
through emotion. In order to truly comprehend language both
messages must be understood.
Researchers have made great strides in understanding the
explicit message, i.e. semantic analysis of language. However,
until recently there has been little interest in understanding
the implicit meaning or emotion of language in text. In
order to achieve the next step of comprehension in language,
recognizing and understanding emotion as well as how it
effects the meaning of language is crucial. To accomplish this
goal, natural language processing and affective computing, two
seemingly disconnected areas of artificial intelligence, must be
brought together. This union has slowly been taking place over
the past few years as researchers make the first steps in trying
This research was performed while the author was a student at The
University of Tokushima.
to understand the affect (emotion) and sentiment (opinion) of
text.
Language processing requires a vast amount of resources,
such as dictionaries, corpora, ontologies, etc. The natural
language processing community over the years has created
these resources for understanding the explicit message, such as
WordNet [7] and the Penn Treebank [10]. However, due to its
infancy, there are only a scarce amount of resources available
for dealing with emotion or opinion in text. To compound the
scarcity problem there is no agreed upon theory of emotion or
even which emotions are basic. This limits the usefulness of
large scale manual creation, because of its limited applicability
and causes much of the manually created resources to be
researcher/research specific.
In order to ignite more research in this area, the amount
of resources must grow and semi-automatic ways of creating
resources must be discovered. This paper attacks the problem
of resource scarcity by proposing an algorithm, which has
some improvements over our previous work [1], for semi-
automatically creating emotion dictionaries using WordNet
(version 2.1 was used). The algorithm takes a list of seed
words and then automatically creates an initial dictionary.
After the initial creation a series of corrective steps are done
that require minimal human interaction. The algorithm is not
tied to any one theory of emotion, which allows for researchers
to create emotion dictionaries that suit their needs.
This paper will continue as follows. First, in section II
background information on affect analysis and semi-automatic
and automatic resource construction is given. Then, in section
III how to define the seed words is discussed. In section IV
how emotional inheritance in seed words is carried out is
explained. Next, in section V the structure of the dictionary
is discussed. Then, in section VI the proposed algorithm
is described. After that, section VII gives the details of an
emotion dictionary constructed with the proposed algorithm.
Then, section VIII gives an evaluation of the constructed
dictionary. Finally, section IX gives future work and makes
concluding remarks.
II. BACKGROUND
Affect and sentiment analysis are subfields of affective
computing dealing with the classification of emotion and opin-

978-0-7695-4300-0/10 $26.00 © 2010 IEEE 629

DOI 10.1109/ICMLA.2010.97
ion respectively. Both have been explored for many different
emotion expressing mediums, such as facial expressions [12]
and speech [14]. Textual affect and sentiment analysis, which
this research aims to help, combines affective computing with
natural language processing to determine the emotion and
opinion of text, such as [15].
As with other text based research, textual affect and senti-
ment analysis requires specialized resources like dictionaries,
corpora and ontologies. However, because of its infancy, only
a sparse amount of resources are available. Much of the
problem comes from that fact that there is no standardized
emotion theory or standardized set of basic emotions. Which
emotions are the most basic and why has been debated
amongst philosophers, psychologists, and cognitive scientists
since the time of the ancient Greeks.
This scarcity of resources has led researchers to manually
create the needed resources, such as [2]. In fact, much of the
research done in this field uses manually annotated resources
designed for the researcher’s current project or painstakingly
modified to fit into the researcher’s theory of emotion. These
resources are often time consuming to create and difficult, if
not impossible, to extend or change.
One common approach is to use humans to annotate or
determine the emotion of words. Cowie et al. created BEEV
(Basic English Emotion Vocabulary) by using the responses of
human participants [4]. There emotion vocabulary is created
using a human-centric approach and as such should describe
emotion well. However, there is a large creation cost and an
equally large update cost. If researchers needed to add an extra
emotion then the creation method used by Cowie et al. may be
prohibitive. Sugimoto et al. had 15 human subjects annotate
the emotional categories of verbs and adjectives found in a
standard dictionary [16].
Some researchers have seen the need to create more auto-
matic methods of creation. Kanayama and Nasukawa created
an unsupervised approach using context coherency for extract-
ing domain-dependent polar terms from unannotated corpora
that achieved 94% precision [9]. Others, have annotated well
known knowledge bases, such as Esuli and Sebastiani who
assigned positive, negative, and objective scores to all of the
senses in WordNet [6].
III. D EFINING S EED W ORDS
The user given seed words include the word, its part-
of-speech, WordNet sense id and emotion category. When
creating the initial dictionary from the seed words a number
of the WordNet relations are used. However, if an “*” is put
in front of the word in the seed word list, the algorithm will
not use any of the relations and only add that word to the
dictionary. This is to allow users to add individual WordNet
senses to the dictionary.
IV. E MOTION I NHERITANCE
Inheritance of emotion in the creation process comes about
through WordNet’s synset, hypernym and derivationally re-
lated form relations. This means that for each seed word, the
630
words in its sysnet, full hypernym tree and any related forms
will inherit (be assigned) the emotion of the seed word. In this
fashion, child senses inherit the emotion of their parents and
members of a synset can inherit from one another.
Normally, this is desirable, however in certain situations the
WordNet hierarchy may cause non-desirable inheritance. An
example of this is shown in figure 1. In this example, the
parent is “pleasantness, sweetness” and would generally have
a positive emotion. Based on the idea of inheritance, all of its
children would be assigned its emotion, which results in all the
siblings having the same emotion assigned. This means that
“disagreeableness” and “agreeableness” would be assigned the
same emotion. However, these two senses are more accurately
described as antonyms and should not be assigned the same
emotion.
Fig. 1. Example of Problematic Inheritance
To counteract this problem, inheritance is done using the
more specific seed word. What this means is that the user
can define the emotion of a parent and its child. The emotion
assigned to the child will override any inheritance from the
parent or siblings. Looking back at figure 1 as an example, the
user has pleasantness as a seed word assigned with emotion
𝑒1. In order to fix the problems in WordNet the user also
defines disagreeableness as a seed word with emotion 𝑒2. All
the children of pleasantness inherit the 𝑒1 emotion except for
disagreeableness, because it was defined as seed word. The
children of disagreeableness will inherit the 𝑒2 emotion and
not the 𝑒1 emotion. In this way the correct emotions can be
assigned to the word senses.
V. E MOTION D ICTIONARY S TRUCTURE
The created dictionary is in XML format and is made up
of information that is useful for both affect and sentiment
analysis. The information within can be used for both shallow
surface level approaches or more involved natural language
processing based approaches. Each entry in the dictionary
contains the word, part-of-speech, WordNet Sense, polarity,
emotion category, probability of the word being an emotion,
probability of the word being the given emotion and WordNet
gloss.
VI. E MOTION D ICTIONARY C REATION A LGORITHM
The proposed algorithm is designed to semi-automatically
create emotion dictionaries from WordNet. The algorithm
automates most of the process and only requires human
intervention for creating a set of initial seed words and for
manual verification. The algorithm is broken down into six
steps: Initial Creation, Multiple Category Verification, Cate-
gory Outlier Correction, Polarity Assignment, Polarity Outlier
Correction and Probability Calculation.
The algorithm starts with an automated initial creation
process that takes a set of user defined seed words. Us-
ing the various relations found within WordNet an initial
emotion dictionary is constructed. After this, a series of
correction/verification steps are performed. These steps start
by verifying the words that have multiple emotions assigned.
Then, words with possible miscategorized emotion are deter-
mined and given to the user to verify or correct. The next two
steps, assign polarity to each of the words and then determines
possible outliers. Finally, the probability that each word is an
emotion and the probability that each word is its assigned
emotion is determined and the final dictionary created. The
following sections will explain the steps of the algorithm in
more detail.
A. Initial Creation
The first step of the proposed algorithm creates the initial
dictionary from the given seed words. To create the dictionary,
WordNet’s synset, hyponym, and derivationally related forms
relations are used. This step is fully automatic and constitutes
the largest part in creating the emotion dictionary.
First, the user defined seed words are read in and parsed.
Then, each of the seed words has its synset retrieved. The seed
word and the words in its synset are then added to an extended
seed word list. Additionally, the seed and the words in synset
are also added to a protected word list. The protected word
list is used to keep the second part of the initial creation step
from assigning extra emotions the seed word. This is done to
ensure correct emotion inheritance as described earlier.
The second part takes the new extended seed word list and
uses WordNet’s relations to further expand the dictionary. For
each of the seed words, its synset, hyponyms and derivation-
ally related forms are extracted. Each of the acquired words
are first checked for in the protected list. The words that are
not protected have the seed’s emotion assigned to them and
are added to the initial dictionary if they are not already in
it. Each of the words in the synset and the hyponym list are
also added to the extended seed word list. This allows for the
full hyponym tree to be extracted as well as additional related
forms.
This step is fully automated and creates a descent, but pos-
sibly noisy, dictionary. Of course, this step, like the algorithm
is not limited to just emotions. It is possible to use this step
for extracting hierarchies from WordNet that could be useful
in information retrieval and extraction.
B. Multiple Category Verification
Typically, word senses should only carry and be assigned
one type of emotion. However, depending on the set of
emotion categories and the theory of emotion being used,
631
certain senses may actually carry multiple emotions. For
example, WordNet defines the first sense of the noun form
of “apprehension” as “fearful expectation or anticipation” 1 .
If there were a “fear” and “anticipation” category, then this
sense should have both emotion categories assigned to it.
Multiple category verification allows the user to examine the
word senses that were assigned multiple emotions and verify
that they should have multiple emotions assigned or correct
them. Errors most typically, will happen due to the WordNet
hierarchy not being the same as the user’s emotion theory.
Errors can also happen if the user does not define seed words
to take advantage of or correct errors in emotion inheritance.
Verification is done by finding the word senses in the
dictionary with multiple emotion categories assigned and
displaying them with extra information to the user for verifi-
cation/correction. The extra information shown includes part-
of-speech, WordNet sense id and WordNet gloss. Additionally,
the set of words that through one of the relations could have
caused the word to be added to the dictionary are displayed.
The set of word senses that have multiple emotions assigned
are displayed in hypernym/hyponym hierarchical order to
further help the user.
The user then selects the word senses and emotions that
should be deleted. When the algorithm deletes the word
sense/emotion pair from the the dictionary it can also delete
the emotion from the synset and hyponym tree if the user
desires. This allows for a more minimal effort on the user’s
part.
C. Category Outlier Correction
It is a strong possibility that the differences in motivations
behind the creation of WordNet and the user’s emotion theory
will cause the WordNet hierarchy to introduce errors when
assigning emotion categories automatically. Because of this,
the category outlier correction step is performed. It tries to
determine possible miscategorized word senses and allows the
user to correct them.
Determining possible miscategorized word senses is done
using a clustering approach. Before the simple clustering can
take place the word senses in the dictionary must be converted
into feature vectors. The vectors contain a feature for each
emotion category defined by the user. Additionally, the feature
vectors are labeled with the emotion of the word senses that
created them.
The values of the features are calculated using the emotions
assigned to the word sense and its hypernym, hyponyms and
derivationally related forms. This is done by first initializing
the values of the features that correspond to the word senses
emotions to 1 and all other values to 0. Then, the word senses
extracted from the hypernym, hyponyms and derivationally
related forms relations are examined one by one. Each sense
that is in the emotion dictionary will cause the feature values
corresponding to the emotions assigned to the sense to be
incremented by 1. Finally, the feature values are normalized
using equation 1.
1 http://wordnet.princeton.edu/perl/webwn?s=apprehension
 𝑓𝑖 Both probabilities are calculated using the emotion dic-
𝑁 𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑(𝑓𝑖 ) = ∑𝑁 (1) tionary and information about the word and its senses in
𝑗=1 𝑓𝑗
WordNet. Equation 4 shows how to calculate the probability
After the feature vectors are created they are clustered that a word carries emotion and equation 5 shows how to
by their labeled emotion category. Then for each emotion calculate the probability that a word carries the given emotion.
category (cluster) a mean vector is created to represent the In the equations, E stands for the set emotion categories and
centroid. The calculation of the mean vector is shown in 𝑊 = {𝑠1 , 𝑠2 , ⋅ ⋅ ⋅ , 𝑠𝑛 } is the set of senses for a word.
equation 2.
Count(𝑠𝑖 ∈ 𝑊 that carry emotion)
〈 ∑∣𝑐𝑖 ∣ ∑∣𝑐𝑖 ∣ ∑∣𝑐𝑖 ∣ 〉 𝑃 (𝐸∣𝑊 ) = (4)
∣𝑊 ∣
𝑗=1 𝑐𝑖 [𝑓1 ] 𝑗=1 𝑐𝑖 [𝑓2 ] 𝑗=1 𝑐𝑖 [𝑓𝑁 ]
𝐶¯𝑖 = , ,⋅⋅⋅, (2)
∣𝑐𝑖 ∣ ∣𝑐𝑖 ∣ ∣𝑐𝑖 ∣
Count(𝑠𝑖 ∈ 𝑊 that carry 𝑒𝑔𝑖𝑣𝑒𝑛 )
𝑃 (𝑒𝑔𝑖𝑣𝑒𝑛 ∣𝑊 ) = (5)
After the mean vectors are calculated for each of the Count(𝑠𝑖 ∈ 𝑊 that carry emotion)
emotion categories, outliers can be detected. This is done by
Each of the word’s senses are taken to be equally probable,
determining which of the emotion categories’ mean vector is
which may or may not be the case in the real use of the word.
closest to each of the senses in the dictionary. The simple
However, the probabilities should aid in using the dictionary
Euclidean distance, equation 3, is used to calculate the distance
in more shallow approaches, where part-of-speech and sense
between the mean vectors and the senses’ feature vectors. The
information may not be used. After the two probabilities are
emotion category whose mean vector has the shortest distance
calculated for each of the words in the dictionary then the final
is presumed to be the true emotion category of the sense. If
dictionary can be created in XML format.
the sense’s assigned emotion categories does not include this
emotion then it will be given to the user so that they can VII. E XAMPLE D ICTIONARY C ONSTRUCTION
determine if the emotion category should be changed or not. This section will detail the process of creating an emotion
 dictionary using the proposed algorithm. First, the basic emo-
𝑁
∑ tions that were chosen will be discussed. Then, statistical in-
𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑣, 𝑐) = ⎷
2
(𝑣𝑖 − 𝑐𝑖 ) (3)
formation on the seed words will be given. Finally, information
𝑖=1
from the results of each step of the proposed algorithm will
D. Polarity Assignment and Outlier Correction be discussed.
The next step in the dictionary creation process is to A. Basic Emotions
assign polarity information. This is done using a user supplied
This first step in building an emotion dictionary is deciding
emotion-polarity mapping. In addition to the mapping a set of
on the basic emotion categories. As has been stated before,
precedence rules can be defined to determine which polarity
there is no one agreed upon theory of what the basic emotions
will be assigned to senses with multiple emotion categories.
should be. This is the reason for needing semi-automatic or
After the initial polarities are assigned, possible outliers
automatic dictionary creating methods, since often times the
can be determined. This process is done in the same way as
categories vary between research to suite the needs of the
category outlier correction except that polarity categories are
application.
used instead of emotion categories for features. This means
that the number of features is equal to the number of polarity
Anger (-) Disgust (-) Sadness (-)
categories and that each category has a mean vector generated.
Shame (-) Fear (-) Apathy(-)
The distance between the vector of each sense and the mean
Surprise (?) Anticipation (?) Agnostic (?)
vectors are computed, using the Euclidean distance, and the
Calmness (+) Joy (+) Love (+)
polarity category with the shortest distance is chosen. The
Desire (+) Courage (+) Praise (+)
senses whose initially assigned polarity does not match the
closest polarity category is displayed to the user for possible Fig. 2. The 15 Basic Emotion Categories with Their Polarities
correction.
The proposed algorithm is capable of creating emotion
E. Probability Calculation dictionaries that are in accordance with any theory of emotion.
In this paper a set of basic emotion categories that draw
The final step in creating a dictionary is to assign probability
heavily from the ideas of Parrot [13] have been chosen. These
information. Currently, two probabilities are assigned to each
categories represent one theory of emotion, which we believe
word. The first probability is the probability that the word
to be beneficial to computer related tasks. Figure 2 shows the
carries emotion and is labeled as “emotionProbability” in
15 basic categories and some of their subcategories. The first 6
the dictionary. The second probability is the probability that
basic emotions have a polarity of “-” and the last 6 have a “+”
the word carries the assigned emotion and is labeled as
polarity. Surprise, Anticipation and Agnostic have a specially
“givenEmotionProbability” in the dictionary.
defined polarity value of “?” denoting that the category itself

632
carries no polarity. The words in these categories have their
polarity decided on an individual basis and possibly change in
different context.
B. Seed Words
A total of 549 seed words (1,088 senses) were taken from
Parrot’s hierarchy and by looking in the WordNet hierarchy.
In total 502 of the senses were nouns, 274 were verbs, 287
were adjectives and 25 were adverbs.
C. Using the Algorithm
The initial creation step resulted in a dictionary containing
a total of 13,509 senses made up of 9,726 unique words. A
total of 364 word senses were assigned multiple emotions
and presented for verification/correction during the multiple
category verification step. Of those, 289 sense/emotion pairs
were deleted from the dictionary.
The next step, category outlier correction, discovered 260
possible outliers. Of those, 92 were determined to be actual
outliers and were corrected. Using the polarities assigned
to the basic emotions, as shown previously, and a simple
precedence of − > + > ? polarities were automatically
assigned to the dictionary2 . Then, polarity outlier correction
was performed and discovered 115 possible outliers. After
manual examination, 43 were determined to be real outliers
and were corrected.
The final step assigned probability information and created
the final XML formatted dictionary. The final dictionary con-
tained a total of 13,166 senses made up of 9,709 words3 . The
total time to create the dictionary was close to 7 person-hours.
In comparison, the manually tagged word level emotion corpus
used in the evaluation section took close to 1,200 person-hours
to complete.
VIII. E VALUATION
To evaluate if the dictionary, we ran an experiment to see
if we could improve the performance of sentence level affect
and sentiment classification. Typically, one would expect this
to be the case, but poor quality dictionaries can actually hurt
performance [11]. As such, if the dictionary is able to improve
performance it is at least a small indication that the dictionary
might be beneficial. With this said, however, state-of-the-art
classification is not the goal, the evaluation is only to examine
the possible usefulness of the emotion dictionary.
Classification was done using support vector machines and
a non-machine learning dictionary based algorithm. A set
of 1,000 English sentences were manually annotated with
emotion and polarity information by ourselves and all tests
were done using 10-fold-cross-validation. The emotion and
polarity of the sentence most often was annotated as the
emotion or polarity of the subject of the sentence.
SVM𝑚𝑢𝑙𝑡𝑖𝑐𝑙𝑎𝑠𝑠 [8] was used as the support vector ma-
chine framework. Four different support vector machines were
2 where − > + denotes that negative polarity has higher precedence than
positive
3 The dictionary falls under the GPL and is available upon request.
633
trained, which were called SVM Standard, SVM Standard
(Pruned), SVM Dictionary (Polarity) and SVM Dictionary
(Emotion). SVM Standard and SVM Standard (Pruned) did
not use the emotion dictionary for classification. Instead, they
used the words found in the training data as binary features,
which is a standard text classification approach. SVM Standard
used all of the words in the training data and SVM Standard
(Pruned) used the maximum relevance criterion to prune the
words down to only the most important ones.
SVM Dictionary (Polarity) and SVM Dictionary (Emotion)
used the emotion dictionary with the user defined polarity
and emotion categories as features respectively. The values for
the features were calculated using the probability information
found in the emotion dictionary.
This was done by looking or emotion words and phrases in
the sentence. Finding the emotion words and phrases was done
by examining each word in the sentence starting at the first.
To detect phrases, n-grams were extracted with the first word
of the n-gram being the currently looked. A 5-gram, 4-gram,
trigram, bigram and the word by itself went through WordNet’s
morphological analysis algorithm and then were looked up in
the dictionary. The longest n-gram found in the dictionary was
taken as the emotion word/phrase and the process continued
starting at the next word after the n-gram. The feature values
were then calculated using equation 6 for SVM Dictionary
(Polarity) and equation 7 for SVM Dictionary (Emotion). In
the equations, 𝑝𝑓 represents the feature’s polarity category, 𝐸
represents the set of of emotion categories, 𝑒𝑓 the feature’s
emotion category, and 𝑤𝑖 the 𝑖th word/phrase in that carries
the feature’s polarity or emotion.
∑
𝐹 𝑉 𝑎𝑙𝑢𝑒(𝑝𝑓 ) = 𝑃 (𝐸∣𝑤𝑖 ) (6)
𝑤𝑖 ∈𝑝𝑓
∑
𝐹 𝑉 𝑎𝑙𝑢𝑒(𝑒𝑓 ) = 𝑃 (𝐸∣𝑤𝑖 ) ∗ 𝑃 (𝑒𝑓 ∣𝑤𝑖 ) (7)
𝑤𝑖 ∈𝑒𝑓
The non-machine learning dictionary based algorithm de-
termines the polarity or emotion simply based on which
one had the largest value. The values for each polarity or
emotion were calculated the same as the feature values using
SVM Dictionary (Polarity) and SVM Dictionary (Emotion)
respectively. For sentiment classification only “+” and “-”
polarities were used for determining the sentence’s polarity. If
the values of “+” and “-” were equal then the chosen polarity
was “?”.
Table I shows the average error rate and the standard devia-
tion for sentiment classification. What can be seen is that all of
the dictionary based approaches greatly reduced the average
error over the non-dictionary based approaches. The lowest
error rate was obtained by SVM Dictionary (Polarity) closely
followed by SVM Dictionary (Emotion). The non-machine
learning dictionary based algorithm (Baseline Dictionary) even
was able to reduce the average error by almost 6% from SVM
Standard (Pruned).
Table II shows the average error rate and the standard de-
viation for affect classification. As one would expect, because
 TABLE I In the future, we hope to expand the dictionary by adding
S ENTIMENT C LASSIFICATION R ESULTS
more seed words to the system. We also are thinking about
Method Average Error Std. Deviation
augmenting the system with some type of machine translation
SVM Standard 32.1% 1.9%
and manual check in order to create a bilingual or multilingual
SVM Standard (Pruned) 31.1% 3.1%
emotion dictionary. Finally, the method was designed for
Baseline Dictionary 25.7% 5.2%
emotion dictionary creation, but the ideas used in the algorithm
SVM Dictionary (Polarity) 18.8% 2.6%
are not limited to this. As such, we also hope to look at
SVM Dictionary (Emotion) 19.0% 2.1%
modifying the method to semi-automatically create domain-
dependent dictionaries.
TABLE II
A FFECT C LASSIFICATION R ESULTS R EFERENCES
[1] D. Bracewell, “Semi-automatic creation of an emotion dictionary using
Method Average Error Std. Deviation wordnet and its evaluation,” in 2008 IEEE Conference on Cybernetics
SVM Standard 62.1% 3.7% and Intelligent Systems,, sep. 2008, pp. 1385 –1389.
[2] Y. H. Cho and K. J. Lee, “Automatic affect recognition using natural
SVM Standard (Pruned) 60.4% 2.1% language processing techniques and manually built affect lexicon,”
Baseline Dictionary 43.6% 0.6% IEICE - Transactions on Information Systems, vol. E89-D, no. 12, pp.
2964–2971, 2006.
SVM Dictionary (Polarity) 89.8% 7.4%
[3] R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias,
SVM Dictionary (Emotion) 44.5% 3.0% W. Fellenz, and J. Taylor, “Emotion recognition in human-computer
interaction,” IEEE Signal Processing Magazine, pp. 32–80, 2001.
[4] R. Cowie, B. A. E. Douglas-Cowie and, J. Taylor, A. Romano, and
W. Fellenz, What a neural net needs to know about emotion words.
the number of classes increased, the average error rate for World Scientific & Engineering Society Press, 1999.
affect classification is much higher. The best results were [5] R. J. Dolan, “Emotion, cognition, and behavior,” Science, vol. 298, no. 8,
pp. 1191–1194, November 2002.
obtained from the non-machine learning dictionary based algo- [6] A. Esuli and F. Sebastiani, “Sentiwordnet: A publicly available lexical
rithm (Baseline Dictionary) and SVM Dictionary (Emotion). resource for opinion mining,” in Proceedings of the fifth international
Surprisingly, the baseline dictionary approach, which only conference on Language Resources and Evaluation, 2006, pp. 22–28.
[7] C. Fellbaum, Ed., WordNet: An Electronic Lexical Database. The MIT
picked the emotion with the highest value, achieved the lowest Press, 1998.
average error and the lowest standard deviation. The results [8] T. Joachims, Making large-scale support vector machine learning prac-
show that the dictionary can reduce the average error for tical. Cambridge, MA, USA: MIT Press, 1999.
[9] H. Kanayama and T. Nasukawa, “Fully automatic lexicon expansion
both sentiment and affect classification for this corpus and we for domain-oriented sentiment analysis,” in Proceedings of the 2006
believe that similar results would be found in other copora. Conference on Empirical Methods in Natural Language Processing
The reduction of average error indicates that the dictionary (EMNLP2006), 2006, pp. 355–363.
[10] M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini, “Building a
could be useful in classification and other research. large annotated corpus of english: the penn treebank,” Comput. Linguist.,
In both affect and sentiment classification the dictionary vol. 19, no. 2, pp. 313–330, 1993.
was able to improve performance by decreasing the average [11] P. McNamee and J. Mayfield, “Comparing cross-language query ex-
pansion techniques by degrading translation resources,” in SIGIR ’02:
error over the standard text classification based support vector Proceedings of the 25th annual international ACM SIGIR conference
machines. Further reductions in error are possible with a on Research and development in information retrieval. New York, NY,
more intelligent system that can understand modifiers, such USA: ACM Press, 2002, pp. 159–166.
[12] C. Padgett and G. Cottrell, “Identifying emotion in static face images,”
as “really” and “wholeheartedly,” and negatives like “not.” in Proceedings of the 2nd Joint Symposium on Neural Computation,
vol. 5, 1995, pp. 91–101.
IX. C ONCLUSION AND F UTURE W ORK [13] W. Parrot, Emotions in Social Psychology. Psychology Press, 2001.
This paper examined an algorithm for the semi-automatic [14] D. Roy and A. Pentland, “Automatic spoken affect classification and
analysis,” in FG ’96: Proceedings of the 2nd International Conference
creation emotion dictionaries using WordNet. The dictio- on Automatic Face and Gesture Recognition (FG ’96). Washington,
naries contain emotion, polarity, probability, part-of-speech, DC, USA: IEEE Computer Society, 1996, p. 363.
and sense information. They are useful for both affect and [15] P. Subasic and A. Huettner, “Affect analysis of text using fuzzy semantic
typing,” IEEE Transactions on Fuzzy Systems, vol. 9, no. 4, pp. 483–496,
sentiment analysis and can be used in both shallow surface August 2001.
level and deeper NLP based algorithms. [16] F. Sugimoto, K. Yazu, M. Murakami, and M. Yoneyama, “A method to
Using the proposed algorithm a sample emotion dictionary classify emotional expressions of text and synthesize speech,” in Con-
trol, Communications and Signal Processing, 2004. First International
was created. The emotion theory behind the dictionary en- Symposium on, 2004, pp. 611–614.
compassed 15 different basic emotion categories. The final
dictionary contained 13,166 word senses made up of 9,709
unique words.
The dictionary was evaluated by examining its usefulness
in affect and sentiment classification. Classification using
the dictionary, even non-machine learning based approaches,
outperformed the standard method greatly. This leads us to
believe that the dictionary might indeed be useful in future
research.

634