Special Issue Published in International Journal of Trend in Research and Development (IJTRD),
ISSN: 2394-9333, www.ijtrd.com
A Comparative Study on Linear Classifiers for
Opinion Mining
1
N.Saranya and 2Dr.R.Gunavathi,
1
M.Sc(SS).,MCA.,M.Phil., Assistant Professor,Department of PG CS, Sree Saraswathi Thyagaraja College, Pollachi.
2
MCA.,M.Phil.,PhD., Head Department of MCA., Sree Saraswathi Thyagaraja college, Pollachi, India
Abstract---Sentimental analysis or opinion mining is one of the
research areas. Sentimental analysis is the method of finding
sentiment such as positive or negative from a text data.
Nowadays, people are often communicating, discussing and
sharing information through internet. Hence, internet is one of
the essential part of human life. The information in it covers a
wide range of areas such as feedback or opinion, academic
information, about products, comments about social issues etc.
It helps people to think and make decision in many things.
Majority of people always listen to others opinion before taking
a final decision. It is necessary to understand the attitudes,
opinions and emotions of the users and. In this paper we are
comparing feature selection techniques of liner classifiers such a
Super Vector Machine and Extreme Learning Machine. The
comparison of these two classifiers is considered for
performance, resources used or support vector kernels and its
computational complexity or speed. From the analysis, best
algorithm is consider for determining the emotional tone of the
user.
Keywords---Machine Learning, Sentiment analysis, Support
Vector Machine, Extreme Learning Machine.
I. INTRODUCTION
opinion analysis has drawn great interest in recent
yearsbecause of the surge in blog posts, movie and restaurant
reviews, etc. being created and shared by Internet users, and the
scope of new applications enabled byunderstanding the
sentiments embedded in that content. For example, extracting
the sentiment of a review can help provide succinct summaries
to readers, and can be very useful inautomatically generating
recommendations for users. Sentiment mainly refers to feelings,
emotions, opinion or attitude. Business owners and advertising
companies often employ sentiment analysis to start new
business strategies and advertising campaign. Sentiment
analysis, which is additionally known as opinion mining,
involves in building a system to gather and examine opinions
regarding the merchandise created in journal posts, comments,
reviews or tweets.
The data is processed through the algorithm and given to the
classifier for processing.Sentiment analysis can be used in
Online Commerce. The websites made an option to allow the
user to record their experience about shopping and product
qualities. It gives detail summary and scores/ ratings. It will
made easy for customers to select the recommended products
based on the interest. Machine learning algorithms are very
helpful to predict and classify whether a particular document
have positive or negative sentiment. Machine learning is
categorized in two types known as supervised and unsupervised
machine learning algorithms. Supervised learning algorithm
uses a labelled dataset where each document of training set is
labelled with appropriate sentiment, whereas, unsupervised
learning include un-labelled dataset where text is not labelled
with appropriate sentiments.
In this paper, the basis of comparison including the widely
used supervised learning techniques on a labeled dataset. (i.e.,)
National Conference on “Digital Transformation – Challenges and Outcomes” (ASAT in CS'17) organized by Department of
Computer Science, St.Anne's First Grade College For Women, Bangalore on 3rd Mar 2017 63 | P a g e
support vector machine and extreme learning machine in order
to evaluate and compare the speed. SVM is known as being
accurate, but at the cost of high computational complexity,
particularly in the learning phase, making it less convenient for
hardware-oriented applications. For each sample of the training
dataset, the training algorithm assigns the sample to one existing
class or creates a new one, making it a non-probabilistic linear
classifier. The comparison between SVM and ELM made in
terms of accuracy, number of hidden neurons / support vectors
and in terms of speed.
Remaining sections are organized as follows: In section 2
delivers some related works on the domain of sentimental
analysis; Section 3 comparative review on approaches to
sentiment analysis, section 4 contributes an overview of our
proposed methodology and the final section 5 is the conclusion
and future work.
II. LITERATURE REVIEW
O’Connor et al., (2010) connect measures of opinion
measured from polls with sentiment measured from text. The
others analyze many surveys on shopper confidence and
political opinion over the 2008 to 2009 amount, and realize they
correlate to sentiment word frequencies in contemporaneous
Twitter messages. Huanget al., (2015) presented an extreme
learning machine presenting an efficient unified solution to
generalized feed-forward neural networks. Unlike ANNs,
however, ELM cannot be easily parallelized, due to the presence
of a pseudo-inverse calculation represented in Xinet al., (2014).
Therefore, a reliable method is proposed to realize a parallel
implementation of ELM that can be applied to large datasets
typical of big data problems. An example represented by Heet
al., (2013) of parallel ELM implementation for regression based
on the MapReduce framework can be found in [11], while
Huanget al., (2016) provides a parallel ensemble method for an
online sequential ELM variant.
Li and Hovy (2014) propose a semi-supervised
bootstrapping algorithmic rule for analyzing China’s foreign
relations from the People’s Daily. These approach addresses
sentiment target clump, subjective lexicons extraction and
sentiment prediction in a unified framework. Totally different
from existing algorithms within the literature, time data is taken
into account in our algorithmic rule through a hierarchical
theorem model to guide the bootstrapping approach.
Danget al., (2010) presented a group of sentiment words
built on sentiment lexicon using a method called lexicon-
enhanced. They have used these words as a new feature. The
experiment were used three features such as Sentiment words
along with content specific and content free features. The
evaluation was performed using 10-fold cross validation. The
dataset used contains reviews about DVD, Books, Digital
cameras, Electronics, Kitchen appliances. The highest overall
accuracy was 84.15%, it is obtained for the product Kitchen
appliances. The experiments show that the combination of F1,
F2, F3was giving more accuracy when compared to the
individual feature set.
 Special Issue Published in International Journal of Trend in Research and Development (IJTRD),
ISSN: 2394-9333, www.ijtrd.com
Celikyilmazet al., (2010) presented a method for
normalizing the noisy tweets and classified them according to
the polarity. They have collected 2 million tweets from
September 2009 to June 2010 using Twitter search API. They
collected tweets related to the mobile operation. To generate
sentimental words they have employed a mixture model
approach, and calculated F-score of each word and the words
with F-score greater than 10 % will be selected as raw words.
As a future work, they suggested a frame work to gain
knowledge of the lexicon that can be extracted from the
collected tweets so we can represent the words such as luv,
lovwww and love as one entity “love”.
Martin et al., (2016) have projected SemEval-2013 Task 2:
Sentiment Analysis in Twitter, including 2 subtasks: A, AN
expression-level subtask, and B, a message level subtask. The
others used crowdsourcing on Amazon Mechanical Turk to
label an outsized Twitter coaching dataset at the side of further
take a look at sets of Twitter and SMS messages for each
subtasks. All datasets utilized in the analysis area unit free to the
analysis community.
Xia et al., (2015) have a tendency to propose a model
referred to as twin sentiment analysis (DSA), to handle this
downside for sentiment classification. They have a tendency to
initial propose a completely unique information growth
technique by making a sentiment-reversed review for every
coaching and take a look at review. On this basis, we have a
tendency to propose a twin coaching algorithmic rule to create
use of original and reversed coaching reviews in pairs for
learning a sentiment classifier, and a twin prediction algorithmic
rule to classify the take a look at reviews by considering 2 sides
of 1 review. They have a tendency to conjointly extend the DSA
framework from polarity (positive-negative) classification to 3-
class (positive negative-neutral) classification, by taking the
neutral reviews into thought.
III. APPROACHES TO OPINION MINING
A. Support Vector Machines
Support Vector Machines are highly effective in many research
and application domains, including text categorization. It have
been shown to outperform Naive Bayes and maximum entropy
classifiers. Zainuddin andSelamat (2014)work is considered
here, the basic idea behind SVMs is to find a separating hyper-
plane with the largest margin in a given higher-dimensional
feature space. The search for this hyper-plane corresponds to a
constrained optimization problem. It is also used for text
classification based on a discriminative classifier. The approach
is based on the principle of structural risk minimization. First
the training data points are separated into two different classes
based on a decided decision criteria or surface. The pseudo code
for SVM is shown in figure 2.
2: while min Ji <−ε:i=1,...,N
Update J, Ji = gidi.
Obtain the minimum of Ji, c = arg min i=1,...,N
Ji. And update the corresponding Lagrange variable αc.
Update g, d.
endwhile
Figure 2: Algorithm for SVM
The decision is based on the support vectors selected in the
training set. Among the different variants of SVM, the
multiclass SVM is used for sentiment analysis. The centroid
classification algorithm first calculates the centroid vector for
every training class. Then the similarities between a document
and all the centroids are calculated and the document is assigned
a class based on these similarities values. The aim of this
experiment is to improve SVM on benchmark datasets by
Panget al., (2004) and Taboadaet al., (2006). The framework
consists of preprocessing, feature extraction, feature selection
and classification stages.
B. Extreme Learning Machine
Onetoet al., (2016) proposed an efficient implementation of
the ELMs on Spark, in order to exploit the benefits of the spark
framework, in the context of big social data analysis. In
particular, an approach to support emotion recognition and
polarity detection in natural language text has been proposed
and evaluated ELM’s output layer can be considered as a linear
system where the output weights can be computed through
simple generalized inverse operation.
The simple, efficient procedure to train the ELM involves the
following steps such as randomly generate hidden node
parameters, Compute the activation matrix and compute the
output weights. They have tested on two affective analogical
reasoning datasets. In particular, two benchmarks are
considered, each one composed by 21743 common-sense
concepts; each concept is represented according to the affective
space model Cambria Gastaldoet al., (2015) and the affective
space 2 model Cambria et al., (2015). Figure 3 shows ELM
algorithm, initially the set of training data is obtained. Then the
kernel matrix is formed.
Candidate SV = {closest pair from opposite classes }
while there are violating points do
Find a violator
candidateSV = candidateSV U violator
if any αp < 0 due to addition of c to S then
candidateSV = candidateSV/p
repeat till all such points are pruned
end if
end while
Figure 3 Algorithm for ELM

1: Initialization:
α = 0, g = Gα − 1, J = g, d = 1, α, g, J, d ∈RN.

Table 1: Comparison Results Of Opinion Mining

Accuracy Testing
Accuracy on Training
on Time Error
Approach Dataset negative class time
positive (minutes) (%)
(%) (minutes)
class (%)
Product Review 17.02
SVM 67.35 84.2 16.15 2.36
dataset
Analogical 14.65
ELM 75.21 96.3 15.03 3.53
reasoning datasets

National Conference on “Digital Transformation – Challenges and Outcomes” (ASAT in CS'17) organized by Department of
Computer Science, St.Anne's First Grade College For Women, Bangalore on 3rd Mar 2017 64 | P a g e
 Special Issue Published in International Journal of Trend in Research and Development (IJTRD),
ISSN: 2394-9333, www.ijtrd.com
The experiments are performed based on the review, accuracy [11]. Dang, Y., Zhang, Y., & Chen, H. (2010). A lexicon-
achieved in sentiment classification task in shown in the table I. enhanced method for sentiment classification: An
In the Table the accuracy of unigram features in positive class as experiment on online product reviews. IEEE Intelligent
well as negative class were analyzed. For improving the Systems, 25(4), 46-53.
accuracy preprocessing unit is added before dividing and testing [12]. Celikyilmaz, A., Hakkani-Tür, D., & Feng, J. (2010,
process.The samples in the dataset should be preprocessed December). Probabilistic model-based sentiment analysis
before performing any type of operation. The main purpose of of twitter messages. In Spoken Language Technology
this step is that reducing the feature set and improves the Workshop (SLT), 2010 IEEE (pp. 79-84). IEEE.
classification performance. Maximum accuracy is obtained in [13]. Martin, V. M. A., David, K., &Bhuvaneswari, R. (2016).
positive class of ELM. Maximum performance for opinion A Survey on Various Techniques for Sentiment Analysis
mining in case of informal text is achieved. To make this and Opinion Mining. Data Mining and Knowledge
effective, need to concentrate on number of factors that affects Engineering, 8(3), 78-82.
the performance of Opinion Mining problem. [14]. Pang, B., & Lee, L. (2004, July). A sentimental
education: Sentiment analysis using subjectivity
CONCLUSION AND FUTURE WORK summarization based on minimum cuts. In Proceedings
A comparative study between two types of learning of the 42nd annual meeting on Association for
methods was considered. Several advantages of SVM is Computational Linguistics (p. 271). Association for
analyzed, such as there is no randomness involved in the Computational Linguistics.
training algorithm, so the result will not be changed by running [15]. Taboada, M., Anthony, C., &Voll, K. (2006, May).
the algorithm multiple times. SVM has active learning which is Methods for creating semantic orientation dictionaries. In
an optimization method for controlling model growth and Proceedings of the 5th Conference on Language
reducing model build time. Since SVM has more testing time it Resources and Evaluation (LREC’06) (pp. 427-432).
is one of the drawback. In ELM, the training time is higher, but [16]. Cambria, E., Gastaldo, P., Bisio, F., &Zunino, R. (2015).
the testing time remains lower. ELM also have high accuracy. An ELM-based model for affective analogical reasoning.
Since both SVM and ELM belong to the class of kernel Neurocomputing, 149, 443-455.
networks, further research will consider similar comparison [17]. Cambria, E., Fu, J., Bisio, F., &Poria, S. (2015, January).
when using different other types of kernels. AffectiveSpace 2: Enabling Affective Intuition for
Concept-Level Sentiment Analysis. In AAAI (pp. 508-
References 514).
[18]. Xia, R., Xu, F., Zong, C., Li, Q., Qi, Y., & Li, T. (2015).
[1]. Oneto, L., Bisio, F., Cambria, E., &Anguita, D. (2016). Dual sentiment analysis: Considering two sides of one
Statistical learning theory and ELM for big social data review. Knowledge and Data Engineering, IEEE
analysis. ieeeCompUTATionAlinTelliGenCemAGAzine, Transactions on, 27(8), 2120-2133.
11(3), 45-55.
[2]. Poria, S., Cambria, E., Winterstein, G., & Huang, G. B.
(2014). Sentic patterns: Dependency-based rules for
concept-level sentiment analysis. Knowledge-Based
Systems, 69, 45-63.
[3]. Zainuddin, N., &Selamat, A. (2014, September).
Sentiment analysis using support vector machine. In
Computer, Communications, and Control Technology
(I4CT), 2014 International Conference on (pp. 333-337).
IEEE.
[4]. Poria, S., Cambria, E., Gelbukh, A., Bisio, F., & Hussain,
A. (2015). Sentiment data flow analysis by means of
dynamic linguistic patterns. IEEE Computational
Intelligence Magazine, 10(4), 26-36.
[5]. O'Connor, B., Balasubramanyan, R., Routledge, B. R., &
Smith, N. A. (2010). From tweets to polls: Linking text
sentiment to public opinion time series. ICWSM, 11(122-
129), 1-2.
[6]. Huang, G., Huang, G. B., Song, S., & You, K. (2015).
Trends in extreme learning machines: a review. Neural
Networks, 61, 32-48.
[7]. Xin, J., Wang, Z., Chen, C., Ding, L., Wang, G., & Zhao,
Y. (2014). ELM∗: distributed extreme learning machine
with MapReduce. World Wide Web, 17(5), 1189-1204.
[8]. Li, J., &Hovy, E. H. (2014, October). Sentiment Analysis
on the People's Daily. In EMNLP (pp. 467-476).
[9]. He, Q., Shang, T., Zhuang, F., & Shi, Z. (2013). Parallel
extreme learning machine for regression based on
MapReduce. Neurocomputing, 102, 52-58.
[10]. Huang, S., Wang, B., Qiu, J., Yao, J., Wang, G., & Yu,
G. (2016). Parallel ensemble of online sequential extreme
learning machine based on MapReduce.
Neurocomputing, 174, 352-367.

National Conference on “Digital Transformation – Challenges and Outcomes” (ASAT in CS'17) organized by Department of
Computer Science, St.Anne's First Grade College For Women, Bangalore on 3rd Mar 2017 65 | P a g e