Documente Academic
Documente Profesional
Documente Cultură
7
ISSN: 1837-7823
R.DHANAPAL2
Principal
K.C.S. Kasi Nadar College of Arts & Science,
R.K.Nagar
Chennai 600 021
URL: www.kcskasinadarcollege.in
E-mail: drdhanapal@gmail.com
Abstract
The process of sentiment analysis is a typical area which requires analysis of various parts of the text to provide
the appropriate results. Since text in general are unstructured, it becomes more difficult for the algorithm to
determine the result. This paper uses machine learning algorithms (Neural Networks and SVM) and J48
Classification algorithm to determine the best approach for determining the polarity of a document for sentiment
analysis. The results infer that SVM performs better than the other techniques in determining the document polarity.
Keywords: Context based mining, Sentiment analysis, SVM, ANN, J48
1. Introduction
In Content Based Image Retrieval (CBIR), we are concentrating on the aspect of retrieving images
corresponding to a query image. In usual text based image search, users will be providing some keywords based on
which images are retrieved. In case of text based search the ability of the user to provide an exact query is limited by
several factors like, colour, texture and such intricate details could not be represented in textual form in a consistent
manner. So the inability to provide proper input will automatically introduce bias or error in the output. So current
generation image search is based on images as input so that the match could be much better than providing text as
input.
The drawback of the current approach is that we are not searching the images in a single well defined context.
The image could be anything and should be matched with all other images in the repository before providing the
output. Image based search and matching has been successful in many domains that are context specific. Say Iris
Scan images when compared to a database containing only Iris images was very successful and similarly, facial
recognition, fingerprint readers etc. are all very reliable because of the fact that the images are all from a single well
defined context.
When it comes to a broad category of images then the drawback of providing an image as input and searching
for similar elements from a repository is that, the user is now handicapped because the context of the search is
missing. Say for example if the user is providing the image of a dog and searching through the repository, then the
context could be any of the following like pet, breed based search, police/sniffer dogs, trained dogs, helper dogs,
diseases suffered by dogs, food for dogs etc. So here by providing an image as input the user is unable to specify the
context that he/she is looking for in the image result.
Human way of looking at an image must be studied from a psychological point of view rather than considering
it as just reading all the pixels and trying to make sense out of it. Human vision or the perception of human vision to
be precise is based on the overall broad context and once we obtain the context then we ignore the local details. This
is completely different from a computerized program. Here semantics and context sensitiveness plays an important
4
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
role. This brings out the need for filling the semantic gap in content based image retrieval. Concentrating on the low
level features alone makes the search results biased and error prone. Also the changes in the luminance or texture or
colour does not change the context of an image and we are looking for the context here.
The core concept in retrieving content from an image is currently based on pixel by pixel analysis of the image.
But human vision doesnt provide the same importance to all the pixels as a computer does. So in order to emulate
human vision through computers, the key is semantics. To provide such a semantic based image retrieval system the
repository as well as the query image must be accompanied by some metadata. Metadata here provides the context.
It could be keywords, descriptions and tags. Even sentiment polarity could be included to make the search much
more effective and context sensitive. Here in this paper we try to bridge the semantic gap by including the sentiment
polarity of the images in CBIR.
The remainder of this paper is structured as follows; section II provides
2. Related works
A lot of research has gone into content based image retrieval. ThijsWesterveld in [1] used Latent Semantic
Indexing to uncover hidden semantics. That work concentrates on including co-occurrence statistics to uncover the
hidden semantic information. The work tries to bring the best of both worlds, image feature (content) and words
(context) into one semantic space. Though the work showed better performance in terms of mono and multilingual
text retrieval, its application to multi-modal and cross modal image retrieval involves a lot of computational
complexity and also its subjectivity complicates the process further.
In [2] David et al proposed several views regarding the importance of context sensitiveness in image retrieval.
They have even quoted examples from newspapers that provides text as well as images in a biased manner favouring
a particular political or religious faction. They have introduced a new platform and a diversity engine architecture
for image retrieval based on opinion analysis, text analysis and content based information retrieval. Though they
have stressed the importance of semantics and context sensitiveness in image retrieval, they have only provided an
overview and have summarized the existing text, image and other multimedia based retrieval systems.
In [3] Liyan et al presented an approach that utilizes context information to learn adaptive rules for automatic
and human in the loop clustering. The work is a bit more context aware as it considers a particular domain of face
tagging and detection. The repository under consideration in their work consists only of human facial images and
hence the context sensitiveness to a broader class is found missing. Large scale context based retrieval of images
requires analysis of millions or even billions of images and hence computationally complex.
In [4] Thanh-Nghi Doan et al have proposed a parallel incremental methodology for power mean SVM based
classification of large scale image datasets and it is proved to handle 1000s of visual classes effectively. Such a
parallel approach towards context sensitive image retrieval could improve the performance and accuracy as well. It
also considers dealing with imbalanced data. In [5] David Ahlstrom et al have shown the effectiveness of simple and
sophisticated tools for video exploration. It provides insights from a real time video search competition for video
exploration.
The next step in web search is based on including users sentiment/opinion effectively and hence providing
context sensitive results. As suggested in [2], the importance of such sentiment analysis is on the rise as the text
mining systems are now being integrated along with multimedia based information retrieval systems. So it is no
more just text or image based search, instead a combination of them all resulting in better results that are reliable in
a wide variety of domains.
Several machine learning based methods are proposed for lexical analysis of text corpus and to infer sentiment
polarity from them. In [6] Blinov et al have proposed a machine learning approach based on Support Vector
Machines (SVM) and maximum entropy method. Their approach has included information about the proportion of
positive and negative words, their colocations, emoticons as such to better identify the context. But their approach is
based on manual formation of emotional dictionaries specifically made for each domain. Since such context based
emotional dictionaries are not so very widely available for all domains, it could not be a scalable solution for general
web based image retrieval systems.
Automated Text Classification is done based on machine learning approaches for a long time now. In [7]
Ikonomakis et al have provided a detailed study of the state of the art in automated text classification using machine
learning approaches. In [8] Stefano et al presented SentiWordNet 3.0 which is the latest edition of lexical resource
specifically designed for opinion mining and sentiment classification applications. The difference between the
5
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
various versions of SentiWordNet and its features are also clearly explained along with the research applications of
such a lexical resource in various automated text classification and sentiment polarity analysis. They have also
mentioned the algorithm for automatic WordNet annotations and how it effectively classifies text into positive,
negative and neutral elements.
Rudy et al in [9] proposed a hybrid approach for sentiment analysis based on rule based classification,
supervised learning and machine learning. They have applied that to movie reviews and product reviews and
reported effective classification of sentiment polarity. Though the results are comparatively good the hybridization
increases the computational complexity of the approach to a greater extent.Bo Pang et al in [10] have considered
sentiment analysis based on positive and negative polarity alone and independent of topic. Naive Bayes, maximum
entropy classification, and support vector machines have been used for sentiment analysis by them and they have
also reported that machine learning approaches are better than human baseline when it comes to sentiment polarity.
3. System architecture
Text
10
01
...
...
...
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
The process of context based image retrieval uses the base information available in the images to retrieve the
context in which they are being used. The context based image retrieval system functions in four phases. The initial
phase deals with analyzing the available data and creating a feature vector. These feature vectors are the information
that is a broken down form of the available data. In order to remove the unnecessary words and to shortlist the
mandatory words needed for the future process, the second phase is performed. This phase removes the stop words
and symbols from the feature vectors to make them more refined. After the process of refinement, the feature matrix
is created by using the reviews and feature vectors. This data serves as the base for performing the context based
sentiment analysis. Machine learning is used for performing this analysis and finding the classification. Figure 1
shows an overall system architecture of the sentiment analysis methodology.
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
a11 a1n
am1 amn
(1)
1 word j reviewi
aij =
(2)
0 Otherwise
Equation (1) shows a sample feature vector matrix, while equation (2) shows the conditions for populating the
feature matrix.
From equation (1) it can be made clear that the rows of the matrix refer to the table and each column refers to
each word determine from the feature vector. The matrix is populated in such a way that if the word occurs in the
given text, then 1 is added to the matrix, and if the word is not present in the given text, then an entry of 0 is added
to the matrix.
The feature matrix is generally found to be large and is used as the base for the machine learning algorithms.
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
TPR
FPR
Figure 3: ROC for J48 (Positive Sentiment)
Figure 3shows the ROC plot for the positive sentiment. From the curve, it can be observed that the accuracy is
approximately 50%. J48 being a primitive classifier, it can be observed that the result obtained is average; hence we
can conclude that a machine learning approach would be a better option.
Figure 4 shows the working of the neural network model. Due to the continually training approach and the very
large data size, the training time of the neural networks seems to be very high. And further, the error rate also seems
to be high. It can be observed from Figure 3 that the error rate is 2.133 and is error reduction rate is also found to be
very low. Hence the option of considering neural networks is eliminated. ENCOG framework is used for
9
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
implementing the neural network model. The neural networks was constructed with three layers. The input and
output layers with no biased neurons, the processing layer with two biased neurons. The input layer was constructed
according to the number of words obtained after pre-processing. In our case it is 3190. Activation Linear and
Activation TanH functions were used in the input and, processing and output layers respectively. Resilient
propagation function was used to train the network. The network design is as follows (Table 1):
Table 1: Neural Network Setup
No Of Layers
3190
3192
ActivationLinear
ActivationTanH
Resilient Propagation
The same data set is considered and analysis is performed using SVM. It uses the RBF kernel function is used
for classification.
K ( xi , x j ) = exp( || xi x j || + r ), > 0
2
(3)
The SVM requires a special format for reading the data. The expected format of input for an SVM is
[label] [index1]:[value1] [index2]:[value2] ...
(4)
The values (value1, value2,valuen) in the given format are normalized within the range -1 to 1. In order to
convert the data into the required format, Max-Min Normalization is used, which is of the form,
(5)
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
FPR
TPR
Figure 6: ROC for SVM
Figure 6 shows the ROC plot, which provides a promising accuracy. Hence after analysis of the results, SVM is
found to work efficiently for the process of context mining. Figure 7 shows the result obtained from SVM Classifier.
6. Conclusion
This paper is an initial implementation for analysis of the available data with the classification algorithms and
to select the appropriate technique for the next level of analysis. Implementation is carried out using data obtained
from the IMDb dataset, and from the results it is clear that SVM works best on the area of context mining. This
process can be further improvised by using one class classification techniques rather than multi-class classification.
Further, our next research proposal will take forward this research into mining levels of polarities rather than
11
International Journal of Computational Intelligence and Information Security, October 2014 Vol. 5, No. 7
ISSN: 1837-7823
providing a single polarity base. Level of polarity can be analyzed and can be used for performing emotion analysis,
which is a deeper form of sentiment analysis.
7. References
[1]
ThijsWesterveld, (2000), Image Retrieval: Content versus Context, University of Twente, Department of
Computer Science, Parlevink Group,PO Box 217, 7500 AE Enschede, The Netherlands.
[2]
David Paul Dupplaw Michael Matthews Richard Johansson Giulia Boato Andrea Costanzo Marco
Fontani Enrico Minack Elena Demidova Roi Blanco Thomas Griffiths Paul Lewis Jonathon Hare
Alessandro Moschitti, (2014), Information extraction from multimedia web documents:an open-source
platform and testbed, Int J Multimed Info Retr 3:97111.
[3]
Liyan Zhang, Dmitri V. Kalashnikov, SharadMehrotra, (2014), Context Assisted Face Clustering
Frameworkwith Human-in-the-Loop, International Journal of Multimedia Information
Retrieval,Volume 3, Issue 2, pp 69-88.
[4]
Thanh-Nghi Doan,Thanh-Nghi Do, Francois Poulet, (2014), Parallel Incremental Power Mean SVM for
the Classificationof Large Scale Image Datasets, International Journal of Multimedia Information
Retrieval,Volume 3, Issue 2, pp 89-96.
[5]
Klaus
Schoeffmann,David
Ahlstrom,
Werner
Bailer,
ClaudiuCobarzan,FrankHopfgartner,KevinMcGuinness, CathalGurrin, ChristianFrisson, Duy-Dinh Le,
Manfred Del Fabro, HongliangBai, Wolfgang Weiss, (2014), The Video Browser Showdown: A Live
Evaluationof Interactive Video Search Tools, International Journal of Multimedia Information
Retrieval,Volume 3, Issue 2, pp 113-127.
[6]
Blinov P. D., Klekovkina M. V., Kotelnikov E. V., Pestov O. A. (2013), Research of lexical approach and
machine learning methods for sentiment analysis.
[7]
[8]
[9]
Rudy Prabowo, Mike Thelwall , (2009), Sentiment Analysis: A Combined Approach, Journal of
Informetrics 3.2 : 143-157.
[10]
[11]
[12]
[13]
[14]
[15]
[16]
Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. (2002), "Thumbs up? Sentiment classification using
machine learning techniques." Proceedings of the ACL-02 conference on Empirical methods in natural
language processing-Volume 10.
[17]
Datasets.
pp. 1
12