International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS-2017)

A Survey on Methodologies used for Semantic

Document Clustering
Aditi Gupta
Post Graduate Student
Department of Computer Science &
Engineering, JSS Academy of
Technical Education, Noida
aditi.gupta1693@gmail.com
Dr. Jyoti Gautam
Head of Department
Department of Computer Science &
Engineering, JSS Academy of
Technical Education, Noida
jyotig@jssaten.ac.in
Mr. Ajay Kumar
Assistant Professor
Department of Computer Science &
Engineering, JSS Academy of
Technical Education, Noida
ajay.itcs@gmail.com

Abstract— Document clustering is a traditional technique, and is
used in multiple fields like data mining, information retrieval,
knowledge discovery from data, pattern recognition etc. Large
volumes of textual data being created in the modern world have
resulted in the rise in importance of document clustering techniques.
Although various document-clustering techniques have been studied
in recent years, clustering quality still remains an area of concern.
Particularly, majority of the present document clustering methods
do not account for the semantic relationships and as a result give
unsatisfactory clustering results. Semantic relationships consider the
context of the usage of the term and do not solely rely on its isolated
meaning. In the recent years, a lot of effort has gone into applying
semantics to document clustering. This paper presents a survey of
various research papers that have been studied and highlights the
merits and demerits of each clustering algorithm. This will give a
direction to future research in a more focused manner.
Here, a survey of various semantic document-clustering
techniques is presented. Section II highlights the fundamentals of
document clustering which includes document clustering,
traditional approach, drawbacks of traditional approach and
semantic approach. Section III describes the related work. In this
section we cover the work that has already been done previously
by the researchers in semantic document clustering, what were
the observation of each study and the resulting research gaps.
Section IV compares the various methods that were used for
clustering and analyses the performance. Section V describes the
merits and demerits of various approaches been used for
document clustering. Section VI presents the summary of the
survey in the form of conclusion.

Keywords— document clustering; evaluation measures; II. FUNDAMENTALS OF DOCUMENT CLUSTERING

semantic; wordnet; word sense disambiguation
I. INTRODUCTION
Clustering [11] is a process of organizing data objects into a set
of non-disjoint classes known as clusters such that objects in the
same cluster are similar to each other and dissimilar to the
objects in another cluster. Document clustering [17] is based on
the similar approach, that is, documents are organized into
meaningful clusters in such a way that documents in the same
cluster represent same topic and those in different cluster
represent different topic. Traditional clustering [8] approach uses
words (terms) of document as feature vector but doesn’t consider
meaningful relationships between these words for document
clustering. Hence, it is not able to produce meaningful clusters
and faces problems such as synonymy, polysemy, and ambiguity.
Semantic document clustering [27] solves these problems by
considering the meaning of the term in the context before
clustering, for this various approaches can be such as Word sense
disambiguation [20] and Wordnet [10].
A. Document Clustering
Document clustering is a traditional approach and is used in
various areas of research such as data mining, information
retrieval, knowledge discovery from data, pattern recognition etc.
Document clustering [7] divides documents into groups based on
similarities of their content. Each cluster consists of documents,
which have similarities within the group (have high intra-cluster
similarity) but at the same time are different documents of other
groups (have low inter-cluster similarity).
Document clustering is different from document classification. In
document classification, the classes are known apriori, and
documents are assigned to such classes, whereas, in document
clustering, the number or membership of classes is not known
from before. Thus, classification can be compared to supervised
machine learning while clustering relates to unsupervised
machine learning. For the clustering process, vector space model
is used to represent documents and its major characteristic is the
high dimensionality of the feature space, which can hinder the
performance of clustering algorithms.

978-1-5386-1887-5/17/$31.00 ©2017 IEEE

671
 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS-2017)
Document pre processing procedure is depicted in Figure 1.
Fig.1: Procedure of Document Preprocessing. Image Source [11]
From the above figure we can conclude that firstly documents are
preprocessed and then clustered. Various steps followed in
preprocessing [6] are tokenization, stop word removal and
stemming. In the stemming process, morphological forms are
normalized into canonical forms. Stop word removal is used to
remove the words which does not convey any meaning such as
‘a’, ‘the’ etc. The tokenization process separates the characters
into tokens by using white spaces and punctuation marks as
separators. After the preprocessing processed documents are
obtained and this can then be clustered using clustering
algorithms.
Document clustering uses two approaches, which are: traditional
approach [8] and semantic approach [12]. Now we will study a
detailed view of traditional document clustering and what its
drawbacks were that lead to the semantic document clustering.
B. Traditional document clustering
In this approach, a document is considered to be a set of words.
The main drawback of this approach is, it does not consider the
meaning of the words that is, it ignores the semantic relationship
between words. It uses words or sequences to form clusters and
further concept weighting [23] approach is used to cluster the
documents. Due to this, it is not able to find meaningful clusters
and differentiate between different clusters.
672
The traditional approach of document clustering is described
below in Figure 2.
Fig. 2: Traditional document-clustering. Image Source [16]
By looking at the above figure of traditional document
clustering, we can conclude that in this approach keywords are
used to compute the similarity and meaning of the terms are not
taken into consideration. Therefore, there are some limitations in
this approach, which are described below.
C. Drawbacks of traditional document clustering
1) Synonymy and Polysemy:
A word may be repeated in different form but will have
the same meaning, such synonym words may exist in
the document but traditional approach does not consider
the meaning, hence these words may end up in different
clusters. Polysemy means same word possessing
different meaning. But here in this approach all of these
words will end up in the same document even though
these words have different meanings.
2) Ambiguity: It is slightly different from polysemy. In this
same word will possess different meaning in different
context. This problem will not be resolved here since
semantic relations are not taken into consideration.
3) High dimensionality: A word may possess different
meaning and when these words are mapped onto feature
space, then may cause a problem of high
dimensionality.
D. Semantic document clustering
Semantics is concerned with the study of meaning. It focuses on
the relation between signifiers like words, phrases and terms. The
meaning of semantic is related with the meaning in language.
Semantic document clustering [11] concerns with partitioning the
 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS-2017)
documents into clusters in such a way that documents in the same
cluster are similar to each other but documents in different
clusters are dissimilar to each other. Various techniques such as
word sense disambiguation [20] and word net mapping [10] are
used to find the dominant sense of the word and cluster the
documents accordingly. The semantic based document clustering
approach is described below in Figure 3.
Fig. 3 Semantic based document clustering. Image Source [15]
From the above figure, we can see that semantic document
clustering includes document-preprocessing, concept weighting
[23] according to the dominant sense using domain ontology and
clustering of the documents. Hence obtained document clusters
are semantically related.
Using Wordnet and Word Sense Disambiguation one can solve
problems of synonymy, polysemy and ambiguity.
WordNet [29] is the lexical database developed by Miller et al. It
is used to include background information for each word such as
their synset IDs, which constitute their different possible senses,
and also different levels of hypernyms and more general terms
for the word. The synsets are sets of synonyms that gather lexical
items having similar significances. Word Sense Disambiguation
[10] is a way of determining the correct sense of a word in a
context using two types of method. Some methods used the
definition of each sense in a dictionary such as WordNet in
which the definition is the gloss assigned to each synset. Other
approaches used the semantic relatedness in an existing semantic
network.
III. RELATED WORK
This section presents survey on various semantic document
clustering approaches that has already been used.
Gaby G. Dagher and Benjamin C.M. Fung [6] proposed a subject
based semantic document clustering approach for digital forensic
investigations in which data mining techniques are used to
support investigations. In this approach clustering is done using
673
wordnet, word sense disambiguation, subject vector space model
(SVSM). But it considered subjects of all clusters to be
independent which results a high dimensionality clustered model.
Ruksana Ater and Yoojin Chung [21] proposed an evolutionary
approach for document clustering in which combination of
Genetic Algorithm and K-Means is used in such a way that
there’s no requirement for pre-specification of the desired
number of clusters. Data set is partitioned into small sets on
which genetic algorithm is applied and hence it avoids problem
of local minima. Its future work focuses on making the algorithm
fully automatic in such a way that there’s no requirement for
parameter specification.
Vivek Kumar Singh, Nisha Tiwari and Shekhar Garg[25]
proposed a document clustering approach using K-means,
Heuristic k-means and fuzzy c-means. It uses different
representation schemes such as tf,tf-idf, Boolean and concludes
that tf is better than Boolean but worse than tf-idf. Out of these
three clustering algorithms, Heuristic k-means produces better
results than K –means and FCM proves to be the robust
algorithm.
Rishiraj Saha Roy and Durga Toshniwal [19] proposed an
approach which uses Naïve Bayesian Concept for fuzzy
clustering of textual documents. In this approach, datasets are
represented as a weighted term matrix using vector space model
and subsequently co-occurrence of semantically linked terms is
observed. Each term is uniquely assigned to a single cluster using
the Naïve Bayesian concept and then documents are assigned to
different clusters. This approach produces better results than
traditional algorithms and doesn’t require any user input for the
clustering. As future work indexing can be used to store the
cluster information and retrieve details relevant to few clusters
only which will help in reducing the computational space and
time.
Thaung Thaung Win and Lin Mon [24] proposed an approach
that uses Fuzzy C-Mean algorithm for document clustering.
Unlike other algorithms, which use crisp partitioning, fuzzy
clustering allows for degree of membership to which a data
belong to different clusters. PBM and F-Measure are used in this
for cluster validation. As future work, Hyperspherical fuzzy C-
Means can also be used for document clustering for providing
different level of fuzziness.
Xiaohui Cui and Thomas E. Potok [26] proposed a document
clustering technique using particle swarm optimization. It uses
K-means, PSO and hybrid of both to cluster the documents and
evaluates the result. Henceforth it concludes that hybrid KPSO
algorithm can generate higher compact clusters than PSO or K-
means alone.
 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS-2017)

Ajit Kumar and I-Jen Chiang [1] proposed a document clustering compared to K-Means alone.
technique which used semantic cliques aggregation. In this
Semantic SSA with bisecting K-Means
approach, features are extracted from document and associations clustering +
Precision,
achieves 10% higher precision value,
are defined among frequently occurring words known as 7. Recall, F-
K-Means 20% higher recall value and 40%
Measure
semantic clique. In the semantic clique each connected [5] higher F-measure value than SSA
component represents a theme and further documents are Frequent
Concept When data size is small FCDC gives
clustered according to the theme. This approach proves to be Based good results but when data size is
immensely useful for automatic document clustering. 8. F-Measure
Document large, obtained clusters are not
Clustering accurate.
Yogesh Kumar Meena, Shashank and Vibhav Prakash Singh[28] [18]
F-Measure, It gives high precision, recall and F-
proposed a text document clustering approach using genetic 9.
Naïve
Recall, Measure values resulting in good
algorithm and discrete differential evolution. Generic algorithm Baye’s [9]
Precision clusters.
is an optimization method, which finds out the best cluster
centers and DDE clusters the documents. GA and DDE solely
require more iterations but their hybrid clusters the documents in
V. MERITS AND DEMERITS
less iterations. As future work, more efficient algorithm can be
designed for large datasets. In this section, Table 2 gives a brief view of merits and demerits
of different document clustering methods.
Amy J.C. Trappey, Charles V. Trappey, Fu-Chiang Hsu and
David W.Hsiao [2] proposed a fuzzy ontological knowledge TABLE II Merits and Demerits of Clustering Algorithms
document clustering methodology. It automatically interprets and
cluster knowledge documents using an ontology schema. It also S. No. Method Merits Demerits
uses fuzzy logic control to match suitable document clusters and Different initial
compares the result with the K-Means approach and concludes K-means tighter the partitioning result in
1. K-Means clusters and is useful different final cluster and
that FODC approach outperforms the K-Means approach. when data size is large. difficult to predict K-
value.
It has low
IV. COMPARISON OF METHODS It requires initial number
Bisecting computational cost and
2. of cluster value (K value)
K-Means produce satisfactory
In this section, Table 1 describes different methods used for clusters.
given by user
document clustering and analyses their performance based on It is stable and allows
different performance evaluation parameters. 3. PSO It is computationally slow.
parallel computation.
It can solve It does not guarantee
TABLE I Comparison of Various Methods of Document Clustering multidimensional optimized results and
Genetic
4. problems and also there is no assurance that
Algorithm
S. Clustering Evaluation solves problems with it will find the global
Performance Analysis multiple solutions. optimum solution.
No. Method Method
Precision and recall gives almost It gives best result for
K-Means Precision, accurate value and proves that the overlapped dataset and It requires a prior
1. Fuzzy
[13] Recall obtained clusters are similar to each 5. in this a term may specification of the
other Algorithm
belong to more than one clusters.
Bisecting Precision, Precision and purity has high values cluster.
2. K-Means Purity, whereas entropy has lower value, and
Computationally
[10] Entropy hence better clustering.
Self- They are very simple, expensive and every SOM
The hybrid of PSO and K-Means 6. Organizing efficient and easy to is different and finds
K-Means + Intra-Cluster, algorithm gives low intra-cluster Map understand different similarities
3.
PSO [22] Inter-Cluster value and high inter-cluster value as among the term vectors.
compared to K-Means or PSO alone.
K-Means + GA achieve 50% higher
K-Means +
precision value than K-Means alone
4. Genetic F-Measure
and 12.5% higher value than GA
[21]
alone. VI. CONCLUSION
DDE + Better fitness values are found out by In this paper, a detailed survey of semantic document clustering
5. Genetic Fitness Value applying combination of GA & DDE
[28] in less iteration.
is presented. It includes survey of traditional and various
F-Measure, Hybrid achieves 11% higher recall semantic document clustering approaches. Various steps such
Ontological as stemming, stopword removal, tokenization and document
Precision, value, 25% higher precision value,
6. Semantic +
Fuzzy [2]
Recall, 11% higher F-Measure value and clustering are followed in both techniques. Moreover, it is
Entropy 33% higher entropy value as observed that semantic based approach for document clustering

674
 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS-2017)
provide better accuracy, result and quality of cluster than
traditional approach because word sense disambiguation and
wordnet are used. Various clustering methodologies have been
compared and analyzed with the help of performance metrics
and their merits and demerits have also been discussed. This
survey paper will be used to give a direction to the proposed
work.
REFERENCES
[1] Ajit Kumar and I-Jen Chiang, “Document Clustering Using Semantic
Cliques Aggregation”, International journal of Computer and
Communications, 2015, 3,Page 28-42.
[2] Amy J.C Trappey, Charles V.Trappey, Fu-Chiang Hsu and David
W.Hsiao, “A fuzzy ontological knowledge document clustering
methodolgy”, IEEE transactions on systems, man, cybernetics, Volume
39, No.3, 2009.
[3] B. Choudhary and P. Bhattacharyya, "Text clustering using semantics,"
The 11th International World Wide Web Conference, WWW2002,
Honolulu, Hawaii, USA, 2002.
[4] D. Buttler, "A short survey of document structure similarity
algorithms", International Conference on Internet Computing, pp. 3-9,
2004.
[5] G.Thilagavathi and J.Anitha, “Document clustering in forensic
investigation by Hybrid approach”, International journal of computer
applications, Volume 91-No.3, April 2014.
[6] Gaby G. Dagher and Benjamin C.M. Fung. “Subject based Semantic
Document Clustering for Digital Forensic Investigations”, In Data &
Knowledge Engineering, Elsevier2013.
[7] H. Gaudani, K. Lakhani, and R. Chhatrala, "Survey of Document
Clustering," International Journal of Computer Science and Mobile
Computing, vol. 3, issue 5, 2014, pp. 871-874.
[8] H. Tar and T. Nyaunt, "Enhancing Traditional Text Documents
Clustering based on Ontology" International Journal of Computer
Applications, vol. 33, no. 10, 2011, pp. 38-42.
[9] How Jing, Yu Tsao, Kuan-Yu Chen and Hsin-Min Wang. “Semantic
Naïve Bayes Classifier for Document Classification”. In International
Joint Conference on Natural Language Processing Page 1117-1123,
October 2013.
[10] Jullian Sedding, "Wordnet based text document clustering" Proceeding
of the 3rd Workshop on robust methods in analysis of natural language
data association for Computational Linguistics, pp. 104-113, 2004.
[11] K. Sathiyakumari, G. Maninekalai, V. Preamsudha, and M. Phil
Scholar, “A survey on various approaches in document clustering.”
International Journal of Computer Technology and Application
(IJCTA) 2, no. 5, CiteseerX (2011): 1534-1539.
[12] K. Shaban, "A Semantic Approach for Document Clustering," Journal
of software 4.5 (2009), Page 391-404.
[13] M. S. Anbarasi, "Ontology Oriented Concept Based Clustering",
International Journal of Research in Eng. and Technology, Vol. 3,
Issue 2, Feb-2014.
[14] Mamta Mahilane, and K. L. Sinha, "A Survey Paper On Different
Techniques Of Document Clustering", International Journal of Current
Engineering and Scientific Research (IJCESR), Volume-2, Issue-
1,2015.
[15] Maitri P. Naik, Harshadkumar B. Prajapati and Vipul K. Dabhi, “A
Survey on Semantic Document Clustering”, Electrical, Computer and
Communication Technologies (ICECCT), IEEE International
conference 2015.
[16] NagmaY .Saiyad, H.B. Prajapati and V.K. Dhabi, “A survey of
document clustering using semantic approach”, International
conference on Electrical, Electronics and Optimization
techniques{ICEEOT), IEEE 2016.
675
[17] PankajJajoo, "Document clustering", M.Tech thesis, IIT, Kharagpur,
2008.
[18] RekhaBaghel and Dr. RenuDhir. "A frequent concepts based document
clustering algorithm," International Journal of computer applications ,
4.5 (2010): 6-12.
[19] RishirajSaha Roy and DurgaToshniwal, “Fuzzy clustering of text
documents using Naïve Bayesian concept”, International conference on
recent trends in information, telecommunication and computing, IEEE
2010.
[20] Roberto Navigli, “Word sense disambiguation : A survey”, ACM
computing surveys(CSUR), Volume 41, Issue 2, 2009.
[21] RuksanaAktar and Yoojin Chung, “An evolutionary approach for
document clustering”, International conference in Electronic
engineering and computer science, Elsevier 2013.
[22] S. Sarkar, "A Comparative Analysis of Particle Swarm Optimization
and K-means Algorithm For Text Clustering Using Nepali Wordnet,"
International Journal on Natural Language Computing (IJNLC) Vol. 3,
No.3, June 2014.
[23] Sapna Gupta, Vikrant Chole, and A. Mahajan. "A Review on
DocumentClustering Using Concept Weight”, International Journal of
Scientific & Engineering Research, Volume 5, Issue 3, March-2014
[24] Thaung Thaung Win and Lin Mon, “Document clustering by Fuzzy C-
Mean algorithm” International Conference on advanced computer
control (ICACC) 2010.
[25] Vivek Kumar Singh, NishaTiwari and ShekharGarg, “Document
clustering using K-Means, Heuristic K-Means and Fuzzy C-Means”,
International conference on computational intelligence and
communication systems 2011.
[26] Xiaohui Cui, Thomas E.Potak and Paul Palathingal, “Document
clustering using Particle Swarm Optimization”, Swarm Intelligence
symposium, IEEE Proceedings 2005.
[27] Y. Wang and J. Hodges, "Document clustering with semantic analysis"
Procedings IEEE of the 39th Annual Hawaii International Conference
on System Science, vol. 3, 2006, HICSS'06.
[28] Yogesh Kumar Meena, Shashank and VibhavPrakash Singh, “Text
document clustering using Genetic algorithm and discrete differential
evolution”, International journal of computer applications, Volume 43-
No. 1, April 2012.
[29] Zakaria Elberrichi, Abdellatif Rahmoun, and Mohamed Amine
Bentaallah, "Using WordNet for Text Categorization", International
Arab Journal Information Technology 5, no. 1 (2008): 16-24.