Intelligent Systems Conference 2017

7-8 September 2017 | London, UK

Improving Arabic Document Clustering using K-

Means Algorithm and Particle Swarm Optimization
Abdullah S. Daoud Ahmed Sallam* Mohamed E. Wheed
Sinai University Faculty of Computers and Faculty of Computers and
North Sinai, Informatics Informatics
Egypt Suez Canal University Suez Canal University
am.said72@yahoo.com Ismailia, Egypt Ismailia, Egypt
sallam_ah@ci.suez.edu.eg mohamed_badawy@ci.suez.edu.eg

Abstract—Document clustering plays a vital role in text
mining fields such as information retrieval, sentiment analysis,
and text organizing. Document clustering aims to automatically
divide a collection of documents based on some aspects of
similarity into groups that are meaningful, useful or both. This
paper aims to improve the clustering task for the Arabic
documents. Recent studies show that partitioning clustering
algorithms are more suitable for clustering process. However, k-
means is the most common algorithm that is being used for
clustering process because of its simplicity and speed. It can only
generate an arbitrary solution because the results depend on the
initial centers for the desired clusters “the seeds”. In this paper, a
new modified k-means algorithm called PSO K-means, supported
by Particle Swarm Optimization (PSO) is applied to enhance the
Arabic document clustering process. Then, an intensive
comparative study between the proposed model and the standard
k-means algorithm is applied. Also, the stemming algorithms
those are being used in Arabic language processing were
assessed. Through the experiments, an evaluation for the new
algorithm is done with three different Arabic data sets. The
results demonstrate that the proposed model can produce more
accurate results compared to the standard k-means algorithm for
Arabic language documents. On the other hand, Arabic light
stemmer is more suitable for the stemming step.
Keywords—Arabic document clustering; NLP; swarm
intelligence; PSO; k-means; features selection; features extraction
I. INTRODUCTION
Nowadays, text is a common method for information
exchange. The importance of this method has led many
researchers to find out suitable methods to analyze natural
language texts to extract useful information. Text mining can
be defined as the process of detecting meaningful and
interesting linguistic patterns from natural language texts [1]
[2] [3]. To aid the process of text mining, many techniques
were developed. These techniques are grouped into two
categories, Clustering and Classification. Document clustering
is a machine learning technique that is used to identify the
similarity between text documents based on their content.
Unlike text classification, document clustering is unsupervised
method, which there is no predefined class [1]. Clustering has
various types of applications like Search Engines, Identifying
Cancerous Data, Students Academics Analysis, and Sentiment
Analysis (SA) [4] [5] [6]. One of the most widely used
techniques in clustering is the k-means algorithm. Based on a
partitioning methodology, the algorithm divides N data objects
(points) into K groups (Clusters) in some similarity aspects
between data points and the clusters centers [7]. Clusters are
formed such that each data point in the cluster has a short
distance to its cluster center. Although the k-means algorithm
is simple, fast, and useful for large data size; it has a crucial
issue that has a great effect on its results. K-means is sensitive
to the initial seeds selection. In this paper, a solution to this
problem is proposed by hybridization of the standard k-means
algorithm with Particle Swarm Optimization (PSO).
A. Challenges, motivation and problem statement
a) Challenges
Arabic document clustering in particular and the
complexity of Arabic natural processing, in general, are
associated with the nature of Arabic language. Some of the
challenges:
• Š‡ ”ƒ„‹… Žƒ‰—ƒ‰‡ Šƒ• ƒ ‘”‡ …‘’Ž‡š
‘”’Š‘Ž‘‰›Ǥ ‡Ǥ‰Ǥǡ –Š‡ ™‘”† ™”‹–‹‰ ሺΔΑΎΘϛሻ Šƒ•
†‹ˆˆ‡”‡– ˆ‘”• Ž‹‡ ሺǡΎΒΘϜϳ ǡϥϮΒΘϜϳ ǡΐΘϜΗ ǡΐΘϜϳ ǡΖΒΘϛ ǡΐΘϛ
ϦΒΘϜϳሻǤ
• ”‡’”‘…‡••‹‰ •–ƒ‰‡ ሺ–‡‹‰ሻ ‹• ‘”‡ …‘’Ž‡šǡ
™Š‡”‡–Š‡”ƒ„‹…Žƒ‰—ƒ‰‡…‘–ƒ‹ƒŽƒ”‰‡†‡‰”‡‡‘ˆ
‹ˆŽ‡…–‹‘ሺΏΎΘϛǦ ΐΘϛ – ΔΒΘϜϣሻǡ™‘”†‰‡†‡”ሺƒ•…—Ž‹‡
ሺΐΘϜϳሻƒ† ‡‹‹‡ሺΐΘϜΗሻሻƒ†’Ž—”ƒŽ‹–‹‡•ሺ‹‰—Žƒ”
ሺΐΘϜϳሻǡ—ƒŽሺΎΒΘϜϳሻƒ†Ž—”ƒŽሺϥϮΒΘϜϳሻሻǤ
• ‘‡ ™‘”†• Šƒ˜‡ †‹ˆˆ‡”‡– •‡ƒ–‹… ‡ƒ‹‰•ǡ
™Š‹…Š …‘–‡š– †‡’‡†‡– ‡Ǥ‰Ǥǡ –Š‡ ™‘”† ሺϖϠΣሻ …‘—Ž†
„‡˜‡”„•Šƒ˜‹‰ሺϖ ˴ ˷ϠΣ˴ ሻǡ‘”‘—‡ƒ””‹‰
˴ ˴ϠΣ˴ ሻǡ˜‡”„ˆŽ›ሺϖ
ሺϖ˸ ˴ϠΣ˴ ሻǤ
•  ˆ‡™ ƒŽ‰‘”‹–Š• ƒ”‡ ƒ˜ƒ‹Žƒ„Ž‡ ˆ‘” ƒƒŽ›œ‹‰ –Š‡
”ƒ„‹… Žƒ‰—ƒ‰‡ ƒ† –Š‹• ’ƒ’‡” Š‹‰ŠŽ‹‰Š–• –Š‡
’”‘„Ž‡•‘ˆ–Š‡•‡ƒŽ‰‘”‹–Š•Ǥ
• Š‡ ƒ˜ƒ‹Žƒ„‹Ž‹–› ‘ˆ †ƒ–ƒ•‡–• ˆ‘” –”ƒ‹‹‰ ƒ† –‡•–‹‰
’‘”–‹‘• –‘ ƒ‡ ƒ …‘’ƒ”‹•‘ „‡–™‡‡ ‘–Š‡”
–‡…Š‹“—‡•ƒ”‡ƒ˜ƒ‹Žƒ„Ž‡Ǥ
•  ˆ‡™ –›’‡• ‘ˆ ”‡•‡ƒ”…Š Šƒ˜‡ „‡‡ †‘‡ ˆ‘” –Š‹•
’—”’‘•‡Ǥ

879 | P a g e
978-1-5090-6435-9/17/$31.00 ©2017 IEEE

b) Motivation
The Arabic language is one of the oldest languages, and the
number of Arabic speakers in the world is 168.4 million, the
majority of them lives in the Middle East and North Africa1.
Moreover, and due to religious reasons, millions of Muslims
around the world are learning Arabic to understand the holy
Quran book, which is written in Arabic. Consequently, there
are a huge Arabic data including Feedback, Tweets, and Blogs
on the internet. In addition to, millions of queries submitted to
search engines. For these reasons, this paper looking forward to
enhancing the Arabic document clustering process where, a
few types of researches have been done for this purpose.
c) Problem statement
Given ( ݅ ) a data set of Arabic documents ‫ ܦ‬ൌ
ሼ݀ͳǡ ݀ʹǡ ǥ ǡ ݀ܰሽǡ (݅݅) a desired number of clusters‫ܭ‬, and (݅݅݅) a
fitness function that evaluates the quality of clustering, the goal
is to compute the assignment of the documents to the clusters
as ‫ ܦ‬՜ ሼͳǡʹǡ ǥ ǡ ‫ܭ‬ሽ that minimizes the fitness function.
B. Thesis Contributions
The main contributions of this thesis are as follows:
1) Demonstrate high-performance document clustering in
the context of Arabic text documents comparatively with the
previous work that was done on the used data set.
2) Measure a new modified k-means algorithm enhanced
by Particle Swarm Optimization to generate more accurate
results for Clustering process for Arabic resources.
3) Features selection process has been adopted for Arabic
text clustering problem.
4) Present a new study for clustering Arabic text
documents and showing the performance on a common corpus
under common criteria to make it comparative.
5) On the basis of selected features, this paper introduce a
comparative study for using different types of stemmers to
reveal its effect s to the Clustering process.
II. RELATED WORKS
Document clustering process becomes an interesting point
for many researchers, especially for European Languages. On
the other side, a few studies are performed to serve the Arabic
language. A review to what has been done for document
clustering techniques in general and for the Arabic language in
particular is presented.
Many studies are presented such [9], where an
experimental study has shown that partitioning algorithms are
more suitable and high accurate for clustering large data set
since, they have a linear time complexity. In [9] [10] [11], k-
means algorithm and its variants are implemented for
documents clustering to enhance accuracy and efficiency, these
studies proved that k-means algorithm is more efficient for
clustering high dimensional text documents. In [12] [13] [14],
an experiments that investigated the effect of similarity
measures like Jaccard, Euclidian, and cosine where Euclidian
and Cosine are more accurate than others. In[15] [16] [17],
other semantic similarity measures based on a popular tool
1
http://www.internetworldstats.com/stats7.htm
978-1-5090-6435-9/17/$31.00 ©2017 IEEE
Intelligent Systems Conference 2017
7-8 September 2017 | London, UK
WordNet [18] [19], that provide a semantic relationship
between the documents.
Fuzzy algorithms like fuzzy C-means have been examined
for document clustering purpose [20]. These algorithms permit
a document to belong to more than two clusters and it is useful
for highly overlapped documents datasets. Other techniques
were applied such as self-organized map (SOM) [21].
Other issues for clustering Arabic documents are the
preprocessing phase and the high dimensional documents.
High features space is a challenge task for clustering algorithm.
Dimension reduction techniques are divided into two
approaches: features extraction (FE) and features selection
(FS). In FE, the original features space is reduced into lower
space using some linear transformation such as Latent
Semantic Analysis (LSA) [22] and Principal Components
Analysis (PCA) [23]. While, FS looking for selecting features
that holding adequate information about the text documents
dataset by assessing the performance of selected features in
some aspect of fitness functions [1]. An example of such
algorithms are genetic algorithms (GA) [24] and swarm
intelligence (SI) algorithms like particle swarm optimization
(PSO) [25], ant colony optimization (ACO), and bee colony
optimization (BCO) [26]. Practically, FS is better than FE due
to the fact that FE may lose some useful features in the
transformation process.
A hybridization of the k-means algorithm and PSO
introduced in many studies for European languages document
clustering [27] [3] [28]. These studies proved that this
hybridization process overcomes the standard k-means issue
about the initialization of the seeds, and achieve more powerful
and accurate results according to natural language processing
(NLP) stage.
PSO was utilized by many researchers for clustering the
text documents. Each researcher investigated PSO based on his
language and region. So, this work aim to measure this
hybridization for the Arabic text documents clustering.
For the Arabic language, a few studies have been done for
the document clustering process and the most never use
common data sets under common performance criteria. In [8],
a probabilistic model using Latent Dirichlet Allocation (LDA)
was used for clustering Arabic datasets, and a comparative
study between this model and standard k-means was carried
out. Four Arabic datasets were examined where LDA results
are superior to standard k-means for all used datasets.
In [29], a comparative study was applied between
stemming algorithms designed for Arabic language processing.
They checked the effectiveness of raw, root and light stemming
algorithms in clustering Arabic datasets, using standard k-
means algorithms and demonstrated that light stemmer has
high accurate and powerful than the others.
Some disadvantage of [8] or other works tested k-means
algorithm over Arabic datasets:
• Š‡›†‘ǯ–…ƒ”‡ƒ„‘—––Š‡‹‹–‹ƒŽ•‡Ž‡…–‹‘–‘…Ž—•–‡”
…‡–‡” ’”‘„Ž‡ ™Š‹Ž‡ –Š‹• ™‘” ’”‡•‡– ‹–•
‹’‘”–ƒ…‡‹–Š‡…Ž—•–‡”‹‰’”‘…‡••Ǥ
880 | P a g e

• Š‡› —•‡† ƒ ˜ƒ”‹‘—• ˆ‡ƒ–—”‡• ‡š–”ƒ…–‹‘ –‡…Š‹“—‡•
Ž‹‡‘†‡Ž‹ሾʹͻሿ™Š‡”‡ƒ•–Š‡•‡‘†‡Ž•ƒˆˆ‡…–
–Š‡‘”‹‰‹ƒŽˆ‡ƒ–—”‡••‡–Ǥ
This paper is organized as follows: Section III provides the
methods of representing documents in clustering algorithms
and of computing the similarity between documents. Also, it
describes the limitations of the used algorithm, k-means
algorithm, and the swarm intelligence approach. Section IV
provides a solution to the limitation problem using particle
swarm intelligence. Section V provides the detailed
experimental setup and results for comparing the performance
of the PSO algorithm with the K-means approaches. The
discussion of the experiment’s results is also presented. The
conclusion is in Section VI.
III. BACKGROUND
A. K-means Clustering Algorithm
K-means clustering algorithm belongs to the family of
partitioning algorithms which split data into a set of disjoint
partitions (Clusters). Given a data set with N points, a partition
method constructs ‫ ܭ‬group of data where each group
represents a cluster such that (‫ ܭ‬൏ൌ ܰ) [7]. So, it classifies the
data into ‫ ܭ‬groups meeting the following conditions:
1) Each cluster contains at least one point
2) Each point belongs to exactly one cluster
K-means represent the cluster by its objects mean (center)
[3] where the center is calculated using (1):
ͳ TABLE I.
‫ܥ‬௝ ൌ ෍ †୨ (1)
୨
ௗೕ ‫א‬௦ೕ
Where †୨ denotes the document vector that belongs to
cluster•୨ , ୨ stands for the center of cluster j, ୨ is the number
of document belongs to cluster•୨ .
Based on a distance function e.g. Euclidian [27], k-means
assigns the object to one of the predetermined clusters where
the object has a minimum distance to one of the clusters
centers. Given a data set ‫ ܦ‬with a very large dimension݀, the
Euclidian distance between an object ܺ ൌ ሾ‫ݔ‬ଵ ǡ ‫ݔ‬ଶ ǡ ǥ ǡ ‫ݔ‬ௗ ሿ and
the center of cluster ‫ ܥ‬ൌ ሾܿͳǡ ܿʹǡ ǥ ܿ݀ሿis calculated using (2).
ௗ models are based on different natural swarm systems.
(2)
ඩ෍ሺš୧ െ …୧ ሻଶ
ଵ Colony [26] [30]. This paper primarily focuses on one of the
The quality of the clustering can be measured by a fitness
function or sometimes is called (evaluation function), which
compute the Sum of Squared Error (SSE) i.e., a measure of
how well the selected seeds to represent their documents [2]
[28]. Minimizing the SSE is our goal. The SSE is measured as
below:
௞
ܵܵ‫ ܧ‬ൌ ෍ ෍ሺ‫ ݔ‬െ ܿ௜ ሻଶ (3)
௜ୀଵ ௫‫א‬௖೔
Where ‫ ܭ‬is the number of clusters, x are the objects belong
to the ݅ ௧௛ cluster, and ܿ௜ is the ݅ ௧௛ cluster center [27].
978-1-5090-6435-9/17/$31.00 ©2017 IEEE
Intelligent Systems Conference 2017
7-8 September 2017 | London, UK
K-means results depend on the first selection of the clusters
centers, which is done randomly, where the first step of the
algorithm is to select ‫ ܭ‬randomly documents as initial clusters
centers (seeds). Then the algorithm repeats two steps: re-
assigning each document vector to the closest center and
recalculating the center based on each cluster members until a
stopping criterion is met. As shown in Fig. 1, K-means results
depend on the initial seeds. For example, for seeds D5 and D7,
k-means converges to {{D2,D3,D4,D5,D8},
{D1,D6,D7,D9,D10}}, while, for seeds D4 and D10, k-means
converges to {{D1,D2,D3,D4,D5,D6,D8}, {D7,D9,D10}}, for
‫ = ܭ‬2 in both cases.
Fig. 1. K-means results depends on the initial seed.
So, to obtain good results, it is common to run the k-means
algorithm multiple times with different seeds to achieve a more
compact clustering. K-means can be summarized as in Table 1:
STANDARD K-MEANS ALGORITHM
Algorithm 1
1) Randomly choose K document vectors to set as initial centers.
2) Assign each document vector to the closest cluster center based on
(2).
3) Update the cluster center using equation (1).
4) Repeat step (2) to (3) until either no changing in the clusters centers
or maximum number of iterations is met.
B. Swarm Intelligence
Swarm intelligence models were inspired by natural swarm
systems. Several swarm intelligence models have been
successfully applied in many real-life applications. These
Examples of swarm intelligence models are Ant Colony
Optimization, Particle Swarm Optimization and Artificial Bee
most popular swarm intelligence model that is Particle Swarm
Optimization.
a) Particle Swarm Intelligence
PSO was introduced by Russell Eberhart and James
Kennedy in 1995. PSO draws inspiration from the social
behavior associated with the bird flocks [25].
Particles are individual solutions while the swarm is a
collection of particles that represent the solution space. Each
particle in the swarm is a moving object which can move
through the search space according to certain rules in order to
adjust its motion (i.e. Position and Velocity) and can be
attracted to a better position, in order to find the best solution.
881 | P a g e

PSO contains a fitness function to evaluate each particle in
the swarm at a specific position. The goal is to optimize the
fitness value (i.e. Minimize or Maximize) for determining the
best position in the whole swarm. Fitness function depends on
the problem to be solved.
b) The Original PSO Algorithm
Each particle moves through the solution space with a
velocity, which represents the particle’s speed in a specific
direction and a memory, which store its personal best solution
(ܲ௕௘௦௧ ), which achieved by this particle, and the global best
solution ( ‫ܩ‬௕௘௦௧ ), which achieved by other particles in the
swarm [26].
The original version of the PSO algorithm [27] [26] [25] is
described by the following two simple velocity and position
update equations are shown in (4) and (5) respectively as
follows:
ܸ௜ௗ ሺ‫ ݐ‬൅ ͳሻ ൌ ܹ ‫ܸ כ‬௜ௗ ሺ‫ݐ‬ሻ ൅ ܿଵ ‫ݎ כ‬ଵ
‫ כ‬൫‫݌‬௜ௗ ሺ‫ݐ‬ሻ െ ‫ݔ‬௜ௗ ሺ‫ݐ‬ሻ൯ ൅ ܿଶ ‫ݎ כ‬ଶ
‫ כ‬ቀ‫݌‬௚ௗ ሺ‫ݐ‬ሻ െ ‫ݔ‬௜ௗ ሺ‫ݐ‬ሻቁ
‫ݔ‬௜ௗ ሺ‫ ݐ‬൅ ͳሻ ൌ ‫ݔ‬௜ௗ ሺ‫ݐ‬ሻ ൅ ܸ௜ௗ ሺ‫ݐ‬ሻ
Where:
• – : denote the iteration counter.
Intelligent Systems Conference 2017
7-8 September 2017 | London, UK
TABLE II. ORIGINAL PSO ALGORITHM
Algorithm 2
1) Initialize each particle with a random velocity and position.
2) For each particle’s position (P), repeat the steps from [3] to [4].
3) Evaluate fitness (P) according to a fitness function.
4) IF fitness (P) is better than fitness of Pbest then Pbest=P.
5) Set best of Pbests as Gbest.
6) Update the velocities of all the particles using (4).
7) Move each particle to its new position using (5).
8) Repeat steps (2) to (6) until a stopping criterion is met (e.g., the
maximum number of allowed iterations is reached or a sufficiently good
fitness value is achieved).
IV. PSO K-MEANS ALGORITHM
The dependency of the k-means algorithm to the selection
of the initial seeds is big problem, which has a great effect on
the results [27] [3]. So, we need to perform a global search in
the entire solution space (data set) for the seeds, which gives
the best results. Therefore, we merge PSO with the standard k-
(4) means algorithm to get the optimal solution.
Each particle in the swarm represents one possible solution
for clustering the document data set. Therefore, each particle
represents a set of clusters centers as ܺ݅ ൌ ሾ‫ͳܥ‬ǡ ‫ʹܥ‬ǡ ǥ ǡ ‫݇ܥ‬ሿ ,
(5) where ‫ ܭ‬is the number of clusters, and ‫ ܥ‬is the center of the
cluster. Then, at each iteration, the fitness of each particle is
measured by (3) to identify the local best position (ܲ௕௘௦௧ ) and
the global best position (‫ܩ‬௕௘௦௧ ).
• ǣ ‹• –Š‡ ‹‡”–‹ƒ ™‡‹‰Š– ˆƒ…–‘”Ǥ – ‡•—”‡• –Šƒ– –Š‡
˜‡Ž‘…‹–›‘ˆ‡ƒ…Š’ƒ”–‹…Ž‡‹•‘–…Šƒ‰‡†•—††‡Ž›ǡ„—–
”ƒ–Š‡” –Š‡ ’”‡˜‹‘—• ˜‡Ž‘…‹–› ‘ˆ –Š‡ ’ƒ”–‹…Ž‡ ‹• –ƒ‡
‹–‘…‘•‹†‡”ƒ–‹‘Ǥ
• ୧ୢ ǣ ”‡’”‡•‡–• –Š‡ ”ƒ–‡ ‘ˆ –Š‡ ’‘•‹–‹‘ …Šƒ‰‡
ሺ˜‡Ž‘…‹–›ሻ‘ˆ–Š‡‹୲୦ ’ƒ”–‹…Ž‡‹–Š‡†୲୦ †‹‡•‹‘Ǥ
• š୧ୢ ǣ ”‡’”‡•‡–• –Š‡ ’‘•‹–‹‘ ‘ˆ –Š‡‹୲୦ ’ƒ”–‹…Ž‡ ‹ –Š‡
†–Š†‹‡•‹‘Ǥ
• ’୧ୢ ǣ”‡’”‡•‡–•–Š‡„‡•–’‘•‹–‹‘‘ˆ–Š‡‹୲୦ ’ƒ”–‹…Ž‡‹
–Š‡†୲୦ †‹‡•‹‘Ǥ
• ’୥ୢ ǣ ”‡’”‡•‡–• –Š‡ ’‘•‹–‹‘ ‘ˆ –Š‡ ‰Ž‘„ƒŽ „‡•–
’ƒ”–‹…Ž‡ሺš୥ ሻ‹–Š‡•™ƒ”‹–Š‡†୲୦ †‹‡•‹‘ሺ‹Ǥ‡Ǥǡ
–Š‡’‘•‹–‹‘–Šƒ–‰‹˜‡•–Š‡‰Ž‘„ƒŽ„‡•–ˆ‹–‡••˜ƒŽ—‡ሻǤ
• …ଵ ƒ† …ଶ ǣ –Š‡ …‘‰‹–‹˜‡ ƒ† •‘…‹ƒŽ …‘•–ƒ–•
”‡•’‡…–‹˜‡Ž› ƒ† ƒ”‡ ‘™ ƒ• ƒ……‡Ž‡”ƒ–‹‘
…‘‡ˆˆ‹…‹‡–•Ǥ
• ”ଵ ƒ†”ଶ ǣ”ƒ†‘˜ƒŽ—‡•‹–Š‡”ƒ‰‡‘ˆሺͲǡͳሻǤ
According to (4) and (5), the basic flow of the original
PSO algorithm can be described as shown in Table 2:
978-1-5090-6435-9/17/$31.00 ©2017 IEEE
Fitness function (3) should be minimized such that,
whenever the value of SSE is smaller, the clustering evaluation
becomes more accurate.
The PSO k-means clustering algorithm can be summarized
as in Table 3:
TABLE III. PROPOSED MERGED K-MEANS-PSO CLUSTERING ALGORITHM
Algorithm 3
1) Set the swarm size, the number of iterations (N), and the
number of clusters (K).
2) At the initial stage, each particle randomly select K different
document vectors from the document data set as the initial
clusters centers vectors.
3) For each particle’s position, repeat the steps from [4] to [7].
4) Assign each document to the closest cluster center according to
(2).
5) Update the centers according to (1).
6) Calculate the fitness value for each particle in the swarm
according to (3).
7) Compare the particle’s fitness value with its best solution value
‫݌‬௜ௗ , if current value is better, set ‫݌‬௜ௗ equal the current value.
8) Set the global best position ‫݌‬௚ௗ as the best value encountered
by all particles (Minimum SSE value)
9) Update the velocity and location of each particle using the new
‫݌‬௜ௗ and ‫݌‬௚ௗ according to Equation (3.4) and (3.5) respectively.
10) Repeat the steps [2] to [8] until the number of iterations is
reached.
882 | P a g e

V. EXPERIMENT SETUP, RESULTS AND DISCUSSION
A. Setup
In this section, we describe the data set being used, the
preprocessing stage and reveal the best practice for selecting
PSO parameters that are suitable for processing Arabic
documents. We carried out our experiments using Eclipse
JAVA environment and WEKA (Waikato Environment for
Knowledge Analysis)2.
a) Data set
In this research, we have used three different Arabic
datasets to evaluate the standard k-means and the merged k-
means PSO algorithm, as shown in Table 4. These data sets are
collected by Saad 3 from multiple sites.
TABLE IV. DATA SET DESCRIPTION
Data set Number Of Documents Number Of classes
BBC 4,763 7 times term ܶ௜ occurs in ‫ܦ‬௝ , and ܰ௝ denotes the number of
CNN 5,170 6
OSAC 22,429 10
b) Text preprocessing
Document preprocessing stage is an important step in text
mining, and has a great effect on the results. The goal of the
preprocessing stage is to transform text documents into a
suitable format for the automatic processing [1]. Arabic
documents are preprocessed as follows:
• Tokenization: Documents are tokenized by removing
punctuation marks, digits, numbers, symbols, and other
non-Arabic words. The remaining words are considered
as terms or tokens.
• Stop words removal: Words such as prepositions,
pronouns, and articles that occur frequently and are not
helpful for distinguishing the documents are called
stop-words [33] . Stop-words are usually removed with
the help of predefined list contains common stop-words
such as (ϝ΍ ,Ϧϣ ,ϰϓ ,ϰϠϋ ,ϕϮϓ).
• Stemming: The process of reducing words to their stem
or root, where is used to match different variants of
words. There are two types of stemming [34]:
1) Root Based Stemming: In this type, we reduce the
words to their roots. In the Arabic language, several words
such as (ϢϴϠόΘϟ΍ - ˯ΎϤϠόϟ΍ - ϢϠόϤϟ΍ ) which mean “the education”,
“the scientists” and“the teacher”, respectively are reduced to
one root (ϢϠϋ) which mean “science”. Many algorithms are
used in root extraction. In our experiments ֊ we are going to
use Khoja Stemming Algorithm .
2) Light Stemming: Light stemming does not affect the
semantics of words. It removes prefixes and suffixes from the
words instead of extracting the original root. For example, the
Arabic word (ϢϠόϤϟ΍), which mean “the teacher” is reduced to
(ϢϠόϣ ), which mean “teacher”. Also, it normalizes words into
their normal state from diacritics (Tashkeel) and stretching
character (Tatweel).
2
Weka Software: http://www.cs.waikato.ac.nz/ml/weka/
3
https://sites.google.com/site/motazsite/corpora/osac
978-1-5090-6435-9/17/$31.00 ©2017 IEEE
Intelligent Systems Conference 2017
7-8 September 2017 | London, UK
• Minimum Term Frequency: Low frequency terms are
not useful in discrimination between categories. It was
found that removing these words helps in distinguishing
between the clusters.
c) Document representation
After completing the preprocessing steps, documents are
represented in the vector space model (VSM) [35] [2]. A Term
or a feature in VSM is represented as a single word, and each
document is represented as a vector of weights ݆݀ ൌ
ሾܹଵ௝ ǡ ܹଶ௝ ǡ ǥ ǡ ܹே௝ ሿǡ where N denotes the number of distinct
terms within the document, and ܹ denotes the Term
Frequency-Inverse Document Frequency weight [TF-IDF]. TF-
IDF is calculated using (6).
ܰ
ܹ௜௝ ൌ ܰ௜௝ ‫ ‰‘Ž כ‬ቆ ቇ (6)
ܰ௝
Where ܰ௜௝ is the term frequency that denotes number of
documents in which the term ܶ௜ appears.
d) Parameters
The performance of PSO algorithm is affected by some
parameters such as:
• Swarm Size: The larger number of particles implies that
more search space to be covered, but increasing number
of particles will be time-consuming. Also, swarm size
depends on the size of the data set to be analyzed. In
this research, we tried different numbers of particles on
different data sets. Based on our experiments a
recommended swarm size would be in the range 25 to
35.
• The number of iterations: How many passes are applied
to solve the problem and get an optimal solution. With a
few numbers of iterations, searching would be stopped.
Furthermore, a large number of iterations are a time-
consuming. Number of iterations depends on the data
size.
B. Results
First ࡤ the standard k-means algorithm was applied for each
data set. And then we applied PSO k-means on the same data
sets to compare the results of both algorithms. Next, we repeat
the experiments with different preprocessing options to
measure the stemming effectiveness as follows:
• Stemming using Khoja Algorithm for root extraction.
• Stemming using the light stemming technique, where
normalization is performed.
• No stemming at all (Raw).
a) Clustering evaluation
Minimizing the objective (fitness) function Equation (3) to
achieve higher intra-cluster similarity and lower inter-cluster
similarity, such that, documents within the same cluster are
similar as possible and documents from different clusters are
dissimilar as possible. This is called internal criterion for
evaluating clustering quality. Nonetheless, good scores in an
objective function do not produce efficient results [36]. An
883 | P a g e

alternative to internal criterion is external criteria, which
measure the clustering quality by matching cluster structure to
some a priori knowledge e.g. Precision, Recall, F-Measure and
Accuracy as in (7), (8), (9) and, (10) are used respectively:
ܰ௜௝
‫ ݊݋݅ݏ݅ܿ݁ݎ݌‬ൌ ቆ ቇ (7)
ܰ௝
ܰ௜௝
ܴ݈݈݁ܿܽ ൌ ൬ ൰ (8)
ܰ௜
Where ܰ௜௝ is the number of documents of class ݅ in the
cluster ݆, Nj is the number of documents of cluster݆, and ܰ௝ is
the number of documents of class ݅. F-measure is computed
using precision and recall as below:
‫ܨ‬ሺ݅ǡ ݆ሻ ൌ ሺሺʹ ‫‘‹•‹…‡” כ‬ሺ݅ǡ ݆ሻ
‫ŽŽƒ…‡ כ‬ሺ݅ǡ ݆ሻሻሻȀሺሺ”‡…‹•‹‘ሺ݅ǡ ݆ሻ (9)
Intelligent Systems Conference 2017
7-8 September 2017 | London, UK
C. Discussion
We evaluate each data set according to (7), (8), (9), and
(10). From the experimental results for each data set, it is clear
that PSO k-means algorithm gives better results compared to
standard k-means algorithm. In the new algorithm, the result
doesn’t depend on the initial centers. Moreover, the light
stemmer gives higher accurate clustering solution compared to
other stemming options.
Also, the number of instances for a data set has a great
effect on the accuracy, where CNN’s results are superior to
BBC, and OSAc’s results are better than CNN. This implies
that there is a relationship between data size and accuracy as
shown in Fig. 2.
80

൅ ‡…ƒŽŽሺ݅ǡ ݆ሻሻሻ
……—”ƒ…› 64
65 62

ൌ ሺ‘–ƒŽ‘””‡…–Ž›Žƒ••‹ˆ‹‡†‘…—‡–•ሻ (10) 53 52 52
55

Ȁሺ‘–ƒŽ‘…—‡–•—„‡”ሻ ‫ͲͲͳ כ‬ 48
41 40
Tables 5 to 8 show the Precision, Recall, F-Measure, and
31
Accuracy respectively over applying standard k-means and 32

PSO k-mean for each data set with different stemming options
16
and Minimum Term Frequency=5:

TABLE V. PRECISION OVER BOTH ALGORITHMS BBC CNN OSAc

Light Root Not

Data
Standard k-means PSO-kmeans
set
- No Light Root No Light Root Fig. 2. Relationship between data size and accuracy.
BBC 0.15 0.28 0.25 0.50 0.46 0.27
VI. CONCLUSION & FUTURE WORK
CNN 0.59 0.61 0.45 0.62 0.61 0.60
OSAC 0.6 0.6 0.5 0.6 0.7 0.6 A. Conclusion
In this paper, one of the crucial issues of the k-means
TABLE VI. RECALL OVER BOTH ALGORITHMS algorithm was investigated, which is the dependency of the
Data selection to the clusters centers and its effects on clustering
Standard k-means PSO-kmeans
set Arabic documents. Therefore, we use Particle Swarm
- No Light Root No Light Root Optimization (PSO) as a search strategy to increase the search
BBC 0.26 0.39 0.31 0.47 0.41 0.46 space in the entire data set, i.e. broad area that all particles try
CNN 0.47 0.45 0.44 0.53 0.55 0.55 to cover to find best solution. So, merging k-means algorithm
OSAC 0.5 0.4 0.4 0.5 0.6 0.5 with PSO implies to avoid k-means limitation to the initial
selection for clusters centers. PSO avoids standard k-means
TABLE VII. F-MEASURE OVER BOTH ALGORITHMS algorithm’s limitation.
Data Using three different Arabic corpora BBC, CNN, and
Standard k-means PSO-kmeans
set OSAc, PSO k-means algorithm gives better results over than
- No Light Root No Light Root applying the standard algorithm to the same data sets. This
BBC 0.15 0.27 0.22 0.33 0.33 0.25 implies that globalize the search space for selecting the clusters
CNN 0.44 0.41 0.39 0.51 0.52 0.51 centers is an important task in the Arabic clustering process.
OSAC 0.4 0.5 0.4 0.5 0.4 0.5 Also, Arabic stemming step has a great effect on the whole
task. Arabic Light stemmer algorithm is more accurate than the
TABLE VIII. ACCURACY OVER BOTH ALGORITHMS root stemmer.
Data These research results are more accurate than those at [8] to
Standard k-means PSO-kmeans
set the same used data sets, in a comparison with applying k-
- No Light Root No Light Root means individually, and their proposed technique, according to
BBC 31.1 34.3 28 40.7 41.3 31 the parameters setting for the proposed algorithm. Also, Arabic
CNN 47 51.1 40.7 52.2 53.2 52.7 light stemmer algorithm is powerful for the Arabic document
OSAC 53 52.1 50 62 64.5 55 clustering process which does not affect the semantics of
words. On the other hand, results are affected from the wrongs
due to stemming step, where some modern words are not

884 | P a g e
978-1-5090-6435-9/17/$31.00 ©2017 IEEE

stemmed correctly. Therefore, The Arabic stemming
algorithms should be enhanced to solve this problem due to its
effects on the similarity measure.
B. Future work
In future we can concentrate on these points:
• Dealing with Arabic dialects problem.
• Using another distance function of k-means algorithms,
which support semantic methods.
• Using other stemming algorithms for the Arabic
language, that solves the stemming problem for the
modern words.
REFERENCES
[1] H. Khalifa, "New Techniques for Arabic Document Classification,"
2013.
[2] Ismael and N. Hashimah, "Text Document Preprocessing and
Dimension Reduction Techniques for Text Document Clustering," in
International Conference on Artificial Intelligence with Applications in
Engineering and Technology, 2014.
[3] Kaur, "Evaluation of Document Clustering Approach based on Particle
Swarm Optimization Method for Twitter Dataset," International Journal
of Computer Science and Information Technologies, Vol. 6, 2015.
[4] X. Y. Wang and J. M. Garibaldi, "A COMPARISON OF FUZZY AND
NON-FUZZY CLUSTERING TECHNIQUES IN CANCER
DIAGNOSIS".
[5] O. Oyelade and Obagbuwa., "Application of k-Means Clustering
algorithm for prediction of Students’ Academic Performance,"
International Journal of Computer Science and Information Security,
Vol. 7, No. 1, 2010.
[6] L. Bijuraj, "Clustering and its Applications," 2013.
[7] M. M. a. S. N. Tapas Kanungo, "An Efficient k-Means Clustering
Algorithm: Analysis and Implementation," IEEE TRANSACTIONS ON
PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 24,
NO. 7, 2002.
[8] Kelaiaia and H. Merouani, "Clustering with Probabilistic Topic Models
on Arabic Texts: A Comparative Study of LDA and K-Means," The
International Arab Journal of Information Technology VOL. 13, NO. 2,
2016.
[9] G. K. Michael Steinbach and V. Kumar, "A Comparison of Document
Clustering Techniques".
[10] Jadon and A. Khunteta, "A New Approach of Document Clustering,"
International Journal of Advanced Research in Computer Science and
Software Engineering, 2013.
[11] H. Anaya-Sánchez, "A document clustering algorithm for discovering
and describing topics," Science Direct, 2010.
[12] M. C. Ling Shyu and H. Rubin, "Affinity-Based Similarity Measure for
Web Document Clustering," IEEE, 2004.
[13] S. A. Ghosh, "Impact of Similarity Measures on Web-page Clustering,"
AAAI Technical Report WS-00-01, 2000.
[14] Leuski, "Evaluating Document Clustering for Interactive Information
Retrieval," 2010.
[15] K NIGAM and T. MITCHELL, "Text Classication from Labeled and
Unlabeled Documents using EM," Springer-Verlag,, 2006.
[16] L. Z. M. K. Liping J and Joshua, "Ontology-based Distance Measure for
Text Clustering".
978-1-5090-6435-9/17/$31.00 ©2017 IEEE
Intelligent Systems Conference 2017
7-8 September 2017 | London, UK
[17] Sridevi and Nagaveni, "Ontology based Similarity Measure in
Document Ranking," International Journal of Computer Applications
(0975 - 8887) Volume 1 – No. 26, 2010.
[18] O. Rajendra and Sahay, "Web Document Clustering and Ranking using
Tf-Idf based Apriori Approach," in In: Proceedings of the 2005
International Conference on Computational Intelligence for Modelling.
[19] Rekha and R. D, "A Frequent Concepts Based Document Clustering
Algorithm," International Journal of Computer Applications (0975 –
8887) Volume 4 – No.5, 2010.
[20] Thangamani.M and Thangaraj.P, "Fuzzy Clustering Of Documents ",
Modern Applied Science, Vol. 4, No. 7; July 2010.
[21] H. Y. Richard Freeman and Nigel, "Self-Organising Maps for Tree View
Based Hierarchical Document Clustering," IEEE, 2002.
[22] R. S. and P. A., "Space and Cosine Similarity measures for Text
Document Clustering," International Journal of Engineering Research
\& Technology (IJERT) Vol. 2 Issue 2, 2013.
[23] Ding and X. He, "K-means Clustering via Principal Component
Analysis".
[24] K. Premalatha and Natarajan, "Discrete PSO with GA Operators for
Document Clustering," International Journal of Recent Trends in
Engineering, Vol 1, No. 1, 2009.
[25] J. Kennedy and R. Eberhart, "Particle Swarm Optimization," in IEEE
International Conference on Neural Networks, 1995.
[26] H. Ahmed and J. Glasgow, "Swarm Intelligence: Concepts, Models and
Applications," 2012.
[27] T. E. P. P. P. Xiaohui Cui, "Document Clustering using Particle Swarm
Optimization," Applied Software Engineering Research Group
Computational Sciences and Engineering Division Oak Ridge National
Laboratory Oak Ridge, TN 37831-6085.
[28] M. N. Singh, "The Improved K-Means with Particle Swarm
Optimization," Journal of Information Engineering and
Applications,Vol.3, No.11, 2013.
[29] O. Ghanem and W. Ashour, "Stemming Effectiveness in Clustering of
Arabic Documents," International Journal of Computer Applications (
0975 – 8887), 2012.
[30] R.Jensi, "A SURVEY ON OPTIMIZATION APPROACHES TO TEXT
DOCUMENT CLUSTERING," International Journal on Computational
Sciences \& Applications (IJCSA) Vol.3, No.6., 2013.
[31] H. Aliwy, "Arabic Morphosyntactic Raw Text Part of Speech Tagging
System," 2012.
[32] S. R. EL-BELTAGY and A. RAFEA, "An accuracy-enhanced light
stemmer for arabic text," ACM Transactions on Speech and Language
Processing, 6, 3, Article 9, 2010.
[33] S. Alghamdi and Shahriza., "Arabic web pages clustering and annotation
using semantic class features," Journal of King Saud University,
Computer and Information Sciences, 2014.
[34] Flat clustering, Cambridge University Press.
[35] M. Lashkari and A. Rostami, "EXTENDED PSO ALGORITHM FOR
IMPROVEMENT PROBLEMS K-MEANS CLUSTERING
ALGORITHM," International Journal of Managing Information
Technology (IJMIT) Vol.6, No.3, 2014.
[36] S. P. Deshpande and D. V. M. Thakare, "DATA MINING SYSTEM
AND APPLICATIONS: A REVIEW," International Journal of
Distributed and Parallel systems (IJDPS) Vol.1, No.1, 2010.
[37] M. N. Al-Gedawy, "Detecting Egyptian Dialect Microblogs using a
Boosted PSO-based Fuzzier," Egyptian Computer Science Journal Vol.
39 No. 1, 2015.
885 | P a g e