Abstract—Keyphrase extraction is an essential part of  Statistical Approach: The statistical information of the
contemporary world of internet. Researchers have treated
keyword extraction as a classification problem where the input
candidate words are classified as keywords or non-keywords.
We propose a supervised machine learning method for
keyword extraction using SVM. Noun phrases extracted using
a specified regular expression are considered as candidate
words. The classifier uses a combination of statistical and
linguistic features such as part of speech, frequency, spread etc.
to classify the extracted candidate words. The proposed method
shows promising results in the classification task.
Keywords— Keyphrase extraction, Noun phrases, Feature
extraction, SVM, Confusion matrix
I. INTRODUCTION
In today’s era, tremendous information is available on the
web. Thousands of books, journals, articles, papers etc are
published on a daily basis about a single topic. Whenever
studying any topic, it is nearly impossible to read the
complete documents. Therefore, an efficient information
retrieval method is required to extract relevant information
which generally involves extracting the index terms or
keywords from the documents. Keywords are smallest unit
that play a significant role in determining the main context of
a given document. They can be used to easily identify the
meaning of the entire document which can be easily
exploited by various applications like text summarizer, topic
detector, cataloger, etc.
Automatic keyphrase extraction is defined as “the
automatic selection of important and topical phrases from the
body of a document”[1]. The main goal of keyword extraction
is to extract a set of phrases that help in grasping the central
idea of the document without the need to read it. The readers
of all kinds of articles whether academic, sports, business,
social etc are benefited greatly from the automatically
generated keywords. Keywords of a document are of great
importance to search engines as they help in providing precise
results of the queries. Having such a large importance, the
topic automatic keyword extraction has received great
attention in the recent years. Many automated keyword
extraction systems have been introduced which are based on
following three approaches -
The keyphrase extractor that Turney [1] suggested also
works on a similar approach but uses candidate phrases of
maximum length five. Features used for weighing the
candidate key-phrases are frequency, relative length, first
occurance of phrase, number of words in phrase, etc. Decision
tree is used to detect the keyphrases.
Krulwich and Burkey [3] created an InformationFinder
through “the extraction of semantically significant phrases”.
The tasks which make it innovative are using heuristics to
extract important phrases for learning rather than performing
complex mathematical calculations on them and more
importantly creating decision trees to determine user’s
interest. Finally, the decision tree is transformed into a
“boolean search query string”.
words such as word frequency, word co- occurrence,
term nesting statistics, etc. is used to identify the
keywords. This approach does not require any training
data. The specified features are extracted for the
document and based on these obtained values, the
keywords are listed.
 Linguistic approach: This approach includes usage of
parts of speech such as nouns and adjectives in sentence
semantics. Nouns phrases are extracted and then further
methods are used to weigh them to be identified as a
keyword.
 Machine Learning Based Approach: A more
sophisticated approach than the previous two which
uses the concept of training a classifier using a dataset
containing the documents along with their extracted
keywords. This classifier is then used to predict the
keywords of a document. Various techniques that can
be used are Support Vector Machine (SVM), Naive
Bayes, etc.
Also, the keyword extraction methods can be classified as
extractive or abstractive. The extractive methods directly
extract keywords from the text whereas the abstractive
methods work similar to a human and generate keywords by
using contextual knowledge of the text. These keywords
generated by abstractive methods may not be directly present
in the text and may be a combination of some words of the
text.
The proposed ML method uses extractive approach to find
and then classify the candidate words.
II. LITERATURE SURVEY
Many supervised and unsupervised extractive approaches
have been proposed till date.
The Kea System [2] firstly chooses the candidate phrases
(maximum length three) by cleaning the input text and then
identifying the candidate by removing stopwords and excluding
proper nouns. Next, stemming is applied to get the final list of
candidates. Secondly, selection of key phrases is done on the
basis of two features namely TF-IDF (term Frequency - Inverse
Document Frequency) and first occurrence. Discretization is
then applied to convert features into nominal data which can be
consumed by machine learning classifiers. Lastly, Naive Bayes
is trained to build a classifier on the basis of these features.
Mihalcea and Tarau (2004) [4] proposed the unsupervised
approach to find the keywords using a TextRank model defined
by distance between co-occurrences of same word. The
candidate words are limited by the use of only a certain lexical
units based on part of speech. The vertices or the candidate
words are then ranked and the top T are said to be the extracted
keywords.
Nguyen and Kan (2007) [5] proposed keyphrase
extraction for scientific articles by using morphological
features found in scientific articles.
Wan and Xiao (2008) [6] proposed a single document
keyphrase extraction method using neighborhood knowledge
whereby an expanded document is created by adding neighbor
documents to the concerned document and then keywords are
ranked using a graph utilizing both the local and global
information in the neighbor documents.
Kathait, Tiwari (2017) [7] proposed an unsupervised
approach based on extraction of noun phrases. The candidate
words only contain adjectives and nouns. Firstly, individual
words are scored using a scoring scheme such as TF-IDF and
secondly, the n-grams or phrases are given scores equal to the
sum of scores of individual words present in them. The top
ranked phrases are considered as the output keyphrases.
III. PROPOSED WORK
The proposed supervised keyphrase extraction scheme
uses the DUC-2001 benchmark dataset that has around 300
news articles [6]. The methodology consists of four main
tasks: (1) Candidate Selection (2) Feature Extraction (3)
Training SVM (4) Prediction of keyphrases.
Fig. 1 The training and keyword prediction process
A. Candidate Selection
A set of words and phrases is extracted as candidate
keyphrases using the regular expression [8].
{(<Adjective>* <Noun>+ <Preposition>)? <Adjective>*
<Noun>+}
Phrases following this pattern are noun phrases. Noun
phrases are selected because nouns contains tremendous
amount of information related to the document. Noun
phrases can be better understood by knowing the most
frequently occurring patterns in them i.e. Noun, Adjective
Noun, Noun Noun and Noun Preposition Noun.
All these phrases essentially contain noun as their component
and hence, are termed as noun phrases.
B. Feature Extraction
There are five features that we used to represent the
candidate keyphrase which are explained in detail below:
i. POS sequences: According to Hulth and
Megyesi(2006) [9], keyphrases tend to have
distinctive distribution of Part of Speech sequences
and hence its worth using them as a feature. This is
done using POS tagging which identifies the
individual word in candidate phrases as a common
noun, proper noun, singular noun, plural noun, or
an adjective, preposition, etc. These POS tags of
the candidate then form a part of the feature list.
ii. Position of Keyphrase: The beginning [10] and
concluding part of a document typically contains
more relevant information to the topic being
addressed, thus we use parabolic and line position as
a feature to rank the words such that the candidate
words at the beginning and end of a document are
assigned more weight.
 Line Position: Line position ranks the candidate
phrases in a way that the words in the beginning
of the document have the maximum weight,
which decreases linearly; is least for a candidate
word at the mid of the document and increases
linearly thereafter such that a candidate word at
the end of the document has the maximum weight
again.
 Parabolic Position: Parabolic position ranks or
weighs the candidate words using a similar
approach with a difference that the weight of the
candidate words decreases and increases according
to the equation of a parabola instead of a line.
iii. Spread of Keyphrase in document: The spread of the
word in the document has been found as a beneficial
feature in a closely related task of link generation [11].
A more important keyphrase has more standard
deviation which implies it is widely spread in the text
document and thus proves to be an essential feature.
Hence, spread calculated using standard deviation of
the candidate in the document forms a part of the
feature list.
iv. Frequency: A more important word tends to occur
repeatedly and as a result has higher frequency in the
document [10]. Hence, we consider frequency as a
feature in our proposed work.
v. Occurrences of candidates in the documents:
According to Louis and Gagnon (2013) [12], position
based features such as first and last occurrence have
been found effective in keyword extraction. We used
normalized position of first occurrence of the
candidate and then extended this approach to use the
normalized position of all occurrences of the candidate
in the documents as part of the feature vector.
Values for these features are extracted for the selected
candidate words. All these features are then stored as key value
pairs for each candidate word.
C. Training the SVM classifier
DUC-2001 dataset is split into training and testing data.
The above mentioned features for candidate words are
extracted using the proposed regular expression. During
training, the SVM classifier is provided with these features
along with 1 or 0 indicating whether the candidate is a keyword
or non-keyword respectively. The ratio of sizes of keyword list
and non-keyword list is adjusted to achieve the highest possible
value of f-measure.
D. Prediction of keyphrases
After training the SVM classifier, we test it by predicting
the keyphrases from the testing data. The features of
candidate words are again extracted and input to the trained
model to classify them as a keyword or non-keyword. The
resulting values of precision, recall and f-measure are
recorded.
IV. OBSERVATIONS
The available dataset was split such that about 60 percent
of the total scripts were used for training the classifier and
remaining 40 percent data was used for testing the classifier.
We plot confusion matrix during testing of the classifier and
then describe the performance of our system in terms of the 3
standard metrics i.e. precision, recall and f-measure,
calculated from this confusion matrix. Initially, there was a
large imbalance in the values of both classes (one being the
keyword and other being the non-keyword) of training data.
Training the classifier using this data yields a high value of
precision but a low recall. Whereas completely balancing the
values in both the classes yields a very high recall but a very
low value of precision. Hence, we trained our classifier by
restricting the values in non-keyword class to an optimal
value such that we get the highest possible f-measure on the
testing data. Firstly, we give a brief description of confusion
matrix, precision, recall and f-measure and then their
respective values achieved.
 Confusion Matrix: It is a form of a table that is used to
determine the performance of a classifier using a set
of data having known true values. The three standard
performance metrics used are based on this confusion
matrix.
 Precision: “Precision (also called positive predictive
value) is the fraction of relevant instances among the
retrieved instances”.
 Recall: “ In pattern classification, recall (also known
as sensitivity) is the fraction of relevant instances that
have been retrieved over the total amount of relevant
instances.”
 F-measure: In pattern classification, f-measure is
calculated as the harmonic mean of precision and
recall.
The values of precision, recall and f-measure were
recorded for the testing dataset when trained with different
values of balance of classes.
A. Completely Balanced Dataset
This dataset has around 1500 samples of each class for
training.
Precision Recall F-measure
12.93 82.04 22.33
Table 1: Observations for completely balanced dataset
We noted that recall was very high but precision was very
low resulting in a low f-measure of 22.33%. This is
because the classifier predicted a large number of false
positives i.e. a large no. of wrongly predicted keywords.
The classifier assumes a lot of candidate words as
keywords, a lot of them not actually being the keywords.
Nonetheless the actual keywords are correctly classified.
B. Unbalanced Dataset
Here, we did not impose any restriction on the size of
classes in the training dataset and there was a large imbalance
in the size of two classes.
Precision Recall F-measure
55.07 3.877 7.24
Table 2: Observations for unbalanced dataset
In this case, precision was very high but recall became
very low which again resulted in a very low F-measure of
only 10.6%. The classifier is over-particular and does not
assume many candidate words to be keywords. Therefore,
misses a lot of keywords.
C. Optimally Balanced Dataset
We tried balancing the dataset to get the highest possible
value of f-measure. This was achieved when there were around
1500 samples corresponding to keyword class and 8400
samples corresponding to non-keyword class.
Precision Recall F-measure
30.8 37.75 33.91
Table 3: Observations for optimally balanced dataset
Here, it is seen that F- measure is considerably high.
V. EVALUATION
For the evaluation of the effectiveness of our proposed
system, we compared it to the previously achieved best scores
on DUC 2001 dataset.
The highest achieved precision, recall and f-measure scores as
mentioned by Hasan and Ng [13] are
Precision Recall F-measure
28.8 35.4 31.7
Table 4: The highest achieved values on DUC-2001 dataset[13]
Our achieved scores are considerably higher which are
30.8, 37.75 and 33.91 respectively.
VI. CONCLUSION
We have presented a supervised approach for keyword
extraction. Noun phrases are selected as candidate words
because majority of information of the document is contained
in them. The paper also introduces five major features that
affect the performance of the SVM classifier. The main finding
is that the ratio of values in the two classes of the training set
greatly affects the precision and recall. Since f- measure is the
harmonic mean of precision and recall, it is considered as the
ultimate measure of performance. The optimal balance between
the two classes yields a promisingly high value of f-measure.
We also concluded that part of speech was the most essential
feature since it increased the f- measure value to a great extent.
Although the values of f-measure attained are
encouraging however there is a scope of improvement by
incorporating more features that increase its value. The
proposed model can also be extended to include more parts of
speech such as adverbs in the regular expression used to
extract candidate keyphrases.