A novel approach for semantic similarity
measurement for high quality answer selection in
question answering platform using deep learning
methods
Darshana Vekariya
Department of Computer Engineering,
V.V.P. Engineering College, Rajkot
Gujarat, India
darshanavekariya9@gmail.com
Abstract:
Question retrieval and high-quality answer retrieval is the main task
in Question-answering system. It is a real-world application of NLP
technologies. The major challenges of QA is the exact selection of
high-quality responses w.r.t. given questions and by doing that it will
also minimize the time of finding a similar question and high-quality
answer. A large amount of data available, but getting the right
information accessible when needed is very important. Since many
users have similar informational needs, answers to new questions can
typically be found either in whole or part, from the existing question-
answer archive. Hence, given a new incoming question, it displays a
list of similar or related questions, which could satisfy immediately
response, the information needs to the user, without having to wait
for new questions to be answered by other users. QA engines attempt
to let you ask the question the way you normally ask. Cosine
similarity algorithm is used to compute the distance between the
question and answer pairs and it is used to find the appropriate
answer. QA can have two domains Open Domain QA and Closed
Domain QA. In Open-domain there is no any particular domain, users
can ask any question, the system will give an answer from the web
and give a respective answer to the user. In a closed domain, QA
users can ask a question related to some particular domain.
Keywords:
Question-answering, neural networks, deep learning, memory
networks
Introduction:
QA is important to retrieve questions from information resources on
the web and based on that it is also helpful to retrieve the best
answers. Sources include Yahoo!-answers, Quora, Stack Overflow.
The advantages of Question answer retrieval are: (i) input is used as a
Natural language rather than the keyword; (ii) system obtained
several possible answers directly rather than long document and so
we can increase the efficiency of finding the required answer [1].
There are mainly two types of QA systems. 1)Open-domain and
2)Closed-domain. In open-domain users get an answer from any
domain using a large collection of data[16]. In the open domain,
many subjects are there and Information is added and updated and No
manual work is required to build databases. In closed domains, QA
users can obtain an answer from a specific domain. And extract an
answer from the database(structured data).
Nivid R. Limbasiya
Department of Computer Engineering,
V.V.P. Engineering College, Rajkot
Gujarat, India
nivid.limbasiya.ce@vvpedulink.ac.in
1. Retrieving best answers and Questions
1.1) Retrieving best answers :
QA service is used to restoring the efficient answer to a similar
question.
A major issue in answer selection are:
1) Modeling of complex matching amongst Question and answer.
2) Modeling of the complex relationship among the response of a
given question.
A major challenge for CQA tasks is how to recognize the quality of
response to the correct question. Methods can involve the elaborate
task of extracting numerous features or are not e ffective for longer
answers[17]. As from the Figure 2,we can say that, the high-quality
answers are those which directly identify and explain the question.
Question:
How can I travel from Rajkot to Mumbai?
Expected answer:
Book yourself a ticket to fly to Mumbai.
Not Expected:
By Swimming…
Figure 1. An example of Question answering
1.2) Retrieving best questions :
User's query compared with the historical data and acquire the best
question[2]. The retrieving functional can take a lot of conveniences
and reduces the repeat rate for CQA platforms. Word mismatch and
semantic gap problem are the main challenges for CQA question
retreival[3]. Question retrieval’s main aim to match historical
questions which are relevant or equivalent to resolve user’s query
directly[18]. Whenever a user submits a new question, the system
will check whether similar questions have already been asked and
answered before or not. If it has been asked or answered previously
then the user's query can be resolved directly by returning the
archived questions with their answers.
 Query FAQ pair Relevant There are mainly 3 stages in IR based factoid QA system:
I want to Q: What are the steps to Yes 1. Question processing
remove my fb remove Facebook account? 2. Retrieval of document and passage
account, what A:Go to "Your Account". 3. Answer Retrieval
are the steps of And then next thing to do
that? is select option 2.1.1) Questison processing
"Deactivsate Account". The question can be asked in different forms, the user may ask it in
their own way. Users should recognize what the question is asking for
Q: How to make friends on No before proceeding to answer the question itself. The practice to
facebook? understand the question is known as question processing.
A: To make friends on
facebook, you need to 2.1.2) Retrieval of document and passage
send request to that Main function of document retrieval is to identify documents that
particular person. contain an answer, by utilizing the query which is provided by
question processing phase. And the main aim of passage retrieval is
Figure 2. An example of question retrieval. to reduce the search space for finding the answer from a set of
documents to a fixed number of passages.
2. Different types of question and answering systems:
2.1.3) Answer Retrieval
2.1) Information retrieval (IR) -based question answering This is the final stage of the IR based QA system. For example, if the
user asks “how tall is Mount Everest?” then the corresponding
IR depends on the huge amount of information which is available in answer should be 29,029 feet. But here some limitations are there
the form of text on the Web. such as definition questions which don’t tend to be a particular
In factoid based information retrieval QA, questions are based on the named type entity.
facts and answer is a named entity. (ex. Person, number, location, 2.2) Question and answering based on knowledge
etc...)
Example: Knowledge-based question answering is mainly used to convert the
natural language query into a structure database query.
example:
query = “What is the surname of john??”
it gets converted into database query SQL
SELECT surname FROM table1 WHERE name= ‘john’;
it returns the answer to the user.

2.3) Question and answering based on the story

This method is generally used to asking the question based on the
passage/story given.

3. The main objective of QA are:

Figure 3. Example of IR
In IR based Question answering, the answer is selected from the set of
documents, it just copies the content from the documents, it does not
generate the answer. So if the answer is not present in the given
document then it does not guarantee to answer.
1) To efficiently identify the quality of answers that are relevant to a
given question.
2) Retrieving answers to questions rather than full documents.
3) Finding small segments of text which answer user's questions.
4) To answer one's query directly by finding the most relevant
question from an archive of past question.
5) QA system helps users to retrieve accurate information by
obtaining the correct answer directly, rather than browsing through
large ranked lists of results.

Figure 4. Architecture of a factoid question answering system Figure 5. Overview of the CQA service.
4. Methodology:
Methodology used for retrieval information in CQA is dividing into
following phases.
4.1) Phase 1: Retrieval of high quality answer
The task of any Question Answering system is to get an answer for a
particular question[19]. A new architecture is proposed called Spatio-
Temporal LSTM (ST-LSTM) which is designed to effectively
influence not only the causal influence between the question-response
(how appropriate response is for a given question) but also spatial
and temporal interactions between responses how to form a high-
quality response gradually).
Figure 7. shows the process flow of the question retrieval model.
Whenever user issues a query it will be classified to the ranked list by
question classification[12]. And based on the ranked list, a general
score including the question retrieval score it will describe the
relevance of two questions.
Each question type describes an expected answer which reflects those
asker’s intention. Therefore, targeted question retrieval should pay
attention to search similar questions in particular types [12]. A
question, when understood correctly places some constraints on a
possible answer and determining those constraints becomes an
important task for any QA system. So, to enhance the accuracy of the
system, filtering out non-relevant candidate answers plays an
important part. This is done by Question Classification[20].

Table 1: Types of questions:

Type Purpose Expected answers

REC Recommendation Suggested item and
the reason
FAC finding facts Data, location, or
name
YNO Yes/No Yes/No and reason
HOW How-to question The instructions
WHY Seeking reason The explanation
DEF Definition The definition of
given entity
OPN Seeking opinion Opinions of other
users
TRA Transportation Navigation
instructions
INT Interactive Discussion thread
Figure 6. The overview of the proposed Spatio-Temporal LSTM (ST- MAT Math problem Solution with steps
LSTM) Architecture.
The label describes the possible intentions of a particular question.
ST-LSTM reads the question and its answers successively and gives The sequence in the ranking list reflects the similarity between
out their corresponding output for answer selection task (i.e., answer questions and a type. The more related a type to a query, the more
classification and answer ranking). The answer retrieval component likely it is that the types contain questions relevant to that query. We
extracts a set of candidate answers A for a question Q by two means. can use two machine learning algorithms: Naive Bayes(NB) and
One is obtained by searching webpages from the internet, the other is Support Vector Machine (SVM) for label ranking.
searching for the knowledge-based map database[16]. h(i) is the
hidden output of the ith ST-cell (e.g., h(4) is the hidden output of the 5. List of Algorithms can be used in CQA :
4th ST-cell in the network). (a) Question and its corresponding
answers with different quality in the training set. (b) All question or 5.1) LSTM :
all answer is considered as a single paragraph and use Paragraph2Vec
to generate their distributed representation.

4.2) Phase 2: Question Classification

Figure 7. Question retrieval model


Figure 8. LSTM Architecture(source:
http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
LSTM is an artificial RNN, and it's far used in the subject of deep
learning and it's far succesful to research long-time period
dependencies. LSTMs are specially designed to avoid the long-term
dependency problem[12]. It remember the information for long
periods of time. Figure 6. the exact architecture of LSTM networks.
LSTMs offer finer manage over what is needed via making
adjustments to the inner structure of a neuron, as can be seen in Fig.
7. Here, xt corresponds to the input vector at time step t (or the t th
word), ht−1 is the hidden state vector from the previous time step, and
the calculations for the various internal states such as “forget gate”
(controls which part of the information is propagated to the next time
step), “input gate” (controls which part of the input is considered as
important information), and cell state are also represented[15].
The questions are first surpassed thru word2vec models and their
respective embedding is acquired[12]. The query embedding is then
passed through a recurrent neural community and is similarly brought
to the embedding of memories[12]. The added records is then
surpassed via every other LSTM the encoded vectors from which
might be subsequently handed via softmax layer that makes solution
predictions.
5.2) Word Embedding Algorithm :
It also known as distributed semantic representations which play a
significant role in building continuous word vectors based on the
contexts in a large corpus. They learn a low-dimensional vector for
each vocabulary term in which the similarity between the word
vectors can show the syntactic and semantic similarities between the
corresponding words[13].
There are main two types of word embeddings.
5.2.1) Continuous Bag-of-Words model (CBoW):
It expect a word from sum of all phrase vector in its context.
Figure 8. Overview of the Continuous Bag-of-Words model.
5.2.2) Skip-gram model: It does the reverse of what the CBOW
model does.
5.3) PLSA(Probabilistic Latent Semantic Analysis) Algorithm :
In PSLA model words and documents can be mapped as vector.
Mainly it is used to improve answer validation.
For given sentence set S, a term set W and a topic set Z, the
conditional probability of sentence-term P(s, w) is shown in equation
(1)[14]:
(1)
In equation (1), P(w | z) represents the conditional possibility of
words in latent semantic layers and P(z | s) represents the conditional
possibility of topics in sentences[14].
The model suits with the EM algorithm and export the most useful
P(Z), P(W | Z) and P(Z | S). while a brand new question is coming,
it's far projected to the topic area Z through using EM set of
rules[14]. The similarity of the query and every sentence may be
received through computing the similarity of the probabilistic
distribution among them in the subject matter area.
First, initialize the P(s, w) for each sentence s ∈ 1,2 , ,..., n within the
retrieved files by using the ratio of the frequency of w in s and within
the sentence set; and P(w | z) for every phrase and P(z | s) for every
sentence are iteratively computed by means of EM set of rules. Then
the query constructed from the question is mapped into the subject
space to compute by way of using EM set of rules, preserving P(w |
z) continually[14]. After that, the conditional chance (w | q) Pwq and
(w | s ) Pws for each sentence in the retrieved files is computed
according to the formula (2)-(3)
(2)
(3)
Conclusion:
This paper focused on improving the various deep learning
methodology that have been proposed in question answering system.
The Method is only based on neural architecture and the recurrent
neural network. In this paper we've got discussed different sorts of
algorithms like LSTM, word embedding set of rules and
PLSA(Probabilistic Latent Semantic analysis) which can be used
in question answering system to retrieve the high quality question
and answer. LSTM is easier to train and LSTM can solve the difficult
task and effective at capturing long-term temporal dependencies.
There is a need to look at ways to solve ambiguous questions because
they are actually very common in real life scenarios.
REFERENCES:
1) NIVID LIMBASIYA AND PRATEEK AGRAWAL, “SEMANTIC 14) Han Ren, Donghong Ji, Chong Teng, Jing Wan, “A Web
TEXTUAL SIMILARITY AND FACTORIZATION MACHINE knowledge based approach for complex question
MODEL FOR RETRIEVAL OF QUESTION-ANSWERING” JULY answering”, December 18,2011, AIRS'11 Proceedings of
2019, ADVANCES IN COMPUTING AND DATA SCIENCES
the 7th Asia conference on Information Retrieval
Technology
2) XIN CAO1, GAO CONG1, 2, BIN CUI3, CHRISTIAN S.
15) G. Rohit, Ekta Gautam Dharamshi and Natarajan
JENSEN1, “A GENERALIZED FRAMEWORK OF EXPLORING
CATEGORY INFORMATION FOR QUESTION RETRIEVAL IN Subramanyam, “Approaches to Question Answering Using
COMMUNITY QUESTION ANSWER ARCHIVES” APRIL 26, LSTM and Memory Networks”, January 2019, Advances in
PROCEEDINGS OF THE 19TH INTERNATIONAL CONFERENCE Intelligent Systems and Computing 816, DOI:
ON WORLD WIDE WEB 10.1007/978-981-13-1592-3_15
3) Kai Wang, Tat-Seng, “Exploiting Salient Patterns for 16) Zhenzhen Li,Jiuming Huang, Zhongcheng Zhou, Haoyu
Question Detection and Question Retrieval in Community- Zhang, Shoufeng Chang, Zhijie Huang, “LSTM-based
based Question Answering” August 23,Proceedings of the Deep Learning Models for Answering Ranking”, March
23rd International Conference on Computational 2017, 2016 IEEE First International Conference on Data
Linguistics Science in Cyberspace (DSC)
4) Min Yang, Wenting Tu, Qiang Qu, Zhou Wei, Qiao Liu, Jia 17) Daniel Cohen, W. Bruce Croft, “End to End Long Short
Zhu, “Advanced community question answering by Term Memory Networks for Non-Factoid Question
leveraging external knowledge and multi-task learning”, 1 Answering”, September 2016, Proceedings of the 2016
May 2019, ELSEVIER, Knowledge-Based Systems ACM International Conference on the Theory of
5) Zheqian Chen, Chi Zhang, Zhou Zhao, Chengwei Yao, Information Retrieval
Deng Cai, “Question retrieval for community-based 18) Zheqian Chen, Chi Zhang, Zhou Zhao, Deng Cai,
question answering via heterogeneous social influential “Question Retrieval for Community-based Question
network” ,12 April 2018, ELSEVIER, Neurocomputing Answering via Heterogeneous Network Integration
6) Guangyou Zhoua,∗, Yin Zhoua, Tingting Hea, Wensheng Learning”, November 2016, arXiv:1611.08135v1 [cs.IR]
Wub, “Learning semantic representation with neural 19) Avani Chandurkar, Ajay Bansal, “Information Retrieval
networks for community question answering retrieval”, 1 from a Structured KnowledgeBase”, 2017 IEEE 11th
February 2016, ELSEVIER, Knowledge-Based Systems International Conference on Semantic Computing
7) Dalia Elalfy, Walla Gad, Rasha Ismail, “A hubrid model to 20) Kai Zhang, Wei Wu, Haocheng Wu, Zhoujun Li, Ming
predict best answers in question answering communities” , Zhou, “Question Retrieval with High Quality Answers in
March 2018, Egyptian Informatics journal Community Question Answering”, November 2014,
8) Guangyou Zhou ,Tingting He ,Jun Zhao ,and Po Hu1, Proceedings of the 23rd ACM International Conference on
“Learning Continuous Word Embedding with Metadata for Conference on Information and Knowledge Management
Question Retrieval in Community Question Answering”,
July 2015, 53rd Annual Meeting of the Association for
Computational Linguistics and the 7 th International joint
Conference on Natural Language Processing
9) Yashvardhan Sharmaa , Sahil Guptaa, “Deep Learning
Approaches for Question Answering System”, 2018,
Procedia Computer Science
10) MingTan,CicerodosSantos,BingXiang&BowenZhou,
“LSTM-Based deep learning models for nonfactroid
answer selection”, Mar 2016, arXiv:1511.04108
11) Jiahui Wen, Hongkui Tu, Xiaohui Cheng, Renquan Xie,
Wei Yin, “Joint modelling of users, questions and answers
for answer selection in CQA”, 15 march 2019, Elsevier,
Expert Systems with Applications
12) Wei Wang, Baichuan Li, Irwin King, “Improving Question
Retrieval in Community Question Answering with Label
Ranking”, 3 October 2011, IEEE, The 2011 International
Joint Conference on Neural Networks
13) Nouha Othman, Rim Faiz, Kamel Smaili, “A Word
Embedding based Method for Question Retrieval in
Community Question Answering”, December 2017,
International Conference On Natural Language Processing
and Speech Processing (ICNLSSP 2017), At Casablanca,
Morocco