Deep Learning Based

Complaint Classification for

Indonesia Telecommunication
Company’s Call Center
R E S U LT O F P R E L I M I N A R Y R E S E A R C H

SHINTA DEVI LUKITASARI

FADHIL HIDAYAT

MSC EIS - BANDU NG, O C TO B ER 12 T H 2019

INTRODUCTION
CallMiner
Index 2018
§ 72% of telecommunication
companies’ customers prefer the
call center to solve their problems
than self-service channels,
§ 57% of customers tended to think
about switching to competitors
when they had unpleasant
experiences by contacting the call
center.
Indonesia Telco Company
Call Center Problems
§ The company needs to provide a sufficient number of agents and channels so
that customers do not need to wait for a long time when they want to contact
the call center.
§ The running of business processes in handling complaints depends on the
accuracy of the call center agent in determining the classification of problems.
§ Conversations data will be recorded in a database, and the file should not be
deleted for a certain period. Extensive capacity and significant cost for
maintenance are needed.
Research Problems Validation
• At least 1 of 3 complaints is a repeat complaint submitted because the previous
complaint had not been resolved properly.
• Data obtained from recording machine are formatted in mono channel audio
file, weak audio volume, noisy, and overlapped.
• Conversation speech contain spontaneous reaction, non meaningful words and
the use of vernaculars.
This Paper Describes…
§ The literature review of deep learning algorithm for speech and text
data classification,
§ The methods used in preparing raw data into training data and
building system models,
§ The results of tests conducted on training data and developed
classification systems.
Research Methodology
Research Clarification Interview & Observation

Descriptive Study - I Data Sample Observation &

Literature Review

Prescriptive Study
Data Processing, Classifier Development,
Evaluation

Descriptive Study - II
Analysis
LITERATURE
REVIEW
 Related Works on Call Center Data
Draman, dkk. Nguyen,dkk. Arsikere, dkk. Sun, dkk. Kobayashi,dkk. Yigit,dkk. Chowdury, dkk.
(2017) (2016) (2016) (2016) (2016) (2017) (2017)
ASR Model Customer Customer Modeling
Research ASR Dialog Act Complaint
Data Satisfaction Satisfaction competitiveness
Objectives Development Tagging Distribution
Preparation Analysis Analysis in conversations
29,050 Corpus
90h Records of 23,932 Phone 158 BCC Data Training 4,000 Call 565
Data 4,116 Corpus
Conversations Calls Dialogs 21,926 Corpus Records Conversations
Data Test
Classification
Acoustic Text Mining Topic
N-Gram AM : GMM-SAT Model used Classification
Modeling Feature, used Logistic Modelling
Language & TDNN SVM, Max Model used
Algorithm Logistic Regression, used Decision
Model LM: 4-gram LM Entropy, SVM and DNN
Regression Feeling Index Tree
Bayesian
Baseline of Speech Classification System

Research opportunities:
§ Deep Learning algorithm for
classification,
§ Manual approach for database
preparation,
§ Data labeled based on Dialog Activities
and Context of Complaints,
§ Time analysis as infrastructure
environment impact.
 Related Works on Classification
with Recurrent Neural Network (RNN) gives the Better Result
Abdelwahab,
Ushio, dkk. Klosowski, Jose KJ, dkk. Abudalfa,dkk. Dumpala,dkk. Cai, dkk.
dkk.
(2016) (2018) (2018) (2018) (2018) (2018)
(2018)
Research Speech Act Language Sentiment Intent Sentiment Sentiment Text
Objectives Tagging Modeling Classification Prediction Classification Classification Classification

Data Speech Speech Text Speech Text Speech Text

Research SVM, CNN, Markov Chain, RNN, RNN-

RNN-LSTM RNN RNN CNN, RNN
Methods RNN-LSTM RNN LSTM, RNN-GRU

Accuracy on Accuracy on Accuracy on

Evaluation Accuracy on Accuracy on
Different Accuracy Accuracy Different Different
Parameters Different Epoch Different Epoch
Dataset Dataset Epoch
PROPOSED
WORK
 Data
Preparation
Workflow

No. Group of Contexts Label

1 Information Request
2 Registration
3 Billing Complaint
4 Technical Complaint
5 Non Technical Complaint
Classification
Model
Development

Accuracy Training
Model Epoch (%) Time (s)
LSTM 1 5 72.04 416
LSTM 1 10 85.74 815
GRU 1 5 74.21 318
GRU 1 10 86.61 592
Proposed System Model
 The methodology phase that has been carried out to design
a system model is presented, starting from research
clarification, literature review, training data preparation
until development and testing the classification models.

Conclusion & Future research is needed to evaluate the design in order to

produce higher accuracy by activating optimal functions in
Future Works deep learning algorithms.

To be implemented at operational the system designed

should be aligned with the company’s strategy to
determine the most effective and efficient infrastructure
design.