International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882

Volume 3, Issue 9, December 2014

1269

REINFORCEMENT LEARNING FRAMEWORK FOR MEDICAL IMAGE

SEGMENTATION USING MEDICAL IMAGES
Saravanan. A1, Dr.S.K.Srivasta2 and Dr.S.Gunesaskaran3
1

Research Scholar ,Dept of ECE ,Bharath University Chennai, India

Professor, Dept of ECE St.Joseph College of Engineering Chennai, India
3
Professor And Dean Dept of ECE Rajalakahmi Engineering Collegei, Chennai, India
2

ABSTRACT
Image segmentation is a key procedure required for
medical image comprehension and intensification. Main
aim here is to segment the attractive objects from the
environment by relating the information of the object.
This paper presents a strengthening knowledge structure
for medical image segmentation. This can also be used
for other general purpose image segmentation
progression effectively. The model introduced here is
adaptive not only to a defined objective function but also
to the users intention and prior knowledge. Based on
this concept, a general segmentation framework using
reinforcement learning is proposed. The proposed frame
work can assimilate specific user intention and
behaviour seamlessly in the background. The results of
the previous step itself can be utilised for the purpose of
learning. The method can be established for an implicit
model and it can also be generalized to different image
contents or segmentation requirements based on
reinforcement learning.
Index Terms Cardiac image segmentation, contextspecific segmentation, reinforcement learning.

I. INTRODUCTION
Many efforts have been made and various methods have
been introduced in order to segment medical images
automatically. Generally medical image segmentation
always requires a huge amount of qualified personnel in
the form of observers to ensure the quality of the
segmented medical image. This is because the
automatically segmented medical images are not always
completely reliable. As one of the best image clustering
methods, fuzzy local information C-means is often used
for image segmentation. The effects of noise are avoided
by utilizing the spatial relationship among pixels, but it
often generates boundary zones for the mix pixels
around the edges [1]. An Auto Context Model (ACM)

based automatic liver segmentation algorithm, which

combines ACM, multiatlases, and mean-shift techniques
to segment liver from 3-D CT images [2].
In order to improve the precision and efficiency of
microelectronic bonding, an Improved Adaptive Genetic
Algorithm (IAGA) for the image segmentation and
vision alignment of the solder joints in the
microelectronic chips [3]. Landform mapping is more
important than ever before, yet the automatic recognition
of specific landforms remains difficult. Object-Based
Image Analysis (OBIA) steps out as one of the most
promising techniques for tackling this issue [4]. In
computer vision, semantically accurate segmentation of
an object is considered to be a critical problem. The
different looking fragments of the same object impose
the main challenge of producing a good segmentation
[5].
The parasite segmentation problem has been addressed
via computational image analysis, but only for a few
species and images free of fecal impurities. In routine,
fecal impurities are a real challenge for automatic image
analysis [6].The development of 3-D ultrasonic probes
and 3-D Ultrasound (3DUS) imaging offers new
functionalities that call for specific image processing
developments, an original method for the segmentation
of the Utero-Fetal Unit (UFU) from 3DUS volumes,
acquired during the first trimester of gestation [7]. An
identification system for Chinese Materia Medicas in
microscopic powder images. The imaging processing of
the microscopic powder image is very complex because
of the low contrast, blurry boundaries, overlapping
objects, and messy background [8].
The multiresolution technique is one of the most
important techniques for image segmentation. Wavelet
transformation is a pixel-based method and is widely
used for multiresolution segmentation approaches, but it
suffers the deficiency of modelling the macro texture

www.ijsret.org

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 3, Issue 9, December 2014

pattern of a given image [9]. Gastroenterology imaging

is an essential tool to detect gastrointestinal cancer in
patients.
Computer assisted diagnosis is desirable to help us
improve the reliability of this detection [10]. The wellknown Finite Mixture Model (FMM) has been regarded
as a useful tool for image segmentation application.
However, the pixels in FMM are considered independent
of each other and the spatial relationship between
neighbouring pixels is not taken into account [11].
Segmentation of injured or unusual anatomic structures
in medical imagery is a problem that has continued to
elude fully automated solutions. The goal of easy to- use
and consistent interactive segmentation is transformed
into a control synthesis problem. It presents a processing
chain for the change detection of Arctic glaciers from
multitemporal multipolarization Synthetic Aperture
Radar (SAR) images [13].
Prohibitory traffic signs play an important role in
guiding, warning and regulating traffic system.
As driving over the speed limit is often the major cause
of accidents, detecting this group of prohibitory signs
may reduce this danger [14]. Researches are done to
develop robust methods for segmentation of
multitemporal SAR and optical data and to investigate
the fusion of multitemporal Advanced Synthetic
Aperture Radar (ASAR) and Chinese HJ-1B
multispectral data for detailed urban land cover
mapping [15].
The individualization of an object from a digital image is
a common problem in the field of image processing. A
modified version of the watershed algorithm for image
segmentation is proposed [16]. [17] Introduces a new
approach for the segmentation of skin lesions in
dermoscopic images based on Wavelet Network (WN).
The WN presented here is a member of fixed-grid WNs
that is formed with no need of training.

1270

swelling, blockage of urine flow etc [19]. Identification

of canopy species is done in a Hawaiian tropical forest
using supervised classification applied to airborne hyper
spectral imagery acquired with the Carnegie Airborne
Observatory-Alpha system. A clear advantage is found
in using regularized discriminant analysis, linear
discriminant analysis, and support vector machines [20].
Currently the hand wash screening audit is done
manually by an expert to monitor the hand under
ultraviolet light once its been washed. Hence, there is a
need for more human experts to conduct the screening
manually. Hand wash screening audit by using a vision
system is proposed to automate the process [21].
The Triplet Markov Field (TMF) model recently
proposed is suitable for tackling the non stationary
image segmentation. A hierarchical model for
unsupervised SAR image segmentation [22]. A method
was developed to detect generic objects using a single
query image. The query image could be a typical real
image, a virtual image, or even a hand-drawn sketch of
the object [23]. A real-time system that utilizes hand
gestures to interactively control the presentation is
proposed. The system employs a thermal camera for
robust human body segmentation to handle the complex
background and varying illumination posed by the
projector [24].
The researches have all been carried out with the aim to
improve the quality of certain type of image
segmentation or for a certain field. This reduces the
overall generalizability of the algorithms that have been
developed over the long years for some specific
purposes. [25] Shows the switching between boundary
and region based segmentation to improve the quality of
the low contrast image segmentation. With the
motivation from it reinforcement learning has been used
here to improve the quality of the medical images
segmented.

II. REINFORCEMENT LEARNING

A new FCM-based method for spatially coherent and
noise-robust image segmentation. The segmentation
results, for both synthetic and real images, demonstrate
that our method efficiently preserves the homogeneity of
the regions and is more robust to noise than related
FCM-based methods [18].
Ultrasound imaging plays crucial roles in medical field
to estimate kidney size, position, and appearance and
helps to detect structural abnormalities as well as the
presence of cysts, stones, cancer, congenital anomalies,

Reinforcement learning is an area of machine learning

process inspired by behaviorist psychology, and
concerned with how software agents have to take actions
in an environment so as to maximize some notion of
cumulative reward. The problem, due to the
reinforcement learning generality, is studied in many
other disciplines, such as game theory, control theory,
operations research, information theory, simulationbased optimization, statistics, and genetic algorithms. In

www.ijsret.org

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 3, Issue 9, December 2014

the fields of operations research and control literature

reinforcement learning studies are called approximate
dynamic programming. The problem has been studied in
the theory of optimal control, though most studies there
are concerned with existence of optimal solutions and
their characterization, and not with the learning or
approximation aspects. In economics and game theory,
reinforcement learning may be used to explain how
equilibrium may arise under bounded rationality.

1271

adding the latest result shape into the training set, so that
the variability of the model is increased. Shape
instantiation with relatively few points is often able to
predict the shape reasonably close to the correct contour
for an organ with complex geometry such as the Right
Ventricle (RV). However, to ensure an accurate
constraint, interpolation is needed based on spline
fitting, forming hybrid constraints.

III. IMPLEMENTATION
In machine learning, the environment is typically
formulated as a Markov Decision Process (MDP), and
many reinforcement learning algorithms for this context
are highly related to dynamic programming techniques.
The main difference between the classical techniques
and reinforcement learning algorithms is that the latter
do not need knowledge about the MDP and classical
techniques target large MDPs where exact methods
become infeasible.
Reinforcement learning differs from standard supervised
learning in that correct input/output pairs are never
presented, nor sub-optimal actions explicitly corrected.
Further, there is a focus on on-line performance, which
involves finding a balance between exploration of
uncharted territory and exploitation of current
knowledge. The exploration vs. exploitation trade-off in
reinforcement learning has been most thoroughly studied
through the multi-armed bandit problem and in finite
MDPs.
The required parameters can be given in to the initially
segmented image with the help of the reinforcement
learning. The parameters can be given in to the system
for reinforcement learning with the help of the shape
instantiation. Shape instantiation concept uses a set of
sparse control points to predict the entire shape of the
required object of interest. It can also be used for
examining and predicting the motion of the myocardium,
as well as the intra-operative tracking of 3-D anatomy
and interventional devices. Usually it is based on a
statistical model anda corresponding regression
algorithm. Partial Least Squares Regression (PLSR) can
be chosen for shape instantiation, because it is more
robust than other algorithms such as Multiple Linear
Regression (MLR) and Principal Components
Regression (PCR). The user-example positions and the
appearance-searched positions with high confidence are
the input data, while the output is the entire shape. The
PLSR model is initially trained with a set of training
data. There after it is dynamically updated every time a
new segmentation is acknowledged by the user, i.e., by

Image segmentation algorithms can be categorized

according to the various parameters like appearance,
geometry, motion characteristics and so on. Within each
category, there may be different strategies for making
use of the information for contour localization. Each
strategy can be given a suitable weight and cost
functions as required so that it helps the process of
choosing the suitable algorithm for segmentation
accordingly with good ease and efficiency. For
simplicity, the overall cost function for locating a
specific part of the contour can be described as a
combination of all the candidate strategies from different
categories. (1)
where is the cost function of the ith candidate strategy in
the cth category, is the corresponding intra-category
probability of that strategy being used, nc is the number
of strategies in the category, is the probability of the cth
category being used or the cross-category probability
and m is the number of strategy categories.
Once the algorithms are categorized the input image can
be given and with the required parameters being
specified the suitable selection of algorithms and the
segmentation process can be performed with the help of
the reinforcement learning efficiently. If two or more
candidate strategies are being involved then they can be
combined accordingly so that the efficient segmentation
of the intended medical image can be performed to
obtain the high quality segmented image automatically.
Once the training process is being completed the images
can be segmented with greater ease and lower time
requirements. The various steps involved in the
implementation of the reinforcement learning for
medical image segmentation are as follows
Step 1: Get the image and initialize the shape consisting
of m control points.
Step 2: Do the segmentation to obtain the required
shape, with the cost efficient algorithm.

www.ijsret.org

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 3, Issue 9, December 2014

1272

Step 3: If user not satisfied with the segmentation output,

get the user specified input control points for
segmentation
Step 4: If the control points do not converge then learn
for a new state
Step 5: If no user information is available for control
points, then generalize the knowledge from previous
control points.
Step 6: repeat again from Step 2: to obtain the required
results.
After completing all the above steps the segmented
medical images can be obtained with greater ease and
efficiency. With the states being defined for the known
case it can also be used for the purpose of generalization
when the required parameters are not known perfectly,
thus helping to generalize the frame work more
efficiently for wider range of image segmentation
processes.

b) Left ventricle with the papillary and endocardial

muscles highlighted.
Fig.2. Left ventricle segmented with the combination of
the categorized strategies.

IV. CONCLUSION
Thus an improved method for medical image
segmentation has been proposed and implemented with
greater accuracy. High ability of the reinforcement
learning for getting the accurate segmented images is
being used in this work. Many image segmentation
algorithms involved were always concentrating on single
type of image segmentation or applicable only to a
certain category of image segmentation. Here with the
help of the reinforcement learning and categorizing the
various segmentation algorithms a higher degree of
generalized method for image segmentation is obtained.
Since the quality of the medical image segmentation is
very high the number of qualified personal required for
verification can be reduced considerably.

REFERENCES
Fig. 1. C-T image of a heart affected with HypertrophicCardiomyopathy

[1] Nan Li, Hong Huo, Yu-ming Zhao, Xi Chen, and

Tao Fang A Spatial Clustering Method With Edge
Weighting for Image Segmentation ; (2013)
[2] ; Hongwei Ji, Jiangping He, Xin Yang, Rudi
Deklerck, and Jan Cornelis, Senior Member ACM-Based
Automatic Liver Segmentation From 3-D CT Images by
Combining Multiple Atlases and Improved Mean-Shift
Techniques, IEEE (2013)

a) Left ventricle segmented from a healthy heart C-T

image

[3] Fujun Wang, Junlan Li, Shiwei Liu, Xingyu Zhao,

Dawei Zhang, and Yanling Tian An Improved Adaptive
Genetic Algorithm for Image Segmentation and Vision
Alignment Used in Microelectronic Bonding;.
IEEE(2013)
[4] S. dOleire-Oltmanns, C. Eisank, L. Dragut, and T.
Blaschke An Object-Based Workflow to Extract

www.ijsret.org

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 3, Issue 9, December 2014

Landforms at Multiple Scales From Two Distinct Data

Types ;. IEEE(2013)
[5] Fahim Irfan Alam, Muhammad Iqbal Hasan
Chowdhury, Md. Reza Rabbani, Fateha Khanam Bappee
An Optimized Image Segmentation Algorithm;.
IEEE(2013).
[6] Celso T. N.Suzuki, Jancarlo F. Gomes, Alexandre X.
Falcao, Joao P. Papa, and Sumie Hoshino-Shimizu
Automatic Segmentation and Classification of Human
Intestinal Parasites From Microscopy Images;. IEEE
(2013).
[7] Jeremie Anquez, Elsa D.Angelini, Senior Member,
IEEE, Gilles Grange, and Isabelle Bloch Automatic
Segmentation of Antenatal -D Ultrasound Images; ,
Member, IEEE (2013)
[8] Shen Yan, Yaoli Li, Yixu Song**, Li Lin, Peifa
Jia, Shaoqing Cai Identification of Chinese Materia
Medicas in Microscopic Powder Images;. IEEE(2012)
[9] Chen Zheng, Leiguang Wang, Rongyuan Chen, and
Xiaohui Chen Image Segmentation Using MultiregionResolution MRF Model;. IEEE(2013)
[10] Farhan Riaz, Francisco Baldaque Silva, Mario
Dinis Ribeiro, and Miguel Tavares Coimbra Impact of
Visual
Features
on
the
Segmentation
of
Gastroenterology Images Using Normalized Cuts;,
Member, IEEE (2013)
[11] Hui Zhang, Member, IEEE, Q. M. Jonathan Wu,
Senior Member Incorporating Mean Template Into
Finite Mixture Model for Image Segmentation;, IEEE,
and Thanh Minh Nguyen. IEEE(2013)
[12] Peter Karasev, Ivan Kolesov, Karl Fritscher,
Patricio Vela, Phillip Mitchell, Allen Tannenbaum
Interactive Medical Image Segmentation using PDE
Control of Active Contours ;.IEEE (2013)

1273

[15] Object-Based Fusion of Multitemporal Multiangle

ENVISAT ASAR and HJ-1B Multispectral Data for
Urban Land-Cover Mapping; Yifang Ban, Member,
IEEE, and Alexander Jacob, Student Member,
IEEE.(2013)
[16] Segmentation of Color Image using Adaptive
Thresholding and Masking with Watershed Algorithm;
Md. Habibur Rahman, Md. Rafiqul Islam. IEEE(2013)
[17] Segmentation of Dermoscopy Images Using t
Networks Amir Reza Sadri, Maryam Zekri, Member,
IEEE, Saeed Sadri, Niloofar Gheissari, Mojgan
Mokhtari, and Farzaneh Kolahdouzan. IEEE(2013)
[18] Spatially Coherent Fuzzy Clustering for Accurate
and Noise-Robust Image Segmentation; Ivana
Despotovi,
Student
Member,
IEEE,
Ewout
Vansteenkiste, and Wilfried Philips, Member,
IEEE(2013)
[19] Speckle Noise Reduction and Segmentation of
Kidney Regions From Ultrasound Image; Tanzila
Rahman, Mohammad Shorif Uddin. IEEE(2013)
[20] Tree Species Discrimination in Tropical Forests
Using Airborne Imaging Spectroscopy; Jean-Baptiste
Fret and Gregory P. Asner. IEEE(2013)
[21] Unclean Hand Detection Machine using
VisionSensor; Faradila Naim , Rawaida Jaafar, Nurul
WahidahArshad, Rosyati Hamid, Mohd Najib Razali.
IEEE(2013)
[22] Unsupervised SAR Image Segmentation Using a
Hierarchical TMF Model; Peng Zhang, Ming Li, Yan
Wu, Gaofeng Liu, Hongmeng Chen, and Lu Jia.
IEEE(2013)
[23] A Generic Object Detection Using a Single Query
Image without Training; Bin Xiong, Xiaoqing

[13] Vahid Akbari, Student Member, IEEE, Anthony P.

Doulgeris, Member, IEEE, and Torbjrn Eltoft, Member
Monitoring Glacier Changes Using Multitemporal
Multipolarization SAR Images;, IEEE(2013)
[14] NIGHT MODE PROHIBITORY TRAFFIC SIGNS
DETECTION; Rubel Biswas, Arif Khan, Md. Zahangir
Alom. Mumit Khan. IEEE (2013)

www.ijsret.org