Automatic Segmentation of Nerve Structures in Ultrasound Images
Using Graph Cuts and Gaussian Processes
Julian Gil Gonzalez, Mauricio A. Alvarez
Abstract Peripheral Nerve Blocking (PNB), is a procedure
used for performing regional anesthesia, that comprises the
administration of anesthetic in the proximity of a nerve. Several
techniques have been used with the purpose of locating nerve
structures when the PNB procedure is performed: anatomical
surface landmarks, elicitation of paresthesia, nerve stimulation
and ultrasound imaging. Among those, ultrasound imaging has
gained great attention because it is not invasive and offers an
accurate location of the nerve and the structures around it.
However, the segmentation of nerve structures in ultrasound
images is a difficult task for the specialist, since such images
are affected by echo perturbations and speckle noise. The
development of systems for the automatic segmentation of nerve
structures can aid the specialist for locating nerve structures
accurately. In this paper we present a methodology for the au-
tomatic segmentation of nerve structures in ultrasound images.
An initial step is carried out using Graph Cut segmentation in
order to generate regions of interest; we then use machine
learning techniques with the aim of segmenting the nerve
structure; here, a specific non-linear Wavelet transform is used
for the feature extraction stage, and Gaussian processes for the
classification step. The methodology performance is measured
in terms of accuracy and the dice coefficient. Results show that
the implemented methodology can be used for automatically
segmenting nerve structures.
I. INTRODUCTION
Regional anesthesia offers several advantages in compar-
ison of general anesthesia, mainly because it reduces the
hospital discharge [1]. The regional anesthesia is performed
using a procedure called Peripheral Nerve Blocking (PNB).
PNB consists in the administration of an anesthetic nearby
the nerve structure. The diffusion of the anesthetic through-
out the nerve inhibits its electrical conduction and thus
the transmission of nociceptive information to the central
nervous system [2]. The success of the PNB procedure
depends strongly on the fact that the anesthetic is supplied
in the right place. Hence, the correct location of the nerve
structures is an important step when regional anesthesia is
carried out [3]. Four methods are recognized for finding the
right place in order to perform PNB: Anatomical surface
landmarks, Elicitation of paresthesia, Nerve stimulation and
Ultrasound imaging [3]. Anatomical surface landmarks help
to find the region in which the needle has to be inserted;
however, this method is not very usual in clinical practice due
to anatomical differences among each patient [3]. Inducing
paresthesia during the exploration with a needle indicates
that the needle is in the proximity of a nerve. This method
Julian Gil Gonzalez, M. A. Alvarez
and Alvaro. A. Orozco are with
the Department of Electrical Engineering, Faculty of Engineering, Uni-
versidad Tecnologica de Pereira, Pereira, Colombia. {jugil, malvarez,
aaog}@utp.edu.co
978-1-4244-9270-1/15/$31.00 2015 IEEE 3089
and Alvaro A. Orozco
is not very common in practice due to the inconveniences
imposed upon the patient. Nerve stimulation has been the
most recommended method for performing PNB [3], it
comprises the electrical stimulation in the proximity of a
targeted nerve. The stimulated nerve produce a contraction in
the area innervated by the nerve. Unlike the above methods,
ultrasound imaging enables a non-invasive visualization of
the nerve, the structures around the nerve, the needle used
and the injected anesthetic to ensure adequate distribution
[3]. However, ultrasound imaging segmentation is a difficult
task due to these images are affected by several artifacts
such as attenuation, acoustic shadows, and speckle noise
[4]. Moreover, nerves are not static structures, hence their
location and shape, depends on body position, gravity and
external forces. The above is one of the reasons why the
specialists have to take long training sessions in order to ac-
quiring experience for recognizing nerve structures in ultra-
sound images [2]. The drawbacks associated with ultrasound
imaging can be minimized by developing automatic systems
that help the anesthesiologist to locate the nerves when
PNB procedures are performed. Several methods have been
proposed for segmenting anatomical structures in ultrasound
images. Between the methods developed for the segmenta-
tion in ultrasound images we can recognize, region growing
[4], classification [5], active shape models [6]. Examples
of anatomical structures studied include the thyroid gland
[5], some breast tumors [7], and the kidney [8]. However,
those techniques do not consider the characteristics associ-
ated with nerve structures. Recently, two methodologies [2],
[9] have been proposed for dealing the problem of nerve
segmentation in ultrasound images. The main problem of
the above methodologies, is that they are not automatic and
they require the intervention of a specialist. For instance,
[2], proposes an approach based on Gaussian processes for
classification where the region of interest should be defined
by an expert. On the other hand, [9], proposes a technique
based on Bayesian shape models, where the initial shape in
computed as the mean shape (mean nerve structure) derived
from the training set. However an expert should locate the
probably nerve region.
Unlike the above, we propose a fully automatic approach
for nerve segmentation in ultrasound images. A methodology
based on Graph Cuts segmentation is used in order to define
a region of interest, ROI. Then, we use a classification
scheme based on Gaussian processes in order to define the
region which corresponds to a nerve structure; here, the
feature extraction step is performed using a specific non-
linear Wavelet transform. The methodology performance is
measured in terms of the accuracy and the Dice coefficient.
II. MATERIALS AND METHODS
A. Dataset
The dataset used in this work comes from the Universidad
Tecnologica de Pereira and the Santa Monica Hospital,
Dosquebradas, Colombia. This dataset contains images from
the ulnar and median nerve (both nerves are located in the
forearm). Each image in the dataset was obtained by using a
Sonosite Nano-Maxx device, with a maximum resolution of
640480 pixels. The dataset was labeled by an anesthesiol-
ogist from the Santa Monica Hospital. Figure 1 shows the
types of images belonging to the dataset.
Fig. 1. Images contained in the dataset. Ulnar nerve (left) and median
nerve (right). The manual segmentation for both images are shown.
B. Graph Cuts Segmentation
Images segmentation using Graph Cuts consists in group-
ing the pixels into two clusters Object and Background.
Graph Cut segmentation, defines a non oriented graph G =<
V, >, which is defined by a set of nodes V (each pixel in
the image corresponds to a node in V ), and a set of non-
oriented edges that connect the neighboring nodes [10].
Two terminal nodes called source (S) and sink (T ) are added
to V in order to represent the Object and Background
clusters respectively [11]. Each pixel is connected to each
terminal node by a link named tlink. Moreover, each pixel
is connected to neighboring pixels by edges named nlink
[11]. The aim of the Graph Cut model is to find a subset of
edges C such that they a cost function defined as
E(A) = R(A) + B(A),
where A is a binary vector whose components A p specify the
label assigned to each pixel p V . If A p is 0, then the pixel
p belongs to Background group and if A p is 1, the pixel
belongs to Object cluster, the term R(A) is associated with
the penalty for assigning a pixel p to any class (Object
or Background). Meanwhile, the term B(A) can be seen as
the penalty for a discontinuity among pixels p and q. Finally,
the term 0 specifies a relative importance of the region
properties R(A) in comparison of boundary properties B(A)
(For more details about (1), see [10]).
It is possible to introduce topological constrains, in order to
improve the segmentations results and for reducing the search
space of possible solutions [10]. For introducing topological
constrains, we define two subsets of pixels O and B, these
subsets belong to the clusters Object and Background
respectively. The objective is to perform the segmentation,
in order to minimize the expression (1) such that it satisfies
the constrains. p O : A p = 1, p B : A p = 0.
C. Parametrization
The parametrization is carried out through the non-linear
Wavelet transform. This transform has demonstrated good
performance for modeling the textures associated to nerve
structures (according the results showed in [2]). The non-
linear Wavelet transform is constructed using the lifting
scheme proposed in [12]. The lifting schemes comprise a
modification made on the filters h and g associated to the
discrete Wavelet transform [12].
D. Classification
Let D = {X, y} be the training set with N samples, where
X is the input matrix composed by vectors {x1 , x2 , . . . , xN }
named the features vectors; and y is the target vector,
where yi {1, 1} correspond to the label assigned to ith
sample. The probability of a sample xi belonging to class
yi = 1 given by p(yi |xi ) is related with a latent function
fi . The relationship between p(yi |xi ) and fi is determined
by a squashing function () [13] (we use the sigmoid
function as the squashing function). A Gaussian Processes
prior p(f|X) = N (f|0, K) is placed over the latent function
f, where is the covariance matrix computed using a specific
kernel function (we use a squared exponential for computing
the covariance matrix). It can be shown that the probabilistic
prediction for
R
the new sample x? is given by [13] p(y? =
1|D, x? ) = ( f? )p( f? |D, x? )df? .
The above predictive distribution, has no analytic solution
and should be calculated using analytic approximations [13]
(we use a Laplace approximation in this work). The un-
specified parameters associated to the kernel function are
computed from the negative log of the marginal likelihood.
E. Procedure
The fully automatic segmentation of nerve structures is
performed through the following stages: ROI generation and
image pre-processing; Feature extraction; Classification; and
Post-processing.
(1)
1) ROI generation and image pre-processing: There are
several factors that transform the segmentation into a difficult
task. Among those factors are the noise and artifacts intro-
duced by the ultrasound device which have to be minimized
carefully in order to preserve anatomical details of the
structures in the image. Hence, a pre-processing stage is
necessary for enhancing the anatomical characteristics of the
structures in the image. The ROI generation and image pre-
processing are carried out following the next steps:
ROI generation: The nerves structures occupy a small
region in the ultrasound image. Thus, it is required to
generate a region of interest (ROI) in which the nerve
structure is located, in order to reduce the regions of analysis.
This stage is performed by using Graph Cuts Segmentation,
we use topological constrains in order to indicate that the
3090
pixels located at the edge of the image belong to the set among two binary images [14]. We use DC for comparing
Background (for the Graph Cuts segmentation we used the the segmented image with the image labeled by the special-
software available on http://cbia.fi.muni.cz/projects/graph- ist (considered as the ground-truth). The DC is computed
cut-library.html). Then, we apply the morphological oper- according to [14]. If there is a perfect overlap among two
ations of dilation and erosion, in order to improve the seg- images, then DC = 1, on the other hand, if there is not overlap
mentation results. Finally, we define the ROI by computing DC = 0 [14].
the bounding box of the segmented region.
III. R ESULTS AND DISCUSSION
Pre-processing: A median filter is applied on the generated
ROI with the aim of minimizing the speckle noise. Finally, The methodology proposed for the automatic segmentation
we use the morphological operators opening and closing in of nerve structures is tested on each image belonging to
order to improve the filtering results. the dataset. For performing this task, we first generate
2) Feature Extraction: For the feature extraction stage, the ROI by using the Graph Cut model, then we apply
the ROI is divided in blocks with size M M (in [2], it was a methodology based on machine learning techniques for
defined that M should be equal to four). Each block was blocking the nerve structures from the ROI generated.
characterized by computing the mean, standard deviation and Figure 1, shows the results of the model of Graph Cuts used
the entropy from the bands xHH, xHL, xLH, xLL obtained by for the generation of the ROI and using different values for
calculating the first level non-linear wavelet transform. For the parameter (See (1)).
this work, three different features sets are taken into account,
each of them using a different scheme for computing the
non-linear Wavelet. first set uses primal lifting scheme, the
second set uses dual lifting scheme. Finally, the third set uses
the Primal-Dual lifting scheme.
3) Classification: The segmentation of nerve structures in
ultrasound images configures a binary classification problem
(class 0 for a non-nerve region and class 1 for a nerve
region). The classifier is trained using the features extracted
(a) = 0.1 (b) = 1 (c) = 10
from each block with size 4 4, where each of those blocks
has been labeled by a specialist, as nerve or no-nerve. Fig. 2. Results of the segmentation using Graph Cuts with different values
Usually, there are more blocks labeled as non-nerve than for the parameter .
blocks labeled as nerve, hence it is necessary to ensure the
balance between the numbers of instances labeled as non- Analyzing the results shown in Figure 2, we can see that
nerve and the instances labeled as nerve in order to train the when the value of the parameter increases, there are fewer
classier properly. Now, for the test step, the ROI is divided regions labeled as nerve structures. However, when we use a
in blocks with size M M with 50% of overlapping. A set value of greater than 5, the segmentation failed for some
of features is extracted from each block using the non-linear images. In this sense, we choose = 1 for generating the
Wavelet transform. The trained classifier predicts each block ROI. Figure 3 shows the result of the ROI generated using
as nerve or no-nerve based on the features extracted from the model of Graph Cuts and the methodology proposed.
each of them. For each block classified as nerve, its closest 8
neighbors are analyzed, if there are at least 4 blocks labeled
as non-nerve, so the block analyzed is re-assigned to the
class no-nerve. Finally, the largest connected component is
considered as part of the nerve structure.
4) Post-processing: Using the methodology described
above, we obtain a region in which the nerve structure is
contained. Nevertheless, the shape of the segmented nerve
region is serrated. Hence, a post-processing step is required (a) Original Image (b) Graph Cut Segmen- (c) ROI generated
tation and morphologi-
in order to recover the complete nerve region. This step is cal operations
carried out using an specific region growing method based
on the concepts of convex hull which is proposed in [5]. Fig. 3. Results of the proposed methodology used for generating the ROI

F. Validation Now, the ROIs generated are divided in blocks with

The methodology evaluation in carried out using a leave- size 4 4, then the classifier based on Gaussian processes
one-out cross-validation scheme due to the small number of is used in order to define the corresponding region to a
images in the dataset. For measuring the performance of the nerve. Figure 4, shows the results for the methodology
methodology we use two metrics: the accuracy and the Dice implemented.
coefficient (DC). Comparing the images showed in the Figure 4, it is possible
The DC is used in medical imaging to measure the overlap to see that the proposed methodology can segment the nerve
3091

Fig. 4. Results for the implemented methodology. In the left the original
image is shown. In the right we compare the manual segmentation (red)
and the results of the implemented methodology (white)
structure. However, we note that some regions outside
the nerve structure were segmented as nerve structures.
Table I shows the performance obtained with the proposed
methodology using three different schemes for computing
the non-linear Wavelet transform.
The first configuration implements a parametrization using
the non-linear Wavelet transform based on the primal
lifting scheme (GC+ML1); for the second configuration
we extract the features using the dual lifting scheme to
compute the non-linear Wavelet transform (GC+ML2);
finally, for the third configuration, we use a non-linear
Wavelet transform based on the primal-dual lifting scheme
in the parametrization stage (GC+ML1). Furthermore, we
compare those results with another common segmentation
scheme such as active shape models [6], the model is
initialized with the results obtained with the graph cut
segmentation (GC+ASM). The comparison is carried out in
terms of accuracy and the Dice coefficient.
TABLE I
P ERFORMANCE OF THE IMPLEMENTED METHODOLOGY USED THREE
DIFFERENT CONFIGURATIONS . T HE RESULTS ARE COMPARED WITH A
SEGMENTATION SCHEME BASED IN ACTIVE SHAPE MODELS .
Dice Coefficient Accuracy
GC+ML1 0, 6019 0, 0031 0, 9774 0, 0002
GC+ML2 0, 6057 0, 0052 0, 9776 0, 0004
GC+ML3 0, 6008 0, 0036 0, 9780 0, 0002
GC+ASM 0, 5530 0, 1647 0, 9741 0, 0122
From results shown in Table I we can see that there are not
great differences when we change the way for computing the
non-linear Wavelet transform in the feature extraction step.
Moreover, we note that the implemented methodology has
a better performance in terms of DC when compared to the
model based on active shape models.
IV. C ONCLUSION
We presented a methodology based on Graph Cuts and
machine learning techniques applied to the automatic seg-
mentation of nerve structures in ultrasound images. The
proposed methodology automatically determines a region of
interest by using Graph cuts segmentation. The generated
3092
ROI is then used to segment the nerve structures by using a
classification scheme based on Gaussian Processes. Results
show that the proposed methodology segments accurately
nerve structures in ultrasound images. This methodology can
be used in order to aid the specialist when performing PNB
procedures applied to regional anesthesia. Future work can
be oriented to the development of an automatic segmentation
technique based on shape models where the initialization is
performed using Graph Cuts segmentation.
V. ACKNOWLEDGMENT
This work was developed under the project: Desarrollo
de una metodologa para la segmentacion automatica de re-
giones objetivo en imagenes ultrasonicas a partir de modelos
estadsticos. Aplicacion a los procedimientos de anestesia
regional, with financial support from the Universidad Tec-
nologica de Pereira. J. Gil Gonzalez is funded by Colciencias
under the program: Jovenes Investigadores e Innovadores -
Convocatoria 645 de 2014. Finally we want to thank the
Dr. Diego Salazar from Santa Monica Hospital, who helped
us to acquire the ultrasound images and to label the nerve
structures.
R EFERENCES
[1] Sites, B. D., & Brull, R. (2006). Ultrasound guidance in peripheral re-
gional anesthesia: philosophy, evidence-based medicine, and techniques.
Current Opinion in Anesthesiology, 19(6), 630-639.
[2] Gil, J., Alvarez, M. A., & Orozco, A. A. (2015). Peripheral nerves
segmentation in Ultrasound Images Using Non-linear Wavelets and
Gaussian Processes. In Iberian Conference on Pattern Recognition and
Image Analysis. IbPRIA 2015. To be published.
[3] Shi, J., Schwaiger, J., & Lueth, T. C. (2011, August). Nerve block using
a navigation system and ultrasound imaging for regional anesthesia. In
Engineering in Medicine and Biology Society, EMBC, 2011 Annual
International Conference of the IEEE (pp. 1153-1156). IEEE.
[4] Noble, J. A., & Boukerroui, D. (2006). Ultrasound image segmentation:
a survey. Medical Imaging, IEEE Transactions on, 25(8), 987-1010.
[5] Chang, C. Y., Lei, Y. F., Tseng, C. H., & Shih, S. R. (2010). Thyroid
segmentation and volume estimation in ultrasound images. Biomedical
Engineering, IEEE Transactions on, 57(6), 1348- 1357.
[6] Xue, Z., Li, S. Z., & Teoh, E. K. (2003). Bayesian shape model for
facial feature extraction and recognition. Pattern Recognition, 36(12),
2819-2833.
[7] Liu, B., Cheng, H. D., Huang, J., Tian, J., Tang, X., & Liu, J.
(2010). Fully automatic and segmentation-robust classification of breast
tumors based on local texture analysis of ultrasound images. Pattern
Recognition, 43(1), 280-298.
[8] Xie, J., Jiang, Y., & Tsui, H. T. (2005). Segmentation of kidney from
ultrasound images based on texture and shape priors. Medical Imaging,
IEEE Transactions on, 24(1), 45-57.
[9] Giraldo, J. J., Garca, H. F., Alvarez, M. A., & Orozco, A. A. (2015).
Peripheral Nerve Segmentation Using Speckle Removal and Bayesian
Shape Models. In Iberian Conference on Pattern Recognition and Image
Analysis. IbPRIA 2015. To be published.
[10] Boykov, Y., & Funka-Lea, G. (2006). Graph cuts and efficient ND
image segmentation. International journal of computer vision, 70(2),
109-131.
[11] Al-Agamy, A. O., & Fahmy, A. S. (2011, August). Segmentation
of strain-encoded magnetic resonance images using graph-cuts. In
Intelligent Computer Communication and Processing (ICCP), 2011
IEEE International Conference on (pp. 219-224). IEEE.
[12] Sweldens, W. (1998). The lifting scheme: A construction of second
generation wavelets. SIAM Journal on Mathematical Analysis, 29(2),
511-546.
[13] Rasmussen, C. E. (2006). Gaussian processes for machine learning.
[14] Sampat, M. P., Wang, Z., Gupta, S., Bovik, A. C., & Markey, M. K.
(2009). Complex wavelet structural similarity: A new image similarity
index. Image Processing, IEEE Transactions on, 18(11), 2385-2401.