PROCEEDINGS OF SPIE

SPIEDigitalLibrary.org/conference-proceedings-of-spie

Computerized lung cancer

malignancy level analysis using 3D
texture features

Wenqing Sun, Xia Huang, Tzu-Liang Tseng, Jianying

Zhang, Wei Qian

Wenqing Sun, Xia Huang, Tzu-Liang Tseng, Jianying Zhang, Wei Qian,
"Computerized lung cancer malignancy level analysis using 3D texture
features," Proc. SPIE 9785, Medical Imaging 2016: Computer-Aided
Diagnosis, 978538 (24 March 2016); doi: 10.1117/12.2216329

Event: SPIE Medical Imaging, 2016, San Diego, California, United States

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use

 Computerized Lung Cancer Malignancy Level Analysis Using 3D Texture Features
Wenqing Sun, Xia Huang, Tzu-Liang (Bill) Tseng, Jianying Zhang, Wei Qian

Medical Imaging and Informatics Laboratory,

Department of Electrical & Computer Engineering, University of Texas, El Paso

Abstract:
Based on the likelihood of malignancy, the nodules are classified into five different levels in Lung
Image Database Consortium (LIDC) database. In this study, we tested the possibility of using three-
dimensional (3D) texture features to identify the malignancy level of each nodule. Five groups of
features were implemented and tested on 172 nodules with confident malignancy levels from four
radiologists. These five feature groups are: grey level co-occurrence matrix (GLCM) features, local
binary pattern (LBP) features, scale-invariant feature transform (SIFT) features, steerable features,
and wavelet features. Because of the high dimensionality of our proposed features, multidimensional
scaling (MDS) was used for dimension reduction. RUSBoost was applied for our extracted features
for classification, due to its advantages in handling imbalanced dataset. Each group of features and
the final combined features were used to classify nodules highly suspicious for cancer (level 5) and
moderately suspicious (level 4). The results showed that the area under the curve (AUC) and
accuracy are 0.7659 and 0.8365 when using the finalized features. These features were also tested on
differentiating benign and malignant cases, and the reported AUC and accuracy were 0.8901 and
0.9353.
Key Words: lung cancer, malignancy level, 3D texture features, computed tomography
1. Description of Purpose:
Lung cancer is the leading cause of cancer death for both men and women worldwide. The American
Cancer Society (ACS) reported that the early detection of lung cancer, stage 1, could significantly
increase the survival rate from 2% to 47% compared to the detection at stage 5. However, only 15%
of early stage lung cancers are detected. Computer-aided diagnosis (CADx) system has the potential
to aid radiologists as a second reader and attracted much attention in the last few decades.
Computed tomography (CT) is typically used for lung cancer screening and diagnosis in clinic. A
single CT examination can generate up to 700 axial images creating a challenging task for image
interpretation. From the reported literature, most research groups are mainly analyzing lung images
via two-dimensional (2D) features either from only one single representative slice or from multiple
slices. In addition, three-dimensional (3D) features are more descriptive than 2D features because
they provide not only the complete information on every slice but also the connections between
adjacent slices. A few researchers investigated 3D texture features to distinguish benign and
malignant lung nodules. In this study, we analyzed application of 3D texture features to classify lung
nodules into different malignancy levels. To the best of our knowledge, no other research group has
reported usage of 3D texture features to differentiate lung nodule malignancy levels.

Medical Imaging 2016: Computer-Aided Diagnosis, edited by Georgia D. Tourassi, Samuel G. Armato III,
Proc. of SPIE Vol. 9785, 978538 · © 2016 SPIE · CCC code: 1605-7422/16/$18 · doi: 10.1117/12.2216329

Proc. of SPIE Vol. 9785 978538-1

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 2. Materials and Methods:
2.1 Data
Our dataset consists of 1018 CT cases downloaded from Lung Image Database Consortium and
Image Database Resource Initiative (LIDC/IDRI), which contained CT slices of each patients and
radiologists’ ratings recorded in XML files. For majority of the cases, four experienced thoracic
radiologists diagnosed them and marked the boundary of each nodule with the greatest in-plane
dimension larger than 3 mm. Besides the nodule boundary information, other nodule characteristics
such as subtlety, internal texture, and likelihood of malignancy were also included in the annotation
files. From these 1018 cases, we eliminated cases with no larger than 3 mm nodules, non-nodule
lesions, incomplete cases, and cases with missing truth files. All the selected nodules were marked
by all four radiologists. Because of the inter-observer variation in defining nodule boundaries, we
chose the union region from the markings of all the radiologists.
Because of the different CT scanning protocols across different vendors, the volume resolution
varies across the dataset with the range from 0.5 mm to 3 mm. To avoid the partial volume effects,
bi-cubic interpolation method was used to normalize CT volumes and boundaries marked by the all
four radiologists resulting in isotropic resolution. Each radiologist gave a malignancy likelihood
rating score in the range of 1 to 5, with 1 representing highly unlikely for cancer and 5 representing
highly suspicious for cancer. Figure 1 shows an example of five different nodules with malignancy
level 1 to 5.

Figure 1: An example of nodules with different malignancy levels. Figure a to e represent malignancy level 1 to level 5.

2.2 Computational features

In this study, we modified and redesined the 2D features in our previous work [1-3] to 3D features.
For each cubic ROI, five different types of 3D texture features were extracted: grey level co-
occurrence matrix (GLCM) feature, local binary pattern (LBP) feature, scale-invariant feature
transform (SIFT) feature, steerable feature, and wavelet feature. The number of features for each
feature group are summarized in Table 1.
The first group of feature is 3D GLCM feature. In grey level co-occurrence matrix, the number of
rows and columns is equal to the number of gray levels in original image, and each element
represent the relative frequency of two pixels with given intensity separated by a pixel distance. In
3D cases, we calculated GLCM in 13 directions, including (0, 1, 0), (-1, 1, 0), (-1, 0, 0), (-1, -1, 0),

Proc. of SPIE Vol. 9785 978538-2

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 (0, 1, -1), (0, 0, -1), (0, -1, -1), (-1, 0, -1), (1, 0, -1), (-1, 1, -1), (1, -1, -1), (-1, -1, -1), (1, 1, -1). Then
the mean and standard deviation were computed for every matrix [4].
The second group of feature is 3D LBP feature. It was first proposed as a texture descriptor for 2D
images [5]. From the equally divided blocks of the original image, the extracted LBP features can
describe the local and global textures. We computed the LBP features by comparing each pixel with
the 26 neighbor pixels, and it returns the 26 digits binary code for each pixel. Then the local binary
pattern feature histogram was calculated from the coded image.
In the third feature group, we computed 3D SIFT features. This scale invariant feature transform is
invariant to uniform scaling, orientation, and it is also partially invariant to affine distortion and
illumination changes [6]. Because of this property, it is a classical algorithm in object matching and
action recognition. When creating descriptors, the sub-region was considered in our algorithm
instead of every pixel, thus it is more adaptive to the image noises and subtle distortions. In our
experiment, we took 8 by 8 by 8 neighbors of the key point descriptor and divided them into eight
blocks. Each block we measure the 27 bins orientation histogram, and altogether we get 216 bins
histogram to describe each key point. Mean and variance were measured for all the key points in one
image.
In group four, we tested 3D steerable features. The steerable filters are linear combination of a set of
basis filters with arbitrary orientations [7]. Similar to 2D classical steerable feature extraction, in our
3D procedure we applied non-maximum suppression to the filtered images [8]. From the response
images, we calculated the mean, variance, skewness, entropy, and energy.
The feature group five contains 3D wavelet features, and they provide the spatial and frequency
representation of the CT images. In 3D wavelet transform, high-pass and low-pass filters are used in
all the three dimensions. For each scale, one input image can generate eight sub-band images: HHH,
HHL, HLH, HLL, LHH, LHL, LLH, and LLL, where H represent high frequency band and L
represent low frequency band [9]. In this study, we used two different scales, and thus generating 15
different images. For each image, the mean and variance were calculated, so 30 wavelet features
were generated in this group.
Table 1: Number of calculated features in each feature group

Feature group Number of features

1 26
2 256
3 532
4 5
5 30
Total 849

2.3 Dimension reduction and classification

Because we have a large number of computational features compared to the dataset, dimension
reduction techniques were used. In this study we used multidimensional scaling (MDS), which is an
algorithm placing each object in N dimensional space and at the same time preserving the between

Proc. of SPIE Vol. 9785 978538-3

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 object distaances [10]. Since
S the maalignancy rattings are nott evenly distrributed in ouur dataset (Fiigure
1), we usedd a boosting method callled RUSBooost as our claassification algorithm.
a It is a hybrid
method thaat combines random undder-samplingg (RUS) and the standardd boosting prrocedure
AdaBoost resulting
r in efficient moodeling of thee minority class by remooving majoriity class sam mples
[11]. To fuully take advantage of thee limited dattaset, we useed leave-onee-out methodd for evaluatiion, so
every data will be used d once for ass testing dataa.
3. Computtational Ressults:
From the 1018
1 cases, we acquiredd 172 nodulles with connfident maliggnancy likelihood ratinggs from
four radiollogists. We defined
d the confident
c raatings as at least three ouut of four raadiologists give
g the
same ratingg score to one
o nodule, anda the distrribution of thet confidennt ratings aree listed in figure 2.
The cases with
w ratings 1 or 2 are liikely benign, the cases with
w ratings 4 or 5 are likkely malignaant, and
the score 3 represent inndeterminatee cases. To teest the possiibility of usinng our propoosed 3D feattures to
differentiatte the high and
a low likellihood of maalignancy caases, we usedd each featurre group to classify
c
level 4 andd level 5 casses. The resuults are showwn in Table 2. Then wee tested the performance
p e of our
system by combining all features together. The MDS dim mension redduction methhod was appplied to
each featurre group, and d then we incorporated them
t to the feature
f pool. The reporteed accuracy is i 0.85.
When com mbining all th he features together,
t all the experim ments are baased on the features
f afteer MDS
procedure, because thee total numbeer of features far exceedds the total nuumber of casses.
To have a better
b understanding of the
t ability off these 3D feeatures to distinguish noodules with
different malignancy
m leevels, we callculated andd compared thhe AUCs ussing nodules with differeent
malignancyy levels and the relevantt AUCs are shown
s in Tabble 3.

100
Amount

50

0
1 2 3 4 5
Maalignancy lev
vel
Figure 2: Thhe malignancyy level distributtion of confidennt data.

Table 2: Thhe accuracy and

d AUC value comparison
c of differentiating
d level 4 and 5 cases
c using diffferent feature groups.
g

Feature
F Accuuracy AUC
A
group Without With Without With
MDS MDS MDS MDS
3D
D GLCM 0.7510 0.8158 0.7010 0.7358
3D LBP 0.7950 0.8226 0.7150 0.7526
3D
D SIFT 0.7845 0.8126 0.7245 0.7426
3D Steerable 0.7703 0.8065 0.7003 0.7365
3D Wavelet 0.7438 0.7681 0.6938 0.7181
All N/A 0.8365 N/A 0.7659

Proc. of SPIE Vol. 9785 978538-4

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Table 3: Comparison of AUC on classifying different malignancy level group cases.

Level 2 Level 3 Level 4 Level 5

Level 1 0.6118 0.6518 0.9073 0.9827
Level 2 0.5599 0.8550 0.9641
Level 3 0.8047 0.9575
Level 4 0.7659
Figure 3 shows three ROC curves for classifying malignancy level 4 and level 5, level 1 and level 5
cases, and benign (both level 1 and level 2) and malignant (both level 4 and level 5) cases. The
accuracy and AUC for classifying level 1 and level 5 cases are 0.98 and 0.96; for benign and
malignant cases are 0.94 and 0.89, respectively.

ROC
1r

0.9-

0.8- ,1
0.7 -1

0.6 -j
¢
0.5 -.-.-.
2 0.4 -

0.3 -
- - Level 4 vs5
0.2 - -Level 1 vs 5
-- Benign vs malignant
0.1 -

0
0,1 0.2 0.3 0.4 0.5 0.6 0'3 0.0 0.9
False Positive Rate

Figure 3: Three ROC curves: malignancy level 4 and level 5 cases; level 1 and level 5 cases; benign and malignant cases

4. Conclusions:
In this study, we tested the feasibility of using 3D texture features to classify nodule malignancy
levels. Compared to other 3D lung cancer feature researches, most of the 3D features were first used
for lung cancer diagnosis, and we analyzed the cancer malignancy levels using these features instead
of classifying benign and malignant nodules only [12]. In our experiment, we tested five different
groups of 3D features: GLCM features, LBP features, SIFT features, steerable features, and wavelet
features. Every group of 3D texture features can differentiate the cases highly suspicious for cancer
and moderately suspicious, and using the combined features resulted in better performance with
accuracy of 0.84 and AUC of 0.77. Among these five feature groups, 3D LBP features outperformed
the other feature groups in this dataset.
We also compared the difficulties of distinguishing different malignancy level cases using the 3D
features, and we found level 1 and level 2 are the hardest to classify and level 1 and level 5 are the
easiest. It was also noted that distinguishing level 4 and level 5 cases gives better results than
distinguishing level 1 and level 2 cases.
From the results, we can see distinguishing nodule malignancy levels is significantly harder than
classifying benign and malignant nodules only. However, the 3D features proposed in this study

Proc. of SPIE Vol. 9785 978538-5

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 showed differentiating power for both purposes. Combining with other image based and non-image
based information, 3D features are promising way to precisely diagnose cancer levels. Future work
will investigate more on 3D features and evaluate performance of these features and algorithms from
a larger dataset.
References:
[1] R. Samala, Psychophysical similarity based feature selection for nodule retrieval in CT,
University of Texas, El Paso, (2011).
[2] R. Samala, W. Moreno, Y. You et al., “A novel approach to nodule feature optimization on
thin section thoracic ct,” Academic radiology, 16(4), 418-427 (2009).
[3] R. K. Samala, W. A. Moreno, D. Song et al., "Knowledge based optimum feature selection for
lung nodule diagnosis on thin section thoracic CT." 726036-726036-8.
[4] W. Sun, T.-L. B. Tseng, B. Zheng, J. Zhang, and W. Qian, “A new breast cancer risk analysis
approach using features extracted from multiple sub-regions on bilateral mammograms,” in
SPIE Medical Imaging. International Society for Optics and Photonics, 2015, vol. 9414, p.
941422.
[5] T. Ojala, M. Pietikäinen, and T. Mäenpää, “Multiresolution gray-scale and rotation invariant
texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol.
24, no. 7, pp. 971–987, 2002.
[6] D. G. Lowe, “Object recognition from local scale-invariant features,” Proc. Seventh IEEE Int.
Conf. Comput. Vis., vol. 2, 1999.
[7] W. T. Freeman and E. H. Adelson, “The design and use of steerable filters,” Pattern Anal.
Mach. Intell. IEEE Trans., vol. 13, no. 9, pp. 891–906, 1991.
[8] F. Aguet, M. Jacob, and M. Unser, “Three-dimensional feature detection using optimal
steerable filters,” in Proceedings - International Conference on Image Processing, ICIP, 2005,
vol. 2, pp. 1158–1161.
[9] W. Sun, B. Zheng, F. Lure, T. Wu, J. Zhang, B. Y. Wang, E. C. Saltzstein, and W. Qian,
“Prediction of near-term risk of developing breast cancer using computerized features from
bilateral mammograms,” Comput. Med. Imaging Graph., vol. 38, no. 5, pp. 348–357, 2014.
[10] I. Borg and P. J. F. Groenen, Modern Multidimensional Scaling, vol. 94, no. 445. 2005.
[11] C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano, “RUSBoost: A hybrid
approach to alleviating class imbalance,” IEEE Trans. Syst. Man, Cybern. Part ASystems
Humans, vol. 40, no. 1, pp. 185–197, 2010.
[12] Han, F., Wang, H., Song, B., Zhang, G., Lu, H., Moore, W., ... & Liang, Z. (2013, February).
A new 3D texture feature based computer-aided diagnosis approach to differentiate
pulmonary nodules. In SPIE Medical Imaging (pp. 86702Z-86702Z). International Society for
Optics and Photonics.

Proc. of SPIE Vol. 9785 978538-6

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 25 Sep 2019
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use