ORIGINAL RESEARCH
Application of Computer-Aided
Diagnosis on Breast Ultrasonography
Evaluation of Diagnostic Performances and Agreement
of Radiologists According to Different Levels of Experience
Eun Cho, MD, Eun-Kyung Kim, MD, PhD
Received November 22, 2016, from the Depart-
ment of Radiology, Severance Hospital, Re-
search Institute of Radiological Science (E.C.,
E.-K.K., J.H.Y.), and Department of Research
Affairs, Biostatistics Collaboration Unit
(M.K.S.), Yonsei University College of Medicine,
Seoul, Korea. Manuscript accepted for
publication April 10, 2017.
This study was supported by the research
fund of Samsung Medison Co, Ltd.
Address correspondence to Jung Hyun
Yoon, MD, PhD, Department of Radiology,
Severance Hospital, Research Institute of
Radiological Science, Yonsei University College
of Medicine, 50-1 Yonse-ro, Seodaemun-gu,
Seoul 03-722, Korea.
E-mail: lvjenny@yuhs.ac
Abbreviations
AUC, area under the receiver operating
characteristic curve; BI-RADS, Breast Imag-
ing Reporting and Data System; CAD, com-
puter-aided diagnosis; NPV, negative
predictive value; PPV, positive predictive
value; US, ultrasonography
doi:10.1002/jum.14332
C 2017 by the American Institute of Ultrasound in Medicine | J Ultrasound Med 2017; 00:00–00 | 0278-4297 | www.aium.org
V
, Mi Kyung Song, MS, Jung Hyun Yoon, MD, PhD
Objectives—To investigate the feasibility of a computer-aided diagnosis (CAD) sys-
tem (S-Detect; Samsung Medison, Co, Ltd, Seoul, Korea) for breast ultrasonography
(US), according to radiologists with various degrees of experience in breast imaging.
Methods—From December 2015 to March 2016, 119 breast masses in 116 women
were included. Ultrasonographic images of the breast masses were retrospectively
reviewed and analyzed by 2 radiologists specializing in breast imaging (7 and 1 years
of experience, respectively) and S-Detect, according to the individual ultrasono-
graphic descriptors from the fifth edition of the American College of Radiology
Breast Imaging Reporting and Data System and final assessment categories. Diag-
nostic performance and the interobserver agreement among the radiologists and S-
Detect was calculated and compared.
Results—Among the 119 breast masses, 54 (45.4%) were malignant, and 65
(54.6%) were benign. Compared to the radiologists, S-Detect had higher specificity
(90.8% compared to 49.2% and 55.4%) and positive predictive value (PPV; 86.7%
compared to 60.7% and 63.8%) (all P < .001). Both radiologists had significantly
improved specificity, PPV, and accuracy when using S-Detect compared to US alone
(all P < .001). The area under the receiving operating characteristic curves of the
both radiologists did not show a significant improvement when applying S-Detect
compared to US alone (all P > .05). Moderate agreement was seen in final assess-
ments made by each radiologist and S-Detect (j 5 0.40 and 0.45, respectively).
Conclusions—S-Detect is a clinically feasible diagnostic tool that can be used to
improve the specificity, PPV, and accuracy of breast US, with a moderate degree of
agreement in final assessments, regardless of the experience of the radiologist.
Key Words—breast; Breast Imaging Reporting and Data System; computer-
aided diagnosis; neoplasm; ultrasonography; ultrasound equipment and
products
O
ver the recent decades, breast ultrasonography (US) has
been widely available in everyday practice. Based on that
trend, the American College of Radiology Breast Imaging
Reporting and Data System (BI-RADS) was released and recently
updated.1 The ultrasonographic BI-RADS provides a sonographic lexi-
con and final assessment categories, which have excellent diagnostic
performances when applied to breast US in the differential diagnosis
of breast masses and suggest standardized management for patients.2,3
In spite of the excellent performances reported by using the
ultrasonographic BI-RADS, the final assessments made for breast
Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography
masses by different performers are known to vary signifi-
cantly,4,5 mostly because of the multiple BI-RADS ultra-
sonographic descriptors used for describing breast
lesions and the subjectiveness of US. To increase the
diagnostic accuracy of breast US, several additional ultra-
sonographic techniques have been developed and
applied in clinical practice, such as elastography,6 auto-
mated breast US,7,8 and computer-aided diagnosis
(CAD) systems.9–13 Among these additional imaging
modalities, CAD systems have been reported to enable
efficient interpretation, in which consistent improved
accuracy can be expected.14,15 S-Detect (Samsung Medi-
son, Co, Ltd, Seoul, Korea), a recently developed com-
mercially available imaging analysis program for breast
US, analyzes the morphologic features of a target breast
mass according to the BI-RADS ultrasonographic
descriptors and provides a final assessment, which is
based on analysis of the ultrasonographic features.16
This program may be useful in clinical practice, since it
can be used to provide a second opinion on tumor char-
acterization and guidance in deciding on patient treat-
ment. In a recent study analyzing the diagnostic
performances of S-Detect and a dedicated breast radiol-
ogist,16 S-Detect had equivalent diagnostic performance
and moderate agreement with the radiologist. The use-
fulness of CAD for breast US can differ according to the
level of experience of the radiologist, as reported in a
recent study,17 but the previous study used in-house
software that is not commercially available, which had its
limitations in that the analytic results could differ accord-
ing to different algorithms or models used for analysis.
Therefore, the purpose of this study was to investigate
the clinical feasibility of S-Detect when applied to breast
US by comparing the diagnostic performances and
agreement between S-Detect and radiologists with vari-
ous degrees of experience in breast imaging.
Materials and Methods
This study was a prospective study, and it was approved
by the Institutional Review Board of Severance Hospital.
Informed consent was obtained from all patients for
study inclusion.
Patients
From December 2015 to March 2016, 126 breast
masses in 123 consecutive women who were scheduled
for breast US-guided core needle biopsy, surgical
2 J Ultrasound Med 2017; 00:00–00
excision, or a combination thereof were included in this
study. Among them, 7 women were excluded for non-
mass lesions seen on breast US (n 5 4), since BI-RADS
descriptors are hard to apply on these lesions, and lack
of grayscale images of the targeted breast mass stored in
the US machine for S-Detect analysis (n 5 3). Only elas-
tograms and Doppler US images were stored for the 3
cases with images not fit for the S-Detect analysis.
Finally, 119 breast masses in 116 women were included
in this study.
Ultrasonographic Examinations
Ultrasonographic examinations were performed with a
3–12-MHz linear transducer (RS80A with Prestige;
Samsung Medison, Co, Ltd). Two staff radiologists
(J.H.Y. and E.-K.K.) with 7 and 19 years of experience in
breast imaging, respectively, were involved in image
acquisition. Bilateral whole-breast ultrasonographic
examinations were routinely performed, according to
scanning protocols including representative transverse
and longitudinal images of the breast masses with and
without calipers used for size measurements. The same
radiologist who had performed breast US proceeded
with US-guided core needle biopsy. Both radiologists
had access to the mammographic images obtained
before US, images of ultrasonographic examinations per-
formed in the past, and medical records containing clini-
cal information of the patients.
Image Review and Application of S-Detect
Ultrasonographic images of the 119 breast masses were
retrospectively reviewed and documented for data analy-
sis by 2 breast radiologists (J.H.Y., radiologist 1; and
E.C., radiologist 2) with 7 and 1 years of breast imaging
experience, respectively. All observers were blinded to
the clinical information and pathologic results of each
mass during image review. Breast masses were analyzed
according to the descriptors and final assessment catego-
ries used in the fifth edition of the BI-RADS and final
assessments (Table 1).1 Calcifications were not analyzed
because of the limited data analytic ability of S-Detect
for calcifications.16 The radiologists individually chose
the most appropriate term to describe each lesion for
each descriptor and made a final assessment accordingly.
The ultrasonographic BI-RADS final assessment catego-
ries were made as follows: category 2 (benign), 3 (prob-
ably benign), 4a (low suspicion of malignancy), 4b
(moderate suspicion of malignancy), 4c (high suspicion
of malignancy), and 5 (highly suggestive of malignancy).
 Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography

After image review by the radiologists, S-Detect was Data and Statistical Analyses
applied to the same image the radiologists used for gray- Histopathologic results from US-guided core needle
scale ultrasonographic feature analysis. A region of inter- biopsy, vacuum-assisted excision, or surgery were
est was either automatically or manually drawn along the regarded as the reference standards. Patients with a
border of the mass by S-Detect. Analytic results of S- diagnosis of high-risk lesions, such as atypical ductal
Detect, including the BI-RADS lesion descriptors and hyperplasia, intraductal papilloma, radial scar, and lobu-
final assessments, were immediately displayed and re- lar carcinoma in situ, were recommended to have further
corded for data analysis (Figure 1). After being informed surgical treatment, for which the final pathologic diagno-
of the final assessment made by S-Detect, each radiolog- sis was considered. Final histopathological diagnoses of
ist gave a final assessment for each breast mass, integrat- high-risk lesions were considered benign for the statisti-
ing the analytic results of S-Detect. cal analysis.
Because the final assessment data from S-Detect
were in a dichotomized form, “possibly benign” and
“possibly malignant,” final assessments from the radiol-
Table 1. Ultrasonographic Descriptors Used for Image Analysis ogists based on the BI-RADS were also divided into 2
Characteristic Descriptors groups for statistical analysis: positive assessments con-
Shape Oval, round, irregular
sisted of categories 4a to 5, and negative assessments
Orientation Parallel, nonparallel consisted of categories 2 and 3. Diagnostic performances
Margin Circumscribed, not circumscribed: of the individual radiologists, S-Detect, and the integra-
indistinct, angular, microlobulated, tion of S-Detect with each radiologist were analyzed and
speculated
Echo pattern Anechoic, hyperechoic, complex
compared, including sensitivity, specificity, positive pre-
cystic and solid, hypoechoic, dictive value (PPV), negative predictive value (NPV),
isoechoic, heterogeneous and accuracy. The generalized estimating equation
Posterior acoustic No posterior features, method was used for comparing the diagnostic perform-
features enhancement, shadowing,
combined
ances between radiologists and S-Detect. The area under
the receiver operating characteristic curve (AUC) was

Figure 1. Representative image of setting the region of interest for S-Detect analysis in a 52-year-old woman with a diagnosis of cancer in her left
breast. The region of interest was set automatically along the margin of the breast mass for analysis (green line). After the region of interest was
set, the ultrasonographic features were automatically analyzed by S-Detect, and a final assessment was automatically visualized.

J Ultrasound Med 2017; 00:00–00 3

Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography

obtained and compared by the Delong method. j statis-
tics were calculated to assess the agreement for ultraso-
nographic descriptors and final assessments among the
radiologists, S-Detect, and the integration of each radiol-
ogist and S-Detect. Estimation of the overall j was based
on a study by Landis and Koch18: less than 0 indicated
poor agreement; 0.00 to 0.20 indicated slight agreement;
0.21 to 0.40 indicated fair agreement; 0.41 to 0.60 indi-
cated moderate agreement; 0.61 to 0.80 indicated sub-
stantial agreement; and 0.81 to 1.00 indicated almost
perfect agreement. The weighted least squares approach
of Barnhart and Williamson19 was used to compare j
coefficients. All statistical analyses in this study were
performed with SAS version 9.2 software (SAS Inc,
Cary, NC), P < .05 was considered to have statistical
significance.
Results
Of the 119 breast masses included in this study, 65
(54.6%) lesions were benign, and 54 (45.4%) lesions
were malignant. The mean age 6 SD of the 116 women
included in this study was 48.5 6 12.2 years (range, 20–
83 years). The mean size of the breast masses was
16.9 6 10.7 mm (range, 4–60 mm). Among the 119
breast masses, 39 (33.3%) lesions were diagnosed by
surgery, 8 (6.7%) by US-guided vacuum-assisted
excision, and 103 (86.6%) by US-guided core needle
biopsy. Fourteen (11.8%) lesions were diagnosed as
benign according to ultrasonographic features without
additional biopsy, among which 8 were considered
benign, since they showed benign ultrasonographic fea-
tures along with stability during imaging follow-up for
greater than 2 years, 1 being a cyst, and 5 proven as gal-
actoceles on aspiration. The mean size of the malignant
masses was significantly larger than that of the benign
masses: 17.0 mm (range, 4.0–45.0 mm) and 11.0 mm
(range, 4.0–60.0 mm), respectively (P < .001).
Diagnostic Performances of the Radiologists
Versus S-Detect
The distribution of breast masses with histopathologic
diagnoses analyzed by the radiologists and S-Detect is
summarized in Table 2. There were no significant differ-
ences in S-Detect results between benign and malignant
masses for both radiologists (P > .05).
Table 3 summarizes the diagnostic performances of
the radiologists and S-Detect. The sensitivity and NPV
were significantly higher for both radiologists compared
to S-Detect, whereas the specificity, PPV, and accuracy
were higher for S-Detect (all P < .05). The AUCs of the
radiologists were significantly higher than that of S-
Detect (0.887 and 0.901 compared to 0.815, respec-
tively; P 5 .023 and .004). When the results of S-Detect

Table 2. Distribution of Breast Masses Analyzed by the Radiologists and S-Detect

Radiologist 1 Radiologist 2
Final Assessment S-Detect Benign Malignant Total Pa Benign Malignant Total Pa

2 Total 4 (100.0) 0 (0.0) 4 NA 0 (0.0) 0 (0.0) 0 NA

Probably benign 4 (100.0) 0 (0.0) 4 0 (0.0) 0 (0.0) 0
Possibly malignant 0 (0.0) 0 (0.0) 0 0 (0. 0) 0 (0.0) 0
3 Total 28 (100.0) 3 (100.0) 31 NA 36 (100.0) 3 (100.0) 39 NA
Probably benign 28 (100.0) 3 (100.0) 31 36 (100.0) 3 (100.0) 39
Possibly malignant 0 (0.0) 0 (0.0) 0 0 (0.0) 0 (0.0) 0
4a Total 26 (100.0) 7 (100.0) 33 .052 23 (100.0) 9 (100.0) 32 .121
Probably benign 24 (92.3) 4 (57.1) 28 21 (91.3) 6 (66.7) 27
Possibly malignant 2 (7.7) 3 (42.9) 5 2 (8.7) 3 (33.3) 5
4b Total 1 (100.0) 0 (0.0) 1 NA 4 (100.0) 8 (100.0) 12 .548
Probably benign 1 (100.0) 0 (0.0) 1 2 (50.0) 2 (25.0) 4
Possibly malignant 0 (0.0) 0 (0.0) 0 2 (50.0) 6 (75.0) 8
4c Total 4 (100.0) 24 (100.0) 28 .574 2 (100.0) 16 (100.0) 18 >.999
Probably benign 2 (50.0) 7 (29.2) 9 0 (0.0) 3 (18.7) 3
Possibly malignant 2 (50.0) 17 (70.8) 19 2 (100.0) 13 (81.3) 15
5 Total 2 (100.0) 20 (100.0) 22 >.999 0 (0.0) 18 (100.0) 18 NA
Probably benign 0 (0.0) 1 (5.0) 1 0 (0.0) 1 (5.6) 1
Possibly malignant 2 (100.0) 19 (95.0) 21 0 (0.0) 17 (94.4) 17

Data are presented as number (percent) where applicable. NA indicates not applicable.
a
Comparison between each radiologist and S-Detect in each BI-RADS category analyzed by the radiologist.

4 J Ultrasound Med 2017; 00:00–00

 Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography

were integrated, the specificity, PPV, and accuracy Figure 2. Receiver operating characteristic curves for the radiologists,
showed significant improvement compared to the per- S-Detect, and integration of the radiologists and S-Detect.

formances of the individual radiologists alone (all

P < .05). The sensitivity of radiologist 2 integrated with
the result of S-Detect was lower than that of radiologist
2 alone (P < .05). There was no significant difference
between the sensitivity of radiologist 1 integrated with
the result of S-Detect and that of radiologist 1 alone
(P 5 .170). The AUCs of the radiologists integrated
with S-Detect did not show significant differences when
compared to the AUCs of the radiologists alone (0.895
compared to 0.887 and 0.901 compared to 0.901; all
P > .05; Figure 2).
Agreement for Ultrasonographic Descriptors and
Final Assessment Between the Radiologists and S-
Detect
Agreement between the radiologists and S-Detect for
ultrasonographic descriptors and final assessments are
summarized in Table 4. Between the radiologists, agree-
ments were substantial for shape (j 5 0.72), orientation
(j 5 0.72), and posterior acoustic features (j 5 0.76)
and fair for margin (j 5 0.35) and echo pattern
(j 5 0.33). Moderate agreement (j 5 0.57) was seen
for final assessments between the radiologists. Agree-
ment between radiologist 1 and S-Detect for the ultraso-
nographic descriptors ranged from fair to moderate
(j 5 0.23–0.55). Agreement between radiologist 2 and

Table 3. Diagnostic Performances of the Radiologists With and Without Integration of S-Detect

Diagnostic Radiologist Radiologist

Performance Radiologist 1 Radiologist 2 S-Detect Pa Pb 1 1 S-Detect Pc 2 1 S-Detect Pd
Sensitivity, % 94.4 94.4 72.2 <.001 <.001 87.0 .170 83.3 .040
(88.3–100.0) (88.3–100.0) (60.3–84.2) (78.1–96.0) (73.4–93.3)
Specificity, % 49.2 55.4 90.8 <.001 <.001 86.2 <.001 87.7 <.001
(37.1–61.4) (43.3–67.5) (83.7–97.8) (77.8–94.6) (79.7–95.7)
PPV, % 60.7 63.8 86.7 <.001 <.001 83.9 <.001 84.9 <.001
(50.3–71.2) (53.2–74.3) (76.7–96.6) (74.3–93.6) (75.3–94.5)
NPV, % 91.4 92.3 79.7 .015 .006 88.9 >.999 86.4 .595
(82.2–100.0) (83.9–100.0) (70.6–88.9) (81.1–96.7) (78.1–94.6)
Accuracy, % 69.8 73.1 82.4 .014 .060 86.6 <.001 85.7 .015
(61.5–78.0) (65.1–81.1) (75.5–89.2) (80.4–92.7) (79.4–92.0)
AUC 0.887 0.901 0.815 .023 .004 0.895 >.999 0.901 >.999
(0.826–0.947) (0.846–0.956) (0.745–0.885) (0.835– (0.844–0.958)
0.956)

Values in parentheses are 95% confidence intervals.

a
Comparison between radiologist 1 and S-Detect.
b
Comparison between radiologist 2 and S-Detect.
c
Comparison between radiologist 1 and radiologist 1 1 S-Detect.
d
Comparison between radiologist 2 and radiologist 2 1 S-Detect.

J Ultrasound Med 2017; 00:00–00 5

Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography

S-Detect ranged from fair to substantial (j 5 0.31–
0.68). When the BI-RADS final assessments of the radi-
ologists were dichotomously divided into benign (cate-
gories 2 and 3) and malignant (categories 4a–5), fair and
moderate agreements were seen between the radiologists
and S-Detect (j 5 0.40 and 0.45, respectively).
Interobserver agreements for the shape, orientation,
and posterior features between the radiologists were sig-
nificantly higher than those between the radiologists and
S-Detect (P < .05). For the final assessment categories,
there were no significant differences in the agreement
between the radiologists versus that between the individ-
ual radiologists and S-Detect (P > .05).
Discussion
The ultrasonographic BI-RADS lexicon is widely used
for breast lesion descriptions, but because of the subjec-
tive tendency of US, observer variability is inevitable,
and it can lead to inconsistent diagnoses among per-
formers.2,5 Computer-aided diagnosis systems were
recently applied to overcome the observer variability of
breast US,20 as well as to improve the diagnostic per-
formances.14,15 S-Detect applies a novel feature extrac-
tion technique and support vector machine classifier that
categorizes breast masses into benign or malignant
according to the suggested feature combinations inte-
grated according to the contents of BI-RADS ultrasono-
graphic descriptors.21 In a recent study, Kim et al16
reported that S-Detect had significantly higher specific-
ity, PPV, accuracy, and AUC compared to a dedicated
breast radiologist, with fair to substantial agreement in
ultrasonographic feature analysis for breast masses.
However, in clinical practice, radiologists with different
levels of experience perform breast US, and the useful-
ness of S-Detect may be different according to the level
of experience: for example, radiologists with less experi-
ence may benefit more by using S-Detect.
In our study, S-Detect showed significantly higher
specificity, PPV, and accuracy compared to the radiolog-
ists (all P < .001). When integrating the results of S-
Detect, it led to significant improvements in specificity,
PPV, and accuracy in both radiologists, similar to the
results of previous studies for CAD systems. In addition,
24 of 26 (92.3%) and 21 of 23 (91.3%) benign breast
masses initially assessed as category 4a by the radiologists
were categorized as probably benign by S-Detect (Table
2). Based on our results, S-Detect could be used as an
additional tool with breast US regardless of the level of
experience the radiologist has and may be used to reduce
the number of unnecessary biopsies of benign breast
masses. Although the specificity, PPV, and accuracy were
improved, the AUCs of the radiologists integrated with S-
Detect had no significant differences compared to those
of the radiologists alone (all P > .05). Both of the radiol-
ogists already had very high AUC values; therefore, little
was left to improve, which may have been the cause for
no differences in AUCs after the integration of S-Detect.
Several reports have been published on applying dif-
ferent types of CAD to breast US.16,17,22 These studies
commonly reported that the CAD systems enable
improvement in diagnostic performances of breast US,
especially specificity and accuracy. Shen et al22 suggested
that computer-aided classification systems could be help-
ful in assessing indeterminate category 4 cases. Wang
et al17 concluded that the inclusion of CAD was more
helpful for junior radiologists than the seniors, with
greater improvement in the diagnostic performances in

Table 4. Interobserver Agreement for BI-RADS Sonographic Descriptors and Final Assessment Categories Among the Radiologists and
S-Detect

j, Radiologist 1 j, Radiologist 1 j, Radiologist 2

BI-RADS Lexicon vs Radiologist 2 vs S-Detect Vs S-Detect Pa Pb Pc

Shape 0.72 (0.62–0.83) 0.44 (0.30–0.58) 0.44 (0.30–0.58) <.001 <.001 .996
Orientation 0.72 (0.58–0.85) 0.55 (0.39–0.71) 0.68 (0.54–0.83) .024 .694 .053
Margin 0.35 (0.24–0.46) 0.28 (0.15–0.39) 0.31 (0.19–0.43) .316 .582 .656
Echo pattern 0.33 (0.18–0.48) 0.23 (0.03–0.44) 0.35 (0.19–0.51) .355 .883 .311
Posterior acoustic features 0.76 (0.66–0.86) 0.35 (0.22–0.49) 0.46 (0.32–0.61) <.001 .001 .037
Final assessment 0.57 (0.41–0.73) 0.40 (0.28–0.53) 0.45 (0.33–0.58) .050 .223 .392

Values in parentheses are 95% confidence intervals.

a
Comparison between radiologist 1 versus radiologist 2 and radiologist 1 versus S-Detect.
b
Comparison between radiologist 1 versus radiologist 2 and radiologist 2 versus S-Detect.
c
Comparison between radiologist 1 versus S-Detect and radiologist 2 versus S-Detect.

6 J Ultrasound Med 2017; 00:00–00

 Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography
the junior group. In our study, the specificity, PPV, and
accuracy of both radiologists were improved, supporting
the idea that S-Detect can be relied on to provide a sec-
ond opinion that can be used to decide the next step for
patient treatment. In addition, S-Detect is a commer-
cially available, easy-to-use program that enables imme-
diate visualization of analytic results during real-time
ultrasonographic examinations, which can easily be
incorporated in daily practice.
In our study, agreement for the BI-RADS ultraso-
nographic descriptors between the radiologists and S-
Detect was fair to substantial. For final assessments, the
agreement between the radiologists and the agreement
between each radiologist and S-Detect was moderate,
without significant differences. The level of agreement
for final assessments between each radiologist and S-
Detect were in a similar range as the agreement bet-
ween the radiologists in this study (j 5 0.57), and also
the agreement among radiologists reported in the liter-
ature (j 5 0.28–0.65).2,5,23 In addition, as S-Detect
provides the final assessment in a dichotomized form
of possibly benign and possibly malignant, we consider
this factor to have affected the low levels of agreement
among the radiologists. Further investigation with tech-
nical improvements is anticipated for a more sophisti-
cated algorithm using the multiple final-assessment BI-
RADS ultrasonographic categories.
There were a few limitations to our study. First,
present study included a relatively small number of cases
(n 5 119). Second, the presence of calcifications, which
is a strong predictor for breast malignancy,24,25 was not
included in the analyses because of the limited analytic
ability of S-Detect for microcalcifications. Similarly, non-
mass lesions were not included in this study, which may
have affected the results. Third, representative still
images stored for analysis were used during image analy-
sis by the radiologists and S-Detect. Variability also exists
in selecting representative images of a breast mass.
Fourth, 60 cases (50.4%) of breast masses included this
study were diagnosed by core needle biopsy only. The
false-negative rate of core needle biopsy was less than
2% in the literature.26 Therefore, this factor may have
had little effect on our results.
In conclusion, S-Detect is a clinically feasible diag-
nostic tool that can be used to improve the specificity,
PPV, and accuracy of breast US, with a moderate degree
of agreement in final assessments, regardless of the expe-
rience of the radiologist.
J Ultrasound Med 2017; 00:00–00
References
1. D’Orsi CJ, Sickles EA, Mendelson EB, et al. ACR BI-RADSV Atlas:
R
Breast Imaging Reporting and Data System. 5th ed. Reston, VA: Ameri-
can College of Radiology; 2013.
2. Lazarus E, Mainiero MB, Schepps B, Koelliker SL, Livingston LS. BI-
RADS lexicon for US and mammography: interobserver variability
and positive predictive value. Radiology 2006; 239:385–391.
3. Yoon JH, Kim MJ, Moon HJ, Kwak JY, Kim EK. Subcategorization
of ultrasonographic BI-RADS category 4: positive predictive value
and clinical factors affecting it. Ultrasound Med Biol 2011; 37:
693–699.
4. Hong AS, Rosen EL, Soo MS, Baker JA. BI-RADS for sonography:
positive and negative predictive values of sonographic features. AJR
Am J Roentgenol 2005; 184:1260–1265.
5. Lee HJ, Kim EK, Kim MJ, et al. Observer variability of Breast Imaging
Reporting and Data System (BI-RADS) for breast ultrasound. Eur J
Radiol 2008; 65:293–298.
6. Au FW, Ghai S, Moshonov H, et al. Diagnostic performance of quan-
titative shear wave elastography in the evaluation of solid breast
masses: determination of the most discriminatory parameter. AJR Am
J Roentgenol 2014; 203:W328–W336.
7. Kim EJ, Kim SH, Kang BJ, Kim YJ. Interobserver agreement on the
interpretation of automated whole breast ultrasonography. Ultrasonog-
raphy 2014; 33:252–258.
8. Shin HJ, Kim HH, Cha JH, Park JH, Lee KE, Kim JH. Automated
ultrasound of the breast for diagnosis: interobserver agreement on
lesion detection and characterization. AJR Am J Roentgenol 2011; 197:
747–754.
9. Chang RF, Wu WJ, Moon WK, Chen DR. Automatic ultrasound
segmentation and morphology based diagnosis of solid breast tumors.
Breast Cancer Res Treat 2005; 89:179–185.
10. Chen CM, Chou YH, Han KC, et al. Breast lesions on sonograms:
computer-aided diagnosis with nearly setting-independent features
and artificial neural networks. Radiology 2003; 226:504–514.
11. Drukker K, Giger ML, Vyborny CJ, Mendelson EB. Computerized
detection and classification of cancer on breast ultrasound. Acad Radiol
2004; 11:526–535.
12. Kim KG, Cho SW, Min SJ, Kim JH, Min BG, Bae KT. Computerized
scheme for assessing ultrasonographic features of breast masses. Acad
Radiol 2005; 12:58–66.
13. Sehgal CM, Cary TW, Kangas SA, et al. Computer-based margin
analysis of breast sonography for differentiating malignant and benign
masses. J Ultrasound Med 2004; 23:1201–1209.
14. Horsch K, Giger ML, Vyborny CJ, Lan L, Mendelson EB, Hendrick
RE. Classification of breast lesions with multimodality computer-
aided diagnosis: observer study results on an independent clinical data
set. Radiology 2006; 240:357–368.
7
Cho et al—Application of Computer-Aided Diagnosis on Breast Ultrasonography
15. Horsch K, Giger ML, Vyborny CJ, Venta LA. Performance of
computer-aided diagnosis in the interpretation of lesions on breast
sonography. Acad Radiol 2004; 11:272–280.
16. Kim K, Song MK, Kim EK, Yoon JH. Clinical application of S-Detect
to breast masses on ultrasonography: a study evaluating the diagnostic
performance and agreement with a dedicated breast radiologist. Ultra-
sonography 2017; 36:3–9.
17. Wang Y, Jiang S, Wang H, et al. CAD algorithms for solid breast
masses discrimination: evaluation of the accuracy and interobserver
variability. Ultrasound Med Biol 2010; 36:1273–1281.
18. Landis JR, Koch GG. The measurement of observer agreement for
categorical data. Biometrics 1977; 33:159–174.
19. Barnhart HX, Williamson JM. Weighted least-squares approach for
comparing correlated kappa. Biometrics 2002; 58:1012–1019.
20. Singh S, Maxwell J, Baker JA, Nicholas JL, Lo JY. Computer-aided
classification of breast masses: performance and interobserver variabil-
ity of expert radiologists versus residents. Radiology 2011; 258:73–80.
8 J Ultrasound Med 2017; 00:00–00
21. Lee JH, Seong YK, Chang CH, et al. Computer-aided lesion diagnosis
in B-mode ultrasound by border irregularity and multiple sonographic
features. In: SPIE Medical Imaging. Bellingham, WA: International
Society for Optics and Photonics; 2013:86701–86707.
22. Shen WC, Chang RF, Moon WK. Computer aided classification sys-
tem for breast ultrasound based on Breast Imaging Reporting and
Data System (BI-RADS). Ultrasound Med Biol 2007; 33:1688–1698.
23. Park CS, Lee JH, Yim HW, et al. Observer agreement using the ACR
Breast Imaging Reporting and Data System (BI-RADS)–ultrasound,
first edition (2003). Korean J Radiol 2007; 8:397–402.
24. Sickles EA. Mammographic features of “early” breast cancer. AJR Am J
Roentgenol 1984; 143:461–464.
25. Sickles EA. Breast calcifications: mammographic evaluation. Radiology
1986; 160:289–293.
26. Schueller G, Jaromi S, Ponhold L, et al. US-guided 14-gauge core-nee-
dle breast biopsy: results of a validation study in 1352 cases. Radiology
2008; 248:406–413.