I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
1
5
,
No
.
1
,
Feb
r
u
ar
y
2
0
2
6
,
p
p
.
861
~
877
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
1
.
p
p
8
6
1
-
8
7
7
861
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
H
y
brid
te
x
ture
-
d
eep f
ea
ture
fusio
n
f
o
r ma
mm
o
g
ra
m
cla
ss
ificatio
n:
a
p
a
tient
-
l
ev
el,
ca
lib
ra
ted
ev
a
lua
tion
M
uh
a
m
m
a
d Sub
a
li
1
,
L
u
lu
M
a
wa
dd
a
h Wi
s
ud
a
wa
t
i
2
,
T
e
re
s
a
3
1
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
c
s
E
n
g
i
n
e
e
r
i
n
g
,
F
a
c
u
l
t
y
o
f
E
n
g
i
n
e
e
r
i
n
g
,
C
e
n
d
e
k
i
a
A
b
d
i
t
a
ma
U
n
i
v
e
r
s
i
t
y
,
Ta
n
g
e
r
a
n
g
,
I
n
d
o
n
e
si
a
2
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
c
s
,
F
a
c
u
l
t
y
o
f
I
n
d
u
st
r
i
a
l
T
e
c
h
n
o
l
o
g
y
,
G
u
n
a
d
a
r
ma
U
n
i
v
e
r
si
t
y
,
D
e
p
o
k
,
I
n
d
o
n
e
s
i
a
3
D
e
p
a
r
t
me
n
t
o
f
N
u
r
si
n
g
,
F
a
c
u
l
t
y
o
f
N
u
r
si
n
g
,
C
e
n
d
e
k
i
a
A
b
d
i
t
a
m
a
U
n
i
v
e
r
si
t
y
,
Ta
n
g
e
r
a
n
g
,
I
n
d
o
n
e
si
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Au
g
1
8
,
2
0
2
5
R
ev
is
ed
Dec
2
9
,
2
0
2
5
Acc
ep
ted
J
an
2
2
,
2
0
2
6
We
p
ro
p
o
se
a
li
g
h
twe
ig
h
t
c
o
m
p
u
ter
-
a
id
e
d
d
iag
n
o
sis
(CAD
)
fra
m
e
wo
rk
th
a
t
fu
se
s
fo
u
r
s
u
b
-
b
a
n
d
d
isc
re
te
wa
v
e
let
tran
sfo
rm
g
ra
y
-
lev
e
l
c
o
-
o
c
c
u
rre
n
c
e
m
a
tri
x
(DWT
–
G
LCM
)
tex
t
u
r
e
fe
a
tu
re
s
with
fin
e
-
tu
n
e
d
Re
sN
e
t
-
50
e
m
b
e
d
d
in
g
s
u
n
d
e
r
a
strict,
p
a
t
ien
t
-
lev
e
l,
lea
k
-
fre
e
e
v
a
lu
a
ti
o
n
p
r
o
to
c
o
l.
Ex
p
e
rime
n
ts
we
re
c
o
n
d
u
c
ted
o
n
two
p
u
b
li
c
d
a
tas
e
ts:
m
a
m
m
o
g
ra
p
h
ic
ima
g
e
a
n
a
ly
sis
so
c
iety
(
M
IAS
)
(n
o
rm
a
l
v
s.
a
b
n
o
rm
a
l)
a
n
d
c
u
ra
ted
b
re
a
st
ima
g
in
g
su
b
se
t
o
f
th
e
d
i
g
it
a
l
d
a
tab
a
se
fo
r
sc
re
e
n
in
g
m
a
m
m
o
g
ra
p
h
y
(
CBI
S
-
DD
S
M
)
(b
e
n
ig
n
v
s.
m
a
li
g
n
a
n
t)
.
F
i
v
e
-
fo
ld
c
ro
ss
-
v
a
li
d
a
ti
o
n
(CV)
wa
s
c
o
n
fi
n
e
d
to
t
h
e
train
in
g
p
o
rti
o
n
,
o
p
e
ra
ti
n
g
th
re
s
h
o
l
d
s
we
re
fix
e
d
o
n
t
h
e
v
a
li
d
a
ti
o
n
sp
li
t
to
targ
e
t
h
ig
h
re
c
a
ll
,
a
n
d
th
e
h
e
l
d
-
o
u
t
tes
t
se
t
wa
s
e
v
a
lu
a
ted
o
n
c
e
.
P
e
rfo
rm
a
n
c
e
wa
s
a
ss
e
s
se
d
u
sin
g
a
c
c
u
ra
c
y
,
F
1
-
sc
o
re
,
re
c
e
iv
e
r
o
p
e
ra
ti
n
g
c
h
a
ra
c
teristic
(ROC)
-
a
re
a
u
n
d
e
r
th
e
c
u
r
v
e
(
AUC
)
with
b
o
o
tstrap
9
5
%
c
o
n
f
id
e
n
c
e
in
terv
a
ls
(CI)
,
p
re
c
isio
n
-
re
c
a
ll
(
PR
)
-
AU
C,
a
n
d
c
a
li
b
ra
ti
o
n
m
e
tri
c
s
(Brier
sc
o
re
,
e
x
p
e
c
ted
c
a
li
b
ra
ti
o
n
e
rro
r).
T
h
e
p
ro
p
o
se
d
fu
sio
n
m
o
d
e
l
a
c
h
ie
v
e
d
R
OC
-
AU
C
o
n
M
IAS
(0
.
9
9
2
)
a
n
d
stro
n
g
p
e
rfo
rm
a
n
c
e
o
n
CBIS
-
DD
S
M
(0
.
8
9
6
),
wit
h
c
o
n
siste
n
t
PR
c
h
a
ra
c
teristics
.
Ca
li
b
ra
ti
o
n
a
n
a
l
y
sis
i
n
d
ica
te
d
re
li
a
b
le
p
ro
b
a
b
il
it
y
e
stim
a
tes
a
n
d
c
li
n
i
c
a
ll
y
in
ter
p
re
tab
le
d
e
c
isio
n
s
a
t
a
9
5
%
se
n
siti
v
it
y
o
p
e
ra
ti
n
g
p
o
i
n
t.
A
b
lat
io
n
e
x
p
e
rime
n
ts
re
v
e
a
led
su
b
sta
n
ti
a
l
g
a
in
s
o
v
e
r
tex
t
u
re
-
o
n
ly
b
a
se
li
n
e
s
a
n
d
p
a
rit
y
wit
h
c
o
n
v
o
lu
t
io
n
a
l
n
e
u
ra
l
n
e
two
rk
(
CNN
)
-
o
n
ly
m
o
d
e
ls,
h
ig
h
li
g
h
t
in
g
f
u
sio
n
a
s
a
sim
p
le
y
e
t
we
ll
-
c
a
li
b
ra
ted
a
lt
e
rn
a
ti
v
e
f
o
r
sc
re
e
n
in
g
-
o
r
ien
te
d
wo
r
k
fl
o
ws
.
T
h
is
st
u
d
y
u
n
d
e
r
sc
o
re
s
th
e
n
e
c
e
ss
it
y
o
f
p
a
ti
e
n
t
-
lev
e
l
e
v
a
lu
a
t
io
n
,
e
x
p
li
c
it
o
p
e
ra
ti
n
g
-
p
o
in
t
se
lec
ti
o
n
,
a
n
d
c
a
li
b
ra
ti
o
n
re
p
o
rti
n
g
t
o
e
n
su
re
c
li
n
ica
ll
y
m
e
a
n
in
g
fu
l
CAD
p
e
rfo
r
m
a
n
c
e
in
m
a
m
m
o
g
ra
p
h
y
.
K
ey
w
o
r
d
s
:
C
alib
r
atio
n
C
o
m
p
u
ter
-
aid
ed
d
iag
n
o
s
is
DW
T
-
GL
C
M
Featu
r
e
f
u
s
io
n
Ma
m
m
o
g
r
a
p
h
y
R
esNet
-
50
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Mu
h
am
m
ad
Su
b
ali
Dep
ar
tm
en
t o
f
I
n
f
o
r
m
atics E
n
g
in
ee
r
in
g
,
Facu
lty
o
f
E
n
g
in
ee
r
in
g
,
C
en
d
ek
ia
Ab
d
itam
a
Un
iv
er
s
ity
I
s
lam
ic
R
ay
a
St.,
Kela
p
a
Du
a,
T
an
g
er
a
n
g
,
B
an
ten
,
I
n
d
o
n
esia
E
m
ail: su
b
ali@
u
ca
.
ac
.
id
1.
I
NT
RO
D
UCT
I
O
N
B
r
ea
s
t
ca
n
ce
r
r
em
ain
s
th
e
m
o
s
t
f
r
eq
u
en
tly
d
ia
g
n
o
s
ed
ca
n
ce
r
am
o
n
g
wo
m
en
a
n
d
a
lead
i
n
g
ca
u
s
e
o
f
ca
n
ce
r
-
r
elate
d
m
o
r
tality
wo
r
ld
wid
e
[
1
]
.
T
h
is
b
u
r
d
en
u
n
d
er
s
co
r
es
th
e
u
r
g
en
c
y
o
f
ef
f
ec
tiv
e
s
cr
ee
n
in
g
a
n
d
ea
r
ly
d
etec
tio
n
s
tr
ateg
ies.
Ma
m
m
o
g
r
ap
h
y
r
em
ain
s
th
e
g
o
ld
s
tan
d
ar
d
f
o
r
p
o
p
u
latio
n
-
b
ased
s
cr
ee
n
in
g
an
d
clin
ical
wo
r
k
-
u
p
,
with
r
ep
o
r
tin
g
an
d
m
an
ag
em
en
t
co
m
m
o
n
ly
s
tan
d
ar
d
ized
u
s
in
g
th
e
b
r
ea
s
t
im
ag
in
g
r
ep
o
r
tin
g
an
d
d
ata
s
y
s
tem
(
BI
-
R
ADS
)
lex
ico
n
[
2
]
.
H
o
wev
er
,
m
am
m
o
g
r
a
m
s
ar
e
o
f
te
n
lo
w
-
c
o
n
tr
ast
an
d
af
f
ec
ted
b
y
tis
s
u
e
s
u
p
er
p
o
s
itio
n
.
T
h
is
m
a
k
es
in
t
er
p
r
etatio
n
d
if
f
icu
lt
an
d
in
cr
e
ases
in
ter
-
r
ea
d
er
v
ar
iab
ilit
y
.
T
h
er
ef
o
r
e
,
c
o
m
p
u
ter
-
aid
ed
d
ia
g
n
o
s
is
(
C
AD)
s
y
s
tem
s
h
av
e
b
ee
n
d
e
v
elo
p
ed
to
s
u
p
p
o
r
t
r
ad
io
l
o
g
is
ts
in
lesi
o
n
d
etec
tio
n
an
d
r
is
k
s
tr
atif
icatio
n
.
T
h
is
is
p
ar
ticu
lar
ly
im
p
o
r
tan
t u
n
d
er
lim
ited
d
a
ta
o
r
co
m
p
lex
im
ag
in
g
s
ce
n
ar
i
o
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
1
,
Feb
r
u
ar
y
2
0
2
6
:
861
-
8
7
7
862
E
ar
ly
C
AD
ap
p
r
o
ac
h
es
p
r
i
m
ar
ily
r
elied
o
n
h
an
d
cr
af
te
d
d
escr
ip
to
r
s
to
ca
p
tu
r
e
te
x
tu
r
al
an
d
m
u
ltire
s
o
lu
tio
n
c
h
ar
ac
ter
is
tics
.
Gr
ay
-
le
v
el
co
-
o
cc
u
r
r
en
ce
m
atr
ix
(
GL
C
M)
s
tatis
tics
r
em
ain
a
f
o
u
n
d
atio
n
al
to
o
l
f
o
r
q
u
a
n
tify
in
g
tex
tu
r
al
r
elatio
n
s
h
ip
s
[
3
]
,
wh
ile
wa
v
elet
d
ec
o
m
p
o
s
itio
n
s
p
r
o
v
id
e
s
p
atial
–
f
r
eq
u
en
cy
r
ep
r
esen
tatio
n
s
well
s
u
ited
f
o
r
b
r
ea
s
t
tis
s
u
e
an
aly
s
is
[
4
]
.
B
u
ild
in
g
o
n
th
ese
id
ea
s
,
B
er
b
ar
[
5
]
in
tr
o
d
u
ce
d
h
y
b
r
id
s
u
b
-
im
ag
e
d
escr
ip
to
r
s
(
wav
elet
-
co
n
to
u
r
let
(
CT
)
/C
T
2
an
d
s
tatis
tical
-
tex
tu
r
e
(
ST
)
-
GL
C
M)
co
u
p
led
with
s
u
p
p
o
r
t
v
ec
to
r
m
ac
h
in
es
(
SVMs
)
an
d
r
ep
o
r
ted
s
tr
o
n
g
r
esu
lts
o
n
d
ig
ital
d
atab
ase
f
o
r
s
cr
ee
n
in
g
m
am
m
o
g
r
ap
h
y
(
DDSM
)
an
d
m
a
m
m
o
g
r
ap
h
ic
im
ag
e
an
aly
s
is
s
o
ciety
(
M
I
AS
)
.
Ab
d
u
llah
et
a
l.
[
6
]
p
r
o
p
o
s
e
d
a
tex
t
u
r
e
-
an
al
y
s
is
p
ip
elin
e
u
s
in
g
a
m
u
lti
-
class
SVM
an
d
r
ep
o
r
te
d
9
8
%
ac
c
u
r
ac
y
f
o
r
ea
r
ly
lesi
o
n
d
etec
t
io
n
,
alth
o
u
g
h
th
e
ev
alu
atio
n
p
r
o
to
co
l
was
n
o
t
clea
r
ly
s
p
ec
if
ied
.
W
is
u
d
awa
ti
et
a
l.
[
7
]
d
em
o
n
s
tr
ated
r
o
b
u
s
t
clas
s
if
icatio
n
o
f
n
o
r
m
al/ab
n
o
r
m
al
a
n
d
b
en
ig
n
/m
alig
n
an
t
[
8
]
tu
m
o
r
s
u
s
in
g
2
D
-
d
is
cr
ete
wav
elet
tr
an
s
f
o
r
m
(
DW
T
)
–
GL
C
M
co
m
b
in
ed
with
ar
tific
ial
n
eu
r
al
n
etwo
r
k
(
ANN
)
(
b
ac
k
p
r
o
p
ag
atio
n
n
eu
r
al
n
etwo
r
k
(
B
PNN)
)
.
T
h
ese
s
tu
d
ies
u
n
d
er
s
co
r
e
th
e
v
alu
e
o
f
en
g
i
n
ee
r
ed
tex
t
u
r
al
f
ea
t
u
r
es,
th
o
u
g
h
g
e
n
er
aliza
tio
n
is
o
f
ten
lim
ited
b
y
d
ataset
s
ize
an
d
ev
alu
atio
n
in
co
n
s
is
ten
cies
.
W
ith
th
e
ad
v
en
t
o
f
d
ee
p
lear
n
in
g
,
m
am
m
o
g
r
a
p
h
y
C
AD
h
as
u
n
d
er
g
o
n
e
a
p
a
r
a
d
ig
m
s
h
if
t.
C
o
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
s
(
C
NN
s
)
lear
n
h
ier
ar
ch
ical
s
em
an
tics
d
ir
ec
tly
f
r
o
m
im
a
g
e
s
,
en
ab
lin
g
s
tr
o
n
g
er
g
en
er
aliza
tio
n
co
m
p
ar
ed
to
h
an
d
cr
a
f
ted
d
escr
ip
to
r
s
.
Mu
d
u
li
et
a
l.
[
9
]
em
p
lo
y
ed
C
NNs
o
n
m
u
lti
-
m
o
d
al
in
p
u
ts
(
m
am
m
o
g
r
ap
h
y
+u
ltra
s
o
u
n
d
)
with
ac
cu
r
ac
ies
a
b
o
v
e
9
0
%
ac
r
o
s
s
d
atasets
.
Ma
h
m
o
o
d
et
a
l.
[
1
0
]
lev
er
ag
ed
tr
an
s
f
er
lear
n
i
n
g
,
au
g
m
en
tatio
n
,
an
d
p
r
ep
r
o
ce
s
s
in
g
to
ac
h
iev
e
ar
ea
u
n
d
er
th
e
cu
r
v
e
(
AUC)
≈
0
.
9
9
f
o
r
b
en
ig
n
-
m
alig
n
an
t
d
is
cr
im
in
atio
n
.
Petr
in
i
et
a
l.
[
1
1
]
in
tr
o
d
u
ce
d
a
two
-
v
iew
E
f
f
icie
n
tNet
ar
ch
itectu
r
e
th
a
t
ag
g
r
eg
ates
b
ilater
al
c
r
an
io
ca
u
d
al
(
CC
)
an
d
m
e
d
io
later
al
o
b
liq
u
e
(
MLO
)
v
iews,
ac
h
iev
in
g
AUC
0
.
9
3
4
u
n
d
e
r
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
(
C
V)
an
d
0
.
8
4
8
o
n
th
e
o
f
f
icial
s
p
lit
o
n
cu
r
ated
b
r
ea
s
t
im
a
g
in
g
s
u
b
s
et
(
C
B
I
S)
o
f
th
e
DDSM.
C
o
m
p
ar
ativ
e
ev
alu
ati
o
n
s
s
h
o
w
th
e
s
u
p
e
r
io
r
ity
o
f
r
esid
u
al
n
etwo
r
k
s
s
u
ch
as
R
esNet
-
5
0
o
v
e
r
v
is
u
al
g
eo
m
etr
y
g
r
o
u
p
v
er
s
io
n
1
6
(
VGG1
6
)
o
n
MI
AS
[
1
2
]
,
wh
ile
Sab
er
et
a
l.
[
1
3
]
b
en
ch
m
ar
k
ed
m
u
ltip
le
tr
an
s
f
er
-
lear
n
in
g
b
ac
k
b
o
n
es,
ac
h
iev
in
g
n
ea
r
-
c
eilin
g
im
a
g
e
-
lev
el
m
etr
ics
(
AUC≈0
.
9
9
5
)
.
R
ec
en
t
wo
r
k
s
also
h
ig
h
lig
h
t
a
r
ch
itectu
r
al
in
n
o
v
a
tio
n
s
s
u
ch
as
C
NN
–
tr
an
s
f
o
r
m
er
h
y
b
r
id
s
[
1
4
]
a
n
d
f
ast
leak
y
r
esid
u
al
n
etwo
r
k
with
class
im
b
alan
ce
r
e
d
u
ctio
n
(
Fas
tLe
ak
y
R
esNet
-
C
I
R
)
[
1
5
]
an
d
s
y
s
tem
atic
r
e
v
iews
em
p
h
asize
b
o
t
h
th
eir
p
r
o
m
is
e
an
d
lim
itatio
n
s
[
1
6
]
,
[
1
7
]
.
Alo
n
g
s
id
e
C
NN
-
o
n
ly
p
ip
elin
e
s
,
h
y
b
r
id
f
r
am
ewo
r
k
s
h
a
v
e
b
e
en
ex
p
lo
r
ed
to
co
m
b
in
e
c
o
m
p
lem
en
tar
y
s
tr
en
g
th
s
o
f
h
an
d
c
r
af
ted
an
d
d
ee
p
f
ea
tu
r
es
[
1
8
]
,
[
1
9
]
.
Fo
r
ex
am
p
le,
Sajid
et
a
l.
[
1
8
]
in
t
eg
r
ated
h
an
d
cr
af
ted
an
d
d
ee
p
f
ea
tu
r
es
in
a
u
n
if
ied
f
r
am
ewo
r
k
f
o
r
b
r
ea
s
t
ca
n
ce
r
class
if
icatio
n
.
Sh
au
k
at
et
a
l.
[
1
9
]
co
m
b
in
ed
d
ee
p
C
NN
f
ea
tu
r
es
with
h
an
d
cr
af
t
ed
tex
tu
r
e
f
ea
tu
r
es
(
e.
g
.
,
Ga
b
o
r
an
d
wav
elet)
f
o
r
b
r
ea
s
t
ca
n
ce
r
d
etec
tio
n
in
m
am
m
o
g
r
a
m
an
d
u
ltra
s
o
u
n
d
im
ag
es
.
Mo
r
e
r
ec
en
tly
,
Das
et
a
l
.
[
2
0
]
ap
p
lied
R
esNet
-
5
0
to
b
r
ea
s
t
ca
n
ce
r
m
ag
n
etic
r
eso
n
a
n
ce
im
ag
in
g
(
MRI
)
im
ag
es
an
d
r
ep
o
r
te
d
9
2
.
0
1
%
ac
cu
r
ac
y
.
T
h
ese
r
esu
lts
s
u
p
p
o
r
t
th
e
r
o
b
u
s
tn
ess
o
f
r
esid
u
al
a
r
ch
ite
ctu
r
es
in
clin
ical
im
a
g
in
g
.
D
esh
p
an
d
e
et
a
l.
[
2
1
]
em
p
lo
y
e
d
tr
an
s
f
er
lear
n
in
g
with
R
esNe
t
-
5
0
o
n
m
am
m
o
g
r
am
s
an
d
r
ep
o
r
ted
9
3
.
4
%
ac
cu
r
ac
y
.
T
r
an
s
f
er
lear
n
in
g
also
r
ed
u
ce
d
th
e
tr
ain
in
g
b
u
r
d
e
n
co
m
p
ar
e
d
with
tr
ain
in
g
f
r
o
m
s
cr
atch
.
T
o
a
d
d
r
ess
class
im
b
alan
ce
,
Als
h
am
r
an
i
an
d
Als
h
o
m
r
an
i
[
2
2
]
in
tr
o
d
u
ce
d
a
d
u
al
R
esNet
-
5
0
+
s
y
n
th
etic
m
in
o
r
ity
o
v
er
-
s
am
p
lin
g
tech
n
iq
u
e
(
SMOT
E
)
f
r
a
m
ewo
r
k
.
T
h
e
s
tu
d
y
r
ep
o
r
ted
9
9
%
ac
cu
r
ac
y
o
n
b
al
an
ce
d
s
ets
an
d
9
0
%
o
n
im
b
al
an
ce
d
o
n
es.
T
h
is
r
esu
lt
h
ig
h
li
g
h
ts
th
e
im
p
o
r
ta
n
ce
o
f
b
alan
cin
g
s
tr
ateg
ies in
C
AD.
B
ey
o
n
d
d
ee
p
C
NNs,
f
ea
tu
r
e
-
lev
el
f
u
s
io
n
s
tr
ateg
ies
t
h
at
co
m
b
in
e
h
an
d
c
r
af
ted
an
d
d
ee
p
r
ep
r
esen
tatio
n
s
h
a
v
e
e
m
er
g
ed
as
a
p
r
o
m
is
in
g
d
ir
ec
tio
n
[
2
3
]
,
[
2
4
]
.
R
az
ali
et
a
l.
[
2
3
]
f
u
s
ed
C
NN
em
b
ed
d
i
n
g
s
with
wav
elet
-
s
ca
tter
in
g
f
ea
tu
r
es
an
d
r
ep
o
r
ted
9
8
-
9
9
%
ac
cu
r
ac
y
o
n
I
Nb
r
ea
s
t.
Vijay
alak
s
h
m
i
et
a
l.
[
2
4
]
co
m
b
in
ed
s
h
ea
r
let
tr
an
s
f
o
r
m
s
with
GL
C
M/
g
r
ay
-
lev
el
r
u
n
-
l
en
g
th
m
atr
ix
(
GL
R
L
M
)
an
d
a
b
id
ir
ec
tio
n
al
lo
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
(
B
iLST
M
)
-
C
NN
m
o
d
el,
ac
h
iev
in
g
9
7
.
1
4
%
o
n
MI
AS.
At
p
atien
t
le
v
el,
W
im
m
er
et
a
l.
[
2
5
]
d
em
o
n
s
tr
ated
m
u
lti
-
task
f
u
s
io
n
p
ip
elin
es
th
at
a
g
g
r
eg
at
ed
p
r
ed
ictio
n
s
ac
r
o
s
s
task
s
an
d
v
iews,
y
ield
in
g
AUC
0
.
9
6
2
f
o
r
lesi
o
n
p
r
esen
c
e
an
d
0
.
7
9
1
f
o
r
m
alig
n
a
n
cy
o
n
DDSM/
C
B
I
S
-
DD
SM.
Su
n
e
t
a
l
.
[
2
6
]
ex
ten
d
e
d
th
is
with
an
atten
tio
n
-
g
u
id
ed
d
u
al
-
b
r
an
ch
C
NN
f
o
r
cr
an
i
o
ca
u
d
al
(
C
C
)
an
d
m
ed
io
later
al
o
b
liq
u
e
(
ML
O)
f
u
s
io
n
.
C
o
llectiv
ely
,
t
h
ese
s
tu
d
ies
s
u
g
g
est
th
at
h
y
b
r
id
p
ip
eli
n
es
ca
n
o
u
tp
er
f
o
r
m
C
NN
-
o
n
ly
m
o
d
els,
esp
ec
ially
wh
en
ev
alu
ated
u
n
d
e
r
d
iv
e
r
s
e
o
r
im
b
alan
ce
d
d
atasets
,
b
y
l
ev
er
ag
in
g
th
e
co
m
p
lem
en
tar
y
n
atu
r
e
o
f
tex
tu
r
e
-
b
ased
an
d
d
ee
p
-
lear
n
ed
r
e
p
r
es
en
tatio
n
s
.
I
n
p
ar
allel,
r
ec
e
n
t
AI
tr
en
d
s
r
elev
an
t
to
m
am
m
o
g
r
a
p
h
y
in
c
lu
d
e
s
u
p
er
v
is
ed
c
o
n
tr
asti
v
e
p
r
e
-
tr
ain
in
g
f
r
am
ewo
r
k
s
th
at
b
o
o
s
t
s
cr
ee
n
in
g
p
e
r
f
o
r
m
an
ce
[
2
7
]
.
Do
m
ai
n
ad
a
p
tatio
n
a
n
d
d
o
m
ain
g
en
er
aliza
tio
n
ar
e
also
ex
p
lo
r
ed
to
m
itig
ate
d
is
tr
ib
u
tio
n
s
h
if
ts
an
d
im
p
r
o
v
e
e
x
ter
n
al
v
alid
ity
[
2
8
]
,
[
2
9
]
.
I
n
ad
d
itio
n
,
v
is
io
n
tr
an
s
f
o
r
m
er
(
ViT
)
m
o
d
els
an
d
m
u
lti
-
v
iew
ar
c
h
itectu
r
es
h
av
e
s
h
o
wn
p
r
o
m
is
in
g
r
esu
lts
in
r
ec
en
t
co
m
p
ar
ativ
e
wo
r
k
s
an
d
s
u
r
v
ey
s
[
3
0
]
,
[
3
1
]
.
T
h
ese
d
ev
elo
p
m
en
ts
m
o
tiv
ate
ev
alu
atio
n
f
r
am
ewo
r
k
s
th
at
b
a
lan
ce
ar
ch
itectu
r
al
ad
v
an
ce
s
with
clin
ical
p
r
ac
tic
ality
,
em
p
h
asizin
g
in
ter
p
r
etab
i
lity
,
ca
lib
r
atio
n
,
a
n
d
r
e
p
r
o
d
u
ci
b
ilit
y
.
Ho
wev
er
,
d
esp
ite
th
ese
ad
v
a
n
ce
s
,
th
r
ee
lim
itatio
n
s
r
em
ain
u
n
ad
d
r
ess
ed
.
First,
m
an
y
s
tu
d
ies
r
ely
o
n
r
eg
io
n
s
o
f
in
ter
est
(
R
OI
)
/p
at
ch
-
lev
el
s
p
lits
th
at
r
is
k
p
atien
t
leak
ag
e
an
d
m
ay
in
f
late
p
er
f
o
r
m
an
ce
,
wh
ile
p
atien
t
-
lev
el,
leak
-
f
r
ee
ev
al
u
a
tio
n
is
r
ar
ely
en
f
o
r
ce
d
.
Seco
n
d
,
p
r
o
b
a
b
ilit
y
ca
lib
r
atio
n
is
o
f
ten
n
eg
lecte
d
ev
en
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Hyb
r
id
textu
r
e
-
d
ee
p
fea
tu
r
e
fu
s
io
n
fo
r
ma
mmo
g
r
a
m
cla
s
s
ific
a
tio
n
:
a
p
a
tien
t
-
leve
l …
(
Mu
h
a
mma
d
S
u
b
a
li
)
863
th
o
u
g
h
ca
lib
r
ate
d
o
u
tp
u
ts
ar
e
ess
en
tial
f
o
r
th
r
esh
o
ld
s
elec
tio
n
an
d
r
is
k
co
m
m
u
n
icatio
n
[
3
2
]
,
[
3
3
]
.
T
h
ir
d
,
in
im
b
alan
ce
d
clin
ical
d
atasets
,
p
er
f
o
r
m
an
ce
is
f
r
e
q
u
en
tly
o
v
e
r
-
r
ep
o
r
ted
u
s
in
g
r
ec
eiv
er
o
p
e
r
atin
g
ch
ar
ac
ter
is
tic
(
R
OC
)
cu
r
v
es
alo
n
e,
wh
ile
p
r
ec
is
io
n
-
r
ec
all
(
PR
)
an
aly
s
is
p
r
o
v
id
es
a
m
o
r
e
r
ea
lis
tic
v
iew
o
f
p
o
s
itiv
e
-
class
r
etr
iev
al
[
3
4
]
–
[
3
6
]
.
T
h
ese
lim
itatio
n
s
h
in
d
er
clin
ical
ad
o
p
ti
o
n
an
d
m
o
tiv
ate
th
e
n
ee
d
f
o
r
r
o
b
u
s
t,
in
ter
p
r
etab
le,
an
d
d
e
p
lo
y
m
e
n
t
-
o
r
ien
te
d
C
AD
p
ip
elin
es.
T
h
is
s
tu
d
y
p
r
o
p
o
s
es
a
h
y
b
r
id
f
u
s
io
n
p
ip
elin
e
th
at
co
n
ca
ten
a
tes
D
W
T
–
GL
C
M
tex
tu
r
e
f
ea
t
u
r
es
with
f
in
e
-
tu
n
e
d
R
esNet
-
5
0
em
b
ed
d
in
g
s
,
class
if
ied
v
ia
a
s
h
allo
w
ANN
u
n
d
er
a
p
atien
t
-
lev
el
,
leak
-
f
r
ee
p
r
o
to
c
o
l
(
s
tr
atif
ied
6
0
/2
0
/2
0
s
p
lit;
5
-
f
o
ld
C
V
o
n
tr
ain
in
g
/
v
alid
atio
n
;
t
h
r
esh
o
ld
s
f
ix
e
d
o
n
v
alid
atio
n
an
d
a
p
p
lied
o
n
ce
to
th
e
h
eld
-
o
u
t
test
)
.
B
ey
o
n
d
R
OC
-
AUC,
we
p
r
o
v
id
e
ca
lib
r
atio
n
an
aly
s
is
(
r
eliab
ilit
y
d
ia
g
r
am
s
,
B
r
ier
s
co
r
e,
an
d
ex
p
ec
te
d
ca
lib
r
atio
n
e
r
r
o
r
(
E
C
E
)
)
an
d
p
r
ec
is
io
n
-
r
ec
all
(
PR
)
-
b
ased
ev
alu
atio
n
to
r
e
f
lect
class
im
b
alan
ce
.
Ou
r
co
n
tr
ib
u
tio
n
s
ar
e
t
h
r
ee
f
o
l
d
:
i)
A
s
im
p
le
y
et
ef
f
ec
tiv
e
f
ea
tu
r
e
-
lev
el
f
u
s
io
n
o
f
h
an
d
c
r
af
ted
m
u
ltire
s
o
lu
tio
n
tex
tu
r
es
an
d
d
ee
p
em
b
ed
d
in
g
s
f
o
r
m
am
m
o
g
r
a
m
class
if
icatio
n
.
ii)
Patien
t
-
lev
el,
leak
-
f
r
ee
ev
al
u
atio
n
en
s
u
r
in
g
r
ep
r
o
d
u
ci
b
ilit
y
,
an
d
clin
ical
r
ele
v
an
ce
(
with
ex
ter
n
al
v
alid
atio
n
(
E
V)
ac
r
o
s
s
d
atasets
,
wh
er
e
ap
p
licab
le)
.
iii)
C
o
m
p
r
eh
en
s
iv
e
ass
ess
m
en
t
i
n
clu
d
in
g
ca
lib
r
atio
n
,
h
ig
h
-
r
e
ca
ll
o
p
er
atio
n
,
an
d
PR
-
m
etr
i
cs,
ad
d
r
ess
in
g
cr
itical
g
ap
s
in
th
e
m
am
m
o
g
r
a
p
h
y
C
AD
liter
atu
r
e.
W
e
ex
p
licitly
p
o
s
itio
n
t
h
e
n
o
v
elty
o
f
th
is
wo
r
k
in
its
ca
lib
r
atio
n
-
awa
r
e,
p
atien
t
-
lev
el
e
v
al
u
atio
n
a
n
d
clin
ical
in
ter
p
r
etab
ilit
y
,
an
d
w
e
o
u
tlin
e
atten
tio
n
-
b
ased
g
atin
g
o
r
f
ea
tu
r
e
s
elec
tio
n
as
p
r
a
g
m
atic
ex
ten
s
io
n
s
to
th
e
p
r
esen
t
f
u
s
io
n
d
esig
n
.
T
ak
en
to
g
eth
er
,
t
h
is
wo
r
k
s
it
u
ates
h
y
b
r
id
f
ea
tu
r
e
f
u
s
io
n
with
in
a
clin
ically
g
r
o
u
n
d
ed
ev
alu
atio
n
f
r
am
ewo
r
k
,
alig
n
in
g
with
th
e
b
r
o
a
d
er
p
u
s
h
to
war
d
s
af
e,
r
eliab
le,
an
d
p
atien
t
-
ce
n
ter
ed
A
I
f
o
r
b
r
ea
s
t
ca
n
ce
r
s
cr
ee
n
in
g
.
Sectio
n
2
d
escr
ib
es
th
e
m
ate
r
ials
an
d
m
eth
o
d
s
(
d
atasets
,
p
r
ep
r
o
ce
s
s
in
g
/R
OI
ex
tr
ac
tio
n
,
f
ea
t
u
r
e
ex
tr
ac
tio
n
,
f
u
s
io
n
/class
if
icatio
n
,
an
d
ev
al
u
atio
n
p
r
o
to
co
l)
.
Sectio
n
3
r
e
p
o
r
ts
th
e
r
esu
lts
an
d
d
is
cu
s
s
io
n
,
in
clu
d
in
g
o
v
e
r
all
p
er
f
o
r
m
an
ce
,
a
b
latio
n
s
,
ca
lib
r
atio
n
/PR
an
aly
s
is
,
co
m
p
ar
ativ
e
s
tu
d
ies,
EV
,
an
d
q
u
alitativ
e
er
r
o
r
an
aly
s
is
.
Sectio
n
4
c
o
n
clu
d
es with
im
p
licati
o
n
s
an
d
f
u
tu
r
e
r
esear
ch
d
ir
ec
ti
o
n
s
.
2.
M
E
T
H
O
D
W
e
ad
o
p
t
a
f
u
s
io
n
-
b
ased
C
AD
f
r
am
ewo
r
k
in
Fig
u
r
e
1
,
wh
ich
in
teg
r
ates
s
tatis
tical
tex
tu
r
e
f
ea
tu
r
es
ex
tr
ac
ted
v
ia
DW
T
–
GL
C
M
with
d
ee
p
em
b
ed
d
i
n
g
s
o
b
tai
n
ed
f
r
o
m
a
f
in
e
-
tu
n
e
d
R
esNet
-
5
0
,
to
ad
d
r
ess
two
b
in
ar
y
class
if
icatio
n
task
s
:
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
o
n
th
e
MI
AS
d
ataset
an
d
b
en
ig
n
v
s
.
m
alig
n
an
t
o
n
th
e
C
B
I
S
-
DD
SM
d
atase
t
[
3
7
]
,
[
3
8
]
.
R
OI
s
ar
e
p
r
ep
ar
ed
f
o
llo
win
g
o
u
r
ea
r
lier
p
ip
elin
es
—
au
to
m
atic
cr
o
p
p
i
n
g
,
in
ten
s
ity
-
g
u
id
ed
lo
ca
lizatio
n
,
an
d
m
ask
r
ef
in
em
en
t
as
in
W
is
u
d
awa
ti
et
a
l.
(
n
o
r
m
al
–
ab
n
o
r
m
al
)
[
7
]
a
n
d
W
is
u
d
awa
ti
et
a
l.
(
b
en
ig
n
–
m
alig
n
an
t)
[
8
]
th
en
c
o
n
v
e
r
t
ed
to
8
-
b
it
g
r
ay
s
ca
le
an
d
r
esized
to
2
2
4
×2
2
4
(
co
n
v
e
r
ted
g
r
a
y
→R
GB
f
o
r
R
esNet
-
5
0
)
.
Fo
r
tex
t
u
r
es,
ea
ch
R
OI
is
d
ec
o
m
p
o
s
ed
b
y
a
o
n
e
-
le
v
el
d
b
4
DW
T
in
to
lo
w
-
lo
w
(
LL
)
,
lo
w
-
h
ig
h
(
LH
)
,
h
ig
h
-
lo
w
(
HL
)
,
a
n
d
h
ig
h
-
h
ig
h
(
HH
)
.
On
ea
ch
s
u
b
-
b
a
n
d
,
a
GL
C
M
with
2
5
6
g
r
a
y
lev
els
ar
e
co
m
p
u
te
d
at
p
ix
el
d
is
tan
ce
d
=1
in
f
o
u
r
o
r
ie
n
tatio
n
s
(
0
°,
4
5
°,
9
0
°,
1
3
5
°).
Fro
m
th
e
n
o
r
m
alize
d
GL
C
M
(
,
)
,
we
ex
tr
ac
t
co
n
tr
ast,
co
r
r
elatio
n
,
en
e
r
g
y
,
an
d
h
o
m
o
g
e
n
eity
[
3
]
an
d
a
v
er
ag
e
ac
r
o
s
s
o
r
ien
tatio
n
s
,
y
ield
in
g
1
6
f
ea
tu
r
es
p
er
R
OI
(
4
s
u
b
-
b
a
n
d
s
×4
s
tatis
tics
)
.
Fo
r
d
ee
p
f
ea
t
u
r
es,
a
R
esNet
-
50
(
I
m
ag
eNe
t)
is
f
in
e
-
tu
n
ed
o
n
tr
ain
; 2
0
4
8
-
D
av
g
_
p
o
o
l e
m
b
ed
d
in
g
is
ex
tr
ac
ted
p
er
R
OI
.
T
h
e
1
6
-
D
tex
tu
r
e
v
ec
to
r
an
d
2
0
4
8
-
D
d
ee
p
v
ec
to
r
ar
e
c
o
n
ca
ten
ated
(
2
0
6
4
-
D)
a
n
d
z
-
s
co
r
ed
u
s
in
g
tr
ain
s
tatis
tics
o
n
ly
,
th
en
class
if
ied
b
y
a
s
h
allo
w
ANN
(
in
p
u
t
2
0
6
4
→
h
id
d
en
1
2
8
(
R
eL
U)
→
S
o
f
t
M
ax
-
2
)
with
in
v
er
s
e
-
f
r
e
q
u
en
c
y
cl
ass
weig
h
ts
.
Fo
r
a
leak
-
f
r
ee
ev
al
u
atio
n
p
r
o
to
co
l,
we
em
p
lo
y
a
p
atien
t
-
lev
el
ap
p
r
o
ac
h
,
s
tr
atif
ied
6
0
/
2
0
/2
0
s
p
lit
(
tr
ain
/v
al/test
)
.
Fiv
e
-
f
o
ld
C
V
is
co
n
f
in
ed
to
tr
ain
f
o
r
m
o
d
el
s
elec
tio
n
an
d
ab
latio
n
s
u
m
m
a
r
ies.
T
h
e
o
p
e
r
atin
g
th
r
esh
o
ld
τ
is
ch
o
s
en
o
n
v
al
id
atio
n
to
tar
g
et
9
5
%
s
en
s
itiv
ity
,
th
en
ap
p
lied
u
n
ch
an
g
ed
to
test
,
wh
ich
is
ev
alu
ated
o
n
ce
.
A
cc
u
r
ac
y
,
s
en
s
itiv
ity
,
s
p
ec
if
i
city
,
p
r
ec
is
io
n
,
F1
-
s
co
r
e
,
R
O
C
-
AUC,
P
R
-
AUC
wi
th
9
5
%
b
o
o
ts
tr
ap
co
n
f
id
en
ce
in
ter
v
als
(
C
I
)
,
an
d
ass
ess
ca
lib
r
atio
n
is
r
ep
o
r
t
ed
v
ia
B
r
ier
s
co
r
e
a
n
d
E
C
E
(
1
0
b
in
s
)
.
Fo
r
a
b
latio
n
s
(
tex
tu
r
e
-
o
n
ly
/C
NN
-
o
n
ly
/
f
u
s
io
n
)
,
we
u
s
e
p
air
e
d
f
o
l
d
-
wis
e
AUC
test
s
(
p
air
ed
t
-
test
o
r
W
ilco
x
o
n
,
as
ap
p
r
o
p
r
iate)
.
I
n
ad
d
itio
n
,
t
h
e
d
ec
is
io
n
th
r
esh
o
ld
τ
*
f
ix
e
d
o
n
v
al
(
≈
9
5
%
s
en
s
itiv
ity
)
is
a
p
p
lied
u
n
ch
an
g
ed
to
b
o
th
test
an
d
E
V.
E
V
s
tr
ictly
p
r
ev
en
ts
p
atien
t
o
v
er
lap
with
th
e
s
o
u
r
ce
co
h
o
r
t,
u
s
es
th
e
s
am
e
n
o
r
m
aliza
tio
n
an
d
τ
*
with
o
u
t
r
ef
itti
n
g
,
an
d
r
ep
o
r
ts
AUC
-
E
V
with
9
5
%
b
o
o
ts
tr
ap
C
I
s
(
B
=2
,
0
0
0
)
.
B
ey
o
n
d
th
e
m
ain
ab
latio
n
,
th
r
ee
lig
h
tweig
h
t v
ar
ia
n
ts
ar
e
ev
alu
ated
u
n
d
er
th
e
id
e
n
tical
p
r
o
to
co
l A
is
d
ee
p
er
h
ea
d
; B
is
Mo
n
te
C
ar
lo
(
MC
)
f
ea
tu
r
e
-
d
r
o
p
o
u
t
with
m
ea
n
ag
g
r
eg
atio
n
;
C
is
α
-
g
ated
f
u
s
io
n
with
m
in
im
u
m
r
ed
u
n
d
an
c
y
m
ax
im
u
m
r
elev
an
ce
(
m
R
MR
)
(
α
,
K
ch
o
s
en
o
n
v
al
)
an
d
a
q
u
alitativ
e
er
r
o
r
an
al
y
s
is
(
g
r
ad
ien
t
-
weig
h
ted
class
ac
tiv
atio
n
m
ap
p
i
n
g
(
Gr
ad
-
C
AM
)
)
is
p
r
o
v
id
ed
to
i
llu
s
tr
ate
tr
u
e
p
o
s
itiv
e/tr
u
e
n
e
g
ativ
e/f
alse
p
o
s
itiv
e
/
f
alse
n
eg
ativ
e
f
ailu
r
e
m
o
d
es.
No
r
m
ality
is
s
cr
ee
n
ed
v
ia
a
L
i
llief
o
r
s
test
to
ch
o
o
s
e
b
etwe
en
p
air
ed
t
-
test
an
d
W
ilco
x
o
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
1
,
Feb
r
u
ar
y
2
0
2
6
:
861
-
8
7
7
864
Fig
u
r
e
1
.
Ov
e
r
v
iew
o
f
th
e
p
r
o
p
o
s
ed
f
u
s
io
n
C
AD
f
r
am
ewo
r
k
in
teg
r
atin
g
DW
T
-
GL
C
M
tex
tu
r
e
f
ea
tu
r
es a
n
d
R
esNet
-
5
0
d
ee
p
em
b
ed
d
in
g
s
2
.
1
.
Da
t
a
s
et
T
h
is
s
tu
d
y
em
p
lo
y
s
two
p
u
b
licly
av
ailab
le
m
am
m
o
g
r
a
m
d
atasets
:
th
e
MI
AS
[
3
7
]
an
d
th
e
C
B
I
S
-
DD
SM
[
3
8
]
.
MI
AS,
th
o
u
g
h
lo
wer
in
r
eso
lu
tio
n
,
is
w
id
ely
u
s
ed
f
o
r
n
o
r
m
al
v
s
.
a
b
n
o
r
m
al
class
if
icatio
n
(
1
0
7
n
o
r
m
al
an
d
8
9
a
b
n
o
r
m
al)
.
C
B
I
S
-
DDSM
p
r
o
v
id
es
h
ig
h
-
r
eso
lu
tio
n
im
ag
es
with
p
ath
o
lo
g
y
-
co
n
f
ir
m
ed
R
OI
s
,
s
u
p
p
o
r
tin
g
b
en
i
g
n
v
s
.
m
alig
n
an
t
class
if
icatio
n
(
4
8
1
b
en
ig
n
a
n
d
5
2
7
m
alig
n
a
n
t)
.
All
av
ailab
le
im
ag
es
f
r
o
m
MI
AS
a
n
d
C
B
I
S
-
DDSM
wer
e
in
clu
d
e
d
.
No
s
am
p
les
wer
e
ex
clu
d
e
d
ex
ce
p
t
f
o
r
co
r
r
u
p
ted
o
r
u
n
r
ea
d
a
b
le
f
iles
,
en
s
u
r
in
g
th
at
th
e
d
atasets
wer
e
f
u
lly
r
ep
r
esen
tativ
e.
2
.
2
.
P
re
pro
ce
s
s
ing
a
nd
RO
I
ex
t
ra
ct
io
n
Pre
p
r
o
ce
s
s
in
g
an
d
R
OI
ex
tr
ac
tio
n
f
o
llo
wed
v
alid
ated
p
r
o
ce
d
u
r
es.
I
m
ag
es
wer
e
co
n
v
er
ted
to
g
r
ay
s
ca
le,
co
n
tr
ast
-
en
h
an
ce
d
with
clip
-
lim
ited
ad
ap
tiv
e
h
is
to
g
r
am
eq
u
aliza
tio
n
(
C
L
AHE
)
an
d
d
en
o
is
ed
u
s
in
g
a
3
×3
m
ed
ian
f
ilter
.
Fo
r
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
class
if
icatio
n
,
th
e
f
u
ll b
r
ea
s
t
r
eg
i
o
n
was
s
eg
m
en
ted
v
ia
ad
ap
tiv
e
th
r
esh
o
ld
in
g
,
au
to
m
atic
cr
o
p
p
in
g
,
an
d
m
o
r
p
h
o
lo
g
y
[
7
]
,
w
h
ile
f
o
r
b
en
ig
n
v
s
.
m
alig
n
an
t,
s
u
s
p
icio
u
s
m
ass
e
s
wer
e
is
o
lated
[
8
]
.
R
OI
s
wer
e
th
en
u
s
ed
in
two
p
ip
elin
es:
D
W
T
-
GL
C
M
o
n
th
e
R
OI
,
an
d
C
NN
-
b
ased
m
o
d
els
(
R
esNet
-
5
0
)
with
r
esized
(
2
2
4
×2
2
4
)
an
d
n
o
r
m
alize
d
in
p
u
ts
.
Fig
u
r
e
2
illu
s
tr
ates
r
ep
r
esen
t
ativ
e
p
r
e
p
r
o
ce
s
s
in
g
an
d
R
OI
ex
tr
ac
tio
n
s
tep
s
f
o
r
t
h
e
two
class
if
icatio
n
task
s
.
I
n
Fig
u
r
e
2
(
a
)
,
th
e
b
r
ea
s
t
ar
ea
is
s
eg
m
en
ted
f
r
o
m
th
e
b
ac
k
g
r
o
u
n
d
a
n
d
ar
tifa
cts
(
e.
g
.
,
lab
els
an
d
s
ca
n
n
in
g
n
o
is
e)
u
s
in
g
ad
a
p
tiv
e
th
r
esh
o
ld
i
n
g
,
m
o
r
p
h
o
lo
g
ical
o
p
er
atio
n
s
,
an
d
au
to
m
atic
cr
o
p
p
in
g
,
p
r
o
v
i
d
in
g
a
clea
n
R
OI
f
o
r
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
class
if
icatio
n
o
n
MI
AS.
I
n
Fig
u
r
e
2
(
b
)
,
a
s
u
s
p
icio
u
s
r
eg
io
n
is
is
o
lated
f
r
o
m
th
e
m
am
m
o
g
r
am
u
s
in
g
a
d
ap
tiv
e
th
r
esh
o
ld
i
n
g
a
n
d
m
o
r
p
h
o
lo
g
ical
s
eg
m
en
tatio
n
,
p
r
o
d
u
cin
g
an
R
OI
th
at
co
r
r
es
p
o
n
d
s
to
th
e
b
e
n
ig
n
v
s
.
m
ali
g
n
an
t
class
if
icatio
n
task
o
n
C
B
I
S
-
DDSM.
T
h
ese
s
tep
s
en
s
u
r
e
co
n
s
is
ten
t
R
OI
q
u
ality
ac
r
o
s
s
d
atasets
an
d
p
r
o
v
id
e
s
tan
d
a
r
d
ized
in
p
u
ts
f
o
r
s
u
b
s
eq
u
en
t te
x
t
u
r
e
-
an
d
d
ee
p
-
f
ea
tu
r
e
p
ip
elin
es.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Hyb
r
id
textu
r
e
-
d
ee
p
fea
tu
r
e
fu
s
io
n
fo
r
ma
mmo
g
r
a
m
cla
s
s
ific
a
tio
n
:
a
p
a
tien
t
-
leve
l …
(
Mu
h
a
mma
d
S
u
b
a
li
)
865
(
a)
(
b
)
Fig
u
r
e
2
.
E
x
am
p
les o
f
R
OI
ex
tr
ac
tio
n
o
f
(
a)
s
eg
m
en
te
d
b
r
ea
s
t a
r
ea
f
o
r
n
o
r
m
al
v
s
.
a
b
n
o
r
m
a
l c
lass
if
icatio
n
,
an
d
(
b
)
is
o
lated
m
ass
f
o
r
b
en
i
g
n
v
s
.
m
alig
n
an
t c
lass
if
icatio
n
2
.
3
.
F
e
a
t
ure
ex
t
r
a
ct
io
n
2
.
3
.
1
.
DWT
-
G
L
CM
t
ex
t
ure
f
ea
t
ures
E
ac
h
8
-
b
it
R
OI
was
d
ec
o
m
p
o
s
ed
u
s
in
g
a
o
n
e
-
lev
el
2
D
DW
T
with
a
Dau
b
ec
h
ies
-
4
(
d
b
4
)
b
asis
,
p
r
o
d
u
cin
g
f
o
u
r
s
u
b
-
b
an
d
s
:
L
L
,
L
H,
HL
,
a
n
d
HH.
Fo
r
ea
ch
s
u
b
-
b
an
d
,
th
e
wa
v
elet
co
ef
f
icien
ts
wer
e
lin
ea
r
ly
r
escaled
to
th
e
r
a
n
g
e
[
0
,
2
5
5
]
,
an
d
a
GL
C
M
with
2
5
6
g
r
a
y
l
ev
els
was
co
m
p
u
ted
at
a
p
ix
el
d
is
tan
ce
o
f
d
=
1
in
f
o
u
r
o
r
ien
tatio
n
s
(
0
°,
4
5
°,
9
0
°,
an
d
1
3
5
°).
Fro
m
th
e
n
o
r
m
alize
d
GL
C
M
(
,
)
,
f
o
u
r
s
tan
d
a
r
d
Har
alick
s
tatis
t
ics
[
3
]
wer
e
ex
tr
ac
ted
:
c
o
n
tr
ast
(
lo
ca
l
in
ten
s
ity
v
a
r
iatio
n
)
,
co
r
r
elatio
n
(
li
n
ea
r
d
ep
en
d
en
cy
b
etwe
en
p
i
x
el
p
air
s
)
,
en
e
r
g
y
(
tex
tu
r
al
u
n
if
o
r
m
ity
)
,
an
d
h
o
m
o
g
e
n
eity
(
clo
s
en
ess
o
f
th
e
d
is
tr
ib
u
tio
n
to
th
e
d
iag
o
n
al)
.
T
h
ese
s
tatis
tics
wer
e
av
er
ag
ed
ac
r
o
s
s
th
e
f
o
u
r
o
r
ien
tatio
n
s
,
r
esu
ltin
g
in
a
to
tal
o
f
1
6
tex
tu
r
e
f
ea
tu
r
es
p
er
R
OI
(
4
s
u
b
-
b
a
n
d
s
×4
s
tatis
tics
)
.
He
r
e,
(
,
)
d
en
o
tes
th
e
n
o
r
m
alize
d
G
L
C
M
f
o
r
a
g
iv
en
s
u
b
-
b
an
d
,
wh
ile
,
ar
e
m
ea
n
s
,
an
d
,
ar
e
s
tan
d
ar
d
d
ev
iatio
n
s
o
f
m
ar
g
in
al
d
is
tr
ib
u
t
io
n
s
.
2
.
3
.
2
.
F
ine
-
t
un
ed
ResNet
-
5
0
deep
f
ea
t
ures
T
o
ca
p
tu
r
e
h
ig
h
-
lev
el
m
o
r
p
h
o
lo
g
ical
p
atter
n
s
,
a
R
esNe
t
-
5
0
p
r
etr
ain
ed
o
n
I
m
ag
eNe
t
is
f
in
e
-
tu
n
ed
o
n
th
e
tr
ain
in
g
s
p
lit
o
n
ly
,
f
o
llo
win
g
th
e
p
atien
t
-
lev
el
s
ep
ar
a
tio
n
p
r
o
to
c
o
l.
T
h
e
f
in
al
f
u
lly
co
n
n
ec
ted
lay
er
is
r
ep
lace
d
with
a
two
-
class
cl
ass
if
ier
,
an
d
th
e
n
etwo
r
k
is
t
r
ain
ed
f
o
r
5
e
p
o
ch
s
u
s
in
g
A
d
am
(
lear
n
in
g
r
ate
1
×1
0
⁻⁴)
,
b
atch
s
ize
o
f
8
,
an
d
cr
o
s
s
-
en
tr
o
p
y
lo
s
s
.
A
f
t
e
r
f
i
n
e
-
t
u
n
i
n
g
,
e
a
c
h
R
O
I
i
s
f
o
r
w
a
r
d
e
d
t
h
r
o
u
g
h
t
h
e
n
e
t
w
o
r
k
,
a
n
d
t
h
e
r
e
s
u
l
ti
n
g
2
,
048
-
D
e
m
b
e
d
d
i
n
g
i
s
t
a
k
e
n
f
r
o
m
t
h
e
g
l
o
b
a
l
a
v
e
r
a
g
e
p
o
o
l
i
n
g
(
a
v
g
_
p
o
o
l
)
l
a
y
e
r
.
2
.
4
.
F
e
a
t
ure
f
us
io
n a
nd
cla
s
s
if
ica
t
io
n
T
h
e
f
in
al
f
ea
tu
r
e
r
ep
r
esen
tatio
n
is
o
b
tain
ed
b
y
co
n
ca
ten
atin
g
th
e
1
6
-
D
tex
tu
r
e
v
ec
to
r
an
d
th
e
2
,
048
-
D
d
ee
p
em
b
ed
d
in
g
,
r
esu
ltin
g
in
a
2
,
064
-
D
v
ec
to
r
.
Z
-
s
co
r
e
n
o
r
m
aliza
tio
n
is
ap
p
lied
u
s
in
g
th
e
m
ea
n
an
d
s
tan
d
ar
d
d
ev
iatio
n
f
r
o
m
th
e
t
r
ain
in
g
s
et,
an
d
th
e
s
am
e
p
a
r
am
eter
s
ar
e
ap
p
lied
u
n
ch
a
n
g
e
d
to
v
alid
atio
n
an
d
test
s
e
ts
to
p
r
ev
en
t
leak
ag
e.
C
las
s
if
icatio
n
is
p
er
f
o
r
m
ed
u
s
in
g
a
s
h
allo
w
AN
N
with
an
in
p
u
t
lay
er
o
f
s
ize
2
,
0
6
4
,
o
n
e
h
id
d
en
lay
er
with
1
2
8
R
eL
U
u
n
its
,
an
d
a
s
o
f
tm
ax
-
2
o
u
t
p
u
t
lay
er
.
C
lass
im
b
ala
n
ce
is
ad
d
r
ess
ed
b
y
in
v
er
s
e
-
f
r
eq
u
en
cy
weig
h
tin
g
i
n
th
e
lo
s
s
f
u
n
ctio
n
.
T
r
ain
in
g
u
s
es
Ad
am
o
p
tim
izatio
n
(
lea
r
n
in
g
r
ate
1
×
1
0
⁻⁴)
,
b
atch
s
ize
3
2
,
a
n
d
a
m
a
x
im
u
m
o
f
3
0
ep
o
c
h
s
.
T
h
is
lig
h
tweig
h
t
ANN
h
ea
d
was
d
elib
er
ately
ch
o
s
en
to
en
s
u
r
e
co
m
p
u
tatio
n
al
ef
f
ici
en
cy
an
d
p
r
ac
tical
d
ep
lo
y
ab
ilit
y
in
C
AD
wo
r
k
f
lo
ws,
wh
er
e
r
ap
id
tr
ain
in
g
an
d
in
f
er
e
n
ce
ca
n
b
e
ac
h
iev
ed
o
n
m
o
d
est
h
ar
d
war
e,
in
clu
d
i
n
g
C
PU
-
o
n
l
y
en
v
i
r
o
n
m
e
n
ts
.
W
h
ile
th
e
c
u
r
r
en
t
f
u
s
io
n
m
ec
h
an
is
m
is
im
p
lem
en
ted
t
h
r
o
u
g
h
s
tr
aig
h
tf
o
r
war
d
f
ea
t
u
r
e
co
n
c
aten
atio
n
f
o
r
s
im
p
licity
an
d
r
ep
r
o
d
u
cib
ilit
y
,
we
in
ten
tio
n
ally
ad
o
p
t
s
im
p
le
co
n
ca
ten
atio
n
g
iv
en
o
u
r
d
e
p
lo
y
m
en
t
-
o
r
ien
ted
s
co
p
e
an
d
p
ag
e
co
n
s
tr
ain
ts
;
lig
h
tweig
h
t
atten
tio
n
-
b
ased
f
u
s
io
n
is
d
ef
er
r
ed
t
o
f
u
tu
r
e
wo
r
k
.
Mo
r
e
ad
v
an
ce
d
s
tr
ateg
ies
s
u
ch
as
atten
tio
n
-
b
ased
g
atin
g
,
m
u
l
ti
-
b
r
an
ch
f
u
s
io
n
,
o
r
m
R
MR
f
ea
tu
r
e
s
e
lectio
n
m
ay
f
u
r
th
er
im
p
r
o
v
e
in
ter
p
r
etab
ilit
y
an
d
d
is
cr
im
in
ativ
e
p
o
wer
.
T
h
ese
alter
n
ativ
es,
to
g
eth
er
with
d
ee
p
e
r
class
if
ier
d
esig
n
s
(
e
.
g
.
,
m
u
lti
-
lay
er
p
er
ce
p
tr
o
n
s
with
d
r
o
p
o
u
t
o
r
tr
an
s
f
o
r
m
e
r
-
b
ased
h
ea
d
s
)
,
ar
e
ac
k
n
o
wled
g
ed
as
p
o
ten
tial
f
u
t
u
r
e
ex
te
n
s
io
n
s
to
s
tr
en
g
th
en
r
o
b
u
s
tn
ess
an
d
AI
n
o
v
elty
with
o
u
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
1
,
Feb
r
u
ar
y
2
0
2
6
:
861
-
8
7
7
866
co
m
p
r
o
m
is
in
g
th
e
clin
ical
ef
f
icien
cy
o
f
th
e
p
ip
elin
e.
As
ex
p
lo
r
ato
r
y
ab
latio
n
s
,
we
also
t
ested
two
m
in
im
al
ex
ten
s
io
n
s
:
i)
a
s
ca
lar
g
ate
th
at
r
e
-
weig
h
ts
th
e
tex
tu
r
e
b
lo
c
k
b
ef
o
r
e
co
n
c
aten
atio
n
a
n
d
ii
)
o
p
tio
n
al
m
R
MR
to
p
-
k
f
ea
tu
r
e
s
elec
tio
n
o
n
th
e
f
u
s
ed
v
ec
to
r
.
2
.
5
.
E
v
a
lua
t
i
o
n
s
t
ra
t
eg
y
a
nd
m
et
rics
W
e
r
ep
o
r
t
ac
cu
r
ac
y
,
s
en
s
itiv
ity
(
r
ec
all)
,
s
p
ec
if
icity
,
p
r
ec
is
i
o
n
,
F1
-
s
co
r
e
,
R
OC
-
AUC,
an
d
PR
-
AU
C
as
p
r
im
ar
y
p
er
f
o
r
m
a
n
ce
m
etr
i
cs.
CV
o
u
tco
m
es
a
r
e
s
u
m
m
a
r
ized
as
m
ea
n
±
s
tan
d
ar
d
d
e
v
iatio
n
ac
r
o
s
s
f
iv
e
f
o
ld
s
o
n
th
e
tr
ain
in
g
p
o
r
tio
n
(
tr
ain
)
.
On
th
e
h
eld
-
o
u
t
test
s
et,
9
5
%
b
o
o
ts
tr
ap
CI
f
o
r
R
OC
-
AUC
a
r
e
co
m
p
u
te
d
u
s
in
g
B
=2
,
0
0
0
p
atien
t
-
le
v
el
r
esam
p
l
es
[
3
6
]
.
Fo
r
c
o
m
p
a
r
ativ
e
an
al
y
s
is
,
f
u
s
io
n
is
ev
alu
ated
ag
ain
s
t
tex
tu
r
e
-
o
n
ly
a
n
d
C
NN
-
o
n
ly
b
aselin
es
u
s
in
g
p
ai
r
ed
s
tatis
tical
test
s
o
n
f
o
ld
-
wis
e
AUC.
A
L
illi
ef
o
r
s
n
o
r
m
ality
test
(
α
=0
.
0
5
)
[
3
9
]
is
u
s
ed
to
d
eter
m
in
e
wh
et
h
er
a
p
air
ed
t
-
test
o
r
a
W
ilco
x
o
n
s
ig
n
ed
-
r
an
k
test
[
4
0
]
is
ap
p
lied
.
C
alib
r
atio
n
is
ass
e
s
s
ed
v
ia
B
r
ier
s
co
r
e
an
d
E
C
E
(
E
C
E
;
1
0
b
in
s
)
,
with
r
eliab
ilit
y
d
iag
r
am
s
in
clu
d
ed
wh
er
e
r
elev
a
n
t
[
3
2
]
,
[
3
3
]
.
Fo
r
clin
ical
in
ter
p
r
etab
ilit
y
,
a
d
ec
is
io
n
th
r
esh
o
ld
is
s
elec
ted
o
n
th
e
v
alid
atio
n
s
p
lit
to
tar
g
et
9
5
%
s
en
s
itiv
ity
an
d
is
ap
p
lied
u
n
ch
an
g
ed
to
t
h
e
test
s
et,
with
s
p
ec
if
icity
,
p
r
ec
is
io
n
,
an
d
F1
-
s
co
r
e
r
ep
o
r
ted
at
th
is
f
ix
e
d
o
p
e
r
atin
g
p
o
in
t.
Fo
r
EV
,
we
u
s
ed
th
e
v
alid
atio
n
-
f
ix
e
d
th
r
esh
o
l
d
(
τ
)
with
o
u
t
an
y
r
e
f
itti
n
g
an
d
en
s
u
r
ed
n
o
p
atien
t
o
v
e
r
lap
b
etwe
en
s
o
u
r
ce
an
d
ex
te
r
n
al
co
h
o
r
ts
;
all
n
o
r
m
aliza
tio
n
p
ar
am
eter
s
wer
e
co
m
p
u
ted
o
n
tr
ai
n
o
n
l
y
an
d
ap
p
lied
u
n
c
h
an
g
e
d
to
v
al/test
/EV
.
As
an
ex
p
lo
r
ato
r
y
s
tep
,
we
al
s
o
ap
p
lied
p
o
s
t
-
h
o
c
tem
p
er
atu
r
e
s
ca
lin
g
(
T
S)
to
ca
lib
r
ate
th
e
So
f
tMa
x
o
u
tp
u
ts
.
T
h
e
tem
p
er
atu
r
e
p
ar
am
eter
T
was
o
p
tim
ized
o
n
th
e
v
alid
atio
n
s
p
lit
b
y
m
in
im
izin
g
th
e
n
eg
ativ
e
lo
g
-
lik
elih
o
o
d
,
an
d
ca
lib
r
atio
n
was
s
u
b
s
eq
u
en
tly
ev
alu
ated
u
s
in
g
th
e
B
r
ier
s
co
r
e
an
d
(
E
C
E
;
1
0
b
in
s
)
.
Sin
ce
th
e
p
r
im
a
r
y
r
esu
lts
ar
e
r
e
p
o
r
te
d
in
t
h
e
u
n
ca
lib
r
ated
s
ettin
g
,
T
S
is
p
r
esen
ted
o
n
ly
to
illu
s
tr
ate
f
ea
s
ib
ilit
y
an
d
to
h
ig
h
lig
h
t f
u
tu
r
e
d
ir
ec
tio
n
s
in
c
alib
r
atio
n
-
awa
r
e
C
AD
d
esig
n
.
All
ex
p
er
im
en
ts
wer
e
im
p
lem
en
ted
in
MA
T
L
AB
R
2
0
2
3
a
u
s
in
g
th
e
d
ee
p
lear
n
i
n
g
to
o
lb
o
x
an
d
im
ag
e
p
r
o
ce
s
s
in
g
to
o
lb
o
x
.
Stan
d
a
r
d
f
u
n
ctio
n
s
wer
e
em
p
l
o
y
ed
f
o
r
R
OC
/P
R
co
m
p
u
tatio
n
an
d
b
o
o
ts
tr
ap
CI
.
Pre
p
r
o
ce
s
s
in
g
,
n
o
r
m
aliza
tio
n
,
an
d
th
r
esh
o
ld
f
itti
n
g
wer
e
d
er
i
v
ed
e
x
clu
s
iv
ely
f
r
o
m
th
e
tr
ain
in
g
/v
alid
atio
n
d
ata
an
d
ap
p
lied
u
n
ch
an
g
ed
to
t
h
e
h
eld
-
o
u
t
test
(
an
d
E
V)
s
et
to
en
s
u
r
e
a
f
u
lly
leak
-
f
r
ee
ev
alu
atio
n
an
d
r
ep
r
o
d
u
cib
ilit
y
o
f
all
s
tep
s
.
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
3
.
1
.
O
v
er
a
ll
perf
o
r
m
a
nce
o
f
t
he
pro
po
s
ed
f
us
io
n m
o
del
T
h
is
s
ec
tio
n
r
ep
o
r
ts
th
e
p
r
im
a
r
y
p
er
f
o
r
m
a
n
ce
o
f
th
e
p
r
o
p
o
s
e
d
f
u
s
io
n
-
b
ased
C
AD
f
r
am
ewo
r
k
o
n
b
o
th
d
atasets
.
W
e
s
u
m
m
ar
ize
CV
d
is
cr
im
in
atio
n
(
C
V
-
AUC,
m
ea
n
±
SD)
,
an
d
h
eld
-
o
u
t
test
m
etr
ics
ac
cu
r
ac
y
,
s
en
s
itiv
ity
,
s
p
ec
if
icity
,
p
r
ec
is
io
n
,
F1
-
s
co
r
e,
an
d
R
OC
-
AUC
-
ea
ch
with
9
5
%
C
I
.
T
h
e
test
s
et
s
ize
(
N)
is
s
h
o
wn
in
th
e
d
ataset
lab
el
f
o
r
cla
r
ity
.
Un
less
o
th
er
wis
e
n
o
ted
,
al
l
r
esu
lts
ar
e
p
r
e
-
ca
lib
r
atio
n
.
T
ab
le
1
p
r
esen
ts
th
e
o
v
er
all
r
esu
lts
,
s
h
o
win
g
th
at
th
e
p
r
o
p
o
s
ed
m
o
d
el
ac
h
iev
es
h
ig
h
d
is
cr
im
in
atio
n
ac
r
o
s
s
b
o
th
MI
AS
an
d
C
B
I
S
-
DD
SM
d
atasets
,
wi
th
m
in
im
al
d
eg
r
ad
atio
n
b
etwe
en
CV
an
d
h
eld
-
o
u
t
e
v
alu
atio
n
.
CV
-
AUC
is
m
ea
n
±
SD
o
v
er
5
f
o
ld
s
.
9
5
%
C
I
f
o
r
ac
cu
r
ac
y
/s
en
s
itiv
ity
/s
p
ec
if
icity
/p
r
ec
is
io
n
:
W
ils
o
n
;
F1
-
s
co
r
e
:
b
o
o
ts
tr
ap
;
R
OC
-
AU
C
: b
o
o
ts
tr
ap
(
B
=2
0
0
0
)
.
T
est s
et
s
ize
is
s
h
o
wn
in
th
e
d
ataset
lab
el.
T
ab
le
1
.
Ov
e
r
all
p
er
f
o
r
m
an
ce
o
f
th
e
p
r
o
p
o
s
ed
f
u
s
io
n
m
o
d
el
o
n
MI
AS a
n
d
C
B
I
S
-
DDSM
d
atasets
D
a
t
a
s
e
t
/
Ta
s
k
CV
-
A
U
C
(
mea
n
±
S
D
)
Te
st
a
c
c
u
r
a
c
y
%
(
9
5
%
C
I
)
Te
st
s
e
n
si
t
i
v
i
t
y
%
(
9
5
%
C
I
)
Te
st
s
p
e
c
i
f
i
c
i
t
y
%
(
9
5
%
C
I
)
Te
st
p
r
e
c
i
s
i
o
n
%
(
9
5
%
C
I
)
Te
st
F1
-
sc
o
r
e
%
(
9
5
%
C
I
)
R
O
C
-
A
U
C
(
9
5
%
C
I
)
M
I
A
S
(
N
=
3
9
)
N
o
r
mal
v
s.
A
b
n
o
r
m
a
l
0
.
9
9
7
±
0
.
0
0
7
9
7
.
4
4
(
8
6
.
5
–
9
9
.
9
)
1
0
0
.
0
0
(
8
0
.
5
–
1
0
0
)
9
5
.
4
5
(
7
7
.
2
–
9
9
.
9
)
9
4
.
4
4
(
7
2
.
7
–
9
9
.
9
)
9
7
.
1
4
(
9
0
.
0
–
1
0
0
.
0
)
0
.
9
9
2
[
0
.
9
6
8
–
1
.
0
0
0
]
C
B
I
S
-
D
D
S
M
(
N
=
2
0
1
)
B
e
n
i
g
n
v
s
.
M
a
l
i
g
n
a
n
t
0
.
9
9
2
±
0
.
0
0
4
8
4
.
5
8
(
7
8
.
8
–
8
9
.
3
)
8
6
.
6
7
(
7
8
.
6
–
9
2
.
5
)
8
2
.
2
9
(
7
3
.
2
–
8
9
.
3
)
8
4
.
2
6
(
7
6
.
0
–
9
0
.
6
)
8
5
.
4
5
(
7
9
.
8
–
9
0
.
2
)
0
.
8
9
6
[
0
.
8
4
5
–
0
.
9
3
8
]
3
.
1
.
1
.
M
I
AS
—
no
rm
a
l v
s
.
a
b
no
rm
a
l
T
h
e
f
u
s
i
o
n
m
o
d
el
a
ch
ie
v
e
d
C
V
-
AUC=
0
.
9
9
7
±
0
.
0
0
7
a
c
r
o
s
s
f
iv
e
s
tr
ati
f
i
e
d
f
o
l
d
s
,
i
n
d
ic
a
tin
g
h
i
g
h
s
tab
ilit
y
d
u
r
i
n
g
t
r
ai
n
i
n
g
.
On
t
h
e
h
e
ld
-
o
u
t
tes
t
s
et
,
AUC=
0
.
9
9
2
(
9
5
%
C
I
:
0
.
9
6
8
-
1
.
0
0
0
)
,
w
it
h
a
cc
u
r
ac
y
=9
7
.
4
4
%
(
8
6
.
5
-
9
9
.
9
)
,
s
en
s
iti
v
it
y
=
1
0
0
.
0
0
%
(
8
0
.
5
-
1
0
0
)
,
s
p
e
ci
f
ic
it
y
=
9
5
.
4
5
%
(
7
7
.
2
-
9
9
.
9
)
,
p
r
ec
is
i
o
n
=
9
4
.
4
4
%
(
7
2
.
7
-
9
9
.
9
)
,
an
d
F1
-
s
co
r
e=
9
7
.
1
4
%
(
9
0
.
0
-
1
0
0
.
0
)
.
Fig
u
r
e
3
p
r
esen
ts
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
p
r
o
p
o
s
ed
f
u
s
io
n
m
o
d
el
o
n
MI
AS.
T
h
e
R
OC
cu
r
v
e
in
F
ig
u
r
e
3
(
a)
d
em
o
n
s
tr
ates
n
ea
r
-
p
er
f
ec
t
class
s
ep
ar
atio
n
,
with
th
e
cu
r
v
e
clo
s
ely
ap
p
r
o
ac
h
in
g
t
h
e
to
p
-
lef
t
co
r
n
er
.
T
h
e
c
o
n
f
u
s
io
n
m
atr
i
x
in
F
ig
u
r
e
3
(
b
)
co
n
f
i
r
m
s
th
at
all
a
b
n
o
r
m
al
ca
s
es
wer
e
co
r
r
ec
tly
i
d
en
tifie
d
,
with
n
o
f
alse
n
eg
ativ
es
—
a
cr
itical
r
e
q
u
ir
em
en
t
in
s
cr
ee
n
i
n
g
s
ce
n
a
r
io
s
.
T
h
e
lear
n
in
g
cu
r
v
e
in
Fig
u
r
e
4
in
d
icate
s
r
ap
id
co
n
v
e
r
g
en
ce
a
n
d
m
i
n
im
al
o
v
er
f
itti
n
g
th
r
o
u
g
h
o
u
t t
r
ain
in
g
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Hyb
r
id
textu
r
e
-
d
ee
p
fea
tu
r
e
fu
s
io
n
fo
r
ma
mmo
g
r
a
m
cla
s
s
ific
a
tio
n
:
a
p
a
tien
t
-
leve
l …
(
Mu
h
a
mma
d
S
u
b
a
li
)
867
(
a)
(
b
)
Fig
u
r
e
3
.
Pro
p
o
s
ed
f
u
s
io
n
m
o
d
el
o
n
MI
AS
of
(
a)
R
OC
cu
r
v
e
an
d
(
b
)
c
o
n
f
u
s
io
n
m
atr
ix
Fig
u
r
e
4
.
L
ea
r
n
in
g
cu
r
v
es f
o
r
t
h
e
ANN
h
ea
d
o
n
f
u
s
io
n
f
ea
tu
r
es (
MI
AS)
3
.
1
.
2
.
CB
I
S
-
DDS
M
—
benig
n
v
s
.
m
a
lig
na
nt
Fo
r
C
B
I
S
-
DD
SM,
C
V
-
AU
C
=0
.
9
9
2
±
0
.
0
0
4
an
d
test
AUC=0
.
8
9
6
(
9
5
%
C
I
:
0
.
8
4
5
–
0
.
9
3
8
)
.
T
est
p
er
f
o
r
m
a
n
ce
was
ac
cu
r
ac
y
=8
4
.
5
8
%
(
7
8
.
8
-
8
9
.
3
)
,
s
en
s
itiv
ity
=8
6
.
6
7
%
(
7
8
.
6
-
9
2
.
5
)
,
s
p
ec
if
icity
=8
2
.
2
9
%
(
7
3
.
2
-
8
9
.
3
)
,
p
r
ec
is
io
n
=8
4
.
2
6
%
(
7
6
.
0
–
9
0
.
6
)
,
an
d
F1
-
s
co
r
e=
8
5
.
4
5
%
(
7
9
.
8
–
9
0
.
2
)
.
Fig
u
r
e
5
p
r
esen
ts
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
p
r
o
p
o
s
ed
f
u
s
io
n
m
o
d
el
o
n
C
B
I
S
-
DDS
M.
T
h
e
R
OC
cu
r
v
e
in
Fig
u
r
e
5
(
a)
d
em
o
n
s
tr
ates
h
ig
h
d
is
cr
im
in
atio
n
b
u
t
with
a
less
s
teep
r
is
e
co
m
p
ar
e
d
to
MI
AS,
r
e
f
lectin
g
th
e
g
r
ea
ter
d
if
f
icu
lty
o
f
b
en
ig
n
–
m
alig
n
an
t
s
ep
a
r
atio
n
.
T
h
e
co
n
f
u
s
io
n
m
atr
ix
in
Fig
u
r
e
5
(
b
)
s
h
o
ws
b
alan
ce
d
d
etec
tio
n
p
er
f
o
r
m
a
n
ce
b
u
t
with
s
o
m
e
f
alse
p
o
s
itiv
es
an
d
f
alse
n
e
g
ativ
es.
L
ea
r
n
in
g
c
u
r
v
es
in
Fig
u
r
e
6
in
d
icate
s
tab
le
tr
ain
in
g
with
v
alid
atio
n
ac
cu
r
ac
y
ar
o
u
n
d
8
5
-
9
0
%,
s
u
g
g
esti
n
g
th
e
m
o
d
el
g
en
er
alize
s
r
ea
s
o
n
ab
ly
well
b
u
t
co
u
ld
b
e
n
ef
it
f
r
o
m
ad
d
itio
n
al
r
e
g
u
lar
izatio
n
.
Ov
er
all,
th
e
p
r
o
p
o
s
ed
f
u
s
io
n
ap
p
r
o
ac
h
d
em
o
n
s
tr
ates
ex
ce
llen
t
g
en
er
aliza
tio
n
in
n
o
r
m
al
v
s
.
a
b
n
o
r
m
al
class
if
icatio
n
(
MI
AS)
an
d
co
m
p
etitiv
e
p
er
f
o
r
m
a
n
ce
in
th
e
m
o
r
e
c
h
a
llen
g
in
g
b
en
ig
n
v
s
.
m
alig
n
an
t
task
(
C
B
I
S
-
DDS
M
)
.
T
h
e
in
teg
r
atio
n
o
f
h
an
d
cr
a
f
ted
DW
T
-
GL
C
M
f
ea
tu
r
es
w
ith
d
ee
p
s
em
an
tic
R
esNet
-
5
0
f
ea
tu
r
es c
o
n
tr
ib
u
te
s
to
r
o
b
u
s
t d
etec
tio
n
ac
r
o
s
s
d
if
f
er
en
t le
s
io
n
ty
p
es.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
1
,
Feb
r
u
ar
y
2
0
2
6
:
861
-
8
7
7
868
No
tab
ly
,
th
e
MI
AS r
esu
lts
s
h
o
w
th
e
s
y
s
tem
’
s
ab
ilit
y
to
ac
h
ie
v
e
p
e
r
f
ec
t
s
en
s
itiv
ity
,
a
cr
itica
l
f
ac
to
r
in
ea
r
ly
b
r
ea
s
t
ca
n
ce
r
d
etec
tio
n
.
I
n
C
B
I
S
-
DDSM,
wh
ile
s
en
s
i
tiv
ity
r
em
ain
ed
h
ig
h
,
a
tr
ad
e
-
o
f
f
with
s
p
ec
if
icity
was
o
b
s
er
v
ed
,
c
o
n
s
is
ten
t
with
ty
p
ical
s
cr
ee
n
in
g
s
y
s
tem
b
eh
av
io
r
f
a
v
o
r
in
g
r
ec
all.
T
h
ese
f
i
n
d
in
g
s
s
u
p
p
o
r
t
th
e
clin
ical
p
o
ten
tial
o
f
h
y
b
r
id
f
u
s
io
n
C
AD
s
y
s
tem
s
,
p
ar
ticu
l
ar
ly
in
s
ce
n
ar
io
s
wh
er
e
b
o
th
tex
tu
r
e
an
d
d
ee
p
f
ea
tu
r
es c
ar
r
y
c
o
m
p
lem
e
n
tar
y
d
iag
n
o
s
tic
in
f
o
r
m
atio
n
.
(
a)
(
b
)
Fig
u
r
e
5
.
Pro
p
o
s
ed
f
u
s
io
n
m
o
d
el
o
n
C
B
I
S
-
DDSM
of
(
a)
R
OC
cu
r
v
e
an
d
(
b
)
c
o
n
f
u
s
io
n
m
at
r
ix
Fig
u
r
e
6
.
L
ea
r
n
in
g
cu
r
v
es f
o
r
t
h
e
ANN
h
ea
d
o
n
f
u
s
io
n
f
ea
tu
r
es (
C
B
I
S
-
DD
SM)
3
.
2
.
Abla
t
io
n
s
t
ud
y
—
t
ex
t
ur
e
-
o
nly
v
s
.
f
ine
-
t
un
ed
ResNet
-
5
0
v
s
.
f
us
io
n
W
e
ev
alu
ated
th
r
ee
ar
ch
itec
tu
r
al
v
ar
ian
ts
u
n
d
e
r
an
id
en
tical,
leak
-
f
r
ee
p
r
o
t
o
co
l
(
p
at
ien
t
-
lev
el
6
0
/2
0
/2
0
s
p
lit;
s
tr
atif
ied
5
-
f
o
ld
C
V
o
n
th
e
tr
ain
in
g
s
et;
id
e
n
tical
ANN
h
ea
d
an
d
s
ch
ed
u
le)
:
i)
tex
tu
r
e
-
o
n
ly
:
4
-
s
u
b
b
an
d
DW
T
–
GL
C
M
f
ea
tu
r
es→ANN,
ii)
C
NN
-
o
n
ly
:
f
in
e
-
tu
n
e
d
R
esNet
-
5
0
em
b
ed
d
in
g
s
→ANN,
an
d
iii)
f
u
s
io
n
:
c
o
n
ca
ten
atio
n
o
f
tex
tu
r
e
an
d
C
NN
f
ea
tu
r
es
→ANN.
T
h
is
s
etu
p
is
o
lates
th
e
co
n
tr
ib
u
tio
n
s
o
f
h
an
d
cr
a
f
ted
tex
tu
r
e
d
escr
ip
t
o
r
s
an
d
lear
n
ed
d
ee
p
r
ep
r
e
s
en
tatio
n
s
.
T
ab
le
2
p
r
esen
ts
ab
latio
n
s
u
m
m
ar
y
(
C
V
-
AUC a
n
d
test
AU
C
)
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Hyb
r
id
textu
r
e
-
d
ee
p
fea
tu
r
e
fu
s
io
n
fo
r
ma
mmo
g
r
a
m
cla
s
s
ific
a
tio
n
:
a
p
a
tien
t
-
leve
l …
(
Mu
h
a
mma
d
S
u
b
a
li
)
869
T
ab
le
2
.
Ab
latio
n
s
u
m
m
ar
y
(
C
V
-
AUC an
d
test
AUC)
(
Valu
e
s
ar
e
av
er
ag
ed
o
v
e
r
5
f
o
ld
s
)
D
a
t
a
s
e
t
M
e
t
h
o
d
CV
-
A
U
C
(
mea
n
±
S
D
)
Te
st
A
U
C
M
I
A
S
Te
x
t
u
r
e
-
o
n
l
y
(DWT
–
G
L
C
M
→A
N
N
)
0
.
7
8
5
±
0
.
0
8
5
0
.
8
5
3
M
I
A
S
C
N
N
-
o
n
l
y
(
R
e
sN
e
t
-
5
0
f
t
→A
N
N
)
0
.
9
9
2
±
0
.
0
1
7
0
.
9
8
4
M
I
A
S
F
u
si
o
n
(
Te
x
t
u
r
e
+
C
N
N
→A
N
N
)
0
.
9
9
7
±
0
.
0
0
4
0
.
9
8
7
C
B
I
S
-
D
D
S
M
Te
x
t
u
r
e
-
o
n
l
y
(DWT
–
G
L
C
M
→A
N
N
)
0
.
8
3
5
±
0
.
0
3
3
0
.
8
0
8
C
B
I
S
-
D
D
S
M
C
N
N
-
o
n
l
y
(
R
e
sN
e
t
-
5
0
f
t
→A
N
N
)
0
.
9
9
6
±
0
.
0
0
2
0
.
8
9
5
C
B
I
S
-
D
D
S
M
F
u
si
o
n
(
Te
x
t
u
r
e
+
C
N
N
→A
N
N
)
0
.
9
9
2
±
0
.
0
0
6
0
.
9
0
5
On
th
e
MI
AS
d
ataset
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
al)
,
f
u
s
io
n
ac
h
i
ev
ed
th
e
h
ig
h
est
cr
o
s
s
-
v
alid
ated
AUC
(
0
.
9
9
7
±
0
.
0
0
4
)
,
m
ar
g
in
ally
o
u
tp
er
f
o
r
m
in
g
C
NN
-
o
n
ly
(
0
.
9
9
2
±
0
.
0
1
7
)
an
d
s
u
b
s
tan
tially
ex
c
ee
d
in
g
tex
tu
r
e
-
o
n
l
y
(
0
.
7
8
5
±
0
.
0
8
5
)
.
Pair
wis
e
co
m
p
ar
is
o
n
s
s
h
o
wed
s
ig
n
if
ica
n
t
g
ain
s
o
f
b
o
th
C
NN
-
o
n
l
y
an
d
f
u
s
io
n
o
v
er
tex
tu
r
e
-
o
n
ly
(
Δ
AUC≈+
0
.
2
1
,
p
≤
0
.
0
4
3
)
,
wh
ile
C
NN
-
o
n
ly
v
er
s
u
s
f
u
s
io
n
was
n
o
t
s
tatis
tically
s
ig
n
if
ican
t
(
p
=0
.
8
5
3
)
.
On
th
e
h
eld
-
o
u
t
test
,
AUC
s
wer
e
0
.
8
5
3
(
tex
tu
r
e
-
o
n
ly
)
,
0
.
9
8
4
(
C
NN
-
o
n
l
y
)
,
an
d
0
.
9
8
7
(
f
u
s
io
n
)
.
Fo
r
th
e
C
B
I
S
-
DDSM
d
ataset
(
b
en
ig
n
v
s
.
m
alig
n
a
n
t)
,
C
NN
-
o
n
ly
an
d
Fu
s
io
n
p
er
f
o
r
m
ed
s
im
ilar
ly
ac
r
o
s
s
f
o
l
d
s
(
0
.
9
9
6
±
0
.
0
0
2
v
s
.
0
.
9
9
2
±
0
.
0
0
6
;
Δ
AU
C
=
-
0
.
0
0
4
,
p
=
0
.
2
0
5
)
,
wi
th
b
o
th
s
ig
n
if
ican
tly
o
u
tp
er
f
o
r
m
in
g
tex
tu
r
e
-
o
n
ly
(
Δ
AUC≥+
0
.
1
5
7
,
p
≤
0
.
0
0
0
5
)
.
T
est AUC
s
wer
e
0
.
8
0
8
(
tex
tu
r
e
-
o
n
ly
)
,
0
.
8
9
5
(
C
NN
-
o
n
ly
)
,
a
n
d
0
.
9
0
5
(
f
u
s
io
n
).
Pair
ed
s
tatis
tics
ar
e
co
m
p
u
te
d
o
n
f
o
ld
-
wis
e
AUC
(
K=
5
)
.
No
r
m
ality
was
s
cr
ee
n
ed
with
L
illi
ef
o
r
s
;
wh
en
v
i
o
lated
we
u
s
ed
W
ilco
x
o
n
s
ig
n
ed
-
r
a
n
k
,
o
th
e
r
wis
e
p
air
ed
t
-
test
s
.
Dif
f
er
en
ce
s
b
etw
ee
n
C
NN
-
o
n
ly
an
d
f
u
s
io
n
wer
e
n
o
t
s
ig
n
if
ican
t
o
n
eith
e
r
d
ataset,
i
n
d
icatin
g
th
at
d
ee
p
f
ea
tu
r
es
d
o
m
in
ate
p
er
f
o
r
m
an
ce
wh
ile
tex
tu
r
e
cu
es p
r
o
v
id
e
c
o
m
p
lem
en
tar
y
b
u
t n
o
t c
o
n
s
is
ten
tly
s
ig
n
if
ican
t g
ain
s
u
n
d
er
th
is
p
r
o
to
co
l.
3
.
2
.
1
.
Co
m
bin
ed
re
s
ults f
o
r
lig
htw
eig
ht
v
a
ria
nts
(A
-
C)
W
e
as
s
ess
ed
th
r
ee
lig
h
tweig
h
t
v
ar
ian
ts
u
n
d
e
r
an
id
en
tica
l
p
atien
t
-
lev
el,
leak
-
f
r
ee
p
r
o
t
o
co
l,
with
o
p
er
atin
g
p
o
in
ts
f
ix
e
d
o
n
th
e
i
n
ter
n
al
v
alid
atio
n
(
v
al
)
s
p
lit an
d
th
en
ev
alu
ated
o
n
ce
o
n
th
e
h
eld
-
o
u
t te
s
t set:
i)
Var
ian
t
A:
d
ee
p
e
r
h
ea
d
:
r
e
p
la
cin
g
th
e
b
aselin
e
ANN
h
ea
d
(
1
2
8
–
R
eL
U→
s
o
f
tm
ax
)
with
a
d
ee
p
er
h
ea
d
(
2
5
6
–
R
eL
U→Dr
o
p
o
u
t
(
0
.
3
)
→
1
2
8
–
R
eL
U→So
f
t
M
ax
)
.
ii)
Var
ian
t
B
:
u
n
ce
r
tain
ty
(
MC
f
ea
tu
r
e
-
d
r
o
p
o
u
t)
:
ag
g
r
e
g
atin
g
T
=3
0
s
to
ch
asti
c
f
o
r
war
d
p
ass
es
v
ia
th
e
m
ea
n
s
co
r
e
(
MC
-
m
ea
n
)
.
Fo
r
MI
AS,
d
ec
is
io
n
th
r
esh
o
ld
τ
is
s
elec
t
ed
o
n
th
e
v
al
s
p
lit
o
f
th
e
MC
-
m
ea
n
m
o
d
el;
f
o
r
C
B
I
S
-
DDSM,
τ
f
o
llo
ws th
e
v
al
o
p
er
atin
g
p
o
in
t o
f
th
e
d
e
ep
er
v
ar
ia
n
t (
as in
d
icate
d
in
T
ab
le
3
)
.
iii)
Var
ian
t
C
:
lig
h
tweig
h
t
f
u
s
io
n
(
α
-
g
ated
+
m
R
MR):
late
f
u
s
io
n
with
a
g
atin
g
f
ac
to
r
α
an
d
m
R
MR
f
ea
tu
r
e
s
elec
tio
n
o
f
s
ize
K
,
wh
er
e
(
α
,
K)
ar
e
ch
o
s
en
o
n
v
al
; th
e
test
s
p
lit r
em
ain
ed
u
n
s
ee
n
u
n
til f
in
al
ev
alu
atio
n
.
T
ab
le
3
s
h
o
ws
th
e
co
m
b
i
n
ed
r
esu
lts
f
o
r
ex
p
e
r
im
en
ts
A
-
C
o
n
MI
AS
an
d
C
B
I
S
-
DDSM
(
p
atien
t
-
lev
el,
leak
-
f
r
ee
,
an
d
th
r
esh
o
ld
s
f
ix
e
d
o
n
v
alid
atio
n
)
.
All
m
etr
ics
ar
e
co
m
p
u
te
d
o
n
th
e
h
el
d
-
o
u
t
test
s
et
;
n
o
r
ef
itti
n
g
o
n
test
.
Fo
r
MI
AS
-
B
,
τ
i
s
s
el
ec
ted
o
n
th
e
MC
-
m
ea
n
VAL
m
o
d
el;
f
o
r
C
B
I
S
-
B
,
τ
f
o
llo
ws
th
e
VAL
o
p
er
atin
g
p
o
in
t o
f
th
e
d
ee
p
er
v
a
r
ian
t,
as
in
d
icate
d
in
th
e
“
k
ey
”
c
o
lu
m
n
.
T
ab
le
3
.
C
o
m
b
i
n
ed
r
esu
lts
f
o
r
ex
p
er
im
en
ts
A
–
C
o
n
M
I
AS a
n
d
C
B
I
S
-
DDSM
(
p
atien
t
-
lev
el,
leak
-
f
r
ee
;
th
r
esh
o
ld
s
f
ix
ed
o
n
v
alid
atio
n
)
S
e
c
t
i
o
n
V
a
r
i
a
n
t
K
e
y
A
c
c
u
r
a
c
y
(
%)
S
e
n
s
i
t
i
v
i
t
y
(
%)
S
p
e
c
i
f
i
c
i
t
y
(
%)
P
r
e
c
i
s
i
o
n
(
%)
F1
-
sc
o
r
e
(
%)
AUC
N
o
r
mal
v
s
.
a
b
n
o
r
m
a
l
(
M
I
A
S
)
A
:
d
e
e
p
e
r
-
v
s.
-
b
a
s
e
l
i
n
e
A
:
b
a
se
l
i
n
e
h
e
a
d
1
2
8
-
R
e
LU
→s
o
f
t
m
a
x
,
τ@
V
A
L
-
9
5
%S
e
n
s
9
0
.
7
7
8
9
.
5
1
9
2
.
0
5
9
2
.
3
6
9
0
.
9
1
0
.
9
2
9
A
:
d
e
e
p
e
r
-
v
s.
-
b
a
s
e
l
i
n
e
A
:
d
e
e
p
e
r
h
e
a
d
2
5
6
→
d
r
o
p
o
u
t
(
0
.
3
)
→
1
2
8
9
0
.
3
8
9
0
.
2
4
9
0
.
5
2
9
0
.
9
1
9
0
.
5
8
0
.
9
2
7
B
:
u
n
c
e
r
t
a
i
n
t
y
B
:
M
C
-
f
e
a
t
u
r
e
-
d
r
o
p
o
u
t
(
me
a
n
)
T=
3
0
,
r
a
t
e
=
0
.
3
0
,
τ@
V
A
L
(
M
C
-
m
e
a
n
)
8
7
.
0
0
9
0
.
0
0
8
4
.
0
0
8
4
.
9
1
8
7
.
3
9
0
.
8
9
0
C:
g
a
t
e
d
+
m
R
M
R
C: α
-
g
a
t
e
d
+
m
R
M
R
(
b
e
st
V
A
L)
α=0
.
0
5
,
K
=
1
2
8
,
τ@
V
A
L
9
0
.
0
0
8
9
.
0
2
9
0
.
9
9
9
0
.
9
1
8
9
.
9
6
0
.
9
2
6
B
e
n
i
g
n
v
s
.
m
a
l
i
g
n
a
n
t
(
C
B
I
S
-
D
D
S
M
)
A
:
d
e
e
p
e
r
-
v
s.
-
b
a
s
e
l
i
n
e
A
:
b
a
se
l
i
n
e
h
e
a
d
1
2
8
-
R
e
LU
→s
o
f
t
m
a
x
,
τ@
V
A
L
-
9
5
%S
e
n
s
8
3
.
6
5
9
2
.
7
1
7
4
.
5
9
7
9
.
5
9
8
5
.
6
2
0
.
8
5
6
A
:
d
e
e
p
e
r
-
v
s.
-
b
a
s
e
l
i
n
e
A
:
d
e
e
p
e
r
h
e
a
d
2
5
6
→
d
r
o
p
o
u
t
(
0
.
3
)
→
1
2
8
8
2
.
0
2
9
2
.
7
1
7
1
.
3
2
7
7
.
1
7
8
4
.
5
0
0
.
8
4
2
B
:
u
n
c
e
r
t
a
i
n
t
y
B
:
M
C
-
f
e
a
t
u
r
e
-
d
r
o
p
o
u
t
(
me
a
n
)
T=
3
0
,
r
a
t
e
=
0
.
3
0
,
τ
f
r
o
m
V
A
L
(
D
e
e
p
e
r
)
7
9
.
6
0
9
5
.
2
4
6
2
.
5
0
7
3
.
5
3
8
2
.
9
9
0
.
8
9
5
C:
g
a
t
e
d
+
m
R
M
R
C: α
-
g
a
t
e
d
+
m
R
M
R
(
b
e
st
V
A
L)
α=0
.
2
0
,
K
=
6
4
,
τ
@
V
A
L
8
1
.
4
3
8
8
.
5
7
6
3
.
5
4
7
2
.
6
6
7
9
.
8
3
0
.
8
4
2
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
1
,
Feb
r
u
ar
y
2
0
2
6
:
861
-
8
7
7
870
T
ab
le
3
co
n
s
o
lid
ates
th
e
r
esu
lts
o
n
MI
AS
(
n
o
r
m
al
v
s
.
a
b
n
o
r
m
al
)
an
d
C
B
I
S
-
DDSM
(
b
en
ig
n
v
s
.
m
alig
n
an
t)
.
Va
r
ian
t
A
(
d
ee
p
e
r
h
ea
d
)
p
er
f
o
r
m
s
ess
en
tially
o
n
p
a
r
with
th
e
b
aselin
e
h
ea
d
(
MI
AS:
ac
cu
r
ac
y
9
0
.
7
7
%
v
s
.
9
0
.
3
8
%,
AUC
0
.
9
2
9
v
s
.
0
.
9
2
7
;
C
B
I
S
-
DDSM:
8
3
.
6
5
%
v
s
.
8
2
.
0
2
%,
0
.
8
5
6
v
s
.
0
.
8
4
2
)
,
in
d
icatin
g
th
at
p
er
f
o
r
m
a
n
ce
is
lar
g
ely
d
r
iv
en
b
y
th
e
f
in
e
-
tu
n
e
d
R
esNet
-
5
0
r
ep
r
esen
tatio
n
s
r
ath
er
th
an
h
ea
d
d
ep
th
;
d
if
f
er
en
ce
s
b
etwe
en
th
e
d
ee
p
e
r
an
d
b
aselin
e
h
ea
d
s
wer
e
n
o
t
s
tatis
t
ically
s
ig
n
if
ican
t,
co
n
s
is
ten
t
with
th
e
s
m
all
ab
s
o
lu
te
g
ap
s
.
Var
ian
t
B
(
u
n
c
er
tain
ty
,
MC
-
f
ea
tu
r
e
-
d
r
o
p
o
u
t,
MC
-
m
ea
n
ag
g
r
e
g
atio
n
)
p
r
o
v
i
d
es
an
u
n
ce
r
tain
ty
-
awa
r
e
o
p
er
atin
g
p
o
in
t
with
clin
ically
in
ter
p
r
etab
le
tr
ad
e
-
o
f
f
s
(
MI
AS:
ac
cu
r
ac
y
8
7
.
0
0
%,
s
en
s
itiv
ity
9
0
.
0
0
%,
s
p
ec
if
icity
8
4
.
0
0
%,
p
r
ec
is
io
n
8
4
.
9
1
%,
F1
-
s
co
r
e
8
7
.
3
9
%,
AUC
0
.
8
9
0
;
C
B
I
S
-
DDSM:
7
9
.
6
0
%,
9
5
.
2
4
%,
6
2
.
5
0
%,
7
3
.
5
3
%,
8
2
.
9
9
%,
0
.
8
9
5
)
,
an
d
th
e
q
u
a
r
tile
-
wis
e
er
r
o
r
tr
en
d
s
s
u
p
p
o
r
t
o
r
d
e
r
e
d
r
is
k
s
tr
atif
icatio
n
s
u
itab
le
f
o
r
tr
iag
e
o
r
s
ec
o
n
d
-
r
ea
d
er
u
s
e.
Var
ia
n
t
C
(
α
-
g
a
ted
+m
R
MR)
y
ield
s
a
p
r
ag
m
atic,
co
m
p
u
te
-
lig
h
t
en
h
an
ce
m
e
n
t
wh
ile
p
r
eser
v
in
g
th
e
s
am
e
d
ec
is
io
n
p
r
o
to
c
o
l
(
b
est
VAL
s
ettin
g
s
:
MI
A
S
α
=0
.
0
5
,
K=
1
2
8
→
test
ac
cu
r
ac
y
9
0
.
0
0
%,
s
en
s
itiv
ity
8
9
.
0
2
%,
s
p
ec
if
icity
9
0
.
9
9
%,
p
r
ec
is
io
n
9
0
.
9
1
%,
F1
-
s
co
r
e
8
9
.
9
6
%,
AUC
0
.
9
2
6
;
C
B
I
S
-
DD
SM
α
=0
.
2
0
,
K=
6
4
→8
1
.
4
3
%,
8
8
.
5
7
%,
6
3
.
5
4
%,
7
2
.
6
6
%,
7
9
.
8
3
%,
0
.
8
4
2
)
.
O
v
e
r
all,
i)
d
ee
p
er
h
ea
d
s
o
f
f
er
n
eg
lig
ib
le
g
ain
s
o
v
er
a
well
-
tu
n
ed
b
aselin
e,
ii)
MC
-
m
ea
n
u
n
ce
r
tain
ty
ad
d
s
ac
tio
n
ab
le
in
ter
p
r
etab
ilit
y
with
co
n
tr
o
llab
le
r
ec
all
–
s
p
ec
if
icity
tr
ad
e
-
o
f
f
s
,
an
d
iii)
α
-
g
ated
+m
R
MR
p
r
o
v
id
es
m
o
d
est
b
u
t
co
n
s
is
ten
t
im
p
r
o
v
em
e
n
ts
with
o
u
t a
lter
in
g
th
e
leak
-
f
r
ee
ev
alu
atio
n
.
3
.
3
.
Ca
lib
ra
t
i
o
n a
nd
o
pera
t
i
ng
-
po
int
a
na
ly
s
is
C
alib
r
atio
n
p
er
f
o
r
m
an
ce
was
ass
es
s
ed
u
s
in
g
r
eliab
ilit
y
d
iag
r
am
s
with
1
0
eq
u
al
-
f
r
eq
u
e
n
cy
b
in
s
,
alo
n
g
with
B
r
ier
s
co
r
e
a
n
d
E
C
E
.
Fo
r
clin
ical
in
ter
p
r
etab
ilit
y
,
th
e
d
ec
is
io
n
th
r
esh
o
ld
was
f
ix
ed
o
n
v
alid
atio
n
s
et
to
ac
h
iev
e
9
5
%
s
en
s
it
iv
ity
an
d
th
en
ap
p
lied
u
n
ch
an
g
ed
t
o
th
e
h
eld
-
o
u
t
test
s
et
as
s
h
o
wn
o
n
Fig
u
r
e
7
.
On
th
e
MI
AS
d
ataset
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
al)
,
th
e
f
u
s
io
n
m
o
d
el
s
h
o
wed
g
o
o
d
ca
lib
r
atio
n
(
B
r
ier
s
co
r
e=
0
.
0
3
4
;
E
C
E
=3
.
5
%).
At
th
e
v
alid
atio
n
-
s
elec
ted
th
r
esh
o
ld
,
th
e
m
o
d
el
ac
h
iev
ed
a
h
ig
h
-
r
ec
all
o
p
er
atin
g
p
o
in
t
with
s
en
s
itiv
ity
=1
0
0
.
0
%,
s
p
ec
if
icity
=9
.
0
9
%,
p
r
ec
is
io
n
=4
5
.
9
5
%,
an
d
F1
-
s
co
r
e
=
6
2
.
9
6
%.
T
h
is
tr
ad
e
-
o
f
f
em
p
h
asizes
s
en
s
itiv
ity
,
wh
ich
is
d
esira
b
le
f
o
r
s
cr
ee
n
in
g
,
wh
ile
also
r
ef
le
ctin
g
th
e
s
m
all
test
s
ize
(
N=
3
9
)
.
On
th
e
C
B
I
S
-
DDSM
d
ataset
(
b
en
ig
n
v
s
.
m
alig
n
an
t
)
,
ca
lib
r
atio
n
was
m
o
d
er
ate
(
B
r
ier
s
co
r
e=
0
.
1
3
9
;
E
C
E
=1
0
.
6
%),
with
m
ild
o
v
er
c
o
n
f
id
en
ce
at
h
ig
h
er
p
r
e
d
icted
p
r
o
b
a
b
ilit
ies.
At
th
e
v
alid
atio
n
-
s
elec
ted
th
r
esh
o
ld
(
τ
=0
.
0
6
3
)
,
th
e
test
s
et
p
er
f
o
r
m
an
ce
was
s
en
s
itiv
ity
=9
3
.
3
3
%,
s
p
ec
if
icity
=6
5
.
6
2
%,
p
r
ec
i
s
io
n
=7
4
.
8
1
%,
an
d
F1
-
s
co
r
e
=8
3
.
0
5
%.
T
h
e
r
eliab
i
lity
d
iag
r
am
i
n
Fig
u
r
e
7
(
a)
s
h
o
ws
well
-
ca
lib
r
ated
esti
m
ates
f
o
r
MI
AS,
wh
e
r
ea
s
Fig
u
r
e
7
(
b
)
co
n
f
ir
m
s
th
is
o
v
er
co
n
f
id
en
ce
,
p
ar
ticu
la
r
ly
at
h
ig
h
er
p
r
o
b
ab
ilit
y
b
in
s
.
(
a)
(
b
)
Fig
u
r
e
7
.
R
eliab
ilit
y
(
ca
lib
r
ati
o
n
)
cu
r
v
es with
B
r
ier
s
co
r
e
an
d
E
C
E
of
(
a
)
MI
AS
an
d
(
b
)
C
B
I
S
-
DDSM
As
an
ex
p
lo
r
at
o
r
y
s
tep
,
we
al
s
o
ap
p
lied
p
o
s
t
-
h
o
c
T
S
f
o
r
ca
l
ib
r
atio
n
.
T
S
was
f
itted
o
n
t
h
e
v
alid
atio
n
s
p
lit
b
y
m
in
im
izin
g
th
e
n
eg
ativ
e
lo
g
-
lik
elih
o
o
d
,
r
escalin
g
lo
g
its
as
=
(
/
)
.
B
ec
au
s
e
T
S
is
a
m
o
n
o
to
n
ic
tr
an
s
f
o
r
m
atio
n
,
d
is
cr
im
in
atio
n
m
etr
ics
s
u
ch
as
R
OC
-
AU
C
/P
R
-
AU
C
r
e
m
ain
u
n
ch
an
g
ed
.
On
C
B
I
S
-
DD
SM,
T
S
w
ith
=
3
.
183
im
p
r
o
v
ed
ca
lib
r
atio
n
(
B
r
ier
:
0
.
1
3
8
9
→0
.
1
2
8
5
;
E
C
E
:
1
0
.
6
%→5
.
1
%)
wh
ile
p
r
eser
v
in
g
th
e
s
am
e
v
al
-
f
ix
e
d
o
p
e
r
atin
g
-
p
o
in
t
m
etr
ics
o
n
test
(
s
en
s
itiv
ity
=9
3
.
3
3
%,
s
p
ec
if
icity
=6
5
.
6
2
%,
p
r
ec
is
io
n
=7
4
.
8
1
%,
an
d
F1
-
s
co
r
e
=8
3
.
0
5
%).
On
MI
AS,
T
S
with
=
2
.
863
d
eg
r
ad
e
d
ca
lib
r
atio
n
Evaluation Warning : The document was created with Spire.PDF for Python.