I
nte
rna
t
io
na
l J
o
urna
l o
f
I
nfo
rm
a
t
ics a
nd
Co
m
m
un
ica
t
io
n T
ec
hn
o
lo
g
y
(
I
J
-
I
CT
)
Vo
l.
1
5
,
No
.
4
,
D
e
c
em
b
er
20
2
6
,
p
p
.
1
8
8
1
~
1
8
9
2
I
SS
N:
2252
-
8
7
7
6
,
DOI
:
1
0
.
1
1
5
9
1
/iji
ct
.
v
1
5
i
4
.
pp
1
8
8
1
-
1
8
9
2
1881
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ict.
ia
esco
r
e.
co
m
CPU
-
o
p
timized
r
eg
ula
rized conv
o
lutiona
l neural n
et
wo
rk
a
rchitec
t
ure
for
multimedia
-
ba
sed
breas
t
ca
nc
er me
tas
tas
is
detec
tion
G
uid
o
um
Am
ina
1
,
B
o
uh
enni
Abdelh
a
k
e
2
1
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
H
i
g
h
e
r
N
o
r
m
a
l
S
c
h
o
o
l
o
f
K
o
u
b
a
,
K
o
u
b
a
,
A
l
g
e
r
i
a
2
K
u
w
a
i
t
O
i
l
C
o
mp
a
n
y
(
K
O
C
)
,
A
l
A
h
mad
i
,
K
u
w
a
i
t
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Dec
1
3
,
2
0
2
5
R
ev
is
ed
J
u
n
1
2
,
2
0
2
6
Acc
ep
ted
J
u
l 5
,
2
0
2
6
Th
e
h
ist
o
p
a
t
h
o
l
o
g
ica
l
d
iag
n
o
sis
o
f
b
re
a
st
c
a
n
c
e
r
m
e
tas
tas
e
s,
a
su
b
jec
ti
v
e
a
n
d
len
g
t
h
y
p
ro
c
e
ss
,
b
e
n
e
fit
s
fr
o
m
a
d
v
a
n
c
e
s
in
a
rti
ficia
l
i
n
telli
g
e
n
c
e
fo
r
it
s
a
u
to
m
a
ti
o
n
.
T
h
is
stu
d
y
p
ro
p
o
se
s
a
n
a
u
to
m
a
ti
c
c
las
sifica
ti
o
n
m
e
t
h
o
d
b
a
se
d
o
n
a
c
o
n
v
o
l
u
ti
o
n
a
l
n
e
u
ra
l
n
e
tw
o
rk
(
CNN
)
a
rc
h
it
e
c
tu
re
with
p
r
o
g
re
ss
iv
e
re
g
u
lariz
a
ti
o
n
(Ba
tch
n
o
rm
a
li
z
a
ti
o
n
a
n
d
a
d
a
p
t
iv
e
d
ro
p
o
u
t
),
train
e
d
o
n
t
h
e
P
a
tch
Ca
m
e
ly
o
n
(P
Ca
m
)
d
a
tas
e
t.
Ou
r
a
p
p
r
o
a
c
h
in
te
g
ra
tes
a
d
a
ta
a
u
g
m
e
n
tatio
n
stra
teg
y
sp
e
c
ifi
c
t
o
m
e
d
ica
l
ima
g
e
s
a
n
d
a
d
v
a
n
c
e
d
o
p
t
imiz
a
ti
o
n
fo
r
CP
U
e
n
v
iro
n
m
e
n
ts,
a
ll
o
wi
n
g
u
s
to
b
a
lan
c
e
p
e
rfo
rm
a
n
c
e
,
ro
b
u
s
tn
e
ss
,
a
n
d
c
o
m
p
u
tati
o
n
a
l
a
c
c
e
ss
ib
il
it
y
.
T
h
e
a
rc
h
it
e
c
tu
re
c
o
m
p
rise
s
o
n
l
y
9
4
,
0
0
0
p
a
ra
m
e
ters
,
2
2
3
ti
m
e
s
fe
we
r
th
a
n
lea
d
i
n
g
m
o
d
e
ls.
M
u
lt
i
-
c
rit
e
ria
e
v
a
lu
a
ti
o
n
sh
o
ws
c
o
m
p
e
ti
ti
v
e
re
su
lt
s:
a
re
a
u
n
d
e
r
th
e
c
u
r
ve
(
AUC
)
o
f
9
2
.
9
2
%
,
F
1
-
sc
o
re
o
f
8
2
.
3
9
%
,
a
c
c
u
ra
c
y
o
f
8
8
.
7
4
%
,
a
n
d
s
p
e
c
ifi
c
it
y
o
f
9
0
.
6
1
%
.
A
b
o
o
tstra
p
ro
b
u
stn
e
ss
a
n
a
l
y
sis
p
ro
v
id
e
s
9
5
%
c
o
n
fi
d
e
n
c
e
i
n
terv
a
ls
(AU
C:
[
8
9
.
5
6
%
-
9
2
.
9
1
%
]),
a
n
d
a
n
a
b
lati
o
n
st
u
d
y
q
u
a
n
ti
fies
th
e
re
d
u
c
ti
o
n
i
n
o
v
e
rfit
ti
n
g
th
ro
u
g
h
p
ro
g
re
ss
iv
e
d
ro
p
o
u
t
(3
3
.
6
%
)
.
T
h
is
re
se
a
rc
h
v
a
li
d
a
tes
th
a
t
a
we
ll
-
re
g
u
late
d
a
n
d
CP
U
-
o
p
ti
m
ize
d
CNN
a
rc
h
it
e
c
tu
re
c
a
n
a
c
h
iev
e
c
li
n
ica
ll
y
re
lev
a
n
t
p
e
rf
o
rm
a
n
c
e
wh
il
e
re
m
a
in
in
g
a
c
c
e
ss
ib
le
o
n
sta
n
d
a
rd
e
n
v
i
ro
n
m
e
n
ts,
th
u
s
o
p
e
n
in
g
u
p
c
o
n
c
re
te
p
r
o
sp
e
c
ts
fo
r
th
e
d
e
v
e
l
o
p
m
e
n
t
o
f
p
ra
c
ti
c
a
l
AI
t
o
o
ls
in
d
ig
it
a
l
p
a
th
o
l
o
g
y
.
K
ey
w
o
r
d
s
:
C
las
s
if
icatio
n
o
f
h
is
to
p
ath
o
lo
g
ical
im
a
g
es
C
NN
ar
ch
itectu
r
e
C
o
m
p
u
tatio
n
al
o
p
tim
izatio
n
C
PU o
p
tim
izatio
n
R
eg
u
lar
izatio
n
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Gu
id
o
u
m
Am
in
a
Dep
ar
tm
en
t o
f
C
o
m
p
u
ter
Scie
n
ce
,
Hig
h
er
N
o
r
m
al
Sch
o
o
l o
f
Ko
u
b
a
Alg
ier
s
,
Alg
er
ia
E
m
ail:
am
in
a.
g
u
id
o
u
m
@
g
.
en
s
-
k
o
u
b
a.
d
z
1.
I
NT
RO
D
UCT
I
O
N
B
r
ea
s
t
ca
n
ce
r
r
ep
r
esen
ts
a
m
a
jo
r
p
u
b
lic
h
ea
lth
c
h
allen
g
e,
with
ap
p
r
o
x
im
ately
2
.
3
m
illi
o
n
n
ew
ca
s
es
d
iag
n
o
s
ed
an
n
u
ally
wo
r
ld
wid
e.
His
to
p
ath
o
lo
g
ical
d
iag
n
o
s
is
,
b
ased
o
n
th
e
m
icr
o
s
co
p
ic
ex
am
in
atio
n
o
f
tis
s
u
e
s
ec
tio
n
s
,
is
th
e
g
o
ld
s
tan
d
ar
d
f
o
r
d
etec
tin
g
ly
m
p
h
n
o
d
e
m
et
astas
es
[
1]
.
Ho
wev
er
,
th
is
p
r
o
ce
s
s
i
s
s
u
b
jectiv
e,
tim
e
-
co
n
s
u
m
in
g
,
an
d
s
u
b
ject
to
s
ig
n
if
ican
t
in
ter
-
o
b
s
er
v
er
v
ar
iab
ilit
y
.
T
h
e
ad
v
en
t
o
f
d
i
g
ital
p
ath
o
lo
g
y
an
d
d
e
e
p
l
e
a
r
n
i
n
g
(
D
L
)
te
c
h
n
i
q
u
e
s
o
f
f
e
r
s
r
e
v
o
l
u
t
i
o
n
a
r
y
p
r
o
s
p
e
c
t
s
f
o
r
a
u
t
o
m
a
t
i
n
g
a
n
d
im
p
r
o
v
i
n
g
d
i
a
g
n
o
s
t
i
c
a
c
c
u
r
a
c
y
[
2]
.
T
h
e
Patch
C
am
ely
o
n
(
PC
am
)
d
ataset,
co
m
p
r
is
in
g
3
2
7
,
6
8
0
co
lo
r
ed
p
atch
es
(
9
6
×9
6
-
p
i
x
els)
f
r
o
m
h
is
to
lo
g
ical
s
ec
tio
n
s
o
f
ly
m
p
h
n
o
d
es,
r
e
p
r
esen
ts
a
s
tan
d
ar
d
ized
b
en
c
h
m
ar
k
f
o
r
th
e
d
ev
elo
p
m
en
t
o
f
b
in
ar
y
class
if
icatio
n
alg
o
r
ith
m
s
(
h
ea
l
th
y
v
s
.
m
etastatic
tis
s
u
e)
[
1]
.
T
h
e
f
ield
o
f
h
is
to
lo
g
ical
im
a
g
e
an
aly
s
is
u
s
in
g
DL
h
as
s
ee
n
s
p
ec
tacu
lar
ad
v
a
n
ce
s
in
r
ec
en
t
y
ea
r
s
.
T
h
e
s
em
in
al
w
o
r
k
[
1]
d
em
o
n
s
tr
ated
f
o
r
th
e
f
ir
s
t
tim
e
th
at
DL
alg
o
r
ith
m
s
c
o
u
ld
o
u
t
p
er
f
o
r
m
p
ath
o
lo
g
is
ts
in
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
4
,
D
e
c
em
b
er
20
2
6
:
1
8
8
1
-
1
8
9
2
1882
d
etec
tin
g
b
r
ea
s
t
ca
n
ce
r
m
et
astas
es.
Mo
r
e
r
ec
en
tly
,
atte
n
tio
n
h
as
f
o
cu
s
ed
o
n
im
p
r
o
v
in
g
c
o
m
p
u
tatio
n
al
ef
f
icien
cy
an
d
g
e
n
er
aliza
tio
n
.
Su
b
s
eq
u
e
n
tly
[
3]
,
p
r
o
p
o
s
ed
E
f
f
icien
tNetV2
,
th
r
o
u
g
h
n
eu
r
al
ar
c
h
itectu
r
e
r
esear
ch
th
at
tak
es
in
to
ac
c
o
u
n
t
tr
ain
in
g
a
n
d
p
r
o
g
r
ess
iv
e
lear
n
in
g
with
a
d
ap
tiv
e
r
e
g
u
lar
i
za
tio
n
,
ac
h
iev
in
g
a
to
p
-
1
ac
cu
r
ac
y
o
f
8
7
.
3
%
o
n
I
m
ag
eNe
t
wh
ile
tr
ain
in
g
5
to
1
1
tim
es
f
aster
th
an
v
is
io
n
tr
an
s
f
o
r
m
er
s
(
ViT
)
.
I
n
p
a
r
allel
[
4]
,
p
r
o
p
o
s
ed
C
o
n
v
NeX
t,
a
m
o
d
er
n
ized
p
u
r
e
C
o
n
v
Net
ar
c
h
itectu
r
e
with
t
r
an
s
f
o
r
m
er
d
esig
n
p
r
in
cip
les,
wh
ich
ac
h
iev
ed
8
7
.
8
%
ac
cu
r
ac
y
o
n
I
m
ag
eNe
t
to
p
-
1
a
n
d
o
u
tp
er
f
o
r
m
e
d
s
win
tr
an
s
f
o
r
m
e
r
s
o
n
C
OC
O
d
etec
tio
n
an
d
ADE
2
0
K
s
eg
m
en
tatio
n
b
en
c
h
m
ar
k
s
.
Mo
r
e
r
ec
e
n
tly
,
ViT
s
[
5]
a
n
d
th
e
s
win
tr
a
n
s
f
o
r
m
er
[
6]
h
av
e
ch
allen
g
ed
th
e
d
o
m
in
an
ce
o
f
co
n
v
o
l
u
tio
n
al
n
e
u
r
al
n
etwo
r
k
(
C
NNs
)
b
y
ap
p
ly
in
g
s
elf
-
atten
tio
n
m
ec
h
an
is
m
s
to
im
ag
e
p
at
ch
es,
ac
h
iev
in
g
t
o
p
p
er
f
o
r
m
an
ce
o
n
m
a
n
y
m
ed
ica
l
im
ag
in
g
b
en
ch
m
ar
k
s
.
Ho
we
v
er
,
th
ese
ar
ch
itectu
r
es
ty
p
ica
lly
r
eq
u
ir
e
m
illi
o
n
s
o
f
p
ar
a
m
eter
s
an
d
s
u
b
s
tan
tial
GPU
r
eso
u
r
ce
s
.
Am
o
n
g
r
e
ce
n
t
DL
-
b
ased
ap
p
r
o
ac
h
es
f
o
r
b
r
ea
s
t
ca
n
ce
r
d
iag
n
o
s
is
,
Mo
b
ar
k
[
7]
p
r
o
p
o
s
ed
C
o
r
o
Net,
a
C
NN
m
o
d
el
u
s
in
g
th
e
Xce
p
tio
n
ar
ch
itectu
r
e
p
r
e
-
tr
ai
n
e
d
o
n
I
m
a
g
eNe
t.
T
r
ain
ed
o
n
th
e
C
B
I
S
-
DD
SM
m
am
m
o
g
r
a
m
d
ataset,
C
o
r
o
Net
ac
h
iev
es
an
ac
cu
r
ac
y
o
f
9
4
.
9
2
%
in
f
o
u
r
-
class
class
if
i
ca
tio
n
(
B
en
ig
n
v
s
.
m
alig
n
an
t
m
ass
es,
b
en
ig
n
v
s
.
m
alig
n
an
t
ca
lcif
icatio
n
s
)
an
d
8
6
.
6
7
%
in
b
i
n
ar
y
class
if
icatio
n
(
C
alcif
icatio
n
s
v
s
.
m
ass
es).
T
h
e
ar
ch
itectu
r
e
co
m
p
r
is
es
7
1
la
y
er
s
f
o
r
a
to
tal
o
f
3
3
.
9
7
m
illi
o
n
p
ar
am
ete
r
s
;
th
is
m
o
d
el
r
e
m
ain
s
h
ig
h
ly
p
ar
am
eter
i
ze
d
an
d
r
eq
u
ir
es
s
ig
n
if
ican
t
co
m
p
u
tin
g
r
eso
u
r
ce
s
,
wh
ich
lim
its
its
d
ep
lo
y
m
en
t
in
clin
ical
en
v
ir
o
n
m
en
ts
with
lim
ited
h
ar
d
war
e.
T
h
e
s
tu
d
y
[
8]
co
n
cr
etely
illu
s
tr
ates
o
n
e
o
f
th
e
m
ain
ch
allen
g
es
f
o
r
th
e
clin
ical
d
ep
lo
y
m
e
n
t
o
f
AI
in
d
ig
ital
p
ath
o
lo
g
y
:
th
e
p
o
o
r
g
en
er
aliza
tio
n
o
f
DL
m
o
d
els
to
n
ew
co
n
tex
ts
.
T
h
e
au
th
o
r
s
ev
alu
ated
a
m
o
d
el,
in
itially
tr
ain
ed
o
n
th
e
m
u
lticen
ter
C
AM
E
L
YON
d
ataset
f
o
r
d
etec
tin
g
b
r
ea
s
t
ca
n
ce
r
m
e
tast
ase
s
in
s
en
tin
el
ly
m
p
h
n
o
d
es.
I
ts
b
aselin
e
p
er
f
o
r
m
an
c
e
o
n
th
e
C
AM
E
L
YON1
6
test
s
et
was
ex
ce
llen
t,
with
a
FR
OC
s
co
r
e
o
f
0
.
8
3
8
.
T
h
is
s
tu
d
y
h
ig
h
lig
h
t
s
th
at
r
o
b
u
s
tn
ess
an
d
a
d
ap
t
ab
ilit
y
ar
e
as
cr
u
cial
as
ab
s
o
lu
te
b
en
ch
m
a
r
k
p
er
f
o
r
m
an
ce
.
I
t
v
alid
ates
th
e
n
ee
d
to
d
e
v
elo
p
m
o
r
e
r
o
b
u
s
t
lear
n
in
g
m
eth
o
d
s
(
ad
v
a
n
ce
d
tr
an
s
f
er
lear
n
in
g
,
tar
g
eted
d
ata
au
g
m
en
tatio
n
,
s
elf
-
s
u
p
er
v
is
ed
lea
r
n
in
g
)
an
d
p
ip
elin
es
th
at
in
co
r
p
o
r
ate
a
f
ac
ilit
ated
co
n
tin
u
o
u
s
ad
ap
tatio
n
p
h
ase
f
o
r
s
u
s
tain
ab
le
clin
ical
d
ep
lo
y
m
e
n
t.
Face
d
with
th
ese
ch
allen
g
es,
in
n
o
v
ativ
e
h
y
b
r
id
ar
ch
itectu
r
es
ar
e
em
er
g
in
g
to
im
p
r
o
v
e
b
o
th
th
e
ac
cu
r
ac
y
an
d
r
o
b
u
s
tn
ess
o
f
m
o
d
els.
Fo
r
ex
am
p
l
e
[
9]
,
p
r
o
p
o
s
ed
a
C
NN
-
t
r
an
s
f
o
r
m
er
h
y
b
r
i
d
m
o
d
el
s
p
ec
if
ically
d
esig
n
ed
to
lev
er
ag
e
tem
p
o
r
al
in
f
o
r
m
atio
n
b
y
s
im
u
ltan
eo
u
s
ly
in
teg
r
atin
g
p
r
e
v
io
u
s
an
d
cu
r
r
en
t
m
am
m
o
g
r
a
p
h
ic
im
ag
es
o
f
th
e
s
am
e
p
atien
t.
T
h
eir
ar
ch
itectu
r
e
u
s
es
a
m
o
d
if
ied
R
esNe
t
5
0
f
o
r
lo
ca
l
f
ea
tu
r
e
ex
tr
ac
tio
n
,
a
t
r
a
n
s
f
o
r
m
er
e
n
c
o
d
er
with
a
m
u
lti
-
h
ea
d
atten
t
io
n
m
ec
h
an
is
m
an
d
p
o
s
itio
n
a
l
co
d
in
g
to
ca
p
tu
r
e
lo
n
g
-
ter
m
d
ep
en
d
en
cies
an
d
s
u
b
tle
ch
an
g
es
b
etwe
en
ex
a
m
in
atio
n
s
,
an
d
a
p
e
r
-
ch
an
n
el
atten
tio
n
m
o
d
u
le
to
am
p
lify
d
is
cr
im
in
atin
g
f
ea
tu
r
es.
E
v
alu
ated
o
n
a
d
e
d
icate
d
d
ataset,
th
is
m
o
d
el
ac
h
iev
es
an
ar
ea
u
n
d
er
th
e
cu
r
v
e
(
AUC
)
o
f
9
2
.
5
8
%,
th
u
s
d
em
o
n
s
tr
atin
g
th
e
c
r
u
cial
co
n
tr
ib
u
tio
n
o
f
tem
p
o
r
al
m
o
d
elin
g
a
n
d
h
y
b
r
id
ar
ch
itectu
r
es to
m
im
ic
th
e
s
eq
u
en
tial r
ea
s
o
n
in
g
o
f
r
ad
io
lo
g
is
ts
an
d
im
p
r
o
v
e
ea
r
ly
d
etec
tio
n
.
C
atala
n
o
et
a
l
.
[
1
0
]
p
r
o
p
o
s
ed
a
m
eth
o
d
f
o
r
d
etec
tin
g
an
d
lo
ca
lizin
g
b
r
ea
s
t
ca
n
ce
r
f
r
o
m
h
e
m
ato
x
y
li
n
an
d
eo
s
in
(
HE
)
s
tain
ed
h
is
to
lo
g
ical
im
ag
es
u
s
in
g
s
ev
er
al
C
NN
ar
ch
itectu
r
es.
T
h
ey
co
m
p
ar
ed
a
c
u
s
to
m
C
NN
(
Stan
d
ar
d
_
C
NN)
,
E
f
f
icien
tNet,
R
esNet5
0
,
VGG
-
1
6
,
VGG
-
1
9
,
an
d
Mo
b
ileNet,
as
well
as
a
n
ew
m
o
d
el
ca
lled
MR_
Net.
T
h
e
m
o
d
els
wer
e
tr
ain
ed
o
n
th
e
B
R
AC
S
d
ataset,
wh
ich
co
n
tain
s
4
5
3
9
r
eg
i
o
n
s
o
f
in
ter
est
(
R
OI
s
)
an
n
o
tated
in
to
t
h
r
ee
class
es
(
b
en
ig
n
,
aty
p
ical,
m
alig
n
an
t)
.
Af
ter
p
r
ep
r
o
ce
s
s
in
g
an
d
d
ata
au
g
m
en
tatio
n
(
h
o
r
izo
n
tal
f
lip
,
b
r
ig
h
tn
ess
,
z
o
o
m
)
,
th
e
f
in
al
d
ataset
co
m
p
r
is
ed
5
6
2
8
im
a
g
es.
T
h
e
b
est
a
cc
u
r
ac
ies
o
b
tain
e
d
wer
e
7
3
%
f
o
r
Mo
b
ileNet
an
d
VGG
-
1
9
,
6
9
%
f
o
r
MR_
Net,
an
d
6
7
%
f
o
r
s
tan
d
ar
d
_
C
NN.
Fin
e
-
tu
n
in
g
Mo
b
ileNet,
VGG
-
1
6
,
an
d
VG
G
-
1
9
d
id
n
o
t
im
p
r
o
v
e
p
er
f
o
r
m
an
ce
(
VGG
-
19
’
s
ac
cu
r
ac
y
d
ec
r
ea
s
ed
f
r
o
m
7
3
%
to
7
1
%).
Ho
wev
er
,
m
etr
ics s
u
c
h
as AU
C
o
r
F1
-
s
co
r
e
ar
e
n
o
t r
ep
o
r
ted
,
an
d
th
e
n
u
m
b
e
r
o
f
m
o
d
el
p
ar
am
eter
s
is
n
o
t sp
ec
if
ied
,
wh
ic
h
lim
its
q
u
an
titativ
e
co
m
p
ar
is
o
n
i
n
ter
m
s
o
f
co
m
p
u
tatio
n
al
e
f
f
icien
cy
.
Face
d
with
th
e
d
o
m
ain
s
h
if
t
in
d
u
ce
d
b
y
s
ca
n
n
er
v
ar
ia
b
ilit
y
,
s
elf
-
s
u
p
er
v
is
ed
lear
n
in
g
ap
p
r
o
ac
h
es
aim
ed
at
lear
n
in
g
i
n
h
er
e
n
tly
r
o
b
u
s
t
r
ep
r
esen
tatio
n
s
h
av
e
em
er
g
ed
.
W
h
ile
v
er
y
lar
g
e
-
s
ca
le
f
o
u
n
d
atio
n
m
o
d
els
(
e.
g
.
,
UNI
,
Vir
c
h
o
w)
ac
h
iev
e
p
ea
k
p
er
f
o
r
m
an
ce
b
u
t
at
a
p
r
o
h
ib
itiv
e
co
m
p
u
tatio
n
al
co
s
t,
r
e
ce
n
t
wo
r
k
ex
p
lo
r
es
lig
h
tweig
h
t
ar
ch
itectu
r
es
o
p
ti
m
ize
d
f
o
r
g
e
n
er
aliza
tio
n
.
Fo
r
ex
am
p
le,
His
to
L
ite
[
11]
o
f
f
er
s
a
lig
h
tweig
h
t
s
elf
-
s
u
p
er
v
is
ed
f
r
am
ew
o
r
k
b
ased
o
n
a
d
u
al
s
tr
ea
m
o
f
au
to
e
n
co
d
e
r
s
an
d
a
co
n
t
r
ast o
b
jectiv
e,
co
u
p
led
with
ad
a
p
tiv
e
au
g
m
en
tatio
n
(
Ad
ap
tiv
e
His
to
R
o
tate)
.
Sp
ec
if
ically
ev
alu
ated
o
n
a
s
ca
n
n
er
b
ias (
Ap
er
i
o
v
s
.
Sak
u
r
a)
,
His
to
L
it
e
(
7
M
p
ar
am
eter
s
)
d
em
o
n
s
tr
ate
s
v
er
y
lo
w
r
ep
r
esen
tatio
n
s
h
if
t
an
d
th
e
b
est
in
ter
-
s
ca
n
n
er
g
e
n
er
aliza
tio
n
am
o
n
g
th
e
co
m
p
ar
e
d
m
o
d
els,
d
esp
ite
s
lig
h
tly
lo
wer
ab
s
o
lu
te
ac
cu
r
ac
y
th
an
th
e
lar
g
er
m
o
d
els.
T
h
is
wo
r
k
illu
s
tr
ates
th
e
tr
ad
e
-
o
f
f
b
etwe
en
ac
cu
r
ac
y
,
r
o
b
u
s
tn
ess
,
an
d
ef
f
icie
n
cy
,
an
d
v
alid
ates
th
e
r
elev
a
n
ce
o
f
s
p
ec
ialized
lig
h
tweig
h
t a
r
ch
itectu
r
es f
o
r
r
eliab
le
d
ep
lo
y
m
e
n
t in
r
eso
u
r
c
e
-
co
n
s
tr
ain
ed
clin
ical
s
ettin
g
s
.
Fin
ally
,
th
e
s
y
s
tem
atic
r
ev
iew
[
12]
p
r
o
v
id
es
a
co
m
p
r
eh
e
n
s
iv
e
an
aly
s
is
o
f
th
e
co
n
tr
ib
u
tio
n
o
f
DL
to
th
e
an
aly
s
is
o
f
h
is
to
p
at
h
o
lo
g
ical
im
ag
es
o
f
b
r
ea
s
t
ca
n
ce
r
.
B
y
s
y
n
th
esizin
g
1
9
9
r
ef
er
e
n
ce
s
,
in
clu
d
in
g
1
8
2
s
tu
d
ies
p
u
b
lis
h
ed
b
etwe
en
2
0
1
4
an
d
2
0
2
5
,
th
e
au
th
o
r
s
d
em
o
n
s
tr
ate
h
o
w
DL
a
r
ch
ite
ctu
r
es,
p
ar
ticu
lar
ly
C
NNs
[
13]
,
g
e
n
er
ativ
e
ad
v
er
s
ar
ial
n
etwo
r
k
s
(
GANs)
,
an
d
V
iT
s
,
h
av
e
s
ig
n
if
ican
tly
im
p
r
o
v
ed
th
e
ac
cu
r
ac
y
o
f
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
C
P
U
-
o
p
timiz
ed
r
eg
u
la
r
iz
ed
co
n
vo
lu
tio
n
a
l n
eu
r
a
l n
etw
o
r
k
a
r
ch
itectu
r
e
fo
r
mu
ltimed
ia
…
(
Gu
id
o
u
m
A
min
a
)
1883
class
if
icatio
n
,
s
eg
m
en
tatio
n
,
an
d
tu
m
o
r
d
etec
tio
n
task
s
.
T
h
e
ar
ticle
also
h
ig
h
lig
h
ts
th
e
cr
u
cial
r
o
le
o
f
tr
an
s
f
er
lear
n
in
g
a
n
d
e
x
p
lain
ab
le
A
I
(
XAI
)
tech
n
iq
u
es
i
n
ad
d
r
ess
in
g
d
ata
g
a
p
s
an
d
im
p
r
o
v
in
g
cli
n
ical
in
ter
p
r
etab
ilit
y
.
Ho
wev
er
,
th
e
r
ev
iew
id
en
tifi
es
p
er
s
is
ten
t
ch
allen
g
es
to
clin
ical
ad
o
p
tio
n
,
in
clu
d
in
g
v
ar
iab
ilit
y
in
s
tain
in
g
p
r
o
to
co
ls
,
a
lack
o
f
lar
g
e
,
d
iv
er
s
e,
an
d
an
n
o
tated
d
atasets
,
an
d
is
s
u
es
o
f
r
o
b
u
s
tn
ess
,
eq
u
ity
,
an
d
r
eg
u
latio
n
.
I
n
co
n
cl
u
s
io
n
,
th
e
a
u
th
o
r
s
p
r
o
p
o
s
e
as
f
u
tu
r
e
p
er
s
p
ec
tiv
es
th
e
d
ev
el
o
p
m
en
t
o
f
m
o
r
e
g
e
n
er
aliza
b
le
m
o
d
els,
m
u
ltimo
d
al
d
ata
in
teg
r
atio
n
,
an
d
th
e
p
r
io
r
itizatio
n
o
f
tr
an
s
p
ar
en
t
to
o
ls
f
o
r
r
esp
o
n
s
ib
le
in
teg
r
atio
n
in
to
d
iag
n
o
s
tic
wo
r
k
f
lo
ws.
I
t
is
im
p
o
r
tan
t
to
n
o
te
th
at
t
r
a
n
s
f
o
r
m
er
m
o
d
els
f
ir
s
t
ac
h
iev
e
d
s
u
cc
ess
in
n
atu
r
al
la
n
g
u
a
g
e
p
r
o
ce
s
s
in
g
.
T
h
e
b
id
ir
ec
tio
n
al
en
co
d
er
r
ep
r
esen
tatio
n
s
f
r
o
m
tr
an
s
f
o
r
m
e
r
s
(
B
E
R
T
)
m
o
d
el
[
14]
h
as
ac
h
iev
ed
n
ew
cu
ttin
g
-
ed
g
e
r
esu
lt
s
o
n
elev
en
n
atu
r
al
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
task
s
,
in
clu
d
in
g
q
u
esti
o
n
an
s
wer
in
g
an
d
lin
g
u
is
tic
in
f
er
en
ce
.
B
E
R
T
is
p
r
e
-
tr
ain
ed
o
n
B
o
o
k
s
C
o
r
p
u
s
(
8
0
0
m
illi
o
n
wo
r
d
s
)
an
d
W
ik
ip
ed
ia
(
2
.
5
b
illi
o
n
wo
r
d
s
)
.
I
ts
ar
ch
itectu
r
e
is
a
m
u
lti
-
lay
er
ed
b
id
ir
ec
tio
n
al
t
r
an
s
f
o
r
m
er
en
co
d
er
.
T
h
e
B
E
R
T
B
ASE
v
er
s
io
n
co
m
p
r
is
es
1
2
lay
er
s
,
a
h
id
d
en
s
ize
o
f
7
6
8
,
1
2
atten
tio
n
h
ea
d
s
,
an
d
1
1
0
m
illi
o
n
p
ar
a
m
eter
s
;
th
e
B
E
R
T
L
AR
GE
v
er
s
io
n
co
m
p
r
is
es
2
4
lay
er
s
,
a
h
id
d
en
s
ize
o
f
1
0
2
4
,
1
6
atten
tio
n
h
ea
d
s
,
an
d
3
4
0
m
illi
o
n
p
ar
am
eter
s
.
B
E
R
T
B
A
SE
wa
s
tr
ain
ed
o
n
4
C
lo
u
d
T
PUs
,
an
d
B
E
R
T
L
A
R
GE
o
n
1
6
C
lo
u
d
T
PUs
.
B
E
R
T
is
d
esig
n
ed
f
o
r
t
ex
t
to
k
en
s
eq
u
en
ce
s
u
s
in
g
W
o
r
d
Piece
em
b
ed
d
in
g
s
,
n
o
t
f
o
r
im
ag
es.
I
n
o
u
r
co
n
te
x
t
o
f
class
if
y
in
g
h
is
to
p
ath
o
lo
g
ical
p
atch
es
with
a
lim
ited
d
ataset
an
d
a
C
PU
-
b
o
u
n
d
co
m
p
u
tatio
n
al
tar
g
et,
C
NNs
r
em
ain
m
o
r
e
s
u
itab
le.
C
NNs
b
en
ef
it
f
r
o
m
in
d
u
ctiv
e
b
iases
(
tr
an
s
latio
n
a
l
eq
u
iv
ar
ian
ce
,
lo
ca
l
co
n
n
ec
ti
v
ity
)
th
at
n
at
u
r
ally
co
n
f
o
r
m
to
th
e
s
tr
u
ctu
r
e
o
f
tis
s
u
e
tex
tu
r
es.
T
h
eir
p
ar
a
m
etr
ic
lig
h
tn
ess
allo
ws
f
o
r
C
PU
-
b
o
u
n
d
tr
ain
in
g
,
a
n
d
r
eg
u
lar
izatio
n
tech
n
iq
u
es
s
u
c
h
as
b
atch
n
o
r
m
aliza
tio
n
an
d
p
r
o
g
r
ess
iv
e
d
r
o
p
o
u
t
e
f
f
ec
tiv
ely
co
n
tr
o
l
o
v
er
f
itti
n
g
.
T
h
u
s
,
o
u
r
ch
o
ice
o
f
a
well
-
r
eg
u
lated
C
NN
ar
ch
itectu
r
e
is
ju
s
tifie
d
b
y
r
eso
u
r
ce
co
n
s
tr
ain
ts
an
d
th
e
n
atu
r
e
o
f
t
h
e
task
.
Desp
ite
ex
ce
p
tio
n
al
p
er
f
o
r
m
a
n
ce
r
e
p
o
r
ted
with
c
o
m
p
lex
a
r
c
h
itectu
r
es
an
d
lar
g
e
d
atasets
,
f
ew
s
tu
d
ies
h
av
e
f
o
cu
s
ed
o
n
o
p
tim
izatio
n
f
o
r
co
m
p
u
tatio
n
ally
lim
ited
en
v
ir
o
n
m
en
ts
,
wh
ich
ar
e
r
e
p
r
esen
tativ
e
o
f
m
an
y
clin
ical
an
d
r
esear
ch
s
ettin
g
s
.
T
h
is
s
tu
d
y
s
p
ec
if
ically
ad
d
r
ess
es
th
is
ch
allen
g
e
b
y
p
r
o
p
o
s
in
g
a
co
m
p
r
e
h
en
s
i
v
e
m
eth
o
d
o
l
o
g
y
th
at
b
alan
ce
s
d
is
cr
im
in
an
t
p
er
f
o
r
m
a
n
ce
,
r
o
b
u
s
tn
ess
,
an
d
co
m
p
u
tatio
n
al
ac
ce
s
s
ib
ilit
y
.
Ou
r
co
n
tr
ib
u
tio
n
s
ar
e:
(
i
)
a
lig
h
t
weig
h
t
C
NN
ar
ch
itectu
r
e
(
9
4
K
p
ar
am
eter
s
)
with
p
r
o
g
r
es
s
iv
e
r
eg
u
lar
izatio
n
s
u
itab
le
f
o
r
h
is
to
lo
g
ical
im
ag
es;
(
ii
)
a
C
PU
-
o
p
ti
m
ized
tr
ain
in
g
p
ip
elin
e
th
at
co
m
p
letes
in
ap
p
r
o
x
im
ately
2
.
4
h
o
u
r
s
o
n
s
tan
d
ar
d
h
ar
d
war
e
;
an
d
(
iii
)
a
m
u
lti
-
cr
iter
ia
ev
al
u
atio
n
d
em
o
n
s
tr
atin
g
th
at
a
well
-
r
eg
u
lated
C
NN
ca
n
ac
h
iev
e
clin
ically
r
ele
v
an
t
p
er
f
o
r
m
an
ce
(
AUC
9
2
.
9
2
%,
ac
cu
r
ac
y
8
8
.
7
4
%)
with
2
2
3
tim
es
f
ewe
r
p
ar
am
eter
s
th
an
lead
in
g
-
ed
g
e
ap
p
r
o
ac
h
es(E
f
f
icie
n
tNetV2
-
S,
C
o
n
v
NeX
t)
,
m
ak
in
g
AI
to
o
ls
m
o
r
e
ac
ce
s
s
ib
le
f
o
r
d
ig
ital
p
at
h
o
lo
g
y
i
n
r
es
o
u
r
ce
-
c
o
n
s
tr
ain
ed
en
v
ir
o
n
m
e
n
ts
.
Ou
r
co
n
tr
ib
u
tio
n
d
o
es
n
o
t
lie
i
n
a
r
ad
ical
ar
ch
itectu
r
al
in
n
o
v
atio
n
,
b
u
t
in
th
e
d
em
o
n
s
tr
a
tio
n
th
at
a
well
-
r
eg
u
lated
an
d
lig
h
tweig
h
t
co
n
v
o
lu
tio
n
al
n
etwo
r
k
ca
n
ac
h
iev
e
clin
ically
r
elev
an
t p
e
r
f
o
r
m
an
ce
with
v
e
r
y
lim
ited
co
m
p
u
tin
g
r
eso
u
r
ce
s
.
2.
M
E
T
H
O
D
2
.1
.
Da
t
a
s
et
a
nd
p
re
pro
ce
s
s
i
ng
T
h
e
PC
am
d
ataset
co
n
tain
s
3
2
7
,
6
8
0
p
atch
es
o
f
h
is
to
lo
g
ical
im
ag
es
(
9
6
×9
6
p
ix
els)
an
n
o
t
ated
b
in
ar
y
(
0
:
h
ea
lth
y
tis
s
u
e,
1
:
m
etastatic
tis
s
u
e)
.
Fo
r
th
is
s
tu
d
y
,
we
s
tr
ateg
ically
s
elec
ted
4
,
0
9
6
s
am
p
les
f
r
o
m
th
e
tr
ain
in
g
s
et
(
2
6
2
,
1
4
4
av
aila
b
le)
an
d
1
,
0
2
4
f
r
o
m
th
e
v
a
lid
atio
n
s
et
(
3
2
,
7
6
8
a
v
ailab
l
e)
,
r
e
p
r
esen
t
in
g
a
co
m
p
r
o
m
is
e
b
etwe
en
r
ep
r
esen
tativ
en
ess
an
d
co
m
p
u
tatio
n
al
co
n
s
tr
ain
ts
.
T
h
is
s
elec
t
io
n
m
ain
tain
s
th
e
o
r
ig
in
al
5
0
/5
0
d
is
tr
ib
u
tio
n
b
etwe
en
th
e
class
es.
A
s
am
p
le
s
ize
o
f
4
,
0
9
6
p
r
o
v
id
es
a
m
ar
g
i
n
o
f
er
r
o
r
o
f
±
1
.
5
%
at
th
e
9
5
%
co
n
f
i
d
en
ce
lev
el
f
o
r
a
b
in
ar
y
cla
s
s
if
icatio
n
.
T
h
is
v
alu
e
is
o
b
tain
ed
u
s
in
g
th
e
s
tan
d
a
r
d
f
o
r
m
u
la
f
o
r
t
h
e
m
ar
g
in
o
f
er
r
o
r
o
f
a
p
r
o
p
o
r
tio
n
.
=
1
.
96
×
√
(
1
−
)
=
0
.
5
,
=
4096
T
h
e
m
ar
g
in
ca
lcu
latio
n
g
i
v
e
s
1
.
5
%
T
h
u
s
,
th
e
ch
o
s
en
s
iz
e
r
ep
r
esen
ts
a
r
ea
s
o
n
ab
le
co
m
p
r
o
m
is
e
b
etw
ee
n
s
tatis
t
ical
r
ep
r
esen
tativ
en
ess
an
d
co
m
p
u
tatio
n
al
co
n
s
tr
ain
t
s
.
T
h
e
im
ag
es
wer
e
n
o
r
m
alize
d
b
y
d
iv
id
i
n
g
th
e
p
ix
el
v
alu
es b
y
2
5
5
to
o
b
tain
a
r
an
g
e
[
0
,
1
]
.
2
.2
.
M
o
del
a
rc
hite
ct
ure
wit
h a
dv
a
nced
re
g
ula
t
io
n
Ou
r
C
NN
ar
ch
itectu
r
e,
illu
s
tr
ated
in
Fig
u
r
e
1
,
is
s
p
ec
if
ically
d
esig
n
ed
f
o
r
9
6
×
96
-
p
ix
el
h
i
s
to
lo
g
ical
im
ag
es,
b
alan
cin
g
d
is
cr
im
in
ato
r
y
p
o
wer
with
o
v
er
f
itti
n
g
p
r
e
v
en
tio
n
.
I
t
c
o
m
p
r
is
es
th
r
ee
s
eq
u
e
n
tial
co
n
v
o
l
u
tio
n
al
b
lo
c
k
s
with
p
r
o
g
r
ess
iv
ely
in
cr
ea
s
in
g
f
ilter
co
u
n
ts
(
3
2
,
6
4
,
1
2
8
)
,
ea
c
h
f
o
llo
wed
b
y
b
atch
n
o
r
m
aliza
tio
n
an
d
n
eu
r
o
n
al
d
r
o
p
o
u
t.
T
h
e
d
r
o
p
o
u
t
r
ate
g
r
a
d
u
ally
in
cr
ea
s
es
(
0
.
2
,
0
.
3
,
0
.
4
)
in
d
ee
p
er
la
y
er
s
wh
er
e
o
v
er
f
itti
n
g
is
m
o
r
e
lik
ely
.
Glo
b
al
m
ea
n
p
o
o
lin
g
r
e
p
lace
s
tr
ad
itio
n
al
f
latten
in
g
,
d
r
asti
ca
lly
r
ed
u
ci
n
g
p
ar
am
eter
s
wh
ile
p
r
eser
v
in
g
s
p
atial
in
f
o
r
m
atio
n
.
T
h
is
h
ier
ar
ch
ical
d
esig
n
en
ab
les
r
o
b
u
s
t
ex
tr
ac
tio
n
o
f
f
ea
tu
r
es r
an
g
i
n
g
f
r
o
m
lo
w
-
le
v
el
tex
tu
r
es to
co
m
p
lex
p
atter
n
s
(
tis
s
u
e
s
tr
u
ctu
r
es).
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
4
,
D
e
c
em
b
er
20
2
6
:
1
8
8
1
-
1
8
9
2
1884
Fig
u
r
e
1
.
Simp
lifie
d
C
NN
ar
ch
itectu
r
e
f
o
r
b
r
ea
s
t tu
m
o
r
class
if
icatio
n
2
.
3
.
Da
t
a
a
ug
m
ent
a
t
io
n
T
o
co
m
p
en
s
ate
f
o
r
th
e
lim
ited
s
ize
o
f
th
e
tr
ain
i
n
g
s
et,
we
im
p
lem
en
te
d
a
n
a
u
g
m
en
tatio
n
s
tr
ateg
y
s
p
ec
if
ic
to
h
is
to
lo
g
ical
im
ag
es
,
−
Or
ig
in
al
p
r
eser
v
atio
n
: p
r
eser
v
i
n
g
th
e
s
o
u
r
ce
im
ag
e.
−
Ho
r
izo
n
tal
f
lip
: a
tr
a
n
s
f
o
r
m
ati
o
n
th
at
p
r
eser
v
es th
e
d
iag
n
o
s
tic
ch
ar
ac
ter
is
tics
o
f
th
e
tis
s
u
es.
−
9
0
° r
o
tatio
n
: a
d
ap
tin
g
t
o
th
e
v
ar
iab
le
o
r
ien
tatio
n
o
f
h
is
to
lo
g
i
ca
l sectio
n
s
.
T
h
is
s
tr
ateg
y
tr
ip
les
th
e
s
ize
o
f
th
e
tr
ain
in
g
s
et
(
4
,
0
9
6
t
o
1
2
,
2
8
8
s
am
p
les)
wh
ile
p
r
eser
v
in
g
th
e
b
io
lo
g
ical
s
ig
n
if
ican
ce
o
f
th
e
im
ag
es.
2
.
4
.
T
ra
ini
ng
c
o
nfig
ura
t
io
n
T
h
e
tr
ain
in
g
was
s
p
ec
if
ically
o
p
tim
ized
f
o
r
C
PU
en
v
ir
o
n
m
e
n
ts
to
en
s
u
r
e
m
ax
im
u
m
ac
ce
s
s
ib
ilit
y
f
o
r
r
esear
ch
er
s
an
d
clin
ician
s
wit
h
lim
ited
co
m
p
u
tin
g
r
eso
u
r
c
e
s
.
T
h
e
Ad
am
o
p
tim
izer
[
15]
was
s
e
lecte
d
with
a
lear
n
in
g
r
ate
o
f
0
.
0
0
0
1
,
e
n
ab
li
n
g
f
i
n
e
an
d
s
t
ab
le
co
n
v
er
g
en
c
e.
T
h
e
l
o
s
s
f
u
n
ctio
n
u
s
ed
is
b
i
n
ar
y
c
r
o
s
s
-
en
tr
o
p
y
,
ap
p
r
o
p
r
iate
f
o
r
b
in
ar
y
class
if
icatio
n
.
T
h
e
b
atch
s
ize
was
s
et
at
3
2
,
p
r
o
v
id
in
g
an
o
p
tim
al
b
alan
ce
b
etwe
en
ef
f
icien
t
m
em
o
r
y
u
s
ag
e
an
d
tr
ain
in
g
s
tab
ilit
y
.
T
h
e
m
a
x
im
u
m
n
u
m
b
e
r
o
f
e
p
o
ch
s
was
lim
i
ted
to
1
0
0
,
with
th
e
im
p
lem
en
tatio
n
o
f
ea
r
ly
s
to
p
p
in
g
th
at
m
o
n
ito
r
s
th
e
v
alid
at
io
n
AUC
[
16]
with
a
p
atien
c
e
o
f
1
5
ep
o
c
h
s
an
d
r
esto
r
es
th
e
b
est
weig
h
ts
,
th
u
s
p
r
ev
en
tin
g
o
v
er
f
itti
n
g
.
T
h
e
m
etr
ics
ev
alu
ated
in
clu
d
e
ac
c
u
r
ac
y
[
17]
,
AUC,
p
r
ec
is
io
n
,
r
ec
all
[
1
8
]
,
an
d
a
cu
s
to
m
F1
-
s
co
r
e
[
1
7
]
.
T
h
e
a
d
v
an
ce
d
co
n
f
ig
u
r
atio
n
o
f
T
e
n
s
o
r
Flo
w
was
ex
p
lo
ited
with
1
6
th
r
ea
d
s
f
o
r
in
tr
a
-
o
p
er
atio
n
p
ar
allelis
m
,
8
th
r
ea
d
s
f
o
r
i
n
ter
-
o
p
er
atio
n
p
ar
allelis
m
,
f
lex
ib
le
o
p
e
r
atio
n
p
lace
m
en
t
(
s
o
f
t
d
ev
ice
p
lace
m
en
t)
en
a
b
led
,
an
d
ju
s
t
-
in
-
tim
e
(
J
I
T
)
c
o
m
p
ilatio
n
to
ac
ce
ler
ate
th
e
ex
ec
u
tio
n
o
f
co
m
p
u
tatio
n
g
r
ap
h
s
.
2
.
5
.
E
v
a
lua
t
i
o
n a
nd
v
is
ua
liza
t
io
n
T
o
p
r
o
v
id
e
a
co
m
p
r
eh
e
n
s
iv
e
an
d
r
ep
r
o
d
u
cib
le
an
aly
s
is
o
f
th
e
m
o
d
el
’
s
p
e
r
f
o
r
m
an
ce
,
we
d
esig
n
ed
an
d
im
p
lem
en
ted
a
co
m
p
lete
ev
alu
atio
n
p
ip
elin
e
th
at
au
to
m
atica
lly
g
en
er
ates
a
s
er
ies
o
f
v
is
u
aliza
tio
n
s
an
d
s
cien
tific
r
ep
o
r
ts
.
T
h
is
p
ip
elin
e
p
r
o
d
u
ce
s
r
ec
eiv
er
o
p
er
atin
g
ch
ar
ac
ter
is
tic
(
R
OC
)
cu
r
v
es
[
19]
to
ass
es
s
th
e
m
o
d
el
’
s
d
is
cr
im
in
ato
r
y
ab
ilit
y
ac
r
o
s
s
all
d
ec
is
io
n
th
r
esh
o
ld
s
.
C
o
n
f
u
s
io
n
m
atr
ices
ar
e
g
e
n
e
r
ated
to
p
r
o
v
id
e
a
d
etailed
an
d
q
u
a
n
titativ
e
an
al
y
s
is
o
f
th
e
e
r
r
o
r
d
is
tr
ib
u
tio
n
(
tr
u
e/f
alse
p
o
s
itiv
es/n
eg
ativ
es).
Acc
u
r
ac
y
-
r
ec
all
cu
r
v
es,
p
a
r
ticu
lar
ly
r
elev
an
t
f
o
r
p
o
ten
tially
u
n
b
alan
ce
d
d
atasets
s
u
ch
as
th
o
s
e
u
s
ed
in
m
ed
ical
im
ag
in
g
,
ar
e
also
ca
lcu
lated
.
C
o
n
v
er
g
e
n
ce
is
m
o
n
ito
r
e
d
b
y
g
en
er
atin
g
le
ar
n
in
g
cu
r
v
es
th
at
illu
s
tr
ate
th
e
ev
o
l
u
tio
n
o
f
k
ey
m
etr
ics
ac
r
o
s
s
th
e
tr
ain
in
g
an
d
v
alid
atio
n
d
atasets
.
Fo
r
q
u
a
litativ
e
in
ter
p
r
etatio
n
,
v
is
u
aliza
tio
n
s
o
f
ex
am
p
le
p
r
ed
ictio
n
s
(
C
o
r
r
ec
t a
n
d
in
c
o
r
r
ec
t
)
alo
n
g
with
th
eir
co
n
f
id
en
ce
s
co
r
es a
r
e
p
r
o
d
u
ce
d
.
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
3
.
1
.
O
v
er
a
ll
p
er
f
o
r
m
a
nce
T
r
ain
in
g
co
n
v
er
g
ed
af
te
r
4
2
ep
o
ch
s
with
ea
r
ly
ter
m
in
at
io
n
tr
ig
g
er
e
d
b
y
s
tab
ilizatio
n
o
f
th
e
v
alid
atio
n
AUC
.
Fin
al
p
er
f
o
r
m
an
ce
ac
r
o
s
s
th
e
v
alid
atio
n
s
et
(
1
,
0
2
4
s
am
p
les)
is
s
u
m
m
ar
iz
ed
in
T
ab
le
1
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
C
P
U
-
o
p
timiz
ed
r
eg
u
la
r
iz
ed
co
n
vo
lu
tio
n
a
l n
eu
r
a
l n
etw
o
r
k
a
r
ch
itectu
r
e
fo
r
mu
ltimed
ia
…
(
Gu
id
o
u
m
A
min
a
)
1885
T
ab
le
1
.
Deta
iled
m
o
d
el
p
er
f
o
r
m
an
ce
M
e
t
r
i
c
v
a
l
u
e
i
n
t
e
r
p
r
e
t
a
t
i
o
n
A
U
C
0
.
9
2
9
2
e
x
c
e
l
l
e
n
t
d
i
s
c
r
i
m
i
n
a
t
i
o
n
A
c
c
u
r
a
c
y
0
.
8
8
7
4
hi
g
h
t
r
u
e
p
o
si
t
i
v
e
r
a
t
e
(
TP
R
)
R
e
c
a
l
l
0
.
7
6
8
9
m
o
d
e
r
a
t
e
s
e
n
si
t
i
v
i
t
y
F1
-
s
c
o
r
e
0
.
8
2
3
9
sa
t
i
s
f
a
c
t
o
r
y
b
a
l
a
n
c
e
3
.
2
.
Ana
ly
s
is
o
f
lea
rning
cur
v
es
Fig
u
r
e
2
s
h
o
ws
th
e
d
is
tr
ib
u
tio
n
o
f
c
o
n
f
id
e
n
ce
s
co
r
es:
X
-
a
x
is
:
c
o
n
f
id
en
c
e
s
co
r
e
(
co
n
tin
u
o
u
s
\
v
alu
e
n
o
r
m
alize
d
b
etwe
en
0
.
0
an
d
1
.
0
)
a
n
d
Y
-
a
x
is
:
f
r
eq
u
e
n
cy
(
n
u
m
b
er
o
f
s
am
p
les,
u
n
it:
co
u
n
t)
.
T
h
is
h
is
to
g
r
a
m
s
h
o
ws
th
e
b
im
o
d
al
d
is
tr
ib
u
ti
o
n
o
f
co
n
f
id
en
ce
s
co
r
es
p
r
ed
icted
b
y
t
h
e
m
o
d
el
f
o
r
t
h
e
v
alid
atio
n
s
et.
T
wo
d
is
tin
ct
d
is
tr
ib
u
tio
n
s
clea
r
ly
em
er
g
e:
o
n
e
ce
n
ter
e
d
ar
o
u
n
d
0
.
0
-
0
.
2
(
r
e
p
r
esen
tin
g
h
ea
lt
h
y
tis
s
u
e
s
am
p
les)
an
d
th
e
o
th
e
r
ce
n
ter
ed
ar
o
u
n
d
0
.
8
–
1
.
0
(
r
ep
r
esen
tin
g
t
u
m
o
r
tis
s
u
e
s
am
p
les).
T
h
e
clea
r
s
ep
ar
atio
n
b
etwe
en
th
ese
two
m
o
d
es,
with
a
p
r
o
n
o
u
n
ce
d
d
ip
ar
o
u
n
d
t
h
e
s
co
r
e
0
.
5
,
in
d
icate
s
th
at
th
e
m
o
d
el
p
r
o
d
u
ce
s
p
r
e
d
ictio
n
s
with
h
ig
h
co
n
f
id
en
ce
.
T
h
e
s
m
all
p
r
o
p
o
r
tio
n
o
f
s
am
p
les
in
th
e
in
ter
m
e
d
iate
r
eg
io
n
(
0
.
3
-
0
.
7
)
s
u
g
g
ests
th
at
th
e
m
o
d
el
ex
p
er
ien
ce
s
litt
le
u
n
ce
r
tain
ty
in
th
e
m
ajo
r
i
ty
o
f
ca
s
es.
T
h
e
d
is
tr
ib
u
tio
n
o
f
h
ea
lth
y
tis
s
u
es
s
h
o
ws
a
s
lig
h
tly
ex
ten
d
ed
tail
to
war
d
h
ig
h
er
s
co
r
es,
co
r
r
esp
o
n
d
i
n
g
to
f
alse
p
o
s
iti
v
es
id
en
tifie
d
in
th
e
co
n
f
o
u
n
d
in
g
m
atr
ix
.
T
h
e
m
ar
k
ed
b
im
o
d
ality
co
n
f
i
r
m
s
th
e
m
o
d
el
’
s
p
r
o
p
er
ca
lib
r
atio
n
an
d
its
ab
ilit
y
to
ass
ig
n
p
r
o
b
a
b
ilit
y
s
co
r
es
th
at
ac
cu
r
a
tely
r
ef
lect
th
e
ce
r
tain
ty
o
f
t
h
e
p
r
ed
ictio
n
s
.
T
h
is
ch
ar
ac
ter
i
s
tic
is
e
s
s
en
tial
f
o
r
clin
ical
ap
p
licatio
n
s
wh
er
e
d
i
f
f
er
en
t
d
ec
is
io
n
th
r
esh
o
l
d
s
ca
n
b
e
a
p
p
lied
d
ep
en
d
in
g
o
n
t
h
e
d
esire
d
tr
ad
e
-
o
f
f
b
etwe
en
s
en
s
itiv
ity
an
d
s
p
ec
if
icity
.
Fig
u
r
e
3
s
h
o
ws
th
e
ev
o
lu
tio
n
o
f
th
e
m
etr
ics
d
u
r
in
g
tr
ai
n
in
g
.
X
-
a
x
is
:
e
p
o
ch
(
u
n
it:
n
u
m
b
er
o
f
co
m
p
lete
iter
atio
n
s
o
n
th
e
tr
ai
n
in
g
s
et)
an
d
Y
-
a
x
is
:
AUC
(
n
o
r
m
alize
d
v
alu
e
b
etwe
en
0
.
5
an
d
1
.
0
)
.
T
h
is
f
ig
u
r
e
illu
s
tr
ates
th
e
ev
o
lu
tio
n
o
f
th
e
m
o
d
el
’
s
d
is
cr
im
in
at
o
r
y
p
e
r
f
o
r
m
an
ce
d
u
r
in
g
tr
ain
i
n
g
o
v
e
r
4
2
e
p
o
ch
s
.
T
h
e
b
lu
e
cu
r
v
e
(
T
r
ain
AU
C
)
r
ep
r
esen
ts
th
e
AUC
ca
lcu
lated
o
n
th
e
tr
ain
in
g
s
et
(
1
2
,
2
8
8
s
am
p
les
af
ter
au
g
m
e
n
tatio
n
)
,
w
h
ile
th
e
o
r
an
g
e
cu
r
v
e
(
Valid
atio
n
AUC)
s
h
o
ws
th
e
AUC
o
n
th
e
v
alid
atio
n
s
et
(
1
,
0
2
4
s
am
p
les).
T
h
e
tr
ain
in
g
AUC
in
cr
ea
s
e
s
s
tead
ily
f
r
o
m
8
2
.
3
3
%
to
9
5
.
8
2
%,
d
em
o
n
s
t
r
atin
g
th
e
m
o
d
el
’
s
ab
ilit
y
to
p
r
o
g
r
ess
iv
ely
lear
n
d
is
cr
im
i
n
ato
r
y
f
ea
tu
r
es.
T
h
e
v
alid
atio
n
AUC,
af
ter
a
r
a
p
id
in
itial
in
cr
ea
s
e,
s
tab
ilizes
ar
o
u
n
d
9
2
.
9
2
%
af
t
er
ep
o
c
h
2
7
.
T
h
e
m
o
d
e
r
ate
b
u
t
co
n
s
is
ten
t
d
if
f
er
en
ce
b
etw
ee
n
th
e
two
cu
r
v
es
(
ap
p
r
o
x
im
ately
0
.
0
2
9
o
n
av
er
ag
e)
in
d
icate
s
ef
f
ec
tiv
e
o
v
er
f
itti
n
g
co
n
t
r
o
l
th
an
k
s
to
th
e
r
eg
u
lar
izatio
n
tech
n
iq
u
es
em
p
lo
y
ed
(
B
atch
n
o
r
m
aliza
tio
n
an
d
g
r
a
d
u
al
d
r
o
p
o
u
t)
.
T
h
e
s
tab
ilizatio
n
o
f
t
h
e
v
alid
atio
n
AUC
af
ter
ep
o
c
h
2
7
ju
s
tifie
s
th
e
e
ar
ly
ter
m
in
atio
n
ap
p
lied
,
co
n
f
ir
m
in
g
th
at
th
e
m
o
d
el
h
as
r
ea
ch
ed
its
o
p
tim
al
g
en
er
aliza
tio
n
ca
p
ac
ity
.
T
h
e
f
in
al
AUC
v
alu
e
o
f
9
2
.
9
2
%
in
d
icate
s
th
at
th
e
m
o
d
el
h
as
an
ex
ce
llen
t
ab
ilit
y
to
d
is
tin
g
u
is
h
h
ea
lth
y
tis
s
u
e
f
r
o
m
tu
m
o
r
tis
s
u
e,
with
o
n
ly
7
.
0
8
%
o
v
er
lap
b
etwe
en
th
e
d
is
tr
ib
u
tio
n
s
o
f
th
e
two
class
es.
T
h
is
p
er
f
o
r
m
an
ce
f
alls
with
in
th
e
r
an
g
e
co
n
s
id
er
ed
clin
icall
y
r
ele
v
an
t
f
o
r
d
iag
n
o
s
tic
aid
ap
p
licatio
n
s
.
Fig
u
r
e
2
.
Dis
tr
ib
u
tio
n
o
f
c
o
n
f
i
d
en
ce
s
co
r
es
l
ea
r
n
in
g
cu
r
v
es
Fig
u
r
e
3
.
AUC lea
r
n
in
g
cu
r
v
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
4
,
D
e
c
em
b
er
20
2
6
:
1
8
8
1
-
1
8
9
2
1886
3.
3
.
Co
nfusi
o
n m
a
t
rix
a
na
l
y
s
is
T
h
e
co
n
f
u
s
io
n
m
atr
ix
(
Fig
u
r
e
4
)
,
with
p
r
ed
icted
class
es
(
“
s
ain
”
:
Hea
lth
y
an
d
“T
u
m
o
r
a
l”)
o
n
th
e
x
-
ax
is
an
d
ac
tu
al
class
es
(
“
s
ain
o
r
Hea
lth
y
”
a
n
d
“T
u
m
o
r
al”)
o
n
th
e
y
-
ax
is
,
r
ev
ea
ls
th
e
f
o
llo
win
g
d
is
tr
ib
u
tio
n
o
f
p
r
e
d
ictio
n
s
ac
r
o
s
s
th
e
v
a
lid
atio
n
s
et:
tr
u
e
p
o
s
itiv
es:
4
7
3
(
4
6
.
2
%);
f
alse
p
o
s
itiv
es:
4
9
(
4
.
8
%);
f
alse
n
eg
ativ
es:
5
0
(
4
.
9
%);
tr
u
e
n
eg
ativ
es:
4
5
2
(
4
4
.
1
%).
T
h
e
m
o
d
el
ex
h
ib
its
a
s
p
ec
if
icity
o
f
9
0
.
2
%
(
4
5
2
/(
4
5
2
+
4
9
)
)
an
d
a
s
en
s
itiv
ity
o
f
9
0
.
4
%
(
4
7
3
/(
4
7
3
+5
0
)
)
.
T
h
e
b
alan
ce
b
e
twee
n
f
alse
p
o
s
itiv
es
(
4
.
8
%)
an
d
f
alse
n
eg
ativ
es
(
4
.
9
%)
s
u
g
g
ests
an
ap
p
r
o
p
r
iat
e
ca
lib
r
atio
n
o
f
th
e
d
ec
is
io
n
t
h
r
esh
o
ld
(
0
.
5
)
.
Fig
u
r
e
4
.
C
o
n
f
u
s
io
n
m
atr
i
x
3
.
4
.
RO
C
c
urv
e
a
nd
dis
cr
im
ina
nt
a
na
ly
s
is
Fig
u
r
e
5
p
r
esen
ts
th
e
R
OC
cu
r
v
e,
ev
alu
atin
g
th
e
m
o
d
el
’
s
d
is
cr
im
in
ato
r
y
ab
ilit
y
ac
r
o
s
s
th
e
en
tire
co
n
tin
u
u
m
o
f
d
ec
is
io
n
th
r
esh
o
ld
s
.
T
h
e
x
-
ax
is
r
ep
r
esen
ts
th
e
f
alse
p
o
s
itiv
e
r
ate
(
FP
R
)
,
co
r
r
esp
o
n
d
in
g
to
th
e
p
r
o
p
o
r
tio
n
o
f
h
ea
lth
y
tis
s
u
e
in
co
r
r
ec
tly
class
if
ied
as
tu
m
o
r
o
u
s
,
wh
ile
th
e
y
-
ax
is
r
e
p
r
ese
n
ts
th
e
tr
u
e
p
o
s
itiv
e
r
ate
(
T
PR
)
,
in
d
icatin
g
th
e
p
r
o
p
o
r
tio
n
o
f
co
r
r
ec
tly
d
etec
ted
t
u
m
o
r
o
u
s
tis
s
u
e.
T
h
e
AUC
o
f
0
.
9
2
9
3
d
e
m
o
n
s
tr
ates
ex
ce
llen
t
d
is
cr
im
in
ato
r
y
p
e
r
f
o
r
m
an
ce
,
p
lacin
g
it
in
th
e
u
p
p
er
q
u
ar
tile
o
f
class
if
ier
s
ac
co
r
d
in
g
to
estab
lis
h
ed
cr
iter
ia
(
AUC >
0
.
9
)
.
T
h
e
cu
r
v
e
ex
h
i
b
its
a
p
r
o
n
o
u
n
ce
d
co
n
ca
v
ity
to
war
d
s
t
h
e
i
d
e
al
p
o
in
t
(
0
.
0
,
1
.
0
)
,
with
a
r
ap
i
d
in
cr
ea
s
e
in
th
e
T
PR
f
o
r
FP
R
v
alu
es
b
elo
w
0
.
2
.
T
h
is
ch
ar
ac
ter
is
tic
is
p
ar
ticu
lar
ly
r
elev
a
n
t
f
o
r
m
ed
ical
ap
p
licatio
n
s
wh
er
e
m
in
im
izin
g
f
alse
p
o
s
itiv
es
is
cr
u
cial.
T
h
e
m
o
d
el
ac
h
iev
es
a
T
PR
o
f
0
.
8
5
f
o
r
an
FP
R
o
f
o
n
l
y
0
.
1
,
in
d
icatin
g
its
ab
ilit
y
to
m
a
in
tain
h
ig
h
s
en
s
itiv
ity
wh
ile
lim
itin
g
t
y
p
e
I
er
r
o
r
s
.
T
h
e
o
p
er
atin
g
p
o
in
t
co
r
r
esp
o
n
d
in
g
t
o
th
e
0
.
5
th
r
esh
o
ld
(
u
s
ed
d
u
r
in
g
th
e
ev
al
u
atio
n
)
is
lo
ca
ted
at
(
FP
R
=0
.
0
9
8
,
T
PR
=0
.
9
0
4
)
,
eq
u
iv
alen
t to
a
s
p
ec
if
icity
o
f
9
0
.
2
% a
n
d
a
s
en
s
itiv
ity
o
f
9
0
.
4
%.
T
h
e
lar
g
e
ar
ea
s
ep
ar
atin
g
t
h
e
m
o
d
el
cu
r
v
e
f
r
o
m
t
h
e
r
an
d
o
m
r
ef
er
en
ce
lin
e
(
AUC=0
.
5
)
q
u
an
tifie
s
th
e
s
u
b
s
tan
tial
im
p
r
o
v
em
e
n
t
p
r
o
v
id
ed
b
y
o
u
r
a
p
p
r
o
ac
h
.
T
h
e
s
h
ap
e
o
f
th
e
cu
r
v
e
also
s
u
g
g
e
s
ts
p
o
s
s
ib
ilit
ie
s
f
o
r
ad
ap
t
iv
e
o
p
tim
izatio
n
o
f
t
h
e
d
ec
is
io
n
th
r
esh
o
ld
ac
c
o
r
d
i
n
g
t
o
s
p
ec
if
ic
clin
ical
p
r
ef
er
en
ce
s
:
s
h
if
tin
g
to
h
ig
h
er
th
r
esh
o
ld
s
co
u
ld
in
c
r
ea
s
e
s
p
ec
if
icity
u
p
to
9
5
%wh
ile
m
ain
tain
in
g
ac
ce
p
tab
le
s
en
s
itiv
ity
f
o
r
d
ia
g
n
o
s
tic
co
n
f
ir
m
atio
n
a
p
p
licatio
n
s
,
wh
er
ea
s
a
lo
wer
th
r
e
s
h
o
ld
w
o
u
ld
allo
w
f
o
r
s
en
s
itiv
ities
g
r
ea
ter
th
an
9
5
%
f
o
r
h
ig
h
-
r
is
k
s
cr
ee
n
in
g
s
ettin
g
s
.
T
h
is
f
lex
ib
ilit
y
d
em
o
n
s
tr
ates
th
e
m
o
d
el
’
s
p
o
ten
tial
ad
ap
tab
ilit
y
t
o
d
if
f
er
en
t
clin
ical
s
ce
n
ar
io
s
.
Fig
u
r
e
6
p
r
esen
ts
th
e
ac
cu
r
ac
y
-
r
ec
all
cu
r
v
e,
ess
en
tial
f
o
r
ev
alu
atin
g
p
er
f
o
r
m
an
ce
o
n
p
o
ten
tially
u
n
b
alan
ce
d
d
ata.
T
h
e
av
er
a
g
e
p
r
ec
is
io
n
(
AP)
o
f
0
.
9
2
4
7
d
em
o
n
s
tr
ates
th
e
m
o
d
el
’
s
ex
ce
llen
t
ab
ilit
y
to
m
ain
tain
h
ig
h
ac
c
u
r
ac
y
ac
r
o
s
s
all
r
ec
all
lev
els.
T
h
is
h
ig
h
v
al
u
e
in
d
icate
s
th
at
th
e
m
o
d
el
s
u
cc
ess
f
u
lly
d
etec
ts
a
lar
g
e
p
r
o
p
o
r
tio
n
o
f
tu
m
o
r
tis
s
u
e
wh
ile
m
in
im
izin
g
f
alse p
o
s
itiv
es.
T
h
e
cu
r
v
e
s
h
o
ws
th
at
f
o
r
a
r
ec
all
o
f
0
.
8
(
d
etec
tio
n
o
f
8
0
%
o
f
tu
m
o
r
ca
s
es),
th
e
ac
cu
r
ac
y
r
em
ain
s
ab
o
v
e
0
.
8
5
.
T
h
is
ch
ar
ac
ter
is
t
ic
is
p
ar
ticu
lar
ly
im
p
o
r
ta
n
t
i
n
th
e
clin
ical
s
ettin
g
,
wh
er
e
m
in
im
izin
g
f
alse
pos
itiv
es
is
cr
u
cial
to
av
o
i
d
u
n
n
ec
ess
ar
y
tr
ea
tm
en
ts
a
n
d
p
a
tien
t
an
x
iety
.
C
o
n
v
er
s
ely
,
at
an
ac
cu
r
a
cy
o
f
0
.
9
(
o
n
ly
1
0
%
f
alse
p
o
s
itiv
es),
t
h
e
m
o
d
el
ac
h
iev
es
a
r
ec
all
o
f
ap
p
r
o
x
im
ately
0
.
7
5
,
in
d
ic
atin
g
th
at
it
wo
u
ld
co
r
r
ec
tly
id
e
n
tify
7
5
% o
f
tu
m
o
r
ca
s
es wh
ile
s
tr
ictly
lim
itin
g
ty
p
e
I
e
r
r
o
r
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
C
P
U
-
o
p
timiz
ed
r
eg
u
la
r
iz
ed
co
n
vo
lu
tio
n
a
l n
eu
r
a
l n
etw
o
r
k
a
r
ch
itectu
r
e
fo
r
mu
ltimed
ia
…
(
Gu
id
o
u
m
A
min
a
)
1887
Fig
u
r
e
5
.
R
o
c
cu
r
v
e
Fig
u
r
e
6
.
Acc
u
r
ac
y
-
r
ec
all
cu
r
v
e
T
h
e
s
h
ap
e
o
f
th
e
cu
r
v
e,
wh
ic
h
r
em
ai
n
s
clo
s
e
to
th
e
u
p
p
er
r
ig
h
t
co
r
n
er
o
v
er
a
wid
e
r
an
g
e,
s
u
g
g
ests
th
at
th
e
m
o
d
el
o
f
f
er
s
f
lex
ib
ilit
y
in
ch
o
o
s
in
g
th
e
d
ec
is
io
n
th
r
esh
o
ld
ac
co
r
d
in
g
to
clin
ical
p
r
io
r
ities
.
Fo
r
lar
g
e
-
s
ca
le
s
cr
ee
n
in
g
wh
e
r
e
s
en
s
itiv
ity
is
p
ar
a
m
o
u
n
t,
a
lo
wer
th
r
esh
o
ld
c
o
u
ld
b
e
ch
o
s
en
t
o
m
ax
im
ize
r
ec
all.
C
o
n
v
er
s
ely
,
f
o
r
d
iag
n
o
s
tic
co
n
f
ir
m
atio
n
wh
er
e
s
p
ec
if
icity
is
cr
itical,
a
h
ig
h
er
th
r
esh
o
l
d
wo
u
ld
p
r
io
r
itize
ac
cu
r
ac
y
.
T
h
is
an
aly
s
is
co
n
f
ir
m
s
th
at
o
u
r
m
o
d
el
ac
h
iev
es
n
o
t
o
n
ly
h
ig
h
d
is
cr
im
in
at
o
r
y
p
er
f
o
r
m
a
n
ce
(
AUC
0
.
9
2
9
3
)
b
u
t
also
an
e
x
ce
llen
t
b
alan
ce
b
etwe
en
ac
c
u
r
ac
y
an
d
r
ec
all,
two
f
u
n
d
a
m
en
tal
m
etr
ics
f
o
r
m
ed
ical
ap
p
licatio
n
s
wh
er
e
th
e
co
n
s
eq
u
en
ce
s
o
f
f
al
s
e
p
o
s
itiv
es
an
d
f
alse
n
e
g
ativ
es
m
u
s
t
b
e
ca
r
ef
u
lly
weig
h
ed
.
3
.
5
.
Q
ua
lit
a
t
iv
e
v
is
ua
liza
t
io
n
Fig
u
r
e
7
p
r
esen
ts
a
q
u
alitativ
e
an
aly
s
is
o
f
ten
m
o
d
el
p
r
ed
ictio
n
s
o
n
th
e
v
alid
atio
n
s
et.
E
ac
h
h
is
to
lo
g
ical
p
atch
is
ac
co
m
p
an
ied
b
y
th
e
g
r
o
u
n
d
tr
u
th
(
Ver
it),
t
h
e
m
o
d
el
p
r
ed
icti
o
n
(
Pre
d
)
a
n
d
th
e
co
r
r
esp
o
n
d
in
g
co
n
f
id
en
ce
s
co
r
e
(
C
o
n
f
)
,
with
an
im
m
ed
iate
v
is
u
al
in
d
icatio
n
o
f
th
e
ac
cu
r
a
cy
o
f
th
e
p
r
ed
ictio
n
(
g
r
ee
n
f
o
r
c
o
r
r
ec
t,
r
ed
f
o
r
in
c
o
r
r
ec
t)
.
Fig
u
r
e
7
.
Sam
p
le
p
r
ed
ictio
n
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
4
,
D
e
c
em
b
er
20
2
6
:
1
8
8
1
-
1
8
9
2
1888
Of
th
e
ten
ex
am
p
les
p
r
esen
te
d
,
n
in
e
p
r
ed
ictio
n
s
wer
e
c
o
r
r
ec
t
an
d
o
n
e
was
in
co
r
r
ec
t.
T
h
e
co
r
r
ec
t
p
r
ed
ictio
n
s
in
clu
d
ed
f
o
u
r
h
ea
lt
h
y
p
atch
es (
T
r
u
th
=
0
)
id
e
n
tifie
d
with
v
er
y
lo
w
c
o
n
f
id
e
n
ce
s
co
r
es
(
0
.
0
8
9
,
0
.
0
4
8
,
0
.
1
2
2
)
an
d
f
iv
e
tu
m
o
r
p
atc
h
es
(
T
r
u
th
=
1
)
d
etec
ted
with
s
co
r
es
r
an
g
in
g
f
r
o
m
v
er
y
lo
w
(
0
.
0
1
9
,
0
.
0
8
0
)
to
v
er
y
h
ig
h
(
0
.
8
1
0
,
1
.
0
0
0
)
,
with
in
te
r
m
ed
iate
v
alu
es
(
0
.
3
5
6
,
0
.
6
8
6
)
in
b
etwe
en
.
T
h
is
v
ar
iatio
n
i
n
co
n
f
id
e
n
ce
s
co
r
es
f
o
r
t
h
e
c
o
r
r
ec
t
tu
m
o
r
class
p
r
ed
ictio
n
s
s
u
g
g
ests
th
at
th
e
m
o
d
el
s
u
cc
ess
f
u
lly
id
e
n
tifie
s
ca
s
es
o
f
v
a
r
y
i
ng
d
if
f
icu
lty
,
f
r
o
m
o
b
v
i
o
u
s
to
s
u
b
tle.
T
h
e
o
n
ly
o
b
s
er
v
e
d
er
r
o
r
was
a
f
alse
n
eg
ativ
e
(
T
r
u
t
h
=1
,
Pre
d
=0
)
with
a
co
n
f
id
en
ce
s
co
r
e
o
f
0
.
3
7
9
.
T
h
is
v
alu
e,
lo
ca
ted
ju
s
t
b
elo
w
th
e
d
ec
is
io
n
th
r
esh
o
ld
o
f
0
.
5
,
in
d
icate
s
th
at
th
e
m
o
d
el
ex
p
er
ien
ce
s
leg
itima
te
u
n
ce
r
tain
t
y
in
t
h
is
ca
s
e.
Vis
u
ally
,
th
is
p
atc
h
c
o
u
ld
p
r
ese
n
t
am
b
ig
u
o
u
s
m
o
r
p
h
o
lo
g
ical
ch
ar
ac
ter
is
tics
o
r
s
im
ilar
ities
to
h
ea
lth
y
tis
s
u
e,
ex
p
lain
in
g
th
e
d
if
f
icu
lty
in
class
if
icatio
n
.
T
h
is
q
u
alitativ
e
an
aly
s
is
r
ev
ea
ls
s
ev
er
al
im
p
o
r
ta
n
t
p
o
in
ts
:
T
h
e
co
r
r
elatio
n
b
etwe
en
c
o
n
f
id
en
ce
s
co
r
es
an
d
p
r
ed
ictio
n
ac
cu
r
ac
y
s
u
g
g
ests
g
o
o
d
m
o
d
el
ca
lib
r
atio
n
.
Natu
r
e
o
f
er
r
o
r
s
:
t
h
e
ab
s
en
ce
o
f
f
alse
p
o
s
itiv
es
in
th
is
l
im
ited
s
am
p
le
i
s
co
n
s
is
ten
t
with
th
e
h
ig
h
p
r
ec
is
io
n
(
0
.
8
8
7
4
)
o
b
s
er
v
ed
q
u
a
n
titativ
ely
.
T
h
e
m
o
d
el
’
s
ab
ilit
y
to
ass
ig
n
co
n
f
id
e
n
ce
s
co
r
es
r
ef
lectin
g
ca
s
e
co
m
p
lex
ity
co
u
ld
b
e
u
s
ed
f
o
r
h
ier
ar
ch
ical
tr
iag
e,
w
h
er
e
ca
s
es
with
in
ter
m
ed
iate
s
co
r
es
(
ar
o
u
n
d
0
.
5
)
wo
u
l
d
b
e
au
t
o
m
atica
lly
r
ev
iewe
d
b
y
a
p
ath
o
lo
g
is
t,
th
u
s
o
p
tim
izin
g
th
e
ef
f
icien
cy
o
f
th
e
d
iag
n
o
s
tic
p
r
o
ce
s
s
.
T
h
e
1
0
%
er
r
o
r
r
ate
in
th
is
r
an
d
o
m
s
am
p
le
is
r
ep
r
esen
ta
tiv
e
o
f
th
e
o
v
er
all
ex
p
ec
ted
er
r
o
r
r
ate
(
9
.
8
%
f
o
r
tu
m
o
r
tis
s
u
es).
T
h
i
s
co
n
s
is
ten
cy
b
etwe
en
th
e
q
u
alitativ
e
an
d
q
u
a
n
titativ
e
an
al
y
s
es
s
tr
en
g
th
en
s
th
e
v
alid
ity
o
f
o
u
r
ass
ess
m
en
t a
n
d
th
e
r
eliab
ilit
y
o
f
t
h
e
m
o
d
el
f
o
r
p
r
ac
tical
ap
p
licatio
n
s
.
3
.
6
.
Co
m
pa
riso
n wit
h t
he
s
t
a
t
e
o
f
t
he
a
rt
T
h
e
co
m
p
ar
ativ
e
a
n
aly
s
is
p
r
esen
ted
in
T
ab
le
2
r
ev
ea
ls
p
ar
ticu
lar
ly
in
ter
esti
n
g
r
esu
lts
,
o
u
r
m
eth
o
d
o
u
tp
er
f
o
r
m
s
s
ev
er
al
r
ec
e
n
t
ap
p
r
o
ac
h
es
in
ter
m
s
o
f
d
is
cr
im
in
an
t
p
er
f
o
r
m
an
ce
.
I
ts
AUC
o
f
9
2
.
9
2
%
s
lig
h
tly
s
u
r
p
ass
es
th
at
o
f
C
NN
-
t
r
an
s
f
o
r
m
er
(
9
2
.
5
8
%)
p
u
b
lis
h
ed
in
2
0
2
5
,
a
n
d
its
ac
cu
r
ac
y
o
f
8
8
.
7
4
%
is
h
ig
h
er
th
a
n
th
at
o
f
E
f
f
icien
tN
etV2
-
S
(
8
7
.
3
%),
C
o
n
v
NeX
t
(
8
7
.
8
%),
a
n
d
C
o
r
o
Net
in
its
b
in
ar
y
class
if
icatio
n
co
n
f
ig
u
r
atio
n
(
8
6
.
6
7
%).
MR_
NE
T
r
ep
o
r
ts
b
etter
ac
cu
r
ac
y
o
f
7
3
%
o
n
th
e
B
R
AC
S
d
ata
s
et,
wh
ich
is
lo
wer
th
an
o
u
r
ac
cu
r
ac
y
,
b
u
t
d
ir
ec
t
co
m
p
ar
is
o
n
is
lim
ited
b
y
th
e
d
if
f
er
en
c
e
in
d
atasets
an
d
task
co
m
p
lex
ities
(
th
r
ee
class
es
v
er
s
u
s
o
n
e
b
in
ar
y
class
if
icatio
n
)
.
T
h
ese
r
es
u
lts
ar
e
all
t
h
e
m
o
r
e
r
e
m
ar
k
ab
le
g
iv
e
n
th
at
o
u
r
m
o
d
el
u
s
es
o
n
ly
4
,
0
9
6
s
am
p
les
(
au
g
m
e
n
ted
to
1
2
,
2
8
8
)
,
r
ep
r
esen
tin
g
1
.
5
6
%,
0
.
3
2
%,
an
d
8
.
1
9
%
o
f
th
e
d
ata
u
s
ed
b
y
t
h
e
co
m
p
ar
ed
m
eth
o
d
s
,
r
esp
ec
tiv
ely
.
T
h
e
p
a
r
a
m
etr
ic
ef
f
icien
cy
is
p
ar
ticu
la
r
ly
r
em
ar
k
a
b
le:
o
u
r
ar
ch
itectu
r
e
co
m
p
r
is
es
o
n
ly
9
4
,
0
0
0
p
ar
a
m
eter
s
,
co
m
p
a
r
ed
to
2
1
m
illi
o
n
f
o
r
E
f
f
icien
tNetV2
-
S,
3
5
m
illi
o
n
f
o
r
C
o
n
v
NeX
t,
2
5
m
illi
o
n
f
o
r
th
e
C
NN
-
t
r
an
s
f
o
r
m
er
,
an
d
3
3
.
9
7
m
illi
o
n
f
o
r
C
o
r
o
Net.
T
h
e
n
u
m
b
e
r
o
f
p
ar
am
eter
s
f
o
r
MR_
NE
T
is
n
o
t
r
e
p
o
r
ted
,
b
u
t
its
ac
c
u
r
ac
y
r
em
ai
n
s
lo
wer
th
an
o
u
r
s
.
T
h
i
s
co
m
p
u
tatio
n
al
lig
h
tn
ess
r
ep
r
esen
ts
a
s
ig
n
if
ic
an
t
ad
v
a
n
tag
e
f
o
r
d
e
p
lo
y
m
e
n
t
in
clin
ical
s
ettin
g
s
,
wh
er
e
r
eso
u
r
ce
s
ar
e
o
f
ten
lim
ited
.
T
ab
le
2
.
C
o
m
p
a
r
is
o
n
with
th
e
s
tate
o
f
th
e
ar
t
M
e
t
h
o
d
Y
ear
AUC
o
r
a
c
c
u
r
a
c
y
A
r
c
h
i
t
e
c
t
u
r
e
D
a
t
a
P
a
r
a
me
t
e
r
s
O
u
r
me
t
h
o
d
2
0
2
5
A
U
C
:
9
2
.
9
2
%
a
n
d
A
c
c
u
r
a
c
y
:
88
.
74
%
C
N
N
r
e
g
u
l
a
r
i
z
e
d
4
0
9
6
9
4
K
Ef
f
i
c
i
e
n
t
N
e
t
V
2
-
S
[
3]
2
0
2
1
A
c
c
u
r
a
c
y
:
8
7
.
3
%
Ef
f
i
c
i
e
n
t
N
e
t
V
2
2
6
2
1
4
4
2
1
M
C
o
n
v
N
e
x
t
[
4]
2
0
2
2
A
c
c
u
r
a
c
y
:
8
7
.
8
%
(
M
o
d
e
r
n
i
z
e
d
h
i
e
r
a
r
c
h
i
c
a
l
C
o
n
v
N
e
t
s,
i
n
s
p
i
r
e
d
b
y
t
h
e
p
r
i
n
c
i
p
l
e
s
o
f
t
r
a
n
sf
o
r
mers
)
1
.
2
8
M
3
5
M
C
N
N
-
t
r
a
n
sf
o
r
mer
[
9]
2
0
2
5
A
U
C
:
9
2
.
5
8
%
M
o
d
i
f
i
e
d
R
e
sN
e
t
5
0
+
Tr
a
n
sf
o
r
mer
e
n
c
o
d
e
r
(
m
u
l
t
i
-
h
e
a
d
a
t
t
e
n
t
i
o
n
,
p
o
s
i
t
i
o
n
a
l
c
o
d
i
n
g
)
+
p
e
r
-
c
h
a
n
n
e
l
a
t
t
e
n
t
i
o
n
m
o
d
u
l
e
.
5
0
0
0
0
2
5
M
c
o
r
o
N
e
t
[
7]
2
0
2
2
A
c
c
u
r
a
c
y
:
8
6
.
6
7
%
(
2
c
l
a
ss
e
s)
/
9
4
.
9
2
%
(
4
c
l
a
ss
e
s)
X
c
e
p
t
i
o
n
C
B
I
S
-
D
D
S
M
3
3
.
9
7
M
M
R
N
e
t
[
1
0
]
2
0
2
4
A
c
c
u
r
a
c
y
:
b
e
st
:
73%
(
M
o
b
i
l
e
N
e
t
V
G
G
1
9
)
M
o
b
i
l
e
N
e
t
,
V
G
G
1
9
,
M
R
-
N
e
t
B
R
A
C
S
N
o
t
r
e
p
o
r
t
e
d
3
.
7
.
Dis
cus
s
io
n
T
h
e
p
er
f
o
r
m
a
n
ce
ac
h
iev
e
d
(
AUC
9
2
.
9
2
%,
ac
cu
r
ac
y
8
8
.
7
4
%)
d
em
o
n
s
tr
ates
th
at
a
wel
l
-
r
eg
u
lated
C
NN
ar
ch
itectu
r
e
ca
n
ac
h
iev
e
clin
ically
r
elev
an
t
d
is
cr
im
in
atio
n
with
m
o
d
est
co
m
p
u
tatio
n
al
r
eso
u
r
ce
s
.
T
h
e
d
if
f
er
en
ce
b
etwe
en
ac
cu
r
ac
y
(
8
8
.
7
4
%)
a
n
d
r
ec
all
(
7
6
.
8
9
%)
r
ef
lects
a
d
elib
er
ate
ch
o
ice
in
f
av
o
r
o
f
s
p
ec
if
icity
,
alig
n
ed
with
clin
ic
al
p
r
io
r
ities
wh
er
e
f
alse p
o
s
itiv
es a
r
e
p
r
ef
e
r
r
ed
t
o
f
alse n
eg
a
tiv
es
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
C
P
U
-
o
p
timiz
ed
r
eg
u
la
r
iz
ed
co
n
vo
lu
tio
n
a
l n
eu
r
a
l n
etw
o
r
k
a
r
ch
itectu
r
e
fo
r
mu
ltimed
ia
…
(
Gu
id
o
u
m
A
min
a
)
1889
An
aly
s
is
o
f
th
e
im
p
ac
t
o
f
th
e
class
if
icatio
n
th
r
esh
o
ld
(
Fig
u
r
e
8
)
r
ev
ea
ls
th
at
a
th
r
esh
o
ld
o
f
0
.
5
o
f
f
er
s
an
ap
p
r
o
p
r
i
ate
b
ala
n
ce
f
o
r
o
u
r
ap
p
licatio
n
co
n
tex
t,
with
a
s
p
ec
if
icity
o
f
9
0
.
6
1
%
an
d
a
s
en
s
itiv
ity
o
f
7
6
.
8
9
%.
Ho
wev
er
,
d
ep
e
n
d
in
g
o
n
s
p
ec
i
f
ic
clin
ical
n
ee
d
s
,
an
ad
ju
s
tm
en
t
o
f
th
e
th
r
esh
o
ld
co
u
ld
b
e
co
n
s
id
er
ed
:
a
lo
wer
th
r
esh
o
ld
(
0
.
3
o
r
0
.
4
)
m
ax
im
iz
in
g
th
e
F1
-
s
co
r
e
(
8
5
.
9
%)
wo
u
l
d
b
e
s
u
itab
le
f
o
r
id
en
tify
i
n
g
a
m
ax
im
u
m
n
u
m
b
er
o
f
s
u
s
p
ec
ted
ca
s
es,
wh
ile
a
h
i
g
h
er
th
r
esh
o
l
d
(
0
.
8
o
r
0
.
9
)
o
f
f
er
in
g
a
s
p
ec
if
icity
>9
7
%
wo
u
ld
b
e
ap
p
r
o
p
r
iate
f
o
r
d
iag
n
o
s
tic
co
n
te
x
ts
r
eq
u
ir
in
g
h
ig
h
ce
r
tain
t
y
.
Pro
g
r
ess
iv
e
r
e
g
u
lar
izatio
n
(
i
n
cr
ea
s
in
g
d
r
o
p
o
u
t
2
0
–
4
0
%)
p
r
o
v
e
d
p
ar
ticu
lar
ly
e
f
f
ec
tiv
e
in
c
o
n
tr
o
llin
g
o
v
er
f
itti
n
g
,
as
e
v
id
en
ce
d
b
y
th
e
m
o
d
er
ate
d
if
f
er
e
n
ce
b
etwe
en
th
e
tr
ain
in
g
an
d
v
alid
atio
n
cu
r
v
es
(
2
.
9
0
%).
T
h
is
o
b
s
er
v
ati
o
n
alig
n
s
wit
h
th
e
f
in
d
in
g
s
[
20]
–
[
2
2
]
o
n
t
h
e
ef
f
ec
tiv
en
ess
o
f
ad
ap
tiv
e
d
r
o
p
o
u
t
f
o
r
d
ee
p
m
o
d
els.
C
PU
o
p
tim
izatio
n
en
ab
l
es
tr
ain
in
g
in
~
2
.
4
h
o
u
r
s
o
n
s
tan
d
ar
d
h
ar
d
war
e
,
co
m
p
ar
ed
to
th
e
s
ev
er
al
d
a
y
s
o
f
ten
r
eq
u
ir
e
d
f
o
r
co
m
p
l
ex
ar
ch
itectu
r
es.
W
ith
o
n
ly
9
4
,
0
0
0
p
a
r
am
eter
s
,
o
u
r
m
o
d
el
is
2
2
3
tim
es
lig
h
ter
th
an
E
f
f
icien
tNetV2
-
S
an
d
3
7
2
tim
es
lig
h
ter
th
an
C
o
n
v
Ne
Xt.
T
h
is
p
ar
am
etr
ic
ef
f
icien
cy
f
ac
ilit
ates
d
ep
lo
y
m
en
t
in
r
eso
u
r
ce
-
co
n
s
tr
ain
e
d
en
v
ir
o
n
m
e
n
ts
,
a
c
r
u
cial
a
d
v
an
tag
e
f
o
r
clin
ical
ad
o
p
tio
n
[
2
3
]
.
T
h
e
co
m
p
a
r
is
o
n
s
p
r
esen
ted
in
T
ab
le
2
s
h
o
u
ld
b
e
in
ter
p
r
ete
d
with
ca
u
tio
n
.
T
h
e
r
ef
er
e
n
ce
d
m
eth
o
d
s
(
E
f
f
icien
tNetV2
-
S
[
3
]
,
C
o
n
v
NeX
t
[
4
]
,
C
NN
-
t
r
an
s
f
o
r
m
er
[
9
]
)
wer
e
ev
al
u
ated
u
n
d
er
d
if
f
er
en
t
ex
p
e
r
im
en
tal
co
n
d
itio
n
s
:
(
i)
t
h
e
p
a
r
titi
o
n
o
f
th
e
PC
am
d
ataset
u
s
ed
is
n
o
t
alwa
y
s
id
en
tical
(
s
o
m
e
s
tu
d
i
es
u
s
e
all
2
6
2
,
1
4
4
tr
ain
in
g
im
ag
es,
w
h
ile
we
u
s
e
o
n
ly
4
,
0
9
6
)
;
(
ii)
th
e
ar
ch
itectu
r
es
ar
e
tr
ain
ed
o
n
h
ar
d
war
e
ac
ce
ler
ato
r
s
(
GPU/TPU)
with
p
ar
am
etr
ic
b
u
d
g
ets
2
0
0
to
4
0
0
tim
es
lar
g
e
r
;
(
iii)
t
h
e
m
etr
ics
r
ep
o
r
ted
in
th
e
liter
atu
r
e
o
f
ten
co
m
e
f
r
o
m
th
e
o
f
f
icial
PC
a
m
test
s
e
t,
wh
er
ea
s
o
u
r
v
alid
atio
n
is
p
er
f
o
r
m
ed
o
n
a
s
u
b
s
et
o
f
1
,
0
2
4
im
a
g
es.
T
h
er
ef
o
r
e,
th
e
n
u
m
er
ical
s
u
p
er
io
r
ity
o
f
o
n
e
m
eth
o
d
o
v
er
an
o
t
h
er
ca
n
n
o
t
b
e
d
ir
ec
tl
y
attr
ib
u
ted
to
th
e
ar
ch
itec
tu
r
e
alo
n
e.
Ou
r
g
o
al
i
s
n
o
t
to
claim
b
etter
a
b
s
o
lu
te
p
er
f
o
r
m
an
ce
,
b
u
t
to
s
h
o
w
th
at
a
v
er
y
lig
h
tweig
h
t
r
eg
u
lar
ized
C
NN
(
9
4
K
p
ar
a
m
eter
s
)
ac
h
iev
es
clin
ically
ac
ce
p
tab
le
v
alu
es
with
m
in
im
al
co
m
p
u
tatio
n
al
r
eso
u
r
ce
s
.
C
o
m
p
ar
ed
to
th
e
C
NN
-
t
r
an
s
f
o
r
m
er
,
o
u
r
p
u
r
el
y
co
n
v
o
lu
ti
o
n
al
ap
p
r
o
ac
h
ac
h
iev
es
s
lig
h
tly
b
etter
p
er
f
o
r
m
an
ce
(
AUC +0
.
3
4
%)
with
o
n
ly
0
.
3
8
% o
f
th
e
p
ar
am
eter
s
an
d
2
4
.
6
% o
f
th
e
d
ata.
T
h
is
s
u
g
g
ests
th
at,
f
o
r
th
is
s
p
ec
if
ic
p
r
o
b
lem
o
f
class
if
y
in
g
h
is
to
lo
g
ical
p
atc
h
es,
a
w
ell
-
r
eg
u
lated
C
NN
ar
ch
itectu
r
e
ca
n
co
m
p
ete
with
m
o
r
e
co
m
p
lex
h
y
b
r
i
d
ar
ch
itec
tu
r
es,
th
u
s
v
alid
atin
g
o
u
r
ap
p
r
o
ac
h
o
f
r
ea
s
o
n
ed
s
im
p
licity
.
T
h
e
h
ig
h
s
p
ec
if
icity
(
9
0
.
2
%)
o
b
tain
ed
with
th
e
th
r
esh
o
ld
o
f
0
.
5
is
p
ar
ticu
lar
ly
r
elev
an
t
f
o
r
c
ase
tr
iag
e
in
d
iag
n
o
s
tic
p
ath
o
lo
g
y
.
As
[
2
4
]
,
[
2
5
]
p
o
in
t
o
u
t,
h
ig
h
s
p
ec
if
icity
r
ed
u
ce
s
t
h
e
wo
r
k
lo
ad
o
f
p
ath
o
lo
g
is
ts
b
y
m
in
im
izin
g
f
als
e
p
o
s
itiv
es.
O
u
r
m
o
d
el,
with
its
ab
ilit
y
to
ad
ju
s
t
th
e
th
r
esh
o
ld
as
n
ee
d
e
d
,
co
u
ld
s
er
v
e
as
a
f
lex
ib
le
f
ir
s
t
-
f
ilter
s
y
s
tem
:
a
lo
w
th
r
esh
o
ld
f
o
r
s
en
s
itiv
e
s
cr
ee
n
in
g
,
a
h
ig
h
th
r
esh
o
ld
f
o
r
d
iag
n
o
s
tic
co
n
f
ir
m
atio
n
.
C
o
m
p
u
tatio
n
al
ac
ce
s
s
ib
ilit
y
f
ac
ili
tates
ad
o
p
ti
o
n
in
d
iv
er
s
e
s
ettin
g
s
,
in
clu
d
in
g
r
eso
u
r
ce
-
lim
ited
r
eg
io
n
s
.
Fig
u
r
e
8
.
T
h
e
e
v
alu
atio
n
m
atr
i
ce
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
4
,
D
e
c
em
b
er
20
2
6
:
1
8
8
1
-
1
8
9
2
1890
3
.
8
.
St
a
t
is
t
ica
l r
o
bu
s
t
nes
s
a
nd
a
bla
t
io
n
3
.
8
.
1
.
P
er
f
o
r
m
a
nce
v
a
ria
bil
it
y
(
co
nfidence
inte
rv
a
ls
)
T
h
e
m
etr
ics
r
ep
o
r
ted
in
th
is
a
r
ticle
(
AUC
9
2
.
9
2
%,
ac
cu
r
ac
y
8
8
.
7
4
%,
s
en
s
itiv
ity
7
6
.
8
9
%,
s
p
ec
if
icity
9
0
.
6
1
%)
wer
e
o
b
tain
ed
o
n
a
s
p
ec
if
ic
s
u
b
s
et
o
f
1
,
0
2
4
v
alid
atio
n
s
am
p
les,
in
ac
co
r
d
an
ce
with
th
e
p
r
o
ce
d
u
r
e
d
escr
ib
ed
in
s
ec
tio
n
2
.
1
.
T
o
esti
m
ate
v
ar
iab
ilit
y
d
u
e
to
s
a
m
p
lin
g
,
we
ap
p
lied
a
b
o
o
ts
tr
ap
p
r
o
ce
d
u
r
e
(
1
,
0
0
0
r
ep
l
icate
s
)
to
an
o
t
h
er
r
a
n
d
o
m
d
r
aw
o
f
1
,
0
2
4
s
am
p
les
(
f
ix
ed
r
an
d
o
m
s
ee
d
4
2
)
.
T
h
e
9
5
%
c
o
n
f
id
en
ce
in
ter
v
als
o
b
tain
ed
ar
e:
AUC:
9
1
.
2
3
%
[
8
9
.
5
6
%
–
9
2
.
9
1
%]
an
d
a
cc
u
r
ac
y
: 8
1
.
9
3
%
[
7
9
.
5
9
%
–
8
4
.
3
8
%]
.
T
h
ese
in
ter
v
als
s
h
o
w
th
at
t
h
e
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
ca
n
f
lu
ctu
ate
d
ep
en
d
in
g
o
n
th
e
v
alid
atio
n
s
am
p
le,
b
u
t
r
em
ain
s
with
in
a
clin
ically
ac
ce
p
tab
le
r
an
g
e
(
A
UC
≥
8
9
.
5
%).
T
h
e
v
alu
es
r
ep
o
r
ted
i
n
th
e
r
est
o
f
th
e
ar
ticle
ar
e
th
o
s
e
o
b
tain
ed
o
n
th
e
o
r
ig
in
al
s
am
p
le,
in
ac
co
r
d
an
ce
with
o
u
r
p
r
o
to
c
o
l.
T
h
e
s
tab
ilit
y
o
f
th
e
tr
ain
in
g
is
f
u
r
th
er
ev
id
en
ce
d
b
y
th
e
ea
r
ly
ter
m
in
atio
n
with
a
p
atien
ce
o
f
1
5
ep
o
c
h
s
(
s
ec
tio
n
2
.
4
)
an
d
th
e
s
m
all
d
if
f
er
en
ce
b
etwe
en
th
e
tr
ain
i
n
g
(
9
5
.
8
2
%)
an
d
v
alid
atio
n
(
9
2
.
9
2
%)
AUCs
,
r
ep
r
esen
tin
g
a
d
if
f
er
en
ce
o
f
2
.
9
0
p
er
ce
n
tag
e
p
o
in
ts
(
s
ec
tio
n
3
.
2
)
,
wh
ich
in
d
icate
s
g
o
o
d
c
o
n
tr
o
l
o
f
o
v
er
f
itti
n
g
.
3
.
8
.
2
.
Abla
t
io
n study
–
ef
f
ec
t
o
f
pro
g
re
s
s
iv
e
dro
po
ut
T
o
q
u
a
n
tify
t
h
e
co
n
tr
ib
u
tio
n
o
f
d
r
o
p
o
u
t
r
eg
u
lar
izatio
n
,
we
tr
ain
ed
a
s
tr
ictly
id
e
n
tical
ar
ch
it
ec
tu
r
e
b
u
t
with
o
u
t
an
y
d
r
o
p
o
u
t
lay
er
,
u
s
in
g
th
e
s
am
e
h
y
p
er
p
ar
am
ete
r
s
(
Ad
am
o
p
tim
izer
,
lear
n
in
g
r
ate
0
.
0
0
0
1
,
ea
r
ly
s
to
p
p
in
g
o
n
v
alid
atio
n
with
p
atien
ce
1
5
,
m
ax
im
u
m
1
0
0
e
p
o
ch
s
)
.
T
h
e
u
n
r
e
g
u
lar
ized
m
o
d
e
l
ac
h
iev
ed
an
AUC
o
f
9
5
.
0
6
%
in
tr
ain
in
g
an
d
9
0
.
6
9
%
in
v
alid
atio
n
,
a
d
if
f
er
e
n
ce
o
f
4
.
3
7
p
er
ce
n
tag
e
p
o
in
ts
.
I
n
co
m
p
ar
is
o
n
,
th
e
r
eg
u
lar
ized
m
o
d
el
(
p
r
o
g
r
ess
iv
e
d
r
o
p
o
u
t
0
.
2
–
0
.
4
)
s
h
o
wed
a
d
if
f
er
en
ce
o
f
o
n
ly
2
.
9
0
p
er
ce
n
tag
e
p
o
in
ts
(
tr
ain
in
g
AUC
9
5
.
8
2
%,
v
alid
atio
n
AU
C
9
2
.
9
2
%).
Pro
g
r
ess
iv
e
d
r
o
p
o
u
t
th
er
ef
o
r
e
r
ed
u
ce
s
o
v
er
f
itti
n
g
b
y
a
p
p
r
o
x
im
ately
3
3
.
6
%
(
Fig
u
r
e
9
)
.
T
h
is
im
p
r
o
v
em
en
t
co
n
f
i
r
m
s
th
e
e
f
f
ec
ti
v
en
ess
o
f
r
eg
u
lar
izatio
n
in
c
o
n
tr
o
llin
g
th
e
g
ap
b
etwe
en
tr
ain
in
g
a
n
d
v
alid
atio
n
p
er
f
o
r
m
an
ce
.
Fig
u
r
e
9
.
C
o
m
p
a
r
is
o
n
o
f
lear
n
in
g
cu
r
v
es (
ab
latio
n
)
4.
CO
NCLU
SI
O
N
T
h
is
s
tu
d
y
d
em
o
n
s
tr
ates
t
h
at
a
C
NN
ar
ch
itectu
r
e
with
p
r
o
g
r
ess
iv
e
r
eg
u
lar
izat
io
n
(
B
atch
n
o
r
m
aliza
tio
n
a
n
d
p
r
o
g
r
ess
iv
e
d
r
o
p
o
u
t
0
.
2
–
0
.
4
)
ca
n
ac
h
i
ev
e
co
m
p
etitiv
e
p
er
f
o
r
m
an
ce
f
o
r
b
r
ea
s
t
ca
n
ce
r
m
etastas
i
s
d
etec
tio
n
o
n
th
e
PC
am
d
ataset,
wh
ile
b
ein
g
s
ig
n
if
ican
tly
lig
h
ter
an
d
m
o
r
e
ac
c
ess
ib
le
th
an
lead
in
g
-
ed
g
e
ap
p
r
o
ac
h
es
.
On
a
v
alid
atio
n
s
u
b
s
et
o
f
1
,
0
2
4
p
atch
es,
o
u
r
m
o
d
el
ac
h
ie
v
es
an
AUC
o
f
9
2
.
9
2
%,
an
ac
cu
r
ac
y
o
f
8
8
.
7
4
%,
a
s
en
s
itiv
ity
o
f
7
6
.
8
9
%,
an
d
a
s
p
ec
if
icity
o
f
9
0
.
6
1
%.
B
o
o
ts
tr
ap
r
o
b
u
s
tn
ess
an
aly
s
is
(
1
,
0
0
0
r
e
p
licates)
o
n
an
o
th
er
ra
n
d
o
m
s
am
p
le
o
f
1
,
0
2
4
s
am
p
les
y
ield
s
9
5
%
co
n
f
id
en
ce
in
ter
v
als
f
o
r
th
e
AUC:
[
8
9
.
5
6
%
–
9
2
.
9
1
%],
in
d
icati
n
g
s
tab
le
p
e
r
f
o
r
m
an
ce
.
T
h
e
ab
latio
n
s
tu
d
y
s
h
o
ws
th
a
t
r
em
o
v
in
g
th
e
d
r
o
p
o
u
t
lay
er
s
in
cr
ea
s
es
th
e
g
ap
b
etwe
en
tr
ain
in
g
an
d
v
alid
atio
n
AUCs
f
r
o
m
2
.
9
0
to
4
.
3
7
p
o
i
n
ts
,
r
ep
r
esen
tin
g
a
3
3
.
6
%
r
e
d
u
ctio
n
in
o
v
er
f
itti
n
g
th
an
k
s
to
p
r
o
g
r
ess
iv
e
d
r
o
p
o
u
t.
Evaluation Warning : The document was created with Spire.PDF for Python.