I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
pu
t
er
E
ng
ineering
(
I
J
E
CE
)
Vo
l.
15
,
No
.
3
,
J
u
n
e
20
25
,
p
p
.
2820
~
2
8
3
3
I
SS
N:
2088
-
8
7
0
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijece.
v
15
i
3
.
pp
2
8
2
0
-
2
8
3
3
2820
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ec
e.
ia
esco
r
e.
co
m
G
ene
tic
a
lg
o
rith
m
-
a
da
pted
a
ct
iv
a
tion fun
ction o
pti
miza
tion o
f
deep learning
fra
mewo
rk f
o
r
breas
t
ma
ss
cancer
clas
sifica
tion
in ma
mm
o
g
ra
m ima
g
es
No
o
r
F
a
dzila
h Ra
za
li
1
,
I
za
S
a
za
nita
I
s
a
1
,
Siti
No
ra
ini
Su
la
im
a
n
1,
2
,
M
uh
a
m
ma
d K
hu
s
a
iri
O
s
m
a
n
1
,
No
o
r
K
ha
iria
h A.
K
a
ri
m
3
,
D
a
y
a
ng
Su
ha
ida
Awa
ng
Da
m
i
t
1
1
El
e
c
t
r
i
c
a
l
E
n
g
i
n
e
e
r
i
n
g
S
t
u
d
i
e
s
,
U
n
i
v
e
r
si
t
i
T
e
k
n
o
l
o
g
i
M
A
R
A
,
C
a
w
a
n
g
a
n
P
u
l
a
u
P
i
n
a
n
g
,
K
a
m
p
u
s Perm
a
t
a
n
g
P
a
u
h
,
P
u
l
a
u
P
i
n
a
n
g
,
M
a
l
a
y
si
a
2
I
n
t
e
g
r
a
t
i
v
e
P
h
a
r
mac
o
g
e
n
o
mi
c
s I
n
st
i
t
u
t
e
(
i
P
R
O
M
I
S
E)
,
U
i
T
M
P
u
n
c
a
k
A
l
a
m C
a
mp
u
s,
B
a
n
d
a
r
P
u
n
c
a
k
A
l
a
m
,
S
e
l
a
n
g
o
r
,
M
a
l
a
y
si
a
3
D
e
p
a
r
t
me
n
t
o
f
B
i
o
m
e
d
i
c
a
l
I
mag
i
n
g
,
A
d
v
a
n
c
e
d
M
e
d
i
c
a
l
a
n
d
D
e
n
t
a
l
I
n
st
i
t
u
t
e
,
U
n
i
v
e
r
si
t
i
S
a
i
n
s
M
a
l
a
y
s
i
a
B
e
r
t
a
m
,
P
u
l
a
u
P
i
n
a
n
g
,
M
a
l
a
y
si
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Au
g
9
,
2
0
2
4
R
ev
is
ed
Feb
1
6
,
2
0
2
5
Acc
ep
ted
Ma
r
4
,
2
0
2
5
Th
e
c
o
n
v
o
l
u
ti
o
n
a
l
n
e
u
ra
l
n
e
two
rk
(CNN
)
h
a
s
b
e
e
n
e
x
p
lo
re
d
f
o
r
m
a
m
m
o
g
ra
m
c
a
n
c
e
r
c
las
sifica
ti
o
n
to
a
i
d
ra
d
i
o
l
o
g
ists.
CNN
s
re
q
u
ire
m
u
l
ti
p
le
c
o
n
v
o
lu
ti
o
n
a
n
d
n
o
n
-
li
n
e
a
rit
y
re
p
e
ti
ti
o
n
s
to
le
a
rn
d
a
ta
sp
a
rsity
,
b
u
t
d
e
e
p
e
r
n
e
tw
o
rk
s o
ften
fa
c
e
th
e
v
a
n
ish
in
g
g
ra
d
ie
n
t
e
ffe
c
t,
wh
ich
h
i
n
d
e
rs
e
ffe
c
ti
v
e
lea
rn
in
g
.
Th
e
re
c
ti
fied
li
n
e
a
r
u
n
it
(Re
LU)
a
c
ti
v
a
ti
o
n
f
u
n
c
ti
o
n
a
c
ti
v
a
tes
n
e
u
r
o
n
s
o
n
l
y
wh
e
n
th
e
o
u
t
p
u
t
e
x
c
e
e
d
s
z
e
ro
,
l
imit
in
g
a
c
ti
v
a
ti
o
n
a
n
d
p
o
te
n
ti
a
ll
y
lo
we
ri
n
g
p
e
rfo
rm
a
n
c
e
.
Th
is
stu
d
y
p
r
o
p
o
s
e
s
a
n
a
d
a
p
ti
v
e
Re
LU
b
a
se
d
o
n
a
g
e
n
e
ti
c
a
lg
o
rit
h
m
(G
A)
to
d
e
term
in
e
th
e
o
p
ti
m
a
l
th
re
sh
o
ld
f
o
r
n
e
u
r
o
n
a
c
ti
v
a
ti
o
n
,
t
h
u
s
imp
ro
v
i
n
g
t
h
e
re
stricti
v
e
n
a
t
u
r
e
o
f
th
e
o
ri
g
in
a
l
Re
LU.
We
c
o
m
p
a
re
d
p
e
rfo
rm
a
n
c
e
s o
n
th
e
INb
re
a
st an
d
IP
P
T
-
m
a
m
m
o
m
a
m
m
o
g
ra
m
d
a
tas
e
ts u
sin
g
Re
LU
a
n
d
lea
k
y
Re
LU
a
c
ti
v
a
ti
o
n
fu
n
c
ti
o
n
s.
Re
su
lt
s
s
h
o
w
a
c
c
u
ra
c
y
imp
ro
v
e
m
e
n
ts
fr
o
m
9
5
.
0
%
to
9
7
.
0
1
%
fo
r
INb
re
a
st
a
n
d
8
4
.
9
%
to
8
7
.
4
%
fo
r
IP
P
T
-
m
a
m
m
o
with
Re
LU
a
n
d
fro
m
9
3
.
0
3
%
to
9
9
.
0
%
fo
r
IN
b
re
a
st
a
n
d
8
4
.
0
3
%
t
o
9
1
.
0
6
%
f
o
r
IP
P
T
-
m
a
m
m
o
with
lea
k
y
Re
LU.
S
i
g
n
ifi
c
a
n
t
a
c
c
u
ra
c
y
imp
ro
v
e
m
e
n
ts
we
re
o
b
se
rv
e
d
fo
r
b
re
a
st
c
a
n
c
e
r
c
las
sifica
ti
o
n
in
m
a
m
m
o
g
ra
m
s,
d
e
m
o
n
stra
ti
n
g
it
s
p
o
ten
ti
a
l
t
o
a
i
d
ra
d
i
o
l
o
g
ists
wi
th
m
o
re
ro
b
u
st
a
n
d
re
li
a
b
le
d
iag
n
o
stic t
o
o
ls.
K
ey
w
o
r
d
s
:
Activ
atio
n
f
u
n
ctio
n
C
o
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
Dee
p
lear
n
in
g
Ma
m
m
o
g
r
am
Op
tim
izatio
n
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
No
o
r
Fad
zilah
R
az
ali
E
lectr
ical
E
n
g
in
ee
r
in
g
Stu
d
ies,
C
o
lleg
e
o
f
E
n
g
in
ee
r
in
g
,
Un
i
v
er
s
iti T
ek
n
o
lo
g
i M
AR
A
C
awa
n
g
an
Pu
lau
Pin
an
g
,
Kam
p
u
s
Per
m
atan
g
P
au
h
1
3
5
0
0
Per
m
atan
g
Pau
h
,
Pu
lau
Pin
an
g
,
Ma
lay
s
ia
E
m
ail:
f
ad
zilah
7
0
8
@
u
itm
.
ed
u
.
m
y
1.
I
NT
RO
D
UCT
I
O
N
B
r
ea
s
t
ca
n
ce
r
is
th
e
m
o
s
t
c
o
m
m
o
n
ca
n
ce
r
am
o
n
g
w
o
m
en
w
o
r
ld
wid
e,
with
in
cid
e
n
ce
r
ates
in
cr
ea
s
in
g
as
wo
m
en
a
g
e
[
1
]
.
B
r
ea
s
t
ca
n
c
er
ca
s
es
h
av
e
r
is
en
b
y
3
1
%,
m
ak
in
g
it
th
e
s
ec
o
n
d
lead
in
g
ca
u
s
e
o
f
ca
n
ce
r
-
r
elate
d
d
ea
th
s
af
ter
lu
n
g
ca
n
ce
r
[
1
]
.
E
ar
ly
d
etec
tio
n
is
p
o
s
s
ib
le
th
r
o
u
g
h
m
am
m
o
g
r
am
s
cr
ee
n
in
g
s
,
b
u
t
ev
alu
atin
g
m
am
m
o
g
r
a
m
im
ag
es
ty
p
ically
in
v
o
lv
es
m
an
u
al
task
s
s
u
ch
as
ad
ju
s
tin
g
co
n
tr
ast,
d
etec
tin
g
m
ass
es,
an
d
m
ak
in
g
in
itial
d
iag
n
o
s
es.
Au
to
m
atin
g
th
ese
p
r
o
ce
s
s
es
with
co
m
p
u
ter
-
aid
ed
d
etec
tio
n
(
C
AD)
s
y
s
tem
s
ca
n
im
p
r
o
v
e
d
iag
n
o
s
is
ac
cu
r
ac
y
a
n
d
p
atien
t c
ar
e.
T
h
e
ad
v
an
ce
m
e
n
t
o
f
d
ee
p
lea
r
n
in
g
th
r
o
u
g
h
co
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
(
C
NN)
h
as
b
ee
n
u
s
ed
to
d
ev
elo
p
t
h
e
m
am
m
o
g
r
am
C
AD
s
y
s
tem
.
I
ts
ar
ch
itectu
r
e
in
v
o
lv
es
co
n
v
o
lu
tio
n
,
n
o
n
-
lin
ea
r
ity
,
an
d
p
o
o
lin
g
lay
er
s
to
im
p
r
o
v
e
th
e
o
v
e
r
all
C
AD
p
er
f
o
r
m
a
n
ce
[
2
]
,
[
3
]
.
Ho
wev
er
,
C
NNs
ca
n
b
e
af
f
ec
ted
b
y
th
e
v
an
is
h
in
g
g
r
ad
ien
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Gen
etic
a
lg
o
r
ith
m
-
a
d
a
p
ted
a
ctiva
tio
n
fu
n
ctio
n
o
p
timiz
a
tio
n
o
f d
ee
p
le
a
r
n
in
g
…
(
N
o
o
r
F
a
d
z
ila
h
R
a
z
a
li
)
2821
p
r
o
b
lem
[
4
]
,
wh
er
e
weig
h
ts
s
to
p
u
p
d
atin
g
,
f
ailin
g
t
o
ca
p
tu
r
e
co
m
p
lex
f
ea
tu
r
es.
Mu
ltip
le
ac
t
iv
atio
n
f
u
n
ctio
n
s
in
C
NN
ar
ch
itectu
r
e
o
b
tain
th
e
s
p
ar
s
ity
o
f
th
e
lear
n
ed
weig
h
ts
o
n
ea
ch
lay
er
to
lear
n
a
co
m
p
lex
f
ea
tu
r
e
r
ep
r
esen
tatio
n
.
Fig
u
r
e
1
s
h
o
ws
m
u
ltip
le
ac
tiv
atio
n
f
u
n
ctio
n
s
av
ailab
le
to
b
e
u
s
ed
with
in
th
e
C
NN
ar
ch
itectu
r
e.
I
t
in
tr
o
d
u
ce
d
n
o
n
-
lin
ea
r
ity
an
d
h
elp
ed
m
itig
ate
v
a
n
is
h
in
g
g
r
ad
ien
t
p
r
o
b
lem
b
y
d
ea
ctiv
a
tin
g
n
eu
r
o
n
s
in
t
h
e
n
eg
ativ
e
r
eg
i
o
n
,
r
esu
ltin
g
in
s
p
ar
s
ity
,
allo
win
g
co
m
p
lex
f
ea
tu
r
e
lear
n
in
g
,
m
ak
in
g
th
em
c
r
u
cial
in
C
NNs
[
5
]
,
[
6
]
.
R
ec
tifie
d
lin
ea
r
u
n
its
(
R
e
L
U)
as
d
ep
icted
in
Fig
u
r
e
1
(
a)
ar
e
th
e
m
o
s
t
co
m
m
o
n
ly
u
s
ed
ac
tiv
atio
n
f
u
n
ctio
n
s
.
R
eL
U
is
f
av
o
r
ed
f
o
r
its
s
im
p
licity
,
lo
w
co
m
p
u
tatio
n
al
co
s
t,
an
d
ef
f
icien
cy
.
Fig
u
r
e
1
(
b
)
s
h
o
ws
v
ar
ian
ts
o
f
th
e
R
eL
U
th
at
f
o
cu
s
o
n
allo
win
g
th
ese
o
u
tlier
s
to
b
e
ac
ti
v
ated
o
n
th
e
n
e
g
ativ
e
r
e
g
io
n
s
,
s
u
ch
as
th
e
leak
y
R
eL
U,
p
ar
am
etr
ic
R
eL
U
(
Pre
L
U)
,
G
a
u
s
s
ian
lin
ea
r
u
n
its
(
GE
L
U)
,
an
d
ex
p
o
n
en
tial lin
ea
r
u
n
its
(
E
L
U)
.
(
a)
(
b
)
Fig
u
r
e
1
.
Gr
a
p
h
s
o
f
(
a)
R
eL
U
an
d
(
b
)
its
d
er
iv
ativ
es' a
ctiv
ati
o
n
f
u
n
ctio
n
v
ar
ian
ts
u
s
ed
in
C
NN
Ho
wev
er
,
d
u
e
to
th
is
s
lig
h
t a
ll
o
wan
ce
in
th
e
n
eg
ativ
e
r
eg
io
n
co
m
p
ar
ed
to
t
h
e
o
r
ig
in
al
R
eL
U,
th
ey
a
r
e
co
m
p
u
tatio
n
ally
m
o
r
e
ex
p
en
s
iv
e
an
d
r
eq
u
ir
e
a
d
d
itio
n
al
m
em
o
r
y
to
s
to
r
e
th
e
tr
ain
ed
p
ar
a
m
eter
s
.
C
h
o
o
s
in
g
th
e
r
ig
h
t
n
o
n
-
li
n
ea
r
ity
p
ar
am
eter
co
ef
f
icien
t
in
E
L
U
a
n
d
leak
y
R
eL
U
r
eq
u
ir
es
p
r
io
r
k
n
o
wled
g
e
ab
o
u
t
th
e
d
ata
a
n
d
ca
r
ef
u
l tu
n
in
g
t
o
s
elec
t th
e
c
o
r
r
ec
t v
alu
e.
Me
an
wh
ile,
th
e
n
e
wer
GE
L
U
r
eq
u
ir
es th
e
u
s
er
t
o
d
eter
m
i
n
e
wh
eth
er
to
ap
p
ly
th
e
T
a
n
h
a
p
p
r
o
x
im
ati
o
n
m
eth
o
d
f
o
r
t
h
e
u
n
d
e
r
ly
in
g
e
r
r
o
r
f
u
n
ctio
n
,
w
h
ich
also
r
eq
u
i
r
es
p
r
io
r
k
n
o
wled
g
e
o
f
th
e
d
ataset
b
eh
a
v
io
r
.
T
h
er
ef
o
r
e,
co
n
tin
u
o
u
s
r
esear
ch
o
n
ac
t
iv
atio
n
f
u
n
ctio
n
s
r
em
ain
e
d
ac
t
iv
e,
wh
ile
th
e
m
o
r
e
estab
lis
h
ed
an
d
th
e
R
eL
U
ar
ch
itectu
r
e
co
n
tin
u
e
d
to
b
e
u
s
ed
i
n
s
tan
d
ar
d
C
NN
ar
ch
itectu
r
e.
Gen
etic
alg
o
r
ith
m
(
GA)
[
7
]
i
s
a
v
er
s
atile
p
io
n
ee
r
in
g
m
eta
h
eu
r
is
tic
o
p
tim
izatio
n
m
eth
o
d
,
allo
win
g
p
r
ac
titi
o
n
er
s
to
tailo
r
it
to
s
p
ec
if
ic
task
s
.
I
ts
ap
p
licatio
n
s
i
n
v
o
lv
e
im
a
g
e
p
r
o
ce
s
s
in
g
m
o
d
u
les
an
d
co
m
p
u
ter
v
is
io
n
th
r
o
u
g
h
n
atu
r
al
s
elec
tio
n
an
d
ev
o
lu
tio
n
th
r
o
u
g
h
th
e
s
u
r
v
iv
al
o
f
th
e
f
ittes
t c
o
n
ce
p
t
[
8
]
,
[
9
]
.
GA
is
m
o
s
tly
ap
p
lied
f
o
r
f
ea
tu
r
e
s
elec
tio
n
[
9
]
,
[
1
0
]
a
n
d
h
y
p
er
p
ar
am
eter
o
p
tim
izatio
n
[
8
]
,
[
1
1
]
i
n
b
r
ea
s
t
-
r
elate
d
s
tu
d
ies
an
d
in
b
io
m
ed
ical
a
p
p
licatio
n
s
,
i
n
clu
d
in
g
b
r
ain
MRI
[
1
2
]
an
d
C
NN
tr
ain
in
g
h
y
p
er
p
ar
am
e
ter
s
elec
tio
n
[
9
]
.
I
t
ex
p
lo
r
es
th
e
s
o
lu
tio
n
s
p
ac
e,
f
av
o
r
s
b
etter
s
o
lu
tio
n
s
,
an
d
th
e
m
u
tatio
n
s
co
n
tr
ib
u
te
m
o
d
est
r
an
d
o
m
c
h
an
g
es
to
m
ain
tain
g
en
etic
d
iv
er
s
ity
,
wh
ile
cr
o
s
s
o
v
er
co
m
b
in
es
g
en
etic
in
f
o
r
m
atio
n
f
r
o
m
two
p
a
r
en
ts
to
p
r
o
d
u
ce
o
p
tim
al
o
f
f
s
p
r
in
g
th
r
o
u
g
h
s
u
cc
ess
iv
e
g
en
er
atio
n
s
m
ar
k
e
d
b
y
im
p
r
o
v
e
m
en
ts
in
s
o
lu
tio
n
q
u
ality
[
7
]
,
[
1
3
]
.
GA's
v
er
s
atility
an
d
ab
ilit
y
to
i
d
en
tify
n
ea
r
ly
o
p
tim
al
s
o
lu
tio
n
s
m
ak
e
it a
p
o
p
u
lar
o
p
tim
izatio
n
tech
n
iq
u
e.
C
NNs
h
av
e
s
ig
n
if
ican
tly
ad
v
an
ce
d
C
AD
s
y
s
tem
s
,
b
u
t
th
e
y
s
till
f
ac
e
c
h
allen
g
es,
p
ar
tic
u
lar
ly
th
e
v
an
is
h
in
g
g
r
ad
ien
t
p
r
o
b
lem
a
n
d
th
e
"d
y
in
g
R
eL
U"
is
s
u
e.
I
n
"d
y
in
g
R
eL
U,
"
s
o
m
e
o
u
tlier
n
eu
r
o
n
s
b
ec
o
m
e
in
ac
tiv
e
wh
ich
lim
its
th
e
m
o
d
el's
ab
ilit
y
to
lear
n
co
m
p
lex
f
e
atu
r
es
[
1
4
]
.
E
x
is
tin
g
s
o
lu
tio
n
s
,
lik
e
v
ar
ian
ts
o
f
R
eL
U,
ad
d
r
ess
s
o
m
e
o
f
th
ese
is
s
u
es
b
u
t
co
m
e
with
d
r
awb
ac
k
s
,
in
clu
d
i
n
g
h
ig
h
e
r
co
m
p
u
tat
io
n
al
co
s
ts
an
d
th
e
n
ee
d
f
o
r
ex
ten
s
iv
e
p
ar
am
eter
t
u
n
in
g
.
T
h
ese
c
h
allen
g
es r
ed
u
c
e
th
eir
p
r
ac
ticality
in
clin
ical
a
p
p
licatio
n
s
.
T
h
is
s
tu
d
y
o
f
f
er
s
a
s
o
lu
tio
n
b
y
in
tr
o
d
u
cin
g
a
n
o
v
el
ad
a
p
tatio
n
o
f
th
e
R
eL
U
ac
tiv
atio
n
f
u
n
ct
io
n
in
p
r
e
-
tr
ain
ed
C
NNs,
o
p
tim
ized
u
s
in
g
GA.
T
h
e
ad
ap
tiv
e
f
u
n
ctio
n
ad
ju
s
ts
d
y
n
am
ically
to
allo
w
lim
ited
n
eg
ativ
e
o
r
p
o
s
itiv
e
o
u
tp
u
ts
b
ased
o
n
an
a
u
to
m
atica
lly
d
eter
m
in
ed
th
r
es
h
o
ld
.
T
h
is
a
p
p
r
o
ac
h
p
r
eser
v
es
th
e
b
en
e
f
its
o
f
n
o
n
-
lin
ea
r
ity
wh
ile
ad
d
r
ess
in
g
v
a
n
is
h
in
g
g
r
ad
ie
n
ts
.
Un
lik
e
tr
ad
itio
n
al
m
eth
o
d
s
th
at
r
eq
u
ir
e
m
an
u
al
tu
n
in
g
,
GA
au
to
m
atica
lly
f
in
e
-
tu
n
es
ac
tiv
a
tio
n
p
ar
am
eter
s
,
m
ak
in
g
th
e
s
y
s
tem
m
o
r
e
ef
f
icien
t
an
d
b
ette
r
s
u
ited
f
o
r
h
a
n
d
lin
g
co
m
p
lex
d
ata.
B
u
ild
in
g
o
n
p
r
io
r
r
esear
ch
t
h
at
ap
p
lied
GA
t
o
task
s
lik
e
f
ea
tu
r
e
s
elec
tio
n
an
d
h
y
p
er
p
a
r
am
eter
o
p
tim
izatio
n
in
m
am
m
o
g
r
am
an
aly
s
is
,
th
is
s
tu
d
y
tak
es
a
p
i
o
n
ee
r
in
g
s
tep
b
y
u
s
in
g
GA
to
o
p
tim
ize
th
e
n
o
n
-
lin
ea
r
ity
lay
er
its
elf
.
T
h
is
ap
p
r
o
ac
h
d
i
r
ec
tly
tack
les
th
e
"d
y
in
g
R
eL
U"
p
r
o
b
lem
an
d
im
p
r
o
v
es
th
e
m
o
d
el's
r
o
b
u
s
tn
ess
.
T
h
is
s
tu
d
y
co
n
tr
ib
u
tes
to
th
e
f
o
llo
win
g
:
i)
I
n
tr
o
d
u
ctio
n
o
f
m
eta
h
eu
r
is
tic
GA
o
p
tim
izatio
n
with
in
estab
lis
h
ed
C
NN
p
r
e
-
tr
ain
ed
lay
er
s
;
ii)
Me
th
o
d
o
f
ad
ap
te
d
ac
tiv
at
io
n
f
u
n
ctio
n
o
p
tim
izatio
n
o
n
m
am
m
o
g
r
am
d
atasets
;
a
n
d
iii)
Ad
ap
tin
g
d
if
f
er
en
t
ac
tiv
atio
n
f
u
n
ctio
n
s
u
s
in
g
th
e
p
r
o
p
o
s
ed
m
eth
o
d
f
o
r
im
p
r
o
v
ed
p
er
f
o
r
m
an
ce
.
R
e
L
U
A
c
ti
vati
on
F
u
n
c
ti
on
s
N
e
u
r
on
A
c
ti
vi
ty
(0,0)
ELU
P
R
e
L
U
G
E
L
U
A
c
ti
vati
on
F
u
n
c
ti
on
s
N
e
u
r
on
A
c
ti
vi
ty
l
e
a
k
y
R
e
L
U
(0
,0
)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
3
,
J
u
n
e
20
25
:
2
8
2
0
-
2
8
3
3
2822
2.
RE
L
AT
E
D
WO
RK
S
I
n
r
ec
e
n
t
y
ea
r
s
,
C
NN
-
b
ased
s
tu
d
ies
o
n
m
am
m
o
g
r
am
-
r
elate
d
im
ag
es
h
a
v
e
b
ee
n
p
er
f
o
r
m
ed
v
ia
co
m
p
u
ter
v
is
io
n
in
v
ar
io
u
s
s
tag
es,
s
u
ch
as
in
im
ag
e
p
r
e
-
p
r
o
ce
s
s
in
g
[
1
5
]
,
ca
n
ce
r
d
etec
tio
n
an
d
s
eg
m
en
tatio
n
[
1
6
]
,
[
1
7
]
,
an
d
ca
n
ce
r
class
if
icatio
n
[
1
8
]
–
[
2
0
]
.
Sin
ce
t
h
e
r
e
lev
an
t
f
ea
tu
r
es
wer
e
au
t
o
m
atica
lly
ex
tr
ac
ted
,
th
e
co
n
v
o
l
u
ted
c
o
m
b
in
atio
n
o
f
f
e
atu
r
es
ex
tr
ac
ted
f
r
o
m
th
e
tr
ai
n
in
g
im
a
g
es
m
ak
es
it
d
if
f
ic
u
lt
to
in
ter
p
r
et
wh
a
t
h
ap
p
en
e
d
d
u
r
i
n
g
th
e
d
ec
is
io
n
-
m
ak
in
g
p
r
o
ce
s
s
.
T
h
is
r
elate
s
to
in
d
u
cin
g
th
e
n
o
n
-
lin
ea
r
f
u
n
ctio
n
s
with
in
th
e
ar
ch
itectu
r
e
in
th
e
f
o
r
m
o
f
r
e
p
ea
ted
lay
e
r
s
with
co
n
v
o
l
u
tio
n
,
wh
ich
d
is
s
o
ciate
s
th
e
o
r
ig
i
n
al
s
tr
aig
h
tf
o
r
war
d
co
n
v
o
l
u
tio
n
an
d
f
ilter
in
g
p
r
o
c
ess
in
a
C
NN.
I
n
th
e
p
ast,
r
esear
c
h
er
s
h
av
e
p
r
o
p
o
s
ed
s
ev
er
al
in
n
o
v
ativ
e
ap
p
r
o
ac
h
es
r
elate
d
to
ac
ti
v
atio
n
f
u
n
ctio
n
s
to
en
h
an
ce
th
e
p
er
f
o
r
m
an
ce
o
f
C
NNs
in
n
etwo
r
k
a
n
aly
s
is
.
An
ad
a
p
tiv
e
ac
tiv
atio
n
f
u
n
ctio
n
,
n
am
ely
lay
er
-
wis
e
an
d
n
eu
r
o
n
-
wis
e,
was
in
tr
o
d
u
ce
d
with
o
p
tim
izatio
n
to
ac
ce
ler
ate
m
o
d
el
co
n
v
er
g
e
n
ce
as
th
e
ef
f
ec
t
o
f
s
lo
p
e
r
ec
o
v
er
y
d
u
r
in
g
tr
ain
i
n
g
[
2
1
]
.
T
h
is
au
th
o
r
also
p
r
o
p
o
s
ed
a
R
o
wd
y
ac
tiv
atio
n
f
u
n
ctio
n
to
b
e
u
s
ed
with
m
u
ltip
l
e
ty
p
es o
f
tr
ain
ab
le
p
a
r
am
eter
s
a
ch
iev
ed
b
y
in
jectin
g
s
in
u
s
o
id
a
l f
lu
ctu
atio
n
s
[
2
2
]
t
o
less
en
th
e
s
atu
r
atio
n
r
eg
io
n
.
Similar
ly
,
an
o
th
e
r
s
tu
d
y
b
y
[
2
3
]
p
r
o
p
o
s
ed
a
m
o
d
if
ie
d
R
eL
U
-
b
ased
ac
tiv
atio
n
f
u
n
ctio
n
b
y
in
tr
o
d
u
cin
g
a
p
er
io
d
ic
s
in
u
s
o
id
al
f
u
n
ctio
n
o
n
th
e
p
o
s
itiv
e
r
eg
io
n
f
o
r
ac
tiv
atin
g
th
e
n
eu
r
o
n
o
n
a
v
o
ice
o
r
d
e
r
class
i
f
icatio
n
,
s
u
g
g
esti
n
g
im
p
r
o
v
e
d
ac
cu
r
ac
y
in
a
co
n
v
o
l
u
tio
n
al
n
eu
r
al
n
etwo
r
k
-
lo
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
(
C
NN
-
L
STM
)
m
o
d
el.
I
n
a
s
tu
d
y
f
o
r
th
e
d
etec
tio
n
o
f
d
is
tr
ib
u
te
d
d
en
ial
o
f
s
er
v
ice
attac
k
s
o
n
r
ec
u
r
r
en
t
n
eu
r
al
n
etwo
r
k
s
,
m
o
d
if
icatio
n
o
f
t
h
e
h
y
p
er
b
o
lic
tan
g
e
n
t
ac
tiv
atio
n
f
u
n
ctio
n
k
n
o
wn
as
T
an
h
2
wa
s
in
tr
o
d
u
ce
d
.
T
h
e
f
u
n
ctio
n
h
a
s
a
r
an
g
e
o
f
v
alu
es
b
et
wee
n
0
a
n
d
1
in
th
e
b
ell
-
s
h
ap
ed
cu
r
v
e
f
u
n
ctio
n
,
wh
ich
r
e
m
o
v
ed
n
eg
ativ
e
v
alu
es
th
at
ca
u
s
ed
th
e
v
a
n
is
h
in
g
g
r
ad
ien
t
p
r
o
b
lem
in
d
ee
p
lear
n
in
g
[
2
4
]
.
Me
an
wh
ile,
a
n
ad
a
p
tiv
e
n
etwo
r
k
en
h
an
ce
m
e
n
t
u
s
in
g
a
two
-
la
y
er
R
eL
U
was
in
tr
o
d
u
ce
d
f
o
r
t
h
e
b
est
i
n
itializatio
n
o
f
its
n
o
n
-
lin
ea
r
o
p
tim
izatio
n
p
r
o
ce
s
s
o
n
th
e
b
est
least
-
s
q
u
ar
e
to
th
e
b
est
ap
p
r
o
x
im
atio
n
o
f
a
tar
g
et
f
u
n
ctio
n
[
6
]
.
T
h
e
m
et
h
o
d
d
et
er
m
in
ed
th
e
n
u
m
b
er
o
f
n
eu
r
o
n
s
f
o
r
t
h
e
ac
tiv
atio
n
to
b
e
a
d
d
ed
u
n
til
th
e
b
est
ap
p
r
o
x
im
atio
n
lead
s
to
th
e
b
est
m
o
d
el
ac
c
u
r
ac
y
.
Ad
d
itio
n
ally
,
an
a
d
ap
tiv
e
R
eL
U
m
eth
o
d
was
p
r
o
p
o
s
ed
b
y
[
2
5
]
b
y
o
p
tim
izin
g
t
h
e
p
o
s
itio
n
o
f
wh
en
th
e
n
eu
r
o
n
in
th
e
n
eg
at
iv
e
r
eg
io
n
will
b
e
ac
tiv
ated
b
y
r
etain
in
g
t
h
e
s
h
a
p
e
o
f
t
h
e
R
eL
U
o
n
t
h
e
ea
r
ly
lay
er
s
o
f
C
NN,
im
p
r
o
v
in
g
th
e
ac
cu
r
ac
y
o
f
t
h
r
ee
d
if
f
er
en
t
im
ag
e
d
atab
ases
.
Fu
r
th
er
m
o
r
e,
a
m
o
d
if
ied
R
eL
U
was
in
tr
o
d
u
ce
d
b
y
[
2
6
]
,
in
tr
o
d
u
cin
g
a
co
ef
f
icien
t
t
o
an
ex
p
o
n
en
tial
v
alu
e
o
n
th
e
n
eg
ativ
e
-
r
e
g
io
n
s
lo
p
e,
cr
ea
t
in
g
a
s
m
all
-
s
ca
le
n
eg
ativ
e
s
lo
p
e
s
im
ilar
to
th
e
leak
y
R
eL
U
with
th
e
ex
ce
p
tio
n
o
f
th
e
n
eg
ativ
e
s
lo
p
e.
Alth
o
u
g
h
lim
ited
,
s
o
m
e
s
tu
d
ies
h
av
e
f
o
cu
s
ed
o
n
tailo
r
i
n
g
ac
tiv
atio
n
f
u
n
ctio
n
s
s
p
ec
if
ically
f
o
r
m
am
m
o
g
r
a
m
d
atasets
.
I
n
a
s
t
u
d
y
u
s
in
g
a
co
g
e
n
t
ac
tiv
atio
n
f
u
n
ctio
n
b
ased
o
n
th
e
m
o
d
if
ied
T
an
h
f
u
n
ctio
n
,
in
tr
o
d
u
ce
d
to
b
e
ap
p
lied
t
o
m
am
m
o
g
r
am
im
ag
es,
with
th
e
b
e
s
t
ac
cu
r
ac
y
at
9
9
%
[
2
7
]
.
N
ex
t,
r
e
p
lace
m
en
ts
o
f
ac
tiv
atio
n
f
u
n
ctio
n
s
in
C
NN
m
o
d
els,
s
u
ch
as
p
lacin
g
th
e
l
ea
k
y
R
eL
U
[
2
8
]
,
a
n
d
PR
eL
U
[
2
9
]
wer
e
d
o
n
e
to
d
eliv
er
p
r
o
m
is
in
g
p
er
f
o
r
m
a
n
c
e
in
b
r
ea
s
t
ca
n
ce
r
class
if
icatio
n
co
m
p
ar
e
d
to
th
eir
b
ase
n
etwo
r
k
s
.
T
h
ese
f
u
n
ctio
n
s
m
itig
ated
th
e
v
an
is
h
in
g
g
r
ad
i
en
t
p
r
o
b
lem
ca
u
s
ed
b
y
d
y
i
n
g
R
eL
U
o
f
ten
o
b
s
er
v
ed
in
R
e
L
U
-
b
ased
n
etwo
r
k
s
,
im
p
r
o
v
in
g
class
if
icatio
n
ac
cu
r
ac
y
th
r
o
u
g
h
v
a
r
io
u
s
m
et
h
o
d
s
.
T
h
ese
s
tu
d
ies also
r
ev
ea
led
th
at
n
o
n
-
lin
ea
r
ity
i
n
a
n
etwo
r
k
p
lay
s
a
v
ital
r
o
le
in
d
e
f
in
in
g
f
ea
tu
r
es
r
ep
r
esen
tin
g
ea
ch
class
an
d
im
p
r
o
v
in
g
th
e
m
o
d
el’
s
g
en
er
aliza
tio
n
th
r
o
u
g
h
m
ea
n
in
g
f
u
l
d
ata
s
p
ar
s
it
y
.
Ho
wev
er
,
b
esid
es
r
eq
u
ir
in
g
i
n
-
d
ep
th
k
n
o
wled
g
e
o
f
d
ev
elo
p
in
g
th
e
m
ath
em
atica
l
m
o
d
el
o
f
th
e
ac
t
iv
atio
n
f
u
n
ctio
n
s
,
th
ese
m
eth
o
d
s
ar
e
ap
p
lied
to
a
cu
s
to
m
ized
n
etwo
r
k
m
o
d
el
to
th
e
s
p
ec
if
ic
task
s
an
d
ar
e
lim
i
ted
p
ar
ticu
lar
ly
d
u
e
to
th
e
f
i
x
ed
f
u
n
ctio
n
s
.
T
h
is
m
ak
es
it
m
o
r
e
co
m
p
licated
to
ad
a
p
t
an
d
g
en
er
alize
to
o
th
e
r
d
atasets
an
d
task
s
,
af
f
ec
tin
g
th
eir
g
en
e
r
aliza
b
ilit
y
.
T
ab
le
1
s
u
m
m
ar
izes
p
ast
s
tu
d
ies th
at
ex
p
lo
r
ed
t
h
e
m
o
d
if
icatio
n
o
f
ac
tiv
atio
n
f
u
n
ctio
n
s
with
in
n
eu
r
al
n
etwo
r
k
s
o
n
v
ar
io
u
s
ap
p
licatio
n
s
.
T
ab
le
1
.
Pas
t r
elate
d
s
tu
d
ies o
n
m
o
d
if
icatio
n
o
r
in
tr
o
d
u
ctio
n
o
f
n
o
v
el
ac
tiv
atio
n
f
u
n
ctio
n
s
in
n
eu
r
al
n
etwo
r
k
s
S
t
u
d
y
A
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
n
o
v
e
l
t
y
A
p
p
l
i
c
a
t
i
o
n
/
d
a
t
a
se
t
s
[
6
]
T
w
o
-
l
a
y
e
r
R
e
LU
a
r
c
h
i
t
e
c
t
u
r
e
S
e
l
f
-
a
d
j
o
i
n
t
sec
o
n
d
-
o
r
d
e
r
e
l
l
i
p
t
i
c
p
a
r
t
i
a
l
d
i
f
f
e
r
e
n
t
i
a
l
e
q
u
a
t
i
o
n
s
[
2
1
]
La
y
e
r
-
w
i
se
a
n
d
n
e
u
r
o
n
-
w
i
se
a
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
S
e
mei
o
n
,
C
I
F
A
R
-
1
0
,
C
I
F
A
R
-
1
0
0
a
n
d
S
V
H
N
[
2
2
]
R
o
w
d
y
a
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
,
si
n
u
so
i
d
a
l
f
l
u
c
t
u
a
t
i
o
n
s
S
e
mei
o
n
,
C
I
F
A
R
-
1
0
,
C
I
F
A
R
-
1
0
0
a
n
d
S
V
H
N
[
2
3
]
S
i
n
o
so
i
d
a
l
i
n
j
e
c
t
i
o
n
s,
s
i
n
u
s
o
i
d
a
l
f
l
u
c
t
u
a
t
i
o
n
s
V
o
i
c
e
o
r
d
e
r
c
l
a
ssi
f
i
c
a
t
i
o
n
[
2
4
]
M
o
d
i
f
i
e
d
T
a
n
h
a
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
D
i
st
r
i
b
u
t
e
d
d
e
n
i
a
l
o
f
serv
i
c
e
a
t
t
a
c
k
d
e
t
e
c
t
i
o
n
[
2
5
]
A
d
a
p
t
i
v
e
R
e
LU
o
n
[
-
1
0
]
a
n
d
[
0
-
1
]
r
e
g
i
o
n
s
C
i
f
a
r
1
0
,
C
i
f
a
r
1
0
0
,
T
i
n
y
I
ma
g
e
N
e
t
a
n
d
T
e
x
t
d
a
t
a
b
a
s
e
s
[
2
6
]
M
o
d
i
f
i
e
d
R
e
LU
,
n
e
g
a
t
i
v
e
s
l
o
p
e
o
n
t
h
e
n
e
g
a
t
i
v
e
r
e
g
i
o
n
M
u
l
t
i
s
p
e
c
t
r
a
l
i
m
a
g
e
f
o
r
l
a
b
e
l
l
e
d
E
u
r
o
S
A
T
sa
t
e
l
l
i
t
e
i
ma
g
e
s
[
27
]
M
o
d
i
f
i
e
d
T
a
n
h
f
u
n
c
t
i
o
n
f
o
r
c
o
g
e
n
t
a
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
M
I
A
S
d
a
t
a
se
t
,
m
a
mm
o
g
r
a
m c
l
a
ss
i
f
i
c
a
t
i
o
n
Me
an
wh
ile,
b
io
-
in
s
p
ir
ed
m
et
ah
eu
r
is
tic
o
p
tim
izatio
n
m
eth
o
d
s
h
a
v
e
b
ee
n
a
p
p
lied
to
m
a
m
m
o
g
r
am
-
b
ased
s
tu
d
ies,
d
em
o
n
s
tr
atin
g
t
h
eir
ab
ilit
y
to
en
h
a
n
ce
th
e
o
v
e
r
all
p
er
f
o
r
m
a
n
ce
o
f
s
y
s
tem
s
wh
en
co
m
b
i
n
ed
with
C
NN
ar
ch
itectu
r
es.
T
h
ese
alg
o
r
ith
m
s
o
f
f
er
d
is
tin
ct
s
tr
en
g
t
h
s
an
d
h
av
e
b
ee
n
in
teg
r
ated
in
t
o
v
ar
io
u
s
s
tag
es
o
f
m
am
m
o
g
r
a
m
an
aly
s
is
p
ip
elin
e
s
to
ad
d
r
ess
s
p
ec
if
ic
ch
allen
g
e
s
,
im
p
r
o
v
e
s
y
s
tem
p
er
f
o
r
m
a
n
c
e,
an
d
co
n
tr
ib
u
te
to
m
o
r
e
ac
cu
r
ate
an
d
r
eliab
le
b
r
ea
s
t
ca
n
ce
r
d
iag
n
o
s
is
.
Fo
r
in
s
tan
ce
,
in
a
s
tu
d
y
f
o
cu
s
ed
o
n
class
if
y
in
g
b
r
ea
s
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Gen
etic
a
lg
o
r
ith
m
-
a
d
a
p
ted
a
ctiva
tio
n
fu
n
ctio
n
o
p
timiz
a
tio
n
o
f d
ee
p
le
a
r
n
in
g
…
(
N
o
o
r
F
a
d
z
ila
h
R
a
z
a
li
)
2823
ab
n
o
r
m
alities
,
a
GA
-
C
NN
m
o
d
el
ac
h
ie
v
ed
a
n
ac
cu
r
ac
y
o
f
9
8
.
5
%
b
y
o
p
tim
ally
ex
t
r
ac
tin
g
f
ea
tu
r
es
f
r
o
m
s
eg
m
en
ted
r
eg
i
o
n
s
[
3
0
]
.
An
o
t
h
er
f
ea
tu
r
e
s
elec
tio
n
o
p
tim
izatio
n
m
eth
o
d
was
ap
p
lied
u
s
in
g
m
u
lti
-
o
b
jectiv
e
im
p
r
o
v
e
d
an
t
co
lo
n
y
o
p
tim
i
za
tio
n
(
AC
O)
b
ased
o
n
co
r
r
elatio
n
co
ef
f
icien
t,
wh
ich
s
h
o
wed
im
p
r
o
v
ed
class
if
icatio
n
r
ates
wh
en
ap
p
lied
to
m
ac
h
i
n
e
lear
n
in
g
class
if
ier
s
[
1
0
]
.
An
o
th
er
f
ea
t
u
r
e
s
elec
tio
n
o
p
tim
izatio
n
u
tili
zin
g
th
e
ch
a
o
tic
-
cr
o
w
-
s
e
ar
ch
o
p
tim
izatio
n
alg
o
r
ith
m
was
also
em
p
lo
y
ed
to
o
p
tim
ize
ac
r
o
s
s
m
u
ltip
le
m
am
m
o
g
r
a
m
d
atab
ases
,
r
esu
ltin
g
in
en
h
an
ce
d
o
v
er
all
ac
cu
r
ac
y
p
er
f
o
r
m
an
ce
[
3
1
]
.
W
eig
h
ted
av
er
ag
e
g
r
av
itatio
n
al
s
ea
r
ch
al
g
o
r
ith
m
s
wer
e
u
s
ed
as
f
ea
tu
r
e
s
ele
ctio
n
to
d
eter
m
i
n
e
th
e
b
est
tex
tu
r
al
m
am
m
o
g
r
am
f
ea
tu
r
e
p
r
o
d
u
cin
g
g
o
o
d
r
esu
lt
s
[
1
9
]
.
I
n
an
o
th
e
r
s
tu
d
y
,
v
ar
io
u
s
m
etah
eu
r
is
tic
o
p
tim
izatio
n
m
eth
o
d
s
,
in
clu
d
i
n
g
GA,
p
ar
ticle
s
war
m
o
p
tim
izatio
n
(
PS
O)
an
d
a
r
tific
ial
b
ee
co
l
o
n
y
(
AB
C
)
alg
o
r
ith
m
s
,
wer
e
c
o
m
p
ar
ed
ag
ain
s
t
a
n
o
v
el
E
b
o
la
o
p
tim
izatio
n
s
e
ar
ch
alg
o
r
ith
m
(
E
OSA)
f
o
r
C
NN
h
y
p
er
p
ar
am
eter
o
p
tim
izatio
n
[
1
1
]
,
wh
ich
d
em
o
n
s
tr
ated
im
p
r
o
v
ed
p
er
f
o
r
m
an
ce
b
y
im
p
lem
e
n
tin
g
t
h
e
d
is
ea
s
e's
s
u
s
ce
p
tib
le
-
in
f
ec
tio
n
-
r
ec
o
v
er
y
m
o
d
el.
Op
tim
izatio
n
o
f
th
e
m
am
m
o
g
r
am
p
r
e
-
p
r
o
ce
s
s
in
g
s
eg
m
en
t
atio
n
u
tili
zin
g
th
e
a
d
ap
tiv
e
Ker
n
el
-
B
ased
Fu
zz
y
C
u
ck
o
o
Sear
ch
Op
tim
izatio
n
C
lu
s
ter
in
g
is
also
in
tr
o
d
u
ce
d
,
with
f
u
zz
y
-
c
-
m
ea
n
s
clu
s
ter
in
g
p
r
o
b
lem
s
,
im
p
r
o
v
in
g
th
e
Dice
s
im
ilar
ity
in
d
ex
o
n
s
ev
er
al
m
am
m
o
g
r
am
im
a
g
e
test
s
[
3
2
]
.
An
o
th
er
s
tu
d
y
b
y
[
3
3
]
o
p
tim
ized
th
eir
class
if
ier
b
y
u
s
in
g
f
ir
e
f
ly
b
in
a
r
y
g
r
ey
o
p
tim
izatio
n
an
d
m
o
th
f
lam
e
lio
n
o
p
tim
izatio
n
in
a
n
en
s
em
b
le
m
o
d
el
to
class
if
y
th
e
I
Nb
r
ea
s
t
d
ataset.
I
n
s
u
m
m
ar
y
,
th
e
o
p
tim
izatio
n
p
r
o
ce
s
s
es
in
th
ese
s
tu
d
ies
we
r
e
ap
p
lied
af
ter
t
h
e
ex
tr
ac
ted
f
ea
tu
r
es,
an
d
th
ey
d
i
d
n
o
t
d
ir
ec
tly
im
p
ac
t
th
e
tr
ain
in
g
p
ath
o
f
th
e
n
etwo
r
k
,
wh
ic
h
led
to
th
e
ted
io
u
s
p
r
o
ce
s
s
o
f
d
eter
m
in
in
g
wh
ich
ty
p
es
o
f
o
p
tim
izatio
n
alg
o
r
it
h
m
s
s
u
ited
th
e
g
iv
en
task
.
Mo
r
eo
v
er
,
m
o
s
t
o
f
th
e
o
p
tim
izatio
n
alg
o
r
ith
m
s
ar
e
n
ewe
r
an
d
less
estab
lis
h
ed
,
r
eq
u
ir
in
g
ex
ten
s
iv
e
k
n
o
wled
g
e
o
f
th
e
m
eth
o
d
t
o
b
e
ap
p
lied
to
o
t
h
er
task
s
as
well.
Ho
wev
er
,
th
is
s
h
o
ws
th
at
co
m
b
in
in
g
o
p
tim
izatio
n
tec
h
n
iq
u
es
an
d
C
NN
ar
ch
itectu
r
es
h
as
d
em
o
n
s
tr
ated
s
ig
n
if
ican
t
p
o
ten
tial
in
im
p
r
o
v
in
g
th
e
ef
f
ec
tiv
en
ess
o
f
m
am
m
o
g
r
am
-
b
ased
C
AD
f
o
r
d
iag
n
o
s
in
g
b
r
ea
s
t
ca
n
ce
r
.
Su
b
s
tan
tial
en
h
a
n
ce
m
en
ts
in
ac
cu
r
ac
y
an
d
r
eliab
ilit
y
a
r
e
ac
h
iev
ed
b
y
u
tili
zin
g
th
e
ad
v
an
tag
es
o
f
o
p
tim
izin
g
d
if
f
er
en
t
p
h
ases
o
f
t
h
e
ar
ch
itectu
r
e.
C
o
n
tin
u
e
d
p
r
o
g
r
ess
in
th
is
f
ield
,
p
ar
ticu
lar
ly
wh
en
ap
p
lied
with
in
th
e
C
NN
ar
ch
i
tectu
r
e
,
h
as
th
e
p
o
ten
tial
to
r
ev
o
l
u
tio
n
ize
t
h
e
id
e
n
tific
a
tio
n
an
d
tr
ea
tm
e
n
t
o
f
b
r
ea
s
t c
an
ce
r
,
r
esu
ltin
g
in
i
m
p
r
o
v
e
d
p
atien
t
o
u
tco
m
es.
I
n
co
n
clu
s
io
n
,
d
ev
el
o
p
m
en
ts
o
f
C
NN
-
b
ased
an
aly
s
is
o
f
m
am
m
o
g
r
am
-
r
elate
d
im
ag
es
h
av
e
h
ig
h
lig
h
ted
th
e
im
p
o
r
tan
ce
o
f
u
s
in
g
d
if
f
er
e
n
t
n
o
n
-
lin
ea
r
ac
tiv
atio
n
f
u
n
cti
o
n
s
in
im
p
r
o
v
in
g
th
e
n
etwo
r
k
’
s
ca
p
ac
ity
to
ex
t
r
ac
t
co
m
p
lex
in
f
o
r
m
atio
n
.
Ho
we
v
er
,
n
o
v
el
n
o
n
-
lin
ea
r
m
o
d
els
s
u
ch
as
t
h
o
s
e
d
e
m
o
n
s
tr
ated
b
y
[
2
1
]
,
[
2
2
]
a
r
e
im
p
lem
en
ted
t
h
eo
r
etica
lly
an
d
h
av
e
n
o
t
b
ee
n
co
n
d
u
cte
d
o
n
c
r
itical
r
ea
l
-
wo
r
ld
d
atasets
s
u
ch
as
m
ed
ical
im
ag
es.
Oth
er
wis
e,
it
d
o
cu
m
en
ted
th
e
n
ee
d
a
n
d
its
im
p
o
r
tan
ce
well
a
n
d
p
r
o
v
i
d
ed
b
ase
k
n
o
wled
g
e
to
th
e
m
ath
em
atica
l
d
ev
elo
p
m
e
n
t to
im
p
r
o
v
e
t
h
e
n
etwo
r
k
’
s
p
er
f
o
r
m
an
ce
.
3.
M
E
T
H
O
D
T
h
is
s
tu
d
y
p
r
o
p
o
s
ed
a
m
eth
o
d
to
im
p
r
o
v
e
t
h
e
f
in
al
f
ea
tu
r
e
ex
tr
ac
tio
n
p
ath
b
y
in
tr
o
d
u
cin
g
a
GA
-
b
ased
o
p
tim
izatio
n
o
n
th
e
last
R
eL
U
lay
er
in
th
e
C
NN
f
ea
tu
r
e
ex
tr
ac
tio
n
p
ath
way
.
T
h
e
m
o
d
el
im
p
r
o
v
em
en
t
is
d
em
o
n
s
tr
ated
b
y
c
o
m
p
ar
in
g
th
e
m
o
d
el’
s
p
er
f
o
r
m
an
ce
b
ef
o
r
e
an
d
af
ter
o
p
tim
izatio
n
an
d
v
alid
ated
o
n
two
m
am
m
o
g
r
a
m
d
atasets
to
p
r
o
v
e
th
e
h
y
p
o
th
esis
o
f
th
is
s
tu
d
y
.
T
h
is
s
tu
d
y
u
s
es
m
am
m
o
g
r
a
m
im
ag
es
as
in
p
u
t
to
an
o
p
tim
ized
C
NN
p
ath
way
f
o
r
b
est
f
ea
tu
r
e
ex
tr
ac
tio
n
b
ased
o
n
m
etr
ics
p
er
f
o
r
m
an
ce
.
Fir
s
t,
th
e
im
ag
es
wer
e
pr
e
-
p
r
o
ce
s
s
ed
to
o
b
tain
th
e
m
ass
lo
ca
tio
n
,
an
d
th
e
cu
s
to
m
ized
o
p
tim
ized
C
NN
u
s
in
g
GA
was
ap
p
lied
d
u
r
in
g
f
ea
tu
r
e
e
x
tr
ac
tio
n
.
Fin
ally
,
th
e
m
ass
was
class
if
ied
as
b
en
ig
n
o
r
m
alig
n
a
n
t.
All
m
o
d
el
d
ev
e
lo
p
m
en
t
is
d
o
n
e
i
n
a
wo
r
k
s
tatio
n
eq
u
ip
p
ed
with
a
n
I
n
tel®
C
o
r
e
™
i7
-
1
2
7
0
0
2
.
1
GHz
,
GPU
g
r
ap
h
ic
ca
r
d
o
f
1
2
GB
an
d
R
AM
3
2
GB
,
th
r
o
u
g
h
MA
T
L
AB
,
Natick
,
M
ass
ac
h
u
s
etts
:
T
h
e
Ma
th
wo
r
k
s
I
n
c.
p
latf
o
r
m
.
T
h
e
f
o
llo
win
g
s
u
b
s
ec
tio
n
s
d
is
cu
s
s
th
e
d
etails o
f
th
e
p
r
o
p
o
s
ed
m
e
th
o
d
.
3
.
1
.
Da
t
a
s
et
prepa
ra
t
io
n
T
wo
m
am
m
o
g
r
am
d
atasets
ar
e
em
p
lo
y
e
d
in
th
is
s
tu
d
y
t
o
v
alid
ate
th
e
p
r
o
p
o
s
ed
m
eth
o
d
.
First,
an
estab
lis
h
ed
,
p
u
b
licly
av
ailab
le
d
ig
itized
m
am
m
o
g
r
am
o
f
I
N
b
r
ea
s
t
wid
ely
u
s
ed
in
m
u
ltip
le
s
tu
d
ies
[
1
8
]
,
[
3
1
]
,
[
3
3
]
–
[
3
6
]
is
o
b
tain
ed
f
r
o
m
its
s
o
u
r
ce
[
3
7
]
.
T
h
e
im
a
g
es
wer
e
co
llected
f
r
o
m
Un
iv
er
s
id
ad
e
d
e
Po
r
to
,
Po
r
tu
g
al,
f
r
o
m
C
en
tr
o
Ho
s
p
italar
d
e
S.
J
o
ão
B
r
ea
s
t
C
en
ter
,
co
n
tain
in
g
1
1
5
ca
s
es
o
f
b
r
ea
s
t
lesi
o
n
s
.
On
ly
th
e
m
ass
ca
s
es
ar
e
s
elec
ted
in
th
is
s
tu
d
y
,
r
esu
ltin
g
in
1
1
2
im
ag
es
with
b
o
th
b
en
ig
n
an
d
m
alig
n
an
t
m
ass
es.
T
h
e
I
Nb
r
ea
s
t
d
ataset
is
co
m
p
r
is
ed
o
f
2
6
.
8
% d
en
s
er
b
r
ea
s
ts
[
3
7
]
.
Ad
d
itio
n
ally
,
th
is
s
tu
d
y
’
s
s
ec
o
n
d
m
a
m
m
o
g
r
am
d
ataset,
ter
m
ed
t
h
e
I
PP
T
-
m
am
m
o
,
was
r
etr
o
s
p
ec
ti
v
ely
co
llected
at
t
h
e
I
n
s
titu
t
Per
g
ig
ian
d
an
Per
u
b
ata
n
T
er
m
aj
u
(
I
PP
T
)
,
Un
iv
er
s
iti
Sain
s
Ma
lay
s
ia
B
er
tam
,
Ma
la
y
s
ia,
ac
co
r
d
in
g
to
r
elev
a
n
t
la
ws
th
r
o
u
g
h
t
h
e
ap
p
r
o
v
al
o
f
th
e
in
s
titu
tio
n
’
s
h
u
m
an
eth
ics
r
esear
ch
co
m
m
ittee
f
o
r
th
is
r
esear
ch
s
tu
d
y
(
Fil
e:
J
E
Pe
M/2
1
0
9
0
6
2
4
y
ea
r
: 2
0
2
1
)
.
I
t
c
o
n
tain
ed
2
0
0
b
r
ea
s
t
m
ass
lesi
o
n
ca
s
es
with
an
n
o
tated
ab
n
o
r
m
alities
b
y
e
x
p
er
t
r
ad
i
o
lo
g
is
ts
an
d
co
m
p
r
is
ed
6
0
%
d
en
s
e
b
r
ea
s
t
d
e
n
s
ity
.
B
o
th
d
atas
ets
in
clu
d
e
g
r
o
u
n
d
-
tr
u
th
in
f
o
r
m
atio
n
s
u
ch
as
th
e
m
ass
es'
R
an
g
e
o
f
I
n
ter
est
(
R
OI
)
,
b
r
ea
s
t
d
en
s
ity
lev
el,
an
d
ty
p
es o
f
a
b
n
o
r
m
ality
.
T
h
e
im
ag
es
co
n
tain
e
d
with
in
b
o
th
d
atasets
wer
e
ap
p
lied
wit
h
n
o
r
m
aliza
tio
n
an
d
o
t
h
er
m
o
r
p
h
o
lo
g
ical
p
r
o
ce
s
s
in
g
tech
n
i
q
u
es.
T
h
is
is
d
o
n
e
to
ex
t
r
ac
t
o
n
l
y
th
e
b
r
ea
s
t
r
eg
io
n
to
elim
in
ate
u
n
n
ec
ess
ar
y
ar
tef
ac
ts
[
2
0
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
3
,
J
u
n
e
20
25
:
2
8
2
0
-
2
8
3
3
2824
An
im
ag
e
en
h
an
ce
m
en
t
p
r
o
c
e
s
s
b
ased
o
n
h
is
to
g
r
am
eq
u
aliz
atio
n
f
r
o
m
o
u
r
p
r
ev
io
u
s
s
tu
d
y
[
3
8
]
is
ap
p
lied
,
an
d
th
e
p
r
o
ce
s
s
is
d
ep
icted
in
Fig
u
r
e
2
.
Fig
u
r
e
2
.
Pre
-
p
r
o
ce
s
s
in
g
o
f
th
e
in
p
u
t im
ag
es
T
h
e
m
eth
o
d
is
p
er
f
o
r
m
e
d
b
y
a
d
ju
s
tin
g
th
e
co
n
tr
ast
b
ased
o
n
th
e
b
r
ea
s
t
d
en
s
ity
le
v
el
th
at
is
s
u
b
jecte
d
to
th
e
o
b
ject
d
etec
tio
n
m
eth
o
d
.
T
o
less
en
th
e
ef
f
ec
t
o
f
o
v
e
r
f
itti
n
g
d
u
e
t
o
lo
w
tr
ain
ab
le
i
m
ag
es
b
y
u
s
in
g
t
h
is
d
ataset
f
o
r
th
e
d
ee
p
lear
n
in
g
a
p
p
r
o
ac
h
,
g
e
o
m
etr
ical
a
u
g
m
en
tat
io
n
,
s
u
ch
as
r
o
tatio
n
r
an
g
es
o
f
-
2
0
to
3
5
0
d
eg
r
ee
s
,
f
lip
p
in
g
,
an
d
r
an
d
o
m
s
ca
lin
g
o
f
1
.
0
–
1
.
3
m
a
g
n
itu
d
e
,
ar
e
ap
p
lied
to
th
e
im
ag
es
to
in
cr
ea
s
e
th
e
tr
ain
in
g
n
u
m
b
er
an
d
im
p
r
o
v
e
m
o
d
el
c
o
n
v
e
r
g
en
ce
.
T
h
e
d
ataset
is
d
iv
i
d
ed
i
n
to
tr
ain
in
g
,
v
alid
atio
n
,
an
d
test
in
g
d
atasets
at
a
r
atio
o
f
6
5
:
1
5
:2
0
to
test
th
e
p
r
o
p
o
s
e
d
m
eth
o
d
.
3
.
2
.
O
ptimized
CNN
f
ea
t
ure
ex
t
ra
ct
io
n
Nex
t,
th
e
m
ass
in
p
u
t
im
ag
es
a
r
e
f
ed
in
to
th
e
C
NN
f
ea
tu
r
e
ex
tr
ac
tio
n
n
etwo
r
k
th
at
h
as
b
ee
n
m
o
d
if
ied
with
th
e
o
p
tim
izatio
n
p
r
o
ce
s
s
p
r
o
p
o
s
ed
in
th
is
s
tu
d
y
,
as
d
ep
ic
ted
in
Fig
u
r
e
3
.
Fig
u
r
e
3
(
a)
illu
s
tr
ates
th
e
p
r
o
p
o
s
ed
o
v
er
all
d
esig
n
f
o
r
th
e
C
NN
tr
ain
in
g
p
at
h
w
ay
f
o
r
f
ea
tu
r
e
ex
tr
ac
tio
n
an
d
class
if
icatio
n
.
Me
a
n
wh
ile,
Fig
u
r
e
3
(
b
)
s
h
o
ws th
e
m
o
d
if
icatio
n
m
ad
e
o
n
th
e
n
o
n
-
lin
ea
r
ac
tiv
atio
n
la
y
er
f
o
r
th
e
o
p
tim
izatio
n
.
Fig
u
r
e
3
.
T
h
e
ar
ch
itectu
r
e
o
f
t
h
e
p
r
o
p
o
s
ed
m
et
h
o
d
o
l
o
g
y
.
(
a)
C
NN
tr
ain
in
g
an
d
(
b
)
GA
o
p
ti
m
izatio
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Gen
etic
a
lg
o
r
ith
m
-
a
d
a
p
ted
a
ctiva
tio
n
fu
n
ctio
n
o
p
timiz
a
tio
n
o
f d
ee
p
le
a
r
n
in
g
…
(
N
o
o
r
F
a
d
z
ila
h
R
a
z
a
li
)
2825
3
.
2
.
1
.
CNN
t
ra
ini
ng
pa
t
hwa
y
I
n
th
e
f
ea
tu
r
e
e
x
tr
ac
tio
n
p
h
ase,
p
o
o
lin
g
la
y
er
s
an
d
ac
tiv
ati
o
n
f
u
n
ctio
n
s
ar
e
u
tili
ze
d
to
im
p
lem
en
t
s
p
atial
d
o
wn
s
am
p
lin
g
an
d
i
n
tr
o
d
u
ce
n
o
n
-
lin
ea
r
ity
to
t
h
e
m
o
d
el.
T
h
e
esti
m
atio
n
o
f
t
h
e
o
p
tim
al
d
ec
im
al
p
r
o
b
a
b
ilit
y
th
at
a
g
iv
en
class
b
elo
n
g
ed
to
a
m
em
b
er
o
f
a
class
o
cc
u
r
s
o
n
th
e
So
f
tMa
x
f
u
n
cti
o
n
lay
er
d
u
r
in
g
t
h
e
class
if
icatio
n
p
h
ase.
T
o
test
t
h
e
p
r
o
p
o
s
ed
m
eth
o
d
,
th
is
s
tu
d
y
em
p
lo
y
e
d
two
estab
lis
h
ed
p
r
e
-
tr
ain
ed
C
NN
n
etwo
r
k
s
o
f
R
esNet5
0
[
4
]
an
d
Sh
u
f
f
leNe
t
[
3
9
]
.
B
o
th
n
etwo
r
k
s
wer
e
tr
ai
n
ed
o
n
a
m
ass
iv
e
I
m
ag
eNe
t
d
ata
b
ase
to
ex
tr
ac
t
a
n
d
lear
n
h
ig
h
er
a
n
d
s
em
an
tic
f
ea
tu
r
es,
in
clu
d
in
g
p
atter
n
s
in
im
a
g
es
wh
er
e
t
h
e
lear
n
ed
weig
h
ts
ar
e
r
eu
s
ed
as
p
ar
t
o
f
th
e
tr
an
s
f
er
lear
n
in
g
p
r
o
ce
s
s
.
T
h
e
two
C
NNs
s
elec
ted
in
th
is
s
tu
d
y
r
ep
r
esen
t
lig
h
tweig
h
t
C
NN
f
o
r
Sh
u
f
f
leNe
t
an
d
h
ea
v
y
wei
g
h
t
C
NN
f
o
r
R
esNet5
0
with
1
.
2
m
illi
o
n
an
d
2
5
.
5
m
illi
o
n
t
r
ain
in
g
p
a
r
am
eter
s
,
r
esp
ec
tiv
ely
.
B
o
th
ar
ch
itectu
r
es
also
d
if
f
e
r
in
ter
m
s
o
f
s
p
ec
if
ic
r
eq
u
ir
em
en
ts
o
f
s
p
ec
if
ic
task
s
,
s
u
ch
as
co
m
p
u
tatio
n
al
r
eso
u
r
ce
s
an
d
m
o
d
el
s
ize
co
n
s
tr
ain
ts
an
d
u
n
i
q
u
ely
r
ep
r
esen
t
d
if
f
er
e
n
t
ty
p
es
o
f
p
r
e
-
tr
ai
n
ed
C
NN
ar
ch
itectu
r
es
estab
lis
h
ed
in
th
e
n
eu
r
al
n
etwo
r
k
f
ield
.
T
o
u
tili
ze
th
e
p
r
e
-
t
r
ain
ed
n
etwo
r
k
s
with
th
e
p
r
o
p
o
s
e
d
m
eth
o
d
,
b
o
th
C
NN
m
o
d
els’
f
u
lly
co
n
n
ec
ted
lay
er
s
ar
e
r
e
m
o
d
ell
ed
to
o
u
tp
u
t
two
class
es,
b
en
ig
n
an
d
m
alig
n
an
t,
an
d
th
e
tr
ain
i
n
g
p
ar
am
eter
s
ar
e
s
et
th
e
s
am
e
as
lis
ted
in
T
a
b
le
2
.
No
te
th
at
th
e
s
ec
o
n
d
I
P
PT
-
m
am
m
o
d
ataset
r
eq
u
ir
ed
a
h
ig
h
er
m
ax
im
u
m
ep
o
ch
an
d
m
in
i
-
b
atc
h
s
ize
n
u
m
b
er
f
o
r
tr
ain
in
g
as
it
co
n
tain
ed
la
r
g
er
im
ag
e
s
am
p
les
th
an
th
e
I
Nb
r
ea
s
t d
ataset.
T
ab
le
2
.
Hy
p
e
r
p
ar
am
ete
r
s
ettin
g
s
f
o
r
C
NN
tr
ain
in
g
f
o
r
I
Nb
r
ea
s
t a
n
d
I
PP
T
-
m
am
m
o
d
ataset
s
H
y
p
e
r
p
a
r
a
me
t
e
r
s
e
t
t
i
n
g
S
e
t
v
a
l
u
e
/
d
a
t
a
s
e
t
I
N
b
r
e
a
st
IPPT
-
mammo
M
a
x
i
m
u
m
e
p
o
c
h
50
80
M
i
n
i
-
b
a
t
c
h
s
i
z
e
80
1
2
8
Le
a
r
n
i
n
g
r
a
t
e
0
.
0
0
1
S
o
l
v
e
r
A
d
a
p
t
i
v
e
m
o
me
n
t
e
s
t
i
m
a
t
i
o
n
(
A
d
a
m)
L2
R
e
g
u
l
a
r
i
z
a
t
i
o
n
0
.
0
0
0
1
3
.
2
.
2
.
G
enet
ic
a
lg
o
rit
hm
o
ptimiza
t
io
n pa
t
hwa
y
I
n
th
is
s
tu
d
y
,
th
e
p
r
o
p
o
s
ed
o
p
tim
izatio
n
p
ath
way
,
as
in
Fig
u
r
e
3
,
is
co
n
d
u
cted
o
n
th
e
last
ac
tiv
atio
n
f
u
n
ctio
n
d
u
r
in
g
th
e
f
ea
tu
r
e
ex
t
r
ac
tio
n
s
tag
e.
Usi
n
g
th
e
last
ac
tiv
atio
n
f
u
n
ctio
n
to
b
e
m
o
d
i
f
ied
allo
ws
th
e
d
ef
au
lt
ex
tr
ac
tio
n
p
r
o
ce
s
s
o
n
th
e
u
p
p
e
r
lev
el
o
f
th
e
n
etwo
r
k
to
s
h
ap
e
th
e
f
in
al
d
ec
is
io
n
b
o
u
n
d
ar
y
o
u
tp
u
t
f
o
r
th
e
s
p
ec
if
ic
task
.
T
h
e
last
f
ea
tu
r
e
ex
tr
ac
ti
o
n
s
tag
e’
s
R
eL
U
ac
tiv
atio
n
f
u
n
ctio
n
,
lay
er
s
1
7
3
a
n
d
1
6
8
f
o
r
R
esNet5
0
an
d
Sh
u
f
f
leNe
t,
was r
ep
lace
d
with
cu
s
to
m
ized
R
eL
U
an
d
leak
y
R
eL
U
ac
tiv
atio
n
f
u
n
ctio
n
s
.
T
h
e
cu
s
to
m
ized
ac
tiv
atio
n
f
u
n
ctio
n
s
ar
e
b
u
ilt
to
r
ec
eiv
e
a
n
ad
ap
ta
b
le
p
a
r
am
eter
v
alu
e
s
et
as
th
e
th
r
esh
o
ld
,
±
,
th
at
co
n
tr
o
ls
th
e
p
o
s
itio
n
o
f
th
e
ac
tiv
atio
n
g
r
ap
h
to
b
e
ac
tiv
ated
b
y
s
h
if
tin
g
t
h
e
g
r
ap
h
f
r
o
m
its
d
ef
au
lt
p
o
s
itio
n
o
f
(
0
,
0
)
to
th
e
o
p
tim
ized
(
±
,
0
)
o
n
th
e
x
-
ax
is
.
T
o
o
p
tim
ize
th
e
th
r
esh
o
ld
,
GA,
a
m
etah
eu
r
is
tic
o
p
tim
izatio
n
a
p
p
r
o
ac
h
b
ased
o
n
n
atu
r
al
g
en
etic
s
elec
tio
n
b
ased
o
n
th
e
s
u
r
v
iv
al
o
f
t
h
e
f
ittes
t
co
n
ce
p
t
to
f
in
d
th
e
b
est
s
o
lu
tio
n
to
a
p
r
o
b
lem
,
was
em
p
lo
y
ed
[
1
3
]
.
T
h
e
b
est
±
is
d
eter
m
in
ed
th
r
o
u
g
h
g
en
e
tic
o
p
er
ato
r
s
o
f
cr
o
s
s
o
v
er
an
d
m
u
tatio
n
b
ased
o
n
th
e
b
est
co
m
b
in
atio
n
o
f
t
h
e
in
itialized
p
ar
en
t
ch
r
o
m
o
s
o
m
e
as
p
ar
am
eter
s
s
elec
ted
at
th
e
s
tar
t
o
f
th
e
f
ir
s
t
p
o
p
u
latio
n
tr
ai
n
in
g
.
T
h
e
s
ea
r
ch
s
p
ac
e
is
s
et
f
o
r
th
e
t
h
r
esh
o
ld
,
±
with
u
p
p
er
an
d
lo
wer
b
o
u
n
d
s
et
to
[
-
1
1
]
.
T
h
e
GA
is
ev
alu
ated
with
th
e
v
alid
atio
n
d
ataset
d
u
r
in
g
th
e
C
NN
tr
ain
in
g
as
th
e
o
p
tim
al
s
o
lu
tio
n
o
f
th
e
f
itn
e
s
s
f
u
n
ctio
n
.
Giv
en
th
e
p
r
o
b
a
b
ilit
y
,
P
,
o
f
d
r
awin
g
in
d
iv
i
d
u
al,
i
,
in
th
e
in
itia
l
p
o
p
u
latio
n
wh
er
e
it
is
d
ef
in
e
d
b
ased
o
n
th
e
f
it
n
ess
f
u
n
ctio
n
F
(
x,
y)
wh
e
r
e
x
a
n
d
y
ar
e
p
ar
a
m
eter
s
d
ef
in
in
g
th
e
i
ch
ar
ac
ter
is
tics
o
r
g
en
o
ty
p
e
wh
en
GA
is
lo
o
k
in
g
f
o
r
o
p
tim
al
v
alu
e
in
a
s
ea
r
ch
s
p
ac
e
w
ith
in
t
h
e
to
tal
n
u
m
b
er
,
N,
o
f
i
in
a
p
o
p
u
latio
n
,
d
e
f
in
ed
as
in
(
1
)
:
=
|
(
,
)
∑
(
,
)
|
(1
)
T
h
e
C
NN
tr
ain
in
g
u
n
d
er
g
o
es
f
o
r
war
d
an
d
b
ac
k
wa
r
d
p
r
o
p
ag
atio
n
,
as
th
e
b
est
±
is
u
p
d
ated
t
h
r
o
u
g
h
th
e
cu
s
to
m
ized
ac
tiv
atio
n
f
u
n
ctio
n
lay
er
o
n
ea
ch
m
i
n
i
-
b
atc
h
tr
ain
in
g
.
I
n
th
is
s
tu
d
y
,
t
h
e
s
ettin
g
f
o
r
th
e
GA
is
co
n
d
itio
n
e
d
to
allo
w
elite
ch
r
o
m
o
s
o
m
e,
m
u
tatio
n
,
an
d
cr
o
s
s
o
v
er
in
ter
m
ed
iate
in
th
e
s
elec
tio
n
p
r
o
ce
s
s
to
allo
w
a
d
iv
er
s
e
p
o
p
u
latio
n
an
d
p
r
o
v
i
d
e
a
b
ala
n
ce
b
etwe
en
e
x
p
lo
r
at
io
n
an
d
ex
p
l
o
itatio
n
to
f
in
d
t
h
e
o
p
tim
al
s
o
lu
tio
n
.
T
h
e
n
u
m
b
er
o
f
in
d
iv
id
u
als,
i
,
in
th
e
p
o
p
u
latio
n
,
wer
e
elite
a
n
d
p
r
eser
v
ed
in
ea
ch
g
en
e
r
atio
n
is
s
et
t
o
th
e
to
p
2
in
d
iv
id
u
als with
th
e
b
est f
itn
es
s
s
co
r
e.
T
h
e
p
r
o
p
o
s
ed
m
o
d
if
icatio
n
is
m
ad
e
o
n
two
estab
lis
h
ed
ac
tiv
atio
n
f
u
n
ctio
n
s
o
f
R
eL
U
an
d
l
ea
k
y
R
eL
U.
T
h
e
p
r
o
b
a
b
le
th
r
esh
o
ld
s
h
if
ts
in
th
e
n
eg
ativ
e
o
r
p
o
s
itiv
e
n
eu
r
o
n
ac
tiv
ity
r
an
g
e
in
th
e
x
-
ax
is
ar
e
ex
p
ec
ted
to
h
elp
ac
tiv
ate
th
e
d
ea
d
n
eu
r
o
n
th
at
is
p
o
s
s
ib
ly
d
ea
d
in
th
e
d
ef
au
lt
s
ettin
g
s
b
ef
o
r
e
th
e
p
r
o
p
o
s
ed
o
p
tim
izatio
n
.
T
h
is
is
esp
ec
ially
tr
u
e
at
th
e
en
d
o
f
t
h
e
ex
tr
ac
tio
n
s
tag
e
to
allev
iat
e
th
e
d
ea
d
n
eu
r
o
n
p
r
o
b
lem
s
ca
u
s
ed
b
y
t
h
e
u
p
p
er
-
lev
el
R
eL
U,
wh
er
e
a
d
ir
ec
t
e
v
alu
atio
n
ca
n
b
e
m
ad
e
th
r
o
u
g
h
th
e
f
itn
ess
f
u
n
ctio
n
f
o
r
o
p
ti
m
ized
p
er
f
o
r
m
an
ce
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
3
,
J
u
n
e
20
25
:
2
8
2
0
-
2
8
3
3
2826
Fig
u
r
e
4
s
h
o
ws th
e
p
o
s
s
ib
le
lo
ca
tio
n
o
f
θ
±
t
o
n
x
-
ax
is
an
d
its
s
u
b
s
eq
u
en
t lin
ea
r
eq
u
ati
o
n
g
r
a
p
h
f
o
r
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
n
th
e
o
r
ig
in
al
r
e
d
g
r
a
p
h
s
f
o
r
Fig
u
r
e
4
(
a
)
R
eL
U
an
d
Fig
u
r
e
4
(
b
)
leak
y
R
eL
U,
s
h
if
te
d
to
b
lu
e
a
n
d
y
ello
w
g
r
ap
h
s
r
esp
ec
tiv
ely
,
ac
co
r
d
in
g
to
th
e
o
p
tim
ized
θ
±
t
v
alu
e,
with
p
o
s
itiv
e
an
d
n
eg
ativ
e
s
lo
p
es
α
an
d
β
r
etain
ed
at
th
eir
d
ef
au
lt
v
alu
es.
(
a)
(
b
)
Fig
u
r
e
4
.
Activ
atio
n
f
u
n
ctio
n
s
g
r
ap
h
s
o
f
.
(
a
)
R
eL
U
an
d
(
b
)
l
ea
k
y
R
eL
U.
T
h
e
o
r
ig
in
al
R
eL
U
an
d
leak
y
R
eL
U
(
r
ed
)
,
p
o
s
s
ib
le
GAa
-
R
eL
U
an
d
GAa
-
L
ea
k
y
R
eL
U
o
n
t
h
e
n
e
g
ativ
e
th
r
esh
o
ld
(
b
lu
e
)
,
an
d
p
o
s
itiv
e
th
r
esh
o
ld
(
o
r
an
g
e)
,
wh
er
e
θ
i
is
th
e
o
r
ig
in
a
l th
r
esh
o
ld
p
o
s
itio
n
a.
Gen
etic
alg
o
r
ith
m
-
a
d
ap
ted
r
ec
tifie
r
lin
ea
r
u
n
it
(
GAa
-
R
eL
U)
Fig
u
r
e
4
(
a)
s
h
o
ws
th
e
R
eL
U
g
r
ap
h
s
an
d
m
o
d
if
icatio
n
s
p
o
s
s
ib
le
d
u
e
to
th
e
p
r
o
p
o
s
ed
m
eth
o
d
.
No
te
th
at
th
e
o
r
ig
in
al
th
r
esh
o
ld
was
f
ix
e
d
at
=
0
an
d
th
e
s
lo
p
e
β
=1
v
al
u
es
f
o
r
n
eu
r
o
n
ac
tiv
atio
n
≥
(
p
o
s
itiv
e
r
eg
io
n
)
r
etain
ed
f
o
r
all
g
r
a
p
h
s
.
T
h
e
o
r
i
g
in
al
R
eL
U
ac
tiv
atio
n
f
u
n
ctio
n
f(
x)
ReLU
is
as in
(
2
)
.
(
)
=
(
0
,
)
;
=
{
0
,
<
,
≥
,
=
(
0
,
0
)
,
=
1
(2)
I
n
th
e
ca
s
e
o
f
th
e
o
r
ig
in
al
R
eL
U
(
r
ed
)
o
f
Fig
u
r
e
4
(
a)
,
th
e
th
r
e
s
h
o
ld
o
f
is
alwa
y
s
=
(
0
,
0
)
p
o
s
itio
n
,
in
wh
ich
th
e
n
eu
r
o
n
will
b
e
ac
tiv
ated
wh
en
th
e
in
p
u
t
≥
.
T
h
e
s
im
p
licity
o
f
R
eL
U
r
elies
o
n
its
lin
ea
r
eq
u
atio
n
,
wh
ich
p
er
m
an
en
tl
y
d
ea
ctiv
ates
th
e
n
eu
r
o
n
wh
en
th
e
in
p
u
t
is
less
th
an
0
.
T
h
is
im
p
r
o
v
es
th
e
co
m
p
u
tatio
n
ab
ilit
y
to
tr
ain
m
u
ltip
le
lay
er
s
o
f
C
N
N
b
y
in
tr
o
d
u
ci
n
g
s
p
ar
s
ity
with
in
th
e
weig
h
t
u
p
d
ate
[
4
]
.
T
h
i
s
s
tu
d
y
p
r
o
p
o
s
ed
th
e
GAa
-
R
eL
U
ac
tiv
atio
n
f
u
n
ctio
n
,
r
e
p
lacin
g
th
e
f
in
al
R
eL
U
la
y
er
in
th
e
o
r
ig
i
n
al
C
NN
ar
ch
it
ec
tu
r
e,
to
allo
w
a
n
u
n
r
estrictiv
e,
alb
eit
co
n
tr
o
lled
th
r
esh
o
ld
f
o
r
th
e
R
eL
U
th
at
en
ab
les
weig
h
t
u
p
d
ates
f
r
o
m
th
e
n
eu
r
o
n
ac
tiv
atio
n
o
n
th
e
in
p
u
t
x
.
T
h
e
GAa
-
R
eL
U
f
u
n
ctio
n
(
)
−
is
as
in
(
3
)
,
with
its
p
o
s
s
ib
le
g
r
ap
h
s
in
Fig
u
r
e
4
(
b
)
,
wh
er
e
th
e
in
ter
s
ec
tio
n
C
is
o
b
t
ain
ed
au
t
o
m
atica
lly
b
y
r
etain
in
g
th
e
p
o
s
itiv
e
r
eg
io
n
s
lo
p
e
as =
1
an
d
n
eg
ativ
e
r
eg
io
n
s
lo
p
e
α
=
0
,
wh
ile
th
e
g
r
a
p
h
is
s
h
if
ted
b
y
±
v
alu
e
o
n
th
e
x
-
ax
is
o
p
tim
ized
th
r
o
u
g
h
th
e
GA
s
ea
r
ch
d
u
r
in
g
th
e
tr
ain
in
g
lo
o
p
.
(
)
−
=
(
0
,
+
±
)
;
=
{
±
,
<
±
+
,
≥
±
,
at
θ
i
=
(
±
,
0
)
,
=
1
(3
)
b.
Gen
etic
alg
o
r
ith
m
-
a
d
ap
ted
lea
k
y
r
ec
tifie
r
lin
ea
r
u
n
it
(
GAa
-
leak
y
R
eL
U)
Ad
d
itio
n
ally
,
th
e
p
r
o
p
o
s
ed
m
e
th
o
d
is
test
ed
o
n
an
o
th
er
estab
lis
h
ed
ac
tiv
atio
n
f
u
n
ctio
n
o
f
leak
y
R
eL
U
as
d
ep
icted
in
Fig
u
r
e
4
(
b
)
,
wh
e
r
e
th
e
o
r
ig
in
al
leak
y
R
eL
U
is
d
ev
elo
p
ed
to
im
p
r
o
v
e
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
R
eL
U
[
4
0
]
.
T
h
e
lea
k
y
R
eL
U
allo
ws
a
n
o
n
-
ze
r
o
g
r
ad
ie
n
t
to
ac
tiv
at
e
s
p
ec
if
ic
n
eu
r
o
n
s
an
d
im
p
r
o
v
e
th
e
d
ea
d
-
n
e
u
r
o
n
p
r
o
b
lem
in
th
e
R
eL
U
b
y
in
tr
o
d
u
cin
g
a
f
ix
ed
l
o
w
-
v
alu
e
c
o
ef
f
icien
t,
α
,
as
th
e
n
e
g
ativ
e
s
lo
p
e.
T
h
e
α
allo
ws
a
s
m
all
s
k
ew
s
lo
p
e
o
f
th
e
n
eg
at
iv
e
r
eg
io
n
,
th
u
s
‘
leak
ed
’
ac
tiv
ated
n
eu
r
o
n
p
r
ev
e
n
tin
g
d
ea
d
n
eu
r
o
n
p
r
o
b
lem
i
n
R
eL
U.
T
h
e
av
ailab
ilit
y
o
f
th
e
s
e
ac
tiv
ated
n
eu
r
o
n
s
allo
ws
th
e
p
r
eser
v
atio
n
o
f
im
p
o
r
tan
t
f
ea
tu
r
e
lear
n
in
g
a
n
d
im
p
r
o
v
es
p
er
f
o
r
m
a
n
ce
.
I
n
th
is
wo
r
k
,
we
tak
e
f
u
r
th
er
a
d
v
an
t
ag
e
o
f
leak
y
R
eL
U
an
d
im
p
le
m
en
t
o
u
r
p
r
o
p
o
s
ed
o
p
tim
izatio
n
m
eth
o
d
,
GAa
-
lea
k
y
R
eL
U.
I
n
th
is
s
tu
d
y
,
th
e
p
o
s
itiv
e
r
eg
io
n
s
lo
p
e
,
α
,
is
s
et
to
d
ef
au
lt
0
.
0
1
,
an
d
(
4
)
in
d
icate
s
th
e
o
r
ig
in
al
leak
y
R
eL
U
eq
u
atio
n
(
)
,
wh
er
ea
s
(
5
)
in
d
i
ca
tes
th
e
GAa
-
leak
y
R
eL
U
f
u
n
ctio
n
(
)
−
wh
en
ap
p
lie
d
to
th
e
p
r
o
p
o
s
ed
m
eth
o
d
.
A
c
t
i
v
a
t
i
on
Fu
n
c
t
i
on
:
R
e
LU
N
e
ur
o
n
A
c
t
i
v
i
t
y
A
c
t
i
v
a
t
i
on
Fu
n
c
t
i
on
:
l
e
a
k
y
R
e
LU
N
e
uro
n
A
c
t
i
v
i
t
y
S
l
op
e
β
=
1
S
l
op
e
α
=
0
S
l
ope
β
=
1
S
l
ope
α
=
0
.
0
1
(
a
)
(
b
)
A
c
t
i
v
a
t
i
on
Fu
n
c
t
i
on
:
R
e
LU
N
e
ur
o
n
A
c
t
i
v
i
t
y
A
c
t
i
v
a
t
i
on
Fu
n
c
t
i
on
:
l
e
a
k
y
R
e
LU
N
e
uro
n
A
c
t
i
v
i
t
y
S
l
op
e
β
=
1
S
l
op
e
α
=
0
S
l
ope
β
=
1
S
l
ope
α
=
0
.
0
1
(
a
)
(
b
)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Gen
etic
a
lg
o
r
ith
m
-
a
d
a
p
ted
a
ctiva
tio
n
fu
n
ctio
n
o
p
timiz
a
tio
n
o
f d
ee
p
le
a
r
n
in
g
…
(
N
o
o
r
F
a
d
z
ila
h
R
a
z
a
li
)
2827
(
)
=
(
,
)
;
=
{
,
<
+
,
≥
,
=
(
0
,
0
)
,
=
0
.
01
,
=
1
(4)
(
)
−
=
(
,
)
;
=
{
α
x
,
<
±
±
,
≥
±
,
at
θ
i
=
(
±
,
0
)
,
α
=
0
.
01
,
β
=
1
(5
)
T
h
e
C
NN
-
ex
tr
ac
ted
f
ea
tu
r
es
ar
e
th
en
class
if
ied
to
g
et
th
e
b
est
p
er
f
o
r
m
an
ce
s
co
r
es
o
n
th
e
test
in
g
d
ataset.
I
n
th
is
s
tu
d
y
,
th
e
cu
s
to
m
ized
ac
tiv
atio
n
f
u
n
ctio
n
lay
er
is
o
n
ly
ap
p
lied
to
th
e
last
lay
er
o
f
f
ea
tu
r
e
ex
tr
ac
tio
n
.
T
h
is
s
tep
is
d
o
n
e
t
o
allo
w
s
tr
ict
f
ea
tu
r
e
ex
tr
ac
tio
n
an
d
s
p
ar
s
ity
o
n
th
e
u
p
p
e
r
lay
er
d
u
r
i
n
g
s
em
an
tic
f
ea
tu
r
e
ex
tr
ac
tio
n
u
s
in
g
t
h
e
o
r
i
g
in
al
R
eL
U
an
d
o
n
ly
allo
w
ad
ap
tiv
en
ess
o
n
th
e
f
i
n
al
ex
tr
ac
ti
o
n
s
tag
e.
3
.
3
.
P
er
f
o
rma
nce
met
rics
T
h
e
ev
alu
atio
n
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el
is
b
ased
o
n
t
h
e
co
n
f
u
s
io
n
m
atr
ix
(
C
M)
,
wh
ich
in
c
lu
d
es
f
o
u
r
co
m
p
o
n
en
ts
:
T
r
u
e
p
o
s
itiv
e
(
T
P),
f
alse
p
o
s
itiv
e
(
FP
)
,
tr
u
e
n
eg
ativ
e
(
T
N)
,
an
d
f
alse
n
e
g
ativ
e
(
FN)
.
E
ac
h
co
m
p
o
n
en
t
ca
p
tu
r
es
s
p
ec
if
ic
asp
ec
ts
o
f
th
e
m
o
d
el's
p
er
f
o
r
m
an
ce
:
T
P
id
e
n
tifie
s
co
r
r
ec
tl
y
p
r
e
d
icted
p
o
s
itiv
e
m
alig
n
an
t
ca
s
es,
FP
co
u
n
ts
in
co
r
r
ec
tly
p
r
e
d
icted
p
o
s
itiv
e
m
alig
n
an
t
as
b
en
ig
n
ca
s
es,
T
N
r
ep
r
esen
ts
co
r
r
ec
tly
p
r
ed
icted
n
eg
ativ
e
ca
s
es,
an
d
FN
tallies
m
is
s
ed
p
o
s
itiv
e
ca
s
es.
Usi
n
g
th
ese
f
o
u
n
d
atio
n
al
co
m
p
o
n
en
ts
,
th
e
ev
alu
atio
n
p
r
o
ce
ed
s
with
th
e
ca
lcu
latio
n
o
f
m
etr
ics
th
r
o
u
g
h
(
6
)
to
(
9
)
,
en
a
b
lin
g
a
d
etaile
d
ass
es
s
m
en
t
o
f
th
e
m
o
d
el's st
r
en
g
th
s
an
d
wea
k
n
e
s
s
es.
Acc
u
r
ac
y
,
Acc
=
TP
+
TN
TP
+
FN
+
TN
+
FP
.
(
6
)
Sen
s
itiv
ity
/ Reca
ll
=
TP
TP
+
FN
.
(
7
)
Sp
ec
if
icity
=
TN
(
TP
+
FN
)
.
(
8
)
F1
-
s
co
r
e
=
2x
Pr
ecis
i
o
n
X
Recal
l
Pr
ecis
i
o
n
+
Recal
l
.
(
9
)
4.
RE
SU
L
T
AND
DI
SCUS
SI
O
N
4.
1
.
Dif
f
er
ent
pre
-
t
ra
ined mo
dels
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
r
esu
lts
o
f
th
e
p
r
o
p
o
s
ed
m
eth
o
d
ev
alu
ated
o
n
t
h
e
I
Nb
r
ea
s
t
d
a
taset.
T
wo
estab
lis
h
ed
p
r
e
-
tr
ain
e
d
n
etwo
r
k
s
,
R
esNet5
0
an
d
Sh
u
f
f
leNe
t,
wer
e
u
s
ed
f
o
r
th
e
ev
alu
atio
n
.
T
h
e
c
o
m
p
ar
is
o
n
h
ig
h
lig
h
ts
th
e
p
er
f
o
r
m
an
ce
o
f
m
o
d
els
tr
ain
ed
u
n
d
e
r
two
co
n
d
itio
n
s
:
th
e
d
ef
au
lt
R
eL
U
ac
tiv
atio
n
o
f
th
e
p
r
e
-
tr
ain
ed
n
etwo
r
k
s
an
d
th
e
m
o
d
i
f
ied
v
er
s
io
n
s
tr
ain
e
d
with
GAa
-
R
eL
U.
T
h
e
r
esu
lts
ar
e
s
u
m
m
ar
ized
in
T
a
b
le
3
.
B
ased
o
n
th
e
r
esu
lts
in
T
ab
le
3
,
Sh
u
f
f
leNe
t
o
u
t
p
er
f
o
r
m
ed
R
esNet5
0
ac
r
o
s
s
all
te
s
ted
m
etr
ics
f
o
r
I
Nb
r
ea
s
t
ca
n
ce
r
class
if
icatio
n
.
W
h
en
tr
ain
ed
u
s
in
g
th
e
p
r
o
p
o
s
ed
GAa
-
R
eL
U
ac
tiv
atio
n
f
u
n
ctio
n
,
b
o
t
h
m
o
d
els
s
h
o
wed
n
o
tab
le
im
p
r
o
v
e
m
en
t
s
o
v
er
th
eir
d
ef
au
lt
R
eL
U
-
ba
s
ed
co
u
n
te
r
p
ar
ts
.
Sh
u
f
f
leNe
t
ac
h
iev
ed
t
h
e
b
est
p
er
f
o
r
m
an
ce
with
an
ac
cu
r
ac
y
o
f
9
9
.
0
0
%,
an
F1
-
s
co
r
e
o
f
0
.
9
8
5
1
,
s
en
s
itiv
ity
o
f
1
.
0
0
0
,
an
d
s
p
ec
if
icity
o
f
0
.
9
8
5
2
.
R
esNet5
0
al
s
o
s
h
o
wed
im
p
r
o
v
em
en
t,
ac
h
iev
in
g
9
7
.
0
1
%
ac
cu
r
ac
y
,
an
F1
-
s
co
r
e
o
f
0
.
9
5
5
9
,
an
d
s
p
ec
if
ici
ty
o
f
0
.
9
5
5
9
,
w
h
ile
m
ain
tain
in
g
a
s
en
s
itiv
ity
o
f
0
.
9
8
4
8
.
T
h
ese
r
e
s
u
lts
d
em
o
n
s
tr
ate
th
at
th
e
p
r
o
p
o
s
ed
GAa
-
R
eL
U
m
eth
o
d
en
h
an
ce
s
p
er
f
o
r
m
a
n
ce
ef
f
ec
tiv
ely
ac
r
o
s
s
d
if
f
er
en
t
p
r
e
-
tr
ain
ed
n
etwo
r
k
s
,
ac
h
iev
in
g
r
esu
lts
o
n
p
ar
with
o
r
b
etter
th
a
n
d
ef
a
u
lt c
o
n
f
ig
u
r
atio
n
s
,
p
ar
ticu
lar
ly
f
o
r
Sh
u
f
f
le
Net.
T
ab
le
3
.
C
lass
if
icatio
n
o
f
b
r
ea
s
t m
ass
f
o
r
I
Nb
r
ea
s
t d
ataset
u
s
in
g
th
e
o
r
ig
in
al
R
eL
U
v
s
.
p
r
o
p
o
s
ed
GAa
-
R
eL
U
on
d
if
f
er
e
n
t p
r
e
-
t
r
ain
ed
n
etwo
r
k
s
M
e
t
r
i
c
S
h
u
f
f
l
e
N
e
t
R
e
sN
e
t
5
0
R
e
LU
(
D
e
f
a
u
l
t
)
GAa
-
R
e
LU
R
e
LU
(
D
e
f
a
u
l
t
)
GAa
-
R
e
LU
A
c
c
u
r
a
c
y
(
%)
9
7
.
5
1
9
9
.
0
0
9
5
.
0
0
9
7
.
0
1
S
e
n
s
i
t
i
v
i
t
y
/
R
e
c
a
l
l
(
TP
R
)
0
.
9
6
8
7
1
.
0
0
0
0
0
.
9
8
4
8
0
.
9
8
4
8
S
p
e
c
i
f
i
c
i
t
y
(
TFR)
0
.
9
7
7
8
0
.
9
8
5
2
0
.
9
3
3
3
0
.
9
5
5
9
F1
-
s
c
o
r
e
0
.
9
6
2
4
0
.
9
8
5
1
0
.
9
2
8
6
0
.
9
5
5
9
4
.
2
.
Dif
f
er
ent
a
ct
iv
a
t
io
n f
un
ct
io
ns
I
n
th
is
ex
p
er
im
en
t,
test
in
g
an
d
co
m
p
ar
is
o
n
to
ev
alu
ate
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
n
an
o
th
er
estab
lis
h
ed
ac
tiv
atio
n
f
u
n
ctio
n
,
leak
y
R
eL
U.
T
h
e
ex
p
e
r
im
en
t
is
test
ed
b
a
s
ed
o
n
m
etr
ics
o
f
ac
c
u
r
ac
y
,
s
e
n
s
itiv
ity
,
s
p
ec
if
icity
,
an
d
F1
-
s
co
r
es.
T
h
e
R
esNet5
0
r
esu
lt
f
r
o
m
T
ab
le
3
is
th
en
tab
u
lated
ag
ain
in
T
ab
le
4
,
wh
er
e
its
p
er
f
o
r
m
an
ce
is
co
m
p
ar
ed
to
th
e
p
r
o
p
o
s
ed
m
et
h
o
d
ap
p
lied
to
t
h
e
s
ec
o
n
d
ac
ti
v
atio
n
f
u
n
ctio
n
o
f
leak
y
R
eL
U.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
3
,
J
u
n
e
20
25
:
2
8
2
0
-
2
8
3
3
2828
T
ab
le
4
.
C
lass
if
icatio
n
o
f
b
r
ea
s
t m
ass
f
o
r
I
Nb
r
ea
s
t d
ataset
u
s
in
g
th
e
o
r
ig
in
al
R
eL
U
an
d
lea
k
y
R
eL
U
v
s
.
th
e
p
r
o
p
o
s
ed
GAa
-
R
eL
U
an
d
GAa
-
leak
y
R
eL
U
M
e
t
r
i
c
/
D
a
t
a
se
t
I
N
b
r
e
a
st
Re
L
U
(
D
e
f
a
u
l
t
)
G
Aa
-
Re
L
U
l
e
a
k
y
Re
L
U
G
Aa
-
l
e
a
k
y
R
e
L
U
A
c
c
u
r
a
c
y
(
%)
9
5
.
0
0
9
7
.
0
1
9
3
.
0
3
9
9
.
0
0
S
e
n
s
i
t
i
v
i
t
y
/
R
e
c
a
l
l
(
TPR)
0
.
9
8
4
8
0
.
9
8
4
8
0
.
9
6
9
7
0
.
9
6
9
7
S
p
e
c
i
f
i
c
i
t
y
(
TN
R
)
0
.
9
3
3
3
0
.
9
5
5
9
0
.
9
1
1
1
1
.
0
0
0
0
F1
-
s
c
o
r
e
0
.
9
2
8
6
0
.
9
5
5
9
0
.
9
0
1
4
0
.
9
8
4
6
Fro
m
T
ab
le
4
,
th
e
s
u
b
s
titu
tio
n
o
f
R
eL
U
to
leak
y
R
eL
U
f
o
r
th
e
I
Nb
r
ea
s
t
d
ataset
d
ec
r
ea
s
ed
its
o
v
er
all
p
er
f
o
r
m
an
ce
.
Oth
er
m
etr
ics,
s
u
ch
as
ac
cu
r
ac
y
,
d
r
o
p
p
ed
to
9
3
.
0
3
%,
a
n
d
F1
-
s
co
r
e,
s
en
s
itiv
ity
,
an
d
s
p
ec
if
icity
v
alu
es
d
r
o
p
p
e
d
as
well.
T
h
is
s
u
g
g
ests
th
at
s
im
p
ly
r
e
p
lacin
g
th
e
R
eL
U
with
lea
k
y
R
eL
U
d
o
es
n
o
t
n
ec
ess
ar
ily
im
p
r
o
v
e
its
d
y
in
g
R
eL
U
p
r
o
b
lem
with
o
u
t
p
r
o
p
er
k
n
o
wled
g
e
o
f
th
e
v
alu
e
o
f
p
o
s
itiv
e
s
lo
p
e
o
r
c
o
ef
f
icien
t
α
.
Ho
wev
er
,
wh
en
th
e
p
r
o
p
o
s
e
d
m
eth
o
d
o
f
GAa
-
leak
y
R
eL
u
is
im
p
lem
en
ted
,
it
ca
n
e
n
h
an
ce
th
e
o
v
er
all
p
er
f
o
r
m
an
ce
f
o
r
all
m
etr
ics,
s
u
r
p
ass
in
g
th
e
p
er
f
o
r
m
a
n
ce
o
f
th
e
o
r
ig
in
al
R
eL
U
an
d
GA
a
-
R
eL
U
o
n
a
f
ix
ed
co
ef
f
icien
t
α
.
A
1
.
0
0
0
0
p
er
f
o
r
m
an
ce
is
d
em
o
n
s
tr
ated
o
n
t
h
e
s
p
ec
if
icity
v
alu
e.
Oth
er
m
etr
ic
s
o
f
th
e
ac
cu
r
ac
y
o
f
9
9
.
0
0
%,
an
d
an
F1
-
s
co
r
e
o
f
0
.
9
8
4
6
is
also
d
em
o
n
s
tr
ated
.
Fro
m
th
is
r
esu
lt,
it
ca
n
b
e
co
n
cl
u
d
ed
th
at
ev
en
th
o
u
g
h
wh
en
th
e
R
eL
U
is
in
itially
s
u
b
s
titu
ted
with
lea
k
y
R
eL
U,
ca
u
s
in
g
th
e
m
o
d
el
to
p
er
f
o
r
m
l
o
wer
ev
en
with
th
e
in
tr
o
d
u
ctio
n
o
f
α
,
it
is
p
r
o
v
en
t
h
at
is
n
o
t
th
e
ca
s
e
f
o
r
th
is
d
ataset.
Simp
ly
in
tr
o
d
u
cin
g
t
h
e
co
ef
f
icien
t
α
is
u
n
ab
le
to
allo
w
th
e
o
u
tlier
s
with
in
th
e
test
ed
d
ataset
to
co
p
e
well
an
d
ca
u
s
e
o
v
er
co
m
p
e
n
s
atio
n
f
o
r
n
o
t
b
ein
g
ab
le
to
b
e
g
r
o
u
p
ed
p
er
f
ec
tly
with
th
e
f
ix
ed
v
alu
e
o
f
α
d
u
r
in
g
ac
tiv
at
io
n
o
f
th
e
test
in
g
f
ea
tu
r
es
o
n
th
e
tr
ain
ed
m
o
d
el.
Ho
wev
er
,
wh
en
o
p
tim
ized
u
s
in
g
th
e
GAa
-
leak
y
R
eL
U,
th
e
o
r
ig
in
al
leak
y
R
eL
U,
h
av
in
g
a
n
e
two
r
k
lan
d
s
ca
p
e
o
f
a
ty
p
ically
n
o
n
-
co
n
v
ex
,
with
i
ts
o
wn
o
p
tim
izatio
n
m
eth
o
d
u
s
ed
,
s
u
ch
as
Ad
am
,
m
ay
g
et
s
tu
ck
in
d
eter
m
i
n
in
g
its
l
o
ca
l
m
in
im
a.
GA,
as
em
p
l
o
y
ed
in
t
h
is
s
tu
d
y
,
with
t
h
eir
ab
ilit
y
to
ex
p
lo
r
e
d
iv
er
s
e
s
o
lu
tio
n
s
,
h
elp
s
escap
e
th
is
lo
ca
l
m
in
im
u
m
an
d
f
in
d
g
lo
b
al
o
r
n
ea
r
-
g
lo
b
al
o
p
tim
a,
as
d
em
o
n
s
tr
ated
in
b
o
th
ad
ap
ted
ac
tiv
atio
n
f
u
n
ctio
n
r
esu
lts
.
4.
3.
Seco
nd
ma
mm
o
g
ra
m da
t
a
s
e
t
I
n
th
is
ex
p
er
im
e
n
t,
test
in
g
an
d
co
m
p
ar
is
o
n
a
r
e
co
n
d
u
cted
u
s
in
g
th
e
s
ec
o
n
d
d
ataset,
I
PP
T
-
m
am
m
o
t
o
f
u
r
th
er
e
v
alu
ate
th
e
p
r
o
p
o
s
ed
m
eth
o
d
's
ef
f
ec
tiv
en
ess
.
T
h
is
ex
p
er
im
en
t
also
ex
am
in
es
th
e
m
eth
o
d
's
p
er
f
o
r
m
an
c
e
wh
en
ap
p
lied
to
an
o
t
h
er
estab
lis
h
ed
ac
tiv
atio
n
f
u
n
ctio
n
,
leak
y
R
eL
U.
T
ab
le
5
s
h
o
ws
th
e
r
es
u
lt
o
f
class
if
icatio
n
u
s
in
g
th
e
I
PP
T
-
m
am
m
o
d
ataset
u
s
in
g
th
e
o
r
ig
in
al
R
eL
U
v
s
.
p
r
o
p
o
s
ed
GAa
-
R
eL
U
an
d
leak
y
R
eL
U
v
s
GAa
-
leak
y
R
eL
U.
T
ab
le
5
.
C
lass
if
icatio
n
o
f
b
en
i
g
n
an
d
m
alig
n
a
n
t b
r
ea
s
t m
ass
f
o
r
I
PP
T
-
m
am
m
o
d
ataset
u
s
in
g
th
e
o
r
i
g
in
al
R
eL
U
v
s
.
p
r
o
p
o
s
ed
GAa
-
R
eL
U
an
d
leak
y
R
eL
U
v
s
GAa
-
lea
k
y
R
eL
U
M
e
t
r
i
c
/
D
a
t
a
se
t
IPPT
-
mammo
Re
L
U
(
D
e
f
a
u
l
t
)
G
Aa
-
Re
L
U
l
e
a
k
y
Re
L
U
G
Aa
-
l
e
a
k
y
R
e
L
U
A
c
c
u
r
a
c
y
(
%)
8
4
.
9
0
8
7
.
4
0
8
4
.
0
3
9
1
.
6
0
S
e
n
s
i
t
i
v
i
t
y
/
R
e
c
a
l
l
(
TPR)
0
.
8
9
8
6
0
.
8
1
1
6
0
.
9
2
7
5
0
.
9
1
3
0
S
p
e
c
i
f
i
c
i
t
y
(
TN
R
)
0
.
7
8
0
0
0
.
9
6
0
0
0
.
7
2
0
0
0
.
9
2
0
0
F1
-
s
c
o
r
e
0
.
8
7
3
2
0
.
8
8
1
9
0
.
8
7
0
7
0
.
9
2
6
5
T
h
e
m
etr
ics
s
u
g
g
est
o
v
er
all
lo
wer
p
er
f
o
r
m
an
ce
with
th
e
esta
b
lis
h
ed
I
Nb
r
ea
s
t
d
ataset
d
u
e
t
o
a
h
ig
h
e
r
n
u
m
b
er
o
f
d
en
s
er
b
r
ea
s
ts
in
th
e
I
PP
T
-
m
am
m
o
d
ataset.
Gen
er
ally
,
wh
en
ap
p
lied
with
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
f
GAa
-
R
eL
U,
th
e
p
er
f
o
r
m
an
ce
p
atter
n
ex
h
ib
its
im
p
r
o
v
e
m
en
ts
an
d
c
o
n
s
is
ten
tly
o
u
tp
er
f
o
r
m
s
th
e
R
eL
U
(
d
ef
au
lt
)
ac
tiv
atio
n
o
n
all
ev
al
u
atio
n
m
etr
ics,
asid
e
f
r
o
m
th
e
s
en
s
itiv
ity
.
T
h
e
im
p
r
o
v
em
e
n
t
in
ac
cu
r
ac
y
is
o
b
s
er
v
ed
at
8
4
.
9
0
%
to
8
7
.
4
0
%.
Sig
n
if
ica
n
t
im
p
r
o
v
e
m
en
t
is
o
b
s
er
v
ed
,
s
u
ch
as
th
e
F1
-
s
co
r
e
r
elate
s
to
th
e
p
r
ec
is
io
n
an
d
s
en
s
itiv
ity
o
f
a
m
o
d
el.
I
t in
d
ic
ates a
b
etter
r
ep
r
esen
tatio
n
o
f
a
s
y
s
tem
with
an
u
n
b
alan
ce
d
d
ataset,
with
a
s
lig
h
t
in
cr
ea
s
e
to
0
.
8
8
1
9
f
r
o
m
0
.
8
7
3
2
in
th
e
I
PP
T
-
m
am
m
o
d
ataset.
C
o
m
p
ar
in
g
t
h
e
r
esu
lt
to
T
ab
le
4
,
a
b
alan
ce
o
f
s
en
s
itiv
ity
an
d
s
p
ec
if
icity
f
o
r
th
e
p
r
o
p
o
s
ed
m
o
d
el
d
em
o
n
s
tr
ates
th
e
ab
ilit
y
o
f
th
e
m
o
d
el
t
o
co
r
r
ec
tly
class
if
y
th
e
f
alse
p
o
s
itiv
es
an
d
f
alse
n
eg
ativ
es
o
f
th
e
test
in
g
d
ata
o
n
th
e
test
in
g
I
Nb
r
ea
s
t
d
ataset.
T
h
e
s
p
ec
if
icity
was
s
ig
n
if
ican
tly
im
p
r
o
v
ed
with
th
e
GAa
-
R
eL
U
ac
tiv
atio
n
,
r
ea
ch
in
g
0
.
9
6
0
0
co
m
p
a
r
ed
to
its
b
as
e
v
er
s
io
n
o
f
0
.
7
8
0
0
.
Ho
wev
er
,
th
is
also
s
im
u
lt
an
eo
u
s
ly
af
f
ec
ted
th
e
T
PR
,
as
it
s
l
ig
h
tly
d
ec
r
ea
s
ed
to
0
.
8
1
1
6
f
o
r
GAa
-
R
eL
U
an
d
a
s
im
ilar
tr
en
d
is
s
ee
n
o
n
th
e
tes
t
u
s
in
g
GAa
-
leak
y
R
eL
U.
Ov
e
r
all,
th
e
im
p
r
o
v
em
en
ts
i
n
p
er
f
o
r
m
an
ce
m
etr
ics
o
n
th
e
I
PP
T
-
m
am
m
o
d
ataset
f
o
r
b
o
th
b
ase
ac
tiv
atio
n
f
u
n
ctio
n
s
s
h
o
w
th
e
p
r
o
p
o
s
ed
m
eth
o
d
ca
n
b
e
im
p
lem
e
n
ted
o
n
d
if
f
er
en
t
d
atasets
with
o
u
t
p
a
r
am
eter
c
h
an
g
es,
s
u
g
g
esti
n
g
th
e
g
en
er
aliza
b
ilit
y
o
f
th
e
p
r
o
p
o
s
ed
m
eth
o
d
.
T
o
ev
alu
ate
th
e
p
r
o
p
o
s
ed
m
et
h
o
d
with
s
ev
er
al
s
tate
-
of
-
th
e
-
a
r
t m
o
d
els f
r
o
m
p
r
ev
io
u
s
s
tu
d
ies,
T
ab
le
6
p
r
o
v
i
d
es
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Gen
etic
a
lg
o
r
ith
m
-
a
d
a
p
ted
a
ctiva
tio
n
fu
n
ctio
n
o
p
timiz
a
tio
n
o
f d
ee
p
le
a
r
n
in
g
…
(
N
o
o
r
F
a
d
z
ila
h
R
a
z
a
li
)
2829
th
e
p
er
f
o
r
m
an
ce
c
o
m
p
ar
is
o
n
b
etwe
en
th
e
s
tu
d
ies
u
s
in
g
th
e
s
am
e
p
u
b
lic
d
ataset
f
r
o
m
I
Nb
r
ea
s
t
an
d
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
f
th
e
C
NN
m
o
d
el
u
s
in
g
th
e
p
r
o
p
o
s
ed
ac
tiv
atio
n
f
u
n
ctio
n
o
p
tim
izatio
n
m
eth
o
d
.
T
ab
le
6
.
C
o
m
p
ar
is
o
n
o
f
e
v
alu
atio
n
p
er
f
o
r
m
an
ce
f
o
r
class
if
icatio
n
s
tu
d
ies u
s
in
g
th
e
I
Nb
r
ea
s
t d
ataset
S
t
u
d
i
e
s
M
e
t
h
o
d
A
c
c
u
r
a
c
y
S
p
e
c
i
f
i
c
i
t
y
S
e
n
s
i
t
i
v
i
t
y
F1
-
s
c
o
r
e
[
1
8
]
C
N
N
,
w
a
v
e
l
e
t
sc
a
t
t
e
r
i
n
g
9
3
.
5
0
N
/
A
N
/
A
0
.
9
3
3
7
[
3
1
]
C
N
N
,
c
h
a
o
t
i
c
ma
p
F
S
o
p
t
i
mi
z
a
t
i
o
n
9
8
.
4
6
0
.
9
8
6
8
0
.
9
8
2
1
0
.
9
8
3
9
[
3
4
]
C
N
N
,
t
r
a
n
sf
e
r
a
b
l
e
t
e
x
t
u
r
e
9
6
.
8
2
0
.
9
7
6
8
0
.
9
5
9
9
N
/
A
[
3
5
]
C
N
N
9
3
.
0
0
0
.
9
3
8
6
0
.
9
3
8
3
0
.
9
3
0
3
[
3
6
]
C
N
N
,
b
i
d
i
r
e
c
t
i
o
n
a
l
l
o
n
g
s
h
o
r
t
-
t
e
r
m
m
e
mo
r
y
n
e
u
r
a
l
n
e
t
w
o
r
k
(
B
i
LST
M
)
9
2
.
2
6
0
.
9
2
9
5
0
.
8
6
2
1
0
.
8
8
5
3
Th
i
s
S
t
u
d
y
C
N
N
,
G
A
o
p
t
i
m
i
z
e
d
R
e
LU
(
R
e
sN
e
t
5
0
)
9
7
.
0
1
0
.
9
5
5
9
0
.
9
8
4
8
0
.
9
5
5
9
C
N
N
,
G
A
o
p
t
i
m
i
z
e
d
R
e
LU
(
S
h
u
f
f
l
e
N
e
t
)
9
9
.
0
0
0
.
9
8
5
2
1
.
0
0
0
0
0
.
9
8
5
1
C
N
N
,
G
A
o
p
t
i
m
i
z
e
d
l
e
a
k
y
R
e
LU
(
R
e
s
N
e
t
5
0
)
9
9
.
0
0
1
.
0
0
0
0
0
.
9
6
9
7
0
.
9
8
4
6
T
h
e
o
v
er
all
p
er
f
o
r
m
an
ce
r
ev
e
aled
th
at
t
h
e
p
r
o
p
o
s
ed
m
et
h
o
d
o
f
o
p
tim
izin
g
th
e
ac
tiv
atio
n
f
u
n
ctio
n
p
er
f
o
r
m
ed
b
est
o
n
all
th
e
ev
al
u
atio
n
m
etr
ics
ac
r
o
s
s
th
r
ee
m
o
d
els
ex
p
e
r
im
en
ted
in
th
is
s
tu
d
y
.
Ma
q
s
o
o
d
et
a
l.
[
3
4
]
also
ex
h
ib
its
g
o
o
d
ac
cu
r
ac
y
(
9
6
.
8
2
%)
wh
e
n
u
s
in
g
a
tr
an
s
f
er
ab
le
tex
tu
r
e
C
NN
an
d
g
o
o
d
s
p
ec
if
icity
(
0
.
9
7
6
8
)
.
Ho
wev
er
,
th
ey
d
id
n
o
t
p
r
o
v
id
e
o
th
er
e
v
alu
atio
n
s
to
m
ak
e
a
s
o
u
n
d
co
m
p
ar
is
o
n
.
T
h
e
s
tu
d
y
b
y
[
1
8
]
h
as
an
ac
cu
r
ac
y
o
f
9
3
.
5
0
%
an
d
a
l
o
wer
F1
-
s
co
r
e,
p
o
s
s
ib
ly
d
u
e
t
o
th
e
n
o
n
-
au
g
m
e
n
ted
im
a
g
es
u
s
ed
.
I
n
ad
d
itio
n
to
th
e
p
r
o
p
o
s
ed
m
eth
o
d
,
C
h
ak
r
a
v
ar
th
y
et
a
l.
[
3
1
]
p
er
f
o
r
m
ed
co
m
p
ar
ativ
ely
well,
with
9
8
.
4
6
%
ac
cu
r
ac
y
an
d
a
b
alan
ce
d
o
v
e
r
all
s
p
ec
if
icity
an
d
s
en
s
itiv
ity
o
f
0
.
9
8
6
8
an
d
0
.
9
8
2
1
,
r
esp
ec
tiv
ely
.
T
h
is
is
d
u
e
to
o
p
tim
izatio
n
im
p
lem
en
tatio
n
d
u
r
in
g
its
f
ea
t
u
r
e
s
elec
tio
n
p
r
o
ce
s
s
,
d
em
o
n
s
tr
atin
g
th
e
im
p
o
r
ta
n
ce
o
f
in
tr
o
d
u
cin
g
a
s
ig
n
if
ican
t
m
eth
o
d
to
im
p
r
o
v
e
th
e
f
in
al
cl
ass
if
icatio
n
p
er
f
o
r
m
an
ce
as
p
r
o
p
o
s
ed
in
th
is
s
tu
d
y
.
Ho
wev
er
,
th
e
b
est
-
p
er
f
o
r
m
e
d
m
o
d
el
is
attain
ed
b
y
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
n
GAa
-
leak
y
R
eL
U
f
o
r
R
esNet5
0
an
d
GAa
-
R
eL
U
f
o
r
Sh
u
f
f
leNe
t,
with
b
o
th
ac
h
iev
in
g
th
e
b
est a
cc
u
r
ac
y
at
9
9
.
0
0
%.
As d
em
o
n
s
tr
ated
f
r
o
m
t
h
e
r
esu
lt,
m
o
d
if
y
in
g
th
e
th
r
esh
o
ld
o
f
th
e
ac
tiv
atio
n
f
u
n
ctio
n
s
,
s
u
c
h
as
R
eL
U
an
d
leak
y
R
eL
U
,
wh
ich
o
r
ig
in
ally
s
tar
ts
at
(
0
,
0
)
,
to
a
v
alu
e
co
r
r
esp
o
n
d
in
g
to
th
e
d
eg
r
ee
o
f
ad
ap
tab
ilit
y
o
f
th
e
n
o
n
lin
ea
r
f
u
n
ctio
n
in
th
e
f
in
al
f
ea
tu
r
e
e
x
tr
ac
tio
n
p
h
ase
h
as
s
u
b
s
tan
tial
s
ig
n
if
ican
ce
.
T
h
e
f
lex
ib
ilit
y
o
f
th
e
ac
tiv
atio
n
f
u
n
ctio
n
in
v
o
l
v
in
g
n
e
g
ativ
e
n
e
u
r
o
n
ac
tiv
atio
n
h
as
n
u
m
er
o
u
s
s
ig
n
if
ican
t
b
en
ef
its
in
m
o
d
ellin
g
in
t
r
icate
d
ata
an
d
o
p
tim
izin
g
n
e
u
r
al
n
etwo
r
k
s
.
H
en
ce
,
in
co
r
p
o
r
a
tin
g
an
ad
ju
s
tab
le
th
r
esh
o
ld
with
th
e
h
elp
o
f
a
n
o
p
tim
izatio
n
a
lg
o
r
ith
m
,
as
d
em
o
n
s
tr
ated
in
th
is
s
tu
d
y
,
allo
ws
f
lex
ib
ilit
y
th
at
en
ab
les
n
eu
r
o
n
s
to
s
u
s
tain
p
ar
tial
ac
tiv
atio
n
e
v
en
wh
en
p
r
esen
te
d
with
in
p
u
t
s
th
at
f
all
b
elo
w
th
e
th
r
esh
o
ld
.
T
h
is
ap
p
r
o
ac
h
in
d
ir
ec
tly
ad
d
r
ess
es
th
e
is
s
u
e
o
f
th
e
d
y
in
g
R
eL
U
p
r
o
b
lem
,
wh
ic
h
r
esu
lts
in
a
h
ig
h
er
lev
el
o
f
n
e
u
r
o
n
in
v
o
lv
em
e
n
t
in
th
e
lear
n
in
g
p
r
o
ce
s
s
an
d
co
n
s
eq
u
en
tly
en
h
an
ce
s
th
e
m
o
d
el's
ca
p
ac
ity
f
o
r
ex
p
r
ess
io
n
.
5.
CO
NCLU
SI
O
N
T
h
is
s
tu
d
y
in
tr
o
d
u
ce
s
a
m
et
h
o
d
f
o
r
o
p
tim
is
in
g
class
if
icatio
n
d
u
r
in
g
f
ea
tu
r
e
e
x
tr
ac
tio
n
,
f
o
cu
s
in
g
o
n
n
o
n
lin
ea
r
ac
tiv
atio
n
at
th
e
f
in
al
s
tag
e.
T
h
e
GA
o
p
tim
izatio
n
is
u
s
ed
to
ad
ju
s
t
a
C
NN's
th
r
e
s
h
o
ld
o
f
R
eL
U
an
d
leak
y
R
eL
U.
T
h
e
r
esu
lts
s
h
o
w
th
e
b
est
m
o
d
el
p
er
f
o
r
m
ed
u
s
in
g
th
e
o
p
tim
ized
leak
y
R
eL
U
with
9
9
%
ac
cu
r
ac
y
,
h
av
in
g
th
e
b
est
s
p
ec
if
icity
at
1
.
0
0
0
0
f
o
r
th
e
I
Nb
r
ea
s
t
d
ataset,
an
d
9
1
.
6
0
%
ac
cu
r
ac
y
f
o
r
th
e
n
ewly
co
llected
I
PP
T
-
m
am
m
o
d
ataset.
T
h
is
ad
ju
s
tm
en
t im
p
r
o
v
es m
o
d
el
p
er
f
o
r
m
an
ce
,
b
y
s
h
if
tin
g
th
e
n
e
u
r
o
n
ac
tiv
a
tio
n
r
e
g
io
n
af
f
ec
tin
g
th
e
n
o
n
-
lin
ea
r
ity
.
T
e
s
ts
o
n
p
u
b
lic
an
d
p
r
iv
ate
m
am
m
o
g
r
am
d
atasets
with
d
if
f
er
en
t
C
NN
m
o
d
els
u
s
in
g
th
e
m
o
d
if
ied
R
eL
U
an
d
leak
y
R
eL
U
in
d
icate
th
at
th
e
m
eth
o
d
is
g
en
er
aliza
b
le
ac
r
o
s
s
v
ar
io
u
s
ar
ch
itectu
r
es
an
d
d
atasets
.
I
n
s
u
m
m
a
r
y
,
m
o
d
if
y
i
n
g
ac
tiv
atio
n
f
u
n
ctio
n
th
r
esh
o
ld
s
h
elp
s
tailo
r
n
e
u
r
al
n
etwo
r
k
s
to
s
p
ec
if
ic
task
s
,
en
h
an
cin
g
p
e
r
f
o
r
m
an
ce
an
d
g
en
er
aliza
tio
n
,
a
d
d
r
ess
in
g
is
s
u
es
lik
e
th
e
d
y
in
g
R
eL
U,
r
ef
in
in
g
n
o
n
lin
ea
r
ity
,
an
d
in
cr
ea
s
in
g
r
o
b
u
s
tn
ess
ag
ain
s
t
n
o
is
y
d
ata.
A
n
o
ted
lim
itatio
n
is
th
e
lo
n
g
er
GA
tr
ain
in
g
tim
e
r
eq
u
ir
ed
f
o
r
lar
g
er
p
o
p
u
latio
n
s
,
th
o
u
g
h
f
in
al
tr
ain
in
g
is
less
co
m
p
u
tatio
n
ally
in
t
en
s
iv
e.
Fu
tu
r
e
wo
r
k
in
clu
d
es
ap
p
ly
in
g
th
is
m
eth
o
d
to
o
th
er
R
eL
U
-
b
ased
f
u
n
cti
o
n
s
an
d
ex
p
l
o
r
in
g
n
ew
m
etah
eu
r
is
tic
o
p
tim
izatio
n
ap
p
r
o
ac
h
es
to
r
ed
u
c
e
co
m
p
u
tatio
n
al
l
o
ad
.
ACK
NO
WL
E
DG
E
M
E
NT
S
T
h
e
au
th
o
r
s
wo
u
ld
lik
e
to
ex
p
r
ess
th
eir
g
r
atitu
d
e
to
m
em
b
er
s
o
f
th
e
ad
v
an
ce
d
co
n
t
r
o
l
s
y
s
tem
an
d
co
m
p
u
tin
g
r
esear
ch
g
r
o
u
p
(
AC
S
C
R
G)
,
ad
v
an
ce
d
r
eh
ab
ilit
atio
n
en
g
in
ee
r
in
g
in
d
iag
n
o
s
tic
an
d
m
o
n
ito
r
in
g
r
esear
ch
g
r
o
u
p
(
AR
E
DiM)
,
in
teg
r
ativ
e
p
h
ar
m
ac
o
g
en
o
m
ics
in
s
titu
te
(
iPR
OM
I
SE)
,
an
d
C
en
tr
e
f
o
r
E
lectr
ical
E
n
g
in
ee
r
in
g
Stu
d
ies,
Un
iv
er
s
iti
T
ek
n
o
lo
g
i
MA
R
A,
C
awa
n
g
an
Pu
lau
Pin
an
g
f
o
r
th
eir
ass
is
tan
ce
an
d
g
u
i
d
an
ce
d
u
r
in
g
th
e
f
ield
wo
r
k
.
T
h
e
au
t
h
o
r
s
ar
e
g
r
atef
u
l
to
Un
i
v
er
s
iti
T
ek
n
o
lo
g
i
MA
R
A,
C
awa
n
g
an
Pu
lau
Pin
an
g
f
o
r
th
eir
im
m
en
s
e
ad
m
in
is
tr
ativ
e
s
u
p
p
o
r
t.
Sp
ec
ial
th
an
k
s
to
th
e
I
m
ag
in
g
Dep
ar
tm
en
t,
A
d
v
an
ce
d
Me
d
ical
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.