I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
14
,
No
.
5
,
Octo
b
er
2
0
2
5
,
p
p
.
3
7
8
1
~
3
7
8
9
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
14
.i
5
.
p
p
3
7
8
1
-
3
7
8
9
3781
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
An alg
o
rithm f
o
r
training
neural
n
etworks wi
th
L1
r
eg
ula
riza
tion
E
k
a
t
er
ina
G
riba
no
v
a
,
Ro
ma
n
G
er
a
s
im
o
v
D
e
p
a
r
t
m
e
n
t
o
f
A
u
t
o
m
a
t
e
d
C
o
n
t
r
o
l
S
y
s
t
e
m
s
,
F
a
c
u
l
t
y
o
f
C
o
n
t
r
o
l
S
y
s
t
e
m
s
,
T
o
m
s
k
S
t
a
t
e
U
n
i
v
e
r
s
i
t
y
o
f
C
o
n
t
r
o
l
S
y
s
t
e
m
s
a
n
d
R
a
d
i
o
E
l
e
c
t
r
o
n
i
c
s
,
To
ms
k
,
R
u
ssi
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Mar
31
,
2
0
2
5
R
ev
is
ed
J
u
l
22
,
2
0
2
5
Acc
ep
ted
Au
g
6
,
2
0
2
5
Th
is
p
a
p
e
r
p
re
se
n
ts
a
n
e
w
a
lg
o
r
it
h
m
fo
r
b
u
il
d
i
n
g
n
e
u
ra
l
n
e
tw
o
rk
m
o
d
e
ls
th
a
t
a
u
to
m
a
ti
c
a
ll
y
se
lec
ts
th
e
m
o
st
imp
o
rtan
t
fe
a
tu
re
s
a
n
d
p
a
ra
m
e
t
e
rs
wh
il
e
imp
ro
v
i
n
g
p
re
d
icti
o
n
a
c
c
u
ra
c
y
.
Trad
it
io
n
a
l
n
e
u
ra
l
n
e
tw
o
rk
s
o
fte
n
u
se
a
ll
a
v
a
il
a
b
le
in
p
u
t
p
a
ra
m
e
ters
,
lea
d
i
n
g
to
c
o
m
p
lex
m
o
d
e
ls
th
a
t
a
re
sl
o
w
to
train
a
n
d
p
ro
n
e
to
o
v
e
rfit
ti
n
g
.
Th
e
p
ro
p
o
se
d
a
lg
o
rit
h
m
a
d
d
re
ss
e
s
th
is
c
h
a
ll
e
n
g
e
b
y
a
u
to
m
a
ti
c
a
ll
y
i
d
e
n
t
ify
i
n
g
a
n
d
re
t
a
in
in
g
o
n
l
y
t
h
e
m
o
st
sig
n
ifi
c
a
n
t
p
a
ra
m
e
ters
d
u
ri
n
g
train
i
n
g
,
re
su
lt
i
n
g
in
sim
p
ler,
fa
ste
r,
a
n
d
m
o
re
a
c
c
u
r
a
te
m
o
d
e
ls.
We
d
e
m
o
n
stra
te
th
e
p
ra
c
ti
c
a
l
b
e
n
e
fit
s
o
f
th
e
p
ro
p
o
se
d
a
l
g
o
rit
h
m
th
r
o
u
g
h
tw
o
re
a
l
-
wo
rld
a
p
p
li
c
a
ti
o
n
s:
sto
c
k
m
a
rk
e
t
fo
re
c
a
stin
g
u
sin
g
th
e
Wi
ls
h
ire
in
d
e
x
a
n
d
b
u
si
n
e
ss
p
ro
f
it
a
b
il
i
ty
p
re
d
ict
io
n
b
a
se
d
o
n
c
o
m
p
a
n
y
fi
n
a
n
c
ial
d
a
ta.
Th
e
re
su
lt
s
sh
o
w
sig
n
ifi
c
a
n
t
imp
r
o
v
e
m
e
n
ts
o
v
e
r
c
o
n
v
e
n
ti
o
n
a
l
m
e
th
o
d
s:
m
o
d
e
ls
u
se
fe
we
r
p
a
ra
m
e
ters
–
c
re
a
ti
n
g
si
m
p
ler,
m
o
re
i
n
terp
re
tab
le
s
o
lu
ti
o
n
s
–
a
c
h
iev
e
b
e
tt
e
r
p
re
d
icti
o
n
a
c
c
u
ra
c
y
,
a
n
d
r
e
q
u
ire
les
s
train
in
g
ti
m
e
.
Th
e
se
a
d
v
a
n
ta
g
e
s
m
a
k
e
th
e
a
lg
o
rit
h
m
p
a
rti
c
u
larly
v
a
lu
a
b
le
f
o
r
b
u
si
n
e
ss
a
p
p
li
c
a
ti
o
n
s
w
h
e
re
m
o
d
e
l
sim
p
li
c
it
y
,
sp
e
e
d
,
a
n
d
a
c
c
u
ra
c
y
a
re
c
ru
c
ial.
Th
e
m
e
th
o
d
is
e
sp
e
c
ially
b
e
n
e
ficia
l
fo
r
o
r
g
a
n
iza
ti
o
n
s
wit
h
li
m
it
e
d
c
o
m
p
u
tati
o
n
a
l
re
so
u
rc
e
s
o
r
th
a
t
re
q
u
ire
fa
st
m
o
d
e
l
d
e
p
l
o
y
m
e
n
t.
B
y
a
u
t
o
m
a
ti
c
a
ll
y
se
lec
ti
n
g
t
h
e
m
o
s
t
re
lev
a
n
t
fe
a
tu
re
s,
it
re
d
u
c
e
s
th
e
n
e
e
d
f
o
r
m
a
n
u
a
l
fe
a
tu
re
e
n
g
in
e
e
rin
g
a
n
d
h
e
l
p
s
p
ra
c
ti
ti
o
n
e
rs
b
u
il
d
m
o
re
e
fficie
n
t
p
re
d
icti
v
e
m
o
d
e
ls
with
o
u
t
re
q
u
i
rin
g
d
e
e
p
tec
h
n
ica
l
e
x
p
e
rti
se
i
n
n
e
u
ra
l
n
e
tw
o
rk
o
p
t
imiz
a
ti
o
n
.
K
ey
w
o
r
d
s
:
Dr
o
p
o
u
t
I
n
v
er
s
e
p
r
o
b
lem
L
1
r
eg
u
lar
izatio
n
Neu
r
al
n
etwo
r
k
Op
tim
izatio
n
m
eth
o
d
Pru
n
in
g
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
E
k
ater
in
a
Gr
ib
an
o
v
a
Dep
ar
tm
en
t o
f
Au
to
m
ated
C
o
n
tr
o
l Sy
s
tem
s
,
Facu
lty
o
f
C
o
n
t
r
o
l Sy
s
tem
s
T
o
m
s
k
State
Un
iv
er
s
ity
o
f
C
o
n
tr
o
l Sy
s
tem
s
an
d
R
ad
io
E
lect
r
o
n
ics
Av
en
u
e.
L
e
n
in
4
0
,
T
o
m
s
k
,
R
u
s
s
ia
E
m
ail:
ek
ater
in
a.
b
.
g
r
ib
an
o
v
a
@
tu
s
u
r
.
r
u
1.
I
NT
RO
D
UCT
I
O
N
Ma
ch
in
e
lear
n
in
g
m
o
d
els,
p
a
r
ticu
lar
ly
n
eu
r
al
n
etwo
r
k
s
,
f
a
ce
a
f
u
n
d
am
en
tal
ch
allen
g
e
k
n
o
wn
as
o
v
er
f
itti
n
g
-
t
h
e
p
h
en
o
m
en
o
n
wh
er
e
a
m
o
d
el
lear
n
s
th
e
tr
ain
in
g
d
ata
to
o
clo
s
ely
,
in
cl
u
d
in
g
its
n
o
is
e
an
d
s
p
ec
if
ic
p
atter
n
s
,
r
esu
ltin
g
in
p
o
o
r
p
er
f
o
r
m
a
n
ce
wh
en
ap
p
li
ed
to
n
ew,
u
n
s
ee
n
d
ata.
T
h
is
p
r
o
b
lem
b
ec
o
m
es
esp
ec
ially
p
r
o
n
o
u
n
ce
d
as
m
o
d
els
g
r
o
w
in
co
m
p
lex
ity
a
n
d
w
h
en
tr
ain
in
g
d
ata
is
lim
ited
r
elativ
e
to
th
e
n
u
m
b
e
r
o
f
m
o
d
el
p
ar
am
eter
s
.
R
eg
u
la
r
izatio
n
em
er
g
es
as
a
c
r
itical
s
o
lu
tio
n
t
o
ad
d
r
ess
o
v
e
r
f
itti
n
g
b
y
c
o
n
s
tr
ain
in
g
m
o
d
el
co
m
p
lex
ity
an
d
en
c
o
u
r
ag
in
g
s
im
p
ler
,
m
o
r
e
g
e
n
er
aliza
b
le
s
o
lu
tio
n
s
.
T
h
e
co
r
e
p
r
in
cip
le
o
f
r
eg
u
lar
izatio
n
is
to
ad
d
co
n
s
tr
ain
ts
o
r
p
en
alties
th
at
p
r
ev
en
t
th
e
m
o
d
el
f
r
o
m
b
ec
o
m
in
g
o
v
er
ly
co
m
p
lex
,
th
er
eb
y
im
p
r
o
v
in
g
its
ab
ilit
y
t
o
p
er
f
o
r
m
well
o
n
n
ew
d
ata.
T
h
is
co
n
ce
p
t
r
ep
r
esen
ts
a
tr
ad
e
-
o
f
f
b
etwe
en
f
itti
n
g
th
e
tr
ain
in
g
d
ata
p
er
f
ec
tly
an
d
m
ain
tain
in
g
th
e
m
o
d
el'
s
ab
ilit
y
to
g
e
n
er
alize
to
u
n
s
ee
n
ex
a
m
p
les.
Var
io
u
s
r
eg
u
lar
izatio
n
a
p
p
r
o
a
ch
es
h
av
e
b
ee
n
d
e
v
elo
p
e
d
to
t
ac
k
le
th
is
ch
allen
g
e
[
1
]
.
T
h
ese
m
eth
o
d
s
ca
n
b
e
b
r
o
ad
l
y
ca
teg
o
r
ized
in
t
o
s
ev
er
al
ty
p
es:
d
ata
-
b
ased
tech
n
iq
u
es
s
u
ch
as
d
ata
au
g
m
en
tatio
n
th
at
in
cr
ea
s
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
2
0
2
5
:
3
7
8
1
-
3
7
8
9
3782
tr
ain
in
g
s
et
d
iv
er
s
ity
th
r
o
u
g
h
m
o
d
if
icatio
n
s
lik
e
ad
d
in
g
G
au
s
s
ian
n
o
is
e
[
2
]
o
r
g
eo
m
et
r
ic
tr
an
s
f
o
r
m
atio
n
s
;
ar
ch
itectu
r
al
ap
p
r
o
ac
h
es
in
clu
d
in
g
d
r
o
p
o
u
t
[
3
]
wh
ich
r
an
d
o
m
ly
d
ea
ctiv
ates
n
eu
r
o
n
s
d
u
r
in
g
tr
ain
in
g
;
o
p
tim
izatio
n
s
tr
ateg
ies
lik
e
ea
r
ly
s
to
p
p
in
g
b
ased
o
n
v
alid
ati
o
n
p
er
f
o
r
m
a
n
ce
[
4
]
;
an
d
p
en
a
lty
-
b
ased
m
eth
o
d
s
th
at
m
o
d
i
f
y
t
h
e
lo
s
s
f
u
n
cti
o
n
d
ir
ec
tly
.
Am
o
n
g
p
e
n
alty
-
b
ased
r
e
g
u
lar
izatio
n
tech
n
i
q
u
es,
L
1
an
d
L
2
r
eg
u
lar
izatio
n
h
av
e
g
ain
e
d
p
ar
ticu
lar
p
r
o
m
in
e
n
ce
d
u
e
to
th
eir
m
ath
em
atica
l
eleg
a
n
ce
an
d
p
r
ac
tical
ef
f
ec
tiv
en
ess
.
L
2
r
eg
u
lar
izati
o
n
(
also
k
n
o
w
n
as
r
id
g
e
r
e
g
r
ess
io
n
)
ad
d
s
a
p
en
alty
p
r
o
p
o
r
tio
n
al
to
th
e
s
u
m
o
f
s
q
u
ar
ed
weig
h
ts
,
en
co
u
r
ag
i
n
g
s
m
aller
weig
h
t
v
alu
es
an
d
s
m
o
o
th
er
s
o
lu
tio
n
s
.
L
1
r
eg
u
lar
iz
atio
n
,
in
tr
o
d
u
ce
d
b
y
T
ib
s
h
ir
an
i
[
5
]
as
th
e
least
ab
s
o
lu
te
s
h
r
in
k
ag
e
an
d
s
elec
tio
n
o
p
er
ato
r
(
L
ASSO)
m
eth
o
d
,
ap
p
lies
a
p
en
alty
p
r
o
p
o
r
tio
n
al
to
th
e
s
u
m
o
f
ab
s
o
lu
te
weig
h
t
v
alu
es.
T
h
e
d
is
tin
g
u
is
h
in
g
c
h
ar
ac
ter
is
tic
o
f
L
1
r
eg
u
lar
izatio
n
lies
in
its
ab
ilit
y
to
d
r
iv
e
s
o
m
e
w
eig
h
ts
to
ex
ac
tly
ze
r
o
,
ef
f
ec
ti
v
ely
p
er
f
o
r
m
in
g
a
u
to
m
atic
f
e
atu
r
e
s
elec
tio
n
.
T
h
is
p
r
o
p
er
t
y
m
ak
es
L
1
r
e
g
u
lar
iz
atio
n
p
ar
ticu
lar
ly
v
alu
ab
le
in
s
ce
n
ar
io
s
wh
er
e
id
en
tify
in
g
th
e
m
o
s
t
r
elev
an
t
f
ea
tu
r
es
is
as
im
p
o
r
tan
t
as
ac
h
iev
in
g
g
o
o
d
p
r
e
d
ictiv
e
p
er
f
o
r
m
an
ce
.
I
n
co
n
tr
ast,
L
2
r
e
g
u
l
ar
izatio
n
ty
p
ically
s
h
r
in
k
s
weig
h
ts
to
war
d
ze
r
o
with
o
u
t
elim
in
atin
g
th
em
e
n
tire
ly
.
Hy
b
r
id
ap
p
r
o
ac
h
es
lik
e
elastic
Net
[
6
]
co
m
b
in
e
b
o
th
p
e
n
alties,
wh
ile
v
ar
ian
ts
s
u
ch
as
L
0
r
eg
u
l
ar
izatio
n
[
7
]
an
d
L
1
/2
r
e
g
u
l
ar
izatio
n
[
8
]
o
f
f
e
r
d
if
f
er
en
t sp
ar
s
ity
-
in
d
u
cin
g
p
r
o
p
er
ties
.
T
h
e
p
r
ac
tical
ap
p
licatio
n
s
o
f
L
1
r
eg
u
lar
izatio
n
d
e
m
o
n
s
tr
ate
its
s
ig
n
if
ican
ce
ac
r
o
s
s
v
ar
io
u
s
d
o
m
ain
s
.
I
n
f
i
n
an
cial
m
o
d
elin
g
[
9
]
,
L
1
/
2
r
e
g
u
lar
izatio
n
h
as
b
ee
n
s
u
cc
ess
f
u
lly
ap
p
lied
to
n
eu
r
al
n
et
wo
r
k
s
f
o
r
p
r
ed
ictin
g
f
in
an
cial
d
is
tr
ess
,
en
ab
lin
g
a
u
to
m
atic
s
elec
tio
n
o
f
th
e
m
o
s
t
s
ig
n
if
ican
t
f
in
an
cial
a
n
d
n
o
n
-
f
in
an
cial
v
ar
ia
b
les
f
r
o
m
lar
g
e
f
ea
tu
r
e
s
ets.
Simila
r
ly
,
in
cr
ed
it r
is
k
ass
ess
m
en
t
[
1
0
]
,
L
1
r
eg
u
lar
izatio
n
h
elp
ed
elim
in
ate
r
ed
u
n
d
a
n
t
in
f
o
r
m
atio
n
in
d
ee
p
n
eu
r
al
n
e
two
r
k
s
,
s
ig
n
if
ican
tly
im
p
r
o
v
in
g
f
o
r
ec
asti
n
g
p
e
r
f
o
r
m
an
ce
.
M
ed
ical
ap
p
licatio
n
s
[
1
1
]
,
im
ag
e
r
ec
o
g
n
itio
n
task
s
[
1
2
]
,
[
1
3
]
,
an
d
h
ig
h
-
d
im
en
s
i
o
n
al
b
io
lo
g
ical
d
ata
a
n
aly
s
is
h
av
e
all
b
e
n
ef
ited
f
r
o
m
th
e
f
ea
tu
r
e
s
elec
tio
n
ca
p
ab
ilit
ies o
f
L
1
r
eg
u
lar
izatio
n
.
Desp
ite
th
ese
ad
v
an
tag
es,
im
p
lem
en
tin
g
L
1
r
e
g
u
lar
izati
o
n
in
n
eu
r
al
n
etwo
r
k
tr
ain
i
n
g
p
r
esen
ts
s
ig
n
if
ican
t
co
m
p
u
tatio
n
al
c
h
allen
g
es.
T
h
e
a
b
s
o
lu
te
v
al
u
e
f
u
n
ctio
n
in
th
e
L
1
p
en
alty
ter
m
is
n
o
n
-
d
if
f
e
r
en
tiab
le
at
ze
r
o
,
m
ak
in
g
s
tan
d
a
r
d
g
r
a
d
ien
t
-
b
ased
o
p
tim
izatio
n
m
et
h
o
d
s
p
r
o
b
lem
atic.
E
x
is
tin
g
a
p
p
r
o
ac
h
es
in
clu
d
e
h
eu
r
is
tic
m
eth
o
d
s
th
at
ex
clu
d
e
p
ar
am
eter
s
b
ased
o
n
th
eir
co
n
tr
ib
u
tio
n
r
atio
s
[
1
4
]
,
s
m
o
o
th
in
g
tech
n
iq
u
es u
s
in
g
p
iece
wis
e
p
o
ly
n
o
m
ial
a
p
p
r
o
x
im
atio
n
s
[
1
5
]
,
an
d
co
o
r
d
in
at
e
d
escen
t
alg
o
r
ith
m
s
with
cr
o
s
s
-
v
alid
atio
n
[
1
6
]
.
Ho
wev
er
,
th
ese
m
eth
o
d
s
o
f
te
n
s
u
f
f
er
f
r
o
m
c
o
m
p
u
tatio
n
al
in
ef
f
icien
cy
,
c
o
n
v
e
r
g
en
ce
d
if
f
icu
lties
,
o
r
r
eq
u
i
r
e
ex
ten
s
iv
e
h
y
p
e
r
p
ar
am
ete
r
tu
n
i
n
g
.
T
h
e
m
o
tiv
atio
n
f
o
r
th
is
r
esear
ch
s
tem
s
f
r
o
m
th
e
n
ee
d
to
o
v
er
co
m
e
th
ese
co
m
p
u
tatio
n
al
b
ar
r
ier
s
wh
ile
p
r
eser
v
in
g
th
e
v
alu
ab
l
e
f
ea
tu
r
e
s
elec
tio
n
p
r
o
p
er
tie
s
o
f
L
1
r
eg
u
lar
izatio
n
.
C
u
r
r
en
t
m
eth
o
d
s
eith
er
co
m
p
r
o
m
is
e
o
n
th
e
ex
ac
tn
ess
o
f
th
e
L
1
p
en
alty
o
r
r
eq
u
i
r
e
co
m
p
u
tatio
n
ally
e
x
p
en
s
iv
e
p
r
o
ce
d
u
r
es
th
at
lim
it
th
eir
p
r
ac
tical
a
p
p
licab
ilit
y
,
es
p
ec
ially
f
o
r
lar
g
e
-
s
ca
le
n
eu
r
al
n
etwo
r
k
s
.
T
h
is
s
tu
d
y
a
d
d
r
ess
es
th
ese
lim
itatio
n
s
b
y
p
r
o
p
o
s
in
g
a
n
o
v
el
o
p
tim
i
za
tio
n
ap
p
r
o
ac
h
th
at
r
ef
o
r
m
u
lates
th
e
L
1
-
r
eg
u
lar
ize
d
n
e
u
r
al
n
etwo
r
k
tr
ain
in
g
p
r
o
b
lem
as
an
in
v
er
s
e
s
in
g
le
-
p
o
in
t
p
r
o
b
lem
.
Ou
r
co
n
tr
i
b
u
ti
o
n
lies
in
d
ev
elo
p
in
g
a
co
m
p
u
tatio
n
ally
ef
f
icien
t
alg
o
r
ith
m
th
at
m
ain
tai
n
s
th
e
th
eo
r
etica
l
p
r
o
p
er
ties
o
f
L
1
r
eg
u
lar
izatio
n
.
T
h
e
p
r
ac
tical
s
ig
n
if
ican
ce
o
f
th
is
wo
r
k
e
x
ten
d
s
to
ap
p
licatio
n
s
r
eq
u
ir
in
g
b
o
th
h
ig
h
p
r
e
d
ictiv
e
ac
cu
r
ac
y
an
d
m
o
d
el
i
n
ter
p
r
etab
ilit
y
,
in
clu
d
in
g
m
ed
ical
d
iag
n
o
s
is
,
f
in
an
cial
m
o
d
elin
g
,
an
d
s
cien
tific
r
ese
ar
ch
wh
er
e
u
n
d
e
r
s
tan
d
in
g
f
e
atu
r
e
im
p
o
r
tan
ce
is
cr
u
cial
f
o
r
d
ec
is
io
n
-
m
ak
i
n
g
.
2.
M
AT
E
R
I
AL
S AN
D
M
E
T
H
O
D
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
th
eo
r
etica
l
f
o
u
n
d
atio
n
an
d
m
e
th
o
d
o
lo
g
ical
f
r
am
ewo
r
k
f
o
r
th
e
s
tu
d
y
.
T
h
e
f
ir
s
t
c
o
m
p
o
n
en
t c
o
v
er
s
n
e
u
r
al
n
etwo
r
k
a
r
ch
itectu
r
e
an
d
L
1
r
e
g
u
lar
izatio
n
tech
n
iq
u
es
f
o
r
f
ea
t
u
r
e
s
elec
tio
n
an
d
s
p
ar
s
ity
p
r
o
m
o
tio
n
.
T
h
e
s
ec
o
n
d
c
o
m
p
o
n
en
t
d
etails
a
n
o
v
el
t
r
ain
in
g
alg
o
r
ith
m
b
a
s
ed
o
n
c
o
n
s
tr
ain
ed
o
p
tim
izatio
n
p
r
i
n
cip
les with
s
elec
tiv
e
weig
h
t u
p
d
ate
m
ec
h
a
n
is
m
s
.
2
.
1
.
Neura
l net
wo
r
k
A
n
e
u
r
a
l
n
e
t
w
o
r
k
i
s
a
c
o
m
p
u
t
a
t
i
o
n
a
l
m
o
d
e
l
i
n
s
p
i
r
e
d
b
y
t
h
e
w
a
y
b
i
o
l
o
g
i
c
a
l
n
e
u
r
a
l
n
e
tw
o
r
k
s
p
r
o
c
e
s
s
i
n
f
o
r
m
a
t
i
o
n
[
1
7
]
.
A
t
it
s
c
o
r
e
,
a
n
e
u
r
a
l
n
e
t
w
o
r
k
c
o
n
s
is
t
s
o
f
in
t
e
r
c
o
n
n
e
c
t
e
d
p
r
o
c
e
s
s
i
n
g
u
n
i
ts
c
a
ll
e
d
n
e
u
r
o
n
s
o
r
n
o
d
e
s
,
o
r
g
a
n
i
z
e
d
i
n
l
a
y
e
r
s
t
h
a
t
t
r
a
n
s
f
o
r
m
i
n
p
u
t
d
a
t
a
i
n
t
o
m
e
a
n
i
n
g
f
u
l
o
u
t
p
u
t
s
t
h
r
o
u
g
h
l
e
a
r
n
e
d
m
a
t
h
e
m
a
t
i
c
al
o
p
e
r
a
t
i
o
n
s
.
T
h
e
f
u
n
d
a
m
e
n
t
a
l
ar
c
h
i
t
e
c
t
u
r
e
c
o
m
p
r
is
e
s
t
h
r
e
e
m
a
i
n
c
o
m
p
o
n
e
n
t
s
:
a
n
i
n
p
u
t
l
a
y
er
t
h
a
t
r
e
c
e
i
v
es
d
a
t
a
,
o
n
e
o
r
m
o
r
e
h
i
d
d
e
n
l
a
y
e
r
s
t
h
a
t
p
e
r
f
o
r
m
i
n
t
e
r
m
e
d
ia
t
e
c
o
m
p
u
t
a
t
i
o
n
s
,
a
n
d
a
n
o
u
t
p
u
t
l
a
y
e
r
t
h
a
t
p
r
o
d
u
c
e
s
f
i
n
al
p
r
e
d
i
c
t
i
o
n
s
F
i
g
u
r
e
1
.
E
a
c
h
c
o
n
n
e
c
t
i
o
n
b
e
t
w
ee
n
n
e
u
r
o
n
s
h
a
s
a
n
a
s
s
o
c
i
at
e
d
w
ei
g
h
t
t
h
at
d
e
te
r
m
i
n
e
s
t
h
e
s
t
r
e
n
g
t
h
a
n
d
d
i
r
e
c
t
i
o
n
o
f
i
n
f
o
r
m
a
ti
o
n
f
lo
w
.
T
h
e
n
e
t
w
o
r
k
l
e
a
r
n
s
b
y
a
d
j
u
s
t
i
n
g
t
h
e
s
e
w
ei
g
h
t
s
d
u
r
i
n
g
t
r
a
i
n
i
n
g
t
o
m
i
n
i
m
i
z
e
p
r
e
d
i
c
t
i
o
n
e
r
r
o
r
s
.
F
o
r
m
a
t
h
e
m
a
ti
c
a
l
f
o
r
m
u
l
at
i
o
n
,
c
o
n
s
i
d
e
r
i
n
p
u
t
d
a
t
a
X
w
it
h
d
i
m
e
n
s
i
o
n
s
N
×
M
∈
ℝ
,
w
h
e
r
e
N
is
t
h
e
n
u
m
b
e
r
o
f
o
b
s
e
r
v
a
t
i
o
n
s
,
a
n
d
M
i
s
t
h
e
n
u
m
b
e
r
o
f
f
e
a
t
u
r
e
s
.
T
h
e
a
c
t
u
a
l
o
u
t
p
u
t
v
a
r
i
a
b
l
e
Y
h
a
s
d
i
m
e
n
s
i
o
n
N
×
1
.
T
h
e
n
e
t
w
o
r
k
p
r
o
c
e
s
s
es
i
n
f
o
r
m
a
t
i
o
n
t
h
r
o
u
g
h
s
u
c
c
es
s
i
v
e
t
r
a
n
s
f
o
r
m
a
ti
o
n
s
.
T
h
e
h
i
d
d
e
n
l
a
y
e
r
v
a
l
u
e
s
a
r
e
c
o
m
p
u
t
e
d
a
s
:
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
a
lg
o
r
ith
m
fo
r
tr
a
in
in
g
n
eu
r
a
l n
etw
o
r
ks w
ith
L1
r
eg
u
la
r
iz
a
tio
n
…
(
E
ka
teri
n
a
Gri
b
a
n
o
va
)
3783
=
1
(
⋅
0
)
w
h
e
r
e
w
0
a
r
e
t
h
e
w
e
i
g
h
ts
o
f
t
h
e
f
i
r
s
t
l
a
y
e
r
,
f
1
is
t
h
e
a
c
t
i
v
a
ti
o
n
f
u
n
c
t
i
o
n
o
f
t
h
e
f
i
r
s
t
l
a
y
e
r
.
A
c
t
iv
a
t
i
o
n
f
u
n
c
t
i
o
n
s
a
r
e
c
r
u
c
i
a
l
c
o
m
p
o
n
e
n
t
s
t
h
a
t
e
n
a
b
l
e
n
e
u
r
a
l
n
e
t
w
o
r
k
s
t
o
ca
p
t
u
r
e
c
o
m
p
l
e
x
,
n
o
n
-
l
i
n
e
a
r
r
e
l
a
t
i
o
n
s
h
i
p
s
i
n
d
a
t
a
.
B
as
e
d
o
n
t
h
e
v
a
l
u
es
o
f
n
e
u
r
o
n
s
o
f
t
h
e
h
i
d
d
e
n
l
a
y
e
r
l
,
t
h
e
o
u
t
p
u
t
o
f
t
h
e
n
e
u
r
a
l
n
et
w
o
r
k
is
c
a
l
c
u
l
at
e
d
u
s
i
n
g
t
h
e
f
o
r
m
u
l
a
:
=
2
(
⋅
1
)
w
h
e
r
e
w
1
a
r
e
t
h
e
w
e
i
g
h
ts
c
o
n
n
e
c
t
i
n
g
t
h
e
h
i
d
d
e
n
l
a
y
e
r
t
o
t
h
e
o
u
t
p
u
t
l
a
y
e
r
,
a
n
d
f
2
i
s
t
h
e
a
c
t
i
v
ati
o
n
f
u
n
c
t
i
o
n
o
f
t
h
e
o
u
t
p
u
t
l
a
y
e
r
.
B
i
a
s
t
e
r
m
s
a
r
e
in
c
o
r
p
o
r
a
t
e
d
b
y
a
d
d
i
n
g
c
o
l
u
m
n
s
o
f
o
n
e
s
t
o
b
o
t
h
a
n
d
l
m
a
t
r
i
c
es
,
p
r
o
v
i
d
i
n
g
a
d
d
i
t
i
o
n
a
l
f
l
e
x
i
b
i
l
it
y
i
n
m
o
d
e
l
f
i
t
t
i
n
g
.
T
h
e
a
d
j
u
s
t
m
e
n
t
o
f
t
h
e
n
e
t
w
o
r
k
’
s
w
e
i
g
h
ts
is
p
e
r
f
o
r
m
e
d
u
s
i
n
g
o
p
t
i
m
i
z
a
ti
o
n
a
l
g
o
r
i
t
h
m
s
,
t
y
p
i
ca
l
l
y
b
a
s
e
d
o
n
g
r
a
d
i
e
n
t
d
e
s
c
e
n
t
.
T
h
e
f
u
n
c
t
i
o
n
t
o
b
e
o
p
t
i
m
i
z
e
d
is
t
h
e
e
r
r
o
r
f
u
n
c
t
i
o
n
,
w
h
i
c
h
r
e
f
l
e
c
ts
t
h
e
d
e
v
i
a
t
i
o
n
o
f
p
r
e
d
i
c
t
e
d
v
a
l
u
e
s
f
r
o
m
a
ct
u
a
l
v
al
u
e
s
,
as
d
e
f
i
n
e
d
i
n
(
1
)
:
(
)
=
∑
(
−
)
2
=
1
(
1
)
wh
er
e
y
is
th
e
v
ec
to
r
o
f
ac
tu
al
o
u
tp
u
t
v
alu
es;
z
is
th
e
v
ec
to
r
o
f
p
r
ed
icted
v
alu
es;
an
d
N
is
th
e
n
u
m
b
er
o
f
o
b
s
er
v
atio
n
s
.
I
n
class
ical
g
r
ad
ien
t
d
escen
t,
a
f
ter
a
f
o
r
war
d
p
ass
,
a
b
ac
k
war
d
p
ass
is
ex
ec
u
ted
t
o
co
m
p
u
te
g
r
ad
ien
ts
u
s
ed
f
o
r
u
p
d
atin
g
weig
h
ts
(
′
is
th
e
d
er
iv
ativ
e
o
f
th
e
ac
tiv
atio
n
f
u
n
ctio
n
)
.
T
h
u
s
,
th
e
er
r
o
r
at
th
e
o
u
tp
u
t
la
y
er
is
ca
lcu
lated
as:
=
−
=
⊙
2
′
(
)
wh
er
e
⊙
is
th
e
elem
en
t
-
wis
e
m
u
ltip
licatio
n
.
T
h
e
er
r
o
r
at
th
e
h
id
d
en
lay
er
c
an
b
e
d
e
f
in
ed
as:
=
⋅
1
=
⊙
1
′
(
)
C
o
n
s
eq
u
en
tly
,
th
e
weig
h
t u
p
d
ates f
o
r
th
e
f
ir
s
t a
n
d
s
ec
o
n
d
la
y
er
s
ar
e
p
er
f
o
r
m
e
d
as
(
2
)
:
0
=
0
−
⋅
1
=
1
−
⋅
(
2
)
wh
er
e
is
th
e
lear
n
in
g
r
ate.
T
h
e
L
1
r
eg
u
lar
izatio
n
tech
n
iq
u
e
ca
n
b
e
f
o
r
m
u
lated
u
s
in
g
th
e
L
ag
r
an
g
ia
n
ap
p
r
o
a
ch
,
wh
ich
tr
an
s
f
o
r
m
s
th
e
co
n
s
tr
ain
ed
o
p
t
im
izatio
n
p
r
o
b
lem
in
t
o
an
u
n
c
o
n
s
tr
ain
ed
f
o
r
m
.
T
h
is
f
o
r
m
u
la
tio
n
in
tr
o
d
u
ce
s
th
e
r
eg
u
lar
izatio
n
p
ar
am
eter
λ
th
a
t
co
n
tr
o
ls
th
e
tr
a
d
e
-
o
f
f
b
etwe
en
m
o
d
el
f
itti
n
g
ac
cu
r
ac
y
a
n
d
s
p
ar
s
ity
,
allo
win
g
p
r
ac
titi
o
n
er
s
to
ad
ju
s
t
th
e
le
v
el
o
f
f
ea
tu
r
e
s
elec
tio
n
ac
co
r
d
i
n
g
to
th
eir
s
p
ec
if
ic
r
eq
u
ir
em
e
n
ts
.
T
h
e
o
p
tim
ized
f
u
n
ctio
n
(
1
)
with
L
1
r
e
g
u
lar
iz
atio
n
in
L
ag
r
an
g
ian
f
o
r
m
ca
n
b
e
r
ep
r
esen
ted
as
(
3
)
:
̃
(
)
=
(
)
+
‖
‖
1
(
3
)
Fig
u
r
e
1
.
Neu
r
al
n
etwo
r
k
s
tr
u
ctu
r
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
2
0
2
5
:
3
7
8
1
-
3
7
8
9
3784
Gr
ap
h
ically
,
th
e
r
eg
u
lar
izatio
n
ter
m
tak
es
o
n
a
d
iam
o
n
d
s
h
ap
e,
wh
ich
en
co
u
r
a
g
es
s
o
m
e
weig
h
t
co
ef
f
icien
ts
to
b
ec
o
m
e
ze
r
o
,
as
th
e
m
in
im
u
m
is
o
f
ten
r
ea
ch
e
d
at
its
co
r
n
er
s
(
Fig
u
r
e
2
)
.
Fig
u
r
e
2
illu
s
tr
ates
th
e
g
eo
m
etr
ic
in
ter
p
r
etatio
n
o
f
L
1
r
eg
u
la
r
izatio
n
in
a
two
-
d
i
m
en
s
io
n
al
p
ar
am
ete
r
s
p
ac
e,
wh
er
e
th
e
d
iam
o
n
d
-
s
h
ap
ed
co
n
s
tr
ain
t
r
e
g
io
n
(
s
h
o
wn
in
d
ar
k
b
lu
e)
r
e
p
r
esen
ts
th
e
L
1
p
en
alty
ter
m
|
|
w
|
|
₁
≤
t
f
o
r
s
o
m
e
th
r
esh
o
l
d
t
.
T
h
e
ellip
tical
co
n
to
u
r
s
(
s
h
o
wn
in
g
r
ee
n
an
d
lig
h
t
b
lu
e)
r
e
p
r
esen
t
th
e
is
o
-
lin
es
o
f
th
e
er
r
o
r
f
u
n
ctio
n
J
(
w
)
,
with
th
e
in
n
er
m
o
s
t
co
n
to
u
r
in
d
icatin
g
th
e
u
n
co
n
s
tr
ain
ed
m
i
n
im
u
m
o
f
t
h
e
lo
s
s
f
u
n
ctio
n
.
T
h
e
o
p
tim
al
s
o
lu
tio
n
u
n
d
er
L
1
r
eg
u
la
r
izatio
n
o
cc
u
r
s
at
th
e
p
o
in
t
wh
er
e
th
e
s
m
allest
er
r
o
r
f
u
n
ctio
n
co
n
to
u
r
to
u
c
h
es
th
e
d
iam
o
n
d
-
s
h
ap
e
d
co
n
s
tr
ain
t
r
e
g
io
n
.
Du
e
t
o
th
e
s
h
ar
p
co
r
n
er
s
o
f
t
h
e
d
iam
o
n
d
co
n
s
tr
ain
t,
th
is
in
ter
s
ec
tio
n
f
r
eq
u
en
tly
o
cc
u
r
s
at
th
e
ax
es
(
wh
er
e
o
n
e
p
ar
am
ete
r
eq
u
als
ze
r
o
)
,
d
em
o
n
s
tr
atin
g
h
o
w
L
1
r
eg
u
lar
izatio
n
n
atu
r
al
ly
p
r
o
d
u
c
es
s
p
ar
s
e
s
o
lu
tio
n
s
b
y
s
ettin
g
s
o
m
e
c
o
ef
f
icien
ts
to
ex
ac
tly
ze
r
o
.
Fig
u
r
e
2
.
Gr
a
p
h
ical
r
ep
r
esen
tatio
n
o
f
th
e
p
r
o
b
lem
2
.
2
.
Neura
l net
wo
r
k
t
r
a
ini
ng
a
lg
o
rit
h
m
T
h
e
p
r
o
p
o
s
ed
n
eu
r
al
n
etwo
r
k
tr
ain
in
g
alg
o
r
ith
m
is
b
ased
o
n
r
ef
o
r
m
u
latin
g
th
e
m
in
im
izatio
n
p
r
o
b
lem
o
f
f
u
n
ctio
n
(
3
)
in
to
a
co
n
s
t
r
ain
ed
o
p
tim
izatio
n
p
r
o
b
lem
.
T
h
is
r
ef
o
r
m
u
latio
n
en
a
b
les
th
e
ap
p
licatio
n
o
f
m
eth
o
d
s
f
o
r
s
o
lv
in
g
in
v
er
s
e
s
in
g
le
-
p
o
in
t
p
r
o
b
lem
s
[
1
8
]
,
[
1
9
]
to
th
e
n
eu
r
al
n
etwo
r
k
lea
r
n
in
g
p
r
o
ce
s
s
.
I
n
t
h
e
r
ef
o
r
m
u
lated
p
r
o
b
lem
,
t
h
e
s
u
m
o
f
th
e
ab
s
o
lu
te
v
alu
es
o
f
th
e
p
ar
am
ete
r
s
is
m
in
im
ized
wh
ile
th
e
tr
ain
i
n
g
er
r
o
r
is
co
n
s
tr
ai
n
ed
to
a
s
p
ec
if
ied
ta
r
g
et
v
alu
e
,
as
d
ef
in
ed
i
n
(
4
)
:
‖
‖
1
→
(
)
=
∗
(
4
)
wh
er
e
∗
is
th
e
tar
g
et
v
alu
e
o
f
th
e
co
n
s
tr
ain
t f
u
n
ctio
n
.
T
h
e
s
o
lu
tio
n
ap
p
r
o
ac
h
in
v
o
l
v
es
iter
ativ
ely
u
p
d
atin
g
ar
g
u
m
en
ts
ac
co
r
d
in
g
to
th
e
f
o
llo
win
g
r
u
le:
at
ea
ch
iter
atio
n
,
s
elec
t
an
a
r
g
u
m
en
t
f
o
r
wh
ic
h
th
e
a
b
s
o
lu
te
v
alu
e
o
f
th
e
p
a
r
tial
d
er
iv
ati
v
e
o
f
t
h
e
co
n
s
tr
ain
t
f
u
n
ctio
n
is
m
ax
im
ized
,
th
e
n
ad
ju
s
t
its
v
alu
e
u
s
in
g
g
r
ad
ien
t
d
escen
t
[
1
8
]
,
[
1
9
]
.
T
h
e
s
to
p
p
in
g
co
n
d
itio
n
s
f
o
r
iter
atio
n
s
in
clu
d
e
r
ea
ch
i
n
g
th
e
tar
g
et
v
alu
e
o
f
th
e
co
n
s
tr
ain
t
f
u
n
ctio
n
with
a
s
p
ec
if
ied
ac
cu
r
ac
y
o
r
lack
o
f
im
p
r
o
v
em
e
n
t
in
s
o
lu
tio
n
s
,
as
well
as
r
ea
ch
in
g
a
p
r
ed
eter
m
in
ed
n
u
m
b
er
o
f
iter
atio
n
s
.
T
h
er
e
m
ay
b
e
ca
s
es
wh
er
e
th
e
tar
g
et
v
alu
e
is
n
o
t
ac
h
iev
ed
d
u
r
in
g
p
r
o
b
lem
-
s
o
lv
in
g
.
I
n
s
u
ch
in
s
tan
ce
s
,
v
alu
es
o
b
tain
ed
t
h
r
o
u
g
h
o
th
er
s
to
p
p
in
g
cr
iter
ia
a
r
e
c
o
n
s
id
er
ed
as
s
o
lu
tio
n
s
,
b
ein
g
as
clo
s
e
as
p
o
s
s
ib
le
to
th
e
tar
g
et
v
alu
e
o
f
th
e
co
n
s
tr
ain
t
f
u
n
ctio
n
with
th
is
s
ea
r
ch
s
tr
ateg
y
.
I
n
th
is
s
tu
d
y
,
th
e
tar
g
et
v
al
u
e
∗
is
ass
u
m
ed
to
b
e
a
s
m
all
ar
b
itra
r
y
n
u
m
b
e
r
,
u
n
d
er
th
e
a
s
s
u
m
p
tio
n
th
at
th
is
v
alu
e
will
n
o
t
b
e
r
ea
c
h
ed
.
T
h
is
ass
u
m
p
tio
n
is
im
p
lem
en
ted
b
y
ex
clu
d
in
g
f
r
o
m
th
e
s
to
p
p
i
n
g
r
u
les
ac
h
iev
in
g
th
e
tar
g
et
v
alu
e
o
f
th
e
co
n
s
tr
ain
t
f
u
n
ct
io
n
with
s
p
ec
if
ied
ac
cu
r
ac
y
.
T
h
is
ap
p
r
o
ac
h
e
n
s
u
r
es
d
if
f
e
r
e
n
tiab
ilit
y
o
f
th
e
o
b
jectiv
e
f
u
n
ctio
n
an
d
d
o
es
n
o
t
r
eq
u
i
r
e
tu
n
in
g
o
f
th
e
r
eg
u
lar
izatio
n
p
ar
a
m
eter
.
I
t
is
also
wo
r
th
n
o
tin
g
th
at
th
e
in
itial
v
alu
es
o
f
th
e
weig
h
t
c
o
e
f
f
icien
ts
ar
e
s
et
t
o
ze
r
o
,
r
ath
er
th
an
b
ei
n
g
g
en
e
r
ated
r
an
d
o
m
l
y
as
in
well
-
k
n
o
wn
alg
o
r
ith
m
s
.
Fo
r
ea
ch
weig
h
t
co
ef
f
icien
t
w
,
an
ad
d
itio
n
al
v
alu
e
u
is
d
ef
i
n
e
d
to
in
d
icate
its
ap
p
licab
ilit
y
i
n
ca
lcu
latio
n
s
.
T
h
is
f
ea
tu
r
e
ca
n
tak
e
two
v
alu
es:
0
o
r
1
,
r
ef
lectin
g
wh
eth
er
th
e
co
r
r
esp
o
n
d
in
g
weig
h
t
co
ef
f
icien
t
ca
n
b
e
m
o
d
if
ied
in
th
e
cu
r
r
en
t
iter
atio
n
.
Mo
d
if
icatio
n
is
ex
clu
d
ed
i
f
a
d
ju
s
tin
g
th
e
weig
h
t
co
ef
f
icien
t
in
th
e
p
r
ev
io
u
s
iter
atio
n
r
esu
lted
in
a
wo
r
s
en
in
g
o
f
th
e
o
p
tim
ized
er
r
o
r
f
u
n
ctio
n
.
T
h
e
p
r
o
p
o
s
ed
tr
ain
in
g
alg
o
r
ith
m
in
co
r
p
o
r
ates
a
s
elec
tiv
e
weig
h
t
u
p
d
ate
m
ec
h
an
is
m
b
ased
o
n
g
r
ad
ien
t
m
a
g
n
itu
d
e.
T
h
is
ap
p
r
o
ac
h
aim
s
to
im
p
r
o
v
e
c
o
n
v
er
g
en
ce
e
f
f
icien
cy
b
y
p
r
ev
en
tin
g
u
n
n
ec
ess
ar
y
u
p
d
ates
o
f
weig
h
ts
.
T
h
e
m
ec
h
an
is
m
d
y
n
am
ically
ad
ju
s
ts
weig
h
t
m
o
d
if
icatio
n
p
r
io
r
iti
es
d
u
r
in
g
tr
ain
in
g
,
en
s
u
r
in
g
th
at
c
o
m
p
u
tatio
n
al
r
e
s
o
u
r
ce
s
ar
e
f
o
c
u
s
ed
o
n
t
h
e
m
o
s
t b
en
ef
icial
p
ar
am
eter
a
d
ju
s
tm
en
ts
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
a
lg
o
r
ith
m
fo
r
tr
a
in
in
g
n
eu
r
a
l n
etw
o
r
ks w
ith
L1
r
eg
u
la
r
iz
a
tio
n
…
(
E
ka
teri
n
a
Gri
b
a
n
o
va
)
3785
T
h
e
alg
o
r
ith
m
f
o
r
a
s
in
g
le
h
id
d
en
lay
er
will in
clu
d
e
th
e
f
o
llo
win
g
s
tep
s
:
Step
1
.
E
r
r
o
r
f
u
n
ctio
n
ca
lc
u
latio
n
: c
alcu
late
th
e
cu
r
r
en
t e
r
r
o
r
f
u
n
ctio
n
v
alu
e:
J
prev
=
J
(
w
)
(
1
)
.
Step
2
.
Gr
ad
ien
t
c
o
m
p
u
tatio
n
: c
alcu
late
g
r
ad
ien
t
v
alu
es f
o
r
a
ll we
ig
h
t c
o
ef
f
icien
ts
(
2
)
:
0
=
1
=
Step
3
.
Selectiv
e
weig
h
t u
p
d
ate
: d
eter
m
in
e
th
e
m
ax
im
u
m
v
al
u
e
o
f
th
e
p
r
o
d
u
ct
b
etwe
en
th
e
g
r
ad
ien
t v
alu
e
an
d
th
e
ap
p
licab
le
weig
h
t
:
=
{
|
0
|
⋅
0
,
|
1
|
⋅
1
}
I
f
th
is
m
a
x
im
u
m
v
alu
e
co
r
r
esp
o
n
d
s
t
o
th
e
weig
h
t
co
ef
f
icien
ts
co
n
n
ec
tin
g
th
e
h
id
d
e
n
lay
e
r
to
th
e
o
u
tp
u
t
la
y
er
(
j
is
th
e
in
d
ex
o
f
th
e
m
a
x
im
u
m
elem
en
t)
,
th
e
n
th
e
weig
h
t c
o
ef
f
icien
t o
f
w
1
is
ad
ju
s
ted
:
∗
1
=
1
−
⋅
1
wh
er
e
is
th
e
p
a
r
am
eter
d
e
f
in
in
g
th
e
s
tep
o
f
weig
h
t
c
o
ef
f
icien
t
ch
an
g
e.
Oth
er
wis
e,
u
p
d
ate
th
e
weig
h
t
co
ef
f
icien
t c
o
n
n
ec
tin
g
t
h
e
in
p
u
t la
y
er
to
th
e
in
ter
m
ed
iate
lay
er
:
∗
0
=
0
−
⋅
0
Step
4
.
Per
f
o
r
m
an
ce
e
v
alu
at
io
n
an
d
ad
a
p
tatio
n
:
c
alcu
lat
e
th
e
n
ew
er
r
o
r
f
u
n
ctio
n
v
alu
e:
J
new
=
J
(
w
).
I
f
J
new
<
J
prev
,
u
v
alu
es a
r
e
s
et
to
1
f
o
r
all
weig
h
t c
o
ef
f
icien
ts
,
w
=
w
*
,
J
prev
=
J
new
.
Go
to
s
tep
2
.
Oth
er
wis
e,
th
e
v
alu
e
u
f
o
r
th
e
co
r
r
esp
o
n
d
in
g
m
o
d
i
f
ied
weig
h
t
co
ef
f
icien
t
is
s
et
t
o
ze
r
o
:
0
=
0
(
if
0
was
ch
an
g
ed
)
o
r
1
=
0
(
if
1
was c
h
an
g
ed
)
.
Go
t
o
s
tep
2
.
S
t
o
p
p
i
n
g
c
r
i
t
e
r
i
o
n
:
e
i
t
h
e
r
al
l
v
al
u
e
s
o
f
u
a
r
e
e
q
u
a
l
t
o
0
,
o
r
t
h
e
s
p
e
c
i
f
i
e
d
n
u
m
b
e
r
o
f
i
t
e
r
a
ti
o
n
s
h
a
s
b
e
e
n
c
o
m
p
l
e
te
d
.
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
em
p
ir
ical
v
alid
atio
n
o
f
th
e
p
r
o
p
o
s
ed
n
eu
r
al
n
etwo
r
k
tr
ain
in
g
alg
o
r
ith
m
th
r
o
u
g
h
co
m
p
u
tatio
n
al
ex
p
er
im
en
ts
.
T
h
e
ev
alu
atio
n
is
s
t
r
u
ctu
r
ed
in
to
two
m
ain
p
a
r
ts
to
s
y
s
tem
atica
lly
d
em
o
n
s
tr
ate
th
e
alg
o
r
ith
m
'
s
ef
f
ec
tiv
en
ess
ac
r
o
s
s
d
if
f
er
en
t
n
etwo
r
k
ar
c
h
itectu
r
es
an
d
a
p
p
licatio
n
d
o
m
ain
s
.
First,
we
ex
am
in
e
th
e
alg
o
r
ith
m
'
s
p
er
f
o
r
m
an
ce
o
n
s
in
g
le
-
l
ay
er
n
etwo
r
k
s
u
s
in
g
f
in
a
n
cial
tim
e
s
er
ies
d
ata
as
p
r
esen
ted
in
s
ec
tio
n
3
.
1
.
T
h
is
an
aly
s
is
f
o
cu
s
es
o
n
f
ea
tu
r
e
s
elec
tio
n
ca
p
ab
ilit
ies
an
d
p
r
ed
ictio
n
ac
cu
r
ac
y
in
f
o
r
ec
asti
n
g
s
to
ck
m
ar
k
et
in
d
ic
es.
Seco
n
d
,
we
in
v
esti
g
ate
th
e
alg
o
r
ith
m
'
s
ap
p
licatio
n
t
o
n
e
u
r
al
n
etwo
r
k
s
with
h
id
d
en
lay
er
s
u
s
in
g
en
te
r
p
r
is
e
f
in
an
cial
d
ata
as
d
is
cu
s
s
ed
in
s
ec
tio
n
3
.
2
,
em
p
h
asizin
g
p
ar
am
eter
r
ed
u
ctio
n
.
Fo
r
ea
ch
ex
p
er
im
en
tal
s
ettin
g
,
we
co
m
p
ar
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
ag
ai
n
s
t
estab
lis
h
ed
b
aselin
e
ap
p
r
o
ac
h
es,
in
clu
d
in
g
s
tan
d
ar
d
ad
ap
tiv
e
m
o
m
en
t
esti
m
atio
n
(
Ad
am
)
o
p
tim
izatio
n
with
v
ar
io
u
s
L
1
r
eg
u
lar
izatio
n
p
ar
am
eter
s
an
d
d
r
o
p
o
u
t
tech
n
iq
u
es.
T
h
e
r
eliab
ilit
y
o
f
o
u
r
r
esu
lts
is
en
s
u
r
ed
th
r
o
u
g
h
r
ig
o
r
o
u
s
ex
p
e
r
im
en
tal
d
esig
n
,
co
m
p
a
r
is
o
n
with
well
-
estab
lis
h
ed
o
p
tim
izatio
n
m
eth
o
d
s
u
n
d
e
r
id
en
tical
co
n
d
itio
n
s
,
an
d
u
s
e
o
f
d
if
f
er
en
t a
ctiv
atio
n
f
u
n
ctio
n
s
an
d
n
etwo
r
k
co
n
f
ig
u
r
atio
n
s
.
T
h
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
was
im
p
lem
en
te
d
in
Py
th
o
n
u
s
in
g
n
ativ
e
Nu
m
Py
o
p
er
atio
n
s
f
o
r
m
atr
i
x
co
m
p
u
tatio
n
s
an
d
g
r
ad
ie
n
t
ca
lcu
latio
n
s
,
wh
ile
th
e
o
p
tim
izatio
n
m
eth
o
d
Ad
a
m
was
im
p
lem
en
ted
in
Py
th
o
n
u
s
in
g
th
e
Ker
as
lib
r
ar
y
.
Ad
d
itio
n
ally
,
we
im
p
le
m
en
ted
L
1
r
eg
u
lar
izatio
n
tec
h
n
iq
u
es
a
n
d
h
y
p
er
p
ar
am
eter
o
p
tim
izatio
n
u
s
in
g
Gr
id
Sear
c
h
C
V
f
r
o
m
th
e
s
cik
it
-
lear
n
li
b
r
ar
y
[
2
0
]
in
th
e
Ad
am
m
eth
o
d
.
Gr
i
d
Sear
ch
C
V
p
er
f
o
r
m
s
s
ea
r
ch
o
v
er
s
p
ec
if
ie
d
p
ar
am
ete
r
v
alu
es,
e
v
alu
atin
g
ea
ch
c
o
m
b
in
atio
n
th
r
o
u
g
h
cr
o
s
s
-
v
alid
atio
n
to
id
en
tify
o
p
tim
al
h
y
p
er
p
ar
am
e
ter
s
in
clu
d
in
g
th
e
n
u
m
b
er
o
f
tr
ain
in
g
e
p
o
ch
s
a
n
d
b
atch
s
iz
e.
T
h
is
s
y
s
tem
atic
ap
p
r
o
ac
h
en
s
u
r
es
f
air
co
m
p
ar
is
o
n
b
etwe
en
p
r
o
p
o
s
ed
alg
o
r
ith
m
an
d
co
n
v
en
tio
n
al
r
eg
u
la
r
izatio
n
m
eth
o
d
s
b
y
s
elec
tin
g
th
e
b
est p
o
s
s
ib
le
co
n
f
ig
u
r
atio
n
f
o
r
ea
ch
ap
p
r
o
ac
h
.
3
.
1
.
Sin
g
le
-
la
y
er
net
wo
rk
T
o
ev
alu
ate
th
e
f
ea
tu
r
e
s
elec
tio
n
ca
p
a
b
ilit
ies
o
f
p
r
o
p
o
s
ed
al
g
o
r
ith
m
we
c
o
n
d
u
cted
e
x
p
er
i
m
en
ts
u
s
in
g
f
in
an
cial
tim
e
s
er
ies
d
ata
f
r
o
m
th
e
W
ils
h
ir
e
2
5
0
0
to
tal
m
ar
k
et
in
d
e
x
.
T
h
is
d
ataset
r
ep
r
esen
ts
a
r
ea
l
-
wo
r
ld
s
ce
n
ar
io
wh
er
e
i
d
en
tify
in
g
t
h
e
m
o
s
t
r
elev
a
n
t
h
is
to
r
ical
v
alu
es
f
o
r
p
r
e
d
ictio
n
is
cr
u
cial
f
o
r
p
r
ac
tical
ap
p
licatio
n
s
.
T
h
e
ex
p
er
im
e
n
tal
s
etu
p
in
v
o
lv
ed
p
r
ed
ictin
g
f
u
tu
r
e
in
d
ex
v
al
u
es
b
ased
o
n
a
4
0
-
d
ay
h
is
to
r
ical
win
d
o
w,
u
s
in
g
d
aily
d
ata
f
r
o
m
J
an
u
ar
y
2
0
2
1
to
Dec
em
b
er
2
0
2
3
(
7
5
4
t
o
tal
o
b
s
er
v
ati
o
n
s
,
with
th
e
last
2
0
0
o
b
s
er
v
atio
n
s
r
eser
v
ed
f
o
r
te
s
tin
g
)
.
W
e
d
elib
er
ately
ch
o
s
e
th
is
f
in
an
cial
d
ataset
b
ec
au
s
e
it
ex
h
ib
its
th
e
co
m
p
lex
tem
p
o
r
al
d
ep
en
d
en
ci
es
an
d
n
o
is
e
ch
ar
ac
ter
is
tics
ty
p
ical
o
f
r
ea
l
-
w
o
r
ld
p
r
ed
ictio
n
p
r
o
b
lem
s
,
m
ak
in
g
it a
n
ap
p
r
o
p
r
iate
test
b
ed
f
o
r
ev
alu
atin
g
f
ea
tu
r
e
s
elec
tio
n
m
et
h
o
d
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
2
0
2
5
:
3
7
8
1
-
3
7
8
9
3786
T
h
e
r
esu
lts
,
in
clu
d
in
g
m
ea
n
s
q
u
ar
ed
e
r
r
o
r
(
MSE
)
an
d
m
ea
n
a
b
s
o
lu
te
er
r
o
r
(
MA
E
)
f
o
r
t
h
e
te
s
t
s
et,
ar
e
p
r
esen
ted
in
T
ab
le
1
.
T
a
b
le
1
also
lis
ts
th
e
r
eg
u
lar
izatio
n
p
ar
am
eter
v
alu
e
,
with
th
e
ar
ctg
u
s
ed
as
th
e
ac
tiv
atio
n
f
u
n
ctio
n
o
f
s
in
g
l
e
-
lay
er
n
eu
r
al
n
etwo
r
k
.
T
h
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
d
em
o
n
s
tr
ated
s
u
p
er
io
r
p
er
f
o
r
m
an
ce
c
o
m
p
a
r
ed
t
o
tr
a
d
itio
n
al
L
1
r
eg
u
lar
izatio
n
ap
p
r
o
ac
h
es.
W
h
ile
co
n
v
e
n
tio
n
a
l
L
1
r
eg
u
lar
izatio
n
(
λ
=0
.
0
1
to
1
.
0
)
f
ailed
to
p
er
f
o
r
m
ac
tu
al
f
ea
tu
r
e
s
elec
tio
n
–
r
e
tain
in
g
all
4
0
in
p
u
t
f
ea
tu
r
es
wh
ile
o
n
ly
r
ed
u
cin
g
co
ef
f
icien
t
m
a
g
n
itu
d
es
–
th
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
s
u
cc
ess
f
u
lly
id
en
tifie
d
ju
s
t
6
f
ea
t
u
r
es
th
at
y
ield
ed
b
etter
p
r
ed
ictio
n
ac
c
u
r
ac
y
.
T
h
e
r
esu
lts
in
T
ab
le
1
r
ev
ea
l
s
ev
er
al
im
p
o
r
tan
t
f
i
n
d
in
g
s
.
First,
s
tan
d
ar
d
Ad
am
o
p
tim
izati
o
n
with
o
u
t
r
eg
u
lar
izatio
n
ac
h
iev
ed
MSE
=4
.
3
9
×1
0
⁻⁵,
wh
ile
th
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
r
ed
u
ce
d
th
is
to
MSE
=2
.
4
8
×1
0
⁻⁶,
r
ep
r
esen
tin
g
an
1
8
-
f
o
l
d
im
p
r
o
v
em
en
t.
Seco
n
d
,
th
e
co
m
p
u
tat
io
n
al
ef
f
icien
cy
was
en
h
an
ce
d
,
r
ed
u
cin
g
tr
ain
in
g
tim
e
f
r
o
m
1
1
.
3
s
ec
o
n
d
s
to
0
.
4
3
s
ec
o
n
d
s
.
T
h
is
im
p
r
o
v
em
e
n
t
s
tem
s
f
r
o
m
th
e
alg
o
r
ith
m
'
s
ab
ilit
y
to
elim
in
ate
ir
r
elev
an
t p
ar
a
m
eter
s
d
u
r
i
n
g
tr
ain
in
g
r
ath
e
r
th
an
m
er
ely
p
en
a
lizin
g
th
em
.
T
ab
le
1
.
R
esu
lts
o
f
co
m
p
u
tatio
n
al
ex
p
e
r
im
en
ts
b
ased
o
n
th
e
W
ils
h
ir
e
2
5
0
0
i
n
d
e
x
M
e
t
h
o
d
N
u
mb
e
r
o
f
se
l
e
c
t
e
d
f
e
a
t
u
r
e
s
(
p
)
M
S
E
M
A
E
Ti
me
(
sec
o
n
d
s
)
A
d
a
m,
=0
40
4
.
3
9
×
10
-
5
0
.
0
0
5
3
1
1
.
3
A
d
a
m,
=
0
.
0
1
40
2
.
0
3
×
10
-
4
0
.
0
1
0
3
1
2
.
8
A
d
a
m,
=
0
.
5
40
0
.
0
0
2
3
0
.
0
1
0
5
1
2
.
8
A
d
a
m,
=
1
.
0
40
0
.
0
0
4
8
0
.
0
1
0
1
7
.
4
5
P
r
o
p
o
se
d
a
l
g
o
r
i
t
h
m,
=
0
.
1
6
2
.
4
8
×
10
-
6
0
.
0
0
1
3
0
.
4
3
T
h
e
ex
p
er
im
e
n
tal
r
esu
lts
d
em
o
n
s
tr
ate
th
e
ef
f
ec
tiv
en
ess
o
f
t
h
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
in
b
o
t
h
p
r
ed
ictio
n
ac
cu
r
ac
y
an
d
co
m
p
u
tatio
n
al
p
er
f
o
r
m
an
ce
.
Fig
u
r
e
3
d
em
o
n
s
tr
ates
th
e
p
r
ac
tical
p
r
ed
icti
o
n
q
u
ality
,
s
h
o
win
g
clo
s
e
alig
n
m
en
t
b
etwe
en
ac
tu
al
an
d
p
r
ed
icted
v
alu
es
o
n
t
h
e
test
s
et.
T
h
e
an
aly
s
is
ac
r
o
s
s
d
if
f
er
en
t
ac
tiv
atio
n
f
u
n
ctio
n
s
Fig
u
r
e
4
co
n
f
ir
m
s
th
e
alg
o
r
ith
m
'
s
r
o
b
u
s
tn
ess
,
co
n
s
is
ten
tly
ac
h
iev
in
g
lo
wer
er
r
o
r
r
ates
th
an
b
aselin
e
m
eth
o
d
ac
r
o
s
s
v
ar
io
u
s
f
u
n
ctio
n
ty
p
es,
in
clu
d
in
g
s
p
ec
ialized
f
in
an
cial
m
o
d
elin
g
f
u
n
ctio
n
s
lik
e
clo
g
lo
g
m
an
d
tan
h
[
2
1
]
.
T
h
e
n
u
m
b
er
o
f
s
el
ec
ted
f
ea
tu
r
es
u
s
in
g
th
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
v
ar
ied
f
r
o
m
2
to
1
0
f
o
r
d
if
f
er
en
t
ac
tiv
atio
n
f
u
n
cti
o
n
s
,
allo
win
g
f
o
r
h
ig
h
er
ac
c
u
r
ac
y
.
Fig
u
r
e
3
.
Actu
al
a
n
d
p
r
ed
icted
in
d
ex
v
alu
es
Fig
u
r
e
4
.
MSE
v
al
u
es b
y
ac
tiv
atio
n
f
u
n
ctio
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
a
lg
o
r
ith
m
fo
r
tr
a
in
in
g
n
eu
r
a
l n
etw
o
r
ks w
ith
L1
r
eg
u
la
r
iz
a
tio
n
…
(
E
ka
teri
n
a
Gri
b
a
n
o
va
)
3787
3
.
2
.
Neura
l net
wo
r
k
wit
h a
s
ing
le
hid
de
n la
y
er
T
h
e
s
ec
o
n
d
p
h
ase
o
f
o
u
r
e
v
alu
atio
n
e
x
am
in
ed
th
e
alg
o
r
ith
m
'
s
p
er
f
o
r
m
an
ce
o
n
m
o
r
e
co
m
p
lex
ar
ch
itectu
r
es
with
a
s
in
g
le
h
id
d
en
lay
er
[
2
2
]
–
[
2
4
]
u
s
in
g
en
ter
p
r
is
e
f
in
an
cial
d
a
ta.
W
e
u
tili
ze
d
a
co
m
p
r
eh
e
n
s
iv
e
d
ataset
o
f
5
5
1
R
u
s
s
ian
en
ter
p
r
is
es
with
r
ev
en
u
es
ex
ce
ed
in
g
1
0
0
m
illi
o
n
r
u
b
les,
s
p
an
n
i
n
g
2017
-
2
0
2
0
[
2
5
]
.
T
h
e
tr
ain
in
g
d
ataset
co
n
tain
s
in
f
o
r
m
atio
n
ab
o
u
t
th
ese
5
5
1
en
ter
p
r
is
es
f
o
r
2
0
1
7
–
2
0
1
9
,
wh
ile
th
e
test
d
atase
t
in
clu
d
es
d
ata
f
o
r
th
e
s
am
e
en
ter
p
r
is
es
f
o
r
2
0
2
0
.
T
h
e
p
r
ed
ictio
n
task
–
f
o
r
ec
asti
n
g
en
ter
p
r
is
e
p
r
o
f
itab
ilit
y
b
ased
o
n
k
ey
f
in
an
cial
in
d
icato
r
s
(
liq
u
id
ity
r
at
io
,
f
ix
ed
ass
et
s
h
ar
e,
f
in
an
cial
lev
er
ag
e,
an
d
ass
et
tu
r
n
o
v
e
r
)
.
T
h
e
e
x
p
er
im
e
n
tal
d
esig
n
co
m
p
ar
ed
n
etwo
r
k
s
with
2
an
d
1
6
h
id
d
en
n
eu
r
o
n
s
to
ex
am
i
n
e
h
o
w
p
r
o
p
o
s
ed
alg
o
r
it
h
m
p
e
r
f
o
r
m
s
ac
r
o
s
s
d
if
f
er
en
t
c
o
m
p
lex
ity
le
v
els.
T
h
e
ch
o
ice
o
f
s
ig
m
o
id
a
ctiv
atio
n
f
o
r
h
id
d
en
lay
er
s
an
d
lin
ea
r
ac
tiv
atio
n
f
o
r
o
u
tp
u
t
f
o
llo
ws
estab
lis
h
ed
p
r
ac
tices
in
f
in
an
cial
m
o
d
el
in
g
,
en
s
u
r
in
g
o
u
r
r
esu
lts
ar
e
co
m
p
ar
ab
le
t
o
s
tan
d
ar
d
ap
p
r
o
ac
h
es in
th
is
d
o
m
ai
n
.
T
h
e
r
esu
lts
in
T
ab
le
2
r
ev
ea
l
ev
id
en
ce
o
f
th
e
alg
o
r
ith
m
'
s
ef
f
ec
tiv
en
ess
.
Fo
r
th
e
2
-
n
eu
r
o
n
n
etwo
r
k
,
p
r
o
p
o
s
ed
alg
o
r
ith
m
r
ed
u
ce
d
p
ar
am
eter
s
f
r
o
m
1
3
to
8
wh
ile
im
p
r
o
v
i
n
g
MSE
f
r
o
m
1
7
4
.
7
9
t
o
1
5
6
.
2
8
an
d
MA
E
f
r
o
m
8
.
5
4
to
8
.
0
6
.
Mo
r
e
s
ig
n
i
f
ican
tly
,
f
o
r
t
h
e
1
6
-
n
eu
r
o
n
n
et
wo
r
k
,
th
e
alg
o
r
ith
m
ac
h
iev
e
d
d
r
am
atic
p
ar
am
eter
r
ed
u
ctio
n
f
r
o
m
9
7
to
1
3
p
a
r
am
eter
s
wh
ile
m
ain
tain
i
n
g
c
o
m
p
ar
ab
le
ac
cu
r
ac
y
(
MSE
=1
5
5
.
0
1
v
s
.
1
5
6
.
0
0
)
,
d
em
o
n
s
tr
atin
g
e
f
f
ec
tiv
e
m
o
d
el
s
im
p
lific
atio
n
with
o
u
t p
er
f
o
r
m
an
ce
d
eg
r
ad
atio
n
.
P
a
r
t
i
c
u
l
a
r
l
y
n
o
t
e
w
o
r
t
h
y
i
s
t
h
e
a
l
g
o
r
i
t
h
m
s
s
u
p
e
r
i
o
r
i
t
y
o
v
e
r
d
r
o
p
o
u
t
r
e
g
u
l
a
r
i
z
a
t
i
o
n
,
a
w
i
d
e
l
y
-
u
s
e
d
t
e
c
h
n
i
q
u
e
f
o
r
p
r
e
v
e
n
t
i
n
g
o
v
e
r
f
i
t
t
i
n
g
.
W
h
i
l
e
d
r
o
p
o
u
t
w
i
t
h
2
0
%
n
e
u
r
o
n
e
l
i
m
i
n
a
t
i
o
n
y
i
e
l
d
e
d
M
S
E
=
1
9
5
.
8
8
,
p
r
o
p
o
s
e
d
a
l
g
o
r
i
t
h
m
a
c
h
i
e
v
e
d
M
S
E
=
1
5
5
.
0
1
w
i
t
h
s
u
b
s
t
a
n
t
i
a
l
p
a
r
a
m
e
t
e
r
r
e
d
u
c
t
i
o
n
,
i
n
d
i
c
a
t
i
n
g
m
o
r
e
e
f
f
e
c
t
i
v
e
r
e
g
u
l
a
r
i
z
a
t
i
o
n
t
h
r
o
u
g
h
p
a
r
a
m
e
t
e
r
s
e
l
e
c
t
i
o
n
r
a
t
h
e
r
t
h
a
n
r
a
n
d
o
m
e
l
i
m
i
n
a
t
i
o
n
.
T
h
e
c
o
m
p
u
t
a
t
i
o
n
a
l
e
f
f
i
c
i
e
n
c
y
g
a
i
n
s
(
t
r
a
i
n
i
n
g
t
i
m
e
r
e
d
u
c
e
d
f
r
o
m
3
2
.
8
t
o
9
.
1
s
e
c
o
n
d
s
)
f
u
r
t
h
e
r
d
e
m
o
n
s
t
r
a
t
e
p
r
a
c
t
i
c
a
l
a
d
v
a
n
t
a
g
e
s
f
o
r
l
a
r
g
e
-
s
c
a
l
e
a
p
p
l
i
c
a
t
i
o
n
s
.
Fig
u
r
e
5
illu
s
tr
ates
th
e
alg
o
r
ith
m
'
s
s
en
s
i
tiv
ity
to
th
e
η
p
ar
a
m
eter
,
s
h
o
win
g
o
p
tim
al
p
e
r
f
o
r
m
an
ce
at
η
=1
0
⁻⁴,
wh
er
e
an
ef
f
ec
tiv
e
b
alan
ce
b
etwe
en
p
ar
am
eter
s
elec
tio
n
an
d
ac
cu
r
ac
y
is
ac
h
iev
ed
.
At
=1
0
-
2
,
p
ar
am
eter
s
elec
tio
n
was
n
o
t
p
er
f
o
r
m
ed
b
ec
a
u
s
e
th
e
o
p
ti
m
ized
f
u
n
ctio
n
y
ield
ed
h
ig
h
er
v
alu
es
wh
en
th
e
ar
g
u
m
en
ts
c
h
an
g
e
d
s
ig
n
if
ica
n
tly
co
m
p
a
r
ed
to
t
h
eir
ze
r
o
v
al
u
es.
Fo
r
=
1
0
-
6
,
t
h
er
e
was
a
s
lo
w
d
ec
r
ea
s
e
in
th
e
o
p
tim
ized
f
u
n
ctio
n
d
u
e
t
o
th
e
u
s
e
o
f
a
s
m
all
s
tep
s
ize.
T
h
e
co
n
s
is
ten
cy
o
f
im
p
r
o
v
e
m
en
ts
ac
r
o
s
s
b
o
th
s
im
p
le
an
d
co
m
p
lex
ar
ch
itectu
r
es v
ali
d
ates th
e
alg
o
r
ith
m
'
s
g
en
er
al
ap
p
licab
ilit
y
an
d
s
u
g
g
ests
its
p
o
ten
tial f
o
r
b
r
o
ad
er
im
p
lem
en
tatio
n
in
n
eu
r
al
n
et
wo
r
k
o
p
tim
izatio
n
task
s
wh
er
e
p
ar
am
eter
ef
f
icien
cy
an
d
m
o
d
el
in
ter
p
r
etab
ilit
y
ar
e
v
alu
ed
al
o
n
g
s
id
e
p
r
ed
ictiv
e
p
er
f
o
r
m
an
ce
.
T
ab
le
2
.
R
esu
lts
o
f
n
eu
r
al
n
et
wo
r
k
m
o
d
elin
g
with
a
h
id
d
e
n
lay
er
M
e
t
h
o
d
N
u
mb
e
r
o
f
n
e
u
r
o
n
s i
n
a
h
i
d
d
e
n
l
a
y
e
r
N
u
mb
e
r
o
f
n
o
n
-
z
e
r
o
p
a
r
a
m
e
t
e
r
s
M
S
E
M
A
E
Ti
me
(
sec
o
n
d
s
)
A
d
a
m,
=0
2
13
1
7
4
.
7
9
8
.
5
4
3
3
.
2
A
d
a
m,
=
0
.
0
1
2
13
1
7
6
.
5
4
8
.
5
4
2
9
.
9
A
d
a
m,
=
0
.
5
2
13
1
8
2
.
1
9
8
.
5
4
2
9
.
2
A
d
a
m,
=
1
.
0
2
13
1
9
6
.
5
3
8
.
8
9
2
8
.
5
P
r
o
p
o
se
d
a
l
g
o
r
i
t
h
m,
=
0
.
0
0
0
1
2
8
1
5
6
.
2
8
8
.
0
6
0
.
6
A
d
a
m,
=0
16
97
1
5
6
8
.
0
8
2
5
.
8
A
d
a
m,
=
0
.
0
1
16
97
1
5
6
.
9
9
8
.
0
9
2
2
.
2
A
d
a
m,
=
0
.
5
16
97
1
7
8
.
7
8
.
3
4
2
2
.
9
A
d
a
m,
=
1
.
0
16
97
1
9
3
.
4
3
8
.
6
2
3
.
7
A
d
a
m,
d
r
o
p
o
u
t
(
0
.
2
)
16
97
1
9
5
.
8
8
8
.
7
3
2
.
8
P
r
o
p
o
se
d
a
l
g
o
r
i
t
h
m,
=
0
.
0
0
0
1
16
13
1
5
5
.
0
1
8
.
0
8
9
.
1
Fig
u
r
e
5
.
C
h
an
g
es in
MSE
an
d
th
e
n
u
m
b
er
o
f
n
o
n
-
ze
r
o
p
ar
a
m
eter
s
with
v
ar
y
in
g
4.
CO
NCLU
SI
O
N
An
alg
o
r
ith
m
h
as
b
ee
n
d
e
v
elo
p
ed
f
o
r
t
r
ain
in
g
a
n
e
u
r
al
n
e
two
r
k
with
L
1
r
eg
u
lar
izatio
n
,
b
ased
o
n
r
ef
o
r
m
u
latin
g
th
e
o
p
tim
izati
o
n
p
r
o
b
lem
o
f
th
e
lo
s
s
f
u
n
ctio
n
as
an
in
v
er
s
e
s
in
g
le
-
p
o
in
t
p
r
o
b
lem
wh
ile
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
2
0
2
5
:
3
7
8
1
-
3
7
8
9
3788
m
in
im
izin
g
th
e
s
u
m
o
f
m
o
d
u
l
es
o
f
ar
g
u
m
en
ts
.
E
x
p
er
i
m
en
ts
wer
e
co
n
d
u
cted
with
two
test
d
atasets
,
lead
in
g
to
th
e
f
o
llo
win
g
co
n
cl
u
s
io
n
s
.
T
h
e
u
s
e
o
f
L
1
r
eg
u
lar
izatio
n
,
im
p
lem
en
ted
in
th
e
Ker
as
lib
r
ar
y
,
d
id
n
o
t
p
er
f
o
r
m
p
ar
am
eter
s
elec
tio
n
i
n
th
e
e
x
am
in
ed
ca
s
es;
h
o
we
v
er
,
it
d
id
r
ed
u
ce
th
e
ab
s
o
lu
te
v
alu
es
o
f
th
e
weig
h
t
co
ef
f
icien
ts
.
T
h
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
s
u
cc
ess
f
u
lly
p
er
f
o
r
m
ed
f
ea
tu
r
e
s
elec
tio
n
f
o
r
th
e
f
ir
s
t
d
ataset.
E
x
p
er
im
en
ts
o
n
th
e
s
ec
o
n
d
d
ataset
d
em
o
n
s
tr
ated
t
h
at
th
e
alg
o
r
ith
m
ef
f
ec
tiv
el
y
ze
r
o
ed
o
u
t
th
e
weig
h
t
co
ef
f
icien
ts
,
r
esu
ltin
g
in
a
r
e
d
u
ctio
n
in
t
h
e
n
u
m
b
e
r
o
f
ad
j
u
s
tab
le
p
ar
am
eter
s
.
T
h
is
,
i
n
t
u
r
n
,
s
im
p
lifie
d
t
h
e
n
etwo
r
k
ar
c
h
itectu
r
e
an
d
im
p
r
o
v
e
d
m
o
d
el
ac
cu
r
ac
y
.
As
a
r
esu
lt,
th
e
s
tu
d
ied
e
x
am
p
l
es
ac
h
iev
ed
f
aster
p
ar
am
eter
tu
n
i
n
g
an
d
lo
wer
M
SE
an
d
MA
E
v
alu
es.
An
o
th
e
r
ad
v
an
tag
e
o
f
th
is
m
eth
o
d
is
th
at
it
elim
in
ates
th
e
n
ee
d
f
o
r
g
e
n
er
atin
g
r
an
d
o
m
w
eig
h
t
v
alu
es,
th
u
s
r
em
o
v
in
g
it
s
s
to
ch
asti
c
n
atu
r
e.
T
h
is
en
s
u
r
es
th
at
th
e
m
eth
o
d
will
y
ield
id
en
tical
r
esu
lts
ac
r
o
s
s
m
u
ltip
le
r
u
n
s
,
p
r
o
v
id
in
g
s
tab
ilit
y
an
d
r
ep
r
o
d
u
ci
b
ilit
y
o
f
th
e
o
b
tain
ed
d
ata.
Ho
wev
er
,
th
e
m
eth
o
d
h
as
s
o
m
e
d
r
awb
ac
k
s
,
in
clu
d
in
g
h
i
g
h
s
en
s
itiv
ity
to
th
e
ch
o
ice
o
f
t
h
e
p
ar
am
eter
.
I
f
a
n
in
co
r
r
ec
t
v
alu
e
is
s
elec
ted
,
all
weig
h
t
co
ef
f
icien
ts
m
ay
b
ec
o
m
e
ze
r
o
,
as
th
eir
ad
ju
s
tm
en
t
wo
u
ld
lead
to
wo
r
s
e
v
alu
es
o
f
th
e
o
p
tim
ized
f
u
n
cti
o
n
.
Fo
r
t
h
is
r
ea
s
o
n
,
th
e
u
s
e
o
f
th
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
is
co
m
p
licated
f
o
r
ce
r
tain
ac
tiv
atio
n
f
u
n
ctio
n
s
,
s
u
c
h
as
s
in
co
s
ac
tiv
atio
n
f
u
n
ctio
n
.
Fu
tu
r
e
r
esear
ch
will
f
o
c
u
s
o
n
d
e
v
elo
p
in
g
an
d
in
v
esti
g
atin
g
a
h
y
b
r
i
d
alg
o
r
i
th
m
th
at
s
y
n
t
h
esizes
th
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
with
ex
is
tin
g
n
eu
r
al
n
etwo
r
k
tr
ain
in
g
m
eth
o
d
s
.
F
UNDING
I
NF
O
R
M
A
T
I
O
N
T
h
is
s
tu
d
y
was
f
u
n
d
ed
b
y
th
e
R
u
s
s
ian
Scien
ce
Fo
u
n
d
atio
n
,
g
r
an
t
n
u
m
b
er
2
5
-
21
-
0
0
1
2
3
“
Ap
p
licatio
n
o
f
m
eth
o
d
s
f
o
r
s
o
l
v
in
g
in
v
er
s
e
ill
-
p
o
s
ed
p
r
o
b
lem
s
in
m
ac
h
in
e
lear
n
in
g
”:
h
ttp
s
://rscf
.
r
u
/
p
r
o
j
ec
t/2
5
-
21
-
0
0
1
2
3
/.
AUTHO
R
CO
NT
RI
B
UT
I
O
NS ST
A
T
E
M
E
N
T
T
h
is
jo
u
r
n
al
u
s
es
th
e
C
o
n
tr
ib
u
to
r
R
o
les
T
ax
o
n
o
m
y
(
C
R
ed
iT)
to
r
ec
o
g
n
ize
in
d
iv
id
u
al
au
th
o
r
co
n
tr
ib
u
tio
n
s
,
r
ed
u
ce
au
th
o
r
s
h
ip
d
is
p
u
tes,
an
d
f
ac
ilit
ate
co
llab
o
r
atio
n
.
Na
m
e
o
f
Aut
ho
r
C
M
So
Va
Fo
I
R
D
O
E
Vi
Su
P
Fu
E
k
ater
in
a
Gr
ib
an
o
v
a
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
R
o
m
an
Ger
asimo
v
✓
✓
✓
✓
✓
✓
✓
C
:
C
o
n
c
e
p
t
u
a
l
i
z
a
t
i
o
n
M
:
M
e
t
h
o
d
o
l
o
g
y
So
:
So
f
t
w
a
r
e
Va
:
Va
l
i
d
a
t
i
o
n
Fo
:
Fo
r
mal
a
n
a
l
y
s
i
s
I
:
I
n
v
e
s
t
i
g
a
t
i
o
n
R
:
R
e
so
u
r
c
e
s
D
:
D
a
t
a
C
u
r
a
t
i
o
n
O
:
W
r
i
t
i
n
g
-
O
r
i
g
i
n
a
l
D
r
a
f
t
E
:
W
r
i
t
i
n
g
-
R
e
v
i
e
w
&
E
d
i
t
i
n
g
Vi
:
Vi
su
a
l
i
z
a
t
i
o
n
Su
:
Su
p
e
r
v
i
s
i
o
n
P
:
P
r
o
j
e
c
t
a
d
mi
n
i
st
r
a
t
i
o
n
Fu
:
Fu
n
d
i
n
g
a
c
q
u
i
si
t
i
o
n
CO
NF
L
I
C
T
O
F
I
N
T
E
R
E
S
T
ST
A
T
E
M
E
NT
T
h
e
au
th
o
r
s
d
ec
la
r
e
th
at
th
er
e
is
n
o
co
n
f
lict
o
f
in
ter
est
i
n
r
elatio
n
to
th
is
p
ap
e
r
,
as
well
as
th
e
p
u
b
lis
h
ed
r
esear
ch
r
esu
lts
,
in
c
lu
d
in
g
th
e
f
in
a
n
cial
asp
ec
ts
o
f
co
n
d
u
ctin
g
th
e
r
esear
ch
,
o
b
ta
in
in
g
an
d
u
s
in
g
its
r
esu
lts
,
as we
ll a
s
an
y
n
o
n
-
f
in
an
cial
p
er
s
o
n
al
r
elatio
n
s
h
ip
s
.
E
T
H
I
CAL AP
P
RO
V
AL
T
h
e
a
u
t
h
o
r
c
o
n
f
i
r
m
s
t
h
a
t
d
i
d
n
o
t
u
s
e
a
r
t
i
f
i
c
i
al
i
n
t
el
l
i
g
e
n
c
e
t
e
ch
n
o
l
o
g
i
e
s
w
h
e
n
c
r
e
a
ti
n
g
t
h
e
c
u
r
r
e
n
t
w
o
r
k
.
DATA AV
AI
L
AB
I
L
I
T
Y
Data
will b
e
m
ad
e
av
ailab
le
o
n
r
ea
s
o
n
ab
le
r
eq
u
est.
RE
F
E
R
E
NC
E
S
[
1
]
G
.
N
u
t
i
,
A
.
-
I
.
C
r
o
ss
,
a
n
d
P
.
R
i
n
d
l
e
r
,
“
Ev
i
d
e
n
c
e
-
b
a
s
e
d
r
e
g
u
l
a
r
i
z
a
t
i
o
n
f
o
r
n
e
u
r
a
l
n
e
t
w
o
r
k
s,
”
M
a
c
h
i
n
e
L
e
a
rn
i
n
g
a
n
d
K
n
o
w
l
e
d
g
e
Ex
t
r
a
c
t
i
o
n
,
v
o
l
.
4
,
n
o
.
4
,
p
p
.
1
0
1
1
–
1
0
2
3
,
2
0
2
2
,
d
o
i
:
1
0
.
3
3
9
0
/
ma
k
e
4
0
4
0
0
5
1
.
[
2
]
H.
-
X
.
D
o
u
,
X
.
-
S
.
L
u
,
C
.
W
a
n
g
,
H
.
-
Z.
S
h
e
n
,
Y
.
-
W
.
Z
h
u
o
,
a
n
d
L.
-
J.
D
e
n
g
,
“
P
a
t
c
h
M
a
s
k
:
a
d
a
t
a
a
u
g
me
n
t
a
t
i
o
n
st
r
a
t
e
g
y
w
i
t
h
g
a
u
s
si
a
n
n
o
i
se
i
n
h
y
p
e
r
s
p
e
c
t
r
a
l
i
ma
g
e
s
,
”
Re
m
o
t
e
S
e
n
si
n
g
,
v
o
l
.
1
4
,
n
o
.
2
4
,
2
0
2
2
,
d
o
i
:
1
0
.
3
3
9
0
/
r
s1
4
2
4
6
3
0
8
.
[
3
]
N
.
S
r
i
v
a
st
a
v
a
,
G
.
H
i
n
t
o
n
,
A
.
K
r
i
z
h
e
v
sk
y
,
I
.
S
u
t
s
k
e
v
e
r
,
a
n
d
R
.
S
a
l
a
k
h
u
t
d
i
n
o
v
,
“
D
r
o
p
o
u
t
:
a
si
mp
l
e
w
a
y
t
o
p
r
e
v
e
n
t
n
e
u
r
a
l
n
e
t
w
o
r
k
s
f
r
o
m o
v
e
r
f
i
t
t
i
n
g
,
”
J
o
u
r
n
a
l
o
f
M
a
c
h
i
n
e
L
e
a
r
n
i
n
g
R
e
se
a
rc
h
,
v
o
l
.
1
5
,
n
o
.
1
,
p
p
.
1
9
2
9
–
1
9
5
8
,
2
0
1
4
.
[
4
]
M
.
V
.
F
e
r
r
o
,
Y
.
D
.
M
o
s
q
u
e
r
a
,
F
.
J
.
R
.
P
e
n
a
,
a
n
d
V
.
M
.
D
.
B
i
l
b
a
o
,
“
Ea
r
l
y
s
t
o
p
p
i
n
g
b
y
c
o
r
r
e
l
a
t
i
n
g
o
n
l
i
n
e
i
n
d
i
c
a
t
o
r
s
i
n
n
e
u
r
a
l
n
e
t
w
o
r
k
s,”
N
e
u
r
a
l
N
e
t
w
o
r
k
s
,
v
o
l
.
1
5
9
,
p
p
.
1
0
9
–
1
2
4
,
2
0
2
3
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
n
e
u
n
e
t
.
2
0
2
2
.
1
1
.
0
3
5
.
[
5
]
R
.
T
i
b
s
h
i
r
a
n
i
,
“
R
e
g
r
e
ssi
o
n
s
h
r
i
n
k
a
g
e
a
n
d
se
l
e
c
t
i
o
n
v
i
a
t
h
e
l
a
ss
o
,
”
J
o
u
r
n
a
l
o
f
t
h
e
Ro
y
a
l
S
t
a
t
i
st
i
c
a
l
S
o
c
i
e
t
y
S
e
r
i
e
s
B:
S
t
a
t
i
st
i
c
a
l
Me
t
h
o
d
o
l
o
g
y
,
v
o
l
.
5
8
,
n
o
.
1
,
p
p
.
2
6
7
–
2
8
8
,
1
9
9
6
,
d
o
i
:
1
0
.
1
1
1
1
/
j
.
2
5
1
7
-
6
1
6
1
.
1
9
9
6
.
t
b
0
2
0
8
0
.
x
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
a
lg
o
r
ith
m
fo
r
tr
a
in
in
g
n
eu
r
a
l n
etw
o
r
ks w
ith
L1
r
eg
u
la
r
iz
a
tio
n
…
(
E
ka
teri
n
a
Gri
b
a
n
o
va
)
3789
[
6
]
Q
.
Li
,
W
.
Q
i
a
o
,
Y
.
S
h
i
,
W
.
B
a
,
F
.
W
a
n
g
,
a
n
d
X
.
H
u
,
“
T
e
m
p
e
r
a
t
u
r
e
mo
d
e
l
i
n
g
o
f
w
a
v
e
r
o
t
o
r
r
e
f
r
i
g
e
r
a
t
i
o
n
p
r
o
c
e
ss
b
a
se
d
o
n
e
l
a
st
i
c
n
e
t
v
a
r
i
a
b
l
e
sel
e
c
t
i
o
n
a
n
d
d
e
e
p
b
e
l
i
e
f
n
e
t
w
o
r
k
,
”
C
h
e
m
o
m
e
t
ri
c
s
a
n
d
I
n
t
e
l
l
i
g
e
n
t
L
a
b
o
ra
t
o
ry
S
y
st
e
m
s
,
v
o
l
.
2
3
9
,
2
0
2
3
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
c
h
e
m
o
l
a
b
.
2
0
2
3
.
1
0
4
8
7
2
.
[
7
]
Z.
W
e
i
,
Q
.
L
i
,
J
.
W
e
i
,
a
n
d
W
.
B
i
a
n
,
“
N
e
u
r
a
l
n
e
t
w
o
r
k
f
o
r
a
c
l
a
ss o
f
sp
a
r
se
o
p
t
i
mi
z
a
t
i
o
n
w
i
t
h
L
0
-
r
e
g
u
l
a
r
i
z
a
t
i
o
n
,
”
N
e
u
ra
l
N
e
t
w
o
r
k
s
,
v
o
l
.
1
5
1
,
p
p
.
2
1
1
–
2
2
1
,
2
0
2
2
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
n
e
u
n
e
t
.
2
0
2
2
.
0
3
.
0
3
3
.
[
8
]
Z.
X
u
,
H
.
Z
h
a
n
g
,
Y
.
W
a
n
g
,
X
.
C
h
a
n
g
,
a
n
d
Y
.
L
i
a
n
g
,
“
L
1
/
2
r
e
g
u
l
a
r
i
z
a
t
i
o
n
,
”
S
c
i
e
n
c
e
C
h
i
n
a
I
n
f
o
rm
a
t
i
o
n
S
c
i
e
n
c
e
s
,
v
o
l
.
5
3
,
n
o
.
6
,
p
p
.
1
1
5
9
–
1
1
6
9
,
2
0
1
0
,
d
o
i
:
1
0
.
1
0
0
7
/
s
1
1
4
3
2
-
0
1
0
-
0
0
9
0
-
0.
[
9
]
Y
.
C
h
e
n
,
J.
G
u
o
,
J.
H
u
a
n
g
,
a
n
d
B
.
L
i
n
,
“
A
n
o
v
e
l
met
h
o
d
f
o
r
f
i
n
a
n
c
i
a
l
d
i
s
t
r
e
ss
p
r
e
d
i
c
t
i
o
n
b
a
se
d
o
n
s
p
a
r
se
n
e
u
r
a
l
n
e
t
w
o
r
k
s
w
i
t
h
L
1
/
2
r
e
g
u
l
a
r
i
z
a
t
i
o
n
,
”
I
n
t
e
r
n
a
t
i
o
n
a
l
J
o
u
rn
a
l
o
f
Ma
c
h
i
n
e
L
e
a
rn
i
n
g
a
n
d
C
y
b
e
rn
e
t
i
c
s
,
v
o
l
.
1
3
,
n
o
.
7
,
p
p
.
2
0
8
9
–
2
1
0
3
,
2
0
2
2
,
d
o
i
:
1
0
.
1
0
0
7
/
s
1
3
0
4
2
-
0
2
2
-
0
1
5
6
6
-
y.
[
1
0
]
M
.
Y
a
n
g
,
M
.
K
.
L
i
m,
Y
.
Q
u
,
X
.
Li
,
a
n
d
D
.
N
i
,
“
D
e
e
p
n
e
u
r
a
l
n
e
t
w
o
r
k
s
w
i
t
h
L1
a
n
d
L
2
r
e
g
u
l
a
r
i
z
a
t
i
o
n
f
o
r
h
i
g
h
d
i
me
n
si
o
n
a
l
c
o
r
p
o
r
a
t
e
c
r
e
d
i
t
r
i
sk
p
r
e
d
i
c
t
i
o
n
,
”
Ex
p
e
rt
S
y
st
e
m
s
w
i
t
h
A
p
p
l
i
c
a
t
i
o
n
s
,
v
o
l
.
2
1
3
,
M
a
r
.
2
0
2
3
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
e
sw
a
.
2
0
2
2
.
1
1
8
8
7
3
.
[
1
1
]
N
.
Eg
w
u
,
T.
M
r
z
i
g
l
o
d
,
a
n
d
A
.
S
c
h
u
p
p
e
r
t
,
“
N
e
u
r
a
l
n
e
t
w
o
r
k
i
n
p
u
t
f
e
a
t
u
r
e
se
l
e
c
t
i
o
n
u
s
i
n
g
s
t
r
u
c
t
u
r
e
d
l
2
−
n
o
r
m p
e
n
a
l
i
z
a
t
i
o
n
,
”
Ap
p
l
i
e
d
I
n
t
e
l
l
i
g
e
n
c
e
,
v
o
l
.
5
3
,
n
o
.
5
,
p
p
.
5
7
3
2
–
5
7
4
9
,
2
0
2
2
,
d
o
i
:
1
0
.
1
0
0
7
/
s
1
0
4
8
9
-
0
2
2
-
0
3
5
3
9
-
8.
[
1
2
]
B
.
H
a
n
e
t
a
l
.
,
“
H
S
R
:
L
1
/
2
-
r
e
g
u
l
a
r
i
z
e
d
sp
a
r
se
r
e
p
r
e
se
n
t
a
t
i
o
n
f
o
r
f
a
s
t
f
a
c
e
r
e
c
o
g
n
i
t
i
o
n
u
si
n
g
h
i
e
r
a
r
c
h
i
c
a
l
f
e
a
t
u
r
e
s
e
l
e
c
t
i
o
n
,
”
N
e
u
r
a
l
C
o
m
p
u
t
i
n
g
a
n
d
Ap
p
l
i
c
a
t
i
o
n
s
,
v
o
l
.
2
7
,
n
o
.
2
,
p
p
.
3
0
5
–
3
2
0
,
2
0
1
6
,
d
o
i
:
1
0
.
1
0
0
7
/
s
0
0
5
2
1
-
0
1
5
-
1
9
0
7
-
y.
[
1
3
]
E.
P
h
a
i
sa
n
g
i
t
t
i
sa
g
u
l
,
“
A
n
a
n
a
l
y
si
s
o
f
t
h
e
r
e
g
u
l
a
r
i
z
a
t
i
o
n
b
e
t
w
e
e
n
L
2
a
n
d
d
r
o
p
o
u
t
i
n
s
i
n
g
l
e
h
i
d
d
e
n
l
a
y
e
r
n
e
u
r
a
l
n
e
t
w
o
r
k
,
”
i
n
2
0
1
6
7
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
C
o
n
f
e
re
n
c
e
o
n
I
n
t
e
l
l
i
g
e
n
t
S
y
st
e
m
s,
M
o
d
e
l
l
i
n
g
a
n
d
S
i
m
u
l
a
t
i
o
n
(
I
S
MS
)
,
2
0
1
6
,
p
p
.
1
7
4
–
179
,
d
o
i
:
1
0
.
1
1
0
9
/
I
S
M
S
.
2
0
1
6
.
1
4
.
[
1
4
]
L.
G
ö
r
l
i
t
z
,
R
.
L
o
o
s
e
n
,
a
n
d
T.
M
r
z
i
g
l
o
d
,
“
To
p
o
l
o
g
y
o
p
t
i
mi
z
a
t
i
o
n
o
f
a
r
t
i
f
i
c
i
a
l
n
e
u
r
a
l
n
e
t
w
o
r
k
s
u
s
i
n
g
L
1
-
p
e
n
a
l
i
z
a
t
i
o
n
,
”
i
n
Pro
c
e
e
d
i
n
g
s
2
0
.
Wo
r
k
sh
o
p
C
o
m
p
u
t
a
t
i
o
n
a
l
I
n
t
e
l
l
i
g
e
n
c
e
,
2
0
1
0
,
p
p
.
8
0
–
8
7
.
[
1
5
]
K
.
S
.
M
o
h
a
me
d
,
“
B
a
t
c
h
g
r
a
d
i
e
n
t
l
e
a
r
n
i
n
g
a
l
g
o
r
i
t
h
m
w
i
t
h
sm
o
o
t
h
i
n
g
L1
r
e
g
u
l
a
r
i
z
a
t
i
o
n
f
o
r
f
e
e
d
f
o
r
w
a
r
d
n
e
u
r
a
l
n
e
t
w
o
r
k
s,
”
C
o
m
p
u
t
e
rs
,
v
o
l
.
1
2
,
n
o
.
1
,
2
0
2
2
,
d
o
i
:
1
0
.
3
3
9
0
/
c
o
m
p
u
t
e
r
s
1
2
0
1
0
0
0
4
.
[
1
6
]
D
.
C
h
e
t
v
e
r
i
k
o
v
,
Z
.
Li
a
o
,
a
n
d
V
.
C
h
e
r
n
o
z
h
u
k
o
v
,
“
O
n
c
r
o
ss
-
v
a
l
i
d
a
t
e
d
l
a
ss
o
i
n
h
i
g
h
d
i
me
n
s
i
o
n
s,
”
T
h
e
A
n
n
a
l
s
o
f
S
t
a
t
i
s
t
i
c
s
,
v
o
l
.
4
9
,
n
o
.
3
,
p
p
.
1
3
0
0
–
1
3
1
7
,
2
0
2
1
,
d
o
i
:
1
0
.
1
2
1
4
/
2
0
-
A
O
S
2
0
0
0
.
[
1
7
]
I
.
A
.
B
a
sh
e
e
r
a
n
d
M
.
H
a
j
me
e
r
,
“
A
r
t
i
f
i
c
i
a
l
n
e
u
r
a
l
n
e
t
w
o
r
k
s
:
f
u
n
d
a
me
n
t
a
l
s,
c
o
mp
u
t
i
n
g
,
d
e
si
g
n
,
a
n
d
a
p
p
l
i
c
a
t
i
o
n
,
”
J
o
u
r
n
a
l
o
f
Mi
c
r
o
b
i
o
l
o
g
i
c
a
l
M
e
t
h
o
d
s
,
v
o
l
.
4
3
,
n
o
.
1
,
p
p
.
3
–
3
1
,
2
0
0
0
,
d
o
i
:
1
0
.
1
0
1
6
/
S
0
1
6
7
-
7
0
1
2
(
0
0
)
0
0
2
0
1
-
3.
[
1
8
]
E.
G
r
i
b
a
n
o
v
a
,
“
A
l
g
o
r
i
t
h
m
f
o
r
s
o
l
v
i
n
g
t
h
e
i
n
v
e
r
s
e
p
r
o
b
l
e
ms
o
f
e
c
o
n
o
mi
c
a
n
a
l
y
si
s
i
n
t
h
e
p
r
e
se
n
c
e
o
f
l
i
mi
t
a
t
i
o
n
s
,
”
EU
R
EK
A:
Ph
y
s
i
c
s
a
n
d
E
n
g
i
n
e
e
ri
n
g
,
v
o
l
.
1
,
n
o
.
1
,
p
p
.
7
0
–
7
8
,
2
0
2
0
,
d
o
i
:
1
0
.
2
1
3
0
3
/
2
4
6
1
-
4
2
6
2
.
2
0
2
0
.
0
0
1
1
0
2
.
[
1
9
]
E.
G
r
i
b
a
n
o
v
a
,
“
E
l
a
b
o
r
a
t
i
o
n
o
f
a
n
a
l
g
o
r
i
t
h
m
f
o
r
s
o
l
v
i
n
g
h
i
e
r
a
r
c
h
i
c
a
l
i
n
v
e
r
se
p
r
o
b
l
e
ms
i
n
a
p
p
l
i
e
d
e
c
o
n
o
m
i
c
s
,
”
Ma
t
h
e
m
a
t
i
c
s
,
v
o
l
.
1
0
,
n
o
.
1
5
,
p
.
2
7
7
9
,
2
0
2
2
,
d
o
i
:
1
0
.
3
3
9
0
/
m
a
t
h
1
0
1
5
2
7
7
9
.
[
2
0
]
F
.
P
e
d
r
e
g
o
s
a
e
t
a
l
.
,
“
S
c
i
k
i
t
-
l
e
a
r
n
:
M
a
c
h
i
n
e
l
e
a
r
n
i
n
g
i
n
p
y
t
h
o
n
,
”
J
o
u
r
n
a
l
o
f
M
a
c
h
i
n
e
L
e
a
r
n
i
n
g
R
e
s
e
a
r
c
h
,
v
o
l
.
1
2
,
p
p
.
2
8
2
5
–
2
8
3
0
,
2
0
1
1
.
[
2
1
]
G
.
S
.
S
.
G
o
mes
,
T.
B
.
Lu
d
e
r
m
i
r
,
a
n
d
L.
M
.
M
.
R
.
Li
m
a
,
“
C
o
m
p
a
r
i
s
o
n
o
f
n
e
w
a
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
s
i
n
n
e
u
r
a
l
n
e
t
w
o
r
k
f
o
r
f
o
r
e
c
a
st
i
n
g
f
i
n
a
n
c
i
a
l
t
i
me
ser
i
e
s
,
”
N
e
u
ra
l
C
o
m
p
u
t
i
n
g
a
n
d
Ap
p
l
i
c
a
t
i
o
n
s
,
v
o
l
.
2
0
,
n
o
.
3
,
p
p
.
4
1
7
–
4
3
9
,
A
p
r
.
2
0
1
1
,
d
o
i
:
1
0
.
1
0
0
7
/
s
0
0
5
2
1
-
0
1
0
-
0
4
0
7
-
3.
[
2
2
]
A
.
X
u
,
H
.
C
h
a
n
g
,
Y
.
X
u
,
R
.
Li
,
X
.
Li
,
a
n
d
Y
.
Zh
a
o
,
“
A
p
p
l
y
i
n
g
a
r
t
i
f
i
c
i
a
l
n
e
u
r
a
l
n
e
t
w
o
r
k
s
(
A
N
N
s)
t
o
s
o
l
v
e
so
l
i
d
w
a
s
t
e
-
r
e
l
a
t
e
d
i
ssu
e
s:
A
c
r
i
t
i
c
a
l
r
e
v
i
e
w
,
”
W
a
st
e
M
a
n
a
g
e
m
e
n
t
,
v
o
l
.
1
2
4
,
p
p
.
3
8
5
–
4
0
2
,
A
p
r
.
2
0
2
1
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
w
a
sma
n
.
2
0
2
1
.
0
2
.
0
2
9
.
[
2
3
]
F
.
L
o
l
l
i
,
R
.
G
a
m
b
e
r
i
n
i
,
A
.
R
e
g
a
t
t
i
e
r
i
,
E.
B
a
l
u
g
a
n
i
,
T
.
G
a
t
o
s,
a
n
d
S
.
G
u
c
c
i
,
“
S
i
n
g
l
e
-
h
i
d
d
e
n
l
a
y
e
r
n
e
u
r
a
l
n
e
t
w
o
r
k
s
f
o
r
f
o
r
e
c
a
s
t
i
n
g
i
n
t
e
r
mi
t
t
e
n
t
d
e
ma
n
d
,
”
I
n
t
e
rn
a
t
i
o
n
a
l
J
o
u
rn
a
l
o
f
Pro
d
u
c
t
i
o
n
E
c
o
n
o
m
i
c
s
,
v
o
l
.
1
8
3
,
p
p
.
1
1
6
–
1
2
8
,
2
0
1
7
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
i
j
p
e
.
2
0
1
6
.
1
0
.
0
2
1
.
[
2
4
]
Y
.
Lu
o
e
t
a
l
.
,
“
A
d
i
ss
o
l
v
e
d
o
x
y
g
e
n
l
e
v
e
l
s
p
r
e
d
i
c
t
i
o
n
m
e
t
h
o
d
b
a
se
d
o
n
s
i
n
g
l
e
-
h
i
d
d
e
n
l
a
y
e
r
f
e
e
d
f
o
r
w
a
r
d
n
e
u
r
a
l
n
e
t
w
o
r
k
u
si
n
g
n
e
i
g
h
b
o
r
h
o
o
d
i
n
f
o
r
mat
i
o
n
me
t
r
i
c
,
”
A
p
p
l
i
e
d
S
o
f
t
C
o
m
p
u
t
i
n
g
,
v
o
l
.
1
6
7
,
D
e
c
.
2
0
2
4
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
a
s
o
c
.
2
0
2
4
.
1
1
2
3
2
8
.
[
2
5
]
D
.
B
.
V
u
k
o
v
i
c
,
L.
S
p
i
t
si
n
a
,
E.
G
r
i
b
a
n
o
v
a
,
V
.
S
p
i
t
si
n
,
a
n
d
I
.
Ly
z
i
n
,
“
P
r
e
d
i
c
t
i
n
g
t
h
e
p
e
r
f
o
r
ma
n
c
e
o
f
r
e
t
a
i
l
mar
k
e
t
f
i
r
ms:
r
e
g
r
e
ss
i
o
n
a
n
d
m
a
c
h
i
n
e
l
e
a
r
n
i
n
g
met
h
o
d
s,”
M
a
t
h
e
m
a
t
i
c
s
,
v
o
l
.
1
1
,
n
o
.
8
,
2
0
2
3
,
d
o
i
:
1
0
.
3
3
9
0
/
m
a
t
h
1
1
0
8
1
9
1
6
.
B
I
O
G
RAP
H
I
E
S O
F
AUTH
O
RS
Ek
a
te
r
in
a
G
r
ib
a
n
o
v
a
h
o
ld
s
a
Do
c
to
r
o
f
Tec
h
n
ica
l
S
c
ien
c
e
s
fro
m
To
m
sk
S
tate
Un
iv
e
rsity
o
f
C
o
n
tr
o
l
S
y
ste
m
s
a
n
d
Ra
d
i
o
El
e
c
tro
n
ics
,
Ru
ss
ian
F
e
d
e
ra
ti
o
n
,
2
0
2
3
.
S
h
e
is
c
u
rre
n
tl
y
a
P
r
o
fe
ss
o
r
a
t
De
p
a
rtme
n
t
of
Au
to
m
a
ted
Co
n
tro
l
S
y
ste
m
s,
To
m
sk
S
tate
Un
i
v
e
rsit
y
o
f
C
o
n
tr
o
l
S
y
ste
m
s
a
n
d
Ra
d
i
o
El
e
c
tro
n
ics
,
Ru
ss
ian
F
e
d
e
ra
ti
o
n
.
He
r
re
se
a
rc
h
i
n
c
lu
d
e
s
m
e
tah
e
u
risti
c
s
,
g
lo
b
a
l
o
p
ti
m
iza
ti
o
n
,
m
a
c
h
in
e
lea
rn
in
g
,
a
n
d
i
n
v
e
rse
p
ro
b
lem
s.
S
h
e
h
a
s
p
u
b
li
sh
e
d
o
v
e
r
1
0
0
p
a
p
e
rs
in
j
o
u
r
n
a
ls
a
n
d
c
o
n
fe
re
n
c
e
s.
S
h
e
c
a
n
b
e
c
o
n
tac
ted
a
t
e
m
a
il
:
e
k
a
terin
a
.
b
.
g
rib
a
n
o
v
a
@tu
su
r.
r
u
.
Ro
m
a
n
G
e
r
a
sim
o
v
re
c
e
iv
e
d
a
b
a
c
h
e
lo
r'
s
d
e
g
re
e
in
Ap
p
li
e
d
Co
m
p
u
ter
S
c
ien
c
e
fro
m
T
o
m
sk
S
tate
Un
i
v
e
rsity
o
f
Co
n
tr
o
l
S
y
ste
m
s
a
n
d
Ra
d
io
e
lec
tr
o
n
ics
(
TUS
UR)
wit
h
a
jo
b
o
n
“
Th
e
m
o
d
u
le
fo
r
p
r
o
c
e
ss
in
g
b
a
n
k
d
o
c
u
m
e
n
ts
o
n
p
ro
tes
ted
tran
s
a
c
ti
o
n
s
i
n
P
JSC
M
TS
Ba
n
k
o
f
To
m
sk
.
”
S
in
c
e
2
0
2
5
,
h
e
h
a
s
b
e
e
n
a
n
E
n
g
i
n
e
e
r
a
t
th
e
Lab
o
ra
to
r
y
o
f
Im
a
g
e
P
ro
c
e
ss
in
g
a
n
d
Artifi
c
ial
In
telli
g
e
n
c
e
(TUS
UR).
His
re
se
a
r
c
h
in
tere
sts
a
re
in
m
o
d
u
lar
n
e
u
ra
l
n
e
two
r
k
s
a
n
d
p
a
tt
e
rn
re
c
o
g
n
it
i
o
n
.
He
c
a
n
b
e
c
o
n
tac
ted
a
t
e
m
a
il
:
ro
m
a
n
.
s.
g
e
ra
sim
o
v
@t
u
su
r.
r
u
.
Evaluation Warning : The document was created with Spire.PDF for Python.