I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
15
,
No
.
2
,
A
p
r
il
2
0
2
6
,
p
p
.
1
2
1
9
~
1
2
3
5
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
2
.
p
p
1
2
1
9
-
1
2
3
5
1219
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
An ef
ficien
t
a
ppr
o
a
ch f
o
r cyber
-
a
t
t
a
ck det
ec
tion by
using
ma
chine learning
a
nd deep l
ea
rning
alg
o
rithms
Ya
s
ir
H
us
s
ei
n Sha
k
ir
1
,
M
a
h
m
o
ud
M
o
ha
m
ed
Abdelh
a
m
ied
2
,
E
s
ha
q
Aziz
Awa
dh
A
L
M
a
nd
ha
ri
3
,
Ali A
lk
ha
zr
a
j
i
4
,
Na
g
la
a
M
.
Reda
5,
6
1
D
e
p
a
r
t
me
n
t
o
f
E
n
g
i
n
e
e
r
i
n
g
,
C
o
l
l
e
g
e
o
f
G
r
a
d
u
a
t
e
S
t
u
d
i
e
s
,
U
n
i
v
e
r
s
i
t
i
Te
n
a
g
a
N
a
s
i
o
n
a
l
,
K
a
j
a
n
g
,
M
a
l
a
y
s
i
a
2
D
e
p
a
r
t
me
n
t
o
f
D
a
t
a
S
c
i
e
n
c
e
a
n
d
A
r
t
i
f
i
c
i
a
l
I
n
t
e
l
l
i
g
e
n
c
e
,
F
a
c
u
l
t
y
o
f
I
n
f
o
r
ma
t
i
o
n
Te
c
h
n
o
l
o
g
y
,
A
l
-
A
h
l
i
y
y
a
A
m
ma
n
U
n
i
v
e
r
s
i
t
y
,
A
mm
a
n
,
J
o
r
d
a
n
3
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
i
n
g
a
n
d
T
e
c
h
n
o
l
o
g
y
,
G
r
a
d
u
a
t
e
S
c
h
o
o
l
o
f
Te
c
h
n
o
l
o
g
y
,
A
s
i
a
P
a
c
i
f
i
c
U
n
i
v
e
r
si
t
y
o
f
Te
c
h
n
o
l
o
g
y
a
n
d
I
n
n
o
v
a
t
i
o
n
,
K
u
a
l
a
L
u
m
p
u
r
,
M
a
l
a
y
si
a
4
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
F
a
c
u
l
t
y
o
f
S
c
i
e
n
c
e
s,
L
e
b
a
n
e
s
e
U
n
i
v
e
r
si
t
y
,
B
e
i
r
u
t
,
Le
b
a
n
o
n
5
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
F
a
c
u
l
t
y
o
f
C
o
m
p
u
t
e
r
s
a
n
d
I
n
f
o
r
mat
i
o
n
T
e
c
h
n
o
l
o
g
y
,
T
h
e
F
u
t
u
r
e
U
n
i
v
e
r
si
t
y
i
n
Eg
y
p
t
,
C
a
i
r
o
,
E
g
y
p
t
6
D
e
p
a
r
t
me
n
t
o
f
M
a
t
h
e
m
a
t
i
c
s
,
F
a
c
u
l
t
y
o
f
S
c
i
e
n
c
e
,
A
i
n
S
h
a
ms U
n
i
v
e
r
si
t
y
,
C
a
i
r
o
,
E
g
y
p
t
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
May
5
,
2
0
2
5
R
ev
is
ed
J
an
5
,
2
0
2
6
Acc
ep
ted
Feb
6
,
2
0
2
6
Th
e
rise
o
f
c
y
b
e
r
-
a
tt
a
c
k
s
n
e
c
e
ss
it
a
tes
in
tru
si
o
n
d
e
tec
ti
o
n
sy
ste
m
s
(
IDS)
th
a
t
p
ro
v
id
e
h
ig
h
d
e
tec
ti
o
n
a
c
c
u
ra
c
y
a
n
d
c
o
m
p
u
tati
o
n
a
l
e
fficie
n
c
y
.
M
o
s
t
e
x
isti
n
g
m
a
c
h
in
e
le
a
rn
in
g
(M
L)
a
n
d
d
e
e
p
lea
rn
i
n
g
(DL)
a
p
p
ro
a
c
h
e
s
a
re
c
o
m
p
lex
,
tak
e
lo
n
g
train
i
n
g
ti
m
e
,
lac
k
tr
a
n
sp
a
re
n
c
y
,
a
n
d
a
re
h
a
rd
t
o
in
t
e
rp
re
t.
To
a
d
d
re
ss
th
e
se
c
h
a
ll
e
n
g
e
s,
t
h
is
re
s
e
a
rc
h
in
tro
d
u
c
e
s
a
n
e
w
m
e
ta
-
h
e
u
risti
c
IDS
o
p
ti
m
iza
ti
o
n
fra
m
e
wo
rk
u
sin
g
th
e
a
rti
ficia
l
b
e
e
c
o
lo
n
y
(ABC) alg
o
r
it
h
m
.
We
d
e
v
e
lo
p
e
d
two
h
y
b
rid
m
o
d
e
ls,
KN
N
+
Be
e
,
wh
ich
c
o
m
b
in
e
s
ABC
to
a
u
to
m
a
te
fe
a
tu
re
se
lec
ti
o
n
a
n
d
k
-
n
e
a
re
st
n
e
ig
h
b
o
rs
(KN
N)
a
lg
o
r
i
th
m
fi
n
e
-
tu
n
i
n
g
,
a
s
we
ll
a
s
g
a
ted
re
c
u
rre
n
t
u
n
it
(G
RU)
+
Be
e
,
wh
e
re
ABC
o
p
ti
m
ize
s
th
e
G
RU
n
e
two
rk
a
rc
h
it
e
c
tu
re
a
n
d
h
y
p
e
rp
a
ra
m
e
ters
.
By
lev
e
ra
g
i
n
g
sw
a
rm
in
telli
g
e
n
c
e
,
o
u
r
m
o
d
e
ls
imp
r
o
v
e
c
las
sifier
p
e
rfo
rm
a
n
c
e
with
o
u
t
c
o
m
p
le
x
a
rc
h
it
e
c
tu
re
.
We
tes
ted
th
e
p
re
se
n
ted
m
o
d
e
ls
o
n
NS
L
-
KD
D,
UN
S
W
-
NB1
5
,
a
n
d
CIC
-
DD
o
S
2
0
1
9
b
e
n
c
h
m
a
rk
d
a
tas
e
ts.
P
e
rfo
rm
a
n
c
e
wa
s e
v
a
lu
a
ted
a
g
a
in
st
b
o
t
h
c
o
n
v
e
n
ti
o
n
a
l
M
L
a
n
d
so
p
h
i
stica
ted
DL
b
a
se
li
n
e
s.
E
x
p
e
rime
n
tal
re
su
lt
s
in
d
ica
ted
t
h
a
t
t
h
e
h
y
b
ri
d
KN
N
+
Be
e
a
n
d
th
e
G
RU
+
B
e
e
m
o
d
e
ls
c
o
n
siste
n
tl
y
su
r
p
a
ss
e
d
th
e
ir
re
sp
e
c
ti
v
e
b
a
se
li
n
e
a
n
d
p
e
rf
o
rm
c
o
m
p
e
ti
ti
v
e
ly
a
g
a
in
st
to
p
m
e
th
o
d
s.
Op
ti
m
ize
d
c
o
n
stru
c
ts
re
g
istere
d
h
ig
h
a
c
c
u
ra
c
y
,
F
1
-
sc
o
re
,
a
n
d
M
a
tt
h
e
ws
c
o
rre
latio
n
c
o
e
fficie
n
t
(M
CC)
,
b
e
sid
e
s
re
tain
in
g
g
re
a
t
g
e
n
e
ra
li
z
a
b
il
it
y
a
c
ro
ss
v
a
ri
o
u
s
a
tt
a
c
k
sc
e
n
a
rio
s.
Ou
r
p
ro
p
o
sa
l
o
ffe
rs
a
re
a
so
n
a
b
le
c
o
m
p
ro
m
ise
b
e
twe
e
n
d
e
tec
ti
o
n
p
re
c
isio
n
a
n
d
th
rif
ti
n
e
s
s,
m
a
k
in
g
it
a
p
p
r
o
p
riate
c
h
o
ice
fo
r
sc
a
lab
le,
re
a
l
-
ti
m
e
c
y
b
e
r
d
e
fe
n
se
sy
ste
m
s.
K
ey
w
o
r
d
s
:
Attack
d
etec
tio
n
B
ee
alg
o
r
ith
m
C
y
b
er
s
ec
u
r
ity
Dee
p
lear
n
in
g
Gate
d
r
ec
u
r
r
e
n
t u
n
it
K
-
n
ea
r
est n
eig
h
b
o
r
alg
o
r
ith
m
Ma
ch
in
e
lear
n
in
g
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Nag
laa
M.
R
ed
a
Dep
ar
tm
en
t o
f
C
o
m
p
u
ter
Scie
n
ce
,
Facu
lty
o
f
C
o
m
p
u
ter
s
an
d
I
n
f
o
r
m
atio
n
T
ec
h
n
o
lo
g
y
T
h
e
Fu
tu
r
e
Un
iv
er
s
ity
in
E
g
y
p
t
C
air
o
,
E
g
y
p
t
E
m
ail:
n
ag
laa.
s
ae
ed
@
f
u
e.
ed
u
.
eg
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
ex
p
o
n
e
n
tial
g
r
o
wth
o
f
th
e
d
ig
ital
u
n
iv
er
s
e
h
as
o
p
en
e
d
u
n
p
ar
alleled
d
o
o
r
s
to
co
m
m
u
n
icatio
n
,
e
-
co
m
m
er
ce
,
an
d
d
ata
-
b
ased
s
er
v
ices.
Ho
wev
er
,
it
h
as
b
r
o
u
g
h
t
in
cr
itical
v
u
ln
er
ab
i
liti
es
in
in
tr
u
s
io
n
d
etec
tio
n
,
ev
asio
n
o
f
p
h
is
h
in
g
attac
k
s
,
an
d
ev
asio
n
o
f
d
en
ia
l
-
of
-
s
er
v
ice
attac
k
s
.
As
th
e
cy
b
er
th
r
ea
ts
b
ec
o
m
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
26
:
1
2
1
9
-
1
2
3
5
1220
h
ig
h
ly
s
o
p
h
is
ticated
an
d
d
y
n
am
ic
in
n
atu
r
e,
lim
itatio
n
s
o
f
tr
a
d
itio
n
al
r
u
le
-
b
ased
an
d
o
n
e
-
d
ataset
-
b
ased
ap
p
r
o
ac
h
es a
r
e
b
ec
o
m
in
g
a
p
p
ar
en
t.
T
h
e
n
ewe
r
tech
n
o
lo
g
ies
s
u
c
h
as
ar
tific
ial
in
tellig
en
ce
(
AI
)
,
m
ac
h
in
e
lear
n
in
g
(
ML
)
,
an
d
d
ee
p
lear
n
in
g
(
DL
)
h
a
v
e
b
ee
n
h
ig
h
ly
u
tili
ze
d
to
tack
le
th
ese
p
r
o
b
lem
s
,
b
u
t
s
ca
lab
ilit
y
p
r
o
b
l
em
s
,
ac
r
o
s
s
-
d
ataset
g
en
er
aliza
tio
n
an
d
o
p
tim
izatio
n
ef
f
icac
y
ar
e
lef
t
u
n
ch
ec
k
ed
[
1
]
.
T
h
is
h
as
ca
u
s
ed
th
e
ev
o
lu
tio
n
o
f
g
r
av
e
s
ec
u
r
ity
p
r
o
b
lem
s
to
co
v
er
d
ata
f
r
o
m
th
e
v
ast
ch
allen
g
es
p
o
s
ed
b
y
cy
b
er
s
ec
u
r
ity
co
n
s
tr
u
cto
r
s
.
T
h
e
f
o
u
n
d
atio
n
o
f
en
s
u
r
i
n
g
d
ata
s
ec
u
r
ity
an
d
s
ec
r
ec
y
to
e
v
er
y
f
o
r
m
o
f
b
u
s
in
ess
es,
g
o
v
er
n
m
e
n
ts
an
d
ev
en
p
r
iv
ate
i
n
d
iv
id
u
als
is
cy
b
er
s
ec
u
r
ity
.
Data
is
s
en
t
an
d
r
ec
eiv
e
d
o
v
er
th
e
i
n
ter
n
et
in
an
o
p
en
e
n
v
ir
o
n
m
en
t
w
h
er
e
it
is
m
an
ip
u
lated
an
d
h
ijack
ed
b
y
u
n
au
th
o
r
ized
p
ar
ties
.
Den
ial
o
f
s
er
v
ice
(
Do
S),
r
o
o
t
to
lo
ca
l
attac
k
(
R
2
L
)
,
u
s
er
to
r
o
o
t
attac
k
(
U2
R
)
,
an
d
p
r
o
b
in
g
(
Pro
b
e)
a
r
e
am
o
n
g
s
t
th
e
v
er
y
f
r
e
q
u
en
t
cy
b
er
-
attac
k
s
[
2
]
.
Do
S
attac
k
s
p
r
ev
en
t
leg
al
u
s
er
u
tili
za
tio
n
o
f
s
y
s
tem
r
eso
u
r
ce
s
.
I
t is a
k
iller
to
o
l th
at
b
r
in
g
s
d
o
wn
s
er
v
er
s
an
d
b
r
in
g
s
to
a
s
t
an
d
s
to
p
s
er
v
ice
b
y
s
en
d
in
g
a
n
a
b
u
n
d
an
t
am
o
u
n
t
o
f
p
ac
k
ets
an
d
ca
u
s
in
g
a
s
er
v
er
o
r
n
etwo
r
k
t
o
co
m
e
to
s
tan
d
s
till
.
Fu
r
th
er
,
Pr
o
b
e
attac
k
s
ar
e
d
esig
n
ed
t
o
r
ev
e
al
in
f
o
r
m
atio
n
r
eg
ar
d
in
g
a
t
ar
g
et
s
y
s
tem
v
u
ln
e
r
ab
ilit
y
b
y
th
e
en
ac
tm
e
n
t
o
f
ca
r
ef
u
lly
s
elec
ted
s
eq
u
en
ce
s
o
f
ac
tio
n
s
an
d
o
b
s
er
v
atio
n
o
f
th
e
s
y
s
tem
o
r
th
e
in
tr
u
s
io
n
d
etec
tio
n
s
y
s
tem
(
I
DS)
r
ea
ctio
n
s
[
3
]
.
A
P
r
o
b
e
is
cr
af
ted
to
b
e
d
etec
ted
b
y
th
eir
tar
g
et
an
d
r
ep
o
r
tab
le
with
a
s
ig
n
atu
r
e
-
s
p
e
cif
ic
"f
in
g
er
p
r
in
t"
in
th
e
r
ep
o
r
t.
No
n
eth
eless
,
U2
R
attac
k
s
b
e
g
in
to
b
e
s
u
cc
ess
f
u
l
in
estab
lis
h
in
g
a
u
s
er
s
ess
io
n
p
r
e
f
er
en
ti
ally
b
y
an
in
ter
ac
tiv
e
s
h
ell
o
r
b
y
o
p
e
n
in
g
u
p
a
T
E
L
NE
T
win
d
o
w
o
n
t
h
e
r
em
o
te
h
o
s
t
[
4
]
.
T
h
r
o
u
g
h
t
h
e
em
p
lo
y
m
en
t
o
f
a
co
m
b
in
atio
n
o
f
co
n
v
en
tio
n
al
m
eth
o
d
s
,
th
e
i
n
v
ad
e
r
attem
p
ts
to
escalate
h
is
p
r
iv
ileg
es
in
s
t
ep
s
u
n
til
h
e
r
ea
ch
es
p
r
iv
ileg
es
o
f
th
e
s
u
p
er
-
u
s
er
.
An
attac
k
er
in
an
R
2
L
attac
k
ca
r
r
ies
o
u
t
a
r
em
o
te
-
to
-
lo
ca
l
attac
k
b
y
s
en
d
in
g
p
ac
k
ets
to
th
e
p
o
ten
ti
al
h
o
s
t
with
a
g
o
al
o
f
u
n
v
eilin
g
s
ec
u
r
ity
v
u
ln
er
ab
ilit
ies
th
r
o
u
g
h
wh
ich
th
e
in
v
ad
er
m
ay
ex
p
lo
it
a
lo
ca
l
u
s
er
'
s
p
r
iv
ileg
es.
Ad
m
itted
ly
,
p
en
etr
atio
n
as
a
n
au
th
o
r
ize
d
u
s
er
m
ay
b
e
a
cr
u
cial
p
r
ep
ar
atio
n
in
an
ticip
atio
n
o
f
ev
en
tu
all
y
ex
e
cu
tin
g
a
u
s
er
-
to
-
r
o
o
t
att
ac
k
.
Acc
o
r
d
in
g
t
o
Dav
is
et
a
l.
[
5
]
,
an
in
tellig
en
t
ML
m
o
d
el
is
o
f
f
er
ed
to
en
h
an
ce
t
h
e
d
etec
tio
n
o
f
c
y
b
er
attac
k
s
,
v
a
lid
ated
o
n
m
u
ltip
le
d
atasets
.
I
t
s
m
ain
aim
was
to
s
ec
u
r
e
d
ata
ex
ch
an
g
e
b
etwe
en
u
s
er
s
o
v
er
th
e
i
n
ter
n
et.
T
h
e
tr
ip
le
-
lay
er
m
o
d
el,
w
h
ich
c
o
m
b
in
es
s
ig
n
atu
r
e,
n
etwo
r
k
tr
af
f
ic,
a
n
d
m
ac
h
in
e
-
lear
n
in
g
-
e
n
h
an
ce
d
b
eh
a
v
io
r
a
l
f
ea
tu
r
es,
o
f
f
er
s
en
co
u
r
a
g
in
g
im
p
r
o
v
e
m
en
ts
in
d
etec
tio
n
ac
cu
r
ac
y
a
n
d
ea
r
l
y
th
r
ea
t
id
en
tific
atio
n
,
e
v
en
th
o
u
g
h
r
an
s
o
m
war
e
is
s
till
a
q
u
ick
ly
d
ev
elo
p
in
g
an
d
ex
tr
em
ely
lu
c
r
ativ
e
cy
b
er
th
r
e
at
[
6
]
.
O
n
th
e
o
th
er
h
an
d
,
to
d
etec
t
f
r
au
d
u
len
t
b
eh
av
io
r
,
an
o
p
tim
ized
e
x
tr
em
e
lear
n
in
g
m
ac
h
in
e
in
te
g
r
ated
with
s
y
n
th
etic
m
in
o
r
ity
o
v
e
r
-
s
am
p
lin
g
tech
n
iq
u
e
(
SMOT
E
)
was
s
u
g
g
ested
in
o
r
d
er
to
im
p
r
o
v
e
d
etec
tio
n
r
o
b
u
s
tn
ess
an
d
ev
alu
atio
n
r
eliab
ilit
y
,
esp
ec
ially
wh
en
th
er
e
i
s
a
s
ig
n
if
ican
t
d
ata
im
b
alan
ce
in
d
i
g
ital p
ay
m
en
t s
y
s
tem
s
[
7
]
.
Acc
o
r
d
in
g
to
r
ec
e
n
t
r
esear
ch
,
AI
–
s
o
f
twar
e
d
e
f
in
ed
n
etwo
r
k
(
SDN)
f
r
am
ew
o
r
k
s
th
at
d
y
n
am
ically
an
aly
ze
in
ter
n
et
co
n
tr
o
l
m
ess
ag
e
p
r
o
to
co
l
(
I
C
MP)
tr
af
f
ic
an
d
en
f
o
r
ce
a
u
to
m
ated
m
itig
atio
n
r
u
les
g
r
e
atly
in
cr
ea
s
e
th
e
ac
cu
r
ac
y
o
f
Sm
u
r
f
attac
k
d
etec
tio
n
a
n
d
d
ec
r
ea
s
e
r
esp
o
n
s
e
tim
es.
T
h
e
n
ee
d
f
o
r
m
o
r
e
r
o
b
u
s
t
a
n
d
s
elf
-
lear
n
in
g
ar
c
h
itectu
r
es
is
h
ig
h
lig
h
ted
b
y
th
e
f
ac
t
th
at
cu
r
r
en
t
s
o
lu
tio
n
s
s
till
h
av
e
s
h
o
r
tc
o
m
in
g
s
in
ter
m
s
o
f
s
ca
lab
ilit
y
,
r
ea
l
-
tim
e
m
o
d
el
ad
ap
tab
ilit
y
,
an
d
r
o
b
u
s
tn
e
s
s
ag
ain
s
t
ch
an
g
in
g
s
p
o
o
f
i
n
g
tech
n
iq
u
es
[
8
]
.
C
o
m
p
ar
ativ
e
an
aly
s
es
o
f
th
e
p
er
f
o
r
m
a
n
ce
o
f
class
if
ier
s
o
n
s
tan
d
ar
d
b
en
c
h
m
ar
k
s
s
u
ch
as
NSL
-
KD
D
ar
e
av
ailab
le.
T
h
e
class
if
ier
s
in
v
e
s
tig
ated
in
clu
d
e
lo
g
is
tic
r
eg
r
e
s
s
io
n
(
L
R
)
,
s
u
p
p
o
r
t
v
ec
to
r
m
ac
h
in
e
(
SVM)
,
an
d
ar
ti
f
icial
n
eu
r
al
n
etwo
r
k
(
AN
N
)
class
if
ier
s
[
9
]
.
Hy
b
r
id
m
o
d
els
b
ased
o
n
DL
tech
n
iq
u
es
h
elp
in
th
e
d
etec
tio
n
o
f
attac
k
s
in
v
a
r
ied
d
atasets
[
1
0
]
.
Hy
b
r
id
m
o
d
els
f
o
r
f
ea
tu
r
e
s
elec
tio
n
ca
n
th
er
ef
o
r
e
im
p
r
o
v
e
th
e
ef
f
icien
c
y
o
f
th
e
class
if
ier
s
an
d
r
ed
u
ce
co
s
ts
ac
co
r
d
in
g
ly
[
1
1
]
.
R
ec
en
t
wo
r
k
s
o
n
s
u
p
er
v
is
ed
ML
f
o
r
d
i
s
tr
ib
u
ted
d
en
ial
o
f
s
er
v
ice
(
DDo
S)
d
etec
tio
n
[
1
2
]
,
as
well
as
th
e
u
s
e
o
f
th
e
co
n
v
o
lu
tio
n
al
n
eu
r
al
n
etw
o
r
k
-
lo
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
(
C
NN
-
L
STM
)
h
y
b
r
i
d
ap
p
r
o
ac
h
[
1
3
]
,
s
h
o
w
th
eir
ef
f
ec
tiv
en
ess
in
attac
k
class
if
icatio
n
.
Hiar
i
et
a
l.
[
1
4
]
p
r
o
p
o
s
e
an
L
STM
-
b
ased
m
o
d
el
f
o
r
ac
c
u
r
ate
Do
S
attac
k
d
etec
tio
n
u
s
in
g
th
e
NSL
-
KDD
d
ataset.
Stan
d
ar
d
b
en
ch
m
ar
k
s
f
o
r
class
if
icatio
n
in
clu
d
e
th
e
NSL
-
KDD
d
ataset
[
1
5
]
an
d
th
e
C
I
C
-
DDo
S
d
ataset
[
1
6
]
.
Par
am
eter
o
p
tim
izatio
n
tech
n
i
q
u
es,
s
u
c
h
as
th
e
o
p
tim
izatio
n
o
f
k
-
n
ea
r
est
n
eig
h
b
o
r
s
(
KNN)
p
a
r
am
eter
s
[
1
7
]
,
ca
n
th
er
ef
o
r
e
im
p
r
o
v
e
th
e
d
etec
tio
n
p
r
o
ce
s
s
.
Ho
wev
er
,
s
tr
ik
in
g
a
b
alan
ce
b
etwe
e
n
ac
cu
r
ac
y
,
tr
ain
in
g
s
p
ee
d
,
an
d
in
ter
p
r
etab
ilit
y
is
s
till
an
im
p
ed
im
en
t in
r
e
f
er
en
ce
[
1
8
]
.
T
h
is
p
ap
er
p
r
esen
ts
an
ef
f
ic
ien
t
s
o
lu
tio
n
to
o
v
er
c
o
m
e
th
e
lim
itatio
n
o
f
th
e
KNN
cla
s
s
if
ier
b
y
in
tr
o
d
u
cin
g
g
e
n
er
al
h
y
b
r
id
m
o
d
el
(
KNN
+
B
ee
)
to
in
v
o
l
v
e
t
h
e
B
ee
o
p
tim
izer
to
f
i
n
e
-
tu
n
e
all
h
y
p
er
p
ar
am
eter
s
o
f
th
e
KNN
class
if
ier
with
o
u
t
a
r
ed
u
ctio
n
i
n
in
p
u
t
d
im
en
s
io
n
.
T
h
e
u
s
ag
e
o
f
B
ee
is
n
eith
er
to
ch
o
o
s
e
f
ea
tu
r
es,
b
u
t
as
an
ef
f
icien
t
o
p
tim
izer
i
n
s
tr
u
m
e
n
t
to
b
e
u
s
ed
o
v
e
r
th
e
h
y
p
er
p
a
r
am
eter
s
o
f
th
e
KNN
class
if
ier
it
s
elf
.
An
in
teg
r
ated
u
s
ag
e
o
f
B
ee
with
KNN
lear
n
in
g
m
o
d
el
is
ab
le
t
o
d
eliv
er
th
e
b
est
p
o
s
s
ib
le
co
n
f
ig
u
r
atio
n
t
o
ce
r
tain
k
ey
p
ar
am
eter
s
s
u
ch
as
lo
w
s
a
m
p
le
s
p
lit,
n
u
m
b
er
o
f
esti
m
ato
r
s
o
r
n
u
m
b
e
r
n
e
ig
h
b
o
u
r
s
an
d
m
etr
ic.
T
h
e
f
o
r
m
er
is
aim
ed
at
ac
h
iev
in
g
m
ax
im
u
m
p
r
ec
is
io
n
in
class
if
icatio
n
a
n
d
latter
at
m
ain
tain
in
g
lo
w
f
a
ls
e
p
o
s
itiv
es
(
FP
)
.
T
h
e
co
n
tr
ib
u
tio
n
s
ar
e
th
u
s
co
n
clu
d
ed
to
b
e
as
f
o
llo
ws.
First,
a
n
o
v
el
h
y
b
r
id
I
DS
is
p
r
o
p
o
s
ed
,
wh
er
e
th
e
ar
tific
ial
b
ee
co
lo
n
y
(
A
B
C
)
alg
o
r
ith
m
au
to
m
atica
lly
o
p
tim
izes
th
e
h
y
p
er
p
ar
am
et
er
s
o
f
b
o
th
KNN
class
if
ier
en
h
an
cin
g
r
o
b
u
s
tn
ess
.
Seco
n
d
,
a
m
eta
-
h
eu
r
is
ticall
y
en
h
an
ce
d
DL
m
o
d
el
is
d
ev
el
o
p
ed
b
y
in
teg
r
atin
g
AB
C
o
p
tim
izatio
n
with
a
g
at
ed
r
ec
u
r
r
en
t
u
n
it
(
GR
U)
n
etw
o
r
k
,
e
f
f
ec
tiv
ely
ca
p
tu
r
in
g
tem
p
o
r
al
d
e
p
en
d
e
n
cies
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
efficien
t a
p
p
r
o
a
ch
fo
r
cy
b
e
r
-
a
tta
ck
d
etec
tio
n
b
y
u
s
in
g
ma
ch
in
e
lea
r
n
in
g
a
n
d
…
(
Ya
s
ir
Hu
s
s
ein
S
h
a
kir
)
1221
in
n
etwo
r
k
d
ata
th
at
r
ed
u
ce
s
m
o
d
el
co
m
p
le
x
ity
an
d
tr
ain
in
g
co
s
ts
.
T
h
ir
d
,
th
e
p
r
o
p
o
s
al
is
co
m
p
r
eh
en
s
iv
el
y
v
alid
ated
o
n
th
r
ee
h
eter
o
g
e
n
eo
u
s
(
NSL
-
KDD,
UNSW
-
NB
1
5
,
an
d
C
I
C
-
DDo
S2
0
1
9
)
,
co
n
f
ir
m
in
g
th
e
f
r
am
ewo
r
k
'
s
r
eliab
ilit
y
a
n
d
a
d
ap
tab
ilit
y
a
g
ain
s
t
a
wid
e
s
p
e
ctr
u
m
o
f
i
n
tr
u
s
io
n
s
a
n
d
DDo
S
attac
k
s
.
Fo
u
r
th
,
a
m
u
ltid
im
en
s
io
n
al
ev
alu
atio
n
is
em
p
lo
y
ed
u
s
in
g
a
s
u
ite
o
f
m
etr
ics
s
u
ch
as
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all
,
F1
-
s
co
r
e,
Ma
tth
ews
co
r
r
elati
o
n
co
ef
f
icien
t
(
MCC
)
,
tr
ain
i
n
g
an
d
test
in
g
tim
e
t
o
en
s
u
r
e
a
b
alan
ce
d
v
iew
o
f
p
r
ed
ictiv
e
p
o
wer
,
r
o
b
u
s
tn
ess
an
d
co
m
p
u
tatio
n
al
ef
f
icien
c
y
.
Fin
ally
,
th
e
lo
ca
l
in
ter
p
r
etab
le
m
o
d
el
-
ag
n
o
s
tic
ex
p
lan
atio
n
s
(
LIME
)
f
r
am
ew
o
r
k
is
in
teg
r
ated
to
p
r
o
v
id
e
tr
an
s
p
ar
en
t
h
u
m
a
n
u
n
d
er
s
tan
d
a
b
le
ex
p
lan
atio
n
s
f
o
r
m
o
d
el
p
r
e
d
ictio
n
s
f
o
s
ter
in
g
tr
u
s
t a
n
d
f
ac
ilit
atin
g
p
r
ac
tical
d
ep
lo
y
m
en
t i
n
s
ec
u
r
ity
o
p
er
atio
n
s
.
I
n
th
e
p
r
esen
t
p
a
p
er
,
th
e
c
o
n
t
en
t
is
s
p
litt
ed
in
to
s
ix
s
ec
t
io
n
s
.
Sectio
n
2
will
d
is
cu
s
s
p
r
io
r
s
tu
d
ies
in
p
r
ev
io
u
s
wo
r
k
.
S
ec
tio
n
3
g
iv
es
a
b
r
ie
f
tech
n
ical
b
ac
k
g
r
o
u
n
d
o
f
t
h
e
m
ate
r
ials
an
d
m
et
h
o
d
s
in
cl
u
d
in
g
th
e
p
r
o
p
o
s
ed
alg
o
r
ith
m
is
p
r
esen
ted
o
f
s
ev
er
al
d
atasets
.
T
h
e
ex
p
lan
atio
n
ab
o
u
t
th
e
e
x
p
er
i
m
en
tal
r
esu
lts
an
d
d
is
cu
s
s
es th
e
cr
itical
an
aly
ze
is
s
h
o
wn
in
s
ec
tio
n
4
. T
h
e
co
n
clu
s
io
n
o
f
th
e
wo
r
k
c
o
m
es in
s
ec
tio
n
5
.
2.
P
RE
VIOU
S WO
RK
S
Ap
p
licatio
n
o
f
ML
an
d
DL
al
g
o
r
ith
m
s
in
th
e
d
etec
tio
n
o
f
c
y
b
er
-
attac
k
s
g
en
er
ated
s
ig
n
if
i
ca
n
t
in
ter
est
in
n
etwo
r
k
s
ec
u
r
ity
a
n
d
in
te
r
n
et
o
f
th
in
g
s
(
I
o
T
)
in
s
tallatio
n
s
.
T
h
e
liter
atu
r
e
d
em
o
n
s
tr
ates
a
co
n
s
is
ten
t
ev
o
lu
tio
n
f
r
o
m
tr
ad
itio
n
al
ML
m
o
d
els
to
war
d
s
s
o
p
h
i
s
ticated
DL
an
d
h
y
b
r
id
ar
c
h
i
tectu
r
es
to
im
p
r
o
v
e
d
etec
tio
n
ac
cu
r
ac
y
r
o
b
u
s
tn
ess
an
d
ef
f
icien
cy
.
T
h
is
s
ec
tio
n
r
ev
iews
co
n
tr
ib
u
tio
n
s
r
elev
an
t
to
n
etwo
r
k
-
b
ased
in
tr
u
s
io
n
d
etec
tio
n
.
Usi
n
g
th
e
NSL
-
KDD
d
ataset,
R
ed
d
y
et
a
l.
[
9
]
ass
ess
ed
ML
m
o
d
el
s
f
o
r
in
tr
u
s
io
n
d
etec
tio
n
.
T
h
eir
f
i
n
d
in
g
s
s
h
o
wed
th
at
a
p
r
o
p
er
ly
ca
lib
r
ated
f
ee
d
f
o
r
war
d
ANN
p
er
f
o
r
m
e
d
ex
ce
p
tio
n
ally
well,
ac
h
iev
in
g
n
ea
r
ly
f
lawless
s
co
r
es
o
f
9
9
.
7
9
%
F1
-
s
co
r
es
,
1
0
0
%
r
ec
all,
9
9
.
5
8
%
ac
cu
r
ac
y
,
an
d
9
9
.
5
8
%
p
r
ec
is
io
n
.
Simp
ler
m
o
d
els
lik
e
SVM
an
d
L
R
ar
e
s
ti
ll
f
ea
s
ib
le
an
d
u
s
ef
u
l
o
p
tio
n
s
f
o
r
lo
w
-
r
eso
u
r
ce
co
n
tex
ts
,
th
e
p
ap
er
co
n
ten
d
s
,
e
v
en
th
o
u
g
h
th
e
AN
N
p
er
f
o
r
m
ed
b
est.
B
o
ad
i
[
1
9
]
u
s
ed
eig
h
t
ML
m
o
d
els
f
o
r
in
tr
u
s
io
n
d
etec
tio
n
o
f
NSL
-
KDD
d
ataset.
W
ith
th
e
h
ig
h
est
ac
cu
r
ac
y
o
f
8
8
.
3
0
,
r
ec
a
ll
8
2
.
3
0
%,
F1
-
s
co
r
e
o
f
8
8
.
9
0
,
an
d
ar
ea
u
n
d
er
th
e
cu
r
v
e
(
AUC
)
o
f
9
7
.
7
0
%,
r
a
n
d
o
m
f
o
r
est
was
th
e
b
est
p
er
f
o
r
m
er
.
T
h
e
s
tu
d
y
co
n
cl
u
d
es
th
at
en
s
em
b
le
m
eth
o
d
s
lik
e
r
an
d
o
m
f
o
r
est
ar
e
b
est
f
o
r
d
etec
tin
g
co
m
p
lex
p
atter
n
s
wh
ile
s
im
p
ler
m
o
d
els
lik
e
LR
an
d
K
NN
r
em
ain
v
alu
ab
le
f
o
r
in
ter
p
r
etab
ilit
y
an
d
ef
f
icien
cy
.
T
o
ad
d
r
e
s
s
s
o
p
h
i
s
ti
ca
ted
cy
b
e
r
t
h
r
e
at
s
,
K
u
m
ar
e
t
a
l
.
[
2
0
]
d
e
v
elo
p
ed
a
n
o
v
e
l
I
D
S
th
at
in
t
e
g
r
at
es
th
e
f
ea
tu
r
e
le
ar
n
in
g
ca
p
ab
i
l
it
ie
s
o
f
a
C
N
N
w
ith
th
e
en
s
em
b
le
c
la
s
s
if
ic
at
io
n
p
o
wer
o
f
r
an
d
o
m
f
o
r
e
s
t
.
E
v
a
lu
at
ed
o
n
th
e
N
SL
-
KD
D
d
at
a
s
e
t
t
h
e
h
y
b
r
id
m
o
d
e
l
d
em
o
n
s
tr
a
t
ed
s
u
p
e
r
io
r
p
e
r
f
o
r
m
an
c
e,
th
e
b
e
s
t
ac
cu
r
a
cy
o
f
9
8
.
7
3
%
an
d
an
F1
-
s
co
r
e
o
f
9
6
.
5
0
%
.
T
h
e
au
th
o
r
s
h
ig
h
l
ig
h
t
th
e
m
o
d
el
's
r
o
b
u
s
tn
e
s
s
ag
a
in
s
t
n
o
v
e
l
at
ta
ck
s
an
d
cla
s
s
im
b
al
an
c
e
p
o
s
i
t
io
n
in
g
i
t
a
s
a
s
c
al
ab
l
e
s
o
l
u
t
io
n
f
o
r
r
ea
l
-
t
im
e
in
tr
u
s
io
n
d
e
te
ct
io
n
.
Gh
a
jar
i
e
t
a
l.
[
2
1
]
d
es
ig
n
e
d
an
I
DS
b
a
s
ed
o
n
h
y
p
er
d
i
m
en
s
io
n
a
l
co
m
p
u
t
in
g
(
HD
C
)
.
T
h
is
ap
p
r
o
a
ch
i
s
ef
f
e
ct
iv
e
a
t
an
aly
zin
g
h
ig
h
-
d
im
en
s
io
n
a
l
d
at
a
a
n
d
d
e
tec
t
in
g
b
o
th
k
n
o
wn
an
d
u
n
iq
u
e
a
s
s
au
lt
p
a
tt
er
n
s
.
T
h
e
a
u
to
en
co
d
er
m
o
d
e
l
te
s
te
d
o
n
th
e
NS
L
-
K
D
D
d
a
ta
s
e
t
a
ch
iev
e
d
an
a
cc
u
r
ac
y
o
f
9
1
.
5
5
%,
d
em
o
n
s
tr
at
in
g
s
t
r
o
n
g
p
o
t
en
ti
al
f
o
r
s
e
cu
r
in
g
I
o
T
n
et
wo
r
k
s
ag
a
in
s
t
s
o
p
h
i
s
t
ic
at
e
d
th
r
ea
t
s
.
S
h
ar
m
a
an
d
Ku
m
ar
[
1
0
]
p
r
o
v
id
e
a
h
y
b
r
id
C
ap
s
N
et
+
B
i
L
S
T
M
a
DL
m
o
d
el
s
,
in
clu
d
in
g
ca
p
s
u
le
n
et
wo
r
k
s
(
C
ap
s
Ne
t)
an
d
b
id
i
r
ec
tio
n
al
lo
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
(
B
iL
ST
M)
.
E
x
p
er
im
en
ta
l
f
in
d
in
g
s
s
h
o
w
t
h
at
th
e
s
u
g
g
e
s
ted
m
e
th
o
d
i
s
ef
f
e
ct
iv
e,
r
e
ac
h
in
g
a
h
ig
h
d
et
ec
t
io
n
ac
cu
r
a
cy
o
f
9
7
.
8
1
%
,
p
r
ec
i
s
io
n
o
f
9
6
.
0
0
%,
r
e
ca
ll
o
f
9
7
.
0
0
%,
an
d
F
1
-
s
co
r
e
o
f
9
6
.
0
0
%.
T
h
eir
w
o
r
k
e
m
p
h
a
s
ize
s
th
e
g
r
o
w
in
g
ad
o
p
t
io
n
o
f
DL
a
r
ch
it
ec
tu
r
e
s
in
n
e
two
r
k
tr
a
f
f
i
c
m
o
n
ito
r
in
g
,
wh
i
ch
al
ig
n
s
w
ith
t
h
e
b
r
o
ad
er
tr
en
d
o
f
lev
e
r
ag
in
g
co
m
p
lex
m
o
d
e
l
s
f
o
r
im
p
r
o
v
ed
d
et
ec
ti
o
n
a
cc
u
r
ac
y
.
Acc
o
r
d
in
g
to
Ar
aú
j
o
et
a
l.
[
2
2
]
,
tr
ad
itio
n
al
ML
m
eth
o
d
s
wer
e
ev
alu
ated
to
ca
teg
o
r
iz
e
n
etwo
r
k
attac
k
s
o
n
th
r
ee
d
ata
s
ets:
HI
KARI
-
2
0
2
1
,
UNR
-
I
DD,
an
d
UNSW
-
N
B
1
5
an
d
th
en
u
s
ed
a
s
tack
in
g
en
s
em
b
le
o
f
tr
ee
-
b
ased
b
o
o
s
tin
g
class
if
ier
s
-
XGBo
o
s
t,
L
ig
h
tGB
M,
an
d
C
at
-
B
o
o
s
t
-
f
o
r
class
if
icat
io
n
.
T
h
e
s
tack
in
g
en
s
em
b
le
g
av
e
th
e
b
est
p
er
f
o
r
m
an
ce
wh
ile
ac
h
iev
in
g
F1
-
s
co
r
e
o
f
9
3
.
7
0
%
o
n
C
I
C
-
UNS
W
-
NB
1
5
.
T
h
e
r
esu
lts
ar
e
in
v
esti
g
atio
n
,
n
am
ely
o
n
t
h
e
less
in
v
esti
g
ated
UNR
-
I
DD
an
d
C
I
C
-
UNSW
-
N
B
1
5
d
at
asets
.
T
h
r
ee
d
atasets
NSL
-
KDD,
C
I
C
-
I
DS2
0
1
7
,
a
n
d
UNSW
-
NB
1
5
ar
e
u
s
ed
i
n
th
is
s
tu
d
y
to
ass
ess
m
an
y
ML
tech
n
iq
u
es
f
o
r
n
etwo
r
k
attac
k
class
if
icatio
n
[
2
3
]
.
E
x
h
a
u
s
tiv
e
f
ea
tu
r
e
s
elec
tio
n
(
E
FS
)
+
least
s
q
u
ar
e
(
LS
)
-
SVM
is
also
u
s
ed
f
o
r
class
if
icatio
n
.
T
h
e
b
est ac
cu
r
ac
y
o
f
9
3
.
3
0
%,
p
r
ec
is
io
n
o
f
1
.
0
0
,
r
ec
all
o
f
9
8
.
0
0
,
a
n
d
F1
-
s
co
r
e
o
f
9
8
.
0
0
% f
o
r
C
I
C
-
UNS
W
-
N
B
1
5
,
th
e
E
FS
+
L
S
-
SVM
p
r
o
d
u
ce
d
th
e
b
es
t
r
esu
lts
.
W
h
en
co
m
p
ar
ed
to
o
th
er
m
o
d
els,
th
e
m
o
d
el
tak
es
th
e
s
h
o
r
test
am
o
u
n
t
o
f
tim
e
to
tr
ain
o
n
an
y
d
ataset.
T
h
ese
f
in
d
in
g
s
d
em
o
n
s
tr
ate
th
e
L
S
-
SVM
-
b
ased
m
o
d
el'
s
ap
p
r
o
p
r
iaten
ess
f
o
r
r
ea
l
-
tim
e
in
tr
u
s
io
n
d
etec
ti
o
n
ap
p
licatio
n
s
.
R
af
r
astar
a
et
a
l
.
[
2
4
]
t
o
ad
d
r
ess
m
u
lti
-
class
n
etwo
r
k
an
o
m
aly
d
etec
tio
n
.
T
h
r
ee
tr
ee
-
b
ased
en
s
em
b
le
alg
o
r
ith
m
s
ar
e
u
s
ed
in
th
is
s
tu
d
y
,
n
am
ely
r
an
d
o
m
f
o
r
est
,
XGBo
o
s
t,
an
d
Ad
aBo
o
s
t.
T
h
eir
ex
p
er
im
en
t
atio
n
,
co
n
d
u
cted
u
s
in
g
th
e
UNSW
-
N
B
1
5
d
ataset,
y
ield
ed
s
ig
n
if
ican
t
r
esu
lts
,
an
d
th
e
f
e
atu
r
e
s
p
ac
e
was
ef
f
icien
tly
r
ed
u
ce
d
.
T
h
r
ee
o
f
t
h
o
s
e
e
n
s
em
b
le
tech
n
iq
u
es
p
er
f
o
r
m
ed
b
etter
th
a
n
b
aselin
e
m
o
d
els
th
at
em
p
lo
y
ed
a
s
in
g
l
e
d
ec
is
io
n
tr
ee
class
if
ier
wh
en
co
m
b
in
e
d
with
a
n
im
p
r
o
v
e
d
Gin
i
in
d
ex
.
W
ith
9
7
.
3
0
%
ac
c
u
r
ac
y
,
r
ec
all,
an
d
p
r
ec
is
io
n
an
d
a
9
6
.
9
0
%
F1
-
s
c
o
r
e,
XGBo
o
s
t
with
Gin
i
in
d
ex
p
r
o
d
u
ce
d
g
r
ea
test
r
esu
lts
.
T
h
is
m
eth
o
d
ca
n
r
ed
u
ce
th
e
n
u
m
b
er
o
f
ch
ar
ac
ter
is
ti
cs
wh
ile
in
cr
ea
s
in
g
th
e
alg
o
r
ith
m
'
s
s
p
ee
d
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
26
:
1
2
1
9
-
1
2
3
5
1222
Ab
i
r
a
m
as
u
n
d
a
r
i
a
n
d
R
a
m
as
w
am
y
[
1
2
]
d
e
v
e
lo
p
e
d
a
n
e
n
h
a
n
c
ed
d
is
t
r
i
b
u
te
d
DD
o
S
at
ta
ck
d
et
ec
ti
o
n
(
E
D
AD
)
f
r
a
m
ew
o
r
k
u
s
in
g
p
r
i
n
ci
p
l
e
c
o
m
p
o
n
e
n
t
an
al
y
s
is
(
PC
A)
a
n
d
v
a
r
i
o
u
s
M
L
cl
as
s
if
i
er
s
.
O
n
s
e
v
e
r
a
l
C
I
C
-
I
DS
d
atas
ets
,
r
a
n
d
o
m
f
o
r
est
ac
h
i
ev
e
d
t
h
e
b
es
t
ac
cu
r
a
c
y
o
f
9
8
.
9
%
o
n
C
I
C
-
I
DS
2
0
1
7
,
wh
i
le
r
an
d
o
m
f
o
r
est
an
d
KNN
we
r
e
b
est
o
n
C
I
C
-
D
Do
S
2
0
1
9
o
f
9
8
.
7
0
%
,
a
n
d
SV
M
p
e
r
f
o
r
m
ed
t
h
e
b
est
o
n
C
I
C
-
I
DS
2
0
1
8
o
f
9
8
.
7
0
%.
Al
-
H
asa
n
i
et
a
l
.
[
1
3
]
i
n
t
r
o
d
u
ce
d
a
h
y
b
r
i
d
DL
m
o
d
el
t
o
c
o
m
b
at
i
m
p
ac
tf
u
l
DD
o
S
at
tac
k
s
.
T
h
e
f
r
a
m
ew
o
r
k
in
t
eg
r
ates
a
C
N
N
to
ex
tr
ac
t
s
p
atia
l
f
ea
t
u
r
es
wi
th
a
L
S
T
M
n
e
two
r
k
t
o
ca
p
t
u
r
e
t
em
p
o
r
al
d
e
p
en
d
e
n
cies
i
n
t
r
a
f
f
ic
d
at
a.
E
v
al
u
at
ed
o
n
C
I
C
-
DD
o
S
2
0
1
9
d
atas
et
,
m
o
d
el
d
e
m
o
n
s
tr
ate
d
ex
ce
p
t
io
n
al
p
e
r
f
o
r
m
a
n
c
e,
ac
h
i
ev
in
g
9
9
.
6
3
%
ac
c
u
r
a
cy
a
n
d
a
9
9
.
7
1
%
m
ic
r
o
-
AUC,
o
u
t
p
e
r
f
o
r
m
in
g
s
t
a
n
d
al
o
n
e
m
o
d
els
.
T
h
is
h
i
g
h
p
e
r
f
o
r
m
an
ce
d
es
p
i
te
d
a
tase
t
co
m
p
le
x
it
y
a
n
d
cl
ass
i
m
b
al
an
c
e
is
a
ttr
ib
u
t
e
d
t
o
its
d
u
al
c
ap
ab
ilit
y
to
l
ea
r
n
b
o
t
h
s
p
a
tia
l
an
d
te
m
p
o
r
al
p
at
te
r
n
s
.
Gan
k
o
ti
y
a
et
a
l
.
[
2
5
]
d
ev
el
o
p
a
d
e
e
p
c
o
n
v
o
l
u
t
io
n
a
l
n
e
u
r
al
n
et
wo
r
k
(
DC
NN
)
m
o
d
el
to
a
d
d
r
ess
t
h
e
co
n
s
t
r
a
in
ts
o
f
c
lass
i
ca
l
DD
o
S
d
et
ec
ti
o
n
in
d
y
n
a
m
ic
wi
r
el
ess
m
es
h
n
etw
o
r
k
s
(
W
M
Ns)
.
O
n
t
h
e
C
I
C
-
D
Do
S
2
0
1
9
d
at
ase
t,
t
h
ei
r
c
r
o
s
s
-
l
a
y
e
r
s
o
l
u
ti
o
n
o
u
t
p
e
r
f
o
r
m
e
d
o
t
h
e
r
a
p
p
r
o
ac
h
es
i
n
te
r
m
s
o
f
p
ac
k
et
d
eli
v
e
r
y
r
a
ti
o
an
d
en
d
-
to
-
en
d
d
ela
y
.
T
h
e
m
o
d
el
g
ets
b
est
ac
c
u
r
ac
y
o
f
9
9
.
1
4
%
,
p
r
e
cisi
o
n
o
f
9
8
.
8
1
%
,
an
d
F1
-
s
c
o
r
e
o
f
9
7
.
3
4
%
.
T
h
ese
s
o
li
d
r
es
u
lts
o
u
t
p
e
r
f
o
r
m
i
n
d
ic
at
e
th
e
m
o
d
e
l'
s
u
s
e
f
u
l
n
es
s
i
n
a
cc
u
r
at
el
y
r
eli
ab
ly
d
ete
cti
n
g
DD
o
S
at
tac
k
s
i
n
th
e
c
o
m
p
l
ica
te
d
c
o
n
te
x
t
o
f
W
MN
s
.
D
ils
h
a
d
e
t
a
l.
[
2
6
]
u
ti
li
ze
d
t
h
e
X
GB
o
o
s
t
lea
r
n
i
n
g
m
e
th
o
d
wit
h
h
i
g
h
t
o
ta
l
p
r
ec
is
i
o
n
o
f
9
4
.
0
0
%
o
n
t
h
e
C
I
C
-
DD
o
S
2
0
1
9
d
at
aset
.
Alt
h
o
u
g
h
th
e
c
o
r
r
es
p
o
n
d
in
g
p
r
e
ci
s
io
n
7
1
.
0
0
%,
r
ec
all
6
9
.
0
0
%,
an
d
F
1
-
s
c
o
r
e
6
9
.
0
0
%
i
n
d
ic
at
e
f
l
aws
i
n
m
i
n
i
m
izi
n
g
FP
o
v
er
f
a
ls
e
n
e
g
a
ti
v
es
(
FN)
,
s
u
ch
ce
n
t
r
a
ll
y
d
e
v
e
lo
p
ed
s
o
l
u
ti
o
n
s
s
u
cc
e
e
d
in
la
b
o
r
at
o
r
y
ex
p
e
r
i
m
e
n
ts
b
u
t
d
o
n
o
t
ta
k
e
r
e
alis
tic
li
m
i
tat
io
n
s
o
f
d
is
t
r
i
b
u
te
d
co
m
p
u
ti
n
g
li
k
e
t
h
e
in
te
r
n
et
o
f
v
e
h
i
cles
(
I
o
V)
i
n
to
ac
co
u
n
t
.
M
o
v
i
n
g
f
o
r
wa
r
d
f
r
o
m
t
h
es
e
f
o
u
n
d
at
io
n
s
,
f
o
l
lo
w
-
on
r
es
ea
r
c
h
h
as
esta
b
l
is
h
ed
a
f
e
d
er
ate
d
l
ea
r
n
i
n
g
f
r
a
m
e
wo
r
k
a
ch
iev
in
g
c
o
m
p
e
titi
v
e
d
et
ec
ti
o
n
p
er
f
o
r
m
a
n
c
e
wi
th
a
n
o
v
e
r
al
l
av
er
a
g
e
s
c
o
r
e
o
f
9
1
%
ac
r
o
s
s
all
m
o
d
es
o
f
att
ac
k
y
et
wit
h
d
ata
c
o
n
f
i
d
e
n
ti
ali
t
y
a
n
d
c
o
m
p
u
t
ati
o
n
a
l
ef
f
ic
ie
n
c
y
i
n
c
o
n
s
i
d
e
r
a
ti
o
n
.
A
co
n
s
is
t
e
n
t
t
h
em
e
ac
r
o
s
s
t
h
e
li
t
er
at
u
r
e
is
t
h
e
s
a
cr
if
i
ce
o
f
e
f
f
ici
en
cy
,
e
x
p
lai
n
a
b
i
lit
y
,
an
d
t
r
ac
ta
b
l
e
tr
ai
n
i
n
g
t
im
e
in
e
x
c
h
an
g
e
f
o
r
s
li
g
h
t
i
n
c
r
em
en
ts
i
n
p
r
ec
is
i
o
n
.
T
h
e
r
e
is
a
p
r
o
n
o
u
n
c
e
d
g
a
p
f
o
r
a
s
o
l
u
ti
o
n
t
h
at
b
ala
n
c
es
h
i
g
h
d
e
t
ec
ti
o
n
p
er
f
o
r
m
a
n
c
e
wit
h
o
p
e
r
a
tio
n
a
l
p
r
ac
ti
c
ali
ty
.
Me
t
a
-
h
eu
r
is
tic
al
g
o
r
it
h
m
s
o
f
f
e
r
an
i
n
t
er
esti
n
g
d
i
r
e
cti
o
n
to
wa
r
d
s
f
illi
n
g
th
e
g
ap
t
h
r
o
u
g
h
th
e
o
p
ti
m
i
za
t
io
n
o
f
b
asi
c,
m
o
r
e
e
x
p
lai
n
a
b
l
e
m
o
d
els
t
o
r
e
ac
h
co
m
p
et
iti
v
e
ly
s
u
p
e
r
i
o
r
p
er
f
o
r
m
a
n
c
es
o
v
e
r
t
h
ei
r
el
ab
o
r
a
te
co
u
n
t
er
p
a
r
ts
.
I
n
s
u
m
m
ar
y
,
t
h
e
s
tu
d
ies to
d
ate
h
av
e
u
tili
ze
d
s
ev
er
al
d
atasets
s
u
ch
as NSL
-
KD
D,
UNS
W
-
NB
1
5
,
an
d
C
I
C
-
DDo
S2
0
1
9
,
an
d
v
ar
iety
o
f
ML
an
d
DL
alg
o
r
ith
m
s
tech
n
iq
u
es
to
cy
b
er
-
attac
k
d
etec
tio
n
in
n
etwo
r
k
tr
af
f
ic.
Ho
wev
er
,
to
t
h
e
b
est
o
f
th
e
k
n
o
wled
g
e
th
er
e
n
o
p
r
ev
io
u
s
s
tu
d
ies
h
av
e
d
ev
elo
p
e
d
GR
U
an
d
KNN
m
o
d
els
o
p
tim
ized
with
n
atu
r
e
-
in
s
p
ir
ed
alg
o
r
ith
m
s
s
u
ch
as
th
e
AB
C
,
n
am
ely
GR
U
+
B
ee
an
d
KNN
+
B
ee
f
o
r
cy
b
er
-
attac
k
d
etec
tio
n
.
A
c
o
m
p
ar
ativ
e
an
aly
s
is
o
f
th
e
p
r
ev
i
o
u
s
r
elate
d
wo
r
k
is
p
r
esen
ted
in
T
ab
le
1
.
T
ab
le
1
.
An
aly
s
is
o
f
c
o
m
p
ar
is
o
n
f
o
r
th
e
r
e
le
v
an
t w
o
r
k
R
e
f
e
r
e
n
c
e
/
y
e
a
r
D
a
t
a
s
e
t
Te
c
h
n
i
q
u
e
O
u
t
c
o
m
e
(
%)
Li
mi
t
a
t
i
o
n
s
[
9
]
,
2
0
2
5
N
S
L
-
K
D
D
ANN
Ac
c
u
r
a
c
y
=
9
9
.
5
8
N
o
t
r
a
i
n
i
n
g
t
i
me
a
n
d
n
o
t
e
x
p
l
a
i
n
a
b
l
e
[
1
0
]
,
2
0
2
5
U
N
S
W
-
N
B
1
5
H
y
b
r
i
d
C
a
p
sN
e
t
+
B
i
LST
M
Ac
c
u
r
a
c
y
=
9
7
.
8
1
N
o
t
r
a
i
n
i
n
g
t
i
me
a
n
d
n
o
t
e
x
p
l
a
i
n
a
b
l
e
[
1
1
]
,
2
0
2
5
U
N
S
W
-
N
B
1
5
H
y
b
r
i
d
I
G
R
F
-
R
F
E
Ac
c
u
r
a
c
y
=
8
4
.
2
4
N
o
e
x
p
l
a
i
n
a
b
l
e
a
n
d
w
e
a
k
r
e
s
u
l
t
s
[
1
2
]
,
2
0
2
5
C
I
C
-
D
D
o
S
2
0
1
9
P
C
A
+
r
a
n
d
o
m fo
r
e
st
a
n
d
K
N
N
Ac
c
u
r
a
c
y
=
9
8
.
7
0
N
o
t
r
a
i
n
i
n
g
t
i
me
a
n
d
n
o
t
e
x
p
l
a
i
n
a
b
l
e
[
1
3
]
,
2
0
2
5
C
I
C
-
D
D
o
S
2
0
1
9
C
N
N
+
LSTM
Ac
c
u
r
a
c
y
=
9
9
.
6
3
N
o
t
e
x
p
l
a
i
n
a
b
l
e
a
n
d
c
o
mp
l
i
c
a
t
e
d
[
1
8
]
,
2
0
2
5
N
S
L
-
K
D
D
R
a
n
d
o
m
f
o
r
e
s
t
Ac
c
u
r
a
c
y
=
8
8
.
3
0
W
e
a
k
r
e
s
u
l
t
s
[
1
9
]
,
2
0
2
5
N
S
L
-
K
D
D
C
NN
+
r
a
n
d
o
m
f
o
r
e
st
Ac
c
u
r
a
c
y
=
9
8
.
7
3
N
o
t
r
a
i
n
i
n
g
t
i
me
a
n
d
c
o
m
p
l
i
c
a
t
e
d
[
2
0
]
,
2
0
2
5
N
S
L
-
K
D
D
A
u
t
o
e
n
c
o
d
e
r
Ac
c
u
r
a
c
y
=
9
1
.
5
5
S
u
b
o
p
t
i
ma
l
r
e
s
u
l
t
s
a
n
d
n
o
t
e
x
p
l
a
i
n
a
b
l
e
[
2
1
]
,
2
0
2
5
U
N
S
W
-
N
B
1
5
S
t
a
c
k
i
n
g
e
n
s
e
mb
l
e
F1
-
sc
o
r
e
=
9
3
.
7
0
N
o
t
e
n
o
u
g
h
e
v
o
l
u
t
i
o
n
a
n
d
n
o
t
e
x
p
l
a
i
n
a
b
l
e
[
2
2
]
,
2
0
2
5
U
N
S
W
-
N
B
1
5
EFS
+
LS
-
S
V
M
Ac
c
u
r
a
c
y
=
9
3
.
3
0
N
o
t
e
n
o
u
g
h
e
v
o
l
u
t
i
o
n
a
n
d
n
o
t
e
x
p
l
a
i
n
a
b
l
e
[
2
3
]
,
2
0
2
5
C
I
C
-
D
D
o
S
2
0
1
9
D
C
N
N
Ac
c
u
r
a
c
y
=
9
9
.
1
4
N
o
t
e
x
p
l
a
i
n
a
b
l
e
a
n
d
n
o
t
i
m
e
t
r
a
i
n
i
n
g
[
2
4
]
,
2
0
2
5
C
I
C
-
D
D
o
S
2
0
1
9
X
G
B
o
o
st
Ac
c
u
r
a
c
y
=
9
4
.
0
0
N
o
t
e
x
p
l
a
i
n
a
b
l
e
a
n
d
s
u
b
o
p
t
i
m
a
l
r
e
su
l
t
s
3.
M
AT
E
R
I
AL
S AN
D
M
E
T
H
O
D
I
n
th
is
s
ec
tio
n
,
a
co
m
p
r
eh
en
s
i
v
e
s
tu
d
y
o
f
th
e
d
ataset
th
at
was
u
tili
ze
d
in
th
e
ex
p
er
im
en
ts
is
p
r
o
v
id
e
d
.
I
n
ad
d
itio
n
,
th
e
m
eth
o
d
o
lo
g
y
is
p
r
o
p
o
s
ed
f
o
r
d
ev
elo
p
i
n
g
a
n
in
tellig
en
t
m
o
d
el
f
o
r
e
n
h
an
ci
n
g
th
e
d
etec
tio
n
o
f
cy
b
er
attac
k
s
to
d
eter
m
in
e
wh
e
th
er
a
n
etwo
r
k
p
ac
k
et
is
u
n
d
er
attac
k
.
T
h
e
o
v
er
all
r
esear
ch
f
r
am
ewo
r
k
c
o
n
s
is
ts
o
f
m
an
y
m
ain
p
h
ases
:
i)
d
ata
ac
q
u
is
itio
n
,
ii)
p
r
e
-
p
r
o
ce
s
s
in
g
an
d
h
y
p
er
p
ar
a
m
eter
o
p
tim
izat
io
n
u
s
in
g
th
e
AB
C
alg
o
r
ith
m
,
iii)
m
o
d
el
tr
ain
i
n
g
,
iv
)
m
o
d
el
ev
alu
atio
n
,
an
d
v
)
ex
p
lain
a
b
le
AI
in
teg
r
a
tio
n
.
T
h
e
d
etailed
m
eth
o
d
o
l
o
g
y
d
iag
r
am
is
d
e
p
ic
ted
in
Fig
u
r
e
1
.
3
.
1
.
Da
t
a
a
cquis
it
io
n
3
.
1
.
1
.
NSL
-
K
DD
T
h
e
NSL
-
KDD
d
ataset
was
s
p
ec
ially
d
ev
elo
p
ed
t
o
o
f
f
s
et
th
e
u
n
av
o
id
ab
le
im
p
er
f
ec
tio
n
s
o
f
th
e
o
r
ig
in
al
KDD
C
UP
9
9
d
ataset
[
1
4
]
,
t
y
p
ically
u
s
ed
in
in
tr
u
s
io
n
d
etec
tio
n
r
esear
ch
.
Un
lik
e
th
e
o
r
i
g
in
al
d
ataset,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
efficien
t a
p
p
r
o
a
ch
fo
r
cy
b
e
r
-
a
tta
ck
d
etec
tio
n
b
y
u
s
in
g
ma
ch
in
e
lea
r
n
in
g
a
n
d
…
(
Ya
s
ir
Hu
s
s
ein
S
h
a
kir
)
1223
NSL
-
KDD
elim
in
ates
d
u
p
licate
r
o
ws
an
d
ac
h
iev
es
a
m
o
r
e
b
alan
ce
d
n
u
m
b
er
o
f
n
o
r
m
al
an
d
attac
k
s
am
p
les
to
r
ed
u
ce
b
ias
d
u
r
in
g
tr
ai
n
in
g
a
n
d
test
in
g
.
I
t
co
n
tain
s
s
ep
ar
ate
tr
ain
in
g
an
d
test
in
g
s
ets
with
a
s
u
f
f
icien
t
n
u
m
b
er
o
f
s
am
p
les
s
o
th
at
r
esear
c
h
er
s
d
o
n
o
t
n
ee
d
to
r
ely
o
n
r
a
n
d
o
m
s
am
p
lin
g
,
m
ak
in
g
it
p
o
s
s
ib
le
to
u
s
e
th
e
e
n
tire
d
ataset
d
ir
ec
tly
.
T
h
is
lead
s
to
m
o
r
e
s
tab
le
an
d
co
m
p
a
r
ab
le
e
v
alu
atio
n
r
esu
lts
ac
r
o
s
s
d
if
f
er
e
n
t stu
d
ies o
f
I
DSs
.
3
.
1
.
2
.
UNSW
-
NB
1
5
T
h
e
UNSW
-
N
B
1
5
d
ataset
wa
s
g
en
er
ated
in
th
e
UNSW
C
a
n
b
er
r
a
C
y
b
er
R
an
g
e
L
ab
u
s
in
g
th
e
I
XI
A
Per
f
ec
tSt
o
r
m
to
o
l,
wh
ich
p
r
o
d
u
ce
d
r
aw
n
etwo
r
k
tr
a
f
f
ic
c
o
n
tai
n
in
g
a
co
m
b
in
atio
n
o
f
n
o
r
m
al
ac
tiv
ity
an
d
m
o
d
er
n
attac
k
b
e
h
av
io
r
s
.
Ap
p
r
o
x
im
ately
1
0
0
GB
o
f
r
aw
tr
af
f
ic
d
ata
was
ca
p
tu
r
ed
with
th
e
tcp
d
u
m
p
u
tili
ty
an
d
th
en
p
r
o
ce
s
s
ed
u
s
in
g
Ar
g
u
s
an
d
B
r
o
-
I
DS
to
o
ls
to
e
x
tr
a
ct
4
9
f
ea
tu
r
es
alo
n
g
with
class
lab
els.
T
h
e
d
ataset
c
o
v
er
s
n
in
e
ty
p
es
o
f
m
alicio
u
s
ac
tiv
ity
:
f
u
zz
er
s
,
an
aly
s
is
,
b
ac
k
d
o
o
r
s
,
Do
S,
ex
p
lo
its
,
g
en
e
r
ic,
r
ec
o
n
n
aiss
an
ce
,
s
h
ellco
d
e,
an
d
wo
r
m
s
.
C
o
m
p
ar
ed
with
tr
ad
itio
n
al
d
atasets
,
UNSW
-
N
B
1
5
p
r
o
v
id
es
a
m
o
r
e
r
ea
lis
tic
b
len
d
o
f
co
n
tem
p
o
r
ar
y
attac
k
p
atter
n
s
an
d
leg
itima
te
tr
af
f
ic,
m
a
k
in
g
it
a
to
u
g
h
er
an
d
m
o
r
e
e
f
f
ec
ti
v
e
b
en
ch
m
ar
k
f
o
r
ev
alu
atin
g
I
DS m
o
d
els
[
2
7
]
.
3
.
1
.
3
.
CIC
-
DDo
S2
0
1
9
T
h
e
C
I
C
-
DDo
S2
0
1
9
d
ataset
was
in
tr
o
d
u
ce
d
b
y
t
h
e
C
an
ad
i
an
I
n
s
titu
te
f
o
r
C
y
b
er
s
ec
u
r
ity
(
C
I
C
)
as
a
co
m
p
r
eh
e
n
s
iv
e
b
en
c
h
m
ar
k
f
o
r
DDo
S
d
etec
tio
n
.
I
t
in
clu
d
es
a
wi
d
e
r
an
g
e
o
f
attac
k
s
ce
n
a
r
io
s
,
s
u
ch
as
UDP,
T
C
P,
HT
T
P,
an
d
I
C
MP
f
lo
o
d
i
n
g
attac
k
s
,
al
o
n
g
with
b
e
n
ig
n
tr
af
f
ic
th
at
cl
o
s
ely
r
ef
lects
r
ea
l
-
wo
r
ld
co
n
d
itio
n
s
.
T
h
e
d
ata
was
ca
p
tu
r
ed
in
PC
AP
f
o
r
m
at
an
d
th
e
n
co
n
v
er
te
d
in
to
f
lo
w
-
b
ased
r
ec
o
r
d
s
u
s
in
g
C
I
C
Flo
wM
eter
,
wh
ic
h
in
clu
d
es
f
ea
tu
r
es
s
u
ch
as
tim
estam
p
s
,
I
P
ad
d
r
ess
e
s
,
p
o
r
ts
,
p
r
o
to
c
o
ls
,
an
d
f
lo
w
c
h
ar
ac
ter
is
tics
.
T
h
is
d
ataset
is
esp
ec
ially
v
alu
ab
le
f
o
r
in
tr
u
s
io
n
d
etec
tio
n
r
esear
c
h
,
as
it
ca
p
tu
r
es
lar
g
e
-
s
ca
le
m
o
d
er
n
DDo
S
attac
k
b
eh
av
io
r
s
a
n
d
p
r
o
v
id
es lab
ele
d
in
s
tan
ce
s
f
o
r
s
u
p
e
r
v
is
ed
lear
n
in
g
[
1
6
]
.
Fig
u
r
e
1
.
Ov
e
r
all
f
r
am
ewo
r
k
o
f
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
lo
g
y
3
.
2
.
P
re
pro
ce
s
s
ing
da
t
a
s
et
T
o
u
tili
ze
d
u
e
t
o
th
eir
d
i
v
er
s
ity
in
attac
k
ca
teg
o
r
ies
an
d
r
ep
r
esen
tatio
n
o
f
h
eter
o
g
en
eo
u
s
n
etwo
r
k
en
v
ir
o
n
m
en
ts
.
A
s
y
s
tem
atic
p
r
ep
r
o
ce
s
s
in
g
was a
p
p
lied
c
o
m
p
r
is
in
g
th
e
f
o
llo
win
g
s
te
p
s
:
i)
L
ab
el
tr
an
s
f
o
r
m
atio
n
:
t
o
s
im
p
lify
th
e
class
if
icatio
n
task
in
to
a
b
in
ar
y
d
etec
tio
n
p
r
o
b
le
m
,
all
o
r
i
g
in
al
m
u
lti
-
class
attac
k
ca
teg
o
r
ies
s
u
ch
as
Do
S,
Pro
b
e,
R
2
L
,
a
n
d
U2
R
,
wer
e
co
n
s
o
lid
ated
in
to
a
s
in
g
le
“a
ttack
”
class
,
wh
ile
n
o
r
m
al
t
r
af
f
ic
was
lab
eled
as
“b
e
n
ig
n
”
.
T
h
is
en
s
u
r
e
d
th
at
th
e
m
o
d
el
f
o
cu
s
ed
o
n
d
is
tin
g
u
is
h
in
g
b
etwe
en
m
alicio
u
s
an
d
leg
itima
te
tr
a
f
f
ic
to
(
1
)
an
d
(
2
)
.
=
{
1
,
2
,
…
,
}
∈
{
,
,
2
,
2
,
}
(
1
)
(
)
=
{
,
=
,
ℎ
(
2
)
ii)
L
ab
el
en
co
d
in
g
:
t
h
e
ca
teg
o
r
i
ca
l
lab
els
(
b
en
ig
n
a
n
d
attac
k
)
wer
e
c
o
n
v
er
te
d
in
to
n
u
m
er
ical
f
o
r
m
at
t
o
f
ac
ilit
ate
co
m
p
atib
ilit
y
with
ML
alg
o
r
ith
m
s
an
d
s
cik
it
-
lea
r
n
L
ab
elE
n
co
d
er
was
u
s
ed
t
o
r
esu
ltin
g
in
b
in
ar
y
n
u
m
er
ical
lab
els s
u
ch
a
s
0
f
o
r
“
b
en
ig
n
”
an
d
1
f
o
r
“
att
ac
k
”.
iii)
Featu
r
e
s
tan
d
ar
d
izatio
n
:
t
o
n
o
r
m
alize
th
e
r
an
g
e
o
f
n
u
m
er
ical
f
ea
tu
r
es
an
d
im
p
r
o
v
e
th
e
s
tab
ilit
y
o
f
tr
ain
in
g
th
e
“Stan
d
ar
d
Scaler
”
tech
n
iq
u
e
was
ap
p
lied
.
T
r
an
s
f
o
r
m
atio
n
wh
er
e
is
th
e
v
alu
e
o
f
th
e
j
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
26
:
1
2
1
9
-
1
2
3
5
1224
f
ea
tu
r
e
f
o
r
th
e
i
s
am
p
le,
is
th
e
m
ea
n
in
g
,
a
n
d
j
is
th
e
s
tan
d
a
r
d
d
e
v
iatio
n
o
f
t
h
e
jf
ea
t
u
r
e.
T
h
i
s
en
s
u
r
es
th
at
all
f
ea
tu
r
es
h
a
v
e
ze
r
o
m
ea
n
in
g
a
n
d
u
n
it
v
a
r
ian
ce
,
t
h
er
eb
y
m
itig
atin
g
s
ca
le
-
r
elat
ed
b
ias
ac
r
o
s
s
f
ea
tu
r
es.
T
h
er
e
f
o
r
e,
t
h
e
f
ea
tu
r
es a
r
e
s
ca
led
ac
co
r
d
in
g
to
(
3
)
.
=
−
(
3
)
iv
)
Data
s
p
litt
in
g
:
t
h
e
p
r
o
ce
s
s
ed
d
atasets
wer
e
p
ar
titi
o
n
ed
in
to
tr
ain
in
g
an
d
test
in
g
s
u
b
s
ets
u
s
in
g
a
7
0
:3
0
r
atio
.
T
h
e
tr
ain
in
g
s
et
(
7
0
%)
was
ap
p
lied
in
m
o
d
el
d
e
v
elo
p
m
en
t,
an
d
th
e
test
s
et
(
3
0
%)
w
as
r
eser
v
ed
f
o
r
u
n
b
iased
p
er
f
o
r
m
a
n
ce
ev
alu
atio
n
.
A
f
ix
ed
r
an
d
o
m
s
tate
was
ap
p
lied
to
en
s
u
r
e
r
ep
r
o
d
u
cib
ilit
y
o
f
ex
p
er
im
en
tal
r
esu
lts
.
3
.
3
.
P
r
o
po
s
ed
m
et
a
-
heuris
t
i
c
o
ptim
ized
m
o
dels
T
h
e
g
o
al
o
f
th
is
s
tu
d
y
is
to
a
p
p
ly
th
e
AB
C
m
eta
-
h
eu
r
is
tic
alg
o
r
ith
m
t
o
o
p
tim
ize
two
d
if
f
e
r
en
t
ty
p
es
o
f
class
if
ie
r
s
,
n
am
ely
ML
an
d
DL
m
o
d
els.
Sp
ec
if
ically
,
t
h
e
AB
C
alg
o
r
ith
m
is
u
tili
ze
d
to
en
h
a
n
ce
m
o
d
el
p
er
f
o
r
m
an
ce
v
ia
o
p
tim
izin
g
cr
itical
p
ar
am
eter
s
.
T
h
r
o
u
g
h
th
is
o
p
tim
izatio
n
s
tr
ateg
y
,
th
e
s
tu
d
y
aim
s
to
im
p
r
o
v
e
class
if
icatio
n
ac
cu
r
ac
y
,
r
o
b
u
s
tn
ess
,
an
d
o
v
er
all
g
en
er
al
izatio
n
ca
p
ab
ilit
y
ac
r
o
s
s
d
if
f
er
en
t d
atasets
.
3
.
3
.
1
.
K
-
nea
re
s
t
neig
hb
o
rs o
ptim
iza
t
io
n us
ing
a
rt
if
icia
l bee
co
lo
ny
I
n
th
is
h
y
b
r
id
ap
p
r
o
ac
h
,
th
e
AB
C
alg
o
r
ith
m
is
em
p
lo
y
e
d
to
a
u
to
m
ate
s
elec
tio
n
o
f
t
h
e
o
p
tim
al
n
u
m
b
er
o
f
n
eig
h
b
o
r
s
K
an
d
th
e
m
o
s
t
r
elev
an
t
f
ea
t
u
r
e
s
u
b
s
et
an
d
ad
d
r
ess
es
th
e
"c
u
r
s
e
o
f
d
im
en
s
io
n
ality
"
an
d
im
p
r
o
v
es
th
e
in
f
e
r
en
ce
s
p
ee
d
o
f
th
e
KNN
class
if
ier
.
Alg
o
r
ith
m
1
p
r
esen
ts
th
e
p
s
eu
d
o
co
d
e
o
f
th
e
KNN
+
B
ee
p
r
o
p
o
s
ed
d
etec
to
r
f
o
r
s
ec
u
r
in
g
cy
b
er
s
.
I
n
th
e
d
ec
lar
atio
n
,
a
class
n
am
ed
"
b
ee
"
is
d
ef
in
ed
,
w
ith
ea
ch
in
s
tan
ce
o
f
B
ee
r
ep
r
esen
tin
g
a
p
o
s
s
ib
le
s
o
lu
tio
n
with
ce
r
tain
KNN
h
y
p
er
p
ar
am
eter
s
(
lab
elled
m
etr
ic
an
d
n
_
n
eig
h
b
o
r
s
)
.
T
h
e
KNN
m
eth
o
d
will
ta
k
e
in
to
ac
co
u
n
t
two
f
ac
to
r
s
:
t
h
e
d
is
tan
ce
(
m
et
r
ic)
an
d
t
h
e
n
u
m
b
er
o
f
n
eig
h
b
o
r
s
(
n
_
n
eig
h
b
o
r
s
)
.
W
h
er
ea
s
f
itn
es
s
is
th
e
m
o
d
el'
s
ac
cu
r
ac
y
s
co
r
e
wh
en
th
ese
h
y
p
e
r
p
ar
a
m
eter
s
ar
e
em
p
lo
y
e
d
.
B
y
u
tili
zin
g
x
_
t
r
ain
a
n
d
y
_
tr
ain
t
o
ca
lcu
late
t
h
e
ac
cu
r
ac
y
s
co
r
e
o
n
th
e
test
s
et
a
n
d
b
y
tr
ain
in
g
a
KNN
m
o
d
el
with
th
e
B
ee
'
s
h
y
p
er
p
ar
am
eter
s
,
o
n
e
m
ay
ass
es
s
th
e
f
itn
es
s
o
f
th
e
B
ee
.
Nex
t,
th
e
alg
o
r
ith
m
co
m
p
u
tes
th
e
ac
cu
r
ac
y
s
co
r
e,
p
r
ed
icts
th
e
lab
els
(
attac
k
s
)
f
o
r
x
_
test
,
an
d
u
s
es
th
e
ac
cu
r
ac
y
s
co
r
e
to
u
p
d
ate
th
e
B
ee
'
s
f
itn
e
s
s
.
T
h
e
p
r
im
ar
y
p
u
r
p
o
s
e
o
f
th
e
B
ee
alg
o
r
ith
m
is
to
u
s
e
a
s
war
m
o
f
B
ee
s
to
o
p
tim
ize
t
h
e
KNN
h
y
p
er
p
ar
am
eter
s
ac
r
o
s
s
s
ev
er
al
r
o
u
n
d
s
.
W
e
in
itialize
m
etr
ics
b
y
u
s
in
g
a
lis
t
o
f
p
o
te
n
tial
m
etr
ics
f
o
r
d
is
tan
ce
,
a
n
d
B
ee
s
as
a
li
s
t
o
f
B
ee
o
b
jects
ar
e
in
itialized
with
a
r
an
d
o
m
m
etr
ic
an
d
r
an
d
o
m
n
_
n
eig
h
b
o
r
s
(
b
etwe
en
1
an
d
5
0
)
.
T
h
e
K
clo
s
est
p
o
in
ts
ar
e
f
o
u
n
d
b
y
u
s
in
g
t
h
e
m
o
s
t
co
m
m
o
n
ly
u
s
ed
d
i
s
tan
ce
m
etr
ics:
E
u
clid
ea
n
(
4
)
,
Ma
n
h
attan
(
5
)
,
C
h
eb
y
s
h
ev
(
6
)
,
an
d
Min
k
o
wsk
i (
7
)
.
(
,
)
=
√
∑
(
−
)
2
=
1
2
(
4
)
(
,
)
=
∑
|
−
|
=
1
(
5
)
(
,
)
=
(
,
)
(
6
)
(
,
)
=
(
∑
|
−
|
=
1
)
1
⁄
;
≥
1
(
7
)
T
h
e
m
ain
lo
o
p
d
eter
m
in
es
ea
ch
B
ee
'
s
f
itn
es
s
f
o
r
ea
ch
iter
atio
n
,
an
d
ass
em
b
les
th
e
B
ee
s
in
d
im
in
is
h
in
g
o
r
d
er
o
f
f
itn
ess
.
W
e
ch
o
o
s
e
th
e
b
est
B
ee
as
th
e
o
n
e
with
th
e
h
ig
h
est
f
itn
ess
.
T
h
en
,
th
e
r
em
ai
n
in
g
B
ee
s
ar
e
u
p
d
ated
i
n
ac
co
r
d
an
c
e
with
th
e
to
p
B
ee
,
wh
ich
h
as
a
p
r
o
b
ab
ilit
y
o
f
0
.
5
with
c
o
p
ie
s
o
f
th
e
b
est
B
ee
's
m
etr
ic
an
d
n
_
n
eig
h
b
o
r
s
.
I
n
a
d
d
itio
n
,
t
h
e
n
_
n
eig
h
b
o
r
s
a
r
e
ch
an
g
ed
b
y
a
r
a
n
d
o
m
v
alu
e
b
etwe
en
-
5
an
d
5
,
m
ak
in
g
s
u
r
e
it
alwa
y
s
r
em
ain
s
at
least
1
,
an
d
a
n
ew
m
etr
ic
is
as
s
ig
n
ed
at
r
an
d
o
m
.
T
h
e
a
lg
o
r
ith
m
m
im
ic
s
a
s
war
m
in
tellig
en
ce
m
eth
o
d
f
o
r
o
p
tim
izin
g
a
KNN
class
if
ier
'
s
h
y
p
er
p
ar
am
eter
s
.
A
p
o
s
s
ib
le
s
o
lu
tio
n
(
s
et
o
f
h
y
p
er
p
ar
am
eter
s
)
is
r
ep
r
esen
t
ed
b
y
ea
ch
B
ee
.
T
h
e
B
ee
s
ar
e
ev
alu
ated
,
s
o
r
ted
,
an
d
u
p
d
ated
iter
ativ
ely
in
o
r
d
er
to
co
n
v
er
g
e
o
n
an
id
ea
l
s
et
o
f
h
y
p
er
p
a
r
am
eter
s
.
Ma
x
im
izin
g
th
e
KNN
clas
s
if
ier
'
s
ac
cu
r
a
cy
o
n
th
e
test
s
et
is
th
e
aim
.
I
n
co
n
tr
ast,
KNN
g
en
er
ates
a
class
if
icat
io
n
r
ep
o
r
t
(
C
R
)
af
ter
b
ein
g
tr
ain
e
d
with
th
e
o
p
tim
al
p
ar
am
eter
s
to
p
r
ed
ict
a
test
d
at
aset.
T
h
e
p
r
o
p
o
s
ed
p
s
eu
d
o
co
d
e
is
g
iv
en
in
Alg
o
r
ith
m
1.
Alg
o
r
ith
m
1
: K
NN
+
B
ee
cy
b
er
-
attac
k
class
if
ier
I
n
p
u
t: Da
taset (
D)
Ou
tp
u
t: C
o
n
f
u
s
io
n
m
atr
ix
(
C
M)
,
class
if
icatio
n
r
ep
o
r
t
(
C
R
)
1:
B
eg
in
2:
Sp
lit (
D,
X_
tr
ain
,
X_
test
,
Y_
tr
ain
,
Y_
test
)
;
// test_
s
ize=
0
.
3
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
efficien
t a
p
p
r
o
a
ch
fo
r
cy
b
e
r
-
a
tta
ck
d
etec
tio
n
b
y
u
s
in
g
ma
ch
in
e
lea
r
n
in
g
a
n
d
…
(
Ya
s
ir
Hu
s
s
ein
S
h
a
kir
)
1225
3:
I
n
itialize
n
;
4:
Def
in
e
C
lass
B
ee
(
n
_
n
eig
h
b
o
r
s
,
m
etr
ic)
;
5:
C
r
ea
te
a
lis
t ‘
b
ee
s
’
as n
in
s
tan
ce
s
o
f
class
B
ee
;
6:
f
o
r
j
1
t
o
n
d
o
7:
f
o
r
b
ee
b
ee
s
d
o
8:
m
o
d
el
KNN
(
b
ee
.
n
_
n
eig
h
b
o
r
s
,
b
ee
.
m
etr
ic)
;
9:
Mo
d
el
.
f
it (
X_
tr
ai
n
,
y
_
tr
ain
)
;
10:
p
r
ed
Mo
d
el
.
p
r
e
d
ict
(
X_
test
)
;
11:
b
ee
.
f
itn
ess
ac
cu
r
ac
y
_
s
co
r
e
(
y
_
test
,
p
r
ed
)
;
12:
en
d
f
o
r
b
ee
;
13:
Desc_
So
r
t (
b
ee
s
,
b
ee
.
f
itn
ess
)
;
14:
b
est_
b
ee
b
ee
s
[
0
]
;
15:
f
o
r
i
1
t
o
n
do
16:
if
r
an
d
o
m
<
0
.
5
th
e
n
17:
B
ee
s
[
i]
n
ew
B
ee
(
b
est_
b
ee
.
n
_
n
eig
h
b
o
r
s
,
b
est_
b
ee
.
m
etr
ic
)
;
18:
Up
d
ate
(
b
ee
s
[
i]
.
n
_
n
eig
h
b
o
r
s
,
b
ee
s
[
i]
.
m
etr
ic)
;
19:
en
d
if
;
20:
en
d
f
o
r
i ;
21:
en
d
f
o
r
j ;
22:
Mo
d
el
KNN
(
b
est_
b
ee
.
n
_
n
eig
h
b
o
r
s
,
b
est_
b
ee
.
m
etr
ic)
;
23:
Mo
d
el
.
f
it (
X_
tr
ai
n
,
y
_
tr
ain
)
;
24:
p
r
ed
Mo
d
el.
p
r
e
d
ict
(
X_
test
)
;
25:
y
_
p
r
e
d
_
KNN
Mo
d
el
.
p
r
ed
i
ct
(
x
_
test
)
;
26:
C
M
co
n
f
u
s
io
n
_
m
atr
i
x
(
y
_
test
,
y
_
p
r
ed
_
KNN)
;
27:
CR
cla
s
s
if
icatio
n
_
r
ep
o
r
t (
y
_
test
,
y
_
p
r
e
d
_
KNN)
;
28:
r
etu
r
n
C
M,
C
R
;
29:
en
d
KNN+
B
ee
3
.
3
.
2
.
G
a
t
ed
re
curr
ent
un
it
o
ptim
iza
t
io
n us
ing
a
rt
if
icia
l bee
co
lo
ny
T
h
is
wo
r
k
p
r
o
p
o
s
es
a
h
y
b
r
id
in
tr
u
s
io
n
d
etec
tio
n
m
o
d
el
f
o
r
cy
b
er
s
p
ac
e,
c
o
m
b
in
i
n
g
a
G
R
U
n
eu
r
al
n
etwo
r
k
with
AB
C
m
eta
-
h
eu
r
is
tic.
T
h
e
GR
U
s
tr
u
ctu
r
e
is
well
-
o
p
tim
ized
f
o
r
th
e
p
r
eser
v
atio
n
o
f
tem
p
o
r
al
d
ep
en
d
e
n
cies
o
f
s
eq
u
en
tial
n
etwo
r
k
tr
af
f
ic
d
ata.
T
h
e
o
p
ti
m
izatio
n
is
ai
m
ed
at
f
o
u
r
m
a
jo
r
h
y
p
er
p
a
r
am
eter
s
th
at
ar
e
esp
ec
ially
im
p
o
r
tan
t f
o
r
th
e
m
o
d
el'
s
p
er
f
o
r
m
an
ce
an
d
co
m
p
u
tatio
n
al
co
s
ts
:
i)
u
n
its
1
:
n
u
m
b
e
r
o
f
GR
U
ce
lls
o
u
tp
u
t f
r
o
m
t
h
e
f
ir
s
t la
y
er
.
ii)
u
n
its
2
:
it
h
o
ld
s
th
e
n
u
m
b
er
o
f
GR
U
ce
ll
s
in
th
e
s
ec
o
n
d
lay
e
r
.
iii)
d
r
o
p
o
u
t:
d
r
o
p
o
u
t
r
atio
f
o
r
r
e
g
u
lar
izatio
n
to
av
o
i
d
o
v
er
f
itti
n
g
.
iv
)
b
atch
_
s
ize:
n
u
m
b
e
r
o
f
s
am
p
le
s
p
er
g
r
ad
ie
n
t u
p
d
ate.
T
h
e
q
u
ality
o
f
a
ca
n
d
id
ate
s
o
l
u
tio
n
(
a
h
y
p
e
r
p
ar
am
eter
s
s
et)
is
ass
e
s
s
ed
b
y
tr
ain
in
g
a
GR
U
m
o
d
el
o
n
s
u
ch
p
ar
am
eter
s
f
o
r
a
f
ew
ep
o
ch
s
an
d
esti
m
atin
g
th
e
v
alid
at
io
n
ac
cu
r
ac
y
.
T
h
e
tar
g
et
f
u
n
ct
io
n
s
ee
k
s
to
f
in
d
a
m
in
im
u
m
o
f
l
o
s
s
,
ca
lcu
lated
as
(
1
-
v
alid
atio
n
_
ac
c
u
r
ac
y
)
,
th
u
s
im
p
licitly
m
ax
im
izin
g
t
h
e
ac
cu
r
ac
y
.
T
h
e
ess
en
ce
o
f
th
e
GR
U
+
B
ee
al
g
o
r
ith
m
is
in
th
e
B
ee
o
p
tim
iz
er
class
th
at
h
an
d
les
th
e
p
o
p
u
latio
n
o
f
p
o
ten
tial
s
o
lu
tio
n
s
an
d
g
r
ad
u
ally
o
p
ti
m
izes
it.
T
h
e
alg
o
r
ith
m
s
tar
ts
with
a
p
o
p
u
latio
n
o
f
B
ee
s
with
r
an
d
o
m
h
y
p
er
p
ar
am
eter
s
with
in
d
ef
in
ed
lim
its
.
Fo
r
ev
e
r
y
iter
atio
n
an
d
ass
ess
es
th
e
f
itn
ess
o
f
all
B
ee
s
an
d
co
m
es
u
p
with
a
s
o
r
t
o
f
s
o
lu
tio
n
.
I
t
th
en
em
p
lo
y
s
th
e
b
est
s
o
lu
tio
n
s
(
e
lite
an
d
b
est
B
ee
s
s
elec
ted
)
to
s
teer
th
e
s
ea
r
ch
b
y
in
tr
o
d
u
cin
g
n
ew
ca
n
d
i
d
ate
s
o
lu
tio
n
s
with
in
th
eir
n
eig
h
b
o
r
h
o
o
d
.
T
h
e
s
ize
o
f
th
e
p
atch
d
et
er
m
in
es
th
e
s
ize
o
f
th
e
s
ea
r
ch
in
th
e
n
eig
h
b
o
r
h
o
o
d
.
I
t
r
e
p
ea
ts
th
is
p
r
o
ce
s
s
o
f
f
it
n
ess
ass
e
s
s
m
e
n
t,
s
elec
tio
n
,
an
d
s
ea
r
ch
with
in
th
e
n
eig
h
b
o
r
h
o
o
d
f
o
r
a
s
p
ec
if
ied
n
u
m
b
er
o
f
iter
atio
n
s
an
d
c
o
n
v
er
g
es
with
an
o
p
tim
al
h
y
p
er
p
ar
am
eter
s
'
s
et.
Fin
ally
,
a
GR
U
m
o
d
el
is
b
u
ilt an
d
co
m
p
letely
tr
ai
n
ed
o
n
th
e
b
est
-
h
o
led
h
y
p
e
r
p
ar
am
eter
s
.
T
h
is
b
est
-
o
p
tim
ized
m
o
d
el
is
u
tili
ze
d
f
o
r
m
a
k
in
g
th
e
f
in
al
p
r
ed
ictio
n
o
n
th
e
t
est
s
et,
an
d
a
d
etailed
e
v
alu
a
tio
n
is
co
n
d
u
cted
,
in
clu
d
in
g
a
co
n
f
u
s
io
n
m
atr
ix
(
C
M)
an
d
CR
ca
lcu
latio
n
.
Her
e
is
th
e
p
s
eu
d
o
co
d
e
p
r
o
p
o
s
ed
f
o
r
th
e
ab
o
v
e
s
tep
s
.
T
h
e
p
r
o
p
o
s
ed
p
s
eu
d
o
co
d
e
is
g
iv
en
in
Alg
o
r
ith
m
2
.
A
lg
o
r
ith
m
2
: G
R
U
+
B
ee
cy
b
er
-
attac
k
class
if
ier
I
n
p
u
t: Da
taset (
D)
Ou
tp
u
t: C
o
n
f
u
s
io
n
m
atr
ix
(
C
M)
,
class
if
icatio
n
r
ep
o
r
t
(
C
R
)
1:
B
eg
in
2:
Sp
lit (
D,
X_
tr
ain
,
X_
test
,
Y_
tr
ain
,
Y_
test
)
;
// test_
s
ize
=0
.
3
3:
R
esh
ap
e
(
X_
tr
ain
,
X_
test
)
f
o
r
GR
U
in
p
u
t ;
// Sh
ap
e:
(
s
am
p
les,
tim
estep
s
,
f
ea
tu
r
es)
4:
Def
in
e
b
o
u
n
d
s
f
o
r
h
y
p
er
p
a
r
a
m
eter
s
[
u
n
its
1
,
u
n
its
2
,
d
r
o
p
o
u
t
,
b
atch
_
s
ize]
;
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
26
:
1
2
1
9
-
1
2
3
5
1226
5:
I
n
itialize
B
ee
Op
tim
izer
(
n
_
b
e
es,
n
_
elite,
n
_
b
est,
p
atch
_
s
ize,
b
o
u
n
d
s
,
m
ax
_
iter
)
;
6:
f
o
r
j ←
1
to
m
ax
_
iter
d
o
7:
f
o
r
b
e
e
∈
Be
eOp
tim
izer
.
p
o
p
u
latio
n
d
o
8:
p
ar
am
s
←
d
ec
o
d
e
(
b
ee
)
;
9:
m
o
d
el
←
C
o
n
s
tr
u
ct_
GR
U_
Mo
d
el(
p
ar
a
m
s
.
u
n
its
1
,
p
ar
a
m
s
.
u
n
its
2
,
p
ar
am
s
.
d
r
o
p
o
u
t)
;
10:
m
o
d
el.
co
m
p
ile(
o
p
tim
izer
='
Ad
am
'
,
lo
s
s
=
'
b
in
ar
y
_
cr
o
s
s
en
tr
o
p
y
'
,
m
etr
ics=[
'
ac
cu
r
ac
y
'
]
)
;
11:
h
is
to
r
y
←
m
o
d
el.
f
it(
X_
tr
ain
,
Y_
tr
ain
,
ep
o
c
h
s
=3
,
b
atc
h
_
s
ize=
p
ar
am
s
.
b
atch
_
s
ize,
v
alid
atio
n
_
s
p
lit=0
.
2
,
v
er
b
o
s
e=
0
)
;
12:
b
ee
.
f
itn
ess
←
1
-
m
ax
(
h
is
to
r
y
.
v
al_
ac
cu
r
ac
y
)
; //
m
in
im
ize
lo
s
s
13:
en
d
f
o
r
b
ee
;
14:
Desc_
So
r
t (
B
ee
Op
tim
izer
.
p
o
p
u
latio
n
,
b
ee
.
f
itn
ess
)
;
15:
b
est_
b
ee
←
B
ee
Op
tim
izer
.
p
o
p
u
latio
n
[
0
]
;
16:
f
o
r
i ←
1
to
n
_
b
ee
s
d
o
17:
if
r
an
d
o
m
<0
.
5
th
en
18:
B
ee
Op
tim
izer
.
p
o
p
u
latio
n
[
i
]
←
Neig
h
b
o
r
_
Sear
c
h
(
b
est_
b
ee
,
p
atch
_
s
ize
,
b
o
u
n
d
s
)
;
19:
else
20:
B
ee
Op
tim
izer
.
p
o
p
u
latio
n
[
i
]
←
R
an
d
o
m
_
Sear
ch
(
b
o
u
n
d
s
)
;
21:
en
d
if
;
22:
en
d
f
o
r
i ;
23:
en
d
f
o
r
j ;
24:
b
est_
p
ar
am
s
←
d
ec
o
d
e
(
b
est_
b
ee
)
;
25:
f
in
al_
m
o
d
el
←
C
o
n
s
tr
u
ct
_
GR
U_
Mo
d
el(
b
est_
p
ar
a
m
s
.
u
n
its
1
,
b
est_
p
ar
am
s
.
u
n
its
2
,
b
est_
p
ar
am
s
.
d
r
o
p
o
u
t
)
;
26:
f
in
al_
m
o
d
el.
c
o
m
p
ile(
o
p
tim
ize
r
='
Ad
am
'
,
lo
s
s
=
'
b
in
ar
y
_
cr
o
s
s
en
tr
o
p
y
'
,
m
etr
ics=[
'
ac
cu
r
ac
y
'
]
)
;
27:
f
in
al_
m
o
d
el.
f
it(X
_
tr
ain
,
Y
_
tr
a
in
,
ep
o
c
h
s
=1
0
,
b
atc
h
_
s
ize=
b
est_
p
ar
am
s
.
b
atch
_
s
ize,
v
alid
atio
n
_
d
ata=
(
X
_
test
,
Y_
test
)
)
;
28:
y
_
p
r
e
d
_
GR
U
←
f
in
al_
m
o
d
el.
p
r
ed
ict(
X_
test
)
;
29:
y
_
p
r
e
d
_
b
in
a
r
y
←
r
o
u
n
d
(
y
_
p
r
e
d
_
GR
U)
;
30:
C
M
←
co
n
f
u
s
io
n
_
m
atr
ix
(
Y_
t
est,
y
_
p
r
ed
_
b
in
ar
y
)
;
31:
C
R
←
cla
s
s
if
icat
io
n
_
r
ep
o
r
t(
Y
_
test
,
y
_
p
r
e
d
_
b
in
a
r
y
)
;
32:
r
etu
r
n
C
M,
C
R
,
f
in
al_
m
o
d
el
;
33:
en
d
GR
U+
B
ee
3
.
3
.
3
.
H
y
perpa
ra
m
et
er
a
nd
e
x
perim
ent
a
l set
t
ing
s
T
o
en
s
u
r
e
o
f
th
e
p
r
o
p
o
s
ed
h
y
b
r
id
m
o
d
els
an
d
to
r
ea
liz
e
b
est
tr
ad
e
-
o
f
f
b
etwe
en
ac
c
u
r
ac
y
an
d
co
m
p
u
tatio
n
al
co
s
t,
a
s
p
ec
if
i
c
s
et
o
f
h
y
p
er
p
a
r
am
eter
s
was
u
s
ed
.
T
h
e
AB
C
alg
o
r
ith
m
was
co
n
f
ig
u
r
ed
to
ex
p
lo
r
e
o
p
tim
al
p
ar
am
eter
s
p
ac
e
f
o
r
b
o
th
KNN
an
d
GR
U.
T
ab
le
2
s
u
m
m
ar
izes
ex
p
e
r
im
en
tal
en
v
ir
o
n
m
en
t,
o
p
tim
izatio
n
s
ettin
g
s
f
o
r
AB
C
alg
o
r
ith
m
,
a
n
d
s
p
ec
if
ic
h
y
p
e
r
p
ar
am
eter
r
a
n
g
es f
o
r
class
if
ier
m
o
d
els.
T
ab
le
2
.
Hy
p
er
p
ar
a
m
eter
s
an
d
o
p
tim
izatio
n
s
ettin
g
s
f
o
r
p
r
o
p
o
s
ed
m
o
d
el
C
a
t
e
g
o
r
y
P
a
r
a
me
t
e
r
V
a
l
u
e
/
r
a
n
g
e
A
B
C
o
p
t
i
mi
z
a
t
i
o
n
P
o
p
u
l
a
t
i
o
n
s
i
z
e
(
B
e
e
s)
30
-
50
M
a
x
i
t
e
r
a
t
i
o
n
s
1
0
0
Li
mi
t
(
sc
o
u
t
b
e
e
t
r
i
g
g
e
r
)
20
O
b
j
e
c
t
i
v
e
f
u
n
c
t
i
o
n
M
a
x
i
m
i
z
e
a
c
c
u
r
a
c
y
/
M
C
C
K
N
N
+
B
e
e
N
u
mb
e
r
o
f
n
e
i
g
h
b
o
r
s
(
K
)
O
p
t
i
mi
z
e
d
b
y
A
B
C
(
R
a
n
g
e
:
1
–
1
5
)
D
i
st
a
n
c
e
m
e
t
r
i
c
Eu
c
l
i
d
e
a
n
F
e
a
t
u
r
e
s
e
l
e
c
t
i
o
n
B
i
n
a
r
y
A
B
C
(
B
o
o
l
e
a
n
m
a
sk
)
G
R
U
+
B
e
e
H
i
d
d
e
n
u
n
i
t
s
O
p
t
i
mi
z
e
d
b
y
A
B
C
(
3
2
,
6
4
,
1
2
8
)
D
r
o
p
o
u
t
r
a
t
e
O
p
t
i
mi
z
e
d
b
y
A
B
C
(
0
.
2
-
0
.
5
)
A
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
Ta
n
h
/
S
i
g
m
o
i
d
O
p
t
i
mi
z
e
r
S
G
D
(
w
i
t
h
A
B
C
w
e
i
g
h
t
s)
Le
a
r
n
i
n
g
r
a
t
e
0
.
0
1
(
I
n
i
t
i
a
l
)
B
a
t
c
h
si
z
e
64
Ep
o
c
h
s
50
3
.
4
.
B
a
s
eline
m
o
dels
f
o
r
co
m
pa
riso
n
T
o
en
s
u
r
e
a
f
ai
r
an
d
co
n
s
is
ten
t
ev
alu
atio
n
,
s
ev
er
al
wid
ely
ad
o
p
ted
ML
an
d
DL
m
o
d
els
wer
e
s
elec
ted
as
b
aselin
es.
T
h
ese
m
o
d
els
r
ep
r
esen
t
s
tan
d
ar
d
p
r
ac
tices
in
in
tr
u
s
io
n
d
etec
tio
n
r
esear
c
h
.
T
h
ey
en
ab
le
an
o
b
jectiv
e
co
m
p
ar
is
o
n
with
th
e
p
r
o
p
o
s
ed
h
y
b
r
id
f
r
am
ewo
r
k
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
n
efficien
t a
p
p
r
o
a
ch
fo
r
cy
b
e
r
-
a
tta
ck
d
etec
tio
n
b
y
u
s
in
g
ma
ch
in
e
lea
r
n
in
g
a
n
d
…
(
Ya
s
ir
Hu
s
s
ein
S
h
a
kir
)
1227
3
.
4
.
1
.
M
a
chine le
a
rning
m
o
dels
(
ba
s
eline
s
)
L
R
is
a
lin
ea
r
p
r
o
b
a
b
ilis
tic
class
if
ier
co
m
m
o
n
ly
u
s
ed
f
o
r
b
in
ar
y
class
if
icatio
n
d
u
e
to
its
s
im
p
licity
an
d
in
ter
p
r
eta
b
ilit
y
[
2
8
]
.
G
a
u
s
s
i
a
n
n
a
ï
v
e
B
a
y
es
(
G
NB
)
a
s
s
u
m
e
s
f
e
a
t
u
r
e
i
n
d
e
p
e
n
d
e
n
c
e
a
n
d
G
a
u
s
s
i
a
n
d
i
s
t
r
i
b
u
ti
o
n
s
,
o
f
f
e
r
i
n
g
l
o
w
c
o
m
p
u
t
a
t
i
o
n
a
l
c
o
m
p
l
e
x
it
y
a
n
d
s
t
a
b
l
e
p
e
r
f
o
r
m
a
n
c
e
i
n
h
i
g
h
-
d
i
m
e
n
s
i
o
n
a
l
s
et
t
i
n
g
s
[
2
9
]
.
Sto
ch
asti
c
g
r
ad
ien
t
d
escen
t
(
S
GD)
u
p
d
ates
m
o
d
el
p
ar
am
eter
s
iter
ativ
ely
u
s
in
g
in
d
iv
id
u
al
s
am
p
les,
m
ak
in
g
it
s
u
itab
le
f
o
r
lar
g
e
-
s
ca
le
lear
n
i
n
g
s
ce
n
ar
io
s
[
3
0
]
.
L
in
ea
r
d
is
cr
im
in
an
t
an
aly
s
is
(
L
DA)
p
r
o
jects
d
ata
in
to
a
lo
wer
-
d
im
en
s
io
n
al
s
p
ac
e
b
y
m
ax
im
izin
g
class
s
ep
ar
ab
il
ity
an
d
is
o
f
te
n
u
s
ed
as
a
p
r
e
p
r
o
ce
s
s
in
g
o
r
class
if
icatio
n
tech
n
iq
u
e
[
2
9
]
.
KNN
is
a
d
is
tan
ce
-
b
ased
m
eth
o
d
th
at
ass
ig
n
s
lab
els
b
ased
o
n
th
e
m
ajo
r
ity
class
o
f
n
ea
r
b
y
s
am
p
les an
d
is
ef
f
ec
tiv
e
wh
en
lo
ca
l d
ata
s
tr
u
ct
u
r
es a
r
e
well
d
ef
in
ed
[
1
7
]
.
3
.
4
.
2
.
Dee
p
lea
rning
m
o
dels
(
ba
s
elines
)
ANNs
ar
e
n
o
n
lin
ea
r
lear
n
in
g
m
o
d
els
ca
p
ab
le
o
f
ca
p
t
u
r
in
g
c
o
m
p
lex
p
atter
n
s
th
r
o
u
g
h
lay
er
ed
n
e
u
r
o
n
co
n
n
ec
tio
n
s
[
3
1
]
.
C
NNs
au
to
m
atica
lly
ex
tr
ac
t
h
ier
ar
c
h
ical
f
ea
tu
r
es
u
s
in
g
co
n
v
o
l
u
tio
n
al
o
p
er
atio
n
s
an
d
h
a
v
e
s
h
o
wn
s
tr
o
n
g
p
e
r
f
o
r
m
an
ce
in
v
ar
i
o
u
s
cy
b
er
s
ec
u
r
ity
task
s
[
3
1
]
.
R
ec
u
r
r
e
n
t
n
e
u
r
al
n
etwo
r
k
s
(
R
NNs)
ar
e
d
esig
n
ed
f
o
r
s
eq
u
en
tial
d
ata
m
o
d
elin
g
b
u
t
s
u
f
f
er
f
r
o
m
tr
ai
n
in
g
in
s
tab
ilit
y
d
u
e
to
v
an
is
h
in
g
g
r
ad
ie
n
ts
[
3
2
]
.
L
STM
an
d
GR
U
n
etwo
r
k
s
ad
d
r
ess
th
ese
lim
itatio
n
s
u
s
in
g
g
atin
g
m
ec
h
an
is
m
s
to
p
r
eser
v
e
lo
n
g
-
ter
m
d
ep
en
d
e
n
cies,
with
GR
U
b
ein
g
co
m
p
u
tatio
n
ally
m
o
r
e
ef
f
icien
t d
u
e
to
its
s
im
p
ler
ar
ch
itectu
r
e
[
3
1
]
.
3
.
5
.
M
o
del
ev
a
lua
t
i
o
n
Fo
r
m
ea
s
u
r
in
g
th
e
p
e
r
f
o
r
m
an
ce
s
ix
d
if
f
er
en
t
ass
ess
m
en
t
m
etr
ics
h
av
e
b
ee
n
u
s
ed
th
e
y
ar
e
lab
elled
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
F1
-
s
co
r
e,
an
d
MCC
.
W
h
ile
p
r
ec
is
io
n
ass
ess
e
s
th
e
p
er
ce
n
tag
e
o
f
ac
cu
r
ately
an
ticip
ated
p
o
s
itiv
e
ca
s
es,
ac
cu
r
ac
y
m
ea
s
u
r
es
th
e
o
v
er
all
co
r
r
ec
tn
ess
o
f
th
e
m
o
d
el'
s
p
r
ed
ictio
n
s
.
R
ec
all
m
ea
s
u
r
es
h
o
w
well
it
ca
n
d
et
ec
t
p
o
s
itiv
e
in
s
tan
ce
s
,
a
n
d
th
e
F1
-
s
co
r
e
g
iv
es
a
b
alan
ce
d
m
ea
s
u
r
e
o
f
p
r
ec
is
io
n
an
d
r
ec
all.
MCC
hi
g
h
v
alu
e
alwa
y
s
co
r
r
esp
o
n
d
s
to
h
ig
h
v
a
lu
es
f
o
r
ea
ch
o
f
th
e
CM
b
asic
r
ates:
s
en
s
i
tiv
ity
,
s
p
ec
if
icity
,
p
r
ec
is
io
n
,
an
d
n
e
g
ativ
e
p
r
ed
ictiv
e
v
alu
e.
T
h
es
e
m
ea
s
u
r
es
ar
e
ev
alu
ated
f
r
o
m
th
e
r
esu
ltin
g
CM
r
ec
o
r
d
in
g
tr
u
e
p
o
s
itiv
es (
T
P),
FP
,
FN
,
an
d
tr
u
e
n
e
g
ativ
es (
T
N)
,
b
y
th
e
(
8
)
to
(
1
2
)
.
=
+
+
+
+
(
8
)
=
+
(
9
)
=
+
(
1
0
)
1
−
=
2
×
(
×
+
)
(
1
1
)
=
(
×
)
−
(
×
)
(
+
)
×
(
+
)
×
(
+
)
×
(
+
)
(
1
2
)
3
.
6
.
E
x
pla
ina
ble a
rt
if
icia
l in
t
ellig
ence
inte
g
ra
t
io
n (
L
I
M
E
)
Fo
r
en
h
an
ce
m
en
t
o
f
in
ter
p
r
et
ab
ilit
y
an
d
tr
an
s
p
a
r
en
cy
o
f
t
h
e
p
r
o
p
o
s
ed
h
y
b
r
i
d
in
tr
u
s
io
n
d
etec
tio
n
m
o
d
els,
th
e
L
I
ME
f
r
am
ew
o
r
k
was
in
teg
r
ated
in
to
th
e
an
aly
s
is
p
h
ase.
L
I
ME
is
a
m
o
d
el
-
a
g
n
o
s
tic
in
ter
p
r
etab
ilit
y
tech
n
i
q
u
e
wh
ich
aim
s
to
ex
p
lain
in
d
i
v
id
u
al
p
r
ed
ictio
n
s
b
y
ap
p
r
o
x
im
atin
g
th
e
co
m
p
lex
d
ec
is
io
n
b
o
u
n
d
ar
y
o
f
th
e
cla
s
s
if
ier
with
a
lo
ca
lly
in
ter
p
r
e
tab
le
lin
ea
r
m
o
d
el.
T
h
e
ap
p
r
o
ac
h
was
to
ap
p
ly
L
I
ME
o
n
th
e
KNN
+
B
ee
cla
s
s
if
ier
to
in
ter
p
r
et
p
r
ed
ictio
n
o
u
tco
m
es
a
n
d
s
ea
r
c
h
f
o
r
wh
ic
h
f
ea
tu
r
es
ar
e
m
o
s
t
in
f
lu
en
tial
r
eg
a
r
d
in
g
an
“
att
ac
k
”
o
r
“
n
o
r
m
al”
class
.
Fo
r
ea
ch
d
ataset
-
NSL
-
KDD,
UNSW
-
N
B
1
5
,
an
d
C
I
C
-
DDo
S2
0
1
9
-
L
I
ME
ca
m
e
u
p
with
lo
ca
l
ex
p
lan
atio
n
s
b
y
p
er
tu
r
b
in
g
th
e
in
p
u
t
s
am
p
les
an
d
an
aly
zin
g
th
eir
ef
f
ec
t
o
n
m
o
d
el
o
u
tp
u
ts
,
p
r
e
s
en
tin
g
f
ea
tu
r
e
im
p
o
r
tan
ce
v
alu
es
th
at
s
h
o
w
th
e
q
u
a
n
titativ
e
co
n
tr
i
b
u
tio
n
o
f
ev
er
y
attr
ib
u
te
to
th
e
f
in
al
p
r
o
b
ab
ilit
y
o
f
th
e
p
r
e
d
ictio
n
.
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
e
x
p
er
im
en
tal
ac
q
u
ir
ed
f
r
o
m
ev
alu
atin
g
th
e
p
r
o
p
o
s
ed
m
e
ta
-
h
eu
r
is
tic
o
p
tim
ized
d
etec
tio
n
m
o
d
els
(
KNN
+
B
ee
an
d
GR
U
+
B
e
e)
ag
ain
s
t
co
n
v
e
n
tio
n
al
ML
an
d
DL
with
th
r
ee
d
atasets
:
C
I
C
-
DDo
S2
0
1
9
,
UNSW
-
N
B
1
5
,
an
d
NSL
-
KDD.
Var
io
u
s
m
ea
s
u
r
es
wer
e
u
s
ed
to
ev
alu
ate
ea
ch
m
o
d
el’
s
p
er
f
o
r
m
a
n
ce
,
in
clu
d
i
n
g
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all
,
F1
-
s
co
r
e,
an
d
MCC
.
T
r
ain
in
g
tim
e
an
d
test
in
g
tim
e
wer
e
also
co
n
s
id
er
ed
t
o
e
n
s
u
r
e
a
b
alan
ce
d
ev
alu
atio
n
o
f
p
r
ed
ictiv
e
ab
ilit
y
a
n
d
g
en
er
ali
za
tio
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
26
:
1
2
1
9
-
1
2
3
5
1228
4
.
1
.
Ana
ly
s
is
o
n NSL
-
K
DD
da
t
a
s
et
T
a
b
les
3
s
h
o
w
t
h
e
o
u
tc
o
m
es
f
o
r
t
h
e
NS
L
-
KDD
d
at
aset
.
T
h
e
t
h
r
o
u
g
h
ML
m
o
d
e
ls
th
e
K
NN
+
B
ee
h
y
b
r
i
d
ac
h
i
ev
ed
p
e
r
f
o
r
m
an
c
e
wit
h
ac
c
u
r
ac
y
=9
9
.
9
8
%
,
p
r
ec
is
i
o
n
=
9
9
.
9
8
%,
r
ec
a
ll
=
9
9
.
9
8
%,
a
n
d
MCC
=9
9
.
9
5
%
,
t
r
a
d
it
io
n
a
l
m
o
d
e
ls
s
u
c
h
as
L
R
=
9
2
.
3
8
%,
a
n
d
GN
B
=
9
0
.
0
4
%.
T
h
e
o
b
s
e
r
v
at
io
n
i
s
t
h
e
c
o
m
p
u
ta
ti
o
n
al
co
s
t
t
r
a
in
in
g
was
f
ast
0
.
0
4
4
1
MS
,
t
h
e
t
esti
n
g
tim
e
was
3
0
1
.
8
8
1
1
MS
,
in
d
i
ca
ti
n
g
a
s
i
g
n
i
f
ic
a
n
t
tr
ad
e
-
o
f
f
f
o
r
ac
h
ie
v
e
d
p
er
f
o
r
m
a
n
ce
g
a
in
.
T
h
is
d
e
m
o
n
s
tr
ates
t
h
at
A
B
C
o
p
ti
m
iz
ati
o
n
e
f
f
e
cti
v
e
ly
r
e
f
i
n
e
d
t
h
e
K
NN
h
y
p
er
p
ar
am
ete
r
s
m
a
x
i
m
iz
in
g
p
r
ec
is
i
o
n
an
d
g
e
n
er
ali
za
t
io
n
.
DL
th
e
GR
U
+
B
ee
v
a
r
i
a
n
t
a
c
h
ie
v
e
d
h
i
g
h
est
a
cc
u
r
a
cy
=9
9
.
9
2
%
a
n
d
b
al
an
c
ed
p
e
r
f
o
r
m
a
n
c
e
all
ev
al
u
a
ti
o
n
m
et
r
i
cs.
T
h
e
i
n
c
r
e
ase
d
t
r
ai
n
i
n
g
ti
m
e
o
f
1
1
3
.
5
3
MS
th
e
o
p
ti
m
iz
ed
m
o
d
e
l
m
ai
n
t
ai
n
e
d
a
l
o
w
tes
ti
n
g
ti
m
e
o
f
8
.
7
3
M
S p
r
es
e
n
ti
n
g
a
f
a
v
o
r
ab
le
tr
a
d
e
-
o
f
f
w
h
e
r
e
ac
c
u
r
ac
y
is
a
ch
ie
v
e
d
w
it
h
o
n
l
y
a
m
i
n
o
r
co
m
p
u
tati
o
n
a
l
o
v
e
r
h
ea
d
th
r
o
u
g
h
t
r
a
in
in
g
a
n
d
n
o
p
e
n
al
ty
d
u
r
i
n
g
c
r
i
tic
al
in
f
e
r
e
n
ce
task
s
.
T
h
e
in
cl
u
s
i
o
n
o
f
B
ee
o
p
ti
m
i
za
t
io
n
im
p
r
o
v
ed
al
l
p
e
r
f
o
r
m
a
n
ce
m
e
tr
ics
s
co
r
e
c
o
m
p
a
r
e
d
t
o
t
h
e
b
as
eli
n
e
GR
U
c
o
n
f
ir
m
i
n
g
th
e
p
o
t
en
tial
o
f
s
w
ar
m
in
tel
li
g
e
n
c
e
in
f
i
n
e
t
u
n
i
n
g
d
ee
p
a
r
c
h
i
tec
tu
r
es.
T
ab
le
3
.
Per
f
o
r
m
an
ce
m
etr
ics M
L
/DL
th
e
NSL
-
KDD
d
ataset
M
o
d
e
l
s
A
c
c
u
r
a
c
y
(
%)
P
r
e
c
i
s
i
o
n
(
%)
R
e
c
a
l
l
(
%)
F1
-
sc
o
r
e
(
%)
M
C
C
(
%)
Tr
a
i
n
i
n
g
t
i
me
M
/
S
Te
st
i
n
g
t
i
m
e
M/S
LR
9
2
.
38
9
0
.
6
9
9
5
.
5
4
9
2
.
3
8
8
4
.
7
7
0
.
1
0
2
1
0
.
0
0
4
3
GNB
90
.
0
4
9
3
.
5
8
8
7
.
3
6
90
.
0
4
80
.
3
0
0
.
1
3
6
0
0
.
0
4
3
7
S
G
D
9
7
.
7
7
9
7
.
8
0
9
7
.
7
7
9
7
.
7
7
9
5
.
5
6
0
.
2
8
2
1
0
.
0
0
4
6
LD
A
9
6
.
9
9
9
7
.
6
7
9
6
.
6
7
9
7
.
1
7
9
3
.
9
7
0
.
6
6
9
1
0
.
0
0
9
3
K
N
N
9
9
.
9
0
9
9
.
8
9
9
9
.
9
1
9
9
.
9
1
9
9
.
8
0
0
.
0
4
2
7
3
1
.
6
7
9
1
K
N
N
+
B
e
e
9
9
.
9
8
9
9
.
9
8
9
9
.
9
8
9
9
.
9
8
9
9
.
9
5
0
.
0
4
4
1
3
0
1
.
8
8
1
1
LSTM
9
9
.
6
7
9
9
.
5
6
9
9
.
8
2
9
9
.
6
9
9
9
.
3
3
9
8
.
7
2
1
7
.
6
2
C
N
N
9
9
.
5
5
9
9
.
8
8
9
9
.
2
7
9
9
.
5
7
9
9
.
5
7
3
2
.
5
1
4
.
7
7
ANN
-
1
9
9
.
6
5
9
9
.
4
3
9
9
.
9
2
9
9
.
6
7
9
9
.
3
0
4
7
.
4
1
5
.
2
9
ANN
-
2
9
9
.
7
1
9
9
.
5
4
9
9
.
9
2
9
9
.
7
3
9
9
.
4
2
3
4
.
2
0
4
.
6
9
R
N
N
9
9
.
7
9
9
9
.
7
5
9
9
.
8
6
9
9
.
8
1
9
8
.
5
9
1
0
2
.
5
0
9
.
3
6
G
R
U
9
9
.
9
0
9
9
.
9
1
9
9
.
9
0
9
9
.
9
0
9
9
.
8
0
7
6
.
8
8
8
.
4
6
G
R
U
+
B
e
e
9
9
.
9
2
9
9
.
9
1
9
9
.
9
4
9
9
.
9
2
9
9
.
8
4
1
1
3
.
5
3
8
.
7
3
4
.
2
.
Ana
ly
s
is
o
n UN
SW
-
NB
1
5
da
t
a
s
et
T
h
e
KNN
+
B
ee
m
o
d
el
ac
h
iev
ed
p
er
f
o
r
m
a
n
ce
o
n
t
h
e
UNSW
-
N
B
1
5
d
ataset
with
m
etr
ics
o
f
ac
cu
r
ac
y
=9
8
.
5
9
%,
p
r
ec
is
io
n
=9
8
.
5
6
%,
r
ec
all
=9
8
.
5
9
%,
F
1
-
s
co
r
e
=9
8
.
5
9
%,
an
d
MC
C
=9
7
.
2
0
%.
T
h
is
in
d
icate
s
a
m
o
s
t
ef
f
ec
tiv
e
an
d
b
alan
ce
d
class
if
ier
.
A
n
o
tab
le
tr
ad
e
-
o
f
f
is
o
b
s
er
v
ed
in
its
co
m
p
u
tatio
n
al
p
r
o
f
ile,
th
at
tr
ain
in
g
is
v
ir
tu
ally
in
s
tan
tan
eo
u
s
0
.
0
0
1
2
MS,
th
e
i
n
f
er
e
n
ce
s
p
ee
d
is
co
n
s
id
er
ab
ly
s
lo
wer
1
.
0
5
5
8
MS,
DL
ar
ch
itectu
r
es
ANN
-
2
ac
c
u
r
ac
y
=9
8
.
5
3
%,
p
r
ec
is
io
n
=9
7
.
2
7
%,
r
ec
all
=9
9
.
7
9
%
,
a
n
d
F
1
-
s
co
r
e
=9
8
.
5
2
%,
an
d
MCC
=9
7
.
1
0
%,
with
a
tr
ain
in
g
tim
e
o
f
9
.
6
6
MS
an
d
a
test
in
g
tim
e
o
f
0
.
8
6
MS.
C
NN
an
d
ANN
-
1
ac
h
iev
ed
a
b
it
h
ig
h
er
ac
cu
r
ac
y
b
u
t
s
h
o
we
d
m
ar
g
i
n
ally
lo
wer
r
o
b
u
s
tn
ess
to
n
o
is
e
an
d
im
b
alan
ce
.
co
m
p
ar
ed
t
o
o
th
e
r
DL
m
o
d
els h
ig
h
lig
h
tin
g
th
e
c
o
s
t o
f
th
e
B
ee
o
p
tim
izatio
n
at
th
e
p
r
ed
ictio
n
s
tag
e
as sh
o
wn
in
T
a
b
le
4
.
T
ab
le
4
.
Per
f
o
r
m
an
ce
m
etr
ics M
L
/DL
th
e
UNSW
-
N
B
1
5
M
o
d
e
l
s
A
c
c
u
r
a
c
y
(
%)
P
r
e
c
i
s
i
o
n
(
%)
R
e
c
a
l
l
(
%)
F1
-
sc
o
r
e
(
%)
M
C
C
(
%)
Tr
a
i
n
i
n
g
t
i
me
M
/
S
Te
st
i
n
g
t
i
m
e
M/S
LR
94
.
8
3
9
1
.
9
2
98
.
0
1
9
4
.
8
3
8
9
.
8
6
0
.
0
2
3
8
0
.
0
0
0
6
GNB
6
3
.
8
3
9
5
.
8
7
2
6
.
9
9
6
3
.
8
3
3
7
.
6
0
0
.
0
0
8
3
0
.
0
1
1
9
S
G
D
9
8
.
1
3
9
8
.
17
9
8
.
1
3
9
8
.
1
3
9
6
.
3
0
0
.
0
1
7
4
0
.
0
0
0
8
LD
A
9
8
.
1
0
9
6
.
8
0
9
9
.
38
9
8
.
07
9
6
.
2
3
0
.
0
6
5
6
0
.
0
0
2
5
K
N
N
9
8
.
5
6
9
7
.
5
2
9
9
.
5
8
9
8
.
5
4
9
7
.
15
0
.
0
0
1
4
0
.
2
2
4
0
K
N
N
+
B
e
e
9
8
.
5
9
9
8
.
5
6
9
8
.
5
9
9
8
.
5
9
9
7
.
2
0
0
.
0
0
1
2
1
.
0
5
5
8
LSTM
9
6
.
4
0
9
5
.
1
3
9
7
.
6
0
9
6
.
3
5
9
2
.
8
2
1
3
.
5
2
1
.
7
2
C
N
N
9
8
.
2
3
9
6
.
8
7
9
9
.
5
8
9
8
.
2
1
9
6
.
5
0
9
.
3
4
0
.
8
6
ANN
-
1
9
8
.
4
3
9
7
.
9
0
9
8
.
8
0
9
8
.
4
0
9
6
.
8
7
1
7
.
9
7
1
.
1
3
ANN
-
2
9
8
.
5
3
9
7
.
2
7
9
9
.
7
9
9
8
.
5
2
9
7
.
1
0
9
.
6
6
0
.
8
6
R
N
N
9
8
.
4
6
9
7
.
0
7
9
9
.
8
6
9
8
.
4
4
9
6
.
9
4
1
9
.
5
3
1
.
7
0
G
R
U
9
8
.
4
6
9
7
.
5
8
9
9
.
3
1
9
8
.
4
4
9
6
.
9
7
9
.
1
2
0
.
5
1
G
R
U
+
B
e
e
9
8
.
3
3
9
7
.
0
1
9
9
.
6
6
9
8
.
3
1
9
6
.
7
0
1
9
.
7
8
0
.
5
4
4
.
3
.
Ana
ly
s
is
o
n CIC
-
DDo
S2
0
1
9
da
t
a
s
et
I
n
th
e
C
I
C
-
DDo
S2
0
1
9
d
ataset
th
e
SGD
m
o
d
el
ac
h
iev
ed
ac
cu
r
ac
y
is
9
9
.
5
0
%,
m
ain
tai
n
in
g
p
ar
ity
im
p
r
o
v
in
g
co
n
s
is
ten
cy
ac
r
o
s
s
all
m
etr
ics
p
r
ec
is
io
n
is
9
9
.
5
0
%,
r
ec
all
is
9
9
.
5
0
%,
MCC
is
9
9
.
0
0
%
an
d
th
e
m
o
d
els
s
u
ch
as
L
R
is
9
7
%,
an
d
GNB
is
9
0
%,
an
d
u
ltra
-
lo
w
laten
cy
tr
ain
in
g
0
.
0
1
4
1
MS
,
test
in
g
0
.
0
0
0
6
MS,
an
d
i
n
th
e
DL
m
o
d
els,
GR
U
+
B
ee
ac
h
iev
ed
ac
cu
r
ac
y
=9
9
.
7
3
%,
p
r
ec
is
io
n
=9
9
.
8
6
%,
r
ec
all
=9
9
.
5
9
%,
Evaluation Warning : The document was created with Spire.PDF for Python.