I
A
E
S
I
n
t
e
r
n
at
io
n
al
Jou
r
n
al
of
A
r
t
if
ic
ia
l
I
n
t
e
ll
ig
e
n
c
e
(
I
J
-
AI
)
V
ol
. 14, No. 6, D
e
c
e
m
be
r
2025
, pp.
5193
~
5200
I
S
S
N
:
2252
-
8938
,
D
O
I
:
10.11591/
ij
a
i.
v
14
.i
6
.pp
5193
-
5200
5193
Jou
r
n
al
h
om
e
page
:
ht
tp
:
//
ij
ai
.
ia
e
s
c
or
e
.c
om
S
e
n
t
i
m
e
n
t
c
l
ass
i
f
i
c
at
i
on
u
si
n
g gr
ad
i
e
n
t
m
od
u
l
at
i
on
a
n
d
l
aye
r
e
d
at
t
e
n
t
i
on
B
agi
yal
ak
s
h
m
i
N
at
ar
aj
an
, T
. V
e
e
r
a
m
ak
al
i
D
e
pa
r
t
m
e
nt
of
D
a
t
a
S
c
i
e
nc
e
a
nd B
u
s
i
ne
s
s
S
ys
t
e
m
s
, S
c
hool
of
C
om
put
i
ng, S
R
M
I
ns
t
i
t
ut
e
of
S
c
i
e
nc
e
a
nd T
e
c
hnol
ogy,
K
a
t
t
a
nkul
a
t
hur
, I
ndi
a
A
r
t
ic
le
I
n
f
o
A
B
S
T
R
A
C
T
A
r
ti
c
le
h
is
to
r
y
:
R
e
c
e
iv
e
d
A
pr
29, 2025
R
e
vi
s
e
d
A
ug 1, 2025
A
c
c
e
pt
e
d
S
e
p 7, 2025
Sentimen
t
analys
is
is
a
techni
que
for
evalua
ting
text
to
asce
rtain
wh
ether
a
statement
is
positive,
negative,
or
neutral.
Currentl
y,
transfo
rmer
-
based
models
capture
the
contextual
relationships
among
words
in
a
phra
se
and
accomplis
h senti
ment anal
ysis i
n
a nuanced
manner vi
a multi
-
head
att
ention.
This
approach,
with
a
fixed
number
of
layers
and
heads,
struggles
to
fi
nd
the
complex
relations
hips
between
phrases
and
their
semanti
c
structur
es.
To
mitigate
this
issue,
the
suggested
technique
incorpora
tes
the
g
raded
multi
-
head
attention
model
(GMHA)
at
the
base
of
the
distilled
bidire
ctional
encoder
representati
ons
from
transform
ers
(
DistilBER
T
)
model.
It
is
employed
to
augment
the
layers
and
heads
progressi
vely,
capturing
the
relationshi
ps
between
sentences
in
a
sophist
icated
manner.
By
increasi
ng
the
layers
and
heads
the
proposed
model
extracts
long
-
term
and
hiera
rchical
relationshi
ps
from
the
sentence.
Additionally
,
the
attenti
on
s
entient
optimization
technique
is
introduced,
which
improves
model
learni
ng
by
giving
more
weight
to
important
words
in
a
sentence.
During
traini
ng,
the
process
checks
to
see
which
words
(“amazing"
or
"
worst"
)
get
more
attenti
on
and
gives
them
more
weight
in
the
model
update.
This
m
akes
it
easier
for
the
model
to
understand
important
emotions.
Our
suggested model
enhances perfo
rmance in
sentimen
t
explorati
on
, with an accuracy of
96.53%.
This
interpretation
includes
a
comparison
analysis
with
a
nother
contempo
rary framework
.
K
e
y
w
o
r
d
s
:
A
tt
e
nt
io
n s
e
nt
ie
nt
opt
im
iz
a
ti
on
G
r
a
de
d m
ul
ti
-
he
a
d a
tt
e
nt
io
n
H
ie
r
a
r
c
hi
c
a
l
la
ye
r
a
na
ly
s
i
s
N
a
tu
r
a
l
la
ngua
ge
pr
oc
e
s
s
in
g
S
e
le
c
ti
ve
gr
a
di
e
nt
a
dj
us
tm
e
nt
S
e
nt
im
e
nt
a
na
ly
s
is
This is an
open
acce
ss artic
le unde
r the
CC BY
-
SA
license.
C
or
r
e
s
pon
di
n
g A
u
th
or
:
B
a
gi
ya
la
ks
hm
i
N
a
t
a
r
a
ja
n
D
e
pa
r
tm
e
nt
of
D
a
ta
S
c
ie
nc
e
a
nd
B
us
in
e
s
s
S
ys
te
m
s
, S
c
hool
of
C
om
put
in
g
S
R
M
I
ns
ti
tu
te
of
S
c
ie
nc
e
a
nd
T
e
c
hnol
ogy
K
a
tt
a
nkul
a
th
ur
, I
ndi
a
E
m
a
il
:
bn7569@
s
r
m
is
t.
e
du.i
n
1.
I
N
T
R
O
D
U
C
T
I
O
N
S
e
nt
im
e
nt
a
na
ly
s
is
,
c
om
m
onl
y
r
e
f
e
r
r
e
d
to
a
s
opi
ni
on
m
in
i
ng,
is
ut
il
iz
e
d
in
na
tu
r
a
l
la
ngua
g
e
pr
oc
e
s
s
in
g
(
N
L
P
)
.
S
e
nt
im
e
nt
a
na
ly
s
i
s
is
a
ta
s
k
us
e
d
to
pr
oc
e
s
s
in
g
te
xt
ua
l
da
ta
,
in
c
lu
di
ng
pr
oduc
t
r
e
vi
e
w
s
,
c
ons
um
e
r
c
om
m
e
nt
s
,
s
oc
i
a
l
m
e
di
a
m
a
t
e
r
ia
l,
a
nd
ne
w
s
[
1]
.
T
he
s
e
nt
im
e
nt
c
a
n
be
c
a
te
gor
iz
e
d
in
to
th
r
e
e
c
la
s
s
if
ic
a
ti
ons
[
2]
.
P
os
it
iv
e
c
ont
e
nt
w
or
ds
s
ugge
s
t
a
good
a
tt
it
ude
or
c
ont
e
nt
m
e
nt
;
n
e
ga
ti
ve
e
m
ot
io
n
phr
a
s
e
s
s
ig
ni
f
y
di
s
a
ppoi
nt
m
e
nt
,
c
r
it
iq
ue
,
or
a
dv
e
r
s
e
pe
r
s
pe
c
ti
ve
s
;
a
nd
ne
ut
r
a
l
s
e
nt
im
e
nt
th
e
te
xt
c
onve
y
s
no
s
pe
c
if
ic
e
m
ot
io
ns
or
la
c
ks
c
la
r
it
y
[
3]
.
A
s
pe
c
t
-
ba
s
e
d
s
e
nt
im
e
nt
a
na
ly
s
i
s
(
A
B
S
A
)
is
a
te
c
hni
que
in
th
e
f
ie
ld
of
N
L
P
de
s
ig
ne
d
to
di
s
c
e
r
n
th
e
s
e
nt
im
e
nt
di
r
e
c
te
d
a
t
pa
r
ti
c
ul
a
r
e
le
m
e
nt
s
or
a
tt
r
ib
ut
e
s
of
a
pr
oduc
t,
s
e
r
vi
c
e
,
or
s
ubj
e
c
t
w
it
hi
n a
s
pe
c
if
ie
d s
e
nt
e
n
c
e
[
4]
.
I
ni
ti
a
ll
y,
r
ul
e
-
ba
s
e
d
s
y
s
te
m
s
e
m
pl
oye
d
le
xi
c
ons
f
or
th
e
a
n
a
ly
s
i
s
of
s
e
nt
im
e
nt
in
t
e
xt
.
T
he
r
ul
e
-
ba
s
e
d
m
e
th
odol
ogy e
xhi
bi
ts
c
ons
tr
a
in
ts
s
uc
h a
s
s
ubs
ta
nt
ia
l
de
ve
lo
pm
e
nt
e
f
f
or
t,
li
m
it
e
d f
le
xi
bi
li
ty
, a
nd c
ha
ll
e
nge
s
i
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 6, D
e
c
e
m
be
r
2025
:
5193
-
5200
5194
pr
oc
e
s
s
in
g
c
om
pl
e
x
s
e
nt
e
n
c
e
s
[
5]
.
A
s
a
r
e
s
ul
t,
m
a
c
hi
ne
le
a
r
ni
ng
e
m
e
r
ge
d.
D
iv
e
r
s
e
te
c
hni
que
s
,
s
uc
h
a
s
s
uppor
t
ve
c
to
r
m
a
c
hi
ne
s
a
nd
na
iv
e
B
a
ye
s
,
a
r
e
e
m
pl
oye
d
f
or
s
e
nt
im
e
nt
c
la
s
s
if
ic
a
ti
on
[
6]
.
H
ow
e
ve
r
,
it
ne
c
e
s
s
it
a
te
s
s
uf
f
ic
ie
nt
t
r
a
in
in
g da
ta
a
nd i
s
c
ha
ll
e
ngi
ng t
o l
oc
a
te
c
ont
e
xt
-
s
pe
c
if
ic
da
ta
[
7]
.
D
e
e
p
le
a
r
ni
ng
a
ppe
a
r
e
d
s
ig
ni
f
ic
a
nt
ly
to
m
it
ig
a
te
c
ont
e
xt
ua
l
d
e
pe
nde
nc
y.
N
um
e
r
ous
de
e
p
le
a
r
ni
ng
te
c
hni
que
s
ut
il
iz
e
d
in
s
e
nt
im
e
nt
a
na
ly
s
is
c
om
pr
is
e
r
e
c
ur
r
e
nt
ne
ur
a
l
ne
twor
k
(
R
N
N
)
,
lo
ng
s
hor
t
-
te
r
m
m
e
m
or
y
(
L
S
T
M
)
, a
nd
bi
di
r
e
c
ti
ona
l
e
nc
ode
r
r
e
pr
e
s
e
nt
a
ti
ons
f
r
om
t
r
a
ns
f
o
r
m
e
r
s
(
B
E
R
T
)
m
ode
ls
[
8]
.
R
N
N
s
a
nd L
S
T
M
s
e
nc
ount
e
r
e
d
di
f
f
ic
ul
ti
e
s
w
it
h
lo
ng
-
r
a
nge
w
or
d
de
pe
nde
nc
ie
s
,
le
a
di
ng
to
th
e
de
ve
lo
pm
e
nt
of
a
tt
e
nt
io
n
m
ode
ls
,
w
hi
c
h
pr
of
ic
ie
nt
ly
di
s
c
e
r
n
c
r
uc
ia
l
w
or
ds
in
s
e
nt
e
nc
e
s
a
nd
e
n
c
a
ps
ul
a
te
e
xt
e
nde
d
de
pe
nde
n
c
ie
s
in
te
xt
[
9]
.
M
ul
ti
-
he
a
d
a
tt
e
nt
io
n
r
e
qui
r
e
s
s
ig
ni
f
ic
a
nt
c
om
put
a
ti
ona
l
r
e
s
ou
r
c
e
s
a
nd
s
e
ve
r
a
l
hype
r
pa
r
a
m
e
te
r
s
,
w
hi
c
h
a
r
e
di
f
f
ic
ul
t
to
opt
im
iz
e
[
10]
.
T
he
c
onve
nt
io
na
l
m
ul
ti
-
he
a
d
a
tt
e
nt
io
n
di
s
tr
ib
ut
e
s
uni
f
or
m
a
tt
e
nt
io
n
a
m
ong
a
ll
he
a
ds
a
nd l
a
y
e
r
s
, l
e
a
di
ng t
o he
ig
ht
e
ne
d m
e
m
or
y us
a
ge
a
nd c
om
put
a
ti
ona
l
r
e
qui
r
e
m
e
nt
s
[
11]
.
T
o
ove
r
c
om
e
th
is
li
m
it
a
ti
on,
our
pr
opos
e
d
m
ode
l
in
c
or
p
or
a
te
s
gr
a
de
d
m
ul
ti
-
he
a
d
a
tt
e
nt
io
n
(
G
M
H
A
)
, w
hi
c
h gr
a
dua
ll
y r
is
e
s
t
he
numbe
r
of
f
oc
us
e
d he
a
ds
t
h
r
oughout l
a
ye
r
s
. T
hi
s
e
nha
n
c
e
m
e
nt
a
ll
ow
s
t
h
e
m
ode
l
to
e
f
f
e
c
ti
ve
ly
c
a
pt
ur
e
bot
h
s
e
m
a
nt
ic
a
nd
s
ynt
a
c
ti
c
pa
tt
e
r
ns
a
t
va
r
yi
ng
le
ve
l
s
of
gr
a
nul
a
r
it
y.
T
he
m
ode
l
us
e
s
th
e
c
ont
e
xt
ua
l
r
e
le
va
n
c
e
f
il
te
r
(
C
R
F
)
w
e
r
e
e
m
pl
oye
d
to
e
li
m
in
a
te
s
upe
r
f
lu
ous
a
nd
r
e
le
va
nt
in
f
or
m
a
ti
on
be
f
or
e
c
la
s
s
if
ic
a
ti
on.
T
hi
s
a
ll
ow
s
th
e
m
ode
l
to
f
oc
us
on
s
e
nt
im
e
nt
r
e
le
va
nt
a
tt
r
ib
ut
e
s
w
hi
le
di
s
c
a
r
di
ng
e
xt
r
a
ne
ous
da
ta
.
A
ddi
ti
ona
ll
y,
th
is
s
tu
dy
in
tr
oduc
e
s
th
e
n
ove
l
a
ppr
oa
c
h
is
a
tt
e
nt
io
n
-
s
e
nt
ie
nt
gr
a
di
e
nt
opt
im
iz
a
ti
on
(
A
S
G
O
)
,
w
hi
c
h
gi
ve
s
m
or
e
im
po
r
ta
nt
to
th
e
s
e
nt
i
m
e
nt
r
ic
h
te
r
m
s
du
r
in
g
th
e
tr
a
in
in
g
p
r
oc
e
s
s
.
I
t
m
odi
f
ie
s
gr
a
di
e
nt
s
of
s
e
nt
im
e
nt
r
ic
h
w
or
ds
ba
s
e
d
on
a
tt
e
nt
io
n
s
c
or
e
s
in
s
te
a
d
of
tr
e
a
ti
ng
a
ll
w
o
r
ds
e
qua
ll
y.
T
hi
s
w
or
k
e
nh
a
nc
e
s
s
e
nt
im
e
nt
c
la
s
s
if
ic
a
ti
on
s
y
s
te
m
s
by
in
c
or
por
a
ti
ng
va
r
io
us
a
tt
e
nt
io
n
la
ye
r
s
,
f
il
te
r
in
g
te
c
hni
que
s
, a
nd gr
a
di
e
nt
opt
im
iz
a
ti
on a
ppr
oa
c
h
e
s
, l
e
a
di
ng t
o i
m
pr
ove
d r
obus
tn
e
s
s
a
nd i
nt
e
r
pr
e
ta
bi
li
ty
.
2.
M
E
T
H
O
D
2.1. Dat
a
p
r
e
-
p
r
oc
e
s
s
in
g
I
n
N
L
P
,
te
xt
pr
e
pr
oc
e
s
s
in
g
is
a
vi
ta
l
pr
oc
e
s
s
th
a
t
in
c
lu
de
s
c
le
a
ni
ng
a
nd
tr
a
ns
f
or
m
in
g
th
e
da
ta
in
to
a
m
a
c
hi
ne
-
r
e
a
da
bl
e
f
or
m
a
t
[
12]
.
A
m
ong
th
e
m
a
ny
ta
s
ks
in
vol
ve
d
a
r
e
s
to
p
w
or
d
e
li
m
in
a
ti
on,
to
ke
ni
z
a
ti
on,
le
m
m
a
ti
z
a
ti
on,
a
nd
s
te
m
m
in
g. T
he
s
e
s
te
p
s
r
e
duc
e
t
he
noi
s
e
i
n t
he
da
ta
, t
he
r
e
by r
e
nde
r
in
g i
t
e
a
s
ie
r
t
o na
vi
ga
te
a
nd
m
or
e
us
e
f
ul
f
or
a
na
ly
s
is
[
13]
.
T
he
ne
xt
s
te
p
in
vol
ve
s
tr
a
ns
f
or
m
in
g
th
e
te
xt
in
to
to
ke
n
e
m
be
ddi
ng
us
in
g
th
e
D
is
ti
lB
E
R
T
e
m
be
ddi
ng
te
c
hni
que
,
w
hi
c
h
e
s
ta
bl
is
h
e
s
th
e
c
ont
e
xt
-
de
pe
nde
nt
c
onne
c
ti
on
be
twe
e
n
th
e
w
or
ds
i
n t
he
s
e
nt
e
nc
e
[
14]
.
2.2
.
D
is
t
il
B
E
R
T
m
od
e
l
D
is
ti
lB
E
R
T
is
a
pr
e
tr
a
in
e
d
m
ode
l
a
ppr
opr
ia
te
f
or
r
e
c
e
nt
a
ppl
ic
a
ti
ons
,
e
s
pe
c
ia
ll
y
s
e
nt
im
e
nt
a
n
a
ly
s
is
.
I
t
a
c
hi
e
ve
s
a
good
ba
la
nc
e
be
twe
e
n
e
f
f
ic
ie
nc
y
a
nd
a
c
c
ur
a
c
y,
m
a
ki
ng
it
us
e
f
ul
f
o
r
r
e
a
l
-
ti
m
e
ta
s
ks
a
nd
s
it
ua
ti
ons
w
he
r
e
c
om
put
a
ti
ona
l
r
e
s
our
c
e
s
a
r
e
in
a
de
qu
a
te
[
15]
.
D
e
s
pi
te
it
s
s
m
a
ll
s
iz
e
,
it
r
e
ta
in
s
a
hi
gh
le
ve
l
of
a
c
c
ur
a
c
y
in
e
m
ot
io
n
c
la
s
s
if
ic
a
ti
on.
T
he
D
is
ti
lB
E
R
T
m
ode
l
is
a
m
or
e
c
om
pa
c
t
in
te
r
pr
e
ta
ti
on
o
f
th
e
B
E
R
T
m
ode
l,
be
in
g
40%
s
m
a
ll
e
r
a
nd
60%
f
a
s
te
r
[
16]
.
A
lt
hough
B
E
R
T
ha
s
12
tr
a
ns
f
or
m
e
r
e
nc
ode
r
s
,
D
i
s
ti
lB
E
R
T
e
m
pl
oys
onl
y
6
e
nc
od
e
r
s
[
17]
,
how
e
v
e
r
,
it
a
c
hi
e
ve
s
pe
r
f
or
m
a
nc
e
c
om
pa
r
a
bl
e
to
B
E
R
T
in
a
nua
nc
e
d
f
a
s
hi
on,
a
s
de
m
ons
tr
a
te
d
in
th
e
F
ig
ur
e
1.
T
hi
s
s
tu
dy
e
xa
m
in
e
s
th
e
te
xt
ut
il
iz
in
g
th
e
6
-
t
r
a
ns
f
or
m
e
r
m
ode
l
w
it
h
12
a
tt
e
nt
io
n
he
a
ds
,
c
om
m
e
nc
in
g
w
it
h
th
e
tr
a
ns
la
ti
on
of
s
e
nt
e
nc
e
s
in
to
to
ke
ns
by
D
is
ti
lB
E
R
T
e
m
be
ddi
ng, a
nd
s
ubs
e
que
nt
ly
tr
a
ns
f
or
m
in
g
e
a
c
h
to
ke
n
in
to
ve
c
to
r
s
[
18]
.
T
hi
s
f
e
a
tu
r
e
ve
c
to
r
na
vi
ga
te
s
th
e
s
ix
tr
a
ns
f
or
m
e
r
la
ye
r
s
,
e
a
c
h
c
ons
is
ti
ng
of
m
ul
ti
-
he
a
d
s
e
lf
-
a
tt
e
nt
io
n
a
nd
a
f
e
e
df
or
w
a
r
d
ne
twor
k
[
19]
.
E
a
c
h
s
e
lf
-
a
tt
e
nt
io
n
la
ye
r
c
a
lc
ul
a
te
s
th
e
a
tt
e
nt
io
n
s
c
or
e
f
or
e
ve
r
y
to
ke
n,
s
o
c
a
pt
ur
in
g
th
e
in
te
r
-
r
e
la
ti
ons
hi
ps
a
m
ong
to
ke
ns
w
hi
le
c
onc
ur
r
e
nt
ly
e
m
pha
s
iz
in
g
di
f
f
e
r
e
nt
f
a
c
ts
of
th
e
te
xt
[
20]
.
E
a
c
h
to
ke
n
in
th
e
s
e
nt
e
nc
e
is
r
e
pr
e
s
e
nt
e
d
a
s
w
1
, w
2
, …, w
n
,
a
nd w
or
d e
m
be
ddi
ng c
a
n be
e
xpr
e
s
s
e
d by (
1)
.
L
e
t
X
be
t
he
i
nput
s
e
nt
e
nc
e
w
it
h t
he
n w
or
ds
.
=
{
1
,
2
,
3
…
}
(
1)
F
ol
lo
w
in
g
to
ke
ni
z
a
ti
on
a
nd
w
or
d
e
m
be
ddi
ng,
w
e
a
c
qui
r
e
th
e
in
put
r
e
pr
e
s
e
nt
a
ti
on
m
a
tr
ix
E
.
E
is
th
e
e
m
be
ddi
ng
ve
c
to
r
w
it
h
n
r
ow
s
a
nd
d c
ol
um
ns
,
w
it
h
d
in
di
c
a
ti
n
g
th
e
e
m
be
ddi
ng
di
m
e
ns
io
n
of
th
e
D
is
ti
lB
E
R
T
m
ode
l,
w
hi
c
h i
s
768
a
s
i
n (
2)
[
21]
.
=
[
1
,
2
,
…
.
.
,
]
∈
(
2)
T
he
a
tt
e
nt
io
n s
c
or
e
f
or
e
a
c
h t
oke
n i
s
c
om
put
e
d u
s
in
g t
he
(
3)
.
=
(
√
)
(
3)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
Se
nt
ime
nt
c
la
s
s
if
ic
at
io
n us
in
g g
r
adi
e
nt
m
odul
at
io
n and lay
e
r
e
d
…
(
B
agi
y
al
ak
s
hm
i
N
at
ar
aj
an
)
5195
I
n
th
is
in
s
ta
n
c
e
,
A
d
s
ig
ni
f
ie
s
th
e
a
tt
e
nt
io
n
s
c
or
e
f
or
e
a
c
h
to
ke
n,
w
hi
le
Q
,
K
,
a
nd
V
d
e
not
e
th
e
in
put
w
or
ds
.
T
he
hi
dde
n
s
t
a
te
va
lu
e
of
D
is
ti
lB
E
R
T
,
r
e
f
e
r
r
e
d
to
a
s
H
d
,
i
s
c
a
lc
ul
a
te
d
us
in
g
th
e
(
4)
a
nd
r
e
la
ye
d
to
th
e
G
M
H
A
la
ye
r
[
22]
.
=
(
4)
2.3
.
P
r
op
os
e
d
gr
ad
e
d
m
u
lt
i
-
h
e
ad
at
t
e
n
t
io
n
T
he
tr
a
ns
f
or
m
e
r
-
ba
s
e
d
m
ode
l
f
a
c
e
s
c
h
a
ll
e
nge
s
w
it
h
c
om
pl
e
x
a
nd
lo
ng
-
r
a
nge
de
p
e
nde
nc
ie
s
a
nd
e
xi
s
ti
ng
m
ode
l
pos
s
e
s
s
e
s
a
pr
e
de
te
r
m
in
e
d
qua
nt
it
y
of
la
ye
r
s
a
nd
he
a
ds
[
23]
.
I
n
th
is
c
ont
e
xt
,
e
a
c
h
a
tt
e
nt
io
n
la
ye
r
tr
e
a
ts
a
ll
to
ke
ns
e
qua
ll
y,
l
a
c
ki
ng
a
f
oc
us
on
va
r
yi
ng
le
ve
ls
of
a
bs
tr
a
c
ti
on.
th
e
r
e
f
or
e
,
th
is
s
tu
dy
in
c
or
por
a
te
d
G
M
H
A
f
ol
lo
w
in
g
th
e
f
in
a
l
hi
dde
n
la
y
e
r
of
D
is
ti
l
B
E
R
T
.
I
n
th
e
G
H
M
A
m
e
th
od,
th
e
la
ye
r
s
a
r
e
pr
ogr
e
s
s
iv
e
ly
in
c
r
e
a
s
e
d,
e
f
f
e
c
ti
ve
ly
c
a
pt
ur
in
g
th
e
s
e
m
a
nt
ic
in
f
or
m
a
ti
on
in
th
e
te
xt
w
it
h
nua
nc
e
.
T
h
e
in
it
ia
l
la
ye
r
c
ons
is
ts
of
two
he
a
ds
,
de
s
ig
ne
d
to
le
a
r
n
th
e
s
ynt
a
c
ti
c
s
tr
u
c
tu
r
e
a
nd
ba
s
ic
w
or
d
de
pe
nde
nc
ie
s
w
it
hi
n
th
e
s
e
nt
e
nc
e
.
I
n
th
e
s
e
c
ond
la
y
e
r
,
th
e
num
be
r
of
he
a
ds
in
c
r
e
a
s
e
s
by
f
our
to
e
m
pha
s
iz
e
phr
a
s
e
s
a
nd
ne
a
r
by
r
e
la
ti
ons
hi
ps
.
T
he
th
ir
d
la
ye
r
f
ur
th
e
r
e
xpa
nds
th
e
he
a
ds
by
e
ig
ht
to
c
a
pt
ur
e
c
ont
e
xt
ua
l
de
pe
nde
nc
ie
s
a
m
ong
th
e
w
or
ds
.
F
in
a
ll
y,
th
e
f
our
th
la
ye
r
in
c
r
e
a
s
e
s
th
e
he
a
ds
by
twe
l
ve
to
a
ddr
e
s
s
lo
ng
-
r
a
nge
de
pe
nde
nc
ie
s
,
w
it
h
a
s
tr
ong f
oc
us
on s
e
nt
im
e
nt
-
r
ic
h w
or
ds
.
F
ig
ur
e
1. A
r
c
hi
te
c
tu
r
e
of
tr
a
ns
f
or
m
e
r
m
ode
l
[
24]
T
he
D
is
ti
lB
E
R
T
m
ode
l
is
d
e
s
ig
ne
d
f
or
w
id
e
a
ppl
ic
a
ti
on;
how
e
ve
r
,
it
s
gr
a
de
d
m
e
c
h
a
ni
s
m
is
s
pe
c
if
ic
a
ll
y
opt
im
iz
e
d
f
or
s
e
nt
im
e
nt
a
na
ly
s
is
,
e
f
f
e
c
ti
ve
ly
c
a
pt
ur
in
g
la
ngua
ge
nua
nc
e
s
[
25]
.
T
he
pr
opos
e
d
a
r
c
hi
te
c
tu
r
e
r
e
pr
e
s
e
nt
e
d
in
F
ig
ur
e
2.
I
n
th
e
pr
opos
e
d
a
r
c
hi
te
c
tu
r
e
,
th
e
pr
e
-
pr
oc
e
s
s
e
d
r
e
vi
e
w
is
e
m
be
dde
d
us
in
g
D
is
ti
lB
E
R
T
e
m
be
ddi
ng,
w
hi
c
h
i
s
s
ubs
e
que
nt
ly
s
e
nt
to
th
e
D
is
ti
lB
E
R
T
m
ode
l.
T
hi
s
m
ode
l
c
om
pr
is
e
s
a
f
ix
e
d
num
be
r
of
la
ye
r
s
a
nd
he
a
ds
,
s
pe
c
if
ic
a
ll
y
s
ix
la
ye
r
s
a
nd
twe
lv
e
he
a
ds
,
a
ll
ow
in
g
th
e
r
e
vi
e
w
to
be
pr
oc
e
s
s
e
d
th
r
ough
a
ll
th
e
s
e
la
y
e
r
s
a
nd
th
e
a
tt
e
nt
io
n
s
c
or
e
f
or
e
a
c
h
to
ke
n
to
be
c
a
lc
ul
a
te
d.
T
he
s
e
la
ye
r
s
c
a
pt
ur
e
th
e
c
ont
e
xt
ua
l
r
e
la
ti
on
s
hi
ps
a
nd
d
e
pe
nde
nc
ie
s
a
m
ong
th
e
w
or
ds
a
nd
ge
n
e
r
a
te
hi
dde
n
in
f
or
m
a
ti
on,
w
hi
c
h
is
th
e
n
tr
a
ns
m
it
te
d
to
th
e
G
M
H
A
m
odul
e
.
B
y
a
ppl
yi
ng
m
ul
ti
pl
e
a
tt
e
nt
io
n
he
a
ds
hi
e
r
a
r
c
hi
c
a
ll
y
a
c
r
o
s
s
la
ye
r
s
,
th
e
G
H
M
A
im
pr
ove
s
f
e
a
tu
r
e
e
xt
r
a
c
ti
on
a
nd
e
n
a
bl
e
s
t
he
m
ode
l
to
c
a
pt
ur
e
lo
ng
e
r
-
r
a
nge
a
nd
d
e
e
pe
r
de
pe
nde
nc
ie
s
.
T
hi
s
s
tr
a
te
gy
im
pr
ove
s
it
s
a
bi
li
ty
f
or
unde
r
s
ta
n
di
ng
c
om
pl
ic
a
te
d
s
e
m
a
nt
ic
pa
tt
e
r
ns
in
a
t
e
xt
.
G
H
M
A
e
f
f
e
c
ti
ve
ly
i
m
p
r
ove
s
t
he
c
ont
e
xt
ua
l
knowle
dge
, w
hi
c
h
is
s
ubs
e
que
nt
ly
ut
il
iz
e
d f
or
a
ddi
ti
ona
l
a
na
ly
s
is
s
uc
h
a
s
a
tt
e
nt
io
n
s
e
nt
ie
nt
opt
im
iz
a
ti
on
te
c
hni
que
.
T
he
a
tt
e
nt
io
n
s
c
or
e
f
or
e
a
c
h
to
ke
n
is
c
a
lc
ul
a
te
d
by
a
c
qui
r
in
g
th
e
hi
dde
n
in
f
or
m
a
ti
on
H
d
f
r
om
th
e
b
a
s
e
m
ode
l
a
nd
m
ul
ti
pl
yi
ng
it
by
th
e
le
a
r
na
bl
e
w
e
ig
ht
pa
r
a
m
e
te
r
W
h
, a
c
c
or
di
ng t
o
(
5)
a
nd (
6)
.
W
e
r
e
,
ℎ
=
ℎ
,
ℎ
=
ℎ
,
ℎ
=
ℎ
.
ℎ
=
(
ℎ
ℎ
√
)
(
5)
T
he
upda
te
d c
on
c
e
a
le
d r
e
pr
e
s
e
nt
a
ti
on r
e
s
ul
t
of
G
M
H
A
H
h
c
a
n
be
c
om
put
e
d ut
il
iz
in
g t
he
s
uc
c
e
s
s
iv
e
f
or
m
ul
a
.
ℎ
=
ℎ
ℎ
(
6)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 6, D
e
c
e
m
be
r
2025
:
5193
-
5200
5196
F
ig
ur
e
2. P
r
opos
e
d
f
r
a
m
e
w
or
k
f
or
G
M
H
A
a
r
c
hi
te
c
tu
r
e
2.4. At
t
e
n
t
io
n
c
on
t
r
ol
le
r
T
he
a
tt
e
nt
io
n
c
ont
r
ol
l
e
r
d
e
te
r
m
in
e
s
w
h
e
r
e
t
o
s
to
p
la
y
e
r
in
c
r
e
m
e
nt
a
ti
on
a
nd
how
m
a
ny
la
y
e
r
s
s
houl
d
be
a
d
de
d
t
o
th
e
G
H
M
A
. T
h
e
a
tt
e
nt
io
n s
c
or
e
f
or
e
a
c
h
le
v
e
l
i
s
c
o
m
put
e
d a
nd c
om
p
a
r
e
d
w
it
h
th
e
pr
e
vi
ous
l
a
y
e
r
.
T
he
c
o
s
in
e
s
im
il
a
r
it
y m
e
a
s
ur
e
w
a
s
u
s
e
d t
o c
o
m
pa
r
e
t
hi
s
s
c
or
e
w
it
h t
he
s
c
or
e
f
or
t
he
pr
e
vi
ou
s
l
a
ye
r
;
if
t
he
s
c
or
e
is
le
s
s
th
a
n
or
e
qu
a
l
to
th
e
th
r
e
s
hol
d
va
lu
e
of
0.0
01,
it
in
di
c
a
t
e
s
th
a
t
th
e
m
od
e
l
ha
s
a
lr
e
a
dy
l
e
a
r
n
e
d
e
no
ugh
a
bout
th
e
c
ont
e
xt
a
nd
i
s
not
be
n
e
f
it
in
g
m
u
c
h
f
r
o
m
th
e
a
ddi
ti
on
o
f
m
or
e
la
ye
r
s
. T
h
e
c
ont
r
ol
le
r
th
e
n
s
to
p
s
to
a
dd
la
ye
r
s
.
T
he
th
r
e
s
hol
d
va
l
ue
i
s
s
e
t
a
t
0
.001,
w
hi
c
h
i
s
u
s
e
f
ul
f
or
a
ddr
e
s
s
in
g
lo
ng
-
r
a
ng
e
d
e
pe
nde
n
c
ie
s
.
T
hi
s
a
id
s
th
e
m
od
e
l
in
a
vo
id
in
g
poi
nt
l
e
s
s
c
a
l
c
ul
a
t
io
n
s
a
n
d c
o
nc
e
nt
r
a
ti
ng
s
ol
e
ly
o
n pe
r
t
in
e
nt
da
t
a
.
2.5. Con
t
e
xt
u
al
r
e
le
van
c
e
f
il
t
e
r
G
M
H
A
ge
ne
r
a
te
s
a
tt
e
nt
io
n
s
c
or
e
s
w
hi
le
a
l
s
o
of
f
e
r
in
g
s
upe
r
f
lu
ous
in
f
or
m
a
ti
on
a
nd
de
c
e
pt
iv
e
pa
tt
e
r
ns
. T
he
c
ont
e
xt
ua
l
f
il
te
r
m
e
th
odi
c
a
ll
y e
li
m
in
a
te
s
noi
s
e
a
n
d i
r
r
e
le
va
nt
i
nf
or
m
a
ti
on
be
f
or
e
s
e
ndi
ng da
ta
t
o
th
e
c
la
s
s
if
ie
r
. T
h
e
f
or
m
ul
a
f
or
obt
a
in
in
g t
he
hi
dde
n i
nf
or
m
a
ti
on
i
s
s
how
n i
n (
7)
.
=
(
ℎ
)
(
7)
T
he
W
g
de
not
e
s
th
e
tr
a
in
a
bl
e
w
e
ig
ht
m
a
tr
ix
,
w
hi
le
H
g
s
ig
ni
f
i
e
s
hi
dde
n
in
f
or
m
a
ti
on
de
r
iv
e
d
f
r
o
m
G
M
H
A
.
T
he
G
f
pr
ovi
de
s
th
e
a
s
s
e
s
s
e
d
w
e
ig
ht
f
or
e
a
c
h
f
e
a
tu
r
e
.
de
not
e
s
t
he
s
ig
m
oi
d
a
c
ti
va
ti
on
f
unc
ti
on.
T
h
e
f
il
te
r
e
d
out
put
i
s
t
he
n f
or
m
ul
a
te
d i
n (
8)
.
=
⊙
ℎ
(
8)
T
he
a
f
or
e
m
e
nt
io
ne
d
f
or
m
ul
a
de
not
e
s
th
e
c
um
ul
a
ti
ve
f
il
te
r
e
d
f
e
a
tu
r
e
s
tr
a
ns
m
it
te
d
to
th
e
s
ub
s
e
que
nt
le
ve
l
f
or
c
a
te
gor
iz
a
ti
on.
⊙
de
not
e
s
e
le
m
e
nt
-
w
is
e
m
ul
ti
pl
ic
a
ti
on.
2.6
.
A
t
t
e
n
t
io
n
s
e
n
t
ie
n
t
gr
ad
ie
n
t
op
t
im
iz
at
io
n
T
hi
s
opt
im
iz
a
ti
on
s
tr
a
te
gy
in
c
r
e
a
s
e
s
s
e
nt
im
e
nt
a
n
a
ly
s
is
qua
li
ty
by
upda
ti
ng
th
e
w
e
ig
ht
s
of
th
e
m
os
t
e
s
s
e
nt
ia
l
te
r
m
s
in
th
e
s
e
nt
e
n
c
e
w
hi
le
tr
a
in
in
g.
T
r
a
di
ti
ona
l
opt
i
m
iz
e
r
s
,
s
uc
h
a
s
A
da
m
,
upda
te
th
e
w
e
ig
ht
s
of
w
or
ds
dur
in
g
tr
a
in
in
g,
but
th
e
y
tr
e
a
t
a
ll
to
ke
ns
e
qua
ll
y.
O
ur
s
u
gge
s
te
d
m
e
th
od
f
in
ds
th
e
m
os
t
e
s
s
e
nt
ia
l
w
or
d
in
th
e
te
xt
us
in
g
th
e
a
tt
e
nt
io
n
s
c
or
e
f
r
om
D
is
ti
lB
E
R
T
a
nd
G
H
M
A
.
F
or
e
xa
m
pl
e
,
"
T
h
e
pr
oduc
t
is
a
m
a
z
in
g
but
th
e
ba
tt
e
r
y
is
not
good."
I
n
th
is
s
ta
te
m
e
nt
,
th
e
w
or
ds
"
a
m
a
z
in
g"
a
nd
"
not
good"
w
il
l
r
e
c
e
iv
e
m
or
e
a
tt
e
nt
io
n
a
nd
w
e
ig
ht
th
a
n
th
e
ot
he
r
w
or
ds
.
T
hi
s
a
ddi
ti
ona
l
in
f
or
m
a
ti
on
is
pr
ovi
de
d
to
th
e
tr
a
di
ti
ona
l
opt
im
iz
e
r
,
w
hi
c
h
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
Se
nt
ime
nt
c
la
s
s
if
ic
at
io
n us
in
g g
r
adi
e
nt
m
odul
at
io
n and lay
e
r
e
d
…
(
B
agi
y
al
ak
s
hm
i
N
at
ar
aj
an
)
5197
gi
ve
s
hi
ghe
r
w
e
ig
ht
to
ke
y
w
or
ds
dur
in
g
ba
c
kpr
opa
ga
ti
on.
B
y
f
oc
us
in
g
on
th
e
a
ppr
opr
ia
te
w
or
ds
,
th
e
m
ode
l
le
a
r
ns
m
or
e
e
f
f
e
c
ti
ve
ly
a
nd
im
pr
ove
s
it
s
a
bi
li
ty
to
unde
r
s
ta
nd
f
e
e
li
ngs
.
T
he
m
ode
l
us
e
s
th
e
s
ta
nda
r
d
gr
a
di
e
nt
w
it
h t
he
A
da
m
opt
im
iz
e
r
, e
m
pl
oyi
ng t
he
(
9)
.
∇
=
(
9)
N
e
xt
th
e
A
S
G
O
r
e
w
e
ig
ht
th
e
gr
a
di
e
nt
us
in
g
th
e
(
10)
,
th
e
n
th
e
a
m
e
nd
gr
a
di
e
nt
∇
′
us
e
d
to
r
e
ju
ve
na
te
th
e
m
ode
l
pa
r
a
m
e
te
r
us
in
g opti
m
iz
e
r
l
ik
e
A
da
m
.
∇
′
=
∇
⊙
(
+
ℎ
)
(
10)
3.
R
E
S
U
L
T
S
A
N
D
D
I
S
C
U
S
S
I
O
N
T
he
pr
opos
e
d
w
or
k
in
c
lu
de
s
c
ol
le
c
ti
ng
r
e
vi
e
w
c
om
m
e
nt
s
f
r
om
e
-
c
om
m
e
r
c
e
pl
a
tf
or
m
s
th
r
ough
a
ppl
ic
a
ti
on
pr
ogr
a
m
m
in
g
in
te
r
f
a
c
e
s
(
A
P
I
s
)
in
th
e
c
a
te
gor
ie
s
of
m
obi
le
phone
s
a
nd
a
c
c
e
s
s
or
ie
s
,
a
s
w
e
ll
a
s
c
lo
th
in
g
a
nd
a
c
c
e
s
s
or
ie
s
,
a
nd
pe
r
f
or
m
in
g
da
ta
a
nnot
a
ti
on.
W
e
ga
th
e
r
e
d
m
or
e
E
ngl
is
h
la
ngua
ge
c
om
m
e
nt
s
f
r
om
onl
in
e
s
our
c
e
s
,
in
c
lu
di
ng
pos
it
iv
e
,
ne
ga
ti
ve
,
a
nd
n
e
ut
r
a
l
s
e
nt
im
e
nt
s
,
a
s
d
e
pi
c
te
d
in
F
ig
ur
e
3.
T
he
yi
e
ld
e
d
d
a
ta
s
e
t
d
e
ta
il
s
r
e
pr
e
s
e
nt
e
d
in
th
e
T
a
bl
e
1,
it
s
h
ow
s
our
da
ta
s
e
t
is
r
e
a
s
ona
bl
y
ba
la
nc
e
d
a
nd
s
ui
ta
bl
e
f
or
s
e
nt
im
e
nt
a
na
ly
s
i
s
.
F
ig
ur
e
3. R
e
vi
e
w
c
om
m
e
nt
s
da
ta
s
e
t
T
a
bl
e
1. D
a
ta
s
e
t
di
s
tr
ib
ut
io
n
C
l
a
s
s
N
um
be
r
of
c
om
m
e
nt
s
P
os
i
t
i
ve
36965
N
e
ga
t
i
ve
27929
N
e
ut
r
a
l
26376
T
he
m
ode
l
w
a
s
tr
a
in
e
d
a
nd
de
pl
oy
e
d
w
it
h
th
is
da
ta
s
e
t
to
c
ond
uc
t
s
e
nt
im
e
nt
a
na
ly
s
is
on
th
e
r
e
vi
e
w
c
om
m
e
nt
s
.
T
he
m
ode
l
ut
il
iz
e
s
th
e
A
da
m
opt
im
iz
e
r
to
id
e
nt
if
y
th
e
pa
r
a
m
e
te
r
s
ne
c
e
s
s
it
a
ti
ng
upda
te
s
.
T
he
pr
in
c
ip
a
l
s
tu
dy
il
lu
s
tr
a
te
s
th
a
t
th
e
m
ode
l
ut
il
iz
e
s
bot
h
G
M
H
A
a
nd
a
n
A
tt
e
nt
io
n
s
e
nt
ie
nt
opt
im
iz
a
ti
on
m
e
th
od,
a
lo
ngs
id
e
a
pr
e
tr
a
in
e
d
m
ode
l,
he
n
c
e
im
pr
ovi
ng
th
e
e
f
f
i
c
a
c
y
of
s
e
nt
im
e
nt
a
na
ly
s
i
s
ta
s
ks
. T
he
e
xi
s
ti
ng
pr
e
tr
a
in
e
d
m
ode
l,
in
c
lu
de
s
onl
y
s
e
lf
-
a
tt
e
nt
io
n
la
ye
r
,
it
us
e
s
f
i
xe
d
num
be
r
of
node
s
a
nd
la
ye
r
s
f
or
s
e
m
a
nt
ic
a
na
ly
s
is
,
but
va
r
io
us
pr
oc
e
s
s
in
g
la
ye
r
e
s
s
e
nt
ia
l
to
pr
oc
e
s
s
c
om
pos
it
e
in
f
or
m
a
ti
on
c
ont
a
in
e
d
in
th
e
s
e
nt
e
nc
e
.
S
o,
in
our
pr
opos
e
d
m
ode
l
us
e
d
pr
ogr
e
s
s
iv
e
a
tt
e
nt
io
n
on
th
e
s
e
nt
e
nc
e
s
,
it
in
c
r
e
a
s
e
s
th
e
pr
e
c
is
io
n,
r
e
c
a
ll
a
nd
a
c
c
ur
a
c
y
of
th
e
m
ode
l.
T
he
m
ode
l
ha
s
a
pr
e
c
is
io
n
of
96.23%
,
a
n
F
1
-
s
c
or
e
of
96.32%
,
a
r
e
c
a
ll
of
96.53%
, a
nd
a
n a
c
c
ur
a
c
y of
96.53%
. T
he
d
e
pi
c
te
d
G
M
H
A
c
a
n m
a
na
g
e
t
he
c
om
pl
e
x r
e
la
ti
ons
hi
ps
a
m
ong the
t
e
xt
s
.
W
e
ut
il
iz
e
d
our
da
ta
s
e
t
w
it
h
s
e
ve
r
a
l
e
s
ta
bl
i
s
he
d
s
e
nt
im
e
nt
a
na
ly
s
is
m
ode
ls
,
s
e
nt
im
e
nt
a
na
ly
s
i
s
-
a
tt
e
nt
io
n
ga
te
d
r
e
c
ur
r
e
nt
uni
t
(
G
R
U
)
,
s
e
nt
im
e
nt
a
na
ly
s
is
-
a
tt
e
nt
io
n
B
E
R
T
.
S
e
nt
im
e
nt
a
na
ly
s
is
-
a
tt
e
nt
io
n
r
obus
tl
y
opt
im
iz
e
d
B
E
R
T
pr
e
tr
a
in
in
g
a
ppr
oa
c
h
(
R
oB
E
R
T
a
)
a
nd
c
om
pa
r
e
d
th
e
r
e
s
ul
ts
w
it
h
ou
r
s
ugge
s
te
d
m
ode
l.
T
a
bl
e
2 i
ll
us
tr
a
te
s
t
h
e
c
om
pa
r
is
on of
pr
e
c
is
io
n a
c
r
o
s
s
di
f
f
e
r
e
nt
s
e
nt
im
e
nt
a
na
ly
s
is
m
ode
ls
.
T
a
bl
e
2. P
e
r
f
or
m
a
nc
e
e
va
lu
a
ti
on of
s
e
nt
im
e
nt
a
n
a
ly
s
is
m
ode
l
s
M
ode
l
P
r
e
c
i
s
i
on
A
c
c
ur
a
c
y
SA
-
a
t
t
e
nt
i
on
G
R
U
82.6
83.3
SA
-
a
t
t
e
nt
i
on
B
E
R
T
86.3
86.6
SA
-
a
t
t
e
nt
i
on
R
oB
E
R
T
a
90.2
90.4
G
M
H
A
96.23
96.53
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 6, D
e
c
e
m
be
r
2025
:
5193
-
5200
5198
T
hi
s
pr
opos
e
d
r
e
s
e
a
r
c
h
in
ve
s
ti
ga
te
s
th
e
e
f
f
e
c
t
s
of
c
ons
e
c
ut
iv
e
la
ye
r
s
a
nd
p
a
r
a
ll
e
l
a
tt
e
nt
io
n.
T
he
pr
opos
e
d
m
ode
l
e
xhi
bi
ts
s
upe
r
io
r
r
e
s
ul
ts
c
om
pa
r
e
d
t
o
a
n
a
lt
e
r
na
ti
ve
m
ode
l,
w
it
h
th
e
pr
e
c
is
io
n
c
om
pa
r
is
on
il
lu
s
tr
a
te
d
in
F
ig
ur
e
4.
T
h
e
c
ur
r
e
nt
a
ppr
oa
c
he
s
e
m
pl
oyi
ng
a
tt
e
nt
io
n
m
e
c
ha
ni
s
m
s
a
r
e
a
s
s
e
s
s
e
d,
a
s
th
e
in
nova
ti
ve
a
tt
e
nt
io
n
m
e
c
h
a
ni
s
m
is
e
f
f
or
tl
e
s
s
ly
in
c
lu
de
d
in
t
o
th
e
e
xi
s
ti
ng
m
ode
l
r
e
l
e
va
nt
to
our
s
ugg
e
s
te
d
f
r
a
m
e
w
or
k. T
he
c
ur
r
e
nt
m
ode
l
pr
im
a
r
il
y e
nha
nc
e
s
t
he
opt
im
iz
a
ti
on of
ne
ur
a
l
ne
twor
ks
. T
he
e
xi
s
ti
ng a
tt
e
nt
io
n
s
ys
te
m
e
n
c
ode
s
phr
a
s
e
s
a
nd
e
va
lu
a
te
s
th
e
s
e
nt
im
e
nt
s
tr
e
ngt
h
of
w
or
ds
;
ne
ve
r
th
e
le
s
s
,
it
s
e
f
f
ic
a
c
y
is
li
m
it
e
d.
O
ur
s
ugge
s
te
d
m
ode
l
ut
il
iz
e
s
th
e
a
tt
e
nt
io
n
m
e
c
ha
ni
s
m
a
lo
ngs
i
de
a
n
opt
im
iz
a
ti
on
m
e
th
od
to
im
pr
ove
w
e
ig
ht
a
dj
us
tm
e
nt
dur
in
g
ba
c
kpr
opa
ga
ti
on,
he
nc
e
a
ll
oc
a
ti
ng
gr
e
a
te
r
w
e
ig
ht
to
s
ig
ni
f
ic
a
nt
w
or
ds
in
s
id
e
th
e
phr
a
s
e
.
F
ig
ur
e
5 i
ll
us
tr
a
te
s
t
he
a
c
c
ur
a
c
y a
s
s
oc
i
a
ti
on be
twe
e
n our
s
ugge
s
te
d m
ode
l
a
nd t
he
e
s
ta
bl
is
he
d m
ode
l.
F
ig
ur
e
4. P
r
e
c
is
io
n
c
om
pa
r
is
on
F
ig
ur
e
5. A
c
c
ur
a
c
y
c
om
pa
r
is
on
4.
C
O
N
C
L
U
S
I
O
N
T
he
G
M
H
A
a
ppr
oa
c
h
a
c
c
om
pl
is
h
e
s
s
e
nt
im
e
nt
a
na
ly
s
is
on
r
e
vi
e
w
c
om
m
e
nt
s
w
it
h
s
ophi
s
ti
c
a
ti
on.
T
hi
s
nove
l
m
e
th
od
ut
il
iz
e
s
a
G
M
H
A
m
e
c
ha
ni
s
m
a
nd
a
n
op
ti
m
iz
a
ti
on
s
tr
a
te
gy
to
c
a
pt
ur
e
th
e
c
ont
e
xt
ua
l
c
onne
c
ti
ons
a
m
ong wor
ds
i
n a
phr
a
s
e
. I
t
s
y
s
te
m
a
ti
c
a
ll
y de
s
c
r
ib
e
s
t
he
r
e
la
ti
ons
hi
p, i
nc
or
por
a
ti
ng s
ynt
a
c
ti
c
a
nd
s
e
m
a
nt
ic
a
tt
r
ib
ut
e
s
f
r
om
th
e
te
xt
th
r
ough
th
e
g
r
a
dua
l
e
nha
nc
e
m
e
nt
of
a
tt
e
nt
io
n
la
ye
r
s
.
T
hi
s
a
ppr
oa
c
h
e
nha
nc
e
s
th
e
m
od
e
l'
s
e
f
f
ic
a
c
y
a
nd
e
xhi
bi
ts
s
tr
ong
a
dhe
r
e
nc
e
t
o
r
e
c
ogni
z
e
d
N
L
P
pa
r
a
di
gm
s
.
I
t
ga
th
e
r
s
m
or
e
pr
e
c
is
e
da
ta
to
e
nha
nc
e
th
e
m
ode
l'
s
e
f
f
ic
a
c
y.
T
he
pr
opos
e
d
m
e
th
odol
ogy
w
a
s
e
va
lu
a
te
d
us
in
g
a
r
e
a
l
da
ta
s
e
t,
pr
oduc
in
g
e
f
f
e
c
ti
ve
r
e
s
ul
ts
e
m
pl
oyi
ng
our
opt
im
iz
a
ti
on
te
c
hni
que
s
.
W
e
e
m
pl
oy
e
d
th
e
s
a
m
e
r
e
a
l
da
ta
s
e
t
f
or
th
e
pr
e
s
e
nt
m
ode
ls
a
nd
pe
r
f
or
m
e
d
a
na
ly
s
e
s
;
ba
s
e
d
on
th
e
out
c
om
e
s
of
th
is
e
xp
e
r
im
e
nt
,
w
e
c
om
pa
r
e
d
th
e
m
w
it
h
our
pr
opos
e
d
m
ode
l.
T
he
pr
opos
e
d
w
or
k
e
m
pl
oys
a
tt
e
nt
io
n
s
e
nt
im
e
nt
opt
im
iz
a
ti
on
s
tr
a
te
gi
e
s
th
a
t
pr
io
r
it
iz
e
s
e
nt
im
e
nt
in
te
ns
it
y
w
or
ds
dur
in
g
hype
r
pa
r
a
m
e
te
r
tu
ni
ng,
s
o
hi
ghl
ig
ht
in
g
th
e
m
or
e
s
ig
ni
f
ic
a
nt
c
om
pone
nt
s
of
th
e
phr
a
s
e
in
va
r
io
us
a
s
pe
c
ts
of
r
e
vi
e
w
c
om
m
e
nt
s
a
nd
pr
oduc
in
g
m
or
e
a
c
c
ur
a
te
f
in
di
ngs
in
a
nua
nc
e
d
m
a
nn
e
r
.
F
ut
ur
e
e
nde
a
vor
s
w
il
l
in
c
lu
d
e
in
te
gr
a
ti
ng
a
d
di
ti
ona
l
li
ngui
s
ti
c
e
xt
e
r
na
l
knowle
dge
in
to
th
e
m
ode
l
a
nd
e
nde
a
vor
in
g
to
i
m
pl
e
m
e
nt
t
he
m
ode
l
us
in
g a
m
ul
ti
d
om
a
in
da
ta
s
e
t.
0
20
40
60
80
100
S
A
-
A
t
t
e
nt
i
on
G
R
U
S
A
-
A
t
t
e
nt
i
on
B
E
R
T
S
A
-
A
t
t
e
nt
i
on
R
oB
E
R
T
a
G
r
a
de
d M
ul
t
i
-
H
e
a
d A
t
t
e
nt
i
on
P
r
e
c
i
s
i
on
(
%
)
S
e
n
t
i
m
e
n
t
a
n
a
l
y
s
i
s
m
ode
l
0
20
40
60
80
100
S
A
-
A
t
t
e
nt
i
on
GR
U
S
A
-
A
t
t
e
nt
i
on
B
E
R
T
S
A
-
A
t
t
e
nt
i
on
R
oB
E
R
T
a
G
r
a
de
d M
ul
t
i
-
H
e
a
d A
t
t
e
nt
i
on
83.3
86.6
90.4
96.53
A
c
c
u
r
a
c
y
(
%
)
S
e
n
t
i
m
e
n
t
a
n
a
l
y
s
i
s
m
ode
l
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
Se
nt
ime
nt
c
la
s
s
if
ic
at
io
n us
in
g g
r
adi
e
nt
m
odul
at
io
n and lay
e
r
e
d
…
(
B
agi
y
al
ak
s
hm
i
N
at
ar
aj
an
)
5199
F
U
N
D
I
N
G
I
N
F
O
R
M
A
T
I
O
N
T
he
a
ut
hor
s
c
onf
ir
m
th
a
t
no
f
undi
ng,
m
one
y,
or
ot
he
r
a
s
s
is
ta
nc
e
w
a
s
obt
a
in
e
d
in
or
de
r
to
pr
e
pa
r
e
th
is
w
or
k.
A
U
T
H
O
R
C
O
N
T
R
I
B
U
T
I
O
N
S
S
T
A
T
E
M
E
N
T
T
hi
s
jo
ur
na
l
us
e
s
th
e
C
ont
r
ib
ut
or
R
ol
e
s
T
a
xonomy
(
C
R
e
di
T
)
to
r
e
c
ogni
z
e
in
di
vi
dua
l
a
ut
hor
c
ont
r
ib
ut
io
ns
, r
e
duc
e
a
ut
hor
s
hi
p di
s
put
e
s
,
a
nd f
a
c
il
it
a
te
c
ol
la
bo
r
a
ti
on.
N
am
e
o
f
A
u
t
h
or
C
M
So
Va
Fo
I
R
D
O
E
Vi
Su
P
Fu
B
a
g
iy
a
l
a
k
s
hm
i
N
a
t
a
r
a
j
a
n
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
T
. V
e
e
r
a
m
a
ka
li
✓
✓
✓
✓
✓
✓
✓
C
:
C
onc
e
pt
ua
l
i
z
a
t
i
on
M
:
M
e
t
hodol
ogy
So
:
So
f
t
w
a
r
e
Va
:
Va
l
i
da
t
i
on
Fo
:
Fo
r
m
a
l
a
na
l
ys
i
s
I
:
I
nve
s
t
i
ga
t
i
on
R
:
R
e
s
our
c
e
s
D
:
D
a
t
a
C
ur
a
t
i
on
O
:
W
r
i
t
i
ng
-
O
r
i
gi
na
l
D
r
a
f
t
E
:
W
r
i
t
i
ng
-
R
e
vi
e
w
&
E
di
t
i
ng
Vi
:
Vi
s
ua
l
i
z
a
t
i
on
Su
:
Su
pe
r
vi
s
i
on
P
:
P
r
oj
e
c
t
a
dm
i
ni
s
t
r
a
t
i
on
Fu
:
Fu
ndi
ng a
c
qui
s
i
t
i
on
C
O
N
F
L
I
C
T
O
F
I
N
T
E
R
E
S
T
S
T
A
T
E
M
E
N
T
A
ut
hor
s
s
ta
te
no c
onf
li
c
t
of
i
nt
e
r
e
s
t.
D
A
T
A
A
V
A
I
L
A
B
I
L
I
T
Y
D
a
ta
a
va
il
a
bi
li
ty
is
not
a
ppl
ic
a
bl
e
to
th
is
pa
pe
r
a
s
no
ne
w
d
a
ta
w
e
r
e
c
r
e
a
te
d
or
a
na
ly
z
e
d
in
th
is
s
tu
dy.
R
E
F
E
R
E
N
C
E
S
[
1]
Y
.
M
a
o,
Q
.
L
i
u,
a
nd
Y
.
Z
ha
ng,
“
S
e
nt
i
m
e
nt
a
na
l
ys
i
s
m
e
t
hods
,
a
ppl
i
c
a
t
i
ons
,
a
nd
c
ha
l
l
e
nge
s
:
a
s
ys
t
e
m
a
t
i
c
l
i
t
e
r
a
t
ur
e
r
e
vi
e
w
,”
J
our
nal
of
K
i
ng
Saud
U
ni
v
e
r
s
i
t
y
-
C
om
put
e
r
and
I
nf
or
m
at
i
on
S
c
i
e
nc
e
s
,
vol
.
36,
no.
4,
A
pr
.
2024,
doi
:
10.1016/
j
.j
ks
uc
i
.2024.102048.
[
2]
J
.
Y
.
-
L
.
C
ha
n,
K
.
T
.
B
e
a
,
S
.
M
.
H
.
L
e
ow
,
S
.
W
. P
hoong,
a
nd
W
.
K
.
C
he
ng,
“
S
t
a
t
e
of
t
he
a
r
t
:
a
r
e
vi
e
w
o
f
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
ba
s
e
d
on
s
e
que
nt
i
a
l
t
r
a
ns
f
e
r
l
e
a
r
ni
ng,”
A
r
t
i
f
i
c
i
al
I
nt
e
l
l
i
ge
nc
e
R
e
v
i
e
w
,
vol
.
56,
no.
1,
pp.
749
–
780,
A
pr
.
2022,
doi
:
10.1007/
s
10462
-
022
-
10183
-
8.
[
3]
A
.
D
a
z
a
,
N
.
D
.
G
.
R
ue
da
,
M
.
S
.
A
.
S
á
nc
h
e
z
,
W
.
F
.
R
.
E
s
pí
r
i
t
u,
a
nd
M
.
E
.
C
.
Q
ui
ñone
s
,
“
S
e
nt
i
m
e
nt
a
na
l
ys
i
s
on
e
-
c
om
m
e
r
c
e
pr
oduc
t
r
e
vi
e
w
s
us
i
ng
m
a
c
hi
ne
l
e
a
r
ni
ng
a
nd
de
e
p
l
e
a
r
ni
ng
a
l
gor
i
t
hm
s
:
a
bi
bl
i
om
e
t
r
i
c
a
na
l
ys
i
s
,
s
ys
t
e
m
a
t
i
c
l
i
t
e
r
a
t
ur
e
r
e
vi
e
w
,
c
ha
l
l
e
nge
s
a
nd
f
ut
ur
e
w
or
ks
,”
I
nt
e
r
nat
i
onal
J
our
nal
of
I
nf
or
m
at
i
on
M
anage
m
e
nt
D
at
a
I
ns
i
ght
s
,
vol
.
4,
no.
2,
N
ov.
2024,
doi
:
10.1016/
j
.j
j
i
m
e
i
.2024.100267.
[
4]
M
.
E
.
M
ow
l
a
e
i
,
M
.
S
.
A
ba
de
h,
a
nd
H
.
K
e
s
ha
va
r
z
,
“
A
s
pe
c
t
-
ba
s
e
d
s
e
nt
i
m
e
n
t
a
na
l
ys
i
s
u
s
i
ng
a
da
pt
i
ve
a
s
pe
c
t
-
ba
s
e
d
l
e
xi
c
on
s
,”
E
x
pe
r
t
Sy
s
t
e
m
s
w
i
t
h A
ppl
i
c
at
i
ons
, vol
. 148, J
un. 2020, doi
:
10.1016/
j
.e
s
w
a
.2020.113234.
[
5]
N
.
S
a
r
a
s
w
a
t
hi
,
T
.
S
.
R
ooba
,
a
nd
S
.
C
ha
ka
r
a
va
r
t
hi
,
“
I
m
pr
ovi
ng
t
he
a
c
c
ur
a
c
y
of
s
e
nt
i
m
e
nt
a
na
l
y
s
i
s
us
i
ng
a
l
i
ngui
s
t
i
c
r
ul
e
-
ba
s
e
d
f
e
a
t
ur
e
s
e
l
e
c
t
i
on m
e
t
hod i
n t
our
i
s
m
r
e
vi
e
w
s
,”
M
e
as
u
r
e
m
e
nt
:
Se
ns
o
r
s
, vol
. 29,
O
c
t
. 2023, doi
:
10.1016/
j
.m
e
a
s
e
n.2023.100888.
[
6]
M
.
Z
.
M
e
kone
n
e
t
al
.
,
“
A
n
opi
ni
ona
t
e
d
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
us
i
ng
a
r
ul
e
-
ba
s
e
d
m
e
t
hod,”
B
ul
l
e
t
i
n
of
E
l
e
c
t
r
i
c
al
E
ngi
ne
e
r
i
ng
and
I
nf
or
m
at
i
c
s
, vol
. 14, no. 1, pp. 726
–
732, F
e
b. 2025, doi
:
10.11591/
e
e
i
.v14i
1.8568.
[
7]
K
.
L
.
T
a
n,
C
.
P
.
L
e
e
,
K
.
S
.
M
.
A
nba
na
nt
he
n,
a
nd
K
.
M
.
L
i
m
,
“
R
oB
E
R
T
a
-
L
S
T
M
:
a
hybr
i
d
m
ode
l
f
or
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
w
i
t
h
t
r
a
ns
f
or
m
e
r
a
nd r
e
c
ur
r
e
nt
ne
ur
a
l
ne
t
w
or
k,”
I
E
E
E
A
c
c
e
s
s
, vol
. 10, pp. 21517
–
21525, 2022, doi
:
10.1109/
a
c
c
e
s
s
.2022.3152828.
[
8]
C
.
G
upt
a
,
G
.
C
h
a
w
l
a
,
K
.
R
a
w
l
l
e
y,
K
.
B
i
s
ht
,
a
nd
M
.
S
ha
r
m
a
,
“
S
e
nt
i
_A
L
S
T
M
:
S
e
nt
i
m
e
nt
a
na
l
ys
i
s
of
m
ovi
e
r
e
vi
e
w
s
us
i
n
g
a
t
t
e
nt
i
on
-
ba
s
e
d
L
S
T
M
,”
i
n
P
r
oc
e
e
di
ngs
of
3r
d
I
nt
e
r
nat
i
onal
C
onf
e
r
e
nc
e
on
C
om
put
i
ng
I
nf
or
m
at
i
c
s
and
N
e
t
w
or
k
s
,
2021,
pp. 211
–
219
,
doi
:
10.1007/
978
-
981
-
15
-
9712
-
1_18.
[
9]
O
.
N
da
m
a
,
I
.
B
e
ns
a
s
s
i
,
a
nd
E
.
M
.
E
n
-
N
a
i
m
i
,
“
T
he
i
m
pa
c
t
of
B
E
R
T
-
i
nf
us
e
d
de
e
p
l
e
a
r
ni
ng
m
ode
l
s
on
s
e
nt
i
m
e
nt
a
na
l
y
s
i
s
a
c
c
ur
a
c
y
i
n
f
i
na
nc
i
a
l
ne
w
s
,
”
B
ul
l
e
t
i
n
of
E
l
e
c
t
r
i
c
al
E
ngi
ne
e
r
i
ng
and
I
nf
or
m
at
i
c
s
,
vol
.
14,
no.
2,
pp.
1231
–
1240,
A
pr
.
2025
,
doi
:
10.11591/
e
e
i
.v14i
2.8469.
[
10]
G
. Z
ha
o, Y
.
L
uo, Q
.
C
he
n,
a
nd
X
. Q
i
a
n,
“
A
s
p
e
c
t
-
ba
s
e
d
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
vi
a
m
ul
t
i
t
a
s
k l
e
a
r
ni
ng f
or
onl
i
ne
r
e
vi
e
w
s
,”
K
now
l
e
dge
-
B
as
e
d Sy
s
t
e
m
s
, vol
. 264, M
a
r
. 2023, doi
:
10.1016/
j
.knos
ys
.2023.110326.
[
11]
Y
.
H
ua
ng,
X
.
B
a
i
,
Q
.
L
i
u,
H
.
P
e
ng,
Q
.
Y
a
ng,
a
nd
J
.
W
a
ng,
“
S
e
nt
e
nc
e
-
l
e
ve
l
s
e
nt
i
m
e
nt
c
l
a
s
s
i
f
i
c
a
t
i
on
ba
s
e
d
on
m
ul
t
i
-
a
t
t
e
nt
i
on
bi
di
r
e
c
t
i
ona
l
ga
t
e
d s
pi
ki
ng ne
ur
a
l
P
s
ys
t
e
m
s
,
”
A
ppl
i
e
d Sof
t
C
om
put
i
ng
, vol
. 152, F
e
b. 2024, doi
:
10.1016/
j
.a
s
oc
.2024.111231.
[
12]
C
. G
a
n, X
.
F
u, Q
. F
e
ng, Q
.
Z
hu, Y
. C
a
o, a
nd Y
.
Z
hu, “
A
m
ul
t
i
m
oda
l
f
us
i
on ne
t
w
or
k w
i
t
h a
t
t
e
nt
i
on m
e
c
ha
ni
s
m
s
f
or
vi
s
ua
l
–
t
e
xt
ua
l
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
,”
E
x
pe
r
t
Sy
s
t
e
m
s
w
i
t
h A
ppl
i
c
at
i
ons
, vol
. 242, M
a
y 2024, doi
:
10.1016/
j
.e
s
w
a
.2023.122731.
[
13]
R
.
K
.
D
e
y
a
nd
A
.
K
.
D
a
s
,
“
N
e
i
ghbour
a
dj
us
t
e
d
di
s
pe
r
s
i
ve
f
l
i
e
s
opt
i
m
i
z
a
t
i
on
ba
s
e
d
de
e
p
hybr
i
d
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
f
r
a
m
e
w
or
k,”
M
ul
t
i
m
e
di
a T
ool
s
and A
ppl
i
c
at
i
ons
, vol
. 83, no. 24, pp. 64393
–
64416, J
a
n. 202
4, doi
:
10.1007/
s
11042
-
023
-
17953
-
8.
[
14]
F
.
W
a
ng,
Q
.
B
a
o,
Z
.
W
a
ng,
a
nd
Y
.
C
h
e
n,
“
O
pt
i
m
i
z
i
ng
t
r
a
ns
f
or
m
e
r
ba
s
e
d
on
hi
gh
-
pe
r
f
or
m
a
nc
e
opt
i
m
i
z
e
r
f
or
p
r
e
di
c
t
i
ng
e
m
pl
oym
e
nt
s
e
nt
i
m
e
nt
i
n
A
m
e
r
i
c
a
n
s
oc
i
a
l
m
e
di
a
c
ont
e
nt
,
”
i
n
2024
5t
h
I
nt
e
r
nat
i
onal
C
onf
e
r
e
nc
e
on
M
ac
hi
ne
L
e
a
r
ni
ng
and
C
om
put
e
r
A
ppl
i
c
at
i
on
, O
c
t
. 2024, pp. 414
–
418
, doi
:
10.1109/
i
c
m
l
c
a
63499.2024.10753783.
[
15]
J
.
Z
i
m
m
e
r
m
a
nn,
L
.
E
.
C
ha
m
pa
gne
,
J
.
M
.
D
i
c
ke
ns
,
a
nd
B
.
T
.
H
a
z
e
n,
“
A
ppr
oa
c
he
s
t
o
i
m
pr
ove
pr
e
pr
oc
e
s
s
i
ng
f
or
l
a
t
e
nt
D
i
r
i
c
hl
e
t
a
l
l
oc
a
t
i
on
t
opi
c
m
ode
l
i
ng,”
D
e
c
i
s
i
on Suppor
t
S
y
s
t
e
m
s
, vol
. 185, O
c
t
. 2024, doi
:
10.1016/
j
.ds
s
.2024.114310.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 6, D
e
c
e
m
be
r
2025
:
5193
-
5200
5200
[
16]
A
.
W
e
ndl
a
nd,
M
.
Z
e
ne
r
e
,
a
nd
J
.
N
i
e
m
a
nn,
“
I
nt
r
oduc
t
i
on
t
o
t
e
xt
c
l
a
s
s
i
f
i
c
a
t
i
on:
i
m
pa
c
t
of
s
t
e
m
m
i
ng
a
nd
c
om
pa
r
i
ng
T
F
-
I
D
F
a
nd
c
ount
ve
c
t
or
i
z
a
t
i
on
a
s
f
e
a
t
ur
e
e
xt
r
a
c
t
i
on
t
e
c
hni
que
,”
i
n
Sy
s
t
e
m
s
,
Sof
t
w
ar
e
and
Se
r
v
i
c
e
s
P
r
oc
e
s
s
I
m
pr
ov
e
m
e
nt
,
2021,
pp.
289
–
300
,
doi
:
10.1007/
978
-
3
-
030
-
85521
-
5_19.
[
17]
I
.
A
.
K
a
ndhr
o,
F
.
A
l
i
,
M
.
U
ddi
n,
A
.
K
e
ha
r
,
a
nd
S
.
M
a
ni
c
ka
m
,
“
E
xpl
or
i
ng
a
s
pe
c
t
-
ba
s
e
d
s
e
nt
i
m
e
nt
a
na
l
ys
i
s
:
a
n
i
n
-
de
pt
h
r
e
vi
e
w
of
c
ur
r
e
nt
m
e
t
hods
a
nd
pr
os
pe
c
t
s
f
or
a
dva
nc
e
m
e
nt
,”
K
now
l
e
dge
and
I
nf
or
m
at
i
on
Sy
s
t
e
m
s
,
vol
.
66,
no.
7,
pp.
3639
–
3669,
A
pr
.
2024,
doi
:
10.1007/
s
10115
-
024
-
02104
-
8.
[
18]
M
.
J
oj
oa
,
P
.
E
f
t
e
kha
r
,
B
.
N
.
-
K
i
a
,
a
nd
B
.
G
.
-
Z
a
pi
r
a
i
n,
“
N
a
t
ur
a
l
l
a
ngua
ge
pr
oc
e
s
s
i
ng
a
n
a
l
ys
i
s
a
ppl
i
e
d
t
o
C
O
V
I
D
-
19
ope
n
-
t
e
xt
opi
ni
ons
us
i
ng
a
D
i
s
t
i
l
B
E
R
T
m
ode
l
f
or
s
e
nt
i
m
e
nt
c
a
t
e
gor
i
z
a
t
i
on,”
A
I
&
SO
C
I
E
T
Y
,
vol
.
39,
no.
3,
pp.
883
–
890,
N
ov.
2022,
doi
:
10.1007/
s
00146
-
022
-
01594
-
w.
[
19]
A
.
A
r
e
s
he
y
a
nd
H
.
M
a
t
hkour
,
“
E
xpl
or
i
ng
t
r
a
ns
f
or
m
e
r
m
ode
l
s
f
or
s
e
nt
i
m
e
nt
c
l
a
s
s
i
f
i
c
a
t
i
on:
a
c
om
pa
r
i
s
on
of
B
E
R
T
,
R
oB
E
R
T
a
,
A
L
B
E
R
T
, D
i
s
t
i
l
B
E
R
T
, a
nd X
L
N
e
t
,”
E
x
pe
r
t
Sy
s
t
e
m
s
, vol
. 41, no. 11, A
ug. 202
4, doi
:
10.1111/
e
xs
y.13701.
[
20]
B
.
P
a
t
t
a
na
ya
k,
A
.
M
a
j
um
de
r
,
B
.
J
o
t
hi
,
a
nd
K
.
S
.,
“
T
e
xt
s
um
m
a
r
i
z
a
t
i
on
w
i
t
h
D
i
s
t
i
l
B
E
R
T
-
L
S
T
M
,”
i
n
2025
I
nt
e
r
nat
i
onal
C
onf
e
r
e
nc
e
on I
nt
e
l
l
i
ge
nt
Sy
s
t
e
m
s
and
C
om
put
at
i
onal
N
e
t
w
o
r
k
s
, J
a
n. 2025, pp.
1
–
8
, doi
:
10.1109/
i
c
i
s
c
n64258.2025.10934207.
[
21]
A
.
S
.
T
a
l
a
a
t
,
“
S
e
nt
i
m
e
nt
a
na
l
ys
i
s
c
l
a
s
s
i
f
i
c
a
t
i
on
s
ys
t
e
m
u
s
i
ng
hybr
i
d
B
E
R
T
m
ode
l
s
,”
J
our
nal
of
B
i
g
D
at
a
,
vol
.
10,
no.
1,
J
un.
2023, doi
:
10.1186/
s
40537
-
023
-
00781
-
w.
[
22]
V
.
V
a
j
r
obol
,
N
.
A
gga
r
w
a
l
,
U
.
S
hukl
a
,
G
.
J
.
S
a
xe
na
,
S
.
S
i
ngh,
a
nd
A
.
P
undi
r
,
“
E
xpl
a
i
na
bl
e
c
r
os
s
-
l
i
ngua
l
de
pr
e
s
s
i
on
i
de
nt
i
f
i
c
a
t
i
o
n
ba
s
e
d
on
m
ul
t
i
-
he
a
d
a
t
t
e
nt
i
on
ne
t
w
or
k
s
i
n
T
ha
i
c
ont
e
xt
,”
I
nt
e
r
nat
i
onal
J
o
ur
nal
of
I
nf
or
m
at
i
on
T
e
c
hnol
ogy
,
vol
.
17,
no.
5,
pp. 2997
–
3012, O
c
t
. 2023, doi
:
10.1007/
s
41870
-
023
-
01512
-
3.
[
23]
A
.
A
l
j
of
e
y,
S
.
A
.
B
e
l
l
o,
J
.
L
u,
a
nd
C
.
X
u,
“
B
E
R
T
-
P
hi
s
hF
i
nde
r
:
a
r
obus
t
m
ode
l
f
or
a
c
c
ur
a
t
e
phi
s
hi
ng
U
R
L
de
t
e
c
t
i
on
w
i
t
h
opt
i
m
i
z
e
d
D
i
s
t
i
l
B
E
R
T
,”
I
E
E
E
T
r
ans
ac
t
i
ons
on
D
e
pe
ndabl
e
and
Se
c
u
r
e
C
om
put
i
ng
,
vol
.
22,
no.
4,
pp.
4315
–
4329,
J
ul
.
2025,
doi
:
10.1109/
t
ds
c
.2025.3545771.
[
24]
H
.
A
de
l
e
t
al
.
,
“
I
m
pr
ovi
ng
c
r
i
s
i
s
e
ve
nt
s
de
t
e
c
t
i
on
us
i
ng
D
i
s
t
i
l
B
E
R
T
w
i
t
h
H
ung
e
r
G
a
m
e
s
s
e
a
r
c
h
a
l
gor
i
t
hm
,”
M
at
he
m
at
i
c
s
,
vol
.
10,
no. 3, J
a
n. 2022, doi
:
10.3390/
m
a
t
h10030447.
[
25]
A
.
R
.
N
a
i
r
,
R
.
P
.
S
i
ngh,
D
.
G
upt
a
,
a
nd
P
.
K
um
a
r
,
“
E
va
l
ua
t
i
ng
t
he
i
m
pa
c
t
of
t
e
xt
da
t
a
a
ugm
e
nt
a
t
i
on
on
t
e
xt
c
l
a
s
s
i
f
i
c
a
t
i
on
t
a
s
k
s
us
i
ng D
i
s
t
i
l
B
E
R
T
,”
P
r
oc
e
di
a C
om
put
e
r
Sc
i
e
nc
e
, vol
. 235, pp. 102
–
111, 2024,
doi
:
10.1016/
j
.pr
oc
s
.2024.04.013.
B
I
O
G
R
A
P
H
I
E
S
O
F
A
U
T
H
O
R
S
Bagiyalakshmi
Nataraj
an
received
her
Bachelor’s
degree
in
Information
Technology
from
Adhiparasakthi
Engineering
College.
She
subseque
ntly
obtained
her
m
aster’s
degree
in
Information
Technology.
She
served
as
an
Assistant
Profe
ssor
in
the
Department
of
Computer
Science
and
Enginee
ring.
She
is
curre
ntly
pursuing
full
-
t
ime
research
at
the
SRM
Institut
e
of
Science
and
Technology.
Her
research
interests
include
nat
ural
language
processing,
text summariz
ation, and d
eep lea
rning
.
She co
ntacte
d at e
mail:
bn7569@
srmist.edu.in
.
T.
Veeramakali
working
as
an
Associate
Professor
in
the
Dep
artment
of
Data
Scienc
e
and
Business
Systems,
School
of
Computing
at
SRM
Institute
of
Scienc
e
an
d
Technology.
She
graduated
in
Information
Technology
in
2003
at
Sri
Siva
Subramaniya
Nada
r
(SSN) Col
lege of Engi
neering, Chen
nai, Tamil
nadu, India.
She secure
d Master of T
echnology i
n
Information
Technology
in
2007
at
Sathyabama
University
,
Chennai,
India.
She
completed
he
r
Ph.D.
in
the
field
of
Cognitive
Radio
Network
at
Anna
University
in
2018,
Chennai,
India.
She
is
in
teaching
profession
for
more
than
20
ye
ars.
She
has
publ
ished
m
any
papers
in
SCI/Scopus
indexed
journals
and
presented
number
of
papers
in
national
and
international
conference
.
Her
main
area
of
interest
includes
netwo
rks,
machine
learning,
image
p
rocessing
,
and
internet
of
things
.
She
is
a
life
time
member
of
the
professional
bodies
such
as
CSE,
ISTE
,
and
IETE.
She
contacted
at email
:
veeramat@srmist
.edu.in.
Evaluation Warning : The document was created with Spire.PDF for Python.