Inter
national
J
our
nal
of
Inf
ormatics
and
Communication
T
echnology
(IJ-ICT)
V
ol.
15,
No.
4,
December
2026,
pp.
2011
∼
2024
ISSN:
2252-8776,
DOI:
10.11591/ijict.v15i4.pp2011-2024
❒
2011
Ev
aluating
usability
,
accessibility
,
and
persuasion
in
lear
ning
management
systems
Samantha
Jim
´
enez
1,2
,
F
er
nanda
Murillo-Mu
˜
noz
2
,
Leticia
Cer
v
antes
1
,
Norma
Candol
1
,
Ber
enice
F
ong
1
,
Camilo
Cara
v
eo
1
,
Alan
Ram
´
ır
ez-Noriega
3
1
F
acultad
de
Ciencias
de
la
Ingenier
´
ıa
y
T
ecnolog
´
ıa,
Uni
v
ersidad
Aut
´
onoma
de
Baja
California,
Blvd.
Uni
v
ersitario,
T
ijuana,
M
´
exico
2
Full
Stack
Program,
San
Die
go
Global
Kno
wledge
Uni
v
ersity
,
San
Die
go,
US
3
F
acultad
de
Ingenier
´
ıa
Mochis,
Uni
v
ersidad
Aut
´
onoma
de
Sinaloa,
Sinaloa,
M
´
exico
Article
Inf
o
Article
history:
Recei
v
ed
Dec
20,
2025
Re
vised
Aug
6,
2026
Accepted
Aug
28,
2026
K
eyw
ords:
Accessibility
LMS
Persuasion
SUS
Usability
ABSTRA
CT
Learning
management
systems
(LMSs)
are
widely
adopted
in
engineering
ed-
ucation
to
support
content
deli
v
ery
,
foster
collaboration,
and
de
v
elop
digital
competencies.
Ho
we
v
er
,
e
xist
ing
platforms
often
prioritize
functionality
o
v
er
user
eng
agement
and
acces
sibility
.
There
is
limited
research
that
holistically
e
v
aluates
LMS
usability
,
persuasi
v
e
design,
and
accessibility
in
the
conte
xt
of
technical
education,
particularly
during
high-frequenc
y
tasks
such
as
assignment
submission.
This
study
aims
to
e
v
aluate
and
compare
the
usability
,
persuasi
v
e
elements,
and
accessibility
of
blackboard,
can
v
as,
and
Google
Classroom
during
the
assignment
submission
process
from
the
perspecti
v
e
of
engineering
students.
An
e
xperimental,
between-subjects
desi
gn
w
as
emplo
yed
with
engineering
stu-
dents
who
each
interacted
with
one
of
the
three
LMS
platforms.
W
e
e
v
alu-
ated
usability
,
persuasi
v
eness
and
accessibility
.
Can
v
as
w
as
the
most
ef
cient
platform,
with
shorter
task
completion
times
and
fe
wer
errors.
Ho
we
v
er
,
it
re-
cei
v
ed
lo
wer
scores
in
satisf
action
and
persuasi
v
e
design
compared
to
Google
Classroom,
which
stood
out
in
credibilit
y
and
user
guidance
b
ut
sho
wed
more
v
ariability
in
ef
cienc
y
.
None
of
the
platforms
fully
complied
with
k
e
y
acces-
sibility
cri
teria,
such
as
k
e
yboard
operability
and
proper
form
labeling.
These
ndings
highlight
the
importance
of
designing
LMS
platforms
that
balance
us-
ability
,
persuasion,
and
accessibility
to
better
support
eng
agement
and
equity
in
engineering
education.
This
is
an
open
access
article
under
the
CC
BY
-SA
license
.
Corresponding
A
uthor:
Samantha
Jim
´
enez
F
acultad
de
Ciencias
de
la
Ingenier
´
ıa
y
T
ecnolog
´
ıa,
Uni
v
ersidad
Aut
´
onoma
de
Baja
California
Blvd.
Uni
v
ersitario
Baja
California,
M
´
exico
Email:
samantha.jimenez@uabc.edu.mx
1.
INTR
ODUCTION
Learning
management
systems
(LMS)
ha
v
e
become
essential
tools
in
modern
engineering
education,
pro
viding
digital
en
vironments
to
deli
v
er
academic
content,
manage
student
acti
vities,
and
foster
interactions.
These
platforms
grant
students
access
to
technical
resources
such
as
videos,
tutorials,
databases,
and
academic
articles,
while
supporting
synchronous
and
asynchronous
collaboration
[1],
[2].
Be
yond
content
deli
v
ery
,
LMS
platforms
play
a
crucial
role
in
the
de
v
elopment
of
digital
competencie
s
required
by
engineering
students,
including
problem-solving,
communication,
and
collaborati
v
e
skills
[3],
[4].
J
ournal
homepage:
http://ijict.iaescor
e
.com
Evaluation Warning : The document was created with Spire.PDF for Python.
2012
❒
ISSN:
2252-8776
Their
inte
gration
into
higher
education
has
accelerated,
particularly
follo
wing
the
global
shift
to
remote
and
h
ybrid
learning
models.
While
LMS
adoption
continues
to
gro
w
,
the
success
of
these
platforms
depends
hea
vily
on
their
us-
ability
and
capacity
to
eng
age
and
moti
v
ate
students.
Usability—dened
through
met
rics
such
as
ease
of
use,
ef
cienc
y
,
and
user
satisf
action—has
been
widely
e
v
aluated
using
tools
lik
e
the
system
usability
scale
(SUS)
and
the
User
e
xperience
questionnaire
(UEQ-S)
[5],
[6].
Ho
we
v
er
,
recent
studies
emphasize
that
be
yond
usability
,
the
persuasi
v
e
capabilities
of
LMS
plat-
forms—their
ability
to
positi
v
ely
inuence
student
beha
vior
,
moti
v
ation,
and
eng
agement—are
equally
vital
for
enhancing
the
learning
e
xperience
[7],
[8].
Despite
this,
empirical
e
v
aluations
of
persuasi
v
e
strate
gies
in
LMSs
remain
limited,
particularly
in
engineering
education.
Existing
research
has
typically
focused
on
isolated
platforms
or
single
e
v
aluation
dimensions.
F
or
instance,
comparati
v
e
usability
studies
of
blackboard,
Can
v
as,
and
Google
Classroom
ha
v
e
highlighted
mix
ed
results
in
terms
of
ef
cienc
y
,
user
satisf
action,
and
technical
features
[9],
[10].
Moreo
v
er
,
while
some
studies
apply
heuristic
frame
w
orks
such
as
Nielsen’
s
usability
princ
iples
[10],
the
inte
gration
of
persuasion
theory
,
such
as
the
persuasi
v
e
systems
design
(PSD)
model
[8],
is
rarely
e
xplored
in
parallel.
T
o
address
this
g
ap,
the
present
study
conducts
a
comparati
v
e
e
v
aluation
of
three
widely
used
LMS
platforms:
blackboard,
can
v
as,
and
Google
Classroom.
The
e
v
aluation
focuses
on
a
core
academic
task—
assignment
submission—and
simultaneously
analyzes
usability
(ef
cienc
y
,
ef
fe
cti
v
eness,
satisf
action)
and
per
-
suasi
v
e
design,
based
on
established
heuristics
and
the
PSD
model.
Furthermore,
accessibility
considerat
ions
were
included
through
a
checklist
deri
v
ed
from
WCA
G
2.1,
ackno
wledging
the
importance
of
equitable
and
inclusi
v
e
design
in
educational
technology
.
The
ndings
of
thi
s
study
aim
to
pro
vide
actionable
insights
into
ho
w
LMS
platforms
can
be
opti-
mized
to
support
both
task
performance
and
student
eng
agement
in
engineering
programs.
Statistical
analyses,
including
the
Kruskal-W
allis
H
test,
were
applied
to
assess
the
signicance
of
observ
ed
dif
ferences
[11].
Unlik
e
pre
vious
studies
that
focus
primarily
on
usability
or
user
satisf
action,
this
w
ork
inte
grates
three
complementary
perspecti
v
es:
usability
,
accessibility
,
and
persuasi
v
e
design.
By
e
v
aluating
blackboard,
can
v
as,
and
Google
Classroom
through
the
same
academic
task,
the
study
pro
vides
a
comparati
v
e
vie
w
of
ho
w
LMS
platforms
support
engineering
students
in
completing
routine
b
ut
critical
learning
acti
vities.
The
ndings
aim
to
inform
the
design
and
selection
of
LMS
platforms
that
are
not
only
ef
cient,
b
ut
also
eng
aging,
accessible,
and
supporti
v
e
of
equitable
digital
learning
e
xperiences.
Assignment
submission
w
as
selected
as
the
e
v
aluation
task
because
it
represents
one
of
the
most
frequent
and
essential
interactions
between
students
and
LMS
platforms.
This
task
requires
students
to
na
vig
ate
course
content,
locate
an
acti
vity
,
upload
a
le,
conrm
submission,
and
interpret
system
feedback.
Therefore,
it
pro
vides
a
suitable
scenario
for
e
xamining
usability
,
accessibility
,
and
persuasi
v
e
design
features
within
a
realistic
academic
w
orko
w
.
The
remainder
of
this
paper
is
structured
as
foll
o
ws
.
Section
2
re
vie
ws
the
related
w
ork
on
LMS
usability
,
persuasi
v
e
design,
and
accessibility
.
Section
3
describes
the
research
methodology
,
including
details
of
participants,
instruments,
and
e
v
aluation
process.
Section
4
presents
the
results
of
the
empirical
and
heuris-
tic
e
v
aluation,
or
g
anized
by
usability
,
accessibility
,
and
persuasion
dimensions.
Section
5
discusses
the
k
e
y
ndings
in
the
conte
xt
of
engineering
education.
Finally
,
section
6
pro
vides
the
conclusions,
implications,
and
directions
for
future
research.
2.
RELA
TED
W
ORK
LMS
are
digital
platforms
designed
to
support,
manage,
and
enhance
the
teaching
and
learning
pro-
cess.
The
y
pro
vide
online
spaces
for
accessing
academic
materials,
or
g
anizing
acti
vities,
and
f
acilitating
inter
-
action
between
students
and
instructors
[1],
[5],
[12].
The
inte
gration
of
LMS
platforms
in
higher
education
is
especially
rele
v
ant
in
engineering
programs,
where
the
y
contrib
ute
to
the
de
v
elopment
of
technical
competen-
cies,
problem-solving
skills,
and
collaborati
v
e
learning
abilities
[3],
[4],
[13].
The
success
of
LMS
platforms
depends
signicantly
on
their
usability
,
commonly
e
v
aluated
through
indicators
such
as
ef
cienc
y
,
ef
fecti
v
eness,
and
user
satisf
action.
T
ools
lik
e
t
he
SUS
and
the
UEQ-S
are
fre-
quently
used
to
assess
these
dimensions
[5],
[14]-[16].
V
arious
studies
ha
v
e
e
xplored
the
usability
of
LMSs
in
dif
ferent
educational
conte
xts.
F
or
instance,
[17]
and
[18]
assessed
Google
Classroom’
s
usability
,
report-
ing
positi
v
e
student
perceptions
re
g
arding
ease
of
use.
Comparati
v
e
studies
ha
v
e
analyzed
platforms
such
as
Int
J
Inf
&
Commun
T
echnol,
V
ol.
15,
No.
4,
December
2026:
2011–2024
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Inf
&
Commun
T
echnol
ISSN:
2252-8776
❒
2013
blackboard,
Desire2Learn,
Microsoft
T
eams,
and
Google
Classroom,
highlighting
dif
ferences
in
satisf
action,
ef
cienc
y
,
and
error
rates
[16],
[19].
Although
pre
vious
c
omparati
v
e
studies
ha
v
e
e
v
aluated
multiple
LMS
platforms
[20],
most
ha
v
e
fo-
cused
e
xclusi
v
ely
on
usability
metrics
such
as
satisf
action
or
percei
v
ed
ease
of
use.
Consequently
,
other
di-
mensions
that
inuence
students’
interaction,
including
accessibility
and
persuasi
v
e
design,
remain
lar
gely
une
xplored
[10].
Be
yond
usability
,
the
persuasi
v
e
design
of
LMS
platforms
plays
a
crucial
role
in
fostering
student
moti
v
ation,
eng
agement,
and
beha
vioral
change.
Persuasi
v
e
educational
systems
are
interacti
v
e
en
vironments
that
incorporate
strate
gies
to
support
users
in
forming,
modifying,
or
reinforcing
learning-related
attitudes
and
beha
viors
[21],
[22].
Ho
we
v
er
,
most
of
these
studies
e
v
aluate
persuasi
v
e
mechanisms
in
isolation
rather
than
together
with
usability
and
accessibility
.
As
a
result,
little
is
kno
wn
about
ho
w
these
dimensions
complement
one
another
when
students
perform
authentic
academic
tasks
within
LMS
en
vironments
[23].
The
PSD
model
[8]
is
widely
used
to
analyze
and
structure
persuasi
v
e
strate
gies.
This
model
cate
go-
rizes
strate
gies
into
four
groups:
primary
task
support,
dialogue
support,
system
credibility
,
and
social
support.
Research
applying
the
PSD
model
to
LMS
e
v
aluations
has
highlighted
that
while
systems
often
perform
well
in
credibility-related
aspects,
social
support
strate
gies
are
frequently
underutilized
[24].
In
addition
to
usability
and
persuasi
v
e
design,
LMS
platforms
dif
fer
in
the
tools
and
functionalit
ies
the
y
of
fer
for
academic
management
and
student
interaction.
T
able
1
summarizes
k
e
y
features
of
blackboard
learn,
can
v
as,
and
Google
Classroom,
which
may
inuence
the
user
e
xperience.
T
able
1.
Comparison
of
core
features
in
three
LMS
platforms
Feature
Blackboard
learn
Can
v
as
LMS
Google
Classroom
Course
creation
and
editing
Y
es
Y
es
Y
es
Assignment
submission
Y
es
(multiple
formats)
Y
es
(multiple
formats)
Y
es
Grade
center
and
feedback
Y
es
Y
es
Y
es
Calendar
and
scheduling
Y
es
Y
es
Y
es
Inte
grated
communication
tools
Announcements,
messages
Announcements,
discussions
Ne
ws
Synchronous
video
inte
gration
Y
es
Y
es
Y
es
Discussion
or
collaboration
spaces
F
orums
Groups
–
Course
content
or
g
anization
Modules,
pages
Modules,
folders
Themes
Although
usability
,
accessibility
,
and
persuasi
v
e
design
ha
v
e
each
been
in
v
estig
ated
in
pre
vious
LM
S
research,
the
y
ha
v
e
generally
been
studied
as
independent
dimensions.
Most
usability
studies
focus
on
task
performance
and
user
satisf
action,
acces
sibility
e
v
aluations
concentrate
on
compliance
with
standards
such
as
WCA
G,
while
persuasi
v
e
design
research
emphasizes
moti
v
a
tion
and
eng
agement.
Consequently
,
there
is
limited
understanding
of
ho
w
these
complementary
dimensions
interact
when
students
perform
authentic
academic
tasks
within
LMS.
Despite
the
gro
wing
body
of
research
on
LMS,
se
v
eral
limitations
remain.
First,
most
c
o
m
parati
v
e
studies
focus
primarily
on
traditional
usability
metrics,
such
as
ef
cienc
y
,
ef
fecti
v
eness,
or
user
satisf
action,
while
o
v
erlooking
complementary
dimensions
such
as
accessibility
and
persuasi
v
e
design.
Second,
pre
vious
in
v
estig
ations
frequently
e
v
aluate
these
aspects
independently
,
making
it
dif
cult
to
understand
ho
w
the
y
jointly
inuence
students’
interaction
with
LMS
platforms.
Third,
most
studies
assess
general
LMS
usage
or
rely
e
xclusi
v
ely
on
perception-based
questionnaires,
with
relati
v
ely
fe
w
emplo
ying
task-based
e
v
aluations
centered
on
authentic
academic
acti
vities.
Finally
,
comparati
v
e
e
v
aluations
conducted
specically
with
engineering
students
remain
scarce,
despite
the
intensi
v
e
use
of
LMS
platforms
in
engineering
education.
These
limitations
highlight
the
need
for
a
more
comprehensi
v
e
e
v
aluation
frame
w
ork
inte
grating
usability
,
accessibility
,
and
persuasi
v
e
design
within
a
realistic
educational
conte
xt
[25].
This
study
contrib
utes
to
closing
this
g
ap
by
conducting
a
comparati
v
e
e
v
aluation
of
three
widely
used
LMS
platforms:
blackboard
learn,
can
v
as,
and
Google
Classroom.
The
analysis
focuses
on
a
critical
aca-
demic
task—assignment
submiss
ion—and
considers
usability
,
persua
si
v
e
design,
and
accessibili
ty
,
pro
viding
actionable
insights
for
enhancing
LMS
ef
fecti
v
eness
in
engineering
programs.
Evaluating
usability
,
accessibility
,
and
per
suasion
in
learning
mana
g
ement
systems
(Samantha
Jim
´
enez)
Evaluation Warning : The document was created with Spire.PDF for Python.
2014
❒
ISSN:
2252-8776
3.
METHOD
This
section
presents
the
methodology
of
the
e
xperiment
conducted
with
users
to
e
v
aluate
the
LMSs.
W
e
perform
a
usability
analysis
that
e
v
aluates
ef
fecti
v
eness,
ef
cienc
y
,
and
satisf
action.
3.1.
Resear
ch
design
This
research
conducted
a
mix
ed-method
approach
because
it
combined
quantitati
v
e
measures
ob-
tained
from
task
performance
(completion
time,
errors,
assistance
requests,
and
SUS
scores)
with
qualitati
v
e
observ
ations
and
heuristic
e
v
aluations
of
persuasi
v
e
design
and
accessibility
.
This
approach
enabled
a
com-
prehensi
v
e
assessment
of
the
three
LMS
platforms
from
complementary
perspecti
v
es.
The
study
design
w
as
between
subjects,
where
dif
ferent
participants
e
v
aluated
each
platform
to
a
v
oid
bias
and
learning.
3.2.
Use
cases
denition
Assignment
submission
w
as
selected
because
it
represents
one
of
the
most
frequently
e
x
ecute
d
aca-
demic
tasks
across
virtually
all
LMS.
Unlik
e
quizzes,
forums,
or
grading
modules,
assignment
submission
is
a
core
functionality
a
v
ailable
in
blackboard,
can
v
as,
and
Google
Classroom.
This
task
requires
users
to
na
vig
ate
the
interf
ace,
locate
course
resources,
upload
les,
conrm
actions,
and
interpret
system
feedback,
making
it
an
appropriate
representati
v
e
task
for
e
v
aluating
usability
,
accessibility
,
and
persuasi
v
e
design.
The
T
able
2
sho
ws
the
description
of
the
use
cases.
The
generic
steps
and
the
specic
steps
for
each
platform
sho
w
that
the
process
is
similar
on
the
three
platforms.
T
able
2.
Use
case
comparison:
submitting
an
assignment
as
a
student
Step
Blackboard
learn
Can
v
as
LMS
Google
Classroom
Login
to
the
platform
Login
via
uni
v
ersity
portal
Login
to
can
v
as
Login
with
Google
account
Na
vig
ate
to
course
Go
to
courses
Select
from
dashboard
Select
course
Access
assignment
section
Assignments
or
content
Assignments
tab
Classw
ork
tab
Select
assignment
Choose
assignment
Select
assignment
Open
assignment
Attach
or
upload
le
Upload
from
de
vice
Use
start
assignment
and
upload
Add
or
create
and
attach
Add
optional
comment
Optional
te
xt
box
a
v
ailable
Comment
box
a
v
ailable
Optional
comment
to
teacher
Submit
assignment
Click
submit
Click
submit
assignment
Click
turn
in
V
ie
w
conrmation
Conrmation
message
Conrmation
with
timestamp
V
isual
conrmation
sho
wn
Figure
1
represents
a
UML
diagram
of
the
process
of
entering
the
LMS
and
conrming
the
submission
of
a
le.
F
or
the
description
of
the
process
in
Figure
1,
an
e
xpert
user
is
considered
the
prole
of
a
student
who
has
already
had
access
to
and
training
in
the
LMS.
Ho
we
v
er
,
response
times,
methodological
follo
w-up,
and
the
o
v
erall
analysis
of
the
LMS
comparison
change
based
on
this
kno
wledge
base.
Figure
1.
W
orko
w
of
the
assignment
submission
task
Int
J
Inf
&
Commun
T
echnol,
V
ol.
15,
No.
4,
December
2026:
2011–2024
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Inf
&
Commun
T
echnol
ISSN:
2252-8776
❒
2015
3.3.
P
articipants
A
total
of
17
v
olunteers
participated
in
this
study
,
the
study
uses
a
con
v
enience
sampling
t
echnique.
All
participants
wer
e
under
graduate
engineering
students
rec
ruited
from
a
public
uni
v
ersity
in
northwestern
Me
xico.
P
articipation
w
as
v
oluntary
,
and
informed
consent
w
as
obtained
from
all
participants
prior
to
the
e
xperiment.
The
study
follo
wed
a
between-subjects
design,
in
which
each
participant
e
v
aluated
only
one
LMS
platform
to
a
v
oid
learning
ef
fects
and
carry-o
v
er
bias
associated
with
repeated
e
xposure
to
multiple
systems.
The
participant
characteristics
for
the
three
LMS
groups
are
presented
in
T
able
3.
The
blackboard
group
consisted
of
participants
with
heterogeneous
backgrounds,
including
both
e
xperienced
and
rst-time
users,
as
well
as
v
arying
le
v
els
of
computer
procienc
y
.
In
contrast,
participants
assi
gned
to
can
v
as
reported
frequent
computer
use,
although
only
one
participant
had
pre
vious
e
xperience
with
the
platform.
P
articipants
e
v
aluating
Google
Classroom
also
reported
pre
vious
e
xperience
with
digital
learning
en
vironments,
although
none
had
e
xtensi
v
e
e
xperience
with
the
specic
assignment
submission
w
orko
w
used
during
the
e
xperiment.
T
able
3.
P
articipant
characteristics
Characteristic
Blackboard
Can
v
as
Classroom
P
articipants
(n)
7
5
5
Female
4
1
4
Male
3
4
1
Age
range
18–53
20–23
21–48
Pre
vious
e
xperience
with
LMS
Mix
ed
Limited–Moderate
Moderate
Computer
procienc
y
Mix
ed
High
High
*Approximate
range
based
on
participant
records.
Because
the
study
focused
e
xclusi
v
ely
on
engineering
students,
the
ndings
should
be
inter
p
r
eted
within
this
conte
xt.
Engineering
students
typically
possess
higher
le
v
els
of
digital
literac
y
t
han
the
general
uni
v
ersity
population,
which
may
inuence
task
performance
and
percei
v
ed
usability
.
Therefore,
the
results
should
not
be
generalized
to
students
from
other
academic
disciplines
without
further
empirical
v
alidation.
P
articipants
were
assigned
to
one
of
the
three
LMS
platforms
according
to
platform
a
v
ailability
while
ensuring
that
each
participant
interacted
with
only
one
platform
throughout
the
e
xperiment.
3.4.
Instruments
The
instruments
were
selected
to
pro
vide
a
multidimensional
e
v
aluation
of
the
same
assignment
sub-
mission
w
orko
w
.
Rather
than
assessing
usability
,
accessibility
,
and
persuasi
v
e
design
as
independent
system
attrib
utes,
the
study
inte
grated
objecti
v
e
performance
measures,
subjecti
v
e
user
perceptions,
and
e
xpert
in-
spection
within
a
common
methodological
frame
w
ork.
The
task
observ
ation
sheet
captured
ef
cienc
y
and
ef
fecti
v
eness
through
completi
on
time,
errors,
and
assistance
requests,
while
the
SUS
measured
participants’
percei
v
ed
usability
and
satisf
action.
The
persuasi
v
e
design
template
e
xamined
ho
w
students
percei
v
ed
the
presence
of
strate
gies
intended
to
guide,
moti
v
ate,
and
sustain
interaction,
whereas
the
WCA
G
2.1
checklist
assessed
whether
the
same
w
orko
w
supported
accessible
and
inclusi
v
e
interaction.
By
applying
these
instru-
ments
to
the
same
task
and
platforms,
the
methodology
enabled
triangulation
across
beha
vioral,
perceptual,
persuasi
v
e,
and
accessibility-related
e
vidence,
pro
viding
a
more
comprehensi
v
e
e
v
aluation
than
an
y
single
in-
strument
could
of
fer
.
3.4.1.
T
ask
obser
v
ation
sheet
A
structured
task
observ
ation
sheet
w
as
designed
to
record
participants’
performance
during
the
e
x-
perimental
task.
The
instrument
w
as
completed
by
tw
o
e
v
aluators
who
independently
observ
ed
each
participant
throughout
the
e
xperiment.
The
observ
ation
sheet
included
task
completion
time
for
each
attempt,
the
number
of
errors,
and
the
number
of
assistance
requests.
T
ask
completion
time
w
as
measured
in
seconds
from
the
m
oment
participants
initiated
the
assi
gnment
submission
process
until
the
conrmation
message
w
as
displayed.
Errors
were
dened
as
de
viations
from
the
e
xpected
task
o
w
(e.g.,
selecting
an
incorrect
menu
option
or
attempting
to
upload
an
incorrect
le).
Assistance
w
as
recorded
whene
v
er
participants
e
xplicitly
requested
help
or
clarication
from
the
e
v
aluators.
3.4.2.
System
usability
scale
User
satisf
action
w
as
measured
using
the
SUS,
a
standardized
questionnaire
composed
of
ten
item
s
rated
on
a
v
e-point
Lik
ert
scale.
SUS
pro
vides
a
global
measure
of
percei
v
ed
usability
ranging
from
0
to
100,
Evaluating
usability
,
accessibility
,
and
per
suasion
in
learning
mana
g
ement
systems
(Samantha
Jim
´
enez)
Evaluation Warning : The document was created with Spire.PDF for Python.
2016
❒
ISSN:
2252-8776
where
higher
scores
indicate
better
percei
v
ed
usability
.
The
questionnaire
w
as
administered
immediately
after
participants
completed
the
fth
task
repetition.
3.4.3.
P
ersuasi
v
e
design
e
v
aluation
The
persuasi
v
e
characteristics
of
each
LMS
were
e
v
aluated
using
the
heuristic
inspection
method
proposed
by
Jim
´
enez
et
al.
[26],
which
is
based
on
the
PSD
model.
The
i
nstrument
e
v
aluates
persuasi
v
e
strate
gies
grouped
into
four
cate
gories:
primary
task
support,
dialogue
support,
system
credibility
,
and
social
support.
Each
persuasi
v
e
strate
gy
w
as
rated
using
a
v
e-point
Lik
ert
scale
ranging
from
1
(strongly
disagree)
to
5
(strongly
agree),
according
to
the
e
x
t
ent
to
which
participants
percei
v
ed
the
strate
gy
to
be
implemented
in
the
interf
ace.
P
articipants
were
also
encouraged
to
pro
vide
qualitati
v
e
observ
ations
to
support
their
ratings.
3.4.4.
Accessibility
checklist
Accessibility
w
as
e
v
aluated
manually
using
a
checklist
deri
v
ed
from
eight
success
criteria
of
the
web
content
accessibility
guidelines
(WCA
G)
2.1.
The
selected
criteria
focused
on
the
assignment
submission
w
orko
w
and
co
v
ered
the
four
WCA
G
principles:
percei
v
able,
operable,
understandable,
and
rob
ust.
The
inspection
w
as
performed
by
tw
o
e
v
aluators,
who
independent
ly
v
eried
compliance
with
each
criterion
during
task
e
x
ecution.
The
e
v
aluation
focused
on
practical
accessibility
aspects
such
as
k
e
yboard
na
v-
ig
ation,
alternati
v
e
te
xt,
form
labels,
color
contrast,
focus
order
,
error
identication,
instructions,
and
semantic
denition
of
interf
ace
elements.
3.5.
Pr
ocedur
e
The
e
xperiment
w
as
conducted
in
person.
The
researchers
rst
e
xplained
the
task
the
part
icipants
needed
to
complete,
as
well
as
the
specic
platform
the
y
w
ould
be
using.
All
participants
performed
the
task
on
a
laptop
running
the
W
indo
ws
operating
system
and
a
stable
internet
service
connection,
equipped
with
an
e
xternal
mouse
and
k
e
yboard
to
ensure
a
more
comfortable
and
controlled
interaction
with
the
system.
Each
participant
completed
the
task
v
e
times.
The
completion
time
w
as
recorded
for
each
attempt,
and
the
a
v
erage
time
per
participant
w
as
calculated
for
analysis.
In
addition
to
task
timing,
the
researchers
documented
the
number
of
errors
and
instances
of
assista
n
c
e
required.
An
error
w
as
dened
as
an
y
de
viation
from
the
optimal
task
o
w
(e.g.,
clicking
an
incorrect
menu
option
or
uploading
the
wrong
le).
Assistance
w
as
recorded
whene
v
er
participants
requested
help.
T
w
o
e
v
aluators
observ
ed
each
participant
in
real
time
and
recorded
observ
ations
using
a
standardized
e
v
aluation
form.
T
o
maintain
consistenc
y
between
trials,
par
-
ticipants
submitted
the
same
mock
assignment
in
the
v
e
repetit
ions.
Upon
completing
the
nal
attempt,
participants
were
ask
ed
to
complete
the
SUS
via
a
digital
form
to
assess
their
general
satisf
action
with
the
platform.
3.6.
Data
analysis
Descripti
v
e
statistics
(mean
and
st
andard
de
viation)
were
calculated
for
all
quantitati
v
e
v
ariables.
Box-
plots
were
generated
to
visualize
the
distrib
ution
of
task
completion
times
across
platforms.
Since
the
sample
size
w
as
small
and
the
assumptions
of
normality
could
not
be
guaranteed,
dif
ferences
between
groups
were
analyzed
using
the
Kruskal–W
allis
H
test
with
a
signicance
le
v
el
of
p
=
0.05.
4.
RESUL
TS
This
section
presents
the
empirical
ndings
of
t
he
comparati
v
e
e
v
aluation
of
blackboard,
can
v
as,
and
Google
Classroom.
Results
are
or
g
anized
according
to
the
thre
e
e
v
aluation
dimensions
considered
in
this
study:
usability
,
accessibility
,
and
persuasi
v
e
design.
4.1.
Usability
e
v
aluation
The
analysis
is
structured
around
the
three
dimensions
of
usability
as
dened
in
ISO
9241-11:
ef
cienc
y
,
ef
fecti
v
eness,
and
satisf
action.
4.1.1.
Efciency
T
ask
ef
cienc
y
w
as
measured
as
the
a
v
erage
time
(in
seconds)
tak
en
to
complete
the
dened
use
case
in
v
e
repetitions
per
participant.
The
ndings
present
an
analysis
at
the
group
le
v
el
rather
than
per
indi
vidual
because
the
study
design
w
as
between
subjects.
Int
J
Inf
&
Commun
T
echnol,
V
ol.
15,
No.
4,
December
2026:
2011–2024
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Inf
&
Commun
T
echnol
ISSN:
2252-8776
❒
2017
Figure
2
sho
ws
a
box
plot
that
compares
completion
times
between
platforms.
P
articipants
using
can
v
as
achie
v
ed
the
f
astest
a
v
erage
completion
time
(M
=
29.24s,
SD
=
7.08),
follo
wed
by
those
using
black-
board
(M
=
41.36s,
SD
=
24.46).
The
classroom
group
sho
wed
the
slo
west
and
most
v
ariable
performance
(M
=
76.50s,
SD
=
57.14),
with
some
participants
taking
signicantly
longer
than
others.
Figure
2.
T
ask
completion
time
by
platform
(seconds)
T
ask
ef
cienc
y
w
as
measured
as
the
a
v
erage
time
(in
seconds)
required
to
complete
the
assignment
submission
task
across
v
e
repetitions.
Figure
2
compares
the
completion
times
obtained
for
the
three
e
v
aluated
LMSs.
P
articipants
using
can
v
as
obtained
the
lo
west
a
v
erage
completion
time
(M
=
29.24
s,
SD
=
7.08),
fol-
lo
wed
by
blackboard
(M
=
41.36
s,
SD
=
24.46).
Google
Classroom
presented
the
highest
a
v
erage
completion
time
(M
=
76.50
s,
SD
=
57.14),
with
greater
v
ariability
among
participants.
A
Kruskal-W
allis
test
w
as
performed
to
compare
completion
times
across
the
three
platforms.
The
analysis
yielded
H(2)=4.07
(p=0.131),
indicating
that
no
statistically
signicant
dif
ferences
were
found
among
the
e
v
aluated
LMS.
Because
the
study
follo
wed
a
between-s
u
bj
ects
design,
results
are
reported
at
the
group
le
v
el.
4.1.2.
Effecti
v
eness
The
ef
fecti
v
eness
w
as
measured
by
recording
the
number
of
errors
and
assistance
requests
made
by
the
participants
while
completing
the
use
case.
T
able
4
sho
ws
the
a
v
erage
number
of
errors
and
assistance
requests
per
platform.
T
able
4.
Ef
fecti
v
eness
results:
errors
and
assistance
requests
per
platform
Platform
Mean
errors
SD
Mean
assistance
SD
Blackboard
0.57
0.79
0.71
1.25
Can
v
as
0.24
0.17
0.00
0.00
Classroom
0.60
0.55
0.80
1.10
Can
v
as
presented
the
lo
west
a
v
erage
number
of
errors
(M=0.24)
and
no
participant
requested
assi
s-
tance
during
the
task.
Blackboard
and
classroom
sho
wed
similar
a
v
erage
error
rates
(0.57
and
0.60,
respec-
ti
v
ely),
whereas
classroom
presented
the
highest
a
v
erage
number
of
assistance
requests
(M=0.80).
4.1.3.
Satisfaction
User
satisf
action
w
as
measured
using
the
SUS.
T
able
5
presents
the
descripti
v
e
statistics
for
each
platform.
P
articipants
rated
can
v
as
the
highest
in
terms
of
usability
,
with
a
mean
SUS
score
of
82.6
and
lo
w
v
ariability
between
participants,
suggesting
consistent
s
atisf
action.
Classroom
had
a
slightly
lo
wer
a
v
erage
(M
=
76.5)
b
ut
sho
wed
greater
v
ariability
,
including
a
relati
v
ely
lo
w
score
(42.5).
Blackboard
sho
wed
the
lo
west
satisf
action
(M
=
61.1),
with
a
wider
spread
of
responses
ranging
from
37.5
to
80.
Evaluating
usability
,
accessibility
,
and
per
suasion
in
learning
mana
g
ement
systems
(Samantha
Jim
´
enez)
Evaluation Warning : The document was created with Spire.PDF for Python.
2018
❒
ISSN:
2252-8776
T
able
5.
Descripti
v
e
statistics
of
SUS
scores
by
platform
Platform
Mean
SD
Min
Max
Blackboard
61.07
13.98
37.5
80.0
Can
v
as
82.60
3.97
78.0
88.0
Classroom
76.50
21.11
42.5
97.5
According
to
the
SUS
interpretation
guidelines,
can
v
as
f
alls
within
the
e
xcel
lent
usability
range,
class-
room
within
the
good
range,
and
blackboard
within
the
OK
range.
4.2.
Accessibility
e
v
aluation
Accessibility
compliance
w
as
e
v
aluated
using
the
WCA
G
2.1
checklist
described
in
section
3.
T
able
6
summarizes
the
compliance
of
the
three
LMSs
with
the
selected
criteria.
Blackboard
and
can
v
as
complied
with
all
e
v
aluated
criteria,
whereas
Google
Classroom
sho
wed
partial
compliance
in
v
e
of
the
eight
e
v
aluated
criteria,
mainly
related
to
alternati
v
e
te
xt,
k
e
yboard
na
vig
ation,
focus
order
,
and
form
labeling.
T
able
6.
Accessibility
compliance
checklist
(WCA
G
2.1)
for
assignment
submission
task
(BB
=
Blackboard,
CV
=
Can
v
as,
and
GC
=
Classroom)
Nº
WCA
G
2.1
Criterion
Description
BB
CV
GC
1
1.1.1
Non-te
xt
content
(A)
All
icons
or
b
uttons
used
to
upload
les
must
ha
v
e
accessible
alternati
v
e
te
xt
or
labels
for
screen
readers.
x
x
P
artially
(some
b
uttons
lack
alt
te
xt)
2
1.3.1
Info
and
re
lationships
(A)
Input
elds
must
be
properly
labeled
for
assisti
v
e
technologies.
x
x
P
artially
(not
all
labels
are
programmatically
dened)
3
1.4.3
Contrast
(Minimum)
(AA)
T
e
xt
and
b
uttons
must
maintain
suf
cient
contrast.
x
x
x
4
2.1.1
K
e
yboard
(A)
Fully
operable
via
k
e
yboard
only
.
x
x
Does
not
fully
support
k
e
yboard-only
na
vig
ation
5
2.4.3
F
ocus
order
(A)
Logical
na
vig
ation
sequence
is
maintained.
x
x
F
ocus
jumps
or
skips
in
some
vie
ws
6
3.3.1
Error
identication
(A)
Clear
error
messages
when
mistak
es
are
made.
x
x
Limited
7
3.3.2
Labels
or
instructions
(A)
Clear
,
visible
instructions
on
ho
w
to
upload
tasks.
x
x
Instructions
are
minimal
8
4.1.2
Name,
role,
v
alue
(A)
Interacti
v
e
elements
must
be
correctly
dened
for
assisti
v
e
technologies.
x
x
Some
roles/v
alues
missi
ng
or
incorrectly
mapped
4.3.
P
ersuasion
e
v
aluation
This
subsection
presents
the
results
of
the
percei
v
ed
persuasi
v
eness
e
v
aluation
of
the
three
LMS
plat-
forms,
structured
according
to
the
four
cate
gories
of
the
PSD
model.
A
structured
template
based
on
the
PSD
model
w
as
used
to
assess
the
e
xtent
to
which
students
percei
v
ed
the
presence
of
persuasi
v
e
strate
gies
in
each
system’
s
interf
ace.
A
5-point
Lik
ert
scale
w
as
used
for
this
purpose
(1
=
Strongly
disagree
to
5
=
Strongly
agree).
The
maximum
possible
score
w
as
125
points,
deri
v
ed
from
the
sum
of
all
e
v
aluated
strate
gies
across
the
four
PSD
cate
gories.
T
able
7
presents
the
e
v
aluation
results
of
three
LMS:
blackboard,
classroom,
and
can
v
as.
The
answers
are
or
g
anized
according
to
the
persuasi
v
e
strate
gies
used,
grouped
into
four
main
cate
gories
based
on
the
PSD
model:
primary
task
support,
dialogue
support,
system
credibility
support,
and
social
support.
The
total
possible
score
for
the
assessment
w
as
125
points.
The
closer
the
total
score
of
the
LMS
is
to
this
v
alue,
the
greater
the
percei
v
ed
persuasi
v
eness
from
the
students’
perspecti
v
e.
Figure
3
presents
the
o
v
erall
score
of
each
system.
The
results
sho
wed
that
classroom
is
in
rst
place,
blackboard
second,
and
can
v
as
third.
In
Figure
4,
we
present
scores
according
to
each
cate
gory
to
emphasize
where
the
LMS
is
most
prominent.
The
primary
task
support
cate
gory
e
v
aluates
ho
w
ef
fecti
v
ely
the
LMS
simplies
the
tar
get
beha
vior
by
pro
viding
guidance,
personalization,
or
mechanisms
to
practice
the
required
actions.
In
this
cate
gory
,
classroom
scored
highest
(23/30),
follo
wed
by
blackboard
(17/30)
a
nd
can
v
as
(12/30),
indicating
a
more
ef
fecti
v
e
imple-
mentation
of
reduction
and
guidance
strate
gies
in
classroom.
Students
agreed
that
T
unneling
and
Reduction
strate
gies
are
implemented
across
the
three
assessed
LMSs,
whereas
simulation
is
less
emphasized.
Int
J
Inf
&
Commun
T
echnol,
V
ol.
15,
No.
4,
December
2026:
2011–2024
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Inf
&
Commun
T
echnol
ISSN:
2252-8776
❒
2019
The
dialogue
support
cate
gory
encompasses
the
communication
between
the
LMS
and
the
st
udent,
in
v
olving
feedback
and
encouragement
mechanisms
such
as
prizes,
re
w
ards,
reminders,
among
others.
The
scores
from
highest
to
lo
west
are:
classroom
(19/30),
blackboard
in
second
place
(13/
30
)
,
and
can
v
as
in
third
place
(12/30).
There
is
no
similarity
in
the
implemented
strate
gies
across
the
LMSs,
b
ut
the
use
of
reminders,
liking,
and
social
roles
appears
to
dif
ferentiate
classroom
most
signicantly
.
System
credibility
assesses
the
percei
v
ed
credibility
of
the
LMS
from
the
surf
ace,
from
the
initi
al
perception
of
the
UI
to
the
accurac
y
of
the
content.
Classroom
outperformed
the
others
(27/30),
follo
wed
by
blackboard
(14/30),
and
can
v
as
(14/30).
Strate
gies
such
as
e
xpertis
e
and
authority
were
particularly
strong
in
classroom.
Social
support
attempts
to
moti
v
ate
students
by
le
v
eraging
social
inuence
and
interaction
with
peer
s
by
comparing
performances,
competing,
or
of
fering
public
recognition.
Classroom
achie
v
ed
the
hi
ghest
score
in
the
cate
gory
(26/35),
follo
wed
by
blackboard
(18/35),
and
can
v
as
(7/35).
Notably
,
classroom
implemented
more
di
v
erse
strate
gies
such
as
cooperation,
recognition,
and
social
f
acilitation,
which
were
minimally
present
in
can
v
as.
It
is
important
to
note
that
these
results
reect
percei
v
ed
persuasi
v
eness
based
on
self-reported
data
and
may
be
inuenced
by
indi
vidual
dif
ferences
in
f
amiliarity
or
e
xpectations.
T
able
7.
Persuasi
v
e
strate
gy
e
v
aluation
by
platform
(Means
by
cate
gory)
Strate
gy
Blackboard
Classroom
Can
v
as
Reduction
4
5
3
T
unneling
4
5
3
T
ailoring
and
personalization
3
3
2
Self-monitoring
4
4
2
Simulation
1
2
1
Rehearsal
1
4
1
Mean
2.83
3.83
2
Prices
4
2
2
Re
w
ards
1
2
1
Reminders
or
suggestions
2
5
2
Similarity
1
1
2
Liking
3
4
3
Social
role
2
5
2
Mean
2.16
3.16
2
T
rustw
orthiness
2
4
4
Expertise
2
5
3
Surf
ace
credibility
4
5
3
Real
w
orld
feel
1
4
2
Authority
2
5
1
V
eriability
3
4
1
Mean
2.33
4.5
2.33
Social
learning
1
4
1
Social
comparison
4
4
1
Normati
v
e
inuence
4
3
1
Social
f
acilitation
4
4
1
Cooperation
2
5
1
Competition
1
2
1
Recognition
2
4
1
Mean
2.57
3.71
1
Figure
3.
Persuasion
e
v
aluation
total
scores
Evaluating
usability
,
accessibility
,
and
per
suasion
in
learning
mana
g
ement
systems
(Samantha
Jim
´
enez)
Evaluation Warning : The document was created with Spire.PDF for Python.
2020
❒
ISSN:
2252-8776
Figure
4.
Persuasion
e
v
aluation
scores
by
cate
gory
5.
DISCUSSION
This
study
compared
three
widely
used
LMS
from
the
perspecti
v
es
of
usability
,
accessibility
,
and
persuasi
v
e
design
withi
n
the
conte
xt
of
engineering
education.
Ov
erall,
can
v
as
demonstrated
the
highest
ef-
cienc
y
and
ef
fe
cti
v
eness
during
assignment
submission,
whereas
Google
Classroom
ac
hie
v
ed
the
highest
percei
v
ed
persuasion
scores
across
the
four
PSD
cate
gories.
Blackboard
presented
intermediate
performance
b
ut
lo
wer
user
satisf
acti
on
than
the
other
e
v
aluated
platforms.
Re
g
arding
accessibility
,
blackboard
and
can
v
as
complied
with
the
e
v
aluated
WCA
G
2.1
criteria,
whereas
Google
Classroom
presented
partial
compliance
in
se
v
eral
accessibility-related
aspects.
5.1.
Usability
implications
The
usability
ndings
partially
agree
with
pre
vious
comparati
v
e
studies
on
LMS
e
v
aluation.
Gumasing
et
al.
[9]
reported
no
statistically
signicant
dif
ferences
i
n
o
v
erall
usability
among
blackboard,
can
v
as,
and
Google
Classroom,
although
Google
Clas
sroom
obtained
slightly
higher
o
v
erall
SUS
ratings.
In
contrast,
the
present
study
found
that
can
v
as
achie
v
ed
the
highest
percei
v
ed
usability
and
the
lo
west
task
com-
pletion
times.
Although
the
Kruskal-W
allis
analysis
did
not
re
v
eal
statistically
signicant
dif
ferences,
the
descripti
v
e
statistics
consistently
f
a
v
ored
can
v
as
re
g
arding
ef
cienc
y
and
ef
fecti
v
eness.
Similarly
,
Alg
amdi
et
al.
[10]
reported
that
can
v
as
outperformed
blackboard
in
na
vig
ation
and
interf
ace
design
while
blackboard
pro
vided
richer
functionality
.
Our
ndings
are
consistent
with
this
observ
ation,
as
participants
completed
the
assignment
submission
task
more
ef
ciently
in
can
v
as
despite
blackboard
of
fering
comparable
functionality
.
5.2.
P
ersuasion
implications
Unlik
e
usability
,
the
persuasion
e
v
aluation
re
v
ealed
a
dif
ferent
pattern.
Google
Classroom
obtained
the
highest
scores
across
the
four
PSD
dimensions,
part
icularly
in
primary
task
support
and
system
credibil-
ity
.
These
ndings
suggest
that
students
percei
v
ed
Classroom
as
a
platform
that
better
guides
users
through
academic
acti
vities
while
con
v
e
ying
higher
credibility
and
trust.
These
results
align
with
pre
vious
w
ork
by
Orji
et
al.
[23]
and
Ahmad
et
al.
[24]
who
reported
that
persuasi
v
e
mechanisms
embedded
in
educational
s
ystems
contrib
ute
positi
v
ely
to
learner
eng
agement
and
participation.
Lik
e
wise,
Murillo-Mu
˜
noz
et
al.
cite21
emphas
ized
that
persuasi
v
e
educational
systems
can
reinforce
beha
viors
associated
with
learning
persistence
and
moti
v
ation.
Therefore,
although
classroom
w
as
not
the
f
astest
platform,
its
persuasi
v
e
characteristics
may
positi
v
ely
inuence
students’
long-term
interaction
with
the
LMS.
5.3.
Accessibility
implications
The
accessibility
e
v
aluation
demonstrated
that
none
of
the
e
v
aluated
platforms
fully
addressed
e
v
ery
aspect
of
inclusi
v
e
interf
ace
design.
Although
blackboard
and
can
v
as
complied
with
all
manually
e
v
aluated
criteria,
Google
Classroom
presented
partial
compliance
in
se
v
eral
WCA
G
2.1
requirements
related
to
k
e
yboard
na
vig
ation,
form
labeling,
and
focus
management.
These
issues
may
create
additional
barriers
for
students
relying
on
assisti
v
e
technologies
or
alternati
v
e
interaction
methods.
Considering
the
increasing
adoption
of
fully
online
engineering
programs,
accessibility
should
be
considered
a
fundamental
design
requirement
rather
than
an
optional
feature.
Assignment
submission
is
among
the
most
frequent
academic
interactions,
and
e
v
en
minor
accessibility
limitations
may
ne
g
ati
v
ely
af
fect
students’
learning
e
xperience.
Int
J
Inf
&
Commun
T
echnol,
V
ol.
15,
No.
4,
December
2026:
2011–2024
Evaluation Warning : The document was created with Spire.PDF for Python.