Indonesian J our nal of Electrical Engineering and Computer Science V ol. 42, No. 1, April 2026, pp. 149 ∼ 163 ISSN: 2502-4752, DOI: 10.11591/ijeecs.v42.i1.pp149-163 ❒ 149 Student acti vity r ecognition fr om classr oom video: a sur v ey Phuong-Dung Nguy en 1,2,3 , Khanh-Huy en Bui 1,2 , Thi-Lan Le 1,2 1 School of Electrical and Electronic Engineering (SEEE), Hanoi Uni v ersity of Science and T echnology (HUST), Hanoi, V ietnam 2 SigM Laboratory , SEEE, Hanoi Uni v ersity of Science and T echnology (HUST), Hanoi, V ietnam 3 Thuyloi Uni v ersity , Hanoi, V ietnam Article Inf o Article history: Recei v ed Apr 3, 2025 Re vised Feb 20, 2026 Accepted Mar 4, 2026 K eyw ords: Acti vity detection Classroom video Student acti vity recognition ABSTRA CT Student beha vior and acti vity play a crucial role in shaping the classroom atmo- sphere and inuencing the quality of a learning session. Recently , vision-based student acti vity recognition has g ained signicant attention. Ho we v er , recog- nizing student acti vities from classroom videos presents unique challenges due to the nature of the classroom en vironment, such as the pr esence of multiple students and se v ere occlusions. As a result, research in this area has often o v er - look ed these challenges. This study pro vides a detailed and comprehensi v e re- vie w of student acti vity recognition from classroom videos. First, we formalize the problem of student acti vity recognition from videos and cate gorize e xisting methods into three distinct approaches: frame-le v e l, clip-le v el, and continuous recognition. W e then pro vide a detailed analysis of r epresentati v e methods for each approach. In addition, we present a comprehensi v e o v ervie w of publicly a v ailable datasets for student acti vity recognition and discuss k e y open chal- lenges, together with potential future research directions. Our analysis re v eals that: (1) Most e xisting studies focus on frame-le v el recognition, while clip-based and continuous acti vity recognition remain relati v ely undere xplored; (2) there is still a lack of lar ge-scale, standardized benchmark datasets for vision-based stu- dent acti vity recognition; and (3) e xisting research primarily emphasizes recog- nition accurac y , whereas real-time performance and com putational ef cienc y are rarely addressed. This is an open access article under the CC BY -SA license . Corresponding A uthor: Thi-Lan Le School of Electrical and Electronic Engineering (SEEE) Hanoi Uni v ersity of Science and T echnology (HUST) Dai Co V iet, Hanoi, V ietnam Email: lan.lethi1@hust.edu.vn 1. INTR ODUCTION Acti vity recognition in v olv es monitoring and analyzing human beha vior and the surrounding en vi- ronment to identify or infer ongoing acti vities [1]. The main objecti v e of acti vity recognition is to pro vide insights into users’ acti vities, states, and beha viors, enabling proacti v e computational systems to of fer per - sonalized assistance and support. This proce ss often relies on data collected from sensors such as cameras, wearable de vices, and other information sources. In recent years, acti vity recognition has g ained signicant at- tention from the research community thanks to its broad applications in human-computer interaction, abnormal acti vity detection for smart cities, and gesture assessment in ph ysical therap y for patients, etc. In the educational domain, student beha vior and acti vity play a crucial role in shaping the clas sroom atmosphere and determining the quality of a learning session. Con v entional methodology based on teacher observ ations, though commonly practiced, is often subjecti v e and f aces challenges in monitoring all students, J ournal homepage: http://ijeecs.iaescor e .com Evaluation Warning : The document was created with Spire.PDF for Python.
150 ❒ ISSN: 2502-4752 particularly in lar ge classrooms or when visual obstructions occur . T o address these challenges, analyzing vi- sual [2], auditory [3], or ph ysical signals [4] presents promising solutions for automatic classroom assessment. Among these modalities, the visual modality is the most widely used, as cameras are often readily a v ailable in classrooms and can pro vide rich conte xtual information for more accurate student acti vity recognition. Student acti vity recognition from classroom videos an be considered as a sub topic of human acti vity recognition (HAR) ho wwe v er it presents specic challenges. The rst challenge is se v ere occlusion. Students may be obscured by other students or by classroom furniture such as chairs and desks, complicating accurate detection and tracking of indi viduals. The second challenge is the high number of subjects (i.e., students) in the scene. Most acti vity recognition methods assume the presence of only one or tw o subjects. Ho we v er , in a cl assroom conte xt, the number of students is usually high, with dif ferent students potentially eng aging in v arious acti vities at dif ferent times. While numerous surv e ys ha v e been conducted on HAR in general [5]-[8], and within specic domains such as sports [9], [10], there is a lack of surv e ys focusing specically on student acti vity recognition [11]. This study focuses on student acti vity recognition from classroom videos. Specically , we present a detailed and comprehensi v e re vie w of e xisting methods, including those that rely solely on spatial informat ion and those incorporating both spatial and temporal features. Additionally , we pro vide a detailed o v ervie w of datasets col- lected for student acti vity recognition. Furthermore, we discuss v arious open challenges and propose potential future research directions. The k e y contrib utions of this paper are as follo ws: (1) W e dene the problem of student acti vity recognition and cate gorize it into three distinct approaches; (2) W e present a state-of-the-art re vie w of methods and datasets used for student acti vity recognition in classroom set tings; (3) W e identify open challenges and suggest possible directions for future research. 2. STUDENT A CTIVITY RECOGNITION FORMULA TION Student acti vity recognition can be formally dened as follo ws: Gi v en an untrimmed video with T consecuti v e frames, student acti vity recognition aims to detect a set of P of M tubelets P i , each corre- sponding to an instance of an acti vity performed by a student. The set of tubelets is represented as: P = { P 1 , P 2 , P i , . . . , P M } , where each tubelet P i is dened as, P i =  P t i | t = x, x + 1 , . . . , x + N P i , c i (1) with: P t i : is the bounding box at frame t of tubelet i th that represents the spatial localization of the acti vity instance at frame t ; x : is the starting frame where the acti vity instance occurs; N P i is the duration of the acti vity (number of frames in the tubelet); c i : is the acti vity class label associated with the tubelet. Figure 1 illustrates a recognized tubelet P i of class c i is raisi ng hand. The starting frame of the tubelet is 0672 x = #0672 whereas the length of tubelet is 133 ( N P i = 133 ). Figure 1. An e xample of a tubelet output for student acti vity recognition Student acti vity recognition can be formally dened as a mapping from video frames to a set of tubelets. Gi v en an untrimmed video with T consecuti v e frames, we dene a function, f : { I 1 , I 2 , . . . , I T } → P (2) Indonesian J Elec Eng & Comp Sci, V ol. 42, No. 1, April 2026: 149–163 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 151 Where: { I t } T t =1 represents the sequence of video frames; f is a function that e xtracts and recognizes student acti vities; P = { P 1 , P 2 , . . . , P M } is the set of tubelets corresponding to detected student acti vities dened by (1). Thus, the functi on f m aps a sequence of frames to a structured set of tubelets, capturing student acti vities in the classroom. In recent years, se v eral w orks ha v e been dedicated to classroom acti vity recognition and encouraging results ha v e been achie v ed. Some approaches rely solely on spatial inform ation, while others inte grate both spatial and temporal data. Based on the type of information used, e xisting methods for student acti vity recog- nition can be cate gorized into three approaches: Frame-le v el, clip-le v el, and continuous acti vity recognition. Frame-le v el methods aim to detect instances of acti vities of interest within indi vidual image frames. The ma- jority of methods for student acti vity recognition f all into the frame-le v el cate gory thanks to the emer gence and adv ancements in object detection techniques [12]-[21]. Methods of the clip-le v el approach classify pre- se gmented video clips into specic classes. It is w orth noting that the term clips in student acti vity recognition refers to a sequence of re gions (i.e., bounding box es) of the acti vity in the original frames. In the acti vity recog- nition eld, a clip may refer to a sequence of entire frames, assuming that each frame contains only one person. Ho we v er , this is not the case for student acti vity recognition. Although some methods belonging the clip-le v el approach ha v e been proposed [22], [23], these methods ha v e dra wback because in a classroom setting, where man y students may be performing dif ferent acti vities at dif ferent times, determining the appropriate clips is not a straightforw ard task. Continuous student act i vity recognition, which aims to determine the location of the acti vity instance and track its changes o v er time, is the most suitable approach. Ho we v er , due to the challenges of student acti vity recognition, v ery fe w w orks ha v e been successfully de v eloped for this approach [24]. It is w orth noting that in the (1), the frame-l e v el methods can only determine the indi vidual bounding box es P t i for each type of acti vi ty . Ho we v er , the y do not pro vide information on whether the detected bounding box es across frames belong to the same acti vity instance. The clip-le v el approach can determine only the class c i for a gi v en tubelet P i , which is predened. In contrast, continuous acti vity recognition methods enable the determination of both the tubelet P i and its corresponding class c i . In the follo wing sections, the methods for student acti vity recognition in each cate gory will be analyzed. 3. METHODS This section outlines the procedure used to collect, screen, and cate gorize the article for a com prehen- si v e surv e y of student acti vity recognition methods in classroom videos. The o v erall methodology is adapted from the systematic re vie w frame w ork presented in the study [25], and rened to align with the three method- ological cate gories of this study: frame-le v el, clip-le v el, and continuous recognition. The w orko w consists of four major stages-Identication, Screening, Eligibility , and Included-corresponding to the process illustrated in Figure 2. The entire w orko w is designed to accurately reect the progression of research in the period 2013–2025 and to ensure consistenc y between the collected sources and the analytical structure presented in the subsequent sections. Figure 2. F our -stage article selection w orko w: Identication, screening, eligibility , and included 3.1. Step 1: Identication Based on the scope of this surv e y , we conducted a systematic search across se v eral reputable academi c databases, including IEEE Xplore, A CM Digital Library , Else vier ScienceDirect, SpringerLink, Scopus, and Student activity r eco gnition fr om classr oom video: a surve y (Phuong-Dung Nguyen) Evaluation Warning : The document was created with Spire.PDF for Python.
152 ❒ ISSN: 2502-4752 Google Scholar . The search tar geted studies directly related to student acti vity recognition from classroom videos, using tw o main groups of k e yw ords, - K e yw ords related to the tar get domain “student acti vity recognition”, “classroom acti vity recognition”, “student beha vior analysis”, “classroom video”, “hand-raising detection”. - K e yw ords related to technical approaches “object detection”, “pose estimation”, “spatio-temporal action detection”, “sk eleton-based action recognition”, “clip-le v el acti vity recognition”, “tracking”, “Simple online and real-time tracking (SOR T)”, “tubelet detection”. 3.2. Step 2: Scr eening The screening stage w as carried out in tw o sequential ltering steps to remo v e studies that were clearly irrele v ant before conducting full-te xt assessment. - T itle Screening: The titles of all retrie v ed publications were e xamined to quickly eliminat e studies that did not align with the research scope. T itles indicating a focus on generic HAR, human motion analysis without educational rele v ance, or applications unrelated to classroom en vironments were e xcluded. - Abstract Re vie w: The goal w as to determine whether each study e xplicitly addressed student-centered beha viors observ able in classr o om video recordings. Studies were e xcluded if the abstract did not clearly identify student acti vities as the primary tar get, if the beha vioral cate gories were ambiguous, or if the methodological focus did not in v olv e recognition or analysis of student actions. Only studies that satised both criteria-rele v ance in title and clarity in abstract-were retained for the eligibility stage. 3.3. Step 3: Eligibility The el igibility criteria were established to ensure that only studies with direct and meaningful contri- b utions to the problem of student acti vity recognition in classroom settings were included in this surv e y . - Inclusion criteria: • Published within the period 2013–2025. • Appeared in peer -re vie wed journals, international conferences, or book chapters. • F ocused on student acti vity recognition in classroom en vironments, at one of the three le v els: frame-le v el, clip-le v el, or continuous-le v el. • Pro vided a complete description of the methodology , model architecture, processi ng pipeline, and e xperimental protocol. • Reported quantitati v e e v aluation metrics enabling performance comparison (e.g., accurac y , mAP , F1-score). - Exclusion criteria: • Studies addressing general HAR without a classroom conte xt. • System-description papers that lack model e v aluation. • W orks for which the full te xt is not a v ailable. • Non-academic materials such as editorials, k e ynote talks, opinion pieces, or slide presentations. • Studies that do not report or clearly dene bounding box es, tubelets, clips, or action labels, making analysis and comparison infeasible. 3.4. Step 4: Included (Final set of studies) In the initial search phase, a total of 80 publications were retrie v ed from the selected dat abases. Through a multi-stage screening process, studies that did not f all within the scope of student acti vity recogni- tion, did not clearly describe classroom beha viors, or did not address acti vities such as hand-raising, sleeping, standing, or phone usage were e xcluded. Additional studies were remo v ed due to insuf cient methodological detail, lack of quantitati v e results, or misalignment with the three tar get methodological cate gories (frame-le v el, clip-le v el, and continuous recognition). Ultimately , 50 studies met all eligibility criteria and were retained for in-depth analysis. These studies were then ca te gorized into the three main methodological groups dened in this surv e y: frame-le v el, clip-le v el, and continuous recognition. Indonesian J Elec Eng & Comp Sci, V ol. 42, No. 1, April 2026: 149–163 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 153 4. RESUL TS AND DISCUSSIONS 4.1. Frame-le v el appr oach f or student acti vity r ecognition Frame-based methods focus on recognizing tar get acti vities within single image frames. This approach is closely related to object detection in still images, a research area that has attracted signicant attention in computer vision and machine learning. As sho wn in Figure 3, these act i vity recognition methods tak e indi vidual frames as input and generate corresponding outputs. Specically , for each frame, the models produce bounding box es that locate the rele v ant acti vities. While some approaches are designed to identify only one type of acti vity , others are capable of detecting multiple acti vities at the same time. Owing to the rapid progress in object detection techniques, a wide range of frame-based methods for student acti vity recognition has been de v eloped. Figure 3. Frame-le v el approach for student acti vity recognition Among student acti vities, hand-raising has attracted the most attention from the research communi ty as the number of hand-raising can be used to infer the quality of teaching session. Ho we v er , detecting hand- raising from images is challenging due to the occluded body parts and lo w-resolution hands areas. In [12], the authors proposed to apply Lienhart-Maydt method for hand-raising detection. Then, the authors proposed to combine the f ace detection results with those of hand detection within the same frame to enhance the detection accurac y . When both f ace and hand are detected, the relat i v e positions b e tween them are calculated to determine if the person is actually raising their hand. The w ork in [13] proposed a model to detect the gesture of raising a hand using static images as input. Initially , the model selects re gions of interest lik ely to contain the gesture of raising a hand, then uses HSV and YCbCr color space models to detect skin color in the images. The edges are detected using a stable Gaussian lter , and nally , the acti vity is classied as raising a hand or not. W ith the arri v al of deep-learning based object models, recent w orks mainly rely on these detecti on models to detect hand raising from classroom images. Deep learning models for object detection are generally di vided into tw o main types: tw o-stage object detectors (e.g. Re gion-based fully con v olutional netw ork (R- FCN) [26], F aster re gion-based con v olutional neural netw ork (RCNN) [27]) and one-stage object detectors (e.g. Y ou only look once (Y OLO) and its v ariants [28]). The authors in [29] proposed an R-FCN-based method for detecting hand-raising gestures in im ages captured within a real-w orld classroom setting. T o e n ha n c e the detection of hand-raising gestures, e v en in lo w-resolution images, the y inte grated p yramid features into t h e original R-FCN architecture. Specically , the R-FCN netw ork w as impro v ed by incorporating p yramid features into its design. Relying solely on object detection for hand-raising recognition may lead to f alse alarms and mis sed detections. T o address this issue, additional cues such as hand pose information ha v e been incorporated. In [15], the authors proposed a three-step frame w ork consisting of hand-raising detection, pose estimation, and matching steps. In the rst step, an impro v ed R-FCN algorithm w as applied for hand detection. In the second step, a no v el part af nity elds-based (P AF) pose estimation method w as introduced to detect human body Student activity r eco gnition fr om classr oom video: a surve y (Phuong-Dung Nguyen) Evaluation Warning : The document was created with Spire.PDF for Python.
154 ❒ ISSN: 2502-4752 k e ypoints. Finally , a heuristic matching strate gy based on the spatial relationship between detected hands and body k e ypoi nts w as emplo yed to identify students who were raising their hands. Similarly , [18] prese n t ed an approach for recognizing hand-raising gestures consisting of tw o stages. The rst stage uses multi-stage pose estimation to determine candidate hand re gions for each student. Then, the second stage applies a binary clas- sication netw ork to determine the specic gesture type. Both studies relied on the pose estimation technique. Ho we v er , this technique only w ork ed well in simple scenes without much occlusion or clutter . An e xtended non-local module combined with the Libra-RCNN detection model w as proposed in study [30], enhancing image information e xtraction by incorporating local conte xt and spatial correlation be- tween pix els. The model addressed tw o main limitations: the lack of locality-a w areness and the disre g ard for spatial relationships between pix els. Recently , [31] proposed a no v el machine learning approach that is rob ust to vie wpoint v ariations and occlusions. The method le v erages long short-term memory (LSTM) netw orks to detect hand-raising acti v- ity from pose estimation’ s information. Similarly , the authors in [32] proposed a morphology-based analysis method. The proposed method in this w ork utilized Y OLOx for object detection and HrNet for sk eleton es- timation. Students’ sk eleton k e y point data is con v erted into se v eral one-dimensional time series, allo wing for a detailed analysis of hand-raising beha viors. These models help in accurately capturing and analyzing hand-raising actions. F ollo wing a similar approach to impro ving R-FCN, the study [17] detected sleep persons in cl assrooms by le v eraging p yramid feature representation. The y emplo yed a modied R-FCN model inte grated a feature p yramid and deformable con v olution to solv e the challenges in sl eep gesture detection, such as occlusion and di v ersity of gestures. By inte grating both together , the system link ed feature layers at v arious scales. This made the proposed method more ef fecti v e in detecting small sleep gestures. Additionally , the study proposed using deformable con v olution netw orks combined with local multiscale testing, enabling the system to learn the specic characteristics of small-size sleep gestures, such as v arious postures and changes in the bounding box sizes. Some w orks tried to recognize more acti vities. In [33], the authors proposed a method based on CNNs combined with transfer learning. Pre-trained architectures, such as V GG16 and V GG19 , were utilized to e xtract deep features from images. In [19], the authors de v eloped an intelligent system capable of automat- ically recognizing student beha viors in recorded classrooms including raising hands, standing up, and dozing of f. These beha viors are challenging to detect due to scale v ariations, lo w resolution, and imbalanced sample distrib utions. The y enhanced F aster R-CNN by introducing a ne w sc ale-a w are detection head to handle scale v ariations, a feature fusion strate gy for detecting lo w-resolution beha viors with minimal e xtra computation, and Online hard e xample mining (OHEM) to mitig ate se v ere class imbalances. Additionally , the y proposed a technique that combines feature v ectors from dif ferent layers to create the feature map for detection, impro ving the accurac y of detecting small objects, such as sleeping and hand-raising gestures. The study [20] introduced a ne w detection model called GestureDet. This model allo ws for the de- tection of typical student gestures, including raising hands, standing up to speak, and sleeping. The authors enhanced the MobileNetV2 object detection model by inte grating spatial attention, channel attention, and batch attention mechanisms to learn features more rob ustly from data. Additionally , GestureDet’ s lightweight nature allo ws to deplo y it on embedded de vices such as the NV idia Jetson TX2. The study [34] focuses on the task of recognizing se v en common types of student acti vities in the classroom by le v eraging the po wer of pre-trained CNN models. The authors apply transfer learning techniques to ne-tune well-kno wn netw ork architectures such as V GG-16, ResNet-50, Inception V3, and Xception on a self-constructed classroom dataset consisting of more than 4,000 images. The study on ET -Y OLOv5s [35] addre ssed the challenges posed by lo w-resolution classroom en viron- ments and small-scale objects by incorporating an enhanced super -resolution generati v e adv ersarial netw ork (ESRGAN). The ESRGAN module w as emplo yed to enhance image details prior to inputting the images into the detection netw ork. In 2024, J ia and He [36] proposed the SBD model, a h ybrid frame w ork that inte grates Y OLOv5 with the coordinate attention (CA) mechanism to generate high-quality feature maps, replacing the traditional V GG- 19 backbone in the OpenPose frame w ork. The proposed model not only performs object detection b ut also f acilitates detailed analysis of human k e ypoints, thereby impro ving beha vior recognition performance under conditions of partial student occlusion. The authors in [21] de v eloped a student beha vior recognition system based on sk eleton pose estimation Indonesian J Elec Eng & Comp Sci, V ol. 42, No. 1, April 2026: 149–163 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 155 and pers on detection to assess students’ learning attitude. F our main student acti vities are considered including looking, asking, boring and bo wing. Based on the analysis results, thi s system can e v aluate student attitudes to w ards the lesson, pro viding useful information for teachers and education managers. In [2], the authors e v aluated the performance of three object detection models including F aster R-CNN, Y OLOv5, and detection transformer (DETR) for student acti vity recognition on the StudentAct dataset. T able 1 summarizes the methods t hat follo w the frame-le v el approach. Although these methods ha v e achie v ed promising results, the y often suf fer from missed detections and f alse alarms due to their inability to capture the temporal aspect of acti vities. T able 1. Summary of frame-le v el methods for student acti vity recognition Method Description Acti vities Result Le v el Nazar ´ e and Ponti (2013) [12] Identify hand-raising posture by analyzing the positional relation- ship between the f ace and hands Raising hand Achie v ed o v er 60% accurac y on most of the their test data Frame Jesna et al. (2016) [13] Use HSV and YCbCr color space for skin color detection and Gaus- sian lters for edge detection Raising hand Achie v ed 91% accurac y on their hand dataset Frame Zhou et al. (2018) [15] Use Pose estimation to identify k e y body points and combine po- sitional information Raising hand Achie v ed 83% accurac y on their dataset (30 schools) Frame Si et al. (2019) [29] Enhance R-FCN by inte grating the Feature Pyramid to detect hand- raising gestures Raising hand Achie v ed 90% mAP on self- constructed dataset Frame Liao et al. (2019) [18] Utilize multi-stage pose estima- tion to identify the hand-based re- gion Raising hand Achie v ed accurac y of 94 . 76% Frame Buhler et al. (2023) [31] Use LSTM to detect hand-raising actions from pose estimation re- sults Raising hand F1-score of 76% Frame Chen et al. (2024) [32] Use Y OLOx and HrNet for sk ele- ton estimation, transformed into time series Raising hand Analysis of acti vity’ s speed and amplitude Frame Li et al. (2019) [17] Modied R-FCN with feature p yramid and deformable con v olu- tion Sleeping Achie v ed 0 . 74 AP@0.5 Frame Hoang et al. (2019) [33] CNNs (V GG16, V GG19) com- bined with transfer learning 8 acti vities (writing, reading, etc) V GG19 achie v ed 80 . 8% accurac y Frame Zheng et al. (2020b) [19] F aster R-CNN with Scale-a w are Detection Head and OHEM Hand-raising, standing, sleeping Achie v ed 57 . 6% mAP Frame Zheng et al. (2020a) [20] GestureDet: Impro v ed Mo- bileNetV2 with spatial and channel attention Hand-raising, standing, sleeping Achie v ed 74 . 5% mAP Frame Lin et al. (2021) [21] Sk eleton pose estimation and hu- man detection Asking, boring, bo w- ing, looking classes Precision 89% , Recall 91% Frame Nguyen et al. (2022) [2] Ev aluate F aster R-CNN, Y OLOv5 and DETR Standing, sitting, phone, sleeping, hand Y OLOv5 achie v ed best mAP (up to 94 . 3% ) Frame Lina et al. (2022) [35] Inte grate ESRGAN to restore im- age details 11 acti vities (bo wing, drinking, etc) Achie v ed a mAP of 96 . 8% Frame Deshpande and Deshpand (2023) [34] T ransfer learning with V GG-16, ResNet-50, Xception 7 acti vities (discussion, writing, etc) Xception achie v ed 92% accurac y Frame Jia and He (2024) [36] Y OLOv5 with Coordinate Atten- tion replacing V GG-19 in Open- Pose Raising hand, standing, writing, etc Obtained an mAP of 82 . 1% Frame Among the s tudied acti vities, hand-raising has recei v ed the most attention from the research com mu- nity . Ho we v er , its detection performance v aries signicantly across dif ferent methods, ranging from 39.4% AP in [2] to 94.76% accurac y in [18]. A direct comparison of these methods is infeasible, as the y are e v aluated on pri v ate datasets collected by the respecti v e authors. Additionally , other important acti vities, such as using a phone, ha v e not been ful ly e xplored. Future research should focus on impro ving detection performance and e xtending recognition to a broader range of acti vities. Besides using appearance features, some me thods ha v e attempted to inte grate human sk eleton infor - mation to reduce f alse alarms. Ho we v er , pre vious studies were conducted in classroom en vironments with Student activity r eco gnition fr om classr oom video: a surve y (Phuong-Dung Nguyen) Evaluation Warning : The document was created with Spire.PDF for Python.
156 ❒ ISSN: 2502-4752 minimal occlusion and clutter , where human joints could be accurately es timated. In r eal classroom conditions, joint estimation performance may be less reliable, posing additional challenges for acti vity recognition. Finally , from a practical point of vie w in classroom management, it is crucial to recognize complete acti vity instances from be ginning to the end rather than analyzing indi vidual frames as recognizing full in- stances enables appropriate interv entions. F or e xample, detecting complete instances of ne g ati v e acti vities, such as using a phone or sleeping, allo ws measurement of their duration and frequenc y . This information can help inform decisions to adjust classroom content to better eng age the students. 4.2. Clip-le v el appr oach f or student acti vity r ecognition This approach focuses on classifying a predened tubelet (i.e., clip) into an action class as ill ustrated in Figure 4. Once the tubelet is dened, the problem becomes similar to action recognition and classication. Therefore, methods de v eloped for action recognition can be applied to clip-le v el student acti vity recognition [8], [37]. Figure 4. Clip-le v el acti vity recognition approaches In [38], the authors proposed to emplo y a combination of traditional feat ures, including local log- euclidean multi v ariate g aussian (L2EMG) and scale-in v ariant feature transform (SIFT) for student acti vity recognition. These features were used to capture shape and te xture v ariations of students in video cli ps for the classication of v e basic actions, namely raising hands, standing, sitting, writing, and reading. In the s tudy [39], the authors ne-tuned the tw o-stream I3D-ResNet-50 model [40] - an adv anced 3D video classication architecture that uses a 3D con v olution netw ork to directly learn and e xtract spatiotemporal information from video data to e v aluate the performance of the EduNet dataset, which consists of action data from classroom en vironments. Based on the joint estimation algorithm, study [23] proposed a model for recognizing student acti v- ities. The model in the study combines the Alphapose joint es timator and the DD-Net acti vity recognition netw ork, creating a compact deep learning model that still deli v ers high recognition accurac y . First, Alphapose e xtracts joint coordinates from clas sroom videos, di viding them into sequences from the start frame to the end frame of each acti vity for each student, and labels them accordingly . The input to DD-Net is the sequences of joint coordinates of a person performing a single acti vity . Another clip-le v el study introduced in [22], based on the Slo wF ast netw ork, incorporates the multi- scale spatial-temporal attention (MST A) module into the Slo w path, which includes e xtracting multi-scale spatial features, channel attention, and temporal attention to ef fecti v ely utilize channel, temporal, and spatial information at dif ferent scales. Subsequently , the ef cient temporal attention (ET A) module is introduced into the F ast path to enhance the model’ s detection performance and help the model bett er capture action information. Based on e xperimental results, the method with the addition of MST A and ET A impro v ed the mAP by 5 . 63% compared to the original Slo wF ast. The study [3] proposed a Con vNet model consisting of a spatial stream Con vNet using Inception V3 t o e xtract spatial features and a temporal stream Con vNet b uilt with Indonesian J Elec Eng & Comp Sci, V ol. 42, No. 1, April 2026: 149–163 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 157 a CNN comprising v e con v olutional layers with 3×3 k ernels, a stride of 1, a n d the ReLU acti v ation function to e xtract temporal features. The e xtracted features were then concatenated to form a feature v ec tor for the nal prediction. In [41], the authors proposed a no v el methodology based on a Histogram of Actions combined with g aze information. A 3D-CNN w as emplo yed to e xtract spatiotem p or al features from 2-minute video se gments. The model achie v ed an F1-score of up to 90% on a dataset containing 1,414 clips across 13 action cate gories, demonstrating the ef fecti v eness of inte grating action frequenc y with students’ attention direction to assess learning quality . In [42], the authors adopted the X3D architecture and i ntroduced a fusion mechanism that combines RGB features with human sk eleton data. This design enhances recognition performance under chal- lenging conditions such as student occlusion and limited computational resources. Experiments cond uc ted on a lar ge-scale classroom beha vior dataset sho wed that the proposed model achie v ed a T op-1 accurac y of 88.36%, while signicantly reducing the number of parameters and computational cost compared to con v entional 3D CNN models. Some clip-le v el methods ha v e bee n proposed for student acti vity recognition. Ho we v er , these ap- proaches typically rely on the assumption that the clips ha v e be dened a prior . The main dra wback of sequence-le v el methods is that in a classroom setting, where man y students may be performing dif ferent acti v- ities at dif ferent times, determining the appropriate clips for analysis is not a straightforw ard task. 4.3. Continuous acti vity r ecognition appr oach Methods in the continuous acti vity recognition cate gory aim to detect complete instances of each acti vity in untrimmed classroom videos. This in v olv es identifying the spatial location of each acti vity instance within the video frame, typically dened by a bounding box, as well as determining its start and end times. Figure 5 sho ws the input and output of continuous student acti vity recognition. Figure 5. Continuous-le v el acti vity recognition approaches This approach is similar to spatio-temporal action detection in act ion recognition [43]-[45], ho we v er due to the challenges of student acti vity recognition, v ery fe w w orks ha v e been proposed for continuous recog- nition from untrimmed videos[24]. In the study [24], the author proposed a method for continuous hand-raising recognition from untrimmed video. The proposed approach combines hand-raising detection and tracking to follo w the hand-raising acti vity o v er time. The detection model is an impro v ed v ersion of the Libra-RCNN deep learning object detection Student activity r eco gnition fr om classr oom video: a surve y (Phuong-Dung Nguyen) Evaluation Warning : The document was created with Spire.PDF for Python.
158 ❒ ISSN: 2502-4752 netw ork, incorporating an enhanced non-local block and le v eraging correlation features based on distance and position between elements on the feature map. The tracking component utilizes the SOR T object tracking module [46]. T able 2 lists all methods belonging to clip and continuous cate gories. T able 2. Summary of methods belonging to clip-le v el and continuous approach for student acti vity recognition Method Description Acti vities Result Recognition Le v el Lei et al. (2019) [38] Combine traditiona l features L2EMG (Local Log-Euclidean Multi v ariate Gaussian) and SIFT (Scale-In v ariant Feature T ransform) Raising hand, stand- ing, sitting, writing, reading Achie v ed an a v erage accurac y of 82 . 05% Frame-le v el Sharma et al. (2021) [39] Use tw o-stream I3D-ResNet-50 model (RGB and Optical Flo w) for video classication 20 acti vitie s (ar gu- ing, clapping, eating, hand raise, hitting, etc) Accurac y of 72 . 3% on EduNet dataset Clip-le v el Nguyen et al. (2023) [23] Combine Alphapose joint estima- tion with the DD-Net acti vity recog- nition netw ork for sk eleton-based acti vity recognition Raising hand, sleep- ing, standing, sitting, using phone Accurac y , precision, recall and F1-score were 74 . 58% , 78 . 71% , 74 . 58% , and 72 . 15% on Studen- tAct Sk el Clip-le v el Zhang et al. (2023) [22] Combine the Multi-scale Spatial- T emporal Attention (MST A) and the Ef cient T emporal Attention (ET A) to focus on salient features o v er time 7 acti vities mAP of 91 . 1% on SCSB dataset Clip-le v el Liu et al . (2023) [3] Propose a Con vNet model to e xtract spatial and temporal features con- sisting of a Spatial Stream Con vNet and a T emporal Stream Con vNet 7 acti vities Ov erall accurac y of 83 . 0 1% on Ed- uAction dataset Clip-le v el Ahmed et al. (2023) [41] Introduce a method base d on a His- togram of Actions combined with g aze data 13 acti vities (raising hand, ya wn, writing, etc) Achie v ed F1-score of up to 90% Clip-le v el Zou et al. (2025) [42] Inte grate a fusion mechanism be- tween RGB features and human sk eleton data to enhance recognition performance 6 acti vities (raising hand, standing up, listening, etc) Achie v ed an T op-1 Accurac y of 88 . 36% Clip-le v el Le et al. (2023) [24] Combine the impro v ed Libra- RCNN model with the SOR T tracking module to recognize and track hand-raising gestures Raising hand Frame-wise accurac y , T emporal IoU, F1-score, and Le v enshtein score were 90% , 84 . 4% , 83 . 2% and 84 . 3% Continuous Nguyen et al. (2025) [47] Propose a tw o-step method (ST rack4Re) for continuous student acti vity recognition 5 acti vities Y OLOv5 and OC-SOR T were se- lected for the object detection and tracking steps Continuous Bui et al. (2025) [48] Enhanced the ST rack4Re method by incorporating boundi ng box es from virtual trajectories and inte- grating appearance features to in- crease tracking rob ustness. 5 acti vities Achie v ed the highest F1-scores in acti vity recognition, outperforming the baseline by up to 10.9%. Continuous 4.4. Classr oom acti vity r ecognition image and video datasets T o train the recognition models and to e v aluate their performance, se v eral datasets ha v e been col lected and annotated. In the study by [29], the authors b uilt a lar ge-scale hand-raising dataset by recording videos from tw o static cameras in a real classroom with more than 30 students. After the annotation process, the dataset included a total of 40,000 hand-raising samples, with 28,000 samples u s ed for training and 12,000 for testing. This dataset also presented challenges for the hand-raising detection task, such as occlusions, lo w resolution, and v ariations in hand-raising gestures. In [49], a dataset for hand-raising recognition at the frame le v el w as introduced, comprising a total of 22,000 images with 76,000 bounding box annotations. All images had a resolution of 1920 × 1080 pix els. The training set w as collected from four classes in an elementary school, while the testing set consisted of a video featuring 23 students in a meeting room. The signicant dif ferences between the postures of children and adults introduced certain challenges for hand-raising detection. Another similar dataset for hand-raising acti vity , named Class HRP w as also introduced in [24], consisting of 6,733 images recorded o v er multipl e sessions in a 50-student classroom of the uni v ersity . A total of 53,798 hand- raising samples ha v e been annotated, with 4,928 images containing 34,511 samples used for the training set and 1,805 images with 19,287 samples allocated for the test set. Indonesian J Elec Eng & Comp Sci, V ol. 42, No. 1, April 2026: 149–163 Evaluation Warning : The document was created with Spire.PDF for Python.