Application of gesture recognition based on domain transfer learning and SimSiam network in education management

Xiaoxu  He

doi:10.6180/jase.202510_28(10).0019

Application of gesture recognition based on domain transfer learning and SimSiam network in education management

Research Categories

Xiaoxu HeThis email address is being protected from spambots. You need JavaScript enabled to view it.

Shenyang City University, No. 2, Wutong Street., Sujiatun District., Shenyang City, 110100 China

Received: November 17, 2023
Accepted: December 21, 2024
Publication Date: February 27, 2025

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202510_28(10).0019

As one of the important components of augmented reality system, gesture recognition is widely used in virtual reality, intelligent interaction and human-machine interface. In education management, how to improve the level of teaching management and teaching quality, gesture recognition plays a very important role. In view of the problems of negative transfer and poor model generalization in single-source domain transfer learning, which have great impact on gesture recognition, this paper proposes a new novel gesture recognition method based on domain transfer learning and SimSiam network. Firstly, the gesture data set is passed into the SimSiam self-supervised network for training. On this basis, the technique of domain specific feature alignment and domain classifier alignment is adopted. This approach aims to enhance the model’s gesture recognition performance between different users, thus significantly improving the accuracy of cross-user gesture recognition systems. Experimental results show that this proposed method can effectively identify a variety of dynamic gestures with Angle deflection. Compared with the traditional displacement feature method, the average accuracy of the proposed method is increased by 4%, and it can effectively deal with the dynamic gesture recognition problem in the case of palm deflection.

Keywords: education management, gesture recognition, domain transfer learning, SimSiam network

[1] R. Tchantchane, H. Zhou, S. Zhang, and G. Alici, (2023) “A review of hand gesture recognition systems based on noninvasive wearable sensors" Advanced In telligent Systems 5(10): 2300207. DOI: 10.1002/aisy.202300207.
[2] J.Qi,L.Ma,Z.Cui,andY.Yu,(2024)“Computervision based hand gesture recognition for human-robot interac tion: a review" Complex & Intelligent Systems 10(1): 1581–1606. DOI: 10.1007/s40747-023-01173-6.
[3] J. Yu, H. Li, S.-L. Yin, and S. Karim, (2020) “Dynamic gesture recognition based on deep learning in human-to computer interfaces" Journal of Applied Science and Engineering 23(1): 31–38. DOI: 10.6180/jase.202003_23(1).0004.
[4] L. Zongxing, H. Baizheng, C. Yingjie, C. Bingxing, Y. Ligang, H. Haibin, and L. Zhoujie, (2023) “Human machine interaction technology for simultaneous gesture recognition and force assessment: A Review" IEEE Sen sors Journal 23(22): 26981–26996. DOI: 10.1109/JSEN.2023.3314104.
[5] N. Al Mudawi, H. Ansar, A. Alazeb, H. Aljuaid, Y. AlQahtani, A. Algarni, A. Jalal, and H. Liu, (2024) “Innovative healthcare solutions: robust hand gesture recog nition of daily life routines using 1D CNN" Frontiers in Bioengineering and Biotechnology 12: 1401803. DOI: 10.3389/fbioe.2024.1401803.
[6] J. Yu, L. Zhao, et al., (2021) “A novel deep CNN method based on aesthetic rule for user preferential images rec ommendation" Journal of Applied Science and Engi neering 24(1): 49–55. DOI: 10.6180/jase.202102_24(1).0006.
[7] D. Ryumin, D. Ivanko, and E. Ryumina, (2023) “Audio-visual speech and gesture recognition by sensors of mobile devices" Sensors 23(4): 2284. DOI: 10.3390/s23042284.
[8] R. Rastgoo, K. Kiani, S. Escalera, and M. Sabokrou, (2024) “Multi-modal zero-shot dynamic hand gesture recognition" Expert Systems with Applications 247: 123349. DOI: 10.1016/j.eswa.2024.123349.
[9] W.Li, S. Wu, S. Li, X. Zhong, X. Zhang, H. Qiao, M. Kang, J. Chen, P. Wang, and L.-Q. Tao, (2023) “Ges ture recognition system using reduced graphene oxide enhanced hydrogel strain sensors for rehabilitation train ing" ACS Applied Materials & Interfaces 15(38): 45106–45115. DOI: 10.1021/acsmaterialslett.4c00809.
[10] M.Alonazi, H. Ansar, N. Al Mudawi, S. S. Alotaibi, N. A. Almujally, A. Alazeb, A. Jalal, J. Kim, and M. Min, (2023) “Smart healthcare hand gesture recognition using CNN-based detector and deep belief network" IEEE Access 11: 84922–84933. DOI: 10.1109/ACCESS.2023.3289389.
[11] B. I. Alabdullah, H. Ansar, N. A. Mudawi, A. Alazeb, A. Alshahrani, S. S. Alotaibi, and A. Jalal, (2023) “Smart Home Automation-Based Hand Gesture Recog nition Using Feature Fusion and Recurrent Neural Net work" Sensors 23(17): 7523. DOI: 10.3390/s23177523.
[12] A. S. M. Miah, M. A. M. Hasan, and J. Shin, (2023) “Dynamic hand gesture recognition using multi-branch attention based graph and general deep learning model" IEEE Access 11: 4703–4716. DOI: 10.1109/ACCESS.2023.3235368.
[13] M. Muneeb, H. Rustam, and A. Jalal. “Automate appliances via gestures recognition for elderly living assistance”. In: 2023 4th International Conference on Advancements in Computational Sciences (ICACS). IEEE. 2023, 1–6. DOI: 10.1109/ICACS55311.2023.10089778.
[14] J. Liu, X. Wang, C. Wang, Y. Gao, and M. Liu, (2023) “Temporal decoupling graph convolutional network for skeleton-based gesture recognition" IEEE Transactions onMultimedia26:811–823. DOI: 10.1109/TMM.2023.3271811.
[15] D. Zheng, H. Li, and S. Yin, (2020) “Action recognition based on the modified twostream CNN" International Journal of Mathematical Sciences and Computing 6(6): 15–23. DOI: 10.5815/ijmsc.2020.06.03.
[16] H.Liang, L. Fei, S. Zhao, J. Wen, S. Teng, and Y. Xu, (2024) “Mask-guided multiscale feature aggregation net work for hand gesture recognition" Pattern Recognition 145: 109901. DOI: 10.1016/j.patcog.2023.109901.
[17] A.Dayal,M.Aishwarya,S.Abhilash,C.K.Mohan,A. Kumar, and L. R. Cenkeramaddi, (2023) “Adversarial unsupervised domain adaptation for hand gesture recogni tion using thermal images" IEEE Sensors Journal 23(4): 3493–3504. DOI: 10.1109/JSEN.2023.3235379.
[18] S. Yin, H. Li, Y. Sun, M. Ibrar, and L. Teng, (2024) “Data Visualization Analysis Based on Explainable Arti f icial Intelligence: A Survey" IJLAI Transactions on Science and Engineering 2(2): 13–20.
[19] S. Yin, H. Li, A. A. Laghari, T. R. Gadekallu, G. A. Sampedro, and A. Almadhor, (2024) “An anomaly detection model based on deep auto-encoder and capsule graph convolution via sparrow search algorithm in 6G internet-of-everything" IEEE Internet of Things Jour nal 11(18): 29402–29411. DOI: 10.1109/JIOT.2024.3353337.
[20] F. Fu, Y. Gao, Z. Lu, H. Wu, and S. Zhao. “Unsuper vised Continual Learning of Image Representation Via Rememory-Based Simsiam”. In: ICASSP 2024 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, 4980–4984. DOI: 10.1109/ICASSP48485.2024.10448032.
[21] M. Montazerin, E. Rahimian, F. Naderkhani, S. F. Atashzar, S. Yanushkevich, and A. Mohammadi, (2023) “Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high density EMG signals" Scientific reports 13(1): 11000. DOI: 10.1038/s41598-023-36490-w.
[22] F. Chen, S. Zhang, L. Hu, J. Fan, C.-H. Lin, P. Guan, Y. Zhou, T. Wan, S. Peng, C.-H. Wang, et al., (2023) “Bio-inspired artificial perceptual devices for neuromorphic computing and gesture recognition" Advanced Func tional Materials 33(24): 2300266. DOI: 10.1002/adfm.202300266.
[23] G. Li, D. Bai, G. Jiang, D. Jiang, J. Yun, Z. Yang, and Y. Sun, (2023) “Continuous dynamic gesture recognition using surface EMG signals based on blockchain-enabled internet of medical things" Information Sciences 646: 119409. DOI: 10.1016/j.ins.2023.119409.