Bixiao Wu
A modified LSTM model for Chinese sign language recognition using leap motion
Wu, Bixiao; Lu, Zhenyu; Yang, Chenguang
Abstract
At present, there are about 70 million deaf people using sign language in the world, but for most normal people, it is difficult to understand the meaning of the sign language expression. Therefore, it is of great importance to explore the ways of recognising the sign language. In this paper, we propose a dynamic sign language recognition method based on the modified long short-term memory (LSTM) model. Firstly, we use Leap Motion to collect the features of Chinese Sign Language (CSL). LSTM has a good effect in processing time series data, but the parameters of its hidden layer are shared, making it important information lost when dealing with long time series. The attention mechanism can give different attention weights to different features according to the correlation between the input data and output data, so as to enhance the model's attention to key information. Therefore, we combine LSTM with attention mechanism for dynamic sign language recognition. Experimental results show that the recognition accuracy of the modified LSTM model is 99.55%, which is higher than that of LSTM model. Finally, we developed a sign language human-computer interaction system, which verifies the real-time performance and effectiveness of the method proposed in this paper.
Citation
Wu, B., Lu, Z., & Yang, C. (2022). A modified LSTM model for Chinese sign language recognition using leap motion. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (1612-1617). https://doi.org/10.1109/SMC53654.2022.9945287
Conference Name | 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC) |
---|---|
Conference Location | Prague, Czech Republic |
Start Date | Oct 9, 2022 |
End Date | Oct 12, 2022 |
Acceptance Date | Nov 9, 2022 |
Publication Date | Nov 18, 2022 |
Deposit Date | Dec 2, 2022 |
Publicly Available Date | Nov 19, 2024 |
Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
Volume | 2022-October |
Pages | 1612-1617 |
Series ISSN | 2577-1655 |
Book Title | 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC) |
DOI | https://doi.org/10.1109/SMC53654.2022.9945287 |
Keywords | Human computer interaction, Dynamics, Time series analysis, Neural networks, Gesture recognition, Assistive technologies, Real-time systems, Sign language recognition, LSTM, Attention mechanism, Leap Motion |
Public URL | https://uwe-repository.worktribe.com/output/10198165 |
Publisher URL | https://ieeexplore.ieee.org/document/9945287 |
Related Public URLs | https://ieeexplore.ieee.org/xpl/conhome/9945068/proceeding |
Files
This file is under embargo until Nov 19, 2024 due to copyright reasons.
Contact Zhenyu.Lu@uwe.ac.uk to request a copy for personal use.
You might also like
A constrained framework based on IBLF for robot learning with human supervision
(2023)
Journal Article
Handheld device design for robotic teleoperation based on multi-sensor fusion
(2023)
Conference Proceeding
A gripper-like exoskeleton design for robot grasping demonstration
(2023)
Journal Article
Deformation-aware contact-rich manipulation skills learning and compliant control
(2023)
Conference Proceeding
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search