User trainable sign language to speech glove using KNN classifier
Keywords:
Sign Language, Smart Glove, K Nearest Neighbour (KNN) Graphical User interface (GUI)Abstract
A sizable population around the world has some form of hearing or speaking disability. This creates a communication barrier among them and the rest of the world. Sign language was introduced to bridge this gap. The objective is to design a glove that can help translate sign language to text and that can be trained by the user itself if required. To achieve this a glove was designed using five flex sensors, three contact sensors and an accelerometer. Flex sensors were chosen as they are resistive devices that change resistance when bent. Due to their compactness, they can also be easily put on to a glove along with contact sensors and an accelerometer. The data from these sensors is then fed to an Arduino where it is read and processed before being sent to MATLAB via Bluetooth. After getting the values, the smart gesture detection algorithm must be designed so as to improve accuracy. To do this the data from the Arduino is first used to train a KNN model for classification. The model created after training is then used for classification of the gestures. A GUI was designed with control signals that allows the user to make a word from these gestures and then the word is converted to speech. The glove accurately gives us data points that can be used to classify various gestures of the American Sign Language. The interactive GUI developed in MATLAB enables a user to easily use the glove to make and/or edit a word created by the user and then recite it out on a speaker.
References
Anusha L, Usha Devi Y (2016) Implementation Of Gesture Based Voice And Language Translator For Dumb People. International Conference on Communication and Electronic Systems (ICCES) ISBN:978-1-5090- 1066-0.
D. J. Sturman and D. Zeltzer. "A Survey of Glovebased Input", IEEE Comput. Graph. Appl., 14(1), pp. 30-39, 1994.,
S. S. Fels and G. E. Hinton. "Glove-talk: A Neural Network Interface between a Data-glove and a Speech Synthesizer", IEEE Trans. Neural
Networks, 4(1), pp. 2-8, 1993
N.Tubaiz,T,Shanableh and K Assaleh,‖GloveBased Continous Arabic Sign Language Recognition in User-Dependent Mode,‖IEEETranscation on Human-Machine Systems,Vol.45,no.4,pp.526-533 Aug.2015.
L. K. Simone, D. G. Kamper, "Design considerations for a wearable monitor to measure finger posture", J. Neuro Eng. Rehabil., vol. 2, no. 1, pp. 5-5, 2005
R. Gentner, J. Classen, "Development and evaluation of a low-cost sensor glove for assessment of human finger movements in neurophysiological settings", J. Neurosci. Methods, vol. 178, no. 1, pp. 138-147, 2009
Zhou Ren ,Junsong Yuan ,Jingjing Meng , Zhengyou Zhang,‖ Robust Part-Based Hand Gesture Recognition Using Kinect Sensor‖ IEEE
Transactions on Multimedia ,Volume: 15 Issue: 5, pp:1520-9210, 2013.
N. El-Bendary, H. Zawbaa, M. Daoud, A. ellaHassanien, K. Naka-matsu, "Arslat: Arabic sign language alphabets translator", Computer
Information Systems and Industrial Management Applications (CISIM) 2010 , pp. 590-595, Oct 2010.
Kalpattu S. Abhishek, Lee Chun Fai Qubeley and Derek. Ho,‖Glove-based hand Gesture recognition sign language translator using capacitive touch sensor‖, IEEE International Conference on Electron Devices and Solid-State Circuits EDSSC), 2016
EunseokJeong ,Jaehong Lee ,DaeEun Kim ―Finger-gesture Recognition Glove using Velostat‖ 11th International Conference on Control, Automation and Systems, Oct. 2011
M. Elmahgiubi ,MEnnajar ,N.Drawil and M.S Elbuni ―Sign language translator and esture recognition‖,Comuter& information technology (GSCIT),2015 Global Summit on Sousse,2015,PP.1-6
Jamal Haydar, Bayan Dalal,, Shahed Hussainy,, Lina El Khansa, Walid Fahs,‖ ASL Fingerspelling Translator Glove‖ IJCSI International Journal of Computer Science Issues, 2012,Vol. 9, Issue 6, No 1, PP-1694-0814
S. Sidney and E. Geoffrey, ―Glove talk—a neuralnetwork interface between a data-glove and a speech synthesizer,‖ IEEE Transactions on Neural Networks, vol. 4, no. 1, pp. 2–8, 1993
Ahmed H. Aliwy1 and Esraa H. Abdul Ameer,Comparative Study of Five Text Classification Algorithms with their Improvements. International Journal of Applied Engineering Research ISSN 0973-4562 Volume 12, Number 14 (2017) pp. 4309-4319.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2019 COMPUSOFT: An International Journal of Advanced Computer Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
©2023. COMPUSOFT: AN INTERNATIONAL OF ADVANCED COMPUTER TECHNOLOGY by COMPUSOFT PUBLICATION is licensed under a Creative Commons Attribution 4.0 International License. Based on a work at COMPUSOFT: AN INTERNATIONAL OF ADVANCED COMPUTER TECHNOLOGY. Permissions beyond the scope of this license may be available at Creative Commons Attribution 4.0 International Public License.
